Skip to content
CosmosCLIChangelog
Context EnginePricingDocsBlog
CustomersCareersStatus PageTrust CenterSecurity
Talk to SalesSign in
Book demo

DevOps

Codebase health scorecard with regression alerts

Scan every connected repo for agent readiness, track scores over time, and alert the team when tests, hooks, docs, or CI health regress.

codebase healthreadinessdevopsagents.mdcoveragecipre-commitscoringregressionobservability

[ workflow / devops ]

Codebase health scorecard with regression alerts

Cosmos scans each connected repository against health criteria such as AGENTS.md quality, test coverage, pre-commit hooks, CI health, dependency freshness, and docs coverage. Scores are stored across runs so regressions trigger Slack alerts and PR comments. Teams can use the scorecard to prioritize cleanup and gate new agent experts.

08 nodes

07 edges

Trigger[trigger]
Nightly + post-merge

Schedule + main branch push

System step[enumerate]
Enumerate repositories

From environment config

AI Agent step[scan]
Run health scan

6 criteria, each 0–100

AI Agent step[score]
Compute composite score

Weighted average + delta

Decision

Score regressed?

Drop > threshold vs last run

Yes
Output / Result[alert]
Send regression alert

Slack to owner team

YES
AI Agent step[scorecard]
Generate scorecard report

Per-criterion + top 3 actions

Output / Result[publish]
Publish scorecard

Slack / dashboard

Workflow prompt

Paste this into Augment to reproduce the workflow end-to-end.

Build a Cosmos workflow that continuously measures codebase health and alerts when quality regresses.

Trigger: a scheduled run (e.g. nightly), plus on every merge to the main branch of any connected repository.

Steps:
1. Enumerate connected repositories. Pull the list from the Cosmos environment configuration.
2. For each repository, run the health scan. Evaluate the following criteria (each scored 0–100):
   a. **AGENTS.md coverage**: does an AGENTS.md (or equivalent) exist? Does it cover the required sections (codebase overview, coding conventions, test instructions, agent-specific notes)? Score by completeness.
   b. **Test coverage**: pull the latest coverage report from CI. Score against the configured target (default: 80%).
   c. **Pre-commit hooks**: are pre-commit hooks configured? Do they include linting, formatting, and secrets-detection?
   d. **CI pipeline health**: what is the pass rate of the main branch CI pipeline over the past 30 runs? Flag if below 90%.
   e. **Dependency freshness**: how many direct dependencies have a newer major or minor version available? Score by ratio of up-to-date packages.
   f. **Documentation coverage**: for public APIs and exported functions, what percentage have docstrings or inline documentation?
3. Compute a composite health score for each repository (weighted average of the above). Compare to the score from the last run.
4. Decision: "Score regressed?".
   - If the composite score dropped by more than the configured threshold (default: 5 points), post a Slack alert to the repository's owner team with the delta and the specific criteria that regressed.
   - In all cases, update the health score log and continue.
5. Generate a scorecard report. For each repository: the composite score, per-criterion scores, a delta vs last run, and the top 3 recommended actions to improve the lowest-scoring criteria.
6. Post the scorecard to a configured Slack channel or dashboard. For repositories below the minimum threshold, apply a "health-warning" tag in the repository.

Constraints:
- Never fail a CI run based on the health score: it is advisory, not a hard gate (unless explicitly configured otherwise).
- Keep the score history append-only so we can build trend charts over time.
- Always include the recommended actions: a score without guidance is noise.
← All Workflows