Using the Cosmos software factory to better understand our business

I lead the Solutions Architecture team at Augment Code. My team supports large engineering organizations evaluating Cosmos, our agentic orchestration platform, as the software factory their teams build in.

A question I get often is: "Mayur, how healthy are our enterprise pilots?" That answer used to be surprisingly hard to get.

Why it was hard

The data was scattered across systems:

• Product usage showed whether customers were adopting.
• GitHub and PR activity showed whether real work was moving.
• Salesforce held deal stages, close dates, and risks.
• Call transcripts captured what each customer had actually said success looked like.
• Important blockers still lived in people's notes, or their heads.

CRM data described the commercial relationship. Engineering and product systems showed what was actually happening. I needed both.

Historically, getting that view would have meant an analytics project: define the metrics, connect the systems, build the dashboard, and assign someone to keep it current. By the time it shipped, parts of it would already be stale.

Instead, I described the view I wanted to Cosmos.

The pilot-health view

It queried our usage data, reconciled session lineage so the numbers were not inflated, pulled the current deal state from Salesforce, and checked our claims against customer call transcripts.

The result was a ranked view of every pilot: adoption, activity, technical progress, deal context, blockers, and an overall health assessment. Every number traced back to a live source.

Pilot-health view: one row per anonymized account with health status, stage, risk, the technical win or value proof, and a board-level one-liner.

That traceability mattered more than the dashboard itself. It is easy to call a pilot green. It is harder to show the evidence.

Behind each account row was the underlying activity for that pilot, over the same reporting window.

Pilot snapshot: reporting-window activity for a single customer, showing sessions, active users, peak DAU, experts, automations, and PRs.

Changing the view

Then I changed the view by talking to it.

One customer, a life-sciences data platform, was not comfortable that anyone could view a secret's value after creation. It was one of two things standing between the account and rollout. I added it to that account's risk view.

Anyone can pull a metric. What I wanted was a metric checked against the customer's own words, especially the CxOs. So I asked for a board-level one-liner for every account, verified against what the customer had actually said. Cosmos returned one line per pilot and marked whether it matched:

• A life-sciences data platform: "An epic that took ~1 week now ships in ~3 hours. Exceeds the CDO's 75-90% automation goal."
• A maritime-software company: "~55% PR merge rate vs ~26% for a competing agent. Clears the CDO's quality bar. ✓"
• A geospatial-intelligence company: "93% PR merge rate, shared by the champion at the exec readout."
• A revenue-operations platform: "The CTO's exact bar: '30% PR merge-time reduction = done deal.' ✓"

Those were not generic engagement metrics. They were product outcomes checked against the customer's own definition of success. There was no new schema, ticket, or dashboard project. As the questions changed, I changed the view.

The next day, I asked Cosmos to refresh it. It found the changes in Salesforce and updated the affected accounts. One pilot had crossed into a technical win. In another, a previously skeptical CTO had "come around." The refreshed view also identified which information was current and which was one cycle behind.

Then it declined to replace one usage metric with a faster raw pull, because doing so would have made the number look more current but less accurate. That was the moment I trusted it. It optimized for the number I could defend rather than the one that refreshed fastest.

What this means beyond pilot health

The dashboard was useful, but it was not the main point.

Many important business questions depend on engineering reality: what customers are using, what is shipping, where work is blocked, and whether promised outcomes are happening. That reality lives across product telemetry, GitHub, CI, deployments, incidents, and support systems. Salesforce, financial data, and customer conversations add the commercial and human context.

Put them together, and you can answer questions neither side could answer alone. For us, the first question was pilot health. The same underlying evidence could support renewal risk, rollout tracking, account planning, product feedback, or an executive update. The output does not have to be a permanent dashboard. It can be whatever the next decision requires.

It changed which questions felt worth asking. I used to dismiss half the things I wanted to know because answering them would require a new report, integration, or workflow. Once the systems were accessible through Cosmos, the cost of the next question dropped.

But one person producing a good analysis is not organizational change. The real value comes when the workflow is shared: when sales, customer success, product, engineering, and leadership can work from the same evidence; when the definitions are visible; and when the process improves as the team uses it. We could spot blockers earlier, enter customer conversations knowing what mattered, and distinguish genuine progress from an optimistic CRM field.

The business context lives in systems like Salesforce and call notes. The operational truth lives in engineering and product systems.

We need both to run a credible pilot review.

Want to see what this looks like on your own data? Explore Cosmos, or talk to us.