Architecture Drift Score: Is Your Documentation Telling the Truth? - Archyl Blog

The Architecture Drift Score is a number from 0 to 100 for how much of your documented architecture still exists in your codebase. This is the mechanism: the formula, what goes in the denominator, what is deliberately excluded, what the check cannot see, and how to enforce it in CI.

Architecture Drift Score: Is Your Documentation Telling the Truth?

A metric nobody can audit is a metric nobody should act on. So this post is the arithmetic: how the Architecture Drift Score is produced, what lands in the denominator, what we deliberately leave out, and the four things the check cannot see.

The score answers one question. What percentage of your documented architecture still exists in your codebase? It is a number from 0 to 100, computed from a single request to your Git provider, with no AI in the path and no file contents read.

If you want the problem rather than the arithmetic, the architecture drift guide covers what drift is, why it happens, and the other ways to detect it. Start there and come back. This page assumes you already want a number and want to know whether to believe it.

Reading the Number

Open any project in Archyl, click the heartbeat icon in the header, and hit "Compute Drift Score." In a few seconds you get a number:

  • 90-100% — Excellent. Your documentation closely matches the codebase.
  • 70-89% — Good. Mostly accurate, some gaps to address.
  • 50-69% — Fair. Significant drift detected. Time to update.
  • Below 50% — Your documentation is fiction.

Those bands are our judgment about what's worth acting on, not a measurement of anything. The number underneath them is exact.

How the Number Is Computed

Every element in your model is sorted into a bucket, and the score is the share that survived:

score = floor( (matched + 0.5 × partial) / total × 100 )

total = matched + partial + missing_in_code + new_in_code
  • matched — the model says it exists, the repository agrees.
  • missing_in_code — documented, and not found. A container whose directory is gone, a code element whose file was deleted.
  • new_in_code — found in the repository, absent from the model. Undocumented, which is drift in the other direction and counts against you exactly as hard.

partial carries half credit and is reserved for elements that match with differences. Today's checks don't produce it: every element lands in one of the other three, so in practice the score is the matched fraction. We're telling you because a formula with a term that never fires is the kind of thing you should hear from us rather than discover.

Two details that matter when you compare two runs. The result is truncated, not rounded, so 89.9 reports as 89. And undocumented elements enlarge the denominator, which is why adding three new services without documenting them drops your score even though nothing you had written became untrue.

What It Actually Checks

The drift analysis is lightweight by design: one recursive tree request to your Git provider, no AI, no file content fetched. It validates your architecture across five dimensions:

Systems — Does your repository name match the documented system? We use the same PascalCase naming convention as the AI discovery pipeline, with fuzzy matching so EkoAuthz matches a repo named authz.

Containers — Do the top-level directories in your repo correspond to documented containers? frontend/ matches FrontendWebApp. backend/ matches BackendApiServer. Infrastructure containers (databases, queues, monitoring) that don't have source directories are excluded, because they're valid documentation of external services rather than drift. The next section covers what that exclusion costs.

Components — Are the components under each container still valid? If the parent container's directory exists, its components are presumed valid. If the container directory is gone, all its components are flagged.

Code Elements — This is the most precise check. Every code element in your C4 model has a filePath. We verify that each file still exists in the repository. Renamed file? Deleted class? Moved module? The drift score catches it instantly.

Relationships — A relationship is valid if both its source and target elements passed validation. If either endpoint drifted, the relationship is flagged.

The result is a per-element breakdown showing exactly what matched, what's missing, and what's new — not an opaque score, but an actionable report.

What's Excluded From the Denominator

A score is only as honest as the things it refuses to count. Three exclusions, all deliberate:

External systems and people. Anything typed as an external system or a person is dropped before comparison, on both sides. Stripe, your identity provider and "Customer" belong on a System Context diagram and none of them will ever appear in your repository. Counting them as missing would punish you for drawing a correct diagram.

Infrastructure containers with no source directory. A documented container that matches no directory is removed from the container tally rather than counted as drift. Your PostgreSQL instance, your Kafka cluster and your Datadog account are legitimate containers and none of them is a folder.

That rule has a cost and you should know it: a real service directory you deleted is excluded from the container tally too, because the check can't tell "database" from "service we removed last sprint". Its components are not excluded. They still resolve as missing, because their parent container didn't match, so a removed service does show up in the score, one level lower than you would expect to find it.

Code elements with no recorded file path. If a code element in your model has no filePath, there is nothing to verify, so it is skipped rather than guessed at. It scores neither for you nor against you. Generated and vendored paths (vendor/, node_modules/, dist/, target/, __pycache__/ and the rest of the usual list) are filtered out of the file tree before any of this runs.

Why Lightweight Matters

We deliberately chose not to run the full AI discovery pipeline for drift detection. Here's why:

Speed. AI analysis takes minutes for large repositories. Drift scoring takes seconds. You can run it on every push without slowing down your pipeline.

Determinism. AI can produce different results on the same codebase depending on model temperature, prompt variations, and token limits. File path existence is binary — either the file is there or it isn't. Your score is reproducible.

Cost. No AI tokens consumed. No API rate limits hit. Run it a hundred times a day if you want.

Simplicity. The algorithm is auditable. Check file paths, match directory names, verify relationships. No black box.

What the Score Cannot See

Every one of those properties is bought with the same trade: the check reads structure, not code. Four consequences, and none of them is a bug we intend to hide.

Behavioral drift is invisible. If two services keep their names and their directories while the synchronous HTTP call between them becomes a queue message, the score does not move. Nothing structural changed. This is the largest blind spot and there is no cheap fix for it: catching it means reading code or reviewing the model with humans.

A move looks exactly like a deletion. Code elements are validated by exact, case-sensitive file path. Move internal/auth/token.go to internal/identity/token.go without touching a line of it and the element is reported missing. That's technically correct, since the documented path is wrong, and it means a refactor that renames directories drops your score in a way that looks alarming and resolves to a one-line edit per element.

Component accuracy is inherited, not verified. If a container's directory exists, every component under it is presumed valid. The check never looks inside. So a container that still exists but has been gutted and rewritten scores as clean at component level, and the number is more confident about your Level 3 diagram than the evidence supports.

Name matching is generous. Systems and containers are matched by name in three passes: exact case-insensitive, then substring containment either way, then overlapping tokens after splitting PascalCase and kebab-case. EkoAuthz matches a repository called authz; BackendApiServer matches a directory called backend. This is what stops trivial naming differences reporting as drift, and it errs toward giving your model the benefit of the doubt. If you want a strict reading, use the per-element breakdown rather than the headline number.

Taken together, the score is a good measure of whether your model still describes the same system, and a weak measure of whether it describes it correctly. Treat a high score as "no structural surprises", not as "the documentation is right".

Track Trends, Not Just Snapshots

A single score is useful. A trend is powerful.

Every drift computation is stored with its full breakdown. The Overview tab shows a bar chart of your score over time. Click any bar to load that historical report and see exactly what changed.

This turns drift scoring from a one-time audit into a continuous health metric. You can see:

  • Did last week's refactor improve or degrade documentation accuracy?
  • Is drift getting worse over time, and did anything you changed about the workflow slow it down?
  • Which sprint introduced the most undocumented changes?

Enforce It in CI

A metric you don't enforce is a metric you'll ignore. That's why we built a GitHub Action.

on:
  push:
    branches: [main]

jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      - uses: archyl-com/actions/drift-score@v1
        with:
          api-key: ${{ secrets.ARCHYL_API_KEY }}
          organization-id: ${{ secrets.ARCHYL_ORG_ID }}
          project-id: 'your-project-uuid'
          threshold: '70'

Set threshold: '70' and the action fails if your architecture documentation drops below 70% accuracy. The job summary shows a formatted table with the full breakdown — visible directly in your PR checks.

You can also post the score as a PR comment:

- uses: archyl-com/actions/drift-score@v1
  id: drift
  with:
    api-key: ${{ secrets.ARCHYL_API_KEY }}
    organization-id: ${{ secrets.ARCHYL_ORG_ID }}
    project-id: 'your-project-uuid'

- uses: actions/github-script@v7
  if: github.event_name == 'pull_request'
  with:
    script: |
      github.rest.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner,
        repo: context.repo.repo,
        body: '## Architecture Drift: ' +
              '${{ steps.drift.outputs.score }}%\n' +
              'Matched: ${{ steps.drift.outputs.matched-count }}' +
              ' / ${{ steps.drift.outputs.total-elements }}'
      })

Every developer sees the drift impact of their changes before merge. Architecture documentation becomes a first-class citizen in your CI pipeline — alongside tests, linting, and security scans.

MCP: AI Agents That Know Their Accuracy

If you're using Claude Code, Cursor, or any MCP-compatible AI agent with Archyl's MCP server, drift scoring is available as a tool:

compute_drift_score({ projectId: "..." })
get_drift_score({ projectId: "..." })
get_drift_history({ projectId: "..." })
get_drift_details({ scoreId: "..." })

This means an AI agent can check documentation accuracy before it starts working. The get_agent_context tool already provides the full C4 model, ADRs, and conformance rules. Now it can also check how trustworthy that documentation is.

An agent that sees a 45% drift score knows to be cautious about the architecture context it received. An agent that sees 95% can confidently rely on the documented structure. This is the foundation for self-aware AI agents that adjust their behavior based on documentation quality.

Webhook Alerts: Know When Drift Happens

Two new webhook events let you stay informed without checking dashboards:

  • drift.score_computed — Fires every time a drift score finishes computing. Push it to a Slack channel for visibility.
  • drift.score_degraded — Fires when the score drops by 10 or more points from the previous computation. This is your early warning system — architecture is drifting fast.

Configure these in Archyl's webhook settings. They work with Slack, Microsoft Teams, Discord, and any generic HTTP endpoint.

The REST API

For teams that want full programmatic control:

# Trigger computation
curl -X POST https://api.archyl.com/api/v1/drift/compute \
  -H "X-API-Key: $API_KEY" \
  -H "X-Organization-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{"projectId": "your-project-uuid"}'

# Get latest score
curl https://api.archyl.com/api/v1/drift/latest?projectId=...

# Get score history
curl https://api.archyl.com/api/v1/drift/history?projectId=...&limit=20

Computation is asynchronous — the POST returns immediately with a score ID, and you poll until status becomes completed. The GitHub Action handles this automatically.

Where This Sits in the Loop

A score is one step in a cycle: agents and humans read the model, code changes, the score measures the gap, CI holds a threshold, the team reconciles. Without the measurement step the cycle has no feedback and the documentation drifts unchallenged. That argument, and the rest of the case for detecting drift at all, is in the guide.

What this post is responsible for is the measuring step being trustworthy. Hence the formula, the exclusions, and the four things it cannot see.

Getting Started

  1. Open any project in Archyl
  2. Click the heartbeat icon in the header toolbar
  3. Click "Compute Drift Score"
  4. Set up the GitHub Action for continuous monitoring
  5. Configure a Slack webhook for drift.score_degraded alerts

Your architecture documentation either reflects reality or it doesn't. Now you have a number that tells you which one it is, and enough of its arithmetic to argue with it.


The rest of the cluster: architecture drift detection for the problem and the other detection methods, living architecture documentation for the practices that keep a score from sliding back. Definitions: architecture drift. Product page: drift detection.