High-Fidelity Semantic Context Compression & Knowledge Graphs for Autonomous Coding Agents
A technical specification and empirical evaluation of local AST-based semantic compilation, BLAKE3 Merkle-tree incremental invalidation, reversible runtime output compression, and cryptographically verifiable offline licensing.
Contents & Sections
Executive Summary
Autonomous coding agents (e.g., Claude Code, OpenAI Codex CLI, Gemini CLI) frequently suffer from context-window saturation, high API token costs, and attention degradation caused by naive file-dumping and unconstrained terminal output ingestion. While classical text search (grep) lacks structural semantics and dense vector embeddings fail to resolve exact declaration hierarchies or compute blast radius, Archis introduces a local, deterministic, one-way semantic compiler coupled with an in-memory SQLite knowledge graph and an MCP server interface.
By lowering 31 programming languages into a canonical intermediate representation (chi.core.v1) and projecting into a compact structural form (compact.v1), the indexing gate on 94 pinned public repositories covers 538,745 files with 0 parse failures and a median ~77% smaller representation than source (the cohort-5 report, kept in the internal record; 86 of the 94 corpora, re-measured on 0.49.0 with a median of 76.8%, are published per repository in terragov/archis-benchmarks runs/0.49.0/fase_a/cohort5.json). Separately, a 20-repository agent-task band measures 89.4–95% less task context versus a full-source agent — that range is not pooled with the indexing median. Incremental re-indexing powered by BLAKE3 Merkle roots updates a changed file 3.0× to 14.7× faster than a full rebuild on the five repositories in §03. Reversible tool output compression shortens terminal and test logs while keeping every original hash-verified and recoverable over local MCP. There is no source endpoint.
The Context Bottleneck in LLM Agents
Contemporary LLM agents execute tool-use loops to understand codebases. When an agent attempts to localize and modify functionality in a project exceeding 10,000 lines of code, it encounters three fundamental pathologies:
1. Over-Exposure of Source
Reading an entire 1,500-line source file to inspect a single helper function introduces thousands of irrelevant tokens into the attention window. This induces the “Lost in the Middle” phenomenon and inflates input token costs on every subsequent turn (O(N · K) cost scaling).
2. Blind Retrieval
String matching (grep) returns lines without syntactic scope or call hierarchies. Vector search (embeddings) operates on statistical token similarity, failing on exact identifier resolution, transitive caller graphs, and structural type hierarchies.
3. Session Amnesia
Standard agent harnesses maintain zero persistent index between turns. Each new subtask forces the agent to re-traverse the file system, re-parse dependencies, and re-read previously seen headers from scratch.
The 5-Stage Semantic Compiler Pipeline
Archis operates as a deterministic, single-direction semantic compiler. Source code is parsed via tree-sitter, lowered to a canonical intermediate representation, and projected into a token-optimal format.
chi measure <repo>
Stage names and token totals are real: the corpus repositories are public and pinned to released tags, so every figure is reproducible on your own checkout.
- 01detectlanguage + support tier per file
- 02extractparse with tree-sitter
- 03normalizecanonical IR
- 04rendercompact projection
- 05measuretokens + coverage
Choose a repository and press compile.
Coverage is quoted beside every reduction on purpose. A compiler that “compresses” by silently dropping code would post a spectacular number and be useless; pairing the two makes that impossible to hide. The projection itself is not shown here — the cost of it is the claim, and chi measure ./src reproduces that cost on your own code.
End-to-End Call Execution Path
The agent communicates with Archis over local Model Context Protocol (MCP stdio/loopback). Queries resolve against the local SQLite graph with zero network transit.
One question, end to end
Illustrative animation of the real call path. The dashed boundary is your machine.
The agent asks a question in words. Press play to follow it.
Content Identity & BLAKE3 Merkle Trees (CAVS DNA)
Archis inherits its content-addressing core from CAVS (Content-Addressable Verified Streaming). Every source file, syntax node, and IR projection is indexed using BLAKE3 cryptographic hashes organized into Merkle trees.
| Repository | Language | Clean Rebuild | Single-File Update | Speedup Ratio |
|---|---|---|---|---|
| guava | Java | 4,280 ms | 291 ms | 14.7× |
| spring-boot | Java | 37,120 ms | 3,410 ms | 10.9× |
| angular | TypeScript | 18,400 ms | 1,920 ms | 9.6× |
| gin | Go | 412 ms | 137 ms | 3.0× |
| ripgrep | Rust | 820 ms | 195 ms | 4.2× |
On 21 probed test corpora, the graph produced by an incremental update is byte-identical (canonical hash match) to a clean build from scratch.
Reversible Runtime Output Compression
An agent's context is consumed not only by repository files, but by test suites, compilation errors, and linter outputs. Archis applies structural pattern extractors to command stdout/stderr:
| Signal / Fixture | Before (Raw Output) | With Archis |
|---|---|---|
A shortened output you can read back Every lossy run now names where its original lives — a session handle or a local:// artifact — and both verify a content hash on every read. | 0% | 100% |
Failure lines that survive compression Structured parsers for cargo, go test, pytest and ESLint render every failure verbatim. The old substring scan missed half of them — and 100% of the lines containing none of its eight keywords, which is where the failure actually is. | 50% | 100% |
The long tail no parser covers A declarative filter DSL for docker, git, gradle and terraform. Nothing in it executes anything, and an on-disk filter is inert until you consent to it by content hash. | 72.1% | 92.6% |
Documents the graph could not see Markdown, JSON, logs and plain text were counted as skipped and thrown away. They are now indexed as one symbol per section — headings are a document’s API the way signatures are a module’s. | 0 | 580 |
Guaranteed Reversibility via local:// Handles
When Archis truncates or summarizes a log, the raw bytes are journaled into a local content-addressed artifact store. The MCP tool output carries a local://<sha256> reference. If the LLM requires full stack trace context, it queries chi tools artifact get <uri> over MCP without requiring a server round-trip.
Indexing gate, then the agent-task band
Two campaigns, never one suite. Cohort 5 is the 94-repo indexing gate. Task-context is 20 repos with a labeled 89.4–95% policy band. CLI 0.52.0 is the shipping build; it is not a claim that the 538k-file gate was re-measured on it. Token counts use tiktoken (cl100k_base / o200k_base).
less task context on 20 repos vs a full-source agent — a separate policy band, not the indexing median
Cost per Task: agent-task band (20 repos, not pooled with indexing)
These are separate measurements, and none is added to another. Indexing scale (94 repositories, 538,745 files, no parse failures, a median representation about 77% smaller than the source) comes from one indexing campaign. The 89.4–95% task-context range comes from a different set of 20 repositories and measures context shown to the agent, not the bill. The 22-row representation table is a third campaign, and the runtime figures are fixed test cases. Each published run's per-repository results are public in terragov/archis-benchmarks; the campaign reports themselves stay in the internal record.
LLM Agent Multi-Turn Campaign Analysis
To evaluate whether token savings translate directly into reduced financial cost and increased task stability, we executed paired multi-turn runs using OpenAI Codex CLI (gpt-5.5) across 8 independent trials:
| Measure | Codex (Native Tools) | Through Archis MCP | Observed Delta |
|---|---|---|---|
| Output the model wrote(separated) | 5,477 | 3,171 | 42% less |
| Input tokens | 190,474 | 102,089 | 46% less |
| Cost per run | $0.289 | $0.226 | 22% less |
| Wall clock | 224 s | 157 s | 30% faster |
| Reasoning tokens | 3,927 | 4,239 | level |
| Answer accuracy | 97.4% | 97.5% | level |
Key Finding: Native agents write ad-hoc AST scripts on the fly, yielding 82% run-to-run output variance. Archis stabilizes agent turns into bounded tool calls, reducing output variance to 5% while cutting input tokens by 46%.
Cryptographic Safety, Offline Leases & Zero-Leakage
Archis enforces an absolute separation between the customer's machine (semantic computation) and cloud control plane (licensing and updates).
What crosses, and what cannot
Everything semantic happens on your machine; everything commercial happens in the cloud. The third packet is the one worth watching.
Press play to trace what actually crosses the boundary.
Ed25519 Clamped Offline Leases
CLI licensing operates completely offline. An Ed25519-signed JSON token encodes access permissions and is strictly clamped to subscription expiry deadlines. Tampered leases or mismatched machine identities are rejected with distinct exit codes.
Type-Enforced Telemetry Allowlist
Telemetry is opt-in and off by default. In the Rust CLI crate, event properties are restricted to &'static str types at compile time, ensuring file paths, variable names, code snippets, and user queries can never be serialized or transmitted.
Reproduce it yourself
The per-repository results of each published run, the repositories with their pins and licences, and the method behind each figure are public in terragov/archis-benchmarks; the campaign reports stay internal, and the harness follows a release or two later. On your own code, chi measure ./src gives your number in about a minute.