ADR-199: Two-Phase Project Status Token Optimization

Status: Accepted Date: 2026-02-15 Deciders: Hal Casteel (CTO) Project: BIO-QMS (coditect-biosciences-qms-platform) Applies To: All CODITECT projects using /project-status

---

Context

Problem

The /project-status --update command (v1.0) consumed the entire context window (~200K tokens) just reading raw TRACK files before AI narrative generation could begin. This caused:

1. Context compaction mid-execution, losing prior conversation state
2. Session continuation required — the command could not complete in a single pass
3. Unreliable execution — compaction could corrupt the analysis mid-generation
4. Unsustainable for regular use — a command meant to run frequently was too expensive to run at all

Token Budget Analysis (v1.0)

Data Source	Token Cost
17 TRACK files (~2,500 tokens each)	~42,500
MASTER-TRACK-INDEX.md	~4,000
Session logs (7 days)	~5,000
Command definition + system prompts + CLAUDE.md	~30,000
AI narrative generation	~10,000
Total	~90,000+

The AI was reading raw markdown files just to extract structured data (progress percentages, section counts, task checkboxes) that a simple script could compute in milliseconds.

Alternatives Considered

Approach	Pros	Cons	Decision
Two-phase architecture	Zero AI tokens for data extraction; compact brief for narrative	Requires Node.js script enhancement	Accepted
Subagent delegation	Keeps main context clean	Still reads all files (tokens consumed in subagent); subagent output still large	Rejected
Incremental updates	Only re-reads changed TRACKs	Complex git-diff logic; still reads files on first run; cache invalidation	Rejected (future enhancement)
--quick mode only	Zero AI tokens	No narrative analysis — loses the core value of the command	Accepted as supplementary mode

---

Decision

Implement a two-phase architecture where:

• Phase 1 (Node.js): Parse all TRACK files, compute metrics, generate both a full JSON and a compact AI brief — at zero AI token cost
• Phase 2 (AI): Read only the compact brief (~1,500 tokens), generate narrative analysis, merge into JSON via subagent

Architecture

Data Flow Detail

Token Budget Comparison

Command Execution Modes

---

Brief Format Specification

The AI brief (project-status-brief.md) contains exactly these sections, totaling ~115 lines / ~1,500 tokens:

Section	Content	Lines
Summary	Overall %, track counts, section counts	3
Tracks Table	Per-track: letter, name, status, progress, sections, sprint	17+
At-Risk Tracks	0% progress tracks with task counts	variable
Cross-Track Dependencies	Upstream/downstream dependency chains	~25
Sprint Load	Sections per sprint period	~10
Agent Workload	Tracks per agent	~17
Recent Session Logs	Last 7 days of activity	~7
Stalled Sections	In-progress with less than 50% completion	variable

Key design constraint: The brief MUST contain everything the AI needs to generate intelligent narrative. The AI MUST NOT need to read any raw TRACK file, MASTER-TRACK-INDEX, or session log content.

---

Implementation

Files Modified

File	Change
scripts/generate-project-dashboard-data.js	+90 lines: AI brief generation after JSON output
commands/project-status.md	Rewritten as v2.0 with two-phase EXECUTION DIRECTIVE
public/project-status-brief.md	New auto-generated output (created by script)

Critical Directives in Command Definition

The v2.0 command includes explicit prohibitions to prevent token regression:

• NEVER read raw TRACK files — the Node.js script already parsed them
• NEVER read MASTER-TRACK-INDEX.md — the brief contains all dependency/sprint data
• NEVER read the full JSON — only the brief + generate narrative
• Total AI token budget: ~12K (brief read + narrative generation)

---

Consequences

Positive

1. 97% token reduction — ~90,000 to ~12,000 tokens per execution
2. Single-pass execution — no context compaction, no session continuation needed
3. --quick mode — zero AI tokens for metrics-only dashboard refresh
4. Faster execution — Node.js parses all TRACKs in under 1 second vs minutes of AI reading
5. Better AI analysis — AI focuses on narrative intelligence instead of data extraction
6. Scalable — adding more TRACKs only increases Node.js time (negligible), not AI token cost

Negative

1. Dual maintenance — brief format must stay in sync with what AI narrative generation expects
2. Information loss — brief is a lossy compression of raw TRACKs; some nuance may be lost
3. Node.js dependency — requires gray-matter npm package for frontmatter parsing

Risks

Risk	Mitigation
Brief format drift from TRACK format	Script parses TRACK files dynamically, not hardcoded
Brief missing data AI needs	AI can fall back to reading specific TRACK with --track X
Script crashes on malformed TRACK	Script uses defensive parsing with fallback defaults

---

Validation

Metric	v1.0	v2.0	Improvement
AI tokens consumed	~90,000	~12,000	87% reduction
Context compactions	1-2 per run	0	Eliminated
Session continuations needed	Yes	No	Eliminated
Wall-clock time	5-10 min	30-60 sec	5-10x faster
Data accuracy	Direct from source	Computed summary	Equivalent
Narrative quality	Limited by exhausted context	Full context for narrative	Improved

---

Future Enhancements

1. Incremental updates — cache previous brief, only re-parse changed TRACKs (git diff)
2. Streaming brief — stream brief sections to AI as they're computed
3. Multi-project — generate briefs for multiple projects in one script run
4. Brief versioning — version the brief format for backward compatibility

---

ADR Version: 1.0.0 Created: 2026-02-15 Author: CODITECT Core Team

Changelog:

• v1.0.0 - Initial decision documenting two-phase architecture for /project-status token optimization