Skip to main content

Internal Documentation Consolidation Plan Executive Summary

Internal Documentation Consolidation Plan - Executive Summary

Date: December 22, 2025 Scope: /internal/ directory Current State: 390 markdown files Target State: 60 active files + 330 archived Reduction: 85% (active files only)

Visual Overview​

BEFORE (390 files):
internal/
├── architecture/ ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140 files
├── research/     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 150 files
├── project/      ━━━━━━━━━━━━━━━━━━━━━━━ 130 files
├── deployment/   ━━━ 10 files
└── testing/      ━ 6 files

AFTER (60 active files):
internal/
├── architecture/ ━━━━━━━━━━━━ 50 files (keep ADRs)
├── research/     ━━━━━ 20 files (87% reduction!)
├── project/      ━━━━━ 30 files (already done ✅)
├── deployment/   ━━━ 10 files (keep all)
└── testing/      ━ 6 files (keep all)

ARCHIVED (330 files):
internal/archive/
├── research/     ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130 files
├── architecture/ ━━━━━━━━━━━━━━━━━━━ 90 files
└── project/      ━━━━━━━━━━━━━━━━━━━ 100 files

Priority Matrix​

DirectoryFilesPriorityEffortImpact
research/anthropic-research/50 → 8P04hHigh - 84% reduction
research/claude-code-automation/15 → 3P12hHigh - 80% reduction
research/generative-ui/20 → 1P12hHigh - 95% reduction
research/llm-council-pattern/9 → 2P11hMedium - 78% reduction
architecture/claude-4.5-optimization/batches/20 → 1P21hMedium - 95% reduction
Other research/56 → 6P24hMedium - 89% reduction

Week-by-Week Execution Plan​

Week 1: Research Consolidation (P0) ⚡ HIGH IMPACT​

Monday: Anthropic Research (4 hours)

# Create index
touch internal/research/anthropic-research/INDEX.md

# Consolidate 50 → 8 files
✅ Keep: ANTHROPIC-CLAUDE-BEST-PRACTICES.md
✅ Keep: ANTHROPIC-PROMPT-ENGINEERING.md
✅ Keep: ANTHROPIC-AGENT-PATTERNS.md
✅ Keep: ANTHROPIC-TOOL-USE-PATTERNS.md
✅ Keep: ANTHROPIC-MULTI-SESSION-PATTERN-RESEARCH.md
✅ Create: LONG-RUNNING-AGENTS-ANALYSIS.md (consolidate)
✅ Create: SESSION-ANALYSIS-SUMMARY.md (consolidate)
✅ Create: INDEX.md (navigation)

# Archive references
mv internal/research/anthropic-research/anthropic-updates/references/ \
   internal/archive/research/anthropic-references/

Tuesday: Claude Code Automation (2 hours)

# Consolidate 15 → 3 files
✅ Keep: README.md
✅ Create: AUTOMATION-SYSTEM-GUIDE.md (merge 5 files)
✅ Create: ACTIVATION-GUIDE.md (quick start)

# Archive examples
mv internal/research/claude-code-automation/examples/ \
   internal/archive/research/claude-code-examples/

Wednesday: Other Research Topics (4 hours)

# GenUI: 20 → 1 file
✅ Create: GENERATIVE-UI-RESEARCH.md or archive entire directory

# LLM Council: 9 → 2 files
✅ Create: LLM-COUNCIL-PATTERN.md (merge 9 files)
✅ Create: LLM-COUNCIL-IMPLEMENTATION.md

# GDPVal: 7 → 0 (archive)
mv internal/research/GDPVal/ internal/archive/research/

# A16Z Response: 4 → 0 (archive)
mv internal/research/az1.ai-coditect-A16Z-response/ internal/archive/research/

# Session Summaries: 2 → 0 (archive - superseded by CX)
mv internal/research/session-summaries/ internal/archive/research/

Thursday: Research Index & Frontmatter (3 hours)

# Create master research index
touch internal/research/INDEX.md

# Add YAML frontmatter to all 20 files
# Template:
---
title: "Document Title"
audience: contributor
type: research|guide|reference
status: active
created: YYYY-MM-DD
last_updated: YYYY-MM-DD
tokens: XXXX
summary: "One-sentence description"
when_to_read: "When AI agents should read this"
keywords: [anthropic, claude, automation]
---

Friday: Testing & Verification (2 hours)

# Verify all links work
grep -r "internal/research" internal/ docs/

# Test navigation from index files
# Commit and push
git add internal/research internal/archive
git commit -m "docs: Consolidate research/ directory (150 → 20 files, 87% reduction)"
git push

Week 1 Result: 150 → 20 files (87% reduction), all with frontmatter ✅

Week 2: Frontmatter & Archive Structure (P1)​

Monday-Tuesday: Add Frontmatter to Architecture (4 hours)

# Add frontmatter to 50 architecture files
# Focus on non-ADR files (ADRs already have status/date)

# Prioritize:
- database/ files (8)
- system-design/ files (8)
- c4-diagrams/ files (10)
- vision/ files (5)
- integration/ files (5)
- orchestration/ files (5)

Wednesday: Create Archive Structure (2 hours)

# Create archive directories
mkdir -p internal/archive/research
mkdir -p internal/archive/architecture
mkdir -p internal/archive/project

# Already archived (from Week 1):
internal/archive/research/
├── anthropic-references/ (40 files)
├── gdpval/ (7 files)
├── a16z-response/ (4 files)
├── session-summaries/ (2 files)
└── claude-code-examples/ (8 files)

# New archives:
mv internal/architecture/claude-4.5-optimization/batches/ \
   internal/archive/architecture/claude-4-5-batches/

Thursday: Master Index Creation (3 hours)

# Create internal/INDEX.md
# Include:
- Directory structure with file counts
- When to read each major file
- Token estimates
- Quick navigation by topic
- Cross-references to docs/

Friday: Final Cleanup (2 hours)

# Remove empty directories
find internal/ -type d -empty -delete

# Update all README.md files
# Final verification
# Commit and push
git add internal/
git commit -m "docs: Add frontmatter to all active files, create archive structure"
git push

Week 2 Result: 100% frontmatter coverage, clear active/archive separation ✅

File-by-File Consolidation Details​

Research/Anthropic-Research (50 → 8 files)​

Keep These 8 Files:​

  1. INDEX.md (NEW - 500 tokens)

    • Master navigation for all Anthropic research
    • Links to 7 core files
    • Keywords: anthropic, research, index

  2. ANTHROPIC-CLAUDE-BEST-PRACTICES.md (KEEP - 3000 tokens)

    • Core best practices from official docs
    • Keywords: claude, best-practices, patterns

  3. ANTHROPIC-PROMPT-ENGINEERING.md (KEEP - 2500 tokens)

    • Prompt engineering techniques
    • Keywords: prompting, engineering, optimization

  4. ANTHROPIC-AGENT-PATTERNS.md (KEEP - 2000 tokens)

    • Agent architecture patterns
    • Keywords: agents, architecture, patterns

  5. ANTHROPIC-TOOL-USE-PATTERNS.md (KEEP - 2000 tokens)

    • Tool use optimization
    • Keywords: tools, mcp, integration

  6. ANTHROPIC-MULTI-SESSION-PATTERN-RESEARCH.md (KEEP - 4000 tokens)

    • Multi-session management
    • Keywords: sessions, context, continuity

  7. LONG-RUNNING-AGENTS-ANALYSIS.md (NEW - 3000 tokens)

    • Consolidate from anthropic-updates/
    • Keywords: long-running, persistence, state

  8. SESSION-ANALYSIS-SUMMARY.md (NEW - 2500 tokens)

    • Consolidate JSONL-DEDUP-, SESSION-ANALYSIS-
    • Keywords: sessions, analysis, deduplication

Archive/Delete (42 files):​

Archive to internal/archive/research/anthropic-references/:

  • anthropic-updates/references/ (40 files)
    • All reference materials (papers, tutorials, blog posts)
    • Rationale: Historical research, not operational

Delete (superseded by CX system):

  • anthropic-updates/JSONL-DEDUP-COMPONENT-INVENTORY.md
  • anthropic-updates/JSONL-SESSION-DEDUP-WORKFLOW-ANALYSIS.md
  • Rationale: Superseded by production CX system

Research/Claude-Code-Automation (15 → 3 files)​

Keep These 3 Files:​

  1. README.md (UPDATE - 1000 tokens)

    • Overview and navigation
    • Links to setup + activation guides

  2. AUTOMATION-SYSTEM-GUIDE.md (NEW - 4000 tokens)

    • Merge these 5 files:
      • SYSTEM-architecture.md
      • ARCHITECTURE-README.md
      • setup/SETUP-AUTOMATION.md
      • AUTOMATION-SYSTEM-COMPLETE.md
      • examples/claude-code-howtos-integration.md
    • Sections: Architecture, Setup, Examples, Troubleshooting

  3. ACTIVATION-GUIDE.md (KEEP - 1500 tokens)

    • Quick activation steps
    • Copy-paste commands

Archive/Delete (12 files):​

  • Archive examples/ folder → archive/research/claude-code-examples/
  • Delete duplicate README files (multiple README.md variants)

Research/Generative-UI (20 → 1 file OR archive)​

Option 1: Consolidate (if actively used)

  1. GENERATIVE-UI-RESEARCH.md (NEW - 5000 tokens)
    • Consolidate all findings
    • Reference archived artifacts

Option 2: Archive (if historical)

  • Move entire generative-ui/ → archive/research/generative-ui/
  • Rationale: If not actively used in production

Research/LLM-Council-Pattern (9 → 2 files)​

Keep These 2 Files:​

  1. LLM-COUNCIL-PATTERN.md (NEW - 4000 tokens)

    • Merge these 8 files:
      • README.md
      • INTRODUCTION.md
      • architecture.md
      • VALUE-PROPOSITION.md
      • ONE-PAGER.md
      • FAQ.md
      • BATTLECARD.md
      • DIAGRAMS.md

  2. LLM-COUNCIL-IMPLEMENTATION.md (NEW - 3000 tokens)

    • Extracted from:
      • IMPLEMENTATION-BLUEPRINT.md
    • Practical implementation guide

Delete (9 files):​

  • All original files (content merged)

Frontmatter Examples​

Standard Frontmatter Template​

---
title: "Document Title"
audience: contributor  # contributor, architect, researcher
type: spec|guide|report|reference|index
status: active|archived|superseded
created: YYYY-MM-DD
last_updated: YYYY-MM-DD
tokens: 1000-5000
summary: "One-sentence description of purpose"
when_to_read: "Context when AI agents should read this file"
keywords: [key, topics, for, discovery]
---

Example: Research Index File​

---
title: "Anthropic Research Index"
audience: contributor
type: index
status: active
created: 2025-12-22
last_updated: 2025-12-22
tokens: 500
summary: "Master navigation for all Anthropic Claude research and best practices documentation"
when_to_read: "When you need to find Anthropic research docs or understand what research exists"
keywords: [anthropic, claude, research, index, best-practices, agents, sessions]
---

Example: Consolidated Guide​

---
title: "Claude Code Automation System Guide"
audience: contributor
type: guide
status: active
created: 2025-12-22
last_updated: 2025-12-22
tokens: 4000
summary: "Complete guide to CODITECT Claude Code automation system - architecture, setup, examples, and troubleshooting"
when_to_read: "When setting up automation, understanding system architecture, or troubleshooting automation issues"
keywords: [automation, claude-code, setup, architecture, examples, hooks, triggers]
---

Success Criteria​

Completion Checklist​

Week 1 Deliverables:

  • research/ reduced from 150 → 20 files (87% reduction)
  • All 20 files have YAML frontmatter
  • research/INDEX.md created with navigation
  • 130 files moved to archive/research/
  • All internal links updated and verified
  • Git commit with clear message

Week 2 Deliverables:

  • 50 architecture files have frontmatter
  • archive/ directory structure created
  • 90 architecture files archived
  • internal/INDEX.md created
  • All README.md files updated
  • Final verification and commit

Quality Gates​

  • ✅ All active files have complete YAML frontmatter
  • ✅ No broken internal links
  • ✅ Clear when_to_read for AI agent discovery
  • ✅ Token estimates within 20% accuracy
  • ✅ Archive preserves all git history
  • ✅ Directory structure matches target state

Rollback Plan​

If consolidation causes issues:

# Emergency rollback
git revert HEAD
git push

# Or restore from archive
cp -r internal/archive/research/* internal/research/

Risk: Low - All files archived (not deleted), git history preserved

Post-Consolidation Maintenance​

Monthly Tasks​

  1. Review new files added to internal/

    • Ensure frontmatter present
    • Verify proper categorization
    • Update index files

  2. Audit "last_updated" fields

    • Mark stale files as status: archived
    • Move to archive if >6 months untouched

  3. Check for duplicates

    • Grep for similar content
    • Consolidate if found

Quarterly Tasks​

  1. Documentation review

    • Read all status: active files
    • Update or archive as needed
    • Verify accuracy of when_to_read

  2. Index file updates

    • Refresh navigation
    • Add new topics
    • Update token estimates

Plan Complete: December 22, 2025 Execution Start: Week of December 23, 2025 Expected Completion: January 6, 2026 Total Effort: 16-20 hours over 2 weeks