Skip to main content

Checkpoint: MoE Document Classification System

Checkpoint: MoE Document Classification System

Date: December 28, 2025 Session: 2025-12-28-002 Author: Claude Opus 4.5 + Hal Casteel

Executive Summary​

Implemented automatic document classification system using Mixture of Experts (MoE) methodology. The core classification pipeline is fully operational, but automatic hook triggering is blocked by a Claude Code bug (#15585).

Completed Work​

1. MoE Classification System Architecture (ADR-023)​

Status: ✅ Complete

Created comprehensive ADR documenting the 4-phase classification pipeline:

PhaseComponentsPurpose
1. Analysts6 specialists (metadata, structural, content, pattern, semantic, path)Multi-signal analysis
2. ConsensusWeighted votingConfidence calculation
3. JudgesConsistency, Quality, DomainValidation (if < 85%)
4. Deep AnalysisSemanticSimilarity, Contextual, ContentReasoning, CrossRefEdge case resolution

Location: submodules/ops/coditect-document-management/docs/adrs/approved/ADR-023_moe-document-classification.md

2. Pattern Improvements (100% Accuracy)​

Status: ✅ Complete

Tuned detection patterns for all document types:

  • Agents: 100% accuracy
  • Commands: 100% accuracy
  • References: 100% accuracy
  • Guides: 100% accuracy

Key improvement: Added frontmatter detection (type:, component_type:) as primary signal with 0.6 weight.

3. /classify Command v2.0​

Status: ✅ Complete

Upgraded command to use full MoE classifier with all options:

/classify docs/ -r                      # Recursive analysis
/classify docs/ -r --update-frontmatter # Auto-add frontmatter
/classify docs/ -r --output report.json # JSON report
/classify docs/ -r --dry-run            # Preview mode
/classify docs/ -r --exclude "test_*"   # Skip patterns
/classify docs/ -r -v                   # Verbose logging

Location: commands/classify.md

4. Hook Infrastructure​

Status: ✅ Created (⚠️ Not firing due to bug)

Created PostToolUse hook components:

  • hooks/classify-document.sh - Shell wrapper
  • hooks/classify_document.py - Python handler (simplified classifier)
  • Hook registered in ~/.claude/settings.json

5. document-classification-patterns Skill​

Status: ✅ Complete

Created skill with:

  • Classification categories and signals
  • Confidence thresholds
  • Frontmatter templates for each type

Location: skills/document-classification-patterns/SKILL.md

6. GKE Deployment Guide Classification​

Status: ✅ Complete

Classified and added frontmatter to:

  • submodules/cloud/coditect-cloud-infra/docs/deployment/gke-deployment-guide.md
  • Type: guide (95% confidence, AUTO_APPROVED)

Known Issues​

PostToolUse Hook Bug (BLOCKING)​

Issue: Claude Code bug #15585 - PostToolUse hooks configured correctly but never execute

Impact: Automatic classification on Write/Edit operations does not trigger

Workaround: Use /classify command manually after creating documents

Tracking: https://github.com/anthropics/claude-code/issues/15585

Related Issues: #6305, #3179

Component Inventory​

ComponentPathStatus
ADR-023submodules/ops/coditect-document-management/docs/adrs/approved/ADR-023_moe-document-classification.md✅
/classify commandcommands/classify.md✅
Hook shell wrapperhooks/classify-document.sh✅
Hook Python handlerhooks/classify_document.py✅
Skillskills/document-classification-patterns/SKILL.md✅
Full MoE classifiersubmodules/ops/coditect-document-management/scripts/moe_classifier/✅

Commits​

CommitRepoDescription
ad5f7dccoditect-document-managementGuide pattern improvements
98f2c74coditect-document-managementADR-023 MoE classification
ea650bc2coditect-coreClassification system (hooks, command, skill)
6a66a104coditect-core/classify v2.0 with full MoE
0766fa3coditect-cloud-infraGKE Deployment Guide with frontmatter
9deb4821rollout-masterUpdated submodule pointers
8d0314b2rollout-master/classify full MoE pointer update

Remaining Work​

Priority 1: Blocked by Bug Fix​

TaskDescriptionBlocked By
Auto-classificationPostToolUse hook auto-triggerBug #15585
Hook validationTest hook fires on Write/EditBug #15585

Priority 2: Enhancements​

TaskDescriptionEffort
Batch frontmatter updateRun /classify --update-frontmatter on all docs1 hour
Classification reportGenerate baseline classification for all CODITECT docs2 hours
Hook fallbackImplement PreToolUse alternative (may have same bug)1 hour

Priority 3: Future Improvements​

TaskDescriptionEffort
Semantic embeddingsAdd vector similarity for edge cases4 hours
Custom type trainingLearn new document types from examples8 hours
CI/CD integrationAuto-classify in GitHub Actions2 hours
Frontmatter validationEnforce required fields per type2 hours

Metrics​

Classification Accuracy (from testing)​

Document TypeAccuracySample Size
Agent100%10
Command100%10
Reference100%10
Guide100%20
Workflow95%10
ADR100%5
Skill100%10

Component Counts (Post-Implementation)​

{
  "agents": 135,
  "commands": 148,
  "skills": 196,
  "scripts": 238,
  "hooks": 44,
  "workflows": 1152,
  "total": 1918
}

How to Test​

Manual Classification​

# Single file
/classify path/to/file.md

# Directory recursive
/classify docs/ -r

# With frontmatter update
/classify docs/ -r --update-frontmatter

# Dry run
/classify docs/ -r --dry-run

Direct Python Handler​

cd submodules/core/coditect-core
python3 hooks/classify_document.py path/to/file.md

Full MoE Classifier​

python3 submodules/ops/coditect-document-management/scripts/moe_classifier/classify.py docs/ -r

Next Session Recommendations​

  1. Monitor bug #15585 - Check for fix in Claude Code updates
  2. Run batch classification - /classify . -r --update-frontmatter across docs
  3. Generate classification report - Baseline all CODITECT documents
  4. Test PreToolUse alternative - May work if PostToolUse is buggy

References​

Session Context Extracted: 88,428 messages in unified store Export: 2025-12-28-002.txt

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.5 noreply@anthropic.com