CODITECT Mixture of Experts (MoE) System Analysis

Generated: December 31, 2025 Analysis Type: Comprehensive MoE System Review Certainty: HIGH (95%) - Based on direct codebase inspection

Executive Summary

CODITECT implements a sophisticated Mixture of Experts (MoE) architecture for multi-agent orchestration, quality evaluation, and document classification. The system leverages ensemble intelligence through parallel agent execution, weighted consensus calculation, and judge-based validation.

Key Metrics

Metric	Value	Source
Core MoE Commands	5	`/moe-agents`, `/moe-judges`, `/moe-workflow`, `/moe-analyze`, `/moe-judge`
MoE ADRs	2	ADR-008 (Analysis), ADR-009 (Judges)
Python Implementation Files	23	`scripts/moe_classifier/`
Analyst Types	5	structural, content, metadata, semantic, pattern
Judge Types	3	consistency, quality, domain
Type Experts	7	guide, reference, workflow, agent, command, ADR, skill

Skill Capabilities

Pattern Recognition: Identify applicable patterns
Automation: Automate repetitive tasks
Quality: Ensure consistent results

Skill Capabilities

Pattern Recognition: Identify applicable patterns
Automation: Automate repetitive tasks
Quality: Ensure consistent results

Skill Capabilities

Pattern Recognition: Identify applicable patterns
Automation: Automate repetitive tasks
Quality: Ensure consistent results

Skill Capabilities

Pattern Recognition: Identify applicable patterns
Automation: Automate repetitive tasks
Quality: Ensure consistent results

Skill Capabilities

Pattern Recognition: Identify applicable patterns
Automation: Automate repetitive tasks
Quality: Ensure consistent results

System Architecture

Three-Layer MoE Model

┌─────────────────────────────────────────────────────────────────────┐
│                    LAYER 1: EXPERT ASSEMBLY                         │
│                         /moe-agents                                  │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐       │
│  │ Expert  │ │ Expert  │ │ Expert  │ │ Expert  │ │ Expert  │       │
│  │  A      │ │  B      │ │  C      │ │  D      │ │  E      │       │
│  │(Domain) │ │(Domain) │ │(Domain) │ │(Domain) │ │(Domain) │       │
│  └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘       │
│       │          │          │          │          │                │
│       └──────────┴────┬─────┴──────────┴──────────┘                │
│                       ▼                                             │
├─────────────────────────────────────────────────────────────────────┤
│                    LAYER 2: JUDGE EVALUATION                        │
│                         /moe-judges                                  │
│  ┌───────────────┐ ┌───────────────┐ ┌───────────────┐              │
│  │ Consistency   │ │ Quality       │ │ Domain        │              │
│  │ Judge         │ │ Judge         │ │ Judge         │              │
│  │ (Cross-ref)   │ │ (Vote quality)│ │ (CODITECT rules)             │
│  └───────┬───────┘ └───────┬───────┘ └───────┬───────┘              │
│          │                 │                 │                      │
│          └─────────────────┼─────────────────┘                      │
│                            ▼                                         │
├─────────────────────────────────────────────────────────────────────┤
│                    LAYER 3: CONSENSUS & OUTPUT                      │
│                         /moe-workflow                                │
│  ┌─────────────────────────────────────────────────────────┐        │
│  │ Weighted Consensus + Quality Gate + Final Verdict       │        │
│  └─────────────────────────────────────────────────────────┘        │
└─────────────────────────────────────────────────────────────────────┘

Command Reference

1. `/moe-agents` - Expert Assembly

Purpose: Analyzes a use case and assembles an optimal team of specialized agents.

Algorithm:

Break down task into component requirements
Map requirements to agent specializations
Select 3-7 agents with complementary skills
Assign specific tasks to each expert
Create coordination plan for collaboration

Example Use Cases:

Task	Expert Team
Build SaaS Product	senior-architect, backend-architect, frontend-react-typescript-expert, devops-engineer, security-specialist
Competitive Analysis	competitive-market-analyst, market-researcher, business-intelligence-analyst, web-search-researcher
Security Audit	security-specialist, penetration-testing-agent, backend-api-security, compliance-checker-agent

Invocation:

/moe-agents build complete authentication system with OAuth2

2. `/moe-judges` - Judge Panel Evaluation

Purpose: Assembles a panel of specialized judges to evaluate, critique, and score output.

Evaluation Dimensions:

Dimension	Weight	Score Range
Architecture Quality	25%	1-10
Security Posture	25%	1-10
Code Quality	20%	1-10
Test Coverage	15%	1-10
Documentation	15%	1-10

Verdict Categories:

Score	Verdict	Meaning
9-10	EXCELLENT	Production ready, exceeds standards
7-8	APPROVED	Ready with minor improvements
5-6	CONDITIONAL	Needs improvements before approval
3-4	REVISION REQUIRED	Significant issues
1-2	REJECTED	Does not meet minimum standards

Invocation:

/moe-judges evaluate API design for production readiness

3. `/moe-workflow` - Complete Expert→Judge Pipeline

Purpose: Full lifecycle: experts create → judges evaluate → iterate to quality.

7-Phase Workflow:

Phase	Activity	Output
1. Analyze	Understand requirements, define criteria	Requirements doc
2. Assemble	Select 3-7 domain experts	Expert team roster
3. Execute	Experts collaborate on deliverable	Initial output
4. Judge Setup	Select 3-5 evaluation experts	Judge panel
5. Evaluate	Each judge reviews independently	Scores + feedback
6. Iterate	Address feedback until passing	Refined output
7. Deliver	Quality-assured output	Final deliverable

Quality Gates:

Gate	Requirements
Gold	All scores 9+/10 → Ship immediately
Silver	All scores 8+/10 → Ship with monitoring
Bronze	All scores 7+/10 → Ship with caveats
Fail	Any score <7/10 → Iterate

4. `/moe-analyze` - Research with Certainty Scoring

Purpose: Multi-agent research with explicit certainty quantification.

Certainty Scoring Formula:

certainty_score = (
    evidence_support * 0.40 +      # Quality of supporting sources
    source_reliability * 0.25 +    # Credibility of sources
    internal_consistency * 0.20 +  # Agent agreement level
    recency * 0.15                 # Information freshness
)

Certainty Levels:

Score	Level	Action
85-100%	HIGH	Report with confidence
60-84%	MEDIUM	Note limitations, provide sources
30-59%	LOW	Explicitly state uncertainty
0-29%	INFERRED	Require logical inference chain

Implementation Details

Python MoE Classifier (`scripts/moe_classifier/`)

The production implementation provides document classification with:

Components:

core/orchestrator.py - MoEOrchestrator class (416 lines)
core/consensus.py - ConsensusCalculator with weighted voting (245 lines)
core/models.py - Data structures (Document, AnalystVote, ConsensusResult)
judges/base.py - BaseJudge abstract class (134 lines)
analysts/ - 5 analyst implementations

Orchestrator Configuration:

@dataclass
class OrchestratorConfig:
    max_parallel_analysts: int = 5
    max_parallel_judges: int = 3
    analyst_timeout_seconds: float = 30.0
    judge_timeout_seconds: float = 10.0
    enable_deep_analysis: bool = True

Approval Types:

class ApprovalType(Enum):
    AUTO_APPROVED = "auto"        # ≥85% confidence, ≥80% agreement
    JUDGE_APPROVED = "judge"      # 2/3 judges approve
    DEEP_ANALYSIS_APPROVED = "deep"  # Deep analysts resolved
    HUMAN_REVIEW_REQUIRED = "human"  # Needs human review
    ESCALATED = "escalated"       # Failed all approval paths
    PENDING = "pending"           # Awaiting judgment

Consensus Algorithm

From moe-consensus-algorithm.md:

# Thresholds
AUTO_APPROVAL_CONFIDENCE = 0.90     # Skip judges if above
JUDGE_APPROVAL_CONFIDENCE = 0.85    # Accept with judge approval
AGREEMENT_THRESHOLD = 0.60          # Minimum 3/5 analysts agree

# Decision Flow
if confidence >= 0.90:
    → AUTO_APPROVED

elif agreement >= 0.60 and confidence >= 0.85:
    → Invoke 3 judges
    → If 2/3 approve: JUDGE_APPROVED
    → If any veto: ESCALATED

else:
    → ESCALATED (human review)

Optimal Use Cases

When to Use Each Command

Use Case	Command	Why
Complex multi-step task	`/moe-workflow`	End-to-end expert coordination + quality gates
Need team assembly	`/moe-agents`	Intelligent agent selection based on requirements
Evaluate existing output	`/moe-judges`	Multi-perspective quality scoring
Research with citations	`/moe-analyze`	Certainty-quantified research with source tracking
Document classification	`/classify`	Automated MoE document typing

Usage Patterns

Pattern 1: Full-Stack Feature Development

# Step 1: Assemble expert team
/moe-agents build user authentication with OAuth2, JWT, and MFA

# Step 2: Execute with coordination
/agent orchestrator "Coordinate authentication feature..."

# Step 3: Evaluate output
/moe-judges evaluate authentication system for production

# OR use combined workflow
/moe-workflow implement and validate user authentication system

Pattern 2: Research with Validation

# Research with certainty scoring
/moe-analyze --with-search "Enterprise DMS market standards 2024-2025"

# Then judge the research quality
/moe-judges evaluate research findings for accuracy and completeness

Pattern 3: Iterative Quality Improvement

# Initial implementation
/moe-agents build REST API for user management

# First evaluation
/moe-judges evaluate API design (score: 6.5/10)

# Iterate based on feedback
/moe-workflow iterate API design to 8+/10 quality

Agent Roles in MoE

Analyst Agents (Parallel Execution)

Analyst	Focus	Input	Output
`structural`	File structure, paths	File path analysis	Type + confidence
`content`	Document content	Text analysis	Type + confidence
`metadata`	YAML frontmatter	Metadata parsing	Type + confidence
`semantic`	Deep meaning	LLM analysis	Type + confidence
`pattern`	Regex patterns	Pattern matching	Type + confidence

Judge Agents (Validation)

Judge	Validation	Veto Authority
`consistency`	Cross-reference with related docs	Yes
`quality`	Vote distribution, outlier detection	Yes
`domain`	CODITECT-specific rules	Yes

Type Expert Agents (Deep Analysis)

Expert	Document Type	Key Signals
`GuideExpert`	Guides/Tutorials	Steps, prerequisites, troubleshooting
`ReferenceExpert`	API/Specs	Tables, schemas, configuration
`WorkflowExpert`	Processes	Phases, diagrams, checklists
`AgentExpert`	AI Agents	Persona, capabilities, tools
`CommandExpert`	Commands	Invocation, parameters, usage
`ADRExpert`	Decisions	Context, decision, consequences
`SkillExpert`	Patterns	When to use, patterns, I/O

Research Foundation

The MoE system is backed by peer-reviewed research (per ADR-008/009):

Research	Venue	Contribution	Certainty
Semantic Density	NeurIPS 2024	Multi-factor confidence scoring	96%
Self-Consistency (CoT-SC)	ICLR 2022	Internal consistency measurement	97%
Mixture-of-Agents	arXiv 2024	Ensemble wisdom (65.1% AlpacaEval)	90%
UoT Framework	NeurIPS 2024	Explicit uncertainty modeling	93%
Chain-of-Verification	ACL 2024	Evidence validation protocol	92%
G-Eval	EMNLP 2023	Chain-of-thought evaluation	95%
ChatEval	ICLR 2024	Multi-agent referee teams	91%

Best Practices

1. Choose the Right Granularity

Complexity	Command	Team Size
Simple task	`/which` → single agent	1
Moderate task	`/moe-agents`	3-4
Complex task	`/moe-workflow`	5-7
Enterprise task	`/moe-workflow` + iteration	7+

2. Leverage Parallel Execution

The orchestrator runs analysts in parallel:

with ThreadPoolExecutor(max_workers=5) as executor:
    futures = [executor.submit(analyst.analyze, doc) for analyst in analysts]

Maximum parallelism: 5 analysts + 3 judges = 8 parallel agents

3. Use Quality Gates

Gate	When to Use
Gold (9+)	Production deployments, security-critical
Silver (8+)	Standard releases
Bronze (7+)	Development, prototypes

4. Handle Escalations

When consensus fails:

Deep Analysis automatically invoked
Type Experts provide specialized analysis
If still unclear → Human review required

Integration Points

With Orchestrator Agent

# MoE commands invoke orchestrator for coordination
/agent orchestrator "Coordinate MoE workflow for authentication feature..."

With Quality Specialists

# Judges include quality specialists
judges = [
    "codi-qa-specialist",      # Quality assurance
    "code-reviewer",           # Code quality
    "architect-review",        # Architecture
    "security-specialist"      # Security posture
]

With Documentation Writer

# After MoE workflow completes
/agent codi-documentation-writer "Document the API created by MoE workflow"

Performance Characteristics

Metric	Target	Notes
Parallel Analysis	<3s	5 analysts concurrent
Consensus Calculation	<10ms	In-memory
Judge Validation	<1s	Only for 85-90% confidence
Total Pipeline	<5s	End-to-end classification
Throughput	100/min	Batch processing mode
Approval Rate	>85%	Auto + Judge approved

Command Quick Reference

# Expert Assembly
/moe-agents <task-description>

# Judge Evaluation
/moe-judges <what-to-evaluate>

# Complete Workflow
/moe-workflow <task-description>

# Research with Certainty
/moe-analyze <research-question>

# Document Classification
/classify <path> -r --expert

ADR-008: MoE Analysis Framework
ADR-009: MoE Judges Framework
Consensus Algorithm: MoE Consensus Specification
Orchestrator Agent: orchestrator.md
MoE Content Classifier: moe-content-classifier.md

Document Version: 1.0.0 Last Updated: December 31, 2025 Author: CODITECT Analysis System Certainty Level: HIGH (95%) - Direct codebase inspection

Executive Summary​

Key Metrics​

Skill Capabilities​

Skill Capabilities​

Skill Capabilities​

Skill Capabilities​

Skill Capabilities​

System Architecture​

Three-Layer MoE Model​

Command Reference​

1. /moe-agents - Expert Assembly​

2. /moe-judges - Judge Panel Evaluation​

3. /moe-workflow - Complete Expert→Judge Pipeline​

4. /moe-analyze - Research with Certainty Scoring​

Implementation Details​

Python MoE Classifier (scripts/moe_classifier/)​

Consensus Algorithm​

Optimal Use Cases​

When to Use Each Command​

Usage Patterns​

Pattern 1: Full-Stack Feature Development​

Pattern 2: Research with Validation​

Pattern 3: Iterative Quality Improvement​

Agent Roles in MoE​

Analyst Agents (Parallel Execution)​

Judge Agents (Validation)​

Type Expert Agents (Deep Analysis)​

Research Foundation​

Best Practices​

1. Choose the Right Granularity​

2. Leverage Parallel Execution​

3. Use Quality Gates​

4. Handle Escalations​

Integration Points​

With Orchestrator Agent​

With Quality Specialists​

With Documentation Writer​

Performance Characteristics​

Command Quick Reference​

Related Documentation​

Executive Summary

Key Metrics

Skill Capabilities

Skill Capabilities

Skill Capabilities

Skill Capabilities

Skill Capabilities

System Architecture

Three-Layer MoE Model

Command Reference

1. `/moe-agents` - Expert Assembly

2. `/moe-judges` - Judge Panel Evaluation

3. `/moe-workflow` - Complete Expert→Judge Pipeline

4. `/moe-analyze` - Research with Certainty Scoring

Implementation Details

Python MoE Classifier (`scripts/moe_classifier/`)

Consensus Algorithm

Optimal Use Cases

When to Use Each Command

Usage Patterns

Pattern 1: Full-Stack Feature Development

Pattern 2: Research with Validation

Pattern 3: Iterative Quality Improvement

Agent Roles in MoE

Analyst Agents (Parallel Execution)

Judge Agents (Validation)

Type Expert Agents (Deep Analysis)

Research Foundation

Best Practices

1. Choose the Right Granularity

2. Leverage Parallel Execution

3. Use Quality Gates

4. Handle Escalations

Integration Points

With Orchestrator Agent

With Quality Specialists

With Documentation Writer

Performance Characteristics

Command Quick Reference

Related Documentation