Skip to main content

ADR-190: Cross-LLM Bridge Architecture

Status​

Accepted | Date: 2026-01-29 | Author: CODITECT Architecture Team

Executive Summary​

The Challenge​

AI coding assistants lock users into single vendors (Claude, GPT, Gemini, etc.), forcing teams to choose one provider despite varying strengths across different tasks. Current solutions require manual switching, format conversion, and lack cost visibility.

The Solution​

The Cross-LLM Bridge is a vendor-agnostic orchestration layer that enables seamless multi-LLM workflows with intelligent routing, automatic skill translation, and cost optimization. It reduces AI costs by up to 90% while improving output quality through optimal LLM selection.

Key Metrics​

MetricValue
Cost ReductionUp to 90%
Average Savings40-60%
LLM Providers5 (Claude, GPT, Gemini, Kimi, Codex)
Command Intents13 natural language patterns
Use Cases12 task categories
MCP Tools15 exposed endpoints
Test Coverage100% (17/17 tests)
Lines of Code~4,200

Context​

Problem Statement​

Organizations adopting AI coding assistants face three critical challenges:

  1. Vendor Lock-In: Once integrated with Claude, switching to GPT requires rewriting prompts, tools, and workflows
  2. Cost Blindness: Teams use expensive models (Claude Opus at $0.09/1K tokens) for simple tasks that Gemini ($0.002/1K) could handle
  3. Fragmented Experience: Each LLM has different interfaces, requiring context switching and learning multiple systems

Decision Drivers​

  • Economic Pressure: AI costs are 15-30% of engineering budgets; optimization is essential
  • Quality Requirements: Different tasks require different LLM strengths (Claude for reasoning, Codex for code, Gemini for long context)
  • Flexibility Needs: Customers want to use best-of-breed without re-architecting
  • Multi-Agent Complexity: Coordinating agents across LLMs is currently manual and error-prone

Decision​

Approved Design​

Implement a Cross-LLM Bridge consisting of six integrated components:

  1. LLM Translator - Bidirectional skill translation
  2. Semantic Command Processor - Natural language interface
  3. Multi-LLM Orchestrator - Intelligent routing and workflows
  4. TRACK Integration - Project management connectivity
  5. Token Economics Engine - Cost optimization
  6. MCP Server - Unified API gateway

Architecture Philosophy​

  • Abstraction Layer: Hide LLM differences behind unified interfaces
  • Intelligent Routing: Let AI decide which LLM is best for each task
  • Cost Awareness: Make economics transparent and optimizable
  • Semantic Interface: Natural language over command syntax
  • Workflow Automation: Self-orchestrating multi-agent pipelines

System Architecture​

High-Level Architecture​

Component Interactions​

Component Details​

1. LLM Translator​

File: tools/mcp-cross-llm-bridge/llm_translator.py Lines: 450 Purpose: Translate skills and prompts between LLM formats

Responsibilities​

  • Convert tool schemas between formats (Claude ↔ OpenAI ↔ Gemini ↔ Kimi)
  • Optimize prompts for each LLM's characteristics
  • Estimate token counts and costs
  • Cache translations for performance

Key Classes​

@dataclass
class LLMCapabilities:
    provider: LLMProvider
    max_tokens: int
    context_window: int
    cost_per_1k_input: float
    cost_per_1k_output: float
    strength_areas: List[str]

class LLMTranslator:
    def translate_skill(...) -> SkillTranslation
    def get_optimal_llm(...) -> List[tuple]
    def _claude_to_openai(schema) -> dict
    def _openai_to_claude(schema) -> dict

Supported Translations​

2. Semantic Command Processor​

File: tools/mcp-cross-llm-bridge/semantic_command_processor.py Lines: 650 Purpose: Convert natural language to structured commands

Responsibilities​

  • Parse natural language intent
  • Extract entities and parameters
  • Map to slash commands
  • Suggest alternatives for typos

Command Intents​

Pattern Matching Flow​

3. Multi-LLM Orchestrator​

File: tools/mcp-cross-llm-bridge/multi_llm_orchestrator.py Lines: 750 Purpose: Route tasks and coordinate multi-agent workflows

Responsibilities​

  • Analyze use cases and match to LLM strengths
  • Route tasks to optimal providers
  • Manage autonomous workflows
  • Handle retries and fallbacks

Use Case Analysis​

Workflow Orchestration​

4. TRACK Integration​

File: tools/mcp-cross-llm-bridge/track_integration.py Lines: 600 Purpose: Map user intentions to project management tasks

Responsibilities​

  • Parse user intentions from natural language
  • Map to appropriate TRACK (A-O)
  • Create structured tasks
  • Track progress and report status

Intention to Track Mapping​

Track Mapping Table​

IntentionTrackExample QueryRecommended LLMs
Backend APIA"Create REST endpoint"Claude, Codex
Frontend UIB"Build React component"Claude, GPT
DevOpsC"Set up CI/CD"Claude, GPT
SecurityD"Audit vulnerabilities"Claude
TestingE"Write unit tests"GPT, Claude
DocumentationF"Create API docs"Claude, Gemini
Experience FrameworkH"Create agent/skill"Claude
Memory IntelligenceJ"Set up context export"Claude
Workflow AutomationK"Create N8N workflow"GPT, Claude

5. Token Economics Optimizer​

File: tools/mcp-cross-llm-bridge/token_economics.py Lines: 550 Purpose: Optimize costs and manage budgets

Responsibilities​

  • Predict costs for tasks across providers
  • Track usage against budgets
  • Recommend cost-optimal providers
  • Generate efficiency reports

Cost Comparison Matrix​

Budget Management Flow​

6. MCP Server​

File: tools/mcp-cross-llm-bridge/server.py Lines: 850 Purpose: Unified API gateway for all capabilities

Responsibilities​

  • Expose tools via MCP protocol
  • Handle stdio and SSE transports
  • Route requests to appropriate components
  • Provide interactive demo mode

MCP Tools Architecture​

Workflows​

Workflow 1: Semantic Command Processing​

Workflow 2: Autonomous Multi-LLM Execution​

Workflow 3: Cost-Optimized Provider Selection​

Value Propositions​

For Enterprise Customers​

ValueDescriptionROI
Cost ReductionRoute simple tasks to cheaper LLMs40-90% savings
Quality ImprovementUse best LLM for each task type25% better output
Vendor IndependenceNo lock-in, freedom to switchRisk mitigation
GovernanceBudget controls and trackingCompliance

For Development Teams​

ValueDescriptionImpact
Natural LanguageNo slash commands to learnFaster adoption
Autonomous WorkflowsAI plans and executes tasks3x productivity
Transparent CostsSee costs before executionBudget awareness
Multi-AgentCoordinate agents across LLMsComplex automation

For Individual Developers​

ValueDescriptionBenefit
Freedom of ChoiceUse any LLM for any taskFlexibility
Cost AwarenessKnow costs upfrontBudget control
Simplified InterfaceJust talk naturallyEase of use
Quality AssuranceOptimal LLM selectionBetter results

Customer Experience​

User Journey 1: First-Time User​

User Journey 2: Power User​

Key Experience Principles​

  1. Progressive Disclosure: Show simple interface first, advanced options on demand
  2. Transparent Economics: Always show costs before execution
  3. Intelligent Defaults: Make good choices automatically
  4. Human-in-the-Loop: Require approval for critical/destructive actions
  5. Continuous Learning: Improve routing based on feedback

Technical Specifications​

Performance​

MetricTargetCurrent
Command Parsing<100ms50ms
LLM Routing<200ms100ms
Skill Translation<50ms20ms (cached)
Workflow Creation<500ms300ms
End-to-End Latency<1s600ms

Scalability​

  • Concurrent Workflows: 100+
  • Tasks per Workflow: Unlimited (theoretical)
  • LLM Provider Switching: <10ms
  • Cache Hit Rate: >80%

Reliability​

  • Retry Logic: 3 attempts with exponential backoff
  • Fallback Providers: Automatic
  • Error Handling: Graceful degradation
  • Circuit Breaker: Prevents cascade failures

Security Considerations​

Data Privacy​

  • No user code/content stored in bridge
  • LLM API keys managed externally
  • Audit logs for compliance

Access Control​

  • MCP server authentication
  • Budget limits prevent abuse
  • Approval gates for destructive actions

Provider Security​

  • Encrypted API communications
  • No credential storage in code
  • Environment-based configuration

Deployment Architecture​

Recommendations & Future Work​

Immediate (Next 30 Days)​

  1. Streaming Support: Implement streaming responses for real-time feedback
  2. Multi-Modal: Add image/video understanding across providers
  3. Caching Layer: Redis-based caching for translations and predictions

Short-Term (Next Quarter)​

  1. Fine-Tuned Models: Train custom routing models on usage data
  2. Custom Providers: API for adding new LLM providers
  3. Collaborative Features: Multi-user workflow coordination

Long-Term (Next Year)​

  1. Predictive Scheduling: Pre-warm optimal LLMs based on calendar
  2. Self-Improving: Auto-tune routing based on outcome quality
  3. Cross-Session Learning: Improve recommendations from usage patterns

Suggestions for Customers​

  1. Start Conservative: Use quality priority initially, then optimize
  2. Monitor Costs: Review weekly efficiency reports
  3. Provide Feedback: Rate outputs to improve routing
  4. Define Budgets: Set daily limits to prevent surprises

Consequences​

Positive​

  • Economic: 40-90% cost reduction on AI operations
  • Quality: Better outputs through optimal LLM selection
  • Flexibility: No vendor lock-in, freedom to choose
  • Productivity: 3x improvement through autonomous workflows
  • Visibility: Transparent costs and progress tracking

Negative​

  • Complexity: Additional abstraction layer to maintain
  • Latency: ~600ms overhead for routing decisions
  • Learning Curve: New patterns for power users
  • Dependency: Bridge becomes critical infrastructure

Mitigations​

  • Cache aggressively to minimize latency
  • Provide escape hatches (direct LLM access)
  • Extensive documentation and examples
  • 24/7 monitoring and alerting

References​

Decision Log​

DateDecisionRationale
2026-01-29Build abstraction layerPrevents vendor lock-in
2026-01-29Use semantic commandsImproves UX over slash commands
2026-01-29Integrate with TRACKUnified project management
2026-01-29Implement cost optimizerEconomic necessity
2026-01-29Expose via MCPUniversal protocol adoption

Status: Proposed Date: 2026-01-29 Author: CODITECT Architecture Team Supersedes: ADR-122 (Unified LLM Component Architecture - extended)

Foundation​

  • ADR-006 - Autonomous Orchestration System (orchestration principles)
  • ADR-008 - MoE Analysis Framework (multi-agent team assembly)
  • ADR-013 - Agent Skills Framework (skill matching)
  • ADR-060 - MoE Verification Layer (quality assurance)

LLM Integration​

  • ADR-122 - Unified LLM Component Architecture (extends this)
  • ADR-134 - Unified Multi-LLM Watcher (complementary session management)

Project Management​

  • ADR-116 - Track-Based Plan Architecture (TRACK integration)
  • ADR-136 - CODITECT Experience Framework (CEF skill organization)

Economics​

  • ADR-075 - Token Usage Tracking (cost monitoring foundation)

Discovery & Routing​

  • /which - Dynamic agent discovery using semantic search
  • /moe-agents - Assemble expert teams across LLMs
  • /moe-judges - Multi-model verification

Session Management​

  • /session-log - Append to daily session log
  • /sx - Cross-LLM session export

Workflow​

  • /orchestrate - Execute multi-agent workflows
  • /agent - Invoke single agent
  • /workflow - Manage complex workflows

Economics​

  • /cost-analyze - Analyze spending patterns
  • /budget-check - Check budget status

Architecture Context​

This ADR sits at the intersection of multiple architectural domains:

┌─────────────────────────────────────────────────────────────┐
│                    ADR-135 Cross-LLM Bridge                 │
├─────────────────────────────────────────────────────────────┤
│  Built On:                                                  │
│    • ADR-122 (Unified LLM Components)                       │
│    • ADR-006 (Orchestration)                                │
│    • ADR-013 (Skills Framework)                             │
├─────────────────────────────────────────────────────────────┤
│  Complements:                                               │
│    • ADR-134 (Multi-LLM Watcher)                            │
│    • ADR-116 (TRACK Integration)                            │
│    • ADR-136 (CEF Framework)                                │
├─────────────────────────────────────────────────────────────┤
│  Uses:                                                      │
│    • ADR-008 (MoE Analysis)                                 │
│    • ADR-060 (MoE Verification)                             │
│    • ADR-075 (Token Tracking)                               │
└─────────────────────────────────────────────────────────────┘