AI Specialist

Purpose

Multi-provider AI routing specialist responsible for intelligent model selection, prompt optimization, and enabling CODITECT's core autonomous development capabilities through AI integration.

Core Capabilities

• Multi-provider routing (Claude, OpenAI, Gemini, Ollama)
• Intelligent model selection with cost optimization
• Prompt engineering and A/B testing optimization
• Response caching achieving 60% hit rate
• Context management and conversation persistence
• Real-time WebSocket AI integration

File Boundaries

src/ai/                # Primary ownership with full control
├── router.rs         # Main routing logic
├── selector.rs       # Model selection algorithm
├── cache.rs         # Response caching
└── graph_integration.rs # Graph system integration

src/providers/        # AI provider implementations
├── claude.rs        # Anthropic Claude API
├── openai.rs        # OpenAI API
├── gemini.rs        # Google Gemini API
└── ollama.rs        # Local Ollama

src/routing/          # Routing strategies
src/prompts/          # Prompt optimization
├── engine.rs        # Optimization engine
├── templates.rs     # Template library
└── optimizer.rs     # A/B testing

src/context/          # Conversation management

Integration Points

Depends On

• rust-developer: For API implementation patterns
• database-specialist: For conversation persistence
• monitoring-specialist: For usage metrics

Provides To

• frontend-developer: AI service interfaces
• orchestrator: AI coordination capabilities
• All agents: AI-powered assistance

Quality Standards

• Response Time: < 2 seconds end-to-end
• Cost Optimization: 40% reduction vs single provider
• Cache Hit Rate: > 60% for similar requests
• Concurrent Requests: 100+ supported
• Model Accuracy: Task-appropriate selection

CODI Integration

# Session initialization
export SESSION_ID="AI-SPECIALIST-SESSION-N"
codi-log "$SESSION_ID: Starting AI provider implementation" "SESSION_START"

# Development tracking
codi-log "$SESSION_ID: FILE_CLAIM src/ai/router.rs" "FILE_CLAIM"
codi-log "$SESSION_ID: Implementing multi-provider routing" "CREATE"

# Integration milestones
codi-log "$SESSION_ID: Claude provider integration complete" "UPDATE"
codi-log "$SESSION_ID: Cost optimization achieved 42% reduction" "PERFORMANCE"

# Completion
codi-log "$SESSION_ID: AI_READY all providers integrated" "AI_SYSTEM"
codi-log "$SESSION_ID: HANDOFF to frontend for UI integration" "HANDOFF"

Task Patterns

Primary Tasks

1. Provider Integration: Connect AI services with auth
2. Model Selection: Implement cost/quality algorithms
3. Prompt Optimization: A/B test and improve prompts
4. Response Caching: Reduce latency and costs
5. Context Management: Maintain conversation state

Delegation Triggers

• Delegates to database-specialist when: Conversation storage needed
• Delegates to security-specialist when: API key management required
• Delegates to monitoring-specialist when: Usage tracking needed
• Escalates to orchestrator when: Cross-service coordination required

Success Metrics

• Response time < 2 seconds
• Cost reduction > 40%
• Cache hit rate > 60%
• Zero API key exposure
• 99.9% availability

Example Workflows

Workflow 1: Add New Provider

1. Implement provider trait
2. Add authentication handling
3. Map response formats
4. Add to router registry
5. Test performance/costs
6. Update selection algorithm

Workflow 2: Optimize Prompts

1. Collect performance data
2. Identify improvement areas
3. Create prompt variants
4. A/B test with metrics
5. Deploy winning variants
6. Monitor improvements

Common Patterns

// Multi-provider routing
pub struct AIRouter {
    providers: HashMap<ProviderType, Box<dyn AIProvider>>,
    selector: ModelSelector,
    cache: Arc<ResponseCache>,
    metrics: Arc<UsageMetrics>,
}

#[async_trait]
pub trait AIProvider: Send + Sync {
    async fn complete(
        &self,
        request: &CompletionRequest,
    ) -> Result<CompletionResponse, AIError>;
    
    fn get_info(&self) -> ProviderInfo;
    fn estimate_cost(&self, tokens: usize) -> f64;
}

impl AIRouter {
    pub async fn route_request(
        &self,
        request: AIRequest,
        tenant_id: &str,
    ) -> Result<AIResponse, AIError> {
        // Check cache first
        if let Some(cached) = self.cache.get(&request.cache_key()).await? {
            self.metrics.record_cache_hit(tenant_id);
            return Ok(cached);
        }
        
        // Select optimal provider
        let provider = self.selector.select_provider(
            &request.task_type,
            &request.constraints,
            self.get_provider_states().await?,
        )?;
        
        // Route request
        let start = Instant::now();
        let response = provider.complete(&request.into()).await?;
        let duration = start.elapsed();
        
        // Track metrics
        self.metrics.record_request(
            tenant_id,
            &provider.get_info().name,
            response.tokens_used,
            duration,
            provider.estimate_cost(response.tokens_used),
        ).await?;
        
        // Cache response
        self.cache.put(
            request.cache_key(),
            response.clone(),
            Duration::hours(24),
        ).await?;
        
        Ok(response)
    }
}

// Model selection algorithm
pub struct ModelSelector {
    task_models: HashMap<TaskType, Vec<ModelPreference>>,
}

impl ModelSelector {
    pub fn select_provider(
        &self,
        task_type: &TaskType,
        constraints: &Constraints,
        provider_states: Vec<ProviderState>,
    ) -> Result<&Box<dyn AIProvider>, AIError> {
        let preferences = self.task_models.get(task_type)
            .ok_or(AIError::UnknownTaskType)?;
        
        // Score each provider
        let scores: Vec<(usize, f64)> = preferences
            .iter()
            .enumerate()
            .filter_map(|(idx, pref)| {
                let state = provider_states.iter()
                    .find(|s| s.provider == pref.provider)?;
                
                if !state.available || state.rate_limited {
                    return None;
                }
                
                let mut score = pref.base_score;
                
                // Adjust for constraints
                if constraints.max_latency_ms < 1000 {
                    score *= state.avg_latency_ms as f64 / 1000.0;
                }
                
                if constraints.optimize_cost {
                    score *= 1.0 / pref.relative_cost;
                }
                
                Some((idx, score))
            })
            .collect();
        
        // Select highest scoring available provider
        scores.into_iter()
            .max_by(|a, b| a.1.partial_cmp(&b.1).unwrap())
            .map(|(idx, _)| &preferences[idx].provider)
            .ok_or(AIError::NoAvailableProvider)
    }
}

// Prompt optimization engine
pub struct PromptEngine {
    templates: Arc<RwLock<PromptTemplateLibrary>>,
    optimizer: PromptOptimizer,
    graph: Arc<PromptGraph>, // ADR-021 integration
}

impl PromptEngine {
    pub async fn optimize_prompt(
        &self,
        task: &Task,
        context: &Context,
    ) -> Result<OptimizedPrompt, PromptError> {
        // Find similar successful prompts
        let similar = self.graph.find_similar_tasks(task, 10).await?;
        
        // Get base template
        let template = self.templates.read().await
            .get_template(&task.task_type)?;
        
        // Apply optimizations from similar tasks
        let optimized = self.optimizer.optimize(
            template,
            &similar,
            context,
        )?;
        
        // Track for future optimization
        self.graph.record_prompt_usage(
            &optimized,
            task,
        ).await?;
        
        Ok(optimized)
    }
}

// WebSocket AI bridge
pub struct AIWebSocketBridge {
    router: Arc<AIRouter>,
    sessions: Arc<DashMap<String, SessionContext>>,
}

impl AIWebSocketBridge {
    pub async fn handle_message(
        &self,
        session_id: &str,
        message: AIWebSocketMessage,
    ) -> Result<AIWebSocketResponse, AIError> {
        let context = self.sessions.entry(session_id.to_string())
            .or_insert_with(|| SessionContext::new(session_id));
        
        match message {
            AIWebSocketMessage::Complete { prompt, model } => {
                let request = AIRequest {
                    prompt,
                    model: model.or(context.preferred_model.clone()),
                    context: context.to_string(),
                    task_type: TaskType::Completion,
                    constraints: Default::default(),
                };
                
                let response = self.router.route_request(
                    request,
                    &context.tenant_id,
                ).await?;
                
                // Update context
                context.add_exchange(&prompt, &response.content);
                
                Ok(AIWebSocketResponse::Completion {
                    content: response.content,
                    model_used: response.model,
                    tokens: response.tokens_used,
                })
            }
        }
    }
}

Anti-Patterns to Avoid

• Don't expose API keys in logs or responses
• Avoid blocking on AI requests
• Never skip response validation
• Don't ignore rate limits
• Avoid unbounded context growth

References

• ADR-007: AI Router & Caching
• ADR-019: Prompt Engine Server
• ADR-021: Unified Hybrid Graph System
• AI Integration Guide