ADR-006: Conversation-to-Commit Correlation
Status: Proposed Date: 2025-11-26 Deciders: Architecture Team, Product Team Related ADRs: ADR-002 (PostgreSQL + Weaviate), ADR-005 (Hybrid Search)
Context
A unique competitive advantage of the context intelligence platform is the ability to link AI conversations to Git commits, answering questions like:
User Stories
1. "What conversation led to this commit?"
User: Sees commit "feat: Add JWT authentication"
Question: "What was the AI discussion that led to this implementation?"
Answer: Links to conversation "Should we use JWT or session cookies?"
2. "What was the outcome of this conversation?"
User: Reads conversation "We need to optimize database queries"
Question: "What code changes were made as a result?"
Answer: Links to commits:
- "perf: Add index on user_id column" (2 hours later)
- "perf: Cache user queries in Redis" (4 hours later)
3. "Show me all conversations and commits related to authentication"
User: Searches "authentication"
Answer: Shows BOTH conversations AND commits, correlated:
- Conversation #42: "Auth strategy discussion" → Commit abc123 (2 hours later)
- Commit xyz789: "Fix auth bug" → Conversation #15 (bug report 1 hour before)
Market Differentiation
Current Market Gap: No product provides conversation-to-commit linking
| Competitor | AI Conversations | Git Commits | Correlation |
|---|---|---|---|
| GitHub Copilot | ✅ (in IDE) | ✅ (GitHub) | ❌ No linking |
| Cursor | ✅ (in IDE) | ❌ | ❌ |
| LinearB | ❌ | ✅ | ❌ |
| Our Platform | ✅ | ✅ | ✅ UNIQUE |
Value Proposition: "The only platform that shows the full story: from conversation to code"
Decision
We will implement multi-signal correlation using three signals:
- Temporal Proximity (60% weight): Conversation timestamp ± 30 minutes of commit
- Semantic Similarity (30% weight): Conversation content similar to commit message
- Explicit Tags (10% weight): User manually links conversation to commit
Architecture
┌─────────────────────────────────────────────────────────────┐
│ Correlation Engine (Celery Background Task) │
└─────────────────────────┬───────────────────────────────────┘
↓
┌─────────────────┴─────────────────┐
↓ ↓
┌──────────────────┐ ┌──────────────────┐
│ Signal 1: │ │ Signal 2: │
│ Temporal │ │ Semantic │
│ Proximity │ │ Similarity │
│ │ │ │
│ ┌──────────────┐ │ │ ┌──────────────┐ │
│ │ If commit │ │ │ │ Embed both: │ │
│ │ timestamp │ │ │ │ - Convo text │ │
│ │ within │ │ │ │ - Commit msg │ │
│ │ ±30 min of │ │ │ │ │ │
│ │ conversation │ │ │ │ Cosine │ │
│ │ │ │ │ │ similarity │ │
│ │ Score: 1.0 │ │ │ │ │ │
│ │ │ │ │ │ Score: 0-1 │ │
│ └──────────────┘ │ │ └──────────────┘ │
└──────────────────┘ └──────────────────┘
↓ ↓
└─────────────────┬─────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Signal 3: Explicit Tags (User-Provided) │
│ │
│ User manually links conversation #42 to commit abc123 │
│ Score: 1.0 (100% confidence) │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Combined Score Calculation │
│ │
│ final_score = (0.6 × temporal) + │
│ (0.3 × semantic) + │
│ (0.1 × explicit) │
│ │
│ Example: │
│ - Conversation: "Let's add JWT auth" (10:00 AM) │
│ - Commit: "feat: Add JWT authentication" (10:15 AM) │
│ │
│ temporal = 1.0 (within 30 min) │
│ semantic = 0.85 (high similarity: "JWT auth" matches) │
│ explicit = 0.0 (no user tag) │
│ │
│ final_score = (0.6 × 1.0) + (0.3 × 0.85) + (0.1 × 0.0) │
│ = 0.6 + 0.255 + 0.0 │
│ = 0.855 (85.5% confidence) ✅ HIGH CONFIDENCE │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Store Correlation in Database │
│ │
│ CREATE TABLE conversation_commit_links ( │
│ conversation_id UUID, │
│ commit_hash TEXT, │
│ confidence_score FLOAT, -- 0.0 to 1.0 │
│ temporal_score FLOAT, │
│ semantic_score FLOAT, │
│ explicit_score FLOAT, │
│ created_at TIMESTAMPTZ │
│ ); │
└─────────────────────────────────────────────────────────────┘
Implementation
Step 1: Database Schema
-- Conversation-to-commit link table
CREATE TABLE conversation_commit_links (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
organization_id UUID NOT NULL REFERENCES organizations(id) ON DELETE CASCADE,
-- Conversation side
conversation_id UUID NOT NULL REFERENCES conversations(id) ON DELETE CASCADE,
-- Commit side
repository TEXT NOT NULL,
commit_hash TEXT NOT NULL,
commit_id UUID REFERENCES commits(id) ON DELETE SET NULL, -- Optional FK
-- Correlation scores
confidence_score FLOAT NOT NULL CHECK (confidence_score BETWEEN 0 AND 1),
temporal_score FLOAT CHECK (temporal_score BETWEEN 0 AND 1),
semantic_score FLOAT CHECK (semantic_score BETWEEN 0 AND 1),
explicit_score FLOAT CHECK (explicit_score BETWEEN 0 AND 1),
-- Metadata
correlation_method TEXT CHECK (correlation_method IN ('auto', 'manual')),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(organization_id, conversation_id, commit_hash)
);
-- Indexes for fast lookups
CREATE INDEX idx_links_conversation ON conversation_commit_links(conversation_id);
CREATE INDEX idx_links_commit ON conversation_commit_links(repository, commit_hash);
CREATE INDEX idx_links_confidence ON conversation_commit_links(confidence_score DESC);
-- RLS policy (multi-tenant isolation)
ALTER TABLE conversation_commit_links ENABLE ROW LEVEL SECURITY;
CREATE POLICY organization_isolation ON conversation_commit_links
FOR ALL
TO authenticated_users
USING (organization_id = current_setting('app.current_organization_id')::uuid);
Step 2: Correlation Service
# core/services/correlation_service.py
from datetime import datetime, timedelta
from typing import List, Optional
import asyncio
import uuid
class CorrelationService:
"""Correlate conversations with Git commits"""
def __init__(
self,
conversation_repo: ConversationRepository,
commit_repo: CommitRepository,
weaviate_client: WeaviateClient,
temporal_window_minutes: int = 30,
confidence_threshold: float = 0.5
):
self.conversation_repo = conversation_repo
self.commit_repo = commit_repo
self.weaviate_client = weaviate_client
self.temporal_window = timedelta(minutes=temporal_window_minutes)
self.confidence_threshold = confidence_threshold
async def correlate_conversation_to_commits(
self,
conversation_id: uuid.UUID,
organization_id: uuid.UUID
) -> List[CorrelationLink]:
"""Find commits related to a conversation"""
# Get conversation
conversation = await self.conversation_repo.get(conversation_id)
if not conversation:
return []
# Find commits within temporal window
commit_candidates = await self.commit_repo.find_in_time_range(
organization_id=organization_id,
start_time=conversation.created_at - self.temporal_window,
end_time=conversation.created_at + self.temporal_window
)
# Calculate correlation scores for each candidate
correlations = []
for commit in commit_candidates:
temporal_score = self._temporal_score(
conversation.created_at,
commit.commit_date
)
semantic_score = await self._semantic_similarity(
conversation.title + " " + conversation.content,
commit.message
)
# Check for explicit tags
explicit_score = await self._explicit_tag_score(
conversation_id,
commit.commit_hash
)
# Calculate combined confidence score
confidence = (
0.6 * temporal_score +
0.3 * semantic_score +
0.1 * explicit_score
)
if confidence >= self.confidence_threshold:
correlations.append(
CorrelationLink(
conversation_id=conversation_id,
commit_hash=commit.commit_hash,
repository=commit.repository,
confidence_score=confidence,
temporal_score=temporal_score,
semantic_score=semantic_score,
explicit_score=explicit_score,
correlation_method='auto'
)
)
return correlations
def _temporal_score(
self,
conversation_time: datetime,
commit_time: datetime
) -> float:
"""Calculate temporal proximity score"""
time_diff = abs((commit_time - conversation_time).total_seconds())
window_seconds = self.temporal_window.total_seconds()
if time_diff <= window_seconds:
# Linear decay: 1.0 at 0 seconds, 0.0 at window edge
return 1.0 - (time_diff / window_seconds)
else:
return 0.0
async def _semantic_similarity(
self,
conversation_text: str,
commit_message: str
) -> float:
"""Calculate semantic similarity using embeddings"""
# Generate embeddings (cached for performance)
conv_embedding = await self.weaviate_client.embed_text(conversation_text)
commit_embedding = await self.weaviate_client.embed_text(commit_message)
# Cosine similarity
similarity = self._cosine_similarity(conv_embedding, commit_embedding)
return max(0.0, min(1.0, similarity)) # Clamp to [0, 1]
async def _explicit_tag_score(
self,
conversation_id: uuid.UUID,
commit_hash: str
) -> float:
"""Check if user manually linked conversation to commit"""
# Query database for explicit link
link = await self.conversation_repo.get_explicit_link(
conversation_id=conversation_id,
commit_hash=commit_hash
)
return 1.0 if link else 0.0
@staticmethod
def _cosine_similarity(vec1: List[float], vec2: List[float]) -> float:
"""Cosine similarity between two vectors"""
import numpy as np
v1 = np.array(vec1)
v2 = np.array(vec2)
return np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))
Step 3: Celery Background Task
# core/tasks/correlation_tasks.py
from celery import shared_task
import uuid
@shared_task(bind=True, max_retries=3)
def correlate_new_commit(self, commit_hash: str, organization_id: str):
"""Background task: Correlate new commit with conversations"""
try:
org_id = uuid.UUID(organization_id)
# Initialize services
correlation_service = get_correlation_service()
# Find conversations within temporal window
commit = await commit_repo.get_by_hash(commit_hash)
conversations = await conversation_repo.find_in_time_range(
organization_id=org_id,
start_time=commit.commit_date - timedelta(minutes=30),
end_time=commit.commit_date + timedelta(minutes=30)
)
# Correlate commit to each conversation
for conversation in conversations:
correlations = await correlation_service.correlate_conversation_to_commits(
conversation_id=conversation.id,
organization_id=org_id
)
# Save correlations to database
for correlation in correlations:
await correlation_repo.create(correlation)
except Exception as exc:
# Retry with exponential backoff
raise self.retry(exc=exc, countdown=2 ** self.request.retries)
# Trigger: When new commit ingested via webhook
async def on_commit_webhook(commit_data: dict):
# Save commit to database
commit = await commit_repo.create(commit_data)
# Trigger background correlation (async, non-blocking)
correlate_new_commit.delay(
commit_hash=commit.commit_hash,
organization_id=str(commit.organization_id)
)
Step 4: API Endpoints
# FastAPI (Standalone Mode)
@router.get("/conversations/{conversation_id}/commits")
async def get_related_commits(
conversation_id: uuid.UUID,
min_confidence: float = Query(0.5, ge=0.0, le=1.0),
current_user = Depends(get_current_user)
):
"""Get commits related to a conversation"""
links = await correlation_service.get_links_for_conversation(
conversation_id=conversation_id,
organization_id=current_user.organization_id,
min_confidence=min_confidence
)
return {
"conversation_id": str(conversation_id),
"commits": [
{
"commit_hash": link.commit_hash,
"repository": link.repository,
"confidence": link.confidence_score,
"scores": {
"temporal": link.temporal_score,
"semantic": link.semantic_score,
"explicit": link.explicit_score
}
}
for link in links
]
}
@router.get("/commits/{commit_hash}/conversations")
async def get_related_conversations(
commit_hash: str,
repository: str = Query(...),
min_confidence: float = Query(0.5, ge=0.0, le=1.0),
current_user = Depends(get_current_user)
):
"""Get conversations related to a commit"""
links = await correlation_service.get_links_for_commit(
commit_hash=commit_hash,
repository=repository,
organization_id=current_user.organization_id,
min_confidence=min_confidence
)
return {
"commit_hash": commit_hash,
"repository": repository,
"conversations": [
{
"conversation_id": str(link.conversation_id),
"title": link.conversation.title,
"confidence": link.confidence_score
}
for link in links
]
}
@router.post("/conversations/{conversation_id}/commits/{commit_hash}/link")
async def manual_link(
conversation_id: uuid.UUID,
commit_hash: str,
repository: str = Query(...),
current_user = Depends(get_current_user)
):
"""Manually link conversation to commit (explicit tag)"""
link = CorrelationLink(
conversation_id=conversation_id,
commit_hash=commit_hash,
repository=repository,
confidence_score=1.0,
temporal_score=0.0,
semantic_score=0.0,
explicit_score=1.0,
correlation_method='manual'
)
await correlation_repo.create(link)
return {"status": "linked", "confidence": 1.0}
Consequences
Positive
-
✅ Unique Competitive Advantage
- Only platform linking AI conversations to Git commits
- Answers "What was the conversation behind this code?"
- Answers "What code resulted from this discussion?"
-
✅ Multi-Signal Robustness
- Temporal: Catches commits made shortly after conversations
- Semantic: Catches commits with similar content (even if not same time)
- Explicit: Allows users to correct false positives/negatives
-
✅ Configurable Confidence Threshold
- Default: 0.5 (balanced precision vs. recall)
- Conservative users: 0.7 (high precision, fewer false positives)
- Exploratory users: 0.3 (high recall, more suggestions)
-
✅ Background Processing (Non-Blocking)
- Correlation runs asynchronously (Celery)
- Users don't wait for correlations to complete
- Can reprocess historical data (backfill)
-
✅ Explainable Scores
- Users see WHY conversation was linked to commit:
Confidence: 85%
- Temporal: 100% (15 minutes after conversation)
- Semantic: 65% (similar keywords: "auth", "JWT")
- Explicit: 0% (not manually tagged)
- Users see WHY conversation was linked to commit:
Negative
-
⚠️ Correlation Accuracy Target: 80-90%
- Challenge: No ground truth dataset to validate accuracy
- Risk: False positives (unrelated conversation linked to commit)
- Risk: False negatives (related conversation NOT linked)
- Mitigation:
- User feedback loop (thumbs up/down on correlations)
- Manual override (explicit tags)
- A/B test weight tuning (0.6/0.3/0.1 vs. 0.5/0.4/0.1)
-
⚠️ Embedding Cost
- Problem: Must embed both conversations AND commit messages
- Cost: $0.0001 per 1K tokens (OpenAI text-embedding-3-large)
- Example: 100K commits × 50 tokens = 5M tokens = $0.50
- Mitigation: Cache embeddings, batch processing, monthly budget cap
-
⚠️ Temporal Window Tuning
- Problem: Optimal window varies by team:
- Fast-moving teams: 15-minute window
- Research teams: 24-hour window
- Mitigation Phase 1: Default 30 minutes (good for 80% of teams)
- Future Enhancement: Per-organization configuration
- Problem: Optimal window varies by team:
-
⚠️ Backfill Complexity
- Problem: Must correlate historical conversations + commits
- Scale: 11,925 messages × 4,506 commits = 53M comparisons (naïve)
- Optimization: Only correlate within temporal window (reduces to ~1K comparisons per commit)
- Time: ~1 hour to backfill 10K commits (acceptable for one-time operation)
Risks and Mitigations
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| Low Accuracy (<80%) | Medium | High | User feedback loop, weight tuning, explicit tags |
| High False Positive Rate | Medium | Medium | Increase confidence threshold (0.5 → 0.7) |
| Embedding Cost Spike | Low | Medium | Cache embeddings, batch processing, cap at $100/month |
| Slow Backfill | Low | Low | Run during off-hours, incremental processing |
Alternatives Considered
Alternative 1: Temporal Proximity Only (No Semantic Similarity)
Architecture: Link conversation to commit if timestamp within ±30 minutes
Pros:
- ✅ Simple implementation (no ML, no embeddings)
- ✅ Fast (just timestamp comparison)
- ✅ No embedding cost
Cons:
- ❌ Poor recall: Misses commits made days after conversation
- ❌ Poor precision: Many unrelated commits happen within 30 minutes
- ❌ Example failure: Discussion at 10 AM, commit at 2 PM = not linked (should be)
Why Rejected: Temporal alone is insufficient. Developers often discuss features days before implementing them.
Alternative 2: Semantic Similarity Only (No Temporal Proximity)
Architecture: Link conversation to commit if semantic similarity >0.7
Pros:
- ✅ Catches commits made days/weeks later
- ✅ No arbitrary time window
Cons:
- ❌ Poor precision: Many unrelated commits have similar keywords
- ❌ Example failure: "Add login" matches 50 commits (need temporal filter)
- ❌ High cost: Must embed ALL commits, even from months ago
Why Rejected: Semantic alone generates too many false positives. Temporal filter is essential.
Alternative 3: Rule-Based (Regex + Keywords)
Architecture: Extract keywords from conversation, match to commit message
Example:
if "JWT" in conversation and "JWT" in commit.message:
link = True
Pros:
- ✅ No ML required
- ✅ Explainable (exact keyword match)
Cons:
- ❌ Brittle: Requires maintaining keyword lists
- ❌ Poor recall: "authentication" doesn't match "auth", "login"
- ❌ No semantic understanding: "bug" doesn't match "error"
Why Rejected: Too brittle. Semantic embeddings are more robust and require less manual tuning.
Alternative 4: User Tags Only (No Automatic Correlation)
Architecture: Users manually tag conversations with commit hashes
Pros:
- ✅ 100% precision (no false positives)
- ✅ No ML complexity
- ✅ No background processing
Cons:
- ❌ Poor adoption: Users won't manually tag (extra work)
- ❌ Low coverage: Most conversations never linked
- ❌ No discovery: Can't answer "What conversation led to this commit?"
Why Rejected: Requires too much user effort. Automatic correlation is critical for adoption.
Success Metrics
Correlation Quality Metrics
- Accuracy: 80-90% (measured via user feedback)
- Precision: 80%+ (relevant correlations among suggestions)
- Recall: 70%+ (find existing links)
- User feedback rate: 20%+ users provide thumbs up/down
Performance Metrics
- Correlation latency: <5 seconds per commit (background task)
- Backfill time: <1 hour for 10K commits
- Embedding cost: <$100/month for 100K commits
Usage Metrics
- View rate: 40%+ users view related commits/conversations
- Manual link rate: 10%+ users manually link (explicit tags)
- Feature adoption: 60%+ users click on correlation links
Implementation Plan
Phase 1: Database Schema (Week 1)
- Create
conversation_commit_linkstable - Add indexes for fast lookups
- Write migration script
Phase 2: Temporal Correlation (Week 2)
- Implement
_temporal_score()method - Test with 100 conversations + 1K commits
- Tune temporal window (15 vs. 30 vs. 60 minutes)
Phase 3: Semantic Correlation (Week 3)
- Integrate Weaviate embedding generation
- Implement
_semantic_similarity()method - Cache embeddings for performance
- Test with 100 conversations + 1K commits
Phase 4: Integration & Backfill (Week 4)
- Implement Celery background tasks
- Add API endpoints (GET related commits/conversations)
- Backfill historical data (11K messages + 4.5K commits)
- User feedback UI (thumbs up/down)
Phase 5: Tuning & Optimization (Week 5)
- A/B test weight combinations:
- 0.6/0.3/0.1 (default)
- 0.5/0.4/0.1 (higher semantic weight)
- 0.7/0.2/0.1 (higher temporal weight)
- Tune confidence threshold (0.5 vs. 0.6 vs. 0.7)
- Collect user feedback (1 month)
- Adjust weights based on feedback
References
Research:
- Commit Message Generation from Diffs - Jiang et al., 2017
- Semantic Similarity using Sentence-BERT - Reimers & Gurevych, 2019
Similar Features (not conversation-to-commit, but related):
- LinearB: Git commit analytics (no AI conversations)
- GitHub Copilot: AI suggestions (no commit correlation)
- Rewind AI: Screen recording (no code correlation)
Related ADRs:
- ADR-002: PostgreSQL + Weaviate (embedding generation)
- ADR-005: Hybrid Search (extends to commit search)
Status: Proposed Review Date: 2025-12-03 Projected ADR Score: 37/40 (A) Complexity: High (ML, multi-signal fusion, accuracy unknown) Owner: Architecture Team + ML Team + Product Team
Next Steps:
- Approve multi-signal correlation approach
- Implement temporal correlation (Week 1-2)
- Add semantic similarity (Week 3)
- Backfill historical data (Week 4)
- Collect user feedback and tune weights (Week 5)