Circular Fix Detection System

Status: Production Ready Version: 1.0.0 Priority: MEDIUM Inspired By: Auto-Claude patterns (implemented from scratch)

What It Does

Automatically detects when an AI agent is repeatedly attempting similar fixes that have already failed, preventing infinite loops and wasting computational resources.

Example:

Attempt 1: "Fixed import error by adding import statement"
           → Failed: ModuleNotFoundError

Attempt 2: "Updated import to include module"
           → Failed: ModuleNotFoundError
           → Similarity: 30%

Attempt 3: "Modified import statement"
           → 🚨 CIRCULAR DETECTED!
           → Suggestion: "Check if package is installed: pip install requests"

Key Features

✅ Jaccard Similarity Algorithm - Efficient set-based comparison ✅ Token Normalization - Remove stopwords and stem programming terms ✅ Configurable Thresholds - Tune detection sensitivity ✅ Alternative Strategies - Smart suggestions when circular detected ✅ Visual Timeline - See similarity scores at a glance ✅ Auto-Cleanup - Remove history on success ✅ Summary Statistics - Track patterns across all tasks

Quick Start

1. Install (Already Included in CODITECT)

# Files created:
scripts/circular-fix-detector.py          # Main implementation
config/circular-fix-config.json           # Configuration
data/stopwords.txt                        # Programming stopwords
scripts/test-circular-detector.py         # Test suite

2. Run Tests

python3 scripts/test-circular-detector.py
# Output: ✅ ALL TESTS COMPLETED SUCCESSFULLY

3. Basic Usage

from scripts.circular_fix_detector import CircularFixDetector, FixOutcome

detector = CircularFixDetector()

# Record attempt
detector.record_attempt(
    subtask_id="fix-bug-123",
    approach="Updated authentication logic",
    files_modified=["auth.py"],
    outcome=FixOutcome.FAILED
)

# Check if next approach is circular
result = detector.is_circular_fix(
    "fix-bug-123",
    "Modified authentication flow"
)

if result.is_circular:
    print(f"⚠️  {result.reasoning}")
    print(f"Suggested: {result.suggested_strategy.value}")

How It Works

Jaccard Similarity

Similarity = |Intersection| / |Union|

Example:
Text 1: "Fixed import statement in main.py"
Text 2: "Updated import in main.py"

Tokens 1: {fix, import, statement, main}
Tokens 2: {update, import, main}

Intersection: {import, main} = 2
Union: {fix, update, import, statement, main} = 5

Similarity = 2/5 = 40%

Circular Detection

DEFAULT_THRESHOLD = 0.30  # 30% similarity
MIN_SIMILAR = 2           # At least 2 similar attempts

if similar_count >= 2:
    → CIRCULAR DETECTED
    → Suggest alternative strategy

Alternative Strategies

Pattern	Strategy	Action
>5 attempts	`escalate_human`	Get human help
>70% similarity	`different_method`	Try completely different approach
Same 1-2 files	`different_files`	Modify other related files
Default	`analyze_root_cause`	Deeper analysis needed

Configuration

Edit config/circular-fix-config.json:

{
  "similarity_threshold": 0.30,      // 30% word overlap
  "min_similar_attempts": 2,          // At least 2 similar
  "lookback_window": 5,               // Check last 5 attempts
  "auto_cleanup_on_success": true     // Remove on success
}

Tuning:

More sensitive (catch more patterns): Lower threshold to 0.25
Less sensitive (fewer false positives): Raise threshold to 0.40
Earlier detection: Set min_similar_attempts to 2
Conservative: Set min_similar_attempts to 3

File Structure

coditect-core/
├── scripts/
│   ├── circular-fix-detector.py       # Main implementation (19KB)
│   └── test-circular-detector.py      # Test suite (15KB)
├── config/
│   └── circular-fix-config.json       # Configuration (3.5KB)
├── data/
│   └── stopwords.txt                  # Programming stopwords (1.6KB)
├── docs/guides/
│   ├── CIRCULAR-FIX-DETECTION-GUIDE.md    # Complete guide (26KB)
│   └── CIRCULAR-FIX-DETECTION-README.md   # This file
└── .coditect/
    └── fix-attempts.json              # Attempt history (auto-generated)

CLI Examples

# Record an attempt
python3 scripts/circular-fix-detector.py \
  --subtask "fix-login" \
  --approach "Updated login logic" \
  --files src/auth.py \
  --outcome failed

# Check if circular
python3 scripts/circular-fix-detector.py \
  --subtask "fix-login" \
  --approach "Modified login authentication" \
  --check

# Visualize attempt history
python3 scripts/circular-fix-detector.py \
  --subtask "fix-login" \
  --visualize

# Get summary stats
python3 scripts/circular-fix-detector.py --summary

Integration

With Recovery Manager

from scripts.circular_fix_detector import CircularFixDetector, AlternativeStrategy
from scripts.recovery_manager import RecoveryManager

detector = CircularFixDetector()
recovery = RecoveryManager()

# Check before executing fix
result = detector.is_circular_fix(subtask.id, approach)

if result.is_circular:
    if result.suggested_strategy == AlternativeStrategy.ESCALATE_HUMAN:
        recovery.escalate_to_human(subtask.id, result.reasoning)
    elif result.suggested_strategy == AlternativeStrategy.DIFFERENT_FILES:
        alternative_files = find_related_files(subtask)
        approach = create_fix_for_files(alternative_files)

With Orchestrator

class Orchestrator:
    def __init__(self):
        self.detector = CircularFixDetector()

    def execute_subtask(self, subtask):
        for attempt in range(5):
            approach = self.plan_fix(subtask)

            # Check circular
            result = self.detector.is_circular_fix(subtask.id, approach)
            if result.is_circular:
                approach = self.apply_alternative_strategy(result)

            # Execute and record
            outcome = self.execute_fix(approach)
            self.detector.record_attempt(
                subtask.id, approach, subtask.files, outcome
            )

            if outcome == FixOutcome.SUCCESS:
                break

Real-World Example

Problem: Agent stuck in import error loop

# Attempt 1
detector.record_attempt(
    "fix-import",
    "Added import requests statement",
    ["api_client.py"],
    FixOutcome.FAILED,
    "ModuleNotFoundError: No module named 'requests'"
)

# Attempt 2
detector.record_attempt(
    "fix-import",
    "Changed to from requests import *",
    ["api_client.py"],
    FixOutcome.FAILED,
    "ModuleNotFoundError: No module named 'requests'"
)

# Check next attempt
result = detector.is_circular_fix(
    "fix-import",
    "Modified import to use absolute path"
)

# Result:
# is_circular = True
# suggested_strategy = analyze_root_cause
# reasoning = "Detected circular pattern (2 similar attempts).
#              Recommend checking if requests is installed:
#              pip install requests"

Testing

# Run comprehensive test suite
python3 scripts/test-circular-detector.py

# Tests:
# ✅ Jaccard similarity calculation
# ✅ Token normalization with stopwords
# ✅ Circular pattern detection
# ✅ Alternative strategy suggestions
# ✅ Real-world import error scenario
# ✅ Automatic cleanup on success
# ✅ Summary statistics

Performance

Efficiency:

O(n) token normalization
O(n) set operations for Jaccard
O(k) similarity comparisons (k = lookback window)
Total: O(n * k) per check

Storage:

~1KB per attempt record
Default: 5 attempts × 50 subtasks = 250KB
Auto-cleanup on success reduces storage

Speed:

Similarity calculation: <1ms
Circular detection: <5ms
Suitable for real-time orchestration

Roadmap

Future Enhancements

Semantic Similarity - Use embeddings instead of Jaccard
Pattern Learning - Learn common failure patterns over time
Code Fix Suggestions - Suggest specific code changes
Integration Dashboard - Web UI for visualization
Multi-Subtask Analysis - Detect patterns across related tasks

Integration Targets

Recovery Manager (P0)
Orchestrator (P0)
Agent Framework (P1)
Dashboard Metrics (P2)

Complete User Guide - 26KB comprehensive guide
Recovery Manager - Error recovery system
Orchestrator Guide - Multi-agent coordination

Credits

Inspired By: Auto-Claude circular fix detection patterns Implemented By: CODITECT Core Team Implementation: Original from scratch (no code copying) Algorithm: Jaccard similarity with programming-aware normalization

Last Updated: December 22, 2025 Version: 1.0.0 Status: Production Ready License: Proprietary (AZ1.AI INC)

What It Does​

Key Features​

Quick Start​

1. Install (Already Included in CODITECT)​

2. Run Tests​

3. Basic Usage​

How It Works​

Jaccard Similarity​

Circular Detection​

Alternative Strategies​

Configuration​

File Structure​

CLI Examples​

Integration​

With Recovery Manager​

With Orchestrator​

Real-World Example​

Testing​

Performance​

Roadmap​

Future Enhancements​

Integration Targets​

Related Documentation​

Credits​