Circular Fix Detection System - User Guide

Version: 1.0.0 Last Updated: December 22, 2025 Status: Production Ready

Overview

The Circular Fix Detection System automatically detects when an AI agent is repeatedly attempting similar fixes that have already failed, preventing infinite loops and suggesting alternative strategies.

Key Features:

Jaccard similarity-based pattern detection
Configurable thresholds and detection parameters
Alternative strategy suggestions
Visual similarity timeline
Automatic cleanup on success
Summary statistics and reporting

Quick Start

Basic Usage

from scripts.circular_fix_detector import CircularFixDetector, FixOutcome

# Initialize detector
detector = CircularFixDetector()

# Record a fix attempt
detector.record_attempt(
    subtask_id="fix-login-bug",
    approach="Updated authentication logic in login.py",
    files_modified=["src/auth/login.py"],
    outcome=FixOutcome.FAILED,
    error_message="Authentication still failing"
)

# Check if next approach is circular
result = detector.is_circular_fix(
    subtask_id="fix-login-bug",
    current_approach="Modified authentication in login module"
)

if result.is_circular:
    print(f"⚠️  Circular pattern detected!")
    print(f"Suggested: {result.suggested_strategy.value}")
    print(f"Reasoning: {result.reasoning}")

CLI Usage

# Record an attempt
python3 scripts/circular-fix-detector.py \
  --subtask "fix-login-bug" \
  --approach "Updated authentication logic" \
  --files src/auth/login.py \
  --outcome failed

# Check if circular
python3 scripts/circular-fix-detector.py \
  --subtask "fix-login-bug" \
  --approach "Modified authentication" \
  --check

# Visualize history
python3 scripts/circular-fix-detector.py \
  --subtask "fix-login-bug" \
  --visualize

# Get summary statistics
python3 scripts/circular-fix-detector.py --summary

How It Works

1. Jaccard Similarity Algorithm

The system uses Jaccard similarity to compare fix approaches:

Similarity = |Intersection| / |Union|

Example:

Approach 1: "Fixed import statement in main.py"
Approach 2: "Updated import statement in main.py"

Tokens 1: {fixed, import, statement, main}
Tokens 2: {updated, import, statement, main}

Intersection: {import, statement, main}  = 3 words
Union: {fixed, updated, import, statement, main} = 5 words

Similarity = 3/5 = 60%

2. Token Normalization

Before comparison, text is normalized:

Lowercase all text
Remove punctuation (except underscores in identifiers)
Split on whitespace
Remove stopwords (the, a, is, are, fix, update, etc.)
Basic stemming (fixing → fix, updated → update)

Example:

Original: "Fixed the import statement by adding it to main.py"
Tokens: [fix, statement, add, main]
(Removed: the, import, by, it, to)

3. Circular Detection Logic

# Default configuration
SIMILARITY_THRESHOLD = 0.30  # 30% word overlap
MIN_SIMILAR_ATTEMPTS = 2     # At least 2 similar attempts
LOOKBACK_WINDOW = 5          # Check last 5 attempts

# Detection algorithm:
1. Get recent attempts (lookback window)
2. Calculate similarity to each
3. Count attempts above threshold
4. If count >= min_similar_attempts → CIRCULAR

4. Alternative Strategy Suggestions

When circular pattern detected:

Condition	Strategy	Example
>5 attempts	`escalate_human`	"Already tried 7 times. Human intervention recommended."
>70% similarity	`different_method`	"Very high similarity. Try completely different approach."
Same 1-2 files	`different_files`	"Repeatedly modifying auth.py. Try different files."
Default	`analyze_root_cause`	"Detected circular pattern. Recommend deeper root cause analysis."

Configuration

Configuration File

Located at config/circular-fix-config.json:

{
  "detection_parameters": {
    "similarity_threshold": 0.30,
    "description": "Minimum Jaccard similarity (0.0-1.0)"
  },
  "pattern_detection": {
    "min_similar_attempts": 2,
    "description": "Minimum similar attempts to flag as circular"
  },
  "history_management": {
    "lookback_window": 5,
    "description": "Number of recent attempts to analyze"
  },
  "automation": {
    "auto_cleanup_on_success": true,
    "retention_days": 7
  }
}

Tuning Guidelines

Similarity Threshold

"similarity_threshold": 0.30  # Default: 30%

// Tuning:
0.20-0.30  → Catches broad patterns (recommended)
0.30-0.50  → Moderate similarity detection
0.50-0.70  → Only very similar approaches
>0.70      → Almost identical only

Min Similar Attempts

"min_similar_attempts": 2  # Default: 2

// Tuning:
2  → Early detection (may have false positives)
3  → Balanced detection (recommended)
4-5 → Conservative (may miss patterns)

Lookback Window

"lookback_window": 5  # Default: 5

// Tuning:
3-5   → Recent context only (fast, focused)
5-10  → Moderate history (recommended)
10+   → Deep analysis (slower, comprehensive)

Integration Examples

With Recovery Manager

from scripts.circular_fix_detector import CircularFixDetector
from scripts.recovery_manager import RecoveryManager

detector = CircularFixDetector()
recovery = RecoveryManager()

# Attempt a fix
attempt = detector.record_attempt(
    subtask_id=subtask.id,
    approach=fix_description,
    files_modified=modified_files,
    outcome=FixOutcome.FAILED
)

# Check if circular
result = detector.is_circular_fix(subtask.id, next_approach)

if result.is_circular:
    # Use suggested strategy
    if result.suggested_strategy == AlternativeStrategy.ESCALATE_HUMAN:
        recovery.escalate_to_human(subtask.id, result.reasoning)

    elif result.suggested_strategy == AlternativeStrategy.DIFFERENT_FILES:
        # Try fixing in different files
        alternative_files = find_related_files(subtask)
        next_fix = create_fix_for_files(alternative_files)

With Orchestrator

class TaskOrchestrator:
    def __init__(self):
        self.detector = CircularFixDetector()

    def execute_subtask(self, subtask):
        max_attempts = 5

        for attempt_num in range(1, max_attempts + 1):
            # Plan next approach
            approach = self.plan_fix_approach(subtask)

            # Check if circular
            result = self.detector.is_circular_fix(subtask.id, approach)

            if result.is_circular:
                logger.warning(f"Circular pattern: {result.reasoning}")

                # Apply suggested strategy
                if result.suggested_strategy == AlternativeStrategy.DIFFERENT_METHOD:
                    approach = self.generate_alternative_approach(subtask)
                elif result.suggested_strategy == AlternativeStrategy.ESCALATE_HUMAN:
                    return self.escalate_task(subtask)

            # Execute fix
            outcome = self.execute_fix(subtask, approach)

            # Record attempt
            self.detector.record_attempt(
                subtask.id,
                approach,
                subtask.files,
                outcome
            )

            if outcome == FixOutcome.SUCCESS:
                return True

        return False

Real-World Scenarios

Scenario 1: Import Error Loop

# Problem: Agent keeps trying to fix import by modifying import statement
# Solution: System detects circular pattern and suggests installing package

Attempt 1: "Added 'import requests' to file"
           → Failed: ModuleNotFoundError

Attempt 2: "Changed to 'from requests import *'"
           → Failed: ModuleNotFoundError
           → Similarity: 25%

Attempt 3: "Modified import to use absolute path"
           → Circular detected! (2 similar attempts)
           → Strategy: analyze_root_cause
           → Suggestion: "Check if requests is installed: pip install requests"

Scenario 2: Authentication Bug

# Problem: Agent repeatedly modifying same auth.py file
# Solution: System suggests trying different files

Attempt 1: "Fixed authentication in auth.py"
           → Failed

Attempt 2: "Updated login logic in auth.py"
           → Failed
           → Similarity: 40%

Attempt 3: "Modified auth validation in auth.py"
           → Failed
           → Similarity: 35%

Attempt 4: "Changed authentication flow in auth.py"
           → Circular detected!
           → Strategy: different_files
           → Suggestion: "Try modifying user.py, session.py, or database.py"

Scenario 3: Too Many Attempts

# Problem: Agent has tried 7 different approaches, all failed
# Solution: Escalate to human

Attempts 1-7: Various different approaches
              → All failed
              → Circular detected!
              → Strategy: escalate_human
              → Reasoning: "Already tried 7 times. Human intervention needed."

Visualization

Similarity Timeline

python3 scripts/circular-fix-detector.py --subtask fix-auth --visualize

Output:

=== Fix Attempt History for fix-auth ===

Total Attempts: 4

Attempt #1:
  Approach: Fixed authentication error by updating login method...
  Files: src/auth/login.py
  Outcome: failed
  Timestamp: 2025-12-22T10:00:00Z

Attempt #2:
  Approach: Modified login authentication to handle errors...
  Files: src/auth/login.py
  Outcome: failed
  Similarity to previous: [████████░░░░░░░░░░░░] 40.00%
  Timestamp: 2025-12-22T10:05:00Z

Attempt #3:
  Approach: Changed authentication flow in login module...
  Files: src/auth/login.py
  Outcome: failed
  Similarity to previous: [██████░░░░░░░░░░░░░░] 30.00%
  Timestamp: 2025-12-22T10:10:00Z

Attempt #4:
  Approach: Updated error handling in login authentication...
  Files: src/auth/login.py
  Outcome: failed
  Similarity to previous: [█████████░░░░░░░░░░░] 45.00%
  Timestamp: 2025-12-22T10:15:00Z

Testing

Run Test Suite

# Comprehensive tests
python3 scripts/test-circular-detector.py

# Output:
# ✅ Jaccard similarity calculation
# ✅ Circular pattern detection
# ✅ Alternative strategy suggestions
# ✅ Real-world scenarios
# ✅ Automatic cleanup
# ✅ Summary statistics

Manual Testing

from scripts.circular_fix_detector import CircularFixDetector, FixOutcome

detector = CircularFixDetector()

# Test similarity calculation
similarity = detector.calculate_jaccard_similarity(
    "Fixed the import statement",
    "Updated import statement"
)
print(f"Similarity: {similarity:.2%}")  # ~50%

# Test circular detection
for i in range(3):
    detector.record_attempt(
        "test-task",
        f"Fix attempt {i+1} for the bug",
        ["bug.py"],
        FixOutcome.FAILED
    )

result = detector.is_circular_fix("test-task", "Another fix for the bug")
print(f"Circular: {result.is_circular}")  # True

Best Practices

1. Tune Thresholds for Your Use Case

# For tight circular detection (avoid false positives):
config = {
    "similarity_threshold": 0.40,
    "min_similar_attempts": 3
}

# For loose detection (catch more patterns):
config = {
    "similarity_threshold": 0.25,
    "min_similar_attempts": 2
}

2. Record All Attempts

# Always record attempts, even successes
detector.record_attempt(
    subtask_id,
    approach,
    files,
    outcome  # FAILED, SUCCESS, PARTIAL, SKIPPED
)

3. Act on Suggestions

result = detector.is_circular_fix(subtask_id, approach)

if result.is_circular:
    # Don't ignore! Apply suggested strategy
    if result.suggested_strategy == AlternativeStrategy.DIFFERENT_METHOD:
        approach = generate_alternative()
    elif result.suggested_strategy == AlternativeStrategy.ESCALATE_HUMAN:
        notify_human(result.reasoning)

4. Monitor and Adjust

# Periodically review effectiveness
summary = detector.get_summary()

print(f"Circular detections: {summary['circular_count']}")
print(f"Average attempts: {summary['average_attempts_per_subtask']}")

# Tune if needed
if summary['average_attempts_per_subtask'] > 4:
    # Lower threshold to detect earlier
    config['similarity_threshold'] = 0.25

Troubleshooting

Issue: Too Many False Positives

Symptom: System flagging different approaches as circular

Solution:

// Increase threshold or min_similar_attempts
{
  "similarity_threshold": 0.40,  // Up from 0.30
  "min_similar_attempts": 3       // Up from 2
}

Issue: Missing Circular Patterns

Symptom: Agent loops without detection

Solution:

// Decrease threshold and increase lookback
{
  "similarity_threshold": 0.25,  // Down from 0.30
  "lookback_window": 10          // Up from 5
}

Issue: Stopwords Too Aggressive

Symptom: Important technical terms being removed

Solution:

# Edit data/stopwords.txt
# Remove domain-specific terms that should NOT be ignored
# Example: Remove 'import', 'module', 'package' for import errors

API Reference

CircularFixDetector

Methods

__init__(config_path: Optional[Path] = None)

Initialize detector with optional custom config.

detector = CircularFixDetector()
# or
detector = CircularFixDetector(config_path=Path("custom-config.json"))

calculate_jaccard_similarity(text1: str, text2: str) -> float

Calculate similarity between two text strings.

similarity = detector.calculate_jaccard_similarity(
    "Fixed the bug",
    "Updated the bug fix"
)
# Returns: 0.33 (33% similarity)

record_attempt(...) -> FixAttempt

Record a fix attempt.

attempt = detector.record_attempt(
    subtask_id="task-123",
    approach="Fixed by updating import",
    files_modified=["main.py"],
    outcome=FixOutcome.FAILED,
    error_message="Still failing"  # Optional
)

is_circular_fix(subtask_id: str, current_approach: str) -> CircularDetectionResult

Check if current approach would be circular.

result = detector.is_circular_fix(
    "task-123",
    "Modified import statement"
)

if result.is_circular:
    print(result.suggested_strategy.value)
    print(result.reasoning)

get_visualization(subtask_id: str) -> str

Get text visualization of attempt history.

viz = detector.get_visualization("task-123")
print(viz)

get_summary() -> dict

Get summary statistics.

summary = detector.get_summary()
print(summary['total_attempts'])
print(summary['outcomes'])

cleanup_subtask(subtask_id: str)

Remove history for completed subtask.

detector.cleanup_subtask("task-123")

Enums

FixOutcome

FixOutcome.FAILED   # Fix attempt failed
FixOutcome.SUCCESS  # Fix succeeded
FixOutcome.PARTIAL  # Partially successful
FixOutcome.SKIPPED  # Attempt skipped

AlternativeStrategy

AlternativeStrategy.DIFFERENT_FILES       # Try different files
AlternativeStrategy.DIFFERENT_METHOD      # Use different approach
AlternativeStrategy.ESCALATE_HUMAN        # Need human help
AlternativeStrategy.SKIP_SUBTASK          # Skip this subtask
AlternativeStrategy.ANALYZE_ROOT_CAUSE    # Deeper analysis needed

Recovery Manager Guide - Error recovery strategies
Orchestrator Guide - Multi-agent orchestration
Testing Guide - Testing best practices

Last Updated: December 22, 2025 Version: 1.0.0 Status: Production Ready Compliance: CODITECT Documentation Standard v1.0.0

Overview​

Quick Start​

Basic Usage​

CLI Usage​

How It Works​

1. Jaccard Similarity Algorithm​

2. Token Normalization​

3. Circular Detection Logic​

4. Alternative Strategy Suggestions​

Configuration​

Configuration File​

Tuning Guidelines​

Similarity Threshold​

Min Similar Attempts​

Lookback Window​

Integration Examples​

With Recovery Manager​

With Orchestrator​

Real-World Scenarios​

Scenario 1: Import Error Loop​

Scenario 2: Authentication Bug​

Scenario 3: Too Many Attempts​

Visualization​

Similarity Timeline​

Testing​

Run Test Suite​

Manual Testing​

Best Practices​

1. Tune Thresholds for Your Use Case​

2. Record All Attempts​

3. Act on Suggestions​

4. Monitor and Adjust​

Troubleshooting​

Issue: Too Many False Positives​

Issue: Missing Circular Patterns​

Issue: Stopwords Too Aggressive​

API Reference​

CircularFixDetector​

Methods​

Enums​

Related Documentation​

Overview

Quick Start

Basic Usage

CLI Usage

How It Works

1. Jaccard Similarity Algorithm

2. Token Normalization

3. Circular Detection Logic

4. Alternative Strategy Suggestions

Configuration

Configuration File

Tuning Guidelines

Similarity Threshold

Min Similar Attempts

Lookback Window

Integration Examples

With Recovery Manager

With Orchestrator

Real-World Scenarios

Scenario 1: Import Error Loop

Scenario 2: Authentication Bug

Scenario 3: Too Many Attempts

Visualization

Similarity Timeline

Testing

Run Test Suite

Manual Testing

Best Practices

1. Tune Thresholds for Your Use Case

2. Record All Attempts

3. Act on Suggestions

4. Monitor and Adjust

Troubleshooting

Issue: Too Many False Positives

Issue: Missing Circular Patterns

Issue: Stopwords Too Aggressive

API Reference

CircularFixDetector

Methods

Enums

Related Documentation