Council Review Patterns

Council Review Patterns

When to Use This Skill

Use this skill when implementing council review patterns patterns in your codebase.

How to Use This Skill

1. Review the patterns and examples below
2. Apply the relevant patterns to your implementation
3. Follow the best practices outlined in this skill

Level 1: Quick Reference (Under 500 tokens)

Council Structure

interface CouncilReview {
  id: string;
  artifact: string; // Code file or PR
  reviewers: Reviewer[];
  verdicts: Verdict[];
  consensus: ConsensusResult;
}

interface Verdict {
  reviewerId: string;
  score: number; // 0-100
  issues: Issue[];
  recommendation: 'approve' | 'request_changes' | 'reject';
}

interface ConsensusResult {
  finalScore: number;
  decision: 'merge' | 'revise' | 'reject';
  confidence: number;
}

Scoring Dimensions

• Correctness: Does it work? (25%)
• Security: Any vulnerabilities? (25%)
• Performance: Is it efficient? (20%)
• Maintainability: Can it be maintained? (20%)
• Style: Follows conventions? (10%)

Level 2: Implementation Details (Under 2000 tokens)

Anonymized Review Process

function anonymizeReview(review: Verdict): AnonymizedVerdict {
  return {
    id: generateAnonymousId(),
    score: review.score,
    issues: review.issues.map(i => ({
      ...i,
      // Remove any identifying language
      description: sanitizeDescription(i.description),
    })),
    recommendation: review.recommendation,
  };
}

function calculateConsensus(verdicts: Verdict[]): ConsensusResult {
  const scores = verdicts.map(v => v.score);
  const avgScore = scores.reduce((a, b) => a + b, 0) / scores.length;
  const stdDev = Math.sqrt(
    scores.map(s => Math.pow(s - avgScore, 2)).reduce((a, b) => a + b, 0) / scores.length
  );

  return {
    finalScore: avgScore,
    decision: avgScore >= 80 ? 'merge' : avgScore >= 60 ? 'revise' : 'reject',
    confidence: 1 - (stdDev / 50), // Higher agreement = higher confidence
  };
}

Level 3: Complete Reference (Full tokens)

Complete Council Workflow

async function runCouncilReview(artifact: string): Promise<CouncilReview> {
  // 1. Select diverse reviewers
  const reviewers = selectReviewers({
    count: 3,
    expertise: detectExpertiseNeeded(artifact),
    diversify: true,
  });

  // 2. Collect independent reviews
  const verdicts = await Promise.all(
    reviewers.map(r => collectVerdict(r, artifact))
  );

  // 3. Anonymize and share for cross-review
  const anonymized = verdicts.map(anonymizeReview);

  // 4. Calculate consensus
  const consensus = calculateConsensus(verdicts);

  // 5. Generate structured output
  return {
    id: crypto.randomUUID(),
    artifact,
    reviewers,
    verdicts: anonymized,
    consensus,
  };
}

Best Practices:

• Use odd number of reviewers (3, 5)
• Ensure diverse expertise coverage
• Anonymize before cross-review
• Document dissenting opinions
• Track consensus confidence

Success Output

When successful, this skill MUST output:

✅ SKILL COMPLETE: council-review-patterns

Completed:
- [x] Reviewer selection (count: 3, diversity verified)
- [x] Independent verdicts collected
- [x] Verdicts anonymized for cross-review
- [x] Consensus calculated (score: XX/100, confidence: XX%)
- [x] Final decision: [merge|revise|reject]

Outputs:
- Council review ID: {reviewId}
- Final score: {finalScore}/100
- Decision: {decision}
- Confidence: {confidence}%
- Reviewer verdicts: {verdictCount} collected

Completion Checklist

Before marking this skill as complete, verify:

• 3+ reviewers selected with diverse expertise
• Each reviewer provided independent verdict
• All verdicts include scores (0-100) across 5 dimensions
• Verdicts anonymized before consensus calculation
• Consensus score and confidence calculated
• Final decision determined (merge/revise/reject)
• Council review object returned with all fields populated
• Dissenting opinions documented if consensus < 80%

Failure Indicators

This skill has FAILED if:

• ❌ Less than 3 reviewers available or selected
• ❌ Reviewer expertise does not cover artifact domain
• ❌ Any verdict missing required dimensions (correctness, security, performance, maintainability, style)
• ❌ Consensus calculation produces NaN or invalid confidence score
• ❌ Decision threshold logic fails to determine merge/revise/reject
• ❌ Anonymization exposes reviewer identity
• ❌ Standard deviation exceeds 40 points (low consensus confidence)

When NOT to Use

Do NOT use this skill when:

• Single reviewer is sufficient (use standard code review instead)
• Artifact is trivial or low-risk (use automated linting/testing)
• Immediate decision needed (council review requires time for multiple reviewers)
• Reviewers lack domain expertise (use subject matter expert review instead)
• Budget constraints prohibit multiple reviewer time investment
• Review is for informational purposes only (use documentation review instead)

Use alternatives:

• Simple PR review: Standard GitHub pull request review
• Automated quality gates: Lint, test, security scan automation
• Expert consultation: Single domain expert review for specialized topics
• Pair programming: Real-time collaborative review

Anti-Patterns (Avoid)

Anti-Pattern	Problem	Solution
Even number of reviewers (2, 4)	Tie-breaking impossible	Always use odd numbers (3, 5, 7)
Same expertise background	Groupthink, blind spots	Ensure diverse backgrounds (backend, security, performance)
Non-anonymized cross-review	Anchoring bias, peer pressure	Anonymize verdicts before sharing
Ignoring low confidence	Forced consensus masks disagreement	Flag low confidence (<50%) for human escalation
Skipping dissent documentation	Loss of valuable minority opinions	Always document dissenting views when present
Cherry-picking reviewers	Confirmation bias	Use random selection or round-robin from qualified pool
Missing dimension scores	Incomplete quality assessment	Enforce all 5 dimensions: correctness, security, performance, maintainability, style

Principles

This skill embodies the following CODITECT principles:

• #5 Eliminate Ambiguity - Clear consensus decision criteria (merge ≥80, revise ≥60, reject <60)
• #6 Clear, Understandable, Explainable - Transparent scoring across 5 dimensions with documented rationale
• #8 No Assumptions - Explicit verification of reviewer expertise and coverage
• Trust & Transparency - Anonymized review prevents bias while maintaining accountability
• Factual Grounding - Each verdict requires specific issues identified with evidence
• Ambiguity Handling - Low consensus confidence triggers clarification protocol

---

Version: 1.1.0 | Created: 2025-12-22 | Updated: 2026-01-04 | Author: CODITECT Team