Skip to main content

Consistency Judge Agent

Validates cross-analyst agreement and ensures classification consistency. Checks for:

  • Minimum agreement threshold across analysts
  • Confidence distribution consistency
  • No conflicting strong opinions

File: consistency.py

Classes​

ConsistencyJudge​

Judge that validates consistency across analyst votes.

Functions​

evaluate(document, votes)​

Evaluate consistency of analyst votes.