Eval Drift

Eval drift is the gradual loss of credibility of automated model evaluations. When the evaluating model (LLM-as-a-Judge) becomes too similar to the one being evaluated, their shared errors stop being caught, and the benchmark score overstates the actual quality. This forces periodic calibration and human review of the tests.