When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability

An empirical audit of LLM-as-judge systems shows that upgrading or changing the evaluator model leads to inconsistent scoring, highlighting reliability challenges in automated evaluation.

Firefly links to the original publisher. The summary above is AI-generated for orientation and may differ from the source. The “current rank score” decays over time so newer significant stories surface first.