What are you looking for?

When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability

AI·LLM-as-judge scores fluctuate across evaluators even with fixed candidate responses, treating the issue as measurement validity. Across four datasets, scaling Qwen3 from 1.7B to 32B parameters yields only limited adjacent gains, while MiniMax M2-to-M2.7 API upgrades show none; stronger judges reduce but do not eliminate position and verbosity biases, with repeated juries offering little benefit under correlated errors.