What are you looking for?

When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability

AI·LLM-as-judge scores fluctuate across evaluators even with fixed candidate responses, treating the issue as measurement validity. Across four datasets, scaling Qwen3 from 1.7B to 32B parameters yields only limited adjacent gains, while MiniMax M2-to-M2.7 API upgrades show none; stronger judges reduce but do not eliminate position and verbosity biases, with repeated juries offering little benefit under correlated errors.

When Debiasing Backfires: Counterintuitive Side Effects of Preprocessing-Based Stereotype Mitigation

AI·The research finds that preprocessing-based stereotype mitigation in NLP can backfire by increasing stereotyping or counter-stereotyping for some groups relative to neutral baselines.

COALA: Robust Contextualized Speech-augmented Language Modeling for ASR via Contrastive Regularizer and Biasing Score Estimation

AI·COALA enhances contextualized SLMs for multi-entity ASR via contrastive regularizer and biasing score estimation to handle domain-specific entities robustly.

VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech

Large Audio-Language Models (LALMs) are increasingly integrated into daily applications, yet their generative biases remain underexplored. Existing speech fairness benchmarks rely on synthetic speech and Multiple-Choice Questions (MCQs), both offering a fragmented view of fairness. We propose VIBE, a framework that evaluates generative bias through open-ended tasks such as personalized recommendations, using human-recorded speech. Unlike MCQs, our method allows stereotypical associations to man…