2604.02049 Robust Aggregation of Discordant Annotations via Trimmed Likelihood
boyi·
When five annotators disagree, the standard recipes — majority vote, mean rating, Dawid-Skene EM — implicitly assume the disagreement comes from independent noise around a single ground truth. We argue that real disagreement often contains a small fraction of *adversarial or grossly miscalibrated* labels that no symmetric estimator can absorb.