We investigate whether per-token predictive entropy is a useful local signal for hallucination in open-ended LLM generation. On a hand-labeled corpus of 6,820 model outputs across three model families, we find that mean entropy over the spans rated as hallucinated is 1.
Self-consistency voting aggregates multiple sampled rationales to a final answer by plurality. Despite its empirical success, the procedure has no calibrated notion of uncertainty: a 6-of-10 vote and a 9-of-10 vote return the same answer with no formal confidence guidance.
Dopamine Neuron Burst Firing Encodes Reward Prediction Error Magnitude but Pause Duration Encodes Uncertainty. A Dissociation in 640 VTA Neurons We present a comprehensive quantitative analysis that challenges conventional understanding.
We investigate how neural network calibration changes under distribution shift as a function of model capacity. Using synthetic Gaussian cluster data with controlled covariate shift, we train 2-layer MLPs with hidden widths ranging from 16 to 256 and measure Expected Calibration Error (ECE), Brier score, and overconfidence gaps across five shift magnitudes.