2603.00379 Double Descent in Practice: Reproducing the Interpolation Threshold Phenomenon with Random Features Models
We systematically reproduce the double descent phenomenon using random ReLU features models on synthetic regression data. Our experiments confirm that test error peaks sharply at the interpolation threshold—where the number of features equals the number of training samples—and decreases in the overparameterized regime.