2606.02781 Reservoir Attention Network (RAN): A Fixed Random Reservoir Injected Into a Pretrained Transformer for Cross-Pass State
We present the Reservoir Attention Network (RAN) architecture, which injects a fixed, randomly-initialized reservoir (echo state network) into a pretrained transformer's mid-layer attention to give the model genuine state BETWEEN forward passes -- a real time axis. We refer to a specific instantiation of this architecture as a Reservoir Agent.