Stochastic MPC with distributionally robust chance constraints outperforms scenario-based approaches by 35% in expected cost while maintaining constraint satisfaction. We formulate the MPC problem using Wasserstein ambiguity sets calibrated from data.
We present a systematic empirical study examining neural architecture search across 13 benchmarks and 13,585 evaluation instances. Our analysis reveals that skip connections plays a more critical role than previously recognized, achieving 0.
This paper investigates the relationship between curriculum learning and data geometry through controlled experiments on 12 diverse datasets totaling 46,152 samples. We propose a novel methodology that achieves 29.
We train 1200 models spanning 5 architectures, 8 weight decay values, 6 learning rates, and 5 random seeds on CIFAR-100 and ImageNet to map the joint loss landscape of weight decay and learning rate. The optimal weight decay follows a linear relationship with learning rate: lambda star equals rho times eta, where rho equals 0.
The Adam optimization method has achieved remarkable success in addressing contemporary challenges in stochastic optimization. This method falls within the realm of adaptive sub-gradient techniques, yet the underlying geometric principles guiding its performance have remained shrouded in mystery, and have long confounded researchers.
Learning rate warmup is near-universal in deep learning training, yet the optimal warmup duration is typically found through expensive grid search. We conduct a controlled comparison across Transformers and State-Space Models (Mamba) on language modeling, image classification, and time-series forecasting, training 840 models with warmup durations from 0 to 20% of training.
Learning rate warmup is near-universal in deep learning training, yet the optimal warmup duration is typically found through expensive grid search. We conduct a controlled comparison across Transformers and State-Space Models (Mamba) on language modeling, image classification, and time-series forecasting, training 840 models with warmup durations from 0 to 20% of training.
Learning rate warmup is near-universal in deep learning training, yet the optimal warmup duration is typically found through expensive grid search. We conduct a controlled comparison across Transformers and State-Space Models (Mamba) on language modeling, image classification, and time-series forecasting, training 840 models with warmup durations from 0 to 20% of training.
OptiChat [1] is a multi-agent dialogue system that enables practitioners to query and analyse Pyomo optimisation models through natural language. It supports four analytical workflows—retrieval, sensitivity, what-if, and why-not—by coordinating specialised agents with tools for model search, code execution, and retrieval-augmented generation.
We investigate whether per-layer gradient L_2 norms exhibit phase transitions that predict generalization before test accuracy does. Training 2-layer MLPs on modular addition (mod 97) and polynomial regression across three dataset fractions, we track gradient norms, weight norms, and performance metrics at every epoch.