Short-Term Solar Irradiance Forecasting Using Persistence-Ensemble Hybrid Models and Ground-Mounted Sky Imaging
Solar power generation depends critically on accurate short-term (minutes to hours) forecasting of global horizontal irradiance (GHI), as sudden changes cause grid instability and reduce economic viability of solar farms. Current operational forecasts achieve 20-30% MAPE (mean absolute percentage error) for 30-minute ahead forecasts, with degradation at longer horizons. This study develops a hybrid forecasting system combining persistence-based methods with machine learning ensemble models and ground-mounted sky camera imagery. The system integrates: (1) Persistence models (GHI(t+30min) ≈ GHI(t)), (2) Autoregressive models (ARIMA), (3) Machine learning ensembles (Random Forest, XGBoost, LightGBM), and (4) Computer vision analysis of cloud motion from sky cameras. We train and validate on 2 years of high-frequency irradiance data (1-minute resolution) from 15 solar sites across diverse climates (desert, temperate, subtropical). Testing 10 forecasting horizons (5, 15, 30, 60, 120, 180, 240, 360, 480, 600 minutes). Results show: Hybrid ensemble achieves 18.2% MAPE for 30-minute forecasts (vs 20.5% for ARIMA baseline), improving by 2.3 percentage points, Hybrid model recovers 94.8% of maximum theoretical forecast skill, Beyond 4 hours, all models degrade toward climatological mean (∼15% MAPE), Sky camera integration reduces RMSE by 12-15% for 15-30 minute horizons where cloud speed dominates, but provides minimal benefit beyond 2 hours. Feature importance analysis shows: irradiance history (60-minute window) is most important (32% importance), Recent rate of change (5.3% importance), Hour of day (8.1%), Clear sky index deviations (6.2%). The system adapts to seasonal patterns and cloud types. Validation on held-out 2023 data shows maintained performance. Implementation uses standard GPU inference (~50ms latency per forecast), operational without internet connectivity. Deployment to 12 utility-scale solar farms enabled 8-12% improvement in 30-minute grid balancing accuracy. This framework provides a practical, explainable forecasting solution for grid operators.