Foundation models for smart meter data imputation achieve strong accuracy on short gaps but degrade sharply on the multi-hour and multi-day outages that drive billing errors and forecasting failures, according to a January 2025 benchmark from the University of Luxembourg. The study tested 16 models — including TimeGPT, Chronos, Moirai, and TimesFM — against residential AMI data with gaps ranging from 30 minutes to 24 hours, confirming what practitioners have long observed: the 90–95% accuracy typical of generic AI falls well short of the 98–99% threshold required for production energy management at utility scale.
The benchmark reveals an inconvenient inversion. Foundation models outperform statistical and traditional machine learning approaches on gaps under two hours, yet those are precisely the intervals where simpler methods already suffice and business impact is minimal. As gap length extends toward 24 hours — the territory of communication failures, system errors, and storm outages — model performance collapses. Computational cost also rises without proportional gain, making foundation models inefficient for the easy cases and unreliable for the hard ones.
This accuracy gap compounds across three structural failure modes. Long continuous gaps break the temporal assumptions that both statistical and deep learning models rely on. Distributional shift — where outage-period consumption patterns diverge from training data — causes models to hallucinate plausible but wrong load shapes. And the “garbage in, garbage out” problem identified by Arcadia means that even high aggregate accuracy masks catastrophic errors in the specific intervals that determine settlement, demand response verification, and grid planning.
For utilities and data platforms, the implication is clear: benchmark leaderboards on short gaps are a distraction. The operational priority is robustness on the long, messy gaps that correlate with revenue risk and reliability events. That demands hybrid architectures — physics-informed constraints, utility-specific calibration, and human-in-the-loop validation — rather than reliance on any single foundation model. The industry’s next step is not better average accuracy, but guaranteed performance where it counts.
Read the full report at Energy Central.