The most dangerous number in a storage backtest is a hit rate of 100%. It doesn’t mean the model is good. It almost always means the model was allowed to see the price it was supposed to forecast — and a backtest that can see the future will happily promise you a future it can’t deliver.
Schedule on the forecast, settle on reality
The honest test is a strict separation: the optimizer commits to a schedule using only the day-ahead view, then settles at the prices that actually cleared. Do that and it takes losses — on a real ERCOT year the hit rate lands around 84–89%, not 100%. The losing days aren’t a flaw in the write-up; they’re the evidence that the backtest wasn’t quietly cheating. A result with no losing days is describing a strategy no one can run.
Show the uncertainty, not just the point
A single forecast number invites false confidence. Publishing an 80% confidence band and a 0–100 confidence score alongside the realized outturn lets you see where the model is genuinely confident and where it is guessing — which is exactly what an operator needs in order to size a position or stand aside. “Can I trust it when it’s wrong?” is answered, in advance, by whether the model told you it might be.
Let someone open the worst day
Aggregate metrics can launder a bad strategy — a healthy annual number can hide a catastrophic week. The real check is whether you can open the single worst day and read it hour by hour: what the optimizer committed to, and what actually cleared. A drawdown you can inspect is a risk you can underwrite. A smooth equity curve with no bad days to open is not a safer asset; it’s a modeling artifact you should distrust.
Rigor in a signal isn’t the absence of losses — it’s the visibility of them. Model the busts explicitly, carry the uncertainty on every forecast, and make the worst day openable. A backtest that never loses hasn’t removed the risk. It has only hidden it until it shows up with real money.