Posts tagged #measurement discipline
3 posts tagged 'measurement discipline'.
- Price-Based Trading Rules All Failed the Backtest - Switching to an Order-Free Forward RecorderI spent a whole day backtesting whether the price fields in my recommendation reports (entry, stop, target) could squeeze out more return, and every rule I judged fell short of the bar. So instead of putting any of them into live trading, I stood up an order-free forward recorder to judge them months from now. This is that process, plus a pre-registration rule I caught myself breaking.
- Reading Out the Label-Redesign Experiment - Zero Passes, Not "No Effect" but "Can't Measure"Yesterday I froze the rules first; today the results of the label-redesign experiment came in. Zero passes, so I kept the current setup, and an outside AI review corrected my reading from "it doesn't make money" to "the data can't tell whether it makes money."
- A Roadmap for Improving Predictive Power, and the "If You Can't Tell, Keep Current" RuleRe-scoring the same model on the window we actually trade showed its predictive power wasn't as near-zero as it looked. Six years of narrowing-the-universe experiments all came back "can't tell," so I left the universe alone, and I added a non-inferiority gate against a fixed anchor model for retraining deploys.