Overfitting exposure audit of current research
Before adding any new control, find out how exposed your existing work already is. Most researchers cannot say how many variants sit behind their best backtest, which data has already been looked at, or which parameters were tuned by eye. A one-page audit of the live candidate list turns a vague worry into a ranked set of specific problems to fix first.
First tasks
- List every strategy you currently consider promising, with its headline Sharpe
- Estimate the number of variants run behind each one from notebooks and logs
- Score each candidate low, medium or high for overfitting exposure
- Pick the three highest-exposure candidates to re-test first