Don't Trust, Verify: How to Backtest Like a Scientist

You have a strategy. “Buy when RSI is 30.” You look at the chart. “Look! It worked here. And here. And here!”

I call this Confirmation Bias. I know it well because I used to do it. You ignored the 10 times it failed.

Key Findings:

• Slippage Reality: In my backtesting analysis, I found returns are often overstated by 0.5-3% annually because traders ignore real-world slippage.
• Sample Size: I refuse to trade a strategy without a minimum of 100 trades to prove statistical significance.
• The Blind Test: I rely on Bar Replay because it is the only method that replicates the psychological pressure of live execution.

The Scientific Method

1. Hypothesis: “Buying RSI 30 works in an Uptrend.”
2. Experiment: Run a simulation over 2023 data.
3. Result: Win Rate 45%. R:R 1:1.5. Profit Factor 1.2.
4. Conclusion: It is marginally profitable, but not great.

The “Blind” Test

Use Bar Replay. Go back to a random date (e.g., March 12, 2019). Press “Play.” Make decisions candle-by-candle. You will feel the pain of drawdowns. You will feel the doubt. This is the only way to build Conviction.

Optimization vs. Curve Fitting

Be careful. If you tweak the RSI to “28” because it worked better in 2023, you are Curve Fitting. You are tailoring the suit to fit the mannequin, not the human. Robust strategies work across wide parameter ranges (RSI 25-35).

Conclusion

I learned the hard way: if I haven’t backtested it, I don’t own it. I am just renting it from a guru. And when the guru is wrong, I will panic. I own my edge.

Does your strategy work in the market, or does it only work in your imagination?