The method behind every verdict · updated 12 November 2026
We test trading strategies the way your account would live them. Every issue follows the same steps, in this order. Each step says it plainly first, then precisely.
Rules first. We rebuild each strategy from its author's own rules and write down every assumption.
Then we reproduce the author's claim on the author's own window. If it does not reproduce, we say so and ask the author.
Data frozen before the test. Before we run a test, we save its prices in one file. Every number in the issue comes from that file.
Precisely: recent issues print the file's md5 fingerprint and its last session, so anyone can confirm the data did not change.
Clean data, on the exchange's calendar. We use only days the New York Stock Exchange was open, and we remove fake minutes from free data.
Precisely: daily tests use ETF prices with dividends reinvested. Intraday tests use Dukascopy's free 1-minute index CFD prices, bid side, priced like the matching futures contract.
Every daily series runs on the NYSE calendar, with sessions taken from QQQ's and SPY's own price history. In minute data we first remove placeholder minutes: flat candles where open, high, low and close are equal and volume is zero. We take no entries on days the NYSE was closed.
Costs, then more costs. Every trade pays commission and slippage, the gap between the price you see and the price you get. Then we double and triple them. An edge that dies at twice the cost was never there.
The account, not the trade. A strategy can win most of its trades and still make less money than doing nothing. So we run it as an account and count the days its money sits idle.
Precisely: idle cash earns a Treasury bill rate, credited daily; each issue names the series, usually the 4-week bill. We compare with holding QQQ, the bar, and SPY, the floor, over the same days: total return, dividends included, from each ETF's first trading day where the test allows.
Same risk, compared fairly. A strategy that sits in cash swings less than the index. That makes it look safer, and poorer. So we also compare at equal risk.
Precisely: we run the strategy at a fixed leverage of one, two or three times, or scale it, or the index, to the same volatility, and compare the money at the end. A volatility match uses hindsight, so it is an illustration, not a rule you could trade.
Borrowed money pays a retail margin rate when positions are held overnight; Issue 05 charged 5.13%. A strategy that is flat by the close every day carries no overnight financing.
Alone, and as an add-on. Some rules are weak alone but help when kept on top of an index holding. We test both.
Precisely: the add-on holds the index with all the money and adds the strategy's trades on top. Any add-on that is positive on average lifts a backtest, even when that average is luck, so it must also pass the luck checks below.
Years it never saw. Most strategies were built on one stretch of history. We split the data into the years a strategy was tuned on and the years it was not, and compare each half with the index in the same half.
Next-door settings. We nudge every parameter. A real edge still works one step either side, in both halves. A lucky setting works at one value only.
Delete the clever part. We rerun the rule without its special filter. If the result stays, the filter was decoration.
Count the tries. Test ten versions and one will look good by chance. We count every version and raise the bar to match.
Precisely: the t-statistic measures how far a result stands out from luck, and 2 is the usual bar. With ten tries, t = 2 becomes about 2.8.
Real fills. A backtest fills the instant a bar closes, and nobody can. We re-price entries from 0 to 300 seconds late. Our own automated orders fill about 47 seconds after the signal.
Our paper-trading account, which trades without real money, checks the machinery: that orders fill where the backtest says they fill. It cannot prove an edge in a few months, so we never call it a test.
The Retail Gate. Could someone with a day job and an ordinary broker account run it as written? If not, we say what it would take.
Numbers from files, checked twice. A script regenerates every number we publish from stored result files. Nobody types a number into a table. Before anything goes out, a separate check re-derives every headline number. Before we change our test engine, a regression job must reproduce every number we have already published.
How we compute the figures
Yearly figures use elapsed calendar time: days divided by 365.25. Never a count of rows.
Sharpe ratio: the average daily return in excess of T-bill cash, divided by its sample standard deviation (n − 1), times the square root of sessions per elapsed year. On the NYSE calendar that is the square root of 252.
Precision: we store every result at full precision and round once, when we display it.
What the verdicts mean
Fails: no edge after costs.
Loses to the index (also written “trails the index”): the edge is real, but the account trails simply holding QQQ or SPY.
Unproven: the data cannot yet separate a real edge from luck or tuning.
Replicates: the author's claim reproduces on our data.
Works as an add-on: it trails the index alone but helps beside it. We say how strong the evidence is.
Data research: the issue studies data or fills, not a strategy.
On the Scoreboard, each strategy also gets one plain answer to “should you use it?”: No, Only as an add-on, Maybe, or Not a strategy (data research).
Fairness and corrections. We name and link every author, lead with what replicated and offer a right of reply before publishing. When a published number changes, the post gets a dated “Updated” note at the top that says what changed. When a verdict changes, we say so plainly, in the post and on the Scoreboard.
If you wrote one of these rules and think we got it wrong, write to [email protected]. We print replies in full.
Disclaimer. Past Performance Lab discusses how trading strategies behaved on historical data. It is not a financial advisory service, and the author is not a licensed financial adviser. Nothing here is a recommendation to buy, sell or hold any security or to follow any strategy, and nothing is tailored to any reader's circumstances.
When we test a strategy a reader sends us, we publish the result for everyone; we do not answer individual investment questions. We accept no payment from the authors or products we test. Backtested results are hypothetical, and readers should not rely on this publication in making an investment decision. The same disclaimer is on our About page.
Your turn
Send us the strategy you are about to risk money on. Write to [email protected].
Past performance is no guarantee of future results. We check anyway.
This is research on historical data, not investment advice. Full disclaimer.

