A backtesting engine has one job: to answer what would have happened if this rule had been running, using only the information that existed at each moment. Almost every part of building one is about enforcing that second clause, because the data on your disk contains the future and nothing stops your code from reading it. A backtest that is fast, elegant and twenty lines long is usually wrong — not because the arithmetic is wrong, but because it quietly used prices that had not happened yet.
What is a backtesting engine actually for?
Not to tell you whether a strategy makes money. It is a falsification tool. You wrote a rule; the engine's purpose is to give that rule the fairest possible chance to fail on data it has never been tuned to.
That framing matters because it changes what you build. If the goal were a number, you would optimise for speed and convenience. If the goal is falsification, you optimise for one property above all others: the engine must make it structurally impossible to use information that did not exist yet.
Everything below is a consequence of that one requirement.
Note — This page describes how to build the tool. It shows no results from one. Nothing here is a claim about any strategy's performance, and nothing here is investment advice.
Vectorised or event-driven — what is the difference?
A vectorised backtest loads the whole price history into arrays and computes everything with column operations: a signal column, a shifted position column, a returns column, a sum. It is fast, it is short, and it is what almost every tutorial shows.
An event-driven backtest walks forward one bar at a time. The engine hands the strategy one bar, the strategy updates whatever state it keeps, decides, and returns orders that will be filled on a later bar. Nothing else is visible to it.
The difference is not performance. It is that in the vectorised version, the entire future sits in memory at every step, and there is no mechanism preventing a column from reading it. A single misplaced shift silently produces a strategy that traded on tomorrow's data.
When is a vectorised backtest acceptable?
The last row is the one people underestimate. An event-driven engine consumes a bar and emits orders. Live trading consumes a bar and emits orders. If both use the same strategy interface, the code that was tested is the code that runs, and an entire category of deployment bug disappears.
| Question about the strategy | Vectorised | Event-driven |
|---|---|---|
| Is there anything here at all? (a first screen) | Fine, and much faster | Overkill |
| Does the position depend on current holdings? | Hard — needs state | Natural |
| Do you need stops, targets or trailing exits? | Awkward and error-prone | Straightforward |
| Are you sizing from live equity that changes? | Needs a loop anyway | Built in |
| Portfolio of many instruments with shared capital | Very hard to do honestly | The reason the design exists |
| Will this become live trading code? | No — it shares nothing with a live loop | Yes — same interface, different data source |
What happens inside one iteration of the loop?
The correctness of the whole engine lives in the ordering of six steps. Get the order wrong and the output still looks perfectly plausible — that is what makes this dangerous.
Why is same-bar look-ahead the error that ruins most first backtests?
Your rule says: buy when the daily close is above the range high. So the code checks the close, sees it is above, and records a fill at that close.
But you could not have known that close until the session was over. At 3:29 p.m. the price may have been below the level; the close is the last thing that happens. Filling at it means the order was placed using information produced by the fill price itself.
The honest version fills at the next bar's open. That price includes any overnight gap — which on Indian equities is a real and frequent thing, produced by results, board meetings, block deals, rating changes and regulatory news. The gap is exactly the part of the outcome the dishonest version deletes, and it is systematically the part that goes against a breakout entry.
signal computed on bar t → order queued → filled at open of bar t+1- Bar t — the bar whose close produced the signal. Everything up to and including its close is legitimately readable.
- Order queued — the engine holds the order; it does not execute it inside the same iteration.
- Open of bar t+1 — the first price at which a real order could have transacted. The difference between this and bar t's close is the gap you were pretending did not exist.
- The same rule applies to exits, to stops and to position sizing inputs, not only to entries.
Watch out — A stop check has a subtler version of this problem. If a bar's high and low both breach your stop and your target, the bar alone cannot tell you which came first. Assuming the favourable one is a silent, systematic bias across every trade in the run. Assume the unfavourable one, or drop to a finer timeframe to resolve it.
What are the other correctness traps?
- Ignoring the open entirely
- Close-to-close arithmetic is convenient and it deletes the gap. On a market with frequent overnight news, the gap is not noise around the answer — it is a large, one-directional part of it.
- Indicator repainting
- Any value computed using a centred window, or recomputed over the full series after the fact, contains future data at every historical point. If a value at bar t changes when bar t+5 arrives, it cannot be used at bar t.
- Unadjusted or wrongly adjusted prices
- A split or bonus shows up in a raw series as a violent one-day move that never happened economically. Adjust consistently across the whole universe, and know whether your adjustment includes dividends — the answer changes every level on the chart.
- Forward-filled holidays and suspensions
- A stock that did not trade is not a stock that closed flat. Carrying the last price forward manufactures bars the market never printed, and mean-reversion rules feed on them.
- Fills that ignore liquidity
- The engine will happily fill any quantity at any printed price. If your size is a meaningful share of the bar's traded volume, that fill did not exist. Cap participation and reject the trade if the cap is not met.
- Costs added at the end
- Brokerage, STT, exchange charges, SEBI turnover fees, stamp duty, GST and slippage must be subtracted per trade, inside the loop. Applying an average at the end hides the fact that the highest-frequency variant is the one costs kill.
- Circuit limits and non-tradeable bars
- A stock locked at an upper or lower band shows a price you could not have transacted at. Indian equities have daily bands; an engine that ignores them fills orders at prices with no counterparty.
What is survivorship bias, and why is the universe the harder half?
Every trap above is about a single price series. This one is about which series you were allowed to look at, and it is both more damaging and much easier to miss — because the code can be flawless while the answer is still fiction.
If you build your candidate list from an index's membership as it stands today and run it back over ten years, every name in that list is there partly because it survived. The companies that were delisted, suspended, merged away or dropped from the index have been removed from the experiment retroactively. Nothing in your loop is wrong. The universe was assembled with hindsight.
What does point-in-time universe construction actually require?
- A membership history, not a membership list. For each date, which instruments were listed, traded and eligible — including the ones that no longer exist.
- Delisted and suspended names retained in the data, with the reason and the date, so the engine can hold a position into a delisting rather than pretending the position was never opened.
- Corporate-action history keyed by ex-date, applied as of that date rather than retroactively rewritten across the whole series.
- Liquidity filters evaluated as of the decision date. Filtering on average traded volume computed over the full history is the same hindsight problem wearing a different hat.
- Index-membership changes dated. A stock added to an index last year was not in it two years ago, and inclusion itself moves prices.
- Derivatives eligibility dated, if the strategy needs a shortable or futures-tradeable leg. That list is revised, and a name eligible today may not have been.
- A record of what the universe was on each rebalance date, stored with the run, so the same test can be reproduced later rather than re-derived from whatever the data vendor currently returns.
Watch out — If your data source only gives you a current constituent list, you cannot build a point-in-time universe from it — and you should say so in the research log rather than run the test anyway and quietly discount the result later. An untestable assumption recorded is worth more than a clean-looking number.
So why is a twenty-line pandas backtest usually wrong?
Not because pandas is unsuitable. Because of what the twenty lines omit.
The canonical version loads adjusted closes, computes a boolean signal column, shifts it by one, multiplies by daily returns and sums. That produces a number. Here is what it silently assumed.
| What the short version does | What it assumed | Why that is not true |
|---|---|---|
| Multiplies a signal by same-day returns | You transacted at the close that produced the signal | That close was unknown until the session ended. |
| Uses close-to-close returns | There is no gap between sessions | Indian equities gap on results, board meetings, block deals and regulatory news. |
| Uses a fixed notional per position | Capital is unlimited and unshared | Real positions compete for the same capital and margin. |
| Has no stop and no target | The exit rule is the only exit | Intrabar stops change nearly every trade, and a bar cannot say which extreme came first. |
| Subtracts a flat cost, or none | Costs are small and constant | Brokerage, STT, exchange and SEBI charges, stamp duty, GST and slippage scale with trade count and size. |
| Runs on today's constituent list | The universe was knowable in advance | It was assembled from names that survived to today. |
| Reports one number for the whole history | The result is a property of the strategy | It is a property of the strategy, the period, the universe and the parameters together. |
Watch out — None of these is a bug in the sense of throwing an error. Every one of them makes the number better than the truth, and they compound in the same direction. That is the actual problem: the errors are not random, they are biased favourably, which is exactly what makes a wrong backtest persuasive.
How do you test the engine itself?
- 1
Run a strategy that must produce nothing
A rule that never trades must return exactly zero, and a rule that buys and holds must reproduce the instrument's own return minus one round trip of costs. If either is off, the engine is wrong before any strategy is.
- 2
Feed it a series you constructed by hand
Ten bars whose correct outcome you worked out on paper. Assert the engine's fills, quantities and cash balance match, bar by bar. This catches ordering bugs that no real dataset will make visible.
- 3
Shift the entry by one bar deliberately
If moving the fill from the next open to the signal close barely changes the result, either the strategy is very slow-moving or the engine was already peeking. Both are worth knowing.
- 4
Set costs to zero, then to something absurd
The result should degrade smoothly and predictably. A strategy that survives an absurd cost assumption is probably not trading; one that collapses on a small one was never viable.
- 5
Reconcile cash, positions and equity every bar
Cash plus the marked value of positions must equal equity at every single step. This one invariant catches double-counted fills, missed costs and phantom positions.
- 6
Re-run and require an identical result
Same inputs, same seed, same output, byte for byte. If a run is not reproducible you cannot compare two versions of a strategy, which is the only comparison that ever mattered.
- 7
Store the run, not the number
Version of the data, version of the code, the parameters, the universe as of each rebalance, and every order with the reason it was generated. A result you cannot regenerate in six months is an anecdote.
What invalidates a backtest?
The same discipline a chart setup needs. State in advance what would make the result unusable, then check for it.
- Any value used at bar t that would change if bar t+1 arrived. That is look-ahead, whatever it is called in the code.
- A fill at a price that could not have been transacted at — a signal-bar close, a locked circuit, a price outside the bar's range, or a size beyond the bar's liquidity.
- A universe built from a current list. The strategy was choosing from names selected by hindsight.
- Costs applied outside the loop, or estimated rather than itemised for the actual segment and instrument type.
- A result that changes materially when the entry is shifted by one bar. The edge was in the timing assumption, not the rule.
- A result driven by a handful of events. Count the trades that contributed most; if removing three changes the conclusion, there is no conclusion.
- A result you cannot reproduce from the stored run definition.
- Any parameter chosen after seeing the result on the same data. That is not a backtest any more, and walk-forward validation is the next topic for a reason.
Note — A backtest that survives all of this is still only evidence that the rule was not obviously broken on history. It is not a forecast, not a performance claim, and not a reason to size up. Forward testing on live prices with no money at risk is the step between it and anything real.
Key points
Pro tip — Build the engine so that the strategy object physically cannot see the future: hand it one bar, let it return orders, and have the engine — not the strategy — decide the fill price on the following bar. Convenience arguments for breaking that boundary will arrive within a week of writing it. Every one of them is the same request in different clothing, and granting it is how a backtest stops meaning anything.
Frequently asked questions
What is the difference between a vectorised and an event-driven backtest?
A vectorised backtest computes the whole result with array operations over the full price history at once; an event-driven backtest walks forward one bar at a time and hands the strategy only the data available up to that bar. The practical difference is not speed but safety: in the vectorised version the entire future is in memory at every step, so a single misplaced shift silently uses tomorrow's data. An event-driven loop also shares its interface with live trading code, so the logic you tested is the logic that runs.
What is look-ahead bias in backtesting?
Using information at a point in time that was not available at that point. The most common form is same-bar look-ahead: the rule reads the close of a bar to generate a signal and then records the fill at that same close, a price that was not knowable until the bar had already ended. It also appears through indicators computed with centred windows, data revised after the fact, and universes assembled from what exists today.
Why should a backtest enter on the next bar's open instead of the signal bar's close?
Because the next open is the first price at which an order could actually have transacted after the signal existed. It also includes any overnight gap, which on Indian equities is frequent — results, board meetings, block deals, rating changes and regulatory news all move prices between sessions. Filling at the signal bar's close deletes exactly that gap, and for breakout entries the deleted part is systematically unfavourable, so the omission flatters the result on every trade rather than averaging out.
What is survivorship bias and how do you avoid it?
It is the error of running a strategy over a universe built from instruments that still exist today, so companies that were delisted, suspended, merged or removed from an index have been deleted from the experiment after the fact. Avoiding it requires point-in-time universe construction: a dated membership history that includes the names that later disappeared, corporate actions applied as of their ex-dates, and liquidity or eligibility filters evaluated as of the decision date rather than over the full history.
Why is a short pandas backtest usually wrong?
Because of what it omits rather than what it computes. The canonical short version assumes you transacted at the close that produced the signal, ignores overnight gaps by using close-to-close returns, assumes unlimited unshared capital, has no intrabar stop handling, applies a flat cost or none, and runs on a current constituent list. None of these throws an error, and every one of them makes the number better than the truth — they bias in the same direction, which is what makes the output persuasive.
How do you handle a bar where both the stop and the target were hit?
A single bar records only the open, high, low and close, so it cannot tell you which extreme came first. Assuming the favourable one introduces a systematic bias across every trade in the run. The two defensible options are to assume the unfavourable outcome, which makes the test conservative, or to drop to a finer timeframe for that bar so the sequence is actually observable. Whichever you pick, record it as part of the engine's specification.
What tests should a backtesting engine itself pass?
At minimum: a strategy that never trades must return exactly zero; a buy-and-hold must reproduce the instrument's own return minus one round trip of costs; a hand-constructed series of about ten bars must produce fills, quantities and cash balances you worked out on paper; cash plus the marked value of positions must equal equity at every bar; and two runs on identical inputs must produce byte-identical output. Engine bugs are far more common than strategy bugs and are invisible in the result.