Every backtest in the rest of this module is a claim about history. If the history is wrong, the claim is worthless — and the failure is silent. Bad market data does not throw an error or produce an obviously broken chart. It produces a clean, plausible, entirely fictional result, and the tidier that result looks the less likely anyone is to go back and check the input. This topic covers the four ways an Indian equity or futures data set is usually wrong: unadjusted corporate actions, survivorship in the universe, look-ahead in the timing, and unstitched futures contracts. None of them is exotic. All four are present by default in most free sources.
Why is data quality the topic that decides everything else?
Consider the order of dependence. A validation method can only tell you whether a result generalises; it cannot tell you the underlying prices were real. A risk model sizes positions off a volatility estimate computed from that same series. A cost model subtracts brokerage and slippage from returns that were measured on it.
So data sits underneath all of them. A defect there is not one error among many — it is an error that every later stage faithfully carries forward and dresses up.
The uncomfortable part is that data defects usually make results look better, not worse. Splits create phantom crashes that a mean-reversion rule is delighted to buy. Survivorship removes the companies that went to zero. Look-ahead hands your rules tomorrow's information. There is no symmetry here — the errors point in one direction, and it is the flattering one.
Watch out — A free, unadjusted, survivor-only data set will produce a beautiful backtest. That is the danger, not the consolation. If your first test looks excellent, the first thing to suspect is the data — not the idea.
What is a corporate action, and why does it break a price series?
A corporate action changes the share itself rather than the market's opinion of the company. A split divides each share into several; a bonus issue hands out additional shares free; a dividend pays cash out of the company. In each case the traded price steps down on the ex-date, mechanically, without a single holder losing value.
The exchange knows this. NSE and BSE publish the corporate-action record and the ex-dates. Your data source may or may not have applied it. If it has not, the raw series contains a step that your code cannot distinguish from a genuine collapse.
- Stock split
- One share becomes several, with the face value divided in the same ratio. A 1:5 split turns one ₹10 face-value share into five ₹2 shares. Price divides by five; your holding multiplies by five.
- Bonus issue
- Free additional shares from reserves — a 1:1 bonus gives you one extra share per share held. Price roughly halves. Economically similar to a split from a chart's point of view, arithmetically expressed as a ratio of new to existing shares.
- Dividend
- Cash leaves the company, and on the ex-dividend date the price typically opens lower by roughly that amount. Small for one payment, and not small at all when compounded across a decade of holdings.
- Rights issue, demerger, amalgamation
- Less frequent and messier. A demerger in particular splits one listed history into two, and no single price series honestly represents what a holder experienced.
How does the split adjustment arithmetic work?
Take a synthetic company — call it Synthetic Metals Ltd — trading around ₹2,000 before a 1:5 split. On the ex-date it opens near ₹400. Nothing happened to the business overnight.
Adjusting means dividing every price before the ex-date by the split factor, and multiplying every volume before it by the same factor, so the series describes one continuous instrument.
adjusted price = raw price ÷ 5 · adjusted volume = raw volume × 5 (for all bars before the ex-date)- 5 — the split factor. For a 1:1 bonus the factor is 2; for 1:2 bonus, 1.5.
- Applied to every bar strictly before the ex-date. The ex-date bar itself is already post-split.
- A ₹2,000 pre-split close becomes ₹400. The apparent 80% fall disappears.
- Dividends adjust by subtraction or by a ratio factor, not by the same divide — and the two conventions do not give identical series.
Example — Synthetic Metals Ltd is invented and the figures are round for arithmetic. This is not a real corporate action on a real NSE or BSE company on a real date, and no part of it is a call.
What does an unadjusted series actually do to a backtest?
It fabricates events. Concretely, on the ex-date of a 1:5 split an unadjusted daily series shows a single-session fall of about eighty per cent, followed by an immediate flat run at the new level.
- Any rule that buys large falls fires here — on an event where no holder lost anything, so the 'recovery' it appears to catch is arithmetic, not a market move.
- Any rule that sells breakdowns fires here too, exiting a position on a fictional collapse and recording a fictional loss.
- Volatility estimates are contaminated. One phantom eighty per cent day inflates the measured volatility of that stock for as long as your lookback window is, which then distorts every position size derived from it.
- Moving averages, ranges and any lookback statistic straddle the step for the length of their window, so the damage lasts well beyond the single bad bar.
- Returns computed as simple price differences are wrong across the gap, so the equity curve is wrong from that date onward.
- Dividend effects are subtler and cumulative: on a high-payout name held for years, a price-only series understates what a holder actually received.
Adjusted close or unadjusted close — which should you use?
Neither, universally. They answer different questions, and the mistake is using one series for both.
| Unadjusted (raw traded price) | Adjusted | |
|---|---|---|
| What it is | The price actually printed on the exchange that day | The raw price restated so one continuous instrument is described |
| Use it for | Anything that must match reality on a specific date — order prices, tick sizes, price bands, lot value | Return calculations, volatility, lookback statistics, anything spanning a corporate action |
| Fails at | Any calculation crossing a corporate action | Telling you what price you could actually have entered at on a given day |
| Silent failure mode | Phantom crashes that rules react to | Prices that never traded, so a level from a chart no longer matches the series |
| Also note | Historical levels look nothing like the chart you are used to | The whole series changes retroactively every time a new action occurs |
Note — That last row catches people out. An adjusted history is not stable — a split announced next month rewrites every price before it. A backtest you ran today can produce different numbers when re-run next year on the same nominal data set. Store the version you used.
What is survivorship bias, and why is it worse in a screener?
Build a universe from the companies in an index today, then run it back ten years, and you have made every constituent pass a test they could only pass by still being here. The names that were removed, delisted, suspended, wound up or merged away are simply absent — along with everything that happened to them on the way out.
This is not a small correction. Over a decade the Indian listed universe turns over meaningfully: index membership changes on a schedule, and companies leave the exchange entirely through delisting and insolvency proceedings. A universe rebuilt from today's list has quietly deleted the worst outcomes from your sample.
How do you actually avoid survivorship bias?
- 1
Use dated index membership, not today's list
Ask what the index contained on the date each decision was taken. Index providers publish constituent change history; a data set that cannot answer 'who was in this index on that date' cannot support an index-universe backtest.
- 2
Keep delisted and suspended names in your database
Their history should end on the date they stopped trading, not vanish. A symbol that disappears from your files retroactively erases every trade you would have taken in it.
- 3
Decide, in writing, what a delisting does to an open position
Your engine needs a rule. Exiting at the last traded price is one convention. Assuming a total loss is another, harsher one. Silently dropping the position is the only unacceptable answer, and it is the default in most naive engines.
- 4
Screen on point-in-time data, not restated data
Any fundamental filter — market capitalisation, turnover, a financial ratio — must use the value as it was known on that date, not the value as later restated. Restated figures are survivorship's cousin.
- 5
Sanity-check your universe size over time
Count how many symbols your test could trade in each year. If a ten-year test can trade exactly the same names in year one as in year ten, your universe is frozen and the result is not trustworthy.
What is look-ahead bias?
Using, at the moment of a decision, information that did not exist yet. It is the most common defect in retail backtests and the most flattering, because a rule with tomorrow's information can hardly fail.
The classic form is a signal computed from a candle's close that is then executed at that same candle's open. The code looks entirely reasonable — the two values sit on the same row of the same table — and the row is the whole trap. A row is not a moment; it is a period, and the fields inside it became knowable at different times.
Where else does look-ahead hide?
- Same-bar execution: signal from the close, fill at the open of the same bar. Shift the signal by one bar, or acknowledge the test is fiction.
- Corporate results and filings, used from the period they describe rather than from the date they were actually filed with the exchange.
- Index membership again, in its timing form: assuming a stock was a constituent from the day the inclusion was announced rather than the day it took effect.
- Adjusted prices themselves, which encode future corporate actions into past bars. A level from an adjusted series is a level nobody could have drawn at the time.
- Any statistic computed over the whole data set before splitting it — a mean, a standard deviation, a normalisation. The in-sample portion now knows the out-of-sample portion's shape.
- Choosing the universe by looking at which names moved most over the test window. That is look-ahead in the selection rather than in the signal, and it is just as fatal.
- Restated or revised data of any kind, including corrected exchange files, used at its original date.
Pro tip — One question tests every input: at the exact instant this decision is taken, had this number already been published? If the answer is no, or 'I am not sure', delay the input by one bar and re-run. A result that collapses under a one-bar shift was never a result.
How do futures continuous contracts work?
There is no such thing as a ten-year NIFTY futures chart. Each contract has an expiry and dies on it. A long futures history is something you construct by stitching successive contracts together, and how you stitch is a research decision you have to record.
The problem is that the next month's contract does not trade at the same price as the expiring one. Join them raw and every roll leaves a step — a jump your rules will react to as though it were a market move.
Which rollover convention should you use?
The honest position is that no convention is correct — each is a trade-off, and the requirement is that you write down which one you used and re-use it consistently. A strategy whose result changes when you change the roll rule has a dependency on the stitching, and that is worth knowing before it is worth trading.
| Convention | What it does | What it costs you |
|---|---|---|
| Raw stitch | Simply concatenates each contract's prices at the roll date | A false gap at every expiry. Unusable for any return calculation. |
| Back-adjusted (difference) | Shifts all earlier history by the price gap so the join is continuous | Historical prices no longer match what traded; deep history can even go negative on long series. |
| Ratio-adjusted | Scales earlier history by the ratio of the two prices instead of the difference | Percentage returns stay sensible, but absolute historical levels are again not real prices. |
| Roll date choice | Expiry day, a fixed number of sessions before it, or when volume shifts to the next month | Different choices give materially different series. Volume-based rolls are closer to what a trader does. |
What would invalidate your data set — and every test built on it?
Run these against whatever source you are using, before you build anything on top of it. Any single failure means results from that source are not evidence.
The data audit
- Does the series show any single-session move of an implausible size on a large-cap name? Check it against the exchange's corporate-action record.
- Does the source state, in writing, whether prices are adjusted, and for which action types? 'Adjusted' with no method stated is not an answer.
- Are dividends included in the adjustment, or only splits and bonuses? Most free sources handle splits at best.
- Does the source contain companies that have been delisted or suspended, with history ending on the correct date?
- Can it answer what an index contained on a given past date, rather than only today?
- Are there duplicate timestamps, missing trading sessions, or sessions present that were exchange holidays?
- Are there bars where the high is below the low, or the close sits outside the high-low range?
- Does volume ever sit at zero on a liquid name — and does your engine treat that as tradeable?
- For futures: is the roll convention documented, and is the roll date recorded on each bar?
- If you re-download the same window a month later, do you get identical numbers? If not, find out why before trusting either version.
What is the practical minimum for someone starting out?
You do not need a research-grade database on day one. You do need to know precisely which of the defects above your data has, so you know what your results are worth.
A reasonable starting posture: use whatever source you can get, run the audit above, and write the failures down at the top of your research log. Then restrict your conclusions to what that data can honestly support. A test on unadjusted, survivor-only daily data can still teach you how a backtest engine behaves and whether a condition even occurs often enough to study — it simply cannot tell you whether a strategy works.
That distinction, held honestly, is worth more than a paid subscription used carelessly.
Note — This page is education, not investment advice, and nothing on it is a recommendation to buy or sell any security. No data vendor, broker or platform is endorsed here; evaluating any specific source against the audit above is your own responsibility.
Key points
Pro tip — Add one line to the top of every research-log entry: the source, the download date, and which of the four defects on this page it is known to have. Six months later that line is the difference between a result you can interpret and a number you have to throw away.
Frequently asked questions
What is survivorship bias in backtesting?
It is the error of building a test universe from companies that still exist today, which silently removes every name that was delisted, suspended, wound up or dropped from an index during the test window. Because the removed names are disproportionately the worst outcomes, the surviving sample looks better than the real opportunity set ever was. The fix is dated index membership plus keeping delisted history in your database.
What is the difference between adjusted and unadjusted closing price?
The unadjusted close is the price actually printed on the exchange that day. The adjusted close restates history so that splits, bonuses and sometimes dividends do not appear as price moves. Use unadjusted prices when you need to match reality on a specific date, such as order levels or price bands, and adjusted prices for returns, volatility and any calculation that spans a corporate action.
How do you adjust NSE stock prices for a split or bonus?
Divide every price before the ex-date by the ratio and multiply the corresponding volumes by the same ratio, so the series describes one continuous instrument. A 1:5 split uses a factor of five; a 1:1 bonus uses two. The ex-dates come from the exchange's own corporate-action records, and the ex-date bar itself is already on the new basis and must not be adjusted again.
What is look-ahead bias and how do I check for it?
Look-ahead bias is using information at a decision point that had not been published yet, most commonly computing a signal from a candle's close and executing at that same candle's open. The check is to ask, for every input, whether the value was already public at the instant of the decision — and then to shift every signal back by one bar and re-run. If the result collapses, the original was fiction.
Is free stock market data good enough for backtesting?
For learning the tooling, yes. For deciding whether a strategy works, usually not, because free sources are commonly unadjusted for corporate actions, missing delisted companies, and unable to say what an index contained on a past date. The gap between free and paid data is rarely the price series you can see; it is the corrections and the missing history you cannot see missing.
How do you build a continuous futures contract?
By stitching successive expiries into one series and removing the step at each roll, either by shifting earlier history by the price difference or by scaling it by the price ratio. Both make earlier prices differ from what actually traded, so neither is correct in an absolute sense — the requirement is to document the convention and the roll date rule and apply them consistently.
Why does my backtest look great but fail in live trading?
Data defects are one of the first places to look, because all of them push results in the flattering direction: phantom corporate-action moves, a universe stripped of failures, and signals that quietly used information from later in the day. Costs, slippage and overfitting are the other main causes, and each has its own topic later in this module.