Machine Learning vs LLMs — Where Prediction Genuinely Fits

Start with the thing that usually gets skipped: these two systems do not share a design. A large language model — an LLM — is trained to continue text. You give it a sequence of words and it produces the next likely word, then the next, until the answer looks finished. A supervised machine-learning model is trained to map a fixed set of numeric inputs to one specific outcome that you labelled in advance. One machine writes. The other scores. The overlap between them is the word AI and very little else.

The reason this matters is that most retail "AI trading" projects die at exactly this seam. Someone reads that a language model passed a difficult professional exam, concludes it must therefore be able to read a chart, and asks it for a level on Bank Nifty. The answer arrives instantly, in full sentences, with reasoning attached. It looks like analysis. It is a paragraph shaped like analysis, assembled from the enormous number of similar paragraphs the model has read, and its connection to the actual series you pasted is far weaker than the fluency suggests.

The opposite mistake is quieter and just as common. A trader learns some supervised learning, builds a model, and then expects it to behave like the chat window — general, explanatory, willing to tell you why. It cannot. It answers the single question you defined when you chose the label, and it answers with a number. Ask it anything else and it has no view at all, because it has no representation of anything you did not put into the columns.

Hold both pictures at once and most of the confusion drains away. The language model is a reader and a writer that happens to be very good at prose. The predictive model is a narrow measuring instrument that you have to build, calibrate and re-check yourself. Neither of them is "the AI that trades", and no combination of the two becomes that.

When you paste two hundred closing prices into a chat box, the model does not load them into a spreadsheet. It converts them into tokens — the small chunks of text, roughly word-fragments and digit-groups, that the model actually operates on. From that moment the numbers are text. There is no column, no enforced ordering, no arithmetic engine underneath. The model then continues the text, and continuing text after a list of prices is a fundamentally different operation from measuring the series.

This is why arithmetic on pasted data fails in a specific, non-obvious way. Short sums often come out right, because the model has absorbed enormous quantities of arithmetic and the pattern is strong. Longer chains drift. And when they drift, nothing signals it — the wrong total is rendered in exactly the tone of a right one. If your tool has a code interpreter attached, meaning a real sandbox that executes Python over your file, that is a genuinely different situation, because then actual arithmetic happens. Know which of the two you are using before you trust a number.

There is a second limit, and it is more fundamental. The model has a training cutoff — a date after which it has read nothing. It has no live feed unless a separate tool is fetching one on its behalf. So a sentence like "the model thinks this stock is extended" has no referent. The model is producing the words that most plausibly follow your question given the vast amount of market commentary it absorbed, not an assessment of the last twenty candles you supplied.

None of this makes the language model useless for market work. It makes it a reader rather than a calculator. It compresses a three-hundred-page annual report into claims you can go and check. It restates your strategy logic in a form where the gaps become visible. It argues against your thesis when you instruct it to. Those are real, valuable jobs. Price prediction is simply not on the list, and asking for it is a category error rather than a missing feature.

Supervised learning has three moving parts and each has a plain-English name. A feature is an input the model sees — one column of numbers for every row of data. A label is the thing you are asking it to predict, and you must be able to compute that label for past data, otherwise there is nothing to learn from. Training is the fitting process: the algorithm adjusts its internal numbers until its output on the training rows sits as close as it can to the labels you supplied.

Make that concrete on Indian equities. Take the NIFTY 500 constituents, one row per stock per week. Your features might be the distance of price from the prior swing high, whether the last five candles closed inside the range of the sixth, this week’s traded volume divided by its own recent average, and a flag for whether a results date fell inside the window. Your label might be: in the following four weeks, did this stock close above the high of the signal candle before it closed below its low? That is a yes or a no. You can compute it for every past row. That makes it a usable label.

Notice what that example did and did not do. It did not ask for a price. It asked a question with a checkable answer that already exists in history, which is the only sort of question supervised learning can be trained on at all. The model is not forecasting in any mystical sense. It is handed thousands of past situations with their outcomes attached, and asked to find whatever regularity connects the one to the other.

Everything else in this part of the module is downstream of that sentence. If the regularity is real and persists, you have something worth testing properly. If the regularity was an accident of the particular sample you happened to use, you have a very convincing chart of nothing at all. Telling those two apart is genuinely hard, and it is the entire subject of the next article.

Supervised learning rests on an assumption that is normally invisible, because in most fields it is simply true: the relationship you fit on past data still holds on future data. Markets violate it routinely. The technical word is non-stationarity — the statistical behaviour of the series changes over time. Volatility clusters and then subsides. A relationship that held across one long stretch inverts. A structural change, such as a shift in the settlement cycle or in who is providing the liquidity, quietly rewrites the thing your model spent its training fitting.

Compare that with the domains where machine learning has been genuinely transformative. Handwritten digit recognition works because the numeral seven looked the same thirty years ago, looks the same today, and will look the same next year. Speech recognition works because the acoustics of spoken Hindi are not adapting to the fact that a model is now listening. Markets are the opposite. They are made of participants, and participants respond. If a simple, findable pattern pays reliably, it gets traded until it stops paying.

That last sentence is the part retail quant work consistently underweights. Your model is not studying a natural process that is indifferent to being studied. It is studying the aggregate behaviour of people and machines who are also studying it, some of them with far better data and far lower costs than you have. The relationship is adversarial, and adversarial relationships decay.

The practical consequence is that a market model is never finished. It has a shelf life you cannot know in advance, and the expiry date is not announced. Nothing throws an error when a relationship stops holding — the code runs exactly as it always did and the results simply get worse. That is why deliberate monitoring is a structural part of the job rather than an optional extra.

The second obstacle is the ratio of signal to noise. Signal is the part of a future move that is genuinely related to something knowable today. Noise is everything else — a fund rebalancing on a schedule you cannot see, order flow, a headline nobody could have anticipated. In the fields where machine learning looks miraculous, signal dominates: nearly all the information needed to read a handwritten digit is right there in the image. In market data the noise dominates, and it dominates heavily.

That has a direct effect on how much data you need. When the signal is weak, you need far more examples to separate a real relationship from a coincidence. And market data is smaller than it feels. Twenty years of daily bars on one stock is a few thousand rows, and the genuinely distinct market regimes inside that span can probably be counted on your fingers. You hold much less independent information than the row count suggests, because consecutive rows are not independent events.

So what is the honest ceiling? A small statistical edge — a tendency that appears slightly more often than chance would produce, is stable enough to survive on data the model has never seen, and is still large enough to exist after brokerage, taxes and slippage have been deducted. That is a real and genuinely valuable thing to find. It is also nothing at all like the mental image of a model that tells you what the market is going to do.

Anyone quoting a high accuracy figure for a price-direction model has, in almost every case, either measured on data the model already saw, or chosen a period that flattered it, or left costs out, or all three at once. The correct response to a headline accuracy number is not admiration. It is three questions: over what sample, over what period, and after what assumed costs.

Once you stop asking for price forecasts, the useful applications appear quickly, and they share a shape: each replaces a judgement you were already making inconsistently by hand. The first family is classification and ranking. Rather than asking how much a stock moves, you ask which of two states a situation resembles, or you ask the model to order a watchlist so that the setups you would examine first come to the top. Ordering is an easier question than forecasting a level, and it fails more gracefully.

The second is regime labelling. You almost certainly already classify market conditions informally — trending, range-bound, choppy, expanding volatility. Doing that by eye is inconsistent, and worse, it is inconsistent in ways correlated with your mood and your open positions. A model trained on a definition you wrote down applies the same definition every day. The value is not that the machine is smarter. It is that the machine is repeatable, and you are not.

The third is execution and slippage — modelling your own fills. Slippage is the gap between the price you expected and the price you actually got. You have a trade log full of it. A model fitted on your own executions can tell you which conditions widen that gap: time of day, order size relative to typical traded volume, the days around an event. This is one of the few places where your data is genuinely yours, the labels are unambiguous, and the relationship is less adversarial than a price forecast.

The fourth is anomaly detection inside your own pipeline. A feed that arrives late, a corporate action adjusted differently from last quarter, a symbol that changed after a merger — these produce data that looks subtly wrong rather than obviously broken. A model that has learned the normal shape of your daily data flags the abnormal day. That is unglamorous work, and it protects everything downstream of it.

There is a genuinely productive relationship between the two technologies, and it is not the one people reach for. The language model does not sit inside the prediction. It sits around the work: helping you specify the problem precisely, writing and documenting the code, explaining an unfamiliar statistical term in plain language, and — most valuable of all — arguing with your design before you have spent three weeks on it.

Specification is where this pays best. Most failed models fail because the idea was never written down precisely enough to be wrong. "Stocks that consolidate after a volume spike tend to break out" is not a specification. It does not say what consolidation means, over how many candles, how large a spike is relative to what baseline, or what counts as a break-out and by when. A model will happily let you skip all of that and hand you a number anyway. Forcing the definitions into the open, before any code exists, is the highest-leverage thirty minutes in the whole project.

The second use is adversarial review. Ask the model to attack your setup rather than help you build it. Instruct it explicitly to look for information that would not have been available at the moment of the decision, for definitions that quietly reference the future, and for choices you made because they improved a result rather than because they made sense. It will not catch everything. It reliably catches the obvious things you stopped seeing on day three.

What it will not do is tell you whether your idea is any good. It has no way to know, and if you ask, you will get an agreeable answer, because agreeable answers are what the training data rewards. Keep the model on the parts that are checkable — definitions, code, terminology, blind spots — and keep the judgement where it belongs.