News and Sentiment at Machine Speed — Filtering Noise Without Trading the Headline

Start by being precise about the job. You are not asking a model to forecast a reaction. You are asking it to reduce a hundred incoming items to the handful that concern you, so that a human being — you — has time to read those properly. That is a sorting task. Sorting is mechanical, checkable and boring, which is precisely why it suits a machine and why it is the part most tools skip over on their way to the exciting-sounding score.

Three sub-tasks make up almost all of the value. Deduplication: recognising that many items describe one event. Relevance routing: deciding which names on your list a story actually touches. Source grading: separating what a company formally told the exchange from what somebody said a person familiar with the matter told a reporter. Each of these has a right answer you can verify by opening the source, which makes them safe to delegate.

Notice what is missing from that list. Nothing there predicts anything. Nothing there is an opinion about price. The moment a tool crosses from “these twenty items are one event” into “this is bearish for the sector”, it has left the ground where its output is checkable and entered the ground where it is just fluent guessing. You want the first half of that tool and none of the second.

There is a practical reason to insist on the split. During a busy session your scarce resource is attention, not information. Information is free and infinite. A filter that hands you six real events instead of a hundred headlines has given you back most of an hour. A filter that hands you a hundred headlines each wearing a confident label has given you a hundred small decisions to make instead of one.

Clustering means grouping items that describe the same underlying event, however differently they are worded. A model does this well because it compares meaning rather than matching words. “Board approves fund raise” and “Company to raise capital via QIP, board clears plan” share almost no vocabulary and are obviously the same event to any reader. A keyword filter misses that. A language model catches it.

The output you want from clustering is one neutral sentence per group, built only from facts present in the items, plus a list of which sources carried it. That format has a useful side effect: if the group cannot be reduced to one clean sentence, it usually means you have accidentally merged two events, and you go and look. Ambiguity in the summary is a signal, not a defect.

Clustering also exposes something the raw feed hides — how thin a story actually is. Twenty items that all trace back to a single unnamed source are not twenty pieces of evidence. They are one claim, repeated. Volume of coverage feels like confirmation and is not. A group that collapses to “all of these cite the same one report” tells you more about how much weight the story can carry than any sentiment score would.

Where it fails is on genuinely borderline cases. A regulatory change and a company’s response to that regulatory change are two events, closely linked; models will sometimes merge them. Two companies announcing similar things on the same morning may get merged for the same reason. So read the group headings, not just the count. The clustering is a first pass you check, not a filing system you trust blindly.

The second job is deciding which of your names a story is actually about. This matters more than it sounds. Financial writing is full of names used as scenery — a story about one company that lists four competitors for context, a sector piece that names every large player in its second paragraph, an analyst quote that compares a business to an unrelated one. A crude keyword alert fires on all of them.

A useful relevance verdict has three levels rather than two. Direct: the story is about this company — its filing, its results, its regulator, its plant. Indirect: the story is about something this company demonstrably depends on, such as a customer, a supplier, an input cost or a rule that governs its business. Passing mention: the name appears, and nothing in the text connects the event to it. Most alerts on most days are the third kind.

The indirect category is where a language model adds something a filter cannot. It can read “the government revised the duty structure on imported cells” and connect it to a manufacturer that assembles using those cells — because the connection lives in the meaning of the sentence, not in a shared keyword. That is a genuine capability. It is also the category where false positives cluster, because the same reasoning that finds a real link will happily invent a plausible one.

So the rule is simple: a relevance verdict without a stated mechanism is not a verdict. If the model says a story is relevant to a name, it must be able to quote the words in the text that make the connection and state, in one checkable sentence, what would have to be true for the event to reach that business. If it cannot, the flag is decoration.

The third job is grading the source, and it is the one with real money attached. Under India’s listing disclosure framework, a listed company is required to inform the exchanges of material events, and those filings appear on the NSE and BSE corporate announcement pages. That is a primary source: named, dated, and attributable to the company itself. It is a different class of object from a report saying that people familiar with the discussions expect an announcement.

Unconfirmed reports move price. That is the whole problem. A market reacts to a plausible story before anybody can check it, and if the story does not survive contact with a filing, the move unwinds as fast as it came. Buying the reaction to an unconfirmed report is a bet on the report, not on the business — and you usually discover which one you were betting on only after the position is on.

A model is decent at this grading because the linguistic markers are consistent. Filings and official releases carry named authority, dates, regulatory references and specific figures. Unconfirmed reporting carries hedges — sources said, is understood to, may consider, is exploring. Ask for the grade and the phrase that earned it, and you get a check you can eyeball in a second.

What the model cannot do is verify. It can tell you the item reads like an unattributed report. It cannot open nseindia.com and confirm whether a filing exists. That step is yours, it takes under a minute, and skipping it is how people end up trading a headline that was withdrawn before the session closed.

Sentiment scoring works by reading the words and rating how positive or negative the language is. That is a real measurement of a real thing. The mistake is what people do with it. A score of “80% negative” is a compression of vocabulary. It says the article used unhappy words. It says nothing about whether the event described will affect earnings, cash flow, competitive position or anything else that eventually shows up in a price.

Two failure directions follow, and both are common. A minor matter written up in dramatic language scores strongly negative — a routine consumer complaint, a small penalty, an executive’s awkward interview. Meanwhile a filing announcing a genuinely serious development is written in flat regulatory prose and scores neutral, because filings are always written that way. The scoring system is behaving exactly as designed. It is being asked the wrong question.

There is a second, quieter problem. Sentiment on financial text is measured against how the language reads, not against what the market already expected. A result described in glowing terms can still be below what was priced in. Nothing in a language score knows what was priced in. Expectation is the missing variable, and it is the variable that decides most reactions.

None of this makes the score worthless. As a sorting aid — surface the items whose language changed sharply versus the same company’s previous coverage — it is a reasonable trigger to go and look. As a reason to act, it is a number that feels like analysis and is not. Use it to decide what to read. Never use it to decide what to do.

The single habit that upgrades news triage from a toy to a tool is forcing the filter to explain itself. Not “is this relevant to this company” but “which words in this text connect this event to this company, and what would have to be true for it to reach the business”. A verdict is a coin flip you cannot inspect. A stated mechanism is something you can immediately agree or disagree with.

False positives become obvious the moment the reasoning is visible. You will see the model connect a story to a name because both appear in the same sector, which is not a mechanism. You will see it build a chain three links long — this affects an input, which affects a supplier, which supplies this company — where each link is plausible and the chain is not. Written out, that is transparently weak. Delivered as a flag with a coloured dot, it looks like a signal.

The demand for a quoted justification also constrains invention. A model asked to quote the exact words that create the link has to find words that exist. That does not make fabrication impossible, but it makes it visible, because you can search the pasted text for the quote in about two seconds. This is the same discipline that makes filing extraction trustworthy: claims travel with their evidence or they do not travel.

Finally, insist on an explicit null answer. Give the model permission to reply “no relevant names in this text” and it will use it. Withhold that permission and it will find something, because the shape of the request implies that finding something is the job. Most headlines on most days are relevant to none of your names, and a filter that never says so is not filtering.

Put the pieces together and you get a short, repeatable routine rather than a scroll through a feed. Collect, cluster, route, grade, then verify by hand the two or three items that survived. The order matters: deduplicate before you route, or you will assess the same event five times; grade the source before you verify, so you know which items are worth the minute it takes to open a filing.

The routine also has a fixed stopping point, which is the part people skip. Once the surviving items have been checked against their primary sources, triage is over. Everything after that — what the event means for the business, whether it changes your view, whether you do anything at all — is your work, done against a chart and your own notes. Handing that stage to the model is where the process quietly turns into taking headline tips from software.

Keep the surviving items in a simple log with four fields: the one-sentence event, the primary source link, the date you saw it, and one line on what you decided to do about it — including “nothing”. Over a few weeks that log tells you something a feed never will: how many of the events that felt urgent actually mattered, and how many of your reactions to headlines you would repeat.

One warning about speed. The reason this is called machine speed is that the machine is fast. You are not, and you should not try to be. If the entire advantage of a process is that it lets you act on a headline thirty seconds sooner, you are competing on the one dimension where retail infrastructure loses by design. The advantage you are actually buying is that you read the right six things instead of skimming a hundred.