
This is a post about where our high-speed “12-minute” analytics still needs a human. I admit it without embarrassment: it does.
There is an old joke. A fox is faster and smarter than a hare, but cannot catch it. The fox explains: “I work out where he will jump. But the idiot himself has no idea where he is going to jump!”😅. It is usually told as a comforting thought: AI will never catch us while we can behave illogically and unpredictably. I tell it for a different reason. Here, the fox is the algorithm. The hare is not a person, but an error hiding precisely where the algorithm is too certain of its own infallibility.
In September, AI agents compiled a report on the Eastern Economic Forum: 5,001 Telegram posts and 1,614 channels, in 12 minutes of machine time. Plus several hours of mine. In a separate pass, they checked each channel for designation as a foreign agent, an undesirable organisation or an extremist organisation, removing it from the dataset when in doubt. They filtered out more than 400 posts.
Then I sat down to check it manually. And found a channel designated as a foreign agent that had passed every filter. It had been on the register since 2024.
A fresh extract from the register was in the context, but somewhere in the middle. The reason is usually mundane: the model skipped over it (the Lost in the Middle effect) and filled in the answer from memory, which is incomplete and/or outdated.
That is where the “hare” was hiding. It is not that humans are more cunning than algorithms and cannot be predicted. It is that the algorithm makes a mistake precisely where it feels certain, without noticing. From the outside, the designated channel looked like an ordinary news channel with a million subscribers.
The remedy is not always a “smarter model”, but architecture. For example, every register check can be verified by a second agent, or better still an nth agent using other models, with disagreements referred to a human. Cross-checking helps, but does not guarantee the result. Better yet, register matching should not be delegated to an LLM at all. It is a deterministic task: exact matching by ID or username (using a mapping table) against a register extract. It can be solved with code whose errors are reproducible and visible, rather than hidden behind the confident tone of a digital flatterer.
Who thought of adding a second agent or a script, and decided exactly what it should check? A human. For now🤓. The report has been redone, the channel removed from the dataset and the figures recalculated. The updated version is available at the same link.
How much human input does this work involve? Let me answer with a ship analogy. The rudder of a modern vessel weighs around 0.05% of its mass. The other 99.95% is the engine, hull and cargo, all working perfectly: the engine drives, the hull holds and the hold is full. But whether the ship stays in the channel or runs aground is not decided by the engine or the cargo. It comes down to that 0.05%: the rudder. A tiny mechanism that turns the entire vessel. The helmsman does not even register in the ship’s mass: practically zero. Yet the helmsman sets the course.
Our work is similar. Agents do almost everything: read thousands of texts, annotate them, calculate, assemble charts and lay out reports. They are the engine, hull and cargo. In 12 minutes, instead of two days of work by a team of analysts. The rudder is the architecture of the analytical environment and its verification system: verification agents, register-matching scripts, the rule that “disagreements go to a human”, and so on. In itself, this component is tiny, yet it steers the entire analysis. The helmsman is the human: setting the direction, looking at the dataset “out of the box”, considering non-obvious explanations where data is missing, using imagination, inventing that rudder and finally checking what the model is too confident about (a model’s confidence is a poor predictor of its correctness).
The fox is fast and clever. And the ship is heavy. All right, I have overdone the metaphors😅
An agent does not care whether the ship runs aground. That is not a criticism; it simply has no reason to care. The course is set by whoever cares where the ship is going and is willing to question what seems obvious. For now, that is a human.


