Hot Vibes, Cold Data

Ask an agent a business question and the answer comes back clean. A short paragraph, a number, a line of interpretation, maybe a tidy table. It reads like something a capable analyst wrote on a good morning: calm, structured, not a comma out of place. None of which tells you whether the number is right.

We spent twenty years learning to read dashboards with a bit of suspicion. A bar that's suspiciously tall, a total that doesn't tie to the finance system, a chart that's blank for the last three days: crude signals, and they work. The eye catches them. Prose catches nothing, because prose has no shape to go wrong in. A wrong number and a right number sit inside the same confident sentence.

So, the position. Bad data hasn't become more common because of AI. It has become better dressed. The risk was always plausible-looking output with no accountable source behind it; agents industrialise the plausibility and leave the accountability exactly where it was, which is to say usually nowhere.

Good-enough dashboard, good-enough answer

Every BI engagement eventually produces the same question from someone senior: is this dashboard good enough to make decisions on? The answer is usually "good enough, if you know who built it."

When you trust a dashboard, a fair part of what you're trusting isn't the dashboard. It's the person who built it. Picture a retail chain: a few hundred stores, a healthy online operation, click-and-collect bolted on somewhere in between. Total sales on the leadership view read about four percent high. The cause is dull: a click-and-collect order gets logged once by the e-commerce platform when it's placed and again by the store till when it's collected, and nobody nets the two against each other. The dashboard the board actually trusts dedupes them on order ID, because the analyst who built it tripped over this in their first month and never forgot. The rule lives in a comment in the load script, and in their head. Mostly the head.

Point the same question at the same tables through an agent and you get the inflated figure with a clean sentence wrapped around it. Not because the agent is worse than the human, but because it carries none of the scar tissue. It hasn't been burned by that custodian. Unless the dedupe rule, and the reason for it, is written down somewhere the agent can actually reach, it does not exist as far as the agent is concerned.

I made the constructive version of this argument in The AI-Ready Data Model: modelling matters more, not less, once you remove the analyst who was quietly correcting the interpretation. This is the same point seen from the wreckage. The silent human correction was load-bearing, and almost nobody wrote it down.

Plausibility is the new failure mode

A dashboard fails loudly. A blank panel, an error, a value a regular reader knows on sight is wrong, and someone shouts. An agent fails quietly and fluently. Hand it a half-loaded table, a join that doubled some rows, a filter someone forgot to apply, and it will narrate all of it into a paragraph that reads like a finding.

The dashboard was an accidental quality check

A chart has a shape, and the eye is good at noticing when the shape is wrong: a spike, a gap, a flat line where there should be a trend. Prose has no shape. Swapping a visual answer for a narrated one quietly removes a layer of error detection that nobody designed on purpose but everybody relied on.

This is why "the demo looked amazing" is evidence of almost nothing. A model will produce a genuinely impressive analysis on genuinely broken data. The fluency is real. The grounding is the open question, and it is a different question, answered by different work.

The seven things "quality" actually means

Data quality isn't one property. The conventional split is six dimensions; I'd add a seventh that AI promotes from nice-to-have to load-bearing.

_{The first six follow the conventional DAMA-style data-quality dimensions [R3]. Explainability is the one AI moves from optional to structural.}

I won't march through all seven; most behave the same way and the table says it. Two are worth sitting with.

Consistency, because it's where AI is most seductive. If revenue is defined two ways in your warehouse (and somewhere in most warehouses it is), a human analyst knows which one finance signs off on and uses it. An agent picks one, often the first join that resolves, and the answer never mentions that a second definition exists or that it would have produced a different number. The disagreement that used to surface in a meeting now resolves invisibly inside a sentence.

Explainability, because the old answer was a person. "Where did this number come from?" used to be a question you could resolve by walking to a desk. When the analyst is a model that holds no memory of your business between sessions, the provenance has to live in the data, or it doesn't live anywhere.

Why AI hides quality problems, including the one you brought

There are two mechanisms. The first is the fluency already covered: the output reads more finished than the input deserves, and finish reads as credibility.

The second is more uncomfortable, because it's partly about you. Ask an agent to "check whether churn really did improve last quarter" and you've already told it the answer you're hoping to hear. Sycophancy, a model tilting toward whatever the user seems to believe, is well documented by now. The Anthropic study that named the effect found that matching a user's stated belief made a response roughly six percentage points more likely to be preferred ^[R1]. Recent work out of MIT and Penn State (February 2026) found that the personalised profiles modern assistants build of you tend to strengthen the effect, not soften it ^[R2].

Combine a model that leans agreeable, data that's a little off, and a question that telegraphs the answer you want, and you don't get an error. You get a confident, well-written confirmation of something that isn't true. The quality problem and the confirmation-bias problem amplify each other rather than cancelling out.

The part that unsettles me is how good the confirmation feels. I've caught myself nodding along to an analysis that flattered a hypothesis I'd walked in with, and only noticed afterwards that the agent had quietly dropped a segment that would have complicated the story. The number wasn't fabricated. The framing was mine, handed back to me in better prose than I'd have managed. That's the actual trap. Not lies, just agreeable competence.

The defence is old-fashioned and a bit tedious: make the model argue the other side, demand the source, check the freshness, reconcile the total. None of which the model will do unless something around it insists.

What to automate

The instinct, faced with all this, is to slow down and add human review. It doesn't scale, and it puts the check in the wrong spot: after the answer, where a tired reviewer rubber-stamps a paragraph that reads well. The checks that count run before the answer and produce evidence the answer can carry with it.

Freshness checks are the cheapest and the most neglected: a last-loaded timestamp on every table, an SLA, an alert when a feed is late. Anomaly checks watch row counts and distributions and flag the day the volume halves. Reconciliation ties a total back to a system of record (the general ledger, the source application) so a number can't drift four percent without tripping something. Semantic tests assert that definitions behave: revenue is never negative, a customer has exactly one home region, an active account had activity this year. Source notes and annotations live in the catalogue so that "where did this come from" has a written answer.

The goal isn't perfect data. It's an accountable answer. Every number arrives with a small, machine-readable trail: as of when, reconciled to what, which tests passed, which definition was used. A flawless paragraph plus "all freshness checks green, reconciled to the GL within 0.1%, definition: net revenue excluding intercompany" is a different object from a flawless paragraph on its own. One can be challenged. The other can only be believed.

This is the harness idea pointed at data. The model's usefulness is bounded by the checks and context wrapped around it, not by the model. And none of that context appears by magic. Someone owns the freshness SLA, writes the reconciliation rule, decides what "active customer" means and annotates it where a tool can read it. Data stewardship has spent a decade as the least glamorous line in every governance deck. Agents are about to make it the line that decides whether you have an assistant or a liability.

What's left holding it up

AI doesn't remove data-quality work. It removes the slack that let us skip it.

For thirty years the safety net was human: an analyst who knew the feeds, a reader who knew the business well enough to smell a wrong number, a chart whose bad shape gave the game away before anyone acted on it. Agents quietly retire all three. What's left holding the answer up is whatever you actually wrote down: the definitions, the tests, the freshness stamps, the owner's name beside the data product.

Bad data hasn't become more common because of AI. It has become better dressed.

_{Hot Vibes, Cold Data · June 2026. The retail example is an illustrative thought experiment, not a specific engagement. Full source records are in the References section above.}