The Same Answer

For the first time a first-year analyst and a twenty-year partner at the same consulting firm can ask the same question and get the same answer in the same four seconds. That is knowledge equality, and it is real. It was supposed to make the partner cheap.

A software company gave 5,172 customer-support agents a language-model assistant in waves so researchers could watch what changed. Productivity rose 15% on average, 30% for the least experienced, and barely moved for the most experienced. The authors’ description of the mechanism is exact: the tool was capturing and disseminating the behaviors of the most productive agents. 453 professionals doing writing tasks from their own occupations produced better work faster, and the gap between the weakest and strongest narrowed.

So the premise of the fear holds. Producing a competent answer used to be a skill; it is now a utility. It comes out of the wall, and who you are no longer decides whether you get it.

Now measure the same tools from the other end.

758 consultants at BCG were given tasks the researchers had chosen to sit inside what the model could do, and produced work graded 40% higher. They were also given one chosen to sit outside it. It looked exactly like the others. The answer required reading interview notes against a spreadsheet of financials, and the model’s confident version was wrong. With AI the consultants were 19 percentage points less likely to get it right than the consultants working alone. Same people, same tool, opposite result. The researchers called the boundary jagged. Nothing on the surface of an answer tells you which side of it you are on.

In Kenya 640 small-business owners were randomized to a standard business guide or a GPT-4 mentor over WhatsApp and followed for five months. Across all 640 the mentor made no measurable difference. That average was hiding two opposite results: the owners already doing well gained about 15%, and the owners doing badly lost nearly 10%. The researchers checked whether the groups had asked different questions or received different advice. They hadn’t. The difference was which parts of the same answer they chose to act on.

That is what knowledge equality does not equalize. Access to the answer was made equal. The ability to tell whether it is right was not; it was amplified. The tool multiplies whatever you bring to it. If what you bring is the inability to tell a good answer from a plausible one, it gives you more plausible answers faster and in better prose.

The failure mode is new. We have always read fluency as evidence of knowledge, and it was a fair proxy: speaking clearly about a subject in its own vocabulary took the years of actually learning it. The manner was expensive, and the expense was the signal. It is now free. The commoditized answer arrives fluent whether or not anything behind it is right. Sounding like someone who knows has come apart from knowing, and the only test left is whether the thing is true.

In 2023 two New York lawyers filed a federal brief citing six cases that did not exist. ChatGPT had produced them: names, docket numbers, judges, quotations, internal citations to other cases that also did not exist. One lawyer grew uneasy and asked the tool whether the cases were real. It said they were. The judge fined them $5,000. The part worth dwelling on is who noticed: opposing counsel went to look them up and could not find them. The brief was invisible as an error to everyone who did not already know the field, and visible in an afternoon to someone who did.

A second effect makes the first worse. Access to knowledge feels like possession of it. In nine experiments people looked up an explanation online for something like how a zipper works, then rated themselves as knowing more about unrelated subjects they had not looked up at all. That was a search box. Last year 16 experienced open-source developers were randomized task by task to work with or without AI tools, on repositories they had each spent about five years in. With the tools they were 19% slower. Afterwards they believed they had been 20% faster. The commoditization is of the median and not the tail, and even the tail could not feel it.

Why does the tail hold? Hayek answered this in 1945. There is knowledge that can be written down and sent to a central authority, and there is what he called knowledge of the particular circumstances of time and place: what the person on the spot knows about this machine, this customer, this shipment. It never gets written down, so it cannot be centralized, and his point was that it is the kind that decides. AI is the most complete centralization of written knowledge anyone has built, and by construction it holds every kind except that one.

Call them universal facts and scenario facts. A universal fact is true everywhere and stays true: the boiling point of water, the statute of limitations, the standard dose. That is what the models were trained on, and it is exactly why it could be centralized. A scenario fact is true here and now and may not be by the afternoon: this patient’s potassium this morning, this site after last week’s rain. It has to be gathered by someone present. The universal fact tells you what is usually true; the scenario fact tells you whether usually applies today.

Polanyi put it in seven words: we know more than we can tell. Nobody pays for the general case anymore; it is on the wall. They pay for an answer run through the particulars and returned as an opinion: do this and not that, here and now, and here is why. Most of the conditions are not in the prompt, because the person who would have to type them is the person who does not know they matter.

Radiology ran the experiment in public with the most credible forecaster available.

In 2016 Geoffrey Hinton told an audience in Toronto that people should stop training radiologists now. It was “completely obvious” that within five years deep learning would do the job better. Ten years on, radiology has adopted more of the tool than any other field. The FDA has cleared 1,451 AI-enabled medical devices; 1,104 of them are radiology, or 76%. And there are more radiologists, not fewer. Mayo Clinic’s radiology staff has grown 55% since the forecast, to more than 400. This year’s residency match offered 1,478 radiology positions, the most ever, up from 1,412. Roughly 4,300 radiologist jobs sat open this spring, and the benchmark time to fill one is about four months. Hinton has since told the Times he spoke too broadly.

What happened is what the studies predict. The algorithms took the median read: the normal chest film, the clean scan, the fracture where a fracture usually is. Each of the 1,104 produces a flag, and each flag has to be turned into a decision by someone who knows whether it is the patient’s problem or the algorithm’s, who can see that the shadow is the surgery from 2019 and not a mass, and whose name goes on the report. The algorithm cannot be asked why. It cannot be sued. The specialist’s job did not shrink to what the tool left behind; it moved to the point where the tool’s answer meets the particular case, and that turns out to be the whole job.

So here is what I think a skill is now. It is narrower than the word used to be.

It is the ability to tell the right answer from the plausible one in one domain. You can only build that by having been wrong in that domain many times. There is no general detector for plausible-but-wrong: the New York lawyers had a general education, and opposing counsel had read a great many real cases.

It is reading how the general answer bends under this history and these constraints at this moment, and coming back with an opinion rather than a summary. That part of a field never made it into writing because it lives in the gap between the rule and the case in front of you.

And it is being the person who signs: the radiologist’s report, the engineer’s stamp, the lawyer’s name on the brief. The rule the New York lawyers were sanctioned under exists because someone has to have vouched. A tool cannot vouch, and the reason is not a gap in its capabilities. Vouching means bearing the consequence.

None of the three is general. Each is judgment about something. The last essay argued that judgment is what remains once the procedure is automated. This one adds that judgment does not transfer. You cannot spot an invented case unless you have read a great many real ones.

Intelligence became equitable. Being right about something in particular did not. Everyone can get the same answer now and it costs less than it ever has. What the specialist was worth did not fall with the price. It concentrated in the one place the same answer cannot reach: the person who knows how it bends here, who can tell when it is the wrong one, and who will put their name under it. The way to become that person has not changed: years in a single field, until its nuance is as familiar to you as the general answer is to everyone else.