Intelligence Augmentation

Beyond the Fantasy of Technical Sovereignty

“A truly intelligent machine will carry out activities which may best be described as self-improvement.”

~ John McCarthy, Marvin L. Minsky, Nathaniel Rochester, and Claude E. Shannon

In 1956, at Dartmouth, a small group of researchers gave a name to a field they believed could be built. They called it artificial intelligence. the proposal was:

“... every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”

In the decades that followed, the field advanced in bursts. Programs solved logic problems. Expert systems entered firms and hospitals. In 1997, Deep Blue defeated Garry Kasparov in a regulated match under formal rules. In 2016, AlphaGo defeated Lee Sedol in Seoul. By then, machine learning had already entered search, advertising, fraud detection, logistics, and medicine.

The language around these systems grew larger than the systems themselves. Startups called ordinary software AI. Executives spoke as if judgment had become a software procurement category. Researchers, meanwhile, kept finding the same obstacle in new forms. A system could classify, rank, retrieve, or predict, and still fail when the world refused to stay tidy. It could process vast data and still need a person to intervene when confidence fell, context shifted, or consequences sharpened. Many real tasks remain open-ended, and human supervision, interpretation, and correction are not temporary embarrassments in the march toward automation. They are part of the work. More than that, they specify why. Human problem-solving draws on cognitive mapping, selective attention, associative memory, and forms of intuitive reasoning that reduce search space without requiring exhaustive calculation. Machines, by contrast, are often strongest where inputs can be normalized, repeated, and scored against clear objectives. The dispute is not poetry versus engineering. It is a disagreement about what kind of world the system is entering.

Too much writing on AI begins with prophecy. Prophecy is cheap. Procurement is expensive. Hospitals, clinical trials, courts, schools, factories, banks, logistics networks, public agencies, and ordinary offices do not live inside keynote slides. They live inside exception handling. They live inside ambiguity. They live inside the small, repeated fact that the case in front of you is not quite like the last one, and that the difference is exactly where the trouble enters. The AI Labs fixation on ‘growing brains’, machine consciousness, mass unemployment treat the human being as a temporary nuisance standing between the present and full technical sovereignty.

An Intellectual Wildcard

A paper, by Jarrahi, Lutz, and Newlands, states the issue plainly. AI is often treated as an “intellectual wildcard,” and that looseness has practical costs. The examples of Anthropic’s Mythos show this clearly. It clouds judgment about what these systems can and should actually do. It also encourages a false contest between machine intelligence and human intelligence, as if the only serious question were who wins. But their argument is more disciplined than that. Human intelligence, they write, retains forms of general, contextual, analogical, social, and intuitive judgment that current AI does not possess. AI can reveal correlations at speed and scale. It can outperform us on bounded tasks. Yet in the wild, outside the lab, most important tasks are not bounded for very long. Goals are loose. Context shifts. Stakes collide. Trade-offs are political before they are computational. Under those conditions, the old fantasy of clean replacement begins to look less like science and more like AI Labs managerial daydreaming.

I believe this is why the phrase “human in the loop” is often misunderstood. It is usually spoken in the tone of a concession, as if the human were a brake pedal left over from a less efficient age. In the scientific papers the phrase means something sturdier. For example, Zheng and co-authors define human-in-the-loop hybrid-augmented intelligence as a system in which the person remains part of the decision process, especially when the machine produces a low-confidence result. That sounds obvious until one notices what it overturns. It overturns the cult of frictionless automation. It says that verification is not a sign of weakness. It says that judgment is essential when confidence scores stop being enough. It says that the final demand in serious domains is not merely prediction, but accountable interpretation.

Uncooperative

Many organizations buy AI with the hope of removing discretion from work. Discretion is slow. Discretion requires training. Discretion leaves fingerprints. So the dream is a system that will standardize the world. Then the world, in its usual uncooperative manner, keeps sending edge cases, adversarial cases, tragic cases, cases written in poor language, cases with missing data, cases whose facts arrived late, and cases in which the most relevant fact was the one no database had thought to store. The human being, whom management had hoped to demote to ceremonial status, has to be invited back into the room and asked to save the apparatus. Jarrahi and his co-authors have a better name for this than most boardrooms do. They describe such systems as technologies of heteromation, arrangements that still rely on humans as indispensable mediators even when presented as if the machine were operating on its own.

These papers are not defensive or nostalgic, their strongest claim is positive. They argue that the real promise lies not in machine substitution but in mutual augmentation. Jarrahi and his colleagues split the field with admirable economy. There is AI that may exceed human performance on certain tasks, and there is hybrid intelligence, the overlap where humans and AI augment one another. The two products of that overlap are worth holding onto: human-augmented AI and augmented human intelligence. The first reminds us that machines are often trained, corrected, maintained, and propped up by human labor. The second points toward something more hopeful: systems that widen human cognitive bandwidth instead of shrinking human agency.

Distributed Intelligence

We have spent too long speaking as if intelligence were a trophy to be awarded either to the machine or the species. But intelligence in practice is often distributed across arrangements. A pilot with instruments is not less intelligent than a pilot without them. A doctor who uses imaging, statistical models, and structured support is not less a doctor. A historian with archives, search tools, transcription software, and pattern detection is not relieved of interpretation. Quite the opposite. The apparatus expands the field of possible attention, then returns the burden of judgment to the person who must live with the outcome.

Kasparov understood this before many executives did. Once the symbolic duel between man and machine had been staged to exhaustion, another truth appeared. A grandmaster working with AI could play better than either alone, because the machine handled analytical calculation while the human concentrated on strategy and broader positional judgment. That is not a sentimental compromise after defeat. It is a more mature account of work. The point is not that the machine grows more human or that the human learns to imitate the machine. The point is that each party gives the other access to a form of strength it does not natively command.

I think this is where augmentation becomes morally serious. The central question is not whether a machine can produce an answer. Machines produce answers all day. The central question is what kind of human being, professional culture, and institutional order is being formed around the answer. A system that helps a radiologist scan more images without dulling clinical responsibility may be a genuine gain. A system that helps a judge process filings faster while quietly laundering unjust assumptions into official language is not augmentation. It is abdication with software.

Tacit Knowledge

This is why I was struck by the insistence, across these papers, on trust, explainability, and tacit knowledge. Szczerbicki and Nguyen note that a truly intelligent artificial system has yet to be built, and that trust and explainability remain central difficulties. Jarrahi and his co-authors sharpen the point by distinguishing human tacit knowledge from machine tacit knowledge. Humans, as Polanyi put it, know more than we can tell.

Tacit knowledge embodies personal wisdom, insight, and intuition and is intertwined with a wealth of experience over time. By definition, tacit knowledge is hard to express or extract (made explicit). Consequently, much of humans’ tacit knowledge is impossible to be replicated by AI.

Machines too may arrive at outputs whose internal path is not easily articulated to users or even developers. But these two forms of opacity are not symmetrical. Human tacit knowledge is embedded in embodied experience, social learning, and responsibility. Machine opacity is embedded in technical process and abstraction. Confusing the two is one of the slyer conceptual errors of the age. A person who cannot fully verbalize seasoned judgment is still a moral agent. An AI model that cannot explain itself is still an AI model.

The optimistim which I share, is not that opacity disappears, but that institutions can be designed so that opacity does not become an alibi. Zheng’s description of hybrid systems is especially impressive. The machine learns from data, the human intervenes when confidence is low, and the system updates its knowledge base. In other words, the loop is not ornamental. It is epistemic. It is where error is checked, where context is reintroduced, where the AI model is pulled back toward the world. That should not be mistaken for inefficiency. In many settings it is the only honest route to reliability.

Humans at their Best

There is, moreover, something deeply humane in the better version of this arrangement. We are not at our best when we are forced to behave like exhausted calculators. Most bureaucracies do precisely that. They bury people under volume, repetition, search costs, formatting chores, and the dead labor of retrieval. In that environment, to give a human being systems that sort, filter, compare, flag, and surface relevant patterns is not to diminish intelligence. It is to rescue it from clerical ruin. Business is run on support systems, cognitive interfaces, context-aware retrieval, active learning, knowledge sharing, and decision enhancement. Behind those technical phrases should be a very plain ambition: free people to spend less time drowning in procedural sludge and more time on discernment.

Of course, that ambition can be corrupted. Every tool that widens capability can also become an excuse to intensify labor. The same assistant that broadens cognitive bandwidth can become the reason management doubles the caseload. And the same machine learning system that performs beautifully on normalized, repeatable inputs can begin to wobble when faced with non-integrity, ambiguity, or unstructured material, which is exactly where human intelligence retains its advantage. The same system sold as augmentation can become a sly instrument for surveillance, speed-up, and blame transfer. I do not think the answer to that danger is to retreat into a pious anti-technical posture. It is to govern deployment around a simple standard. Does the system leave the human operator more able to understand, contest, redirect, and own the result, or less? If less, one should stop calling it augmentation and speak more honestly.

We should not confuse intelligence with mere throughput. Human intelligence is not just problem-solving power. It includes causal judgment, selective attention, memory, experience, intuition, cognitive mapping, and value judgment in open environments. Zheng and his colleagues are particularly useful on this point. They describe hybrid-augmented intelligence not as a slogan but as a framework built from concrete elements: selective attention that helps an agent screen what is relevant in a crowded environment, cognitive maps that organize relations among events and options, associative memory that links new situations to prior experience, and intuitive reasoning that allows a person to move through complexity without brute-force search. That is why clerical overload is so destructive. It wastes precisely the faculties that are hardest to reproduce. Whilst this may be refreshingly unglamorous. It returns intelligence to lived situations. It also restores dignity to the kinds of work that technical culture often underrates: noticing what is missing, sensing when a category does not fit, asking whether a high-confidence output has arrived for low-quality reasons, recognizing when a recommendation has crossed the line from aid into command.

A Fruitful Path

So yes, I am positive about intelligence augmentation. I think it is the saner path, the more fruitful path, and in the long run the more ambitious path. Replacement flatters the engineer and degrades the institution. Augmentation sets a harder task. It asks us to build systems that do not merely perform, but collaborate. It asks us to design for reciprocity rather than conquest. It asks us to admit that human beings are not residues in the system but sources of context, restraint, reinterpretation, and purpose. That is not a failure of the machine. It is a fact about the world in which the machine must operate.

The future worth wanting is not a world in which the last human signs off on decisions already made elsewhere. It is a world in which technical power is used to widen human range without severing human responsibility. Better retrieval. Better pattern recognition. Better warning systems. Better comparative analysis. Better support for memory, diagnosis, planning, and invention. But also a stronger grip on reasons, consequences, and dissent. I do not find that vision small. I find it adult.

Artificial Intelligence is not made more impressive by pretending it can float free of human judgment. It becomes more impressive when it enters a working partnership with it. The machine can search a million possibilities before dawn. The person can notice that the question was wrong. And in any civilization I would trust, the second achievement still earns the steadier respect.

Stay curious

Colin

Image: The Screwtape Letters by C.S. Lewis. “There are two equal and opposite errors into which our race can fall about the devils. One is to disbelieve in their existence. The other is to believe, and to feel an excessive and unhealthy interest in them.”

Google DeepMind - The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousnes

OpenAI - The AI Jobs Transition Framework Report