In Praise of Tacit Knowledge

Polanyi's Paradox, Work And The Silent Currents of Knowing

Few figures are as simultaneously towering and elusive as Michael Polanyi. Polanyi, polymathic in his pursuits, straddled disciplines with the deftness of a protean thinker, challenging scientific conventions in physical chemistry, economics, philosophy, and the sociology of science. Born in Budapest to a well-educated Jewish family, Polanyi’s formative years were steeped in a culture of intellectual rigor and artistic ferment. After earning a medical degree and later a Ph.D. in physical chemistry, he made groundbreaking contributions to fields ranging from chemical kinetics to crystallography, often collaborating with eminent scientists such as Fritz London and Eugene Wigner. However, his life was marked by upheavals, his forced emigration from Red Vienna and the Nazi’s to England in 1933, coupled with his scientific discussions in Moscow, shaped not only his academic trajectory but also his philosophical outlook.

“A free society is regarded as one that does not engage, on principle, in attempting to control what people find meaningful, and a totalitarian society is regarded as one that does, on principle, attempt such control.”

Polanyi’s transition from scientist to philosopher was driven by his disillusionment with reductionism and his conviction that tacit knowledge, the unspoken, intuitive foundation of all understanding, was critical to both science and society.

Central to his philosophy was the concept of the “republic of science,” a term he coined to describe the scientific community as a self-regulating system. This metaphor underscores his belief in the interdependence of freedom and order, a recurring theme in his work. Science, for Polanyi, is both a tradition and a rebellion. The progress of science relies on freedom of inquiry and thought, mutual criticism, and a decentralized structure, much like a free-market economy.

Tacit Knowing

Polanyi is famous for his economic work, most notably his book: Full Employment and Free Trade. But his work on human nature, skills and magnum opus on Personal Knowledge has equal impact and importance.

He reminded us that there are mysteries we feel but cannot name, truths that govern us as imperceptibly as the subtle dynamics beneath the surface of a wave. In his slim yet monumental The Tacit Dimension, Polanyi offered an idea as radical as it was subversively elegant:

“We can know more than we can tell.”

Knowledge, he proposed, is not merely the explicit, quantifiable information we write in books or code into algorithms, it is embodied, intuitive, and irreducibly personal. Polanyi's research aimed not just to explain this phenomenon but to reclaim the richness of human cognition from the sterile grip of mechanistic formalism. This is not simple intuition or hunches described by Danny Kahneman, “intuition is nothing more and nothing less than recognition.” Similar to Simon’s belief that we have mind models:

“If one could open the lid, so to speak, and see what was in the head of the experienced decision-maker, one would find that he had at his disposal repertoires of possible actions; that he had checklists of things to think about before he acted; and that he had mechanisms in his mind to evoke these, and bring these to his conscious attention when the situations for decisions arose.”

Tacit knowledge is the jagged edge of intelligence, a deeper knowing. David Autor resurrects this idea in the age of artificial intelligence, dubbing it “Polanyi’s Paradox.” Autor observes how our tacit understanding, the driver’s sense of the road, the chess grandmaster’s instinct for a board, the doctor’s diagnostic intuition, eludes even the most sophisticated computational replication. This paradox, a Gordian knot for technology, cuts through debates on automation, labor, and the evolving human role in an increasingly machine-dominated landscape.

Is tacit knowledge the classical case of Poe's Purloined Letter, that momentous document lying casually in front of everybody, and hence overlooked by all?

Knowledge Unspoken

Polanyi's philosophy begins in a deceptively simple observation, we recognize a face in a crowd but often cannot articulate the precise details that trigger our recognition. Gestalt psychology supplies a framework here, suggesting that we perceive wholes before we grasp particulars. This is evident in principles like the law of closure, where our minds instinctively complete incomplete shapes, and the law of similarity, which drives us to group similar elements together. Additionally, the concept of 'prehension', an innate ability to grasp the environment as a unified whole, aligns profoundly with Polanyi's idea of tacit knowledge as an ‘indwelling’ in the world, highlighting a seamless, often unconscious, organization of perception and understanding. Polanyi extends this, arguing that tacit knowledge underpins not only perception but all intellectual and practical acts. It is the substrate of discovery, the invisible glue binding our fragmented awareness into coherent understanding.

In Polanyi’s view, this tacit dimension refutes the reductionist dream that all knowledge can be made explicit, formalized, and universally codified. He argued that formal systems are intrinsically incomplete, as they depend on tacit assumptions and interpretations that lie outside the system’s explicit boundaries. This reliance on unspoken premises narrows the system's scope and risks distorting understanding by ignoring the vital contextual and intuitive elements of knowledge. Polanyi warned that the obsessive drive to formalize could render subjects lifeless, much like dissecting a bird to understand its flight, capturing mechanics at the cost of its essence. As he provocatively framed it, “the process of formalizing all knowledge... destroys its subject matter.” Polanyi says if we do not begin to tap in to this tacit knowledge it is akin to “violent self-immolation of the mind.”

Machines and Their Limits

Before the Industrial Revolution, textiles were produced by skilled artisans using spinning wheels and handlooms in cottage industries. These craftspeople possessed deep tacit knowledge of their trade, the feel of fibers, the rhythm of the loom, and the subtleties of dyeing processes. This expertise was passed down through apprenticeship, embedded in years of hands-on practice.

However, with the advent of mechanization, such as the Spinning Jenny and the power loom, much of this tacit knowledge was deconstructed into explicit, codifiable processes. Machines took over repetitive tasks, and workers' roles shifted from creators to operators. The industrial system relied less on the nuanced skill of the individual and more on the standardized operation of machinery.

MITs David Autor is one of the world’s leading scholars on machines and work, his 2014 policy paper on the history of computerization and the labor market’s transformation, relied heavily on Polanyi's insights about tacit knowledge. Autor sketches out opposing tasks: routine, codifiable work, calculations, data sorting, and other programmable functions, versus those requiring judgment, adaptability, and context. For example, his research highlights the polarization of job tasks in the modern economy. Routine tasks, such as bookkeeping or manufacturing line work, are increasingly automated, while non-routine tasks, like caregiving or creative problem-solving, remain reliant on human expertise. Autor also discusses how automation has reshaped industries, from the decline of clerical roles to the growing demand for roles requiring interpersonal skills and critical thinking. These shifts highlight the essential role of tacit knowledge in fostering adaptability and innovation within human labor. Computers excel in the former, but the latter remains stubbornly human.

In the field of AI, systems can now process vast datasets, outperform humans in pattern recognition, and execute rote tasks with astonishing precision. Yet, as Autor points out, computers falter in seemingly simple tasks requiring tacit knowledge, interpreting the nuanced dynamics of a social conversation, diagnosing subtle symptoms in a medical examination, devising creative strategies in a crisis, or safely navigating chaotic city traffic in bad weather conditions. The paradox lies in this, the more essential the tacit skill, the harder it becomes to formalize or automate.

Labor, Polarization, and the New Economy

Autor identifies a troubling polarization, jobs bifurcating into high-skill, high-wage positions demanding creativity and problem-solving, and low-skill, low-wage roles relying on manual labor. The middle, traditionally dominated by routine, semi-skilled tasks, is shrinking as automation surges. This polarization is both a symptom and a cause of widening economic inequality, with tacit knowledge playing a central, if underappreciated, role. The implications of Polanyi's Paradox are where we need to focus human skillsets in the AI economy.

For instance a surgeon performing complex operations, a teacher fostering critical thought, or an artist capturing ineffable beauty and insight, each operates in realms rich with tacit expertise. Meanwhile, jobs once requiring moderate skill, bookkeeping, assembly-line work, procedural clerical tasks, translation, call center, are increasingly vulnerable to automation, their value commodified by the ruthless efficiency of code.

The Industrial Revolution's impact on textiles is often used to illustrate the modern transformations driven by automation and AI. Both illustrate how the transfer of tacit knowledge into explicit systems reshapes industries, alters employment structures, and creates new demands for skills. It also highlights the persistent value of human creativity and adaptability, as the transition required inventors and innovators to design and refine these machines, a role steeped in tacit ingenuity.

The Human Advantage

If machines are stymied by tacit knowledge, humans flourish in it. Our comparative advantage lies not in replicating computational precision but in embracing the very qualities that machines lack, elements of creativity, emotional intelligence, ethical judgment, and the ability to overcome ambiguity.

However, Jean Monnet offered up a stark warning:

“People only accept change when they are faced with necessity, and only recognise necessity when a crisis is upon them.”

Polanyi’s vision of human action as rooted in tacit knowing suggests a skill mandate we should strive to nurture, in what I call abstract task intensive occupations. As automation reshapes economies, Education systems must shift from teaching codifiable skills to cultivating broader intellectual, social and emotional competencies. Polanyi's emphasis on apprenticeship and mentorship offers a valuable lens here, these methods immerse learners in the tacit dimension of knowledge, enabling them to develop skills and intuition through guided experience. Such approaches foster a deeper understanding, allowing individuals to navigate complex, ambiguous situations with creativity and adaptability. Policymakers, too, must grapple with the dual challenges of ensuring equity in the labor market while preserving spaces where human ingenuity thrives.

A Society of Explorers

Polanyi concluded The Tacit Dimension with a profound assertion, understanding arises from our indwelling in the world, a dynamic intellectual, moral, and social process that no formula can capture. He reminded us:

“What we call ‘the economy’ is merely the ghost of human choices past. The living must decide its future.”

Autor’s engagement with this idea in the context of automation forces us to ask: as machines take on more of what we do, what does it take to have meaningful work?

The answer, I suspect, lies in paradox itself. Our strength is not in what we can codify but in what we cannot, our capacity to innovate, empathize, and imagine. We need to tap into this resolute force, to trust in the tacit, the unspoken, and the profoundly human dimensions, and build our competence in abstract tasks. In the end this is what Polanyi called a ‘society of explorers.’

And remember: “the structure of authority exercised over a society of explorers is different from that to which a dogmatic society submits.”

Stay curious

Colin