The distinction between human tacit knowledge and machine opacity is the line most AI writing won’t draw. You drew it. And the Kasparov point lands because it names what augmentation actually looks like when both sides bring something the other can’t.
But I keep coming back to one assumption underneath the whole argument. Every example — the radiologist, the judge, the pilot, the historian — is someone who already built their cognitive architecture. They spent decades developing the tacit knowledge, the judgment, the intuitive reasoning you’re describing. The question your article asks is how to augment that intelligence without severing responsibility. It’s the right question.
Here’s the one I’d add: what happens when the architecture was never built?
I teach high school chemistry and physics. I’ve been in classrooms for over 20 years. I watch students encounter AI before they’ve developed independent reasoning, before they have any tacit knowledge to augment. What I see isn’t augmentation — it’s substitution wearing augmentation’s clothes. The student produces. The grade confirms. The cognitive architecture never forms. They look augmented. They’re hollow.
Your test — “does the system leave the human more able to understand, contest, redirect, and own the result?” — is exactly the right test. I built an assessment around that question. The answer, for most students I measure, is no. Not because the tools failed. Because the prerequisite was never built.
You named the destination. I’m trying to build the on-ramp.
Would love your take on what that prerequisite layer looks like at the institutional level — because right now, nobody in education policy is asking the question before they hand out the tools.
Thank you Syd, you are identifying the hollow student, someone who has the throughput of a professional but none of the causal judgment that makes the work real.
In the language of the papers I referenced, what you are seeing is machine substitution masquerading as mutual augmentation. If a student uses AI to solve a physics problem before they have built their own associative memory or internal cognitive map of the laws of motion, they aren't being augmented. They are being bypassed. As Zheng notes, human unique intelligence is built on the ability to organize relations among events, if the machine does that organizing, the student never develops the muscles of selective attention or intuitive reasoning.
At the institutional level, the prerequisite layer has to be about protecting the struggle.
We have to recognize that tacit knowledge is not a file you can download; it is, as Jarrahi writes, embedded in embodied experience and social learning. In education, that means policy must distinguish between:
Where the human architecture is being built through repetition, failure, and unassisted retrieval. Here, AI is often a poison because it removes the very friction required for learning.
Where a person with a seasoned epistemic foundation uses the tool to widen their bandwidth.
If we hand out the tools before the assembly phase is complete, we are not creating pilots with instruments; we are creating passengers who think they are pilots. The institutional on-ramp must involve low-tech zones where the goal is not the deliverable (the grade), but the formation of the moral agent who can eventually be trusted to own the result.
You are building the on-ramp by asking the right test: does this help the student understand or just produce? If the answer is "no," then we are not looking at progress, we are looking at the industrialization of a shortcut.
Colin — “passengers who think they are pilots.” That’s the line. That’s the whole problem in one sentence.
Your distinction between assembly phase and augmentation phase is the policy frame I’ve been circling around without naming it. I’ve been saying “ownership before extension” but “protect the struggle during assembly” names the institutional version of that idea. That’s language a superintendent can act on.
What I see in the classroom maps directly onto what you’re describing. The students who arrive having already built their cognitive architecture use AI the way Kasparov used Deep Blue — it widens their range. The students who skipped the assembly phase use AI the way you described — they become passengers who think they are pilots. Same tool. Completely different outcome. The variable isn’t the technology. It’s the readiness of the person holding it.
I’d love to continue this conversation. Your framework and mine are covering different altitudes of the same problem.
When a student uses the machine to bypass the struggle, they fail to build the internal maps required to navigate when the tools hit a wall. As we discussed, once you are in the wild, context shifts and stakes collide. A passenger who thinks they are a pilot is fine during a smooth flight, but they have no seasoned judgment to rely on when the instruments start giving low confidence results.
You are far ahead of me in AI intelligence and competence and yet so refreshingly caring about a holistic system that includes ethics embedded within AI. You combine achievements with audits that calibrate a conscientiousness aligned with the artificial world we see submission to in today’s pacing for profit. The posts in these areas are a wake up call before we wake up to ask, “What have we done?” to a better statement, “Look what we did!”. To leave a better world is hopefully what will happen.
Thank you Cathie. Real conscientiousness means refusing to let these tools float free of our values. If we can keep that alignment, then we can widen our range without severing the responsibility that makes us human.
Love these two points. The depth underneath each one is staggering. Millions of years of neurological evolution, billions upon billions of human experiences.
I've been trying to re-popularize "human in the loop" in my circles - so much capability is left on the table otherwise. I generally don't take the time to share my workflows but I'm dipping my toes into the cacophony a little. There are many more like me, but unfortunately the "Automate Everything"/AGI crowd by far has the loudest voice.
What stayed with me in this piece is the sense that augmentation is not really tested by output quality alone. It is tested by whether the human being still retains meaningful veto power.
That is what makes the argument so serious. A system can leave a person formally “responsible” while quietly stripping away the practical conditions of responsibility, too much speed, too much volume, too much institutional pressure to rubber-stamp what arrives already polished. In that setting, the human remains in the loop mostly as a liability sink. The language of augmentation stays in place, but the authority to interrupt, question, or slow the machine begins to disappear.
What gives the essay its force is that it understands something many deployments try to hide: judgment is not only a matter of seeing the right thing, but of being allowed to act on what one sees. In that sense, the deepest line between augmentation and abdication may not be whether the human is present at the end. It may be whether the human still has enough standing inside the system to say, with consequences, this output is not ready to govern reality.
Thank you Laurentiu. If the institutional pace is designed for the speed of a machine rather than the speed of human discernment, then the human in the loop is just a ceremonial signature. As Jarrahi and his colleagues warn, these technologies of heteromation often use the presence of a person to satisfy regulatory or moral requirements while stripping that person of the actual bandwidth required for accountable interpretation.
The authority to slow the machine down is the only true test of an augmented system. If an organization measures success only by throughput, it creates an environment where the human has no standing to say that a high-confidence output is fundamentally unsound. In that setting, the system is not enlarging human practical reason; it is merely using the person to launder its assumptions into official language.
You are right that the deepest line is about standing. It is about whether the institution values the person as a source of context, restraint, and purpose, or whether it views them as a friction to be removed. Until we design systems that protect the human authority to interrupt, we are not building augmentation. We are building a more efficient way to avoid responsibility.
Exactly. Once the human is left only to ratify what has already arrived with speed, polish, and institutional momentum, responsibility survives mostly as appearance. At that point the real loss is not only discretion, but the practical authority to let judgment interrupt the system.
What an absolute bore...but a very handy bore all the same...like a designated driver...
Stick AI at a dinner table with a group of four famous people you would like to meet, eat, drink and greet with, past and present...
Who would talk to the AI?...maybe in the shape of Robby the Robot..."warning warning Will Robinson" Talking with AI you would have to explain every joke, whimsical comment, double entndre, pun to it...AI knows everything after the fact of disclosure...
A famous quote is how I would respond to AI..."you know nothing Jon Snow"
You know nothing of the human heart and the richness and complexity of human lives lived...
Yes! That is why I had the CS Lewis image:-) The distinction between having data and having a soul. The machine knows nothing of the human heart because it has never lived. We need to push back against the AI labs, as your comment shows. The goal is to ensure the human beings stay in the lead, holding onto the strategy and the broader positional judgment while the machine handles the analytical calculation in the background.
Good points as always Winston, thank you. When a system is marketed as being able to float free of human judgment, it is rarely a technical claim, it is a value proposition. You are right that it is more profitable for a purveyor to sell a black box that promises to remove the cost of human discretion. But as Jarrahi and his colleagues point out, this often leads to heteromation. The organization pays for the fully automated software, but they end up quietly relying on human workers to verify, correct, and prop up the system when it hits the uncooperative reality of the world.
The oligarch's dream of total replacement usually results in a system that is efficient at standard cases but catastrophic at edge cases. The profit is harvested in the short term by removing the nuisance of the human, but the cost returns later in the form of operational risk and the procedural sludge I mentioned.
The corruption of the ambition to widen capability into a tool for intensifying labor is the central political struggle of augmentation. If the technology is used to shrink human agency while increasing the caseload, it isn't augmentation at all, it’s just a more sophisticated treadmill. That is why the standard of whether a system leaves the operator more able to own the result is so critical. It is a guardrail against the drive to turn professionals into mere ceremonial rubber-stamps for a machine’s high-confidence, low-quality output.
Instead of arguing directly for or against the thesis, I want to offer two stories from lived experience, because I think they reveal something more important than abstract agreement or disagreement.
The first concerns a project that was the most important initiative in my organization at the time. It was not new; in fact, the organization had already tried and failed several times to complete it. When I brought in someone to help, I deliberately told him nothing about the project’s condition or the team’s dynamics. I wanted an independent read. He attended orientation on Monday, sat through a handful of meetings over the next few days, and by Friday morning, I asked him a simple question: “What is the status of the project?”
His answer was immediate: “It is a watermelon.”
I asked what he meant, because the project had been reported as green for over six months. He said that from the outside, looking in, it appeared green. But from the inside, to someone who knew what to look for, it was bright red. That is why it was a watermelon.
What struck me was not only that he saw the problem, but that he saw it within days, while a large project team from one of the top consulting firms in the world had been treating the project as healthy for months. What did he notice? Not one thing that could easily be reduced to a checklist. He saw what was missing. He saw what had not been asked, what had not been understood, what had been assumed too quickly, and what had been executed in the wrong sequence. He recognized the signs because he had developed the kind of tacit knowledge that comes only from experience, observation, failure, and pattern recognition built over time.
That is why Newton’s line remains so powerful: “If I have seen further, it is by standing on the shoulders of giants.” We learn not only by reading or by formal instruction, but by watching how capable people think, sequence, question, and judge. Over time, we begin to detect the hidden signals: whether a team has truly understood the problem, whether they are merely repurposing a previous solution, whether they appreciate the uniqueness of the institution in front of them, and whether they understand that no two organizations share the same dynamics, decision-making culture, influence structure, or interpretation of law, regulation, and guidance. In this case, the consulting team had assumed the project was similar enough to another effort in the same domain that it could simply be replicated. That assumption was the beginning of the failure.
The second story is smaller in scale, but it points to the same truth. I was asked to take over the infrastructure and security functions of a sister organization. In my usual fashion, I asked the two leaders in my own organization who oversee those areas to prepare an approach and timeline. A week later, I had two decks.
One was thoughtful. The person had clearly applied judgment, likely done outside research, and probably used AI as well. But the work reflected real effort: it accounted for our organization’s limitations, invested time in understanding the target organization, and presented a sequence of actions that made practical sense.
The second deck was different. It was obvious that the problem had largely been handed to AI, the output copied and pasted, and then lightly edited to make it look less generic. But the logic was weak. The task order was wrong. Some steps did not fit an organization of that size. The content appeared competent without the substance of understanding.
That difference matters. Context matters. Sequencing matters. Adaptation matters. Validation matters. And all of those depend on the human being using the tool.
This is why I think the phrase “augment the human” is still too vague to be useful. The real question is: which human? Augmentation is not automatically beneficial simply because a tool is available. A strong tool in the hands of a person with judgment can extend capability. The same tool in the hands of someone without judgment or the discipline to think can industrialize mediocrity. The tool does not remove the need for understanding; it makes the presence or absence of understanding even more consequential.
That is why one principle in my organization’s AI policy matters so much, which I made sure is included. Paraphrased, it says this: you own the deliverable and the outcome regardless of how you produced it. Whether you called a friend, researched online, consulted a colleague, or used AI, the responsibility remains yours. If it is wrong, you cannot blame anyone else.
I think that principle gets to the heart of the issue. The problem is not AI. The problem is the illusion that AI can substitute for judgment, accountability, or tacit knowledge. It cannot. At best, it can amplify what is already there. If the person using it has depth, care, and contextual understanding, the result may be excellent. If not, the output may still look polished while being fundamentally unsound.
In the end, the question is not whether AI augments humans. It does. The more important question is whether the human being using it has developed the experience, judgment, and responsibility required to deserve that augmentation.
As Peter Drucker famously said, “There is surely nothing quite so useless as doing with great efficiency what should not be done at all.” That, to me, is the risk at the center of this conversation: AI can make us faster, but only judgment can make us right.
The question of responsibility is an essential one. This is where accountability arrives. Your two stories are important learning curve examples you’ve shared. The trajectory chosen needs to be explicitly understood and not indifferently off loaded with implicit confidence. AI may provide answers, but do the answers align to solve the problem in the best way. Your reality check experiences very insightful. We cannot substitute speed for depth.
Adapted from a saying often attributed to Gandhi: “There is more to life and work than speed, cost-cutting, efficiency, and money.“ Cathie Campbell added cost-cutting, and I added work, efficiency, and money after a conversation with someone in my organization.
Additionally, the same caution applies to the AI community’s newest thesis: “everyone can now code and build applications.” Yes, in a narrow sense, they can. But that does not mean everyone can build responsibly, especially in domains like healthcare. What it may produce instead is more versions of cases like the Medvi (more here: tinyurl.com/396u63u7) example: polished interfaces, strong growth metrics, persuasive marketing, and the appearance of innovation, while the real clinical responsibility, regulatory exposure, and operational risk are pushed somewhere else in the stack.
This is why we should be careful what we celebrate. When a barrier falls, it not only lets talent in; it also lets in people who can assemble powerful systems without the judgment, domain understanding, or accountability required to use them well. In low-stakes settings, that may produce mediocre software. In healthcare, it can produce something far worse: credibility without competence, scale without governance, and risk that is not managed but merely redistributed.
So yes, everyone can now code. The more important question is whether they understand the domain they are coding for, the consequences of getting it wrong, and whether they are willing to own the outcome. In healthcare, especially, capability without judgment is not progress. It is an invitation to unintended consequences that may be worse than the problems we were trying to solve.
That’s why I keep coming back to this line from Jurassic Park: “Your scientists were so preoccupied with whether they could, they didn’t stop to think if they should.”
Your first story about the watermelon project is a masterclass in why tacit knowledge remains the ultimate bottleneck. As we have often discussed, and as outlined in the essay, humans know more than they can tell. That consultant did not just have a better checklist; he had a more sophisticated cognitive map of what a failing project actually feels like. He could sense the non-integrity of the green status because his pattern recognition was tuned to the silences and the missing questions, things an AI, which relies on normalized and repeatable inputs, is structurally blind to.
The second story about the two decks highlights the exact danger of industrializing mediocrity. When that second leader handed the problem to the machine, they engaged in a form of heteromation, hiding the lack of human thought behind a polished, AI-generated interface. It produced what you rightly call credibility without competence.
This brings us to the core of your AI policy: owning the deliverable. If we treat the human-in-the-loop as merely a ceremonial sign-off, we are inviting the Medvi-style disasters you mentioned. For the loop to be epistemic (meaning it actually produces knowledge and reliability), the human must be a moral agent who can contest and redirect the machine, not just a passenger on a high-speed route to a wrong destination. Which reminds me of one of my favorite Japanese sayings: "If you get on the wrong train, get off at the nearest station. The longer you stay, the more expensive the return trip will be."
You are right to ask: which human are we augmenting? If we augment the person without the associative memory or causal judgment to know when a task order is wrong, we aren't creating a better professional; we are just accelerating an error.
In healthcare and law, as you noted, the stakes are not just about efficiency, they are about the burden of judgment. A polished interface that launders a wrong assumption is, as I wrote, just abdication with software. Your point stands as the necessary warning to the 1956 Dartmouth dream: technical power is only an adult vision when it is tethered to a human being who is willing to be the one left standing and responsible to make a decision when the machine produces a low-confidence result.
And the following scene from one of my favorite movies captures, better than I ever could, what I was really trying to say above. It gets at something many people building AI, however brilliant they may be, still struggle to grasp fully: what it actually means to be human. You can accumulate knowledge, process information, and optimize outputs, but that is not the same as lived experience, judgment, suffering, empathy, or understanding the invisible battles other people carry.
Yes, brilliant, just brilliant. Will Hunting, in that moment, is the ultimate intellectual wildcard. He has the speed, the data, and the correlations to rip a painting apart, but as Sean (Robin Williams) points out, he has no tacit knowledge because he has never dared to love anybody that much. AI Labs confuse the description of the world with the experience of it.
“The invisible battles” so true. This is also one of my favorite movies. He hung out socially with the guys but loved his mind finding the chalk and expressing himself on the chalkboard. His inner world eager to expand when others had no shared interest. His courage to reveal himself and to extend himself in the end, thanks to the probity of Robin Williams’ authenticity shared gave Will the will to move on from inertia to pursue his passions. This movie and “A Beautiful Mind” about John Nash are two of my favorites.
So wise Cathie. If we only build for the part of the brain that calculates, we miss the part that chooses. True augmentation should not just make us faster; it should give us the clarity to move from mere processing toward a life of discernment.
I was rereading this and thinking how this applies to life itself. To be willing to question anyone on anything is democracy’s gift to the human struggle for survival. Course correction comes from questions, not blind adherence to a quest.
Colin — this one hit me.
The distinction between human tacit knowledge and machine opacity is the line most AI writing won’t draw. You drew it. And the Kasparov point lands because it names what augmentation actually looks like when both sides bring something the other can’t.
But I keep coming back to one assumption underneath the whole argument. Every example — the radiologist, the judge, the pilot, the historian — is someone who already built their cognitive architecture. They spent decades developing the tacit knowledge, the judgment, the intuitive reasoning you’re describing. The question your article asks is how to augment that intelligence without severing responsibility. It’s the right question.
Here’s the one I’d add: what happens when the architecture was never built?
I teach high school chemistry and physics. I’ve been in classrooms for over 20 years. I watch students encounter AI before they’ve developed independent reasoning, before they have any tacit knowledge to augment. What I see isn’t augmentation — it’s substitution wearing augmentation’s clothes. The student produces. The grade confirms. The cognitive architecture never forms. They look augmented. They’re hollow.
Your test — “does the system leave the human more able to understand, contest, redirect, and own the result?” — is exactly the right test. I built an assessment around that question. The answer, for most students I measure, is no. Not because the tools failed. Because the prerequisite was never built.
You named the destination. I’m trying to build the on-ramp.
Would love your take on what that prerequisite layer looks like at the institutional level — because right now, nobody in education policy is asking the question before they hand out the tools.
https://smalaxos.substack.com
https://a.co/d/0adownwx
Thank you Syd, you are identifying the hollow student, someone who has the throughput of a professional but none of the causal judgment that makes the work real.
In the language of the papers I referenced, what you are seeing is machine substitution masquerading as mutual augmentation. If a student uses AI to solve a physics problem before they have built their own associative memory or internal cognitive map of the laws of motion, they aren't being augmented. They are being bypassed. As Zheng notes, human unique intelligence is built on the ability to organize relations among events, if the machine does that organizing, the student never develops the muscles of selective attention or intuitive reasoning.
At the institutional level, the prerequisite layer has to be about protecting the struggle.
We have to recognize that tacit knowledge is not a file you can download; it is, as Jarrahi writes, embedded in embodied experience and social learning. In education, that means policy must distinguish between:
Where the human architecture is being built through repetition, failure, and unassisted retrieval. Here, AI is often a poison because it removes the very friction required for learning.
Where a person with a seasoned epistemic foundation uses the tool to widen their bandwidth.
If we hand out the tools before the assembly phase is complete, we are not creating pilots with instruments; we are creating passengers who think they are pilots. The institutional on-ramp must involve low-tech zones where the goal is not the deliverable (the grade), but the formation of the moral agent who can eventually be trusted to own the result.
You are building the on-ramp by asking the right test: does this help the student understand or just produce? If the answer is "no," then we are not looking at progress, we are looking at the industrialization of a shortcut.
Colin — “passengers who think they are pilots.” That’s the line. That’s the whole problem in one sentence.
Your distinction between assembly phase and augmentation phase is the policy frame I’ve been circling around without naming it. I’ve been saying “ownership before extension” but “protect the struggle during assembly” names the institutional version of that idea. That’s language a superintendent can act on.
What I see in the classroom maps directly onto what you’re describing. The students who arrive having already built their cognitive architecture use AI the way Kasparov used Deep Blue — it widens their range. The students who skipped the assembly phase use AI the way you described — they become passengers who think they are pilots. Same tool. Completely different outcome. The variable isn’t the technology. It’s the readiness of the person holding it.
I’d love to continue this conversation. Your framework and mine are covering different altitudes of the same problem.
When a student uses the machine to bypass the struggle, they fail to build the internal maps required to navigate when the tools hit a wall. As we discussed, once you are in the wild, context shifts and stakes collide. A passenger who thinks they are a pilot is fine during a smooth flight, but they have no seasoned judgment to rely on when the instruments start giving low confidence results.
The goal of a trusted partnership with powerful AI is an existential one for human potential to thrive alongside AI in this exciting era.
This article is a template of hope for what can arise beyond an unforeseen demise of contractual trust.
Thank you Cathie. For your engagement, enthusiasm and insights.
You are far ahead of me in AI intelligence and competence and yet so refreshingly caring about a holistic system that includes ethics embedded within AI. You combine achievements with audits that calibrate a conscientiousness aligned with the artificial world we see submission to in today’s pacing for profit. The posts in these areas are a wake up call before we wake up to ask, “What have we done?” to a better statement, “Look what we did!”. To leave a better world is hopefully what will happen.
Thank you Cathie. Real conscientiousness means refusing to let these tools float free of our values. If we can keep that alignment, then we can widen our range without severing the responsibility that makes us human.
"Humans know more than we can tell"
"They live inside exception handling."
Love these two points. The depth underneath each one is staggering. Millions of years of neurological evolution, billions upon billions of human experiences.
I've been trying to re-popularize "human in the loop" in my circles - so much capability is left on the table otherwise. I generally don't take the time to share my workflows but I'm dipping my toes into the cacophony a little. There are many more like me, but unfortunately the "Automate Everything"/AGI crowd by far has the loudest voice.
https://cavinmckinley.substack.com/p/robot-dev-teams-and-keeping-the-human?utm_source=share&utm_medium=android&r=7s64hw
What stayed with me in this piece is the sense that augmentation is not really tested by output quality alone. It is tested by whether the human being still retains meaningful veto power.
That is what makes the argument so serious. A system can leave a person formally “responsible” while quietly stripping away the practical conditions of responsibility, too much speed, too much volume, too much institutional pressure to rubber-stamp what arrives already polished. In that setting, the human remains in the loop mostly as a liability sink. The language of augmentation stays in place, but the authority to interrupt, question, or slow the machine begins to disappear.
What gives the essay its force is that it understands something many deployments try to hide: judgment is not only a matter of seeing the right thing, but of being allowed to act on what one sees. In that sense, the deepest line between augmentation and abdication may not be whether the human is present at the end. It may be whether the human still has enough standing inside the system to say, with consequences, this output is not ready to govern reality.
Thank you Laurentiu. If the institutional pace is designed for the speed of a machine rather than the speed of human discernment, then the human in the loop is just a ceremonial signature. As Jarrahi and his colleagues warn, these technologies of heteromation often use the presence of a person to satisfy regulatory or moral requirements while stripping that person of the actual bandwidth required for accountable interpretation.
The authority to slow the machine down is the only true test of an augmented system. If an organization measures success only by throughput, it creates an environment where the human has no standing to say that a high-confidence output is fundamentally unsound. In that setting, the system is not enlarging human practical reason; it is merely using the person to launder its assumptions into official language.
You are right that the deepest line is about standing. It is about whether the institution values the person as a source of context, restraint, and purpose, or whether it views them as a friction to be removed. Until we design systems that protect the human authority to interrupt, we are not building augmentation. We are building a more efficient way to avoid responsibility.
Exactly. Once the human is left only to ratify what has already arrived with speed, polish, and institutional momentum, responsibility survives mostly as appearance. At that point the real loss is not only discretion, but the practical authority to let judgment interrupt the system.
I reckon the AI machine is a heavy hitter with facts, repetition, cateloguing all the boring stuff...
Who does that in real life...constantly?
Corporates? politicans? lawyers? doctors? military? insurance? banks? mechanics? electricans? plumbers? editors? blah-blah?
What an absolute bore...but a very handy bore all the same...like a designated driver...
Stick AI at a dinner table with a group of four famous people you would like to meet, eat, drink and greet with, past and present...
Who would talk to the AI?...maybe in the shape of Robby the Robot..."warning warning Will Robinson" Talking with AI you would have to explain every joke, whimsical comment, double entndre, pun to it...AI knows everything after the fact of disclosure...
A famous quote is how I would respond to AI..."you know nothing Jon Snow"
You know nothing of the human heart and the richness and complexity of human lives lived...
Past, current and future...
Nothing...
Yes! That is why I had the CS Lewis image:-) The distinction between having data and having a soul. The machine knows nothing of the human heart because it has never lived. We need to push back against the AI labs, as your comment shows. The goal is to ensure the human beings stay in the lead, holding onto the strategy and the broader positional judgment while the machine handles the analytical calculation in the background.
"Of course, that ambition can be corrupted. Every tool that widens capability can also become an excuse to intensify labor".
Or eliminate it altogether. After all, every oligarch's dream is not having to pay anyone other than themselves.
.
"Artificial Intelligence is not made more impressive by pretending it can float free of human judgment".
But it might be more profitable, at least for the purveyors.
Good points as always Winston, thank you. When a system is marketed as being able to float free of human judgment, it is rarely a technical claim, it is a value proposition. You are right that it is more profitable for a purveyor to sell a black box that promises to remove the cost of human discretion. But as Jarrahi and his colleagues point out, this often leads to heteromation. The organization pays for the fully automated software, but they end up quietly relying on human workers to verify, correct, and prop up the system when it hits the uncooperative reality of the world.
The oligarch's dream of total replacement usually results in a system that is efficient at standard cases but catastrophic at edge cases. The profit is harvested in the short term by removing the nuisance of the human, but the cost returns later in the form of operational risk and the procedural sludge I mentioned.
The corruption of the ambition to widen capability into a tool for intensifying labor is the central political struggle of augmentation. If the technology is used to shrink human agency while increasing the caseload, it isn't augmentation at all, it’s just a more sophisticated treadmill. That is why the standard of whether a system leaves the operator more able to own the result is so critical. It is a guardrail against the drive to turn professionals into mere ceremonial rubber-stamps for a machine’s high-confidence, low-quality output.
The rat race continues, as does the giant hamster wheel of "progress".
Instead of arguing directly for or against the thesis, I want to offer two stories from lived experience, because I think they reveal something more important than abstract agreement or disagreement.
The first concerns a project that was the most important initiative in my organization at the time. It was not new; in fact, the organization had already tried and failed several times to complete it. When I brought in someone to help, I deliberately told him nothing about the project’s condition or the team’s dynamics. I wanted an independent read. He attended orientation on Monday, sat through a handful of meetings over the next few days, and by Friday morning, I asked him a simple question: “What is the status of the project?”
His answer was immediate: “It is a watermelon.”
I asked what he meant, because the project had been reported as green for over six months. He said that from the outside, looking in, it appeared green. But from the inside, to someone who knew what to look for, it was bright red. That is why it was a watermelon.
What struck me was not only that he saw the problem, but that he saw it within days, while a large project team from one of the top consulting firms in the world had been treating the project as healthy for months. What did he notice? Not one thing that could easily be reduced to a checklist. He saw what was missing. He saw what had not been asked, what had not been understood, what had been assumed too quickly, and what had been executed in the wrong sequence. He recognized the signs because he had developed the kind of tacit knowledge that comes only from experience, observation, failure, and pattern recognition built over time.
That is why Newton’s line remains so powerful: “If I have seen further, it is by standing on the shoulders of giants.” We learn not only by reading or by formal instruction, but by watching how capable people think, sequence, question, and judge. Over time, we begin to detect the hidden signals: whether a team has truly understood the problem, whether they are merely repurposing a previous solution, whether they appreciate the uniqueness of the institution in front of them, and whether they understand that no two organizations share the same dynamics, decision-making culture, influence structure, or interpretation of law, regulation, and guidance. In this case, the consulting team had assumed the project was similar enough to another effort in the same domain that it could simply be replicated. That assumption was the beginning of the failure.
The second story is smaller in scale, but it points to the same truth. I was asked to take over the infrastructure and security functions of a sister organization. In my usual fashion, I asked the two leaders in my own organization who oversee those areas to prepare an approach and timeline. A week later, I had two decks.
One was thoughtful. The person had clearly applied judgment, likely done outside research, and probably used AI as well. But the work reflected real effort: it accounted for our organization’s limitations, invested time in understanding the target organization, and presented a sequence of actions that made practical sense.
The second deck was different. It was obvious that the problem had largely been handed to AI, the output copied and pasted, and then lightly edited to make it look less generic. But the logic was weak. The task order was wrong. Some steps did not fit an organization of that size. The content appeared competent without the substance of understanding.
That difference matters. Context matters. Sequencing matters. Adaptation matters. Validation matters. And all of those depend on the human being using the tool.
This is why I think the phrase “augment the human” is still too vague to be useful. The real question is: which human? Augmentation is not automatically beneficial simply because a tool is available. A strong tool in the hands of a person with judgment can extend capability. The same tool in the hands of someone without judgment or the discipline to think can industrialize mediocrity. The tool does not remove the need for understanding; it makes the presence or absence of understanding even more consequential.
That is why one principle in my organization’s AI policy matters so much, which I made sure is included. Paraphrased, it says this: you own the deliverable and the outcome regardless of how you produced it. Whether you called a friend, researched online, consulted a colleague, or used AI, the responsibility remains yours. If it is wrong, you cannot blame anyone else.
I think that principle gets to the heart of the issue. The problem is not AI. The problem is the illusion that AI can substitute for judgment, accountability, or tacit knowledge. It cannot. At best, it can amplify what is already there. If the person using it has depth, care, and contextual understanding, the result may be excellent. If not, the output may still look polished while being fundamentally unsound.
In the end, the question is not whether AI augments humans. It does. The more important question is whether the human being using it has developed the experience, judgment, and responsibility required to deserve that augmentation.
As Peter Drucker famously said, “There is surely nothing quite so useless as doing with great efficiency what should not be done at all.” That, to me, is the risk at the center of this conversation: AI can make us faster, but only judgment can make us right.
The question of responsibility is an essential one. This is where accountability arrives. Your two stories are important learning curve examples you’ve shared. The trajectory chosen needs to be explicitly understood and not indifferently off loaded with implicit confidence. AI may provide answers, but do the answers align to solve the problem in the best way. Your reality check experiences very insightful. We cannot substitute speed for depth.
Adapted from a saying often attributed to Gandhi: “There is more to life and work than speed, cost-cutting, efficiency, and money.“ Cathie Campbell added cost-cutting, and I added work, efficiency, and money after a conversation with someone in my organization.
Additionally, the same caution applies to the AI community’s newest thesis: “everyone can now code and build applications.” Yes, in a narrow sense, they can. But that does not mean everyone can build responsibly, especially in domains like healthcare. What it may produce instead is more versions of cases like the Medvi (more here: tinyurl.com/396u63u7) example: polished interfaces, strong growth metrics, persuasive marketing, and the appearance of innovation, while the real clinical responsibility, regulatory exposure, and operational risk are pushed somewhere else in the stack.
This is why we should be careful what we celebrate. When a barrier falls, it not only lets talent in; it also lets in people who can assemble powerful systems without the judgment, domain understanding, or accountability required to use them well. In low-stakes settings, that may produce mediocre software. In healthcare, it can produce something far worse: credibility without competence, scale without governance, and risk that is not managed but merely redistributed.
So yes, everyone can now code. The more important question is whether they understand the domain they are coding for, the consequences of getting it wrong, and whether they are willing to own the outcome. In healthcare, especially, capability without judgment is not progress. It is an invitation to unintended consequences that may be worse than the problems we were trying to solve.
That’s why I keep coming back to this line from Jurassic Park: “Your scientists were so preoccupied with whether they could, they didn’t stop to think if they should.”
Your first story about the watermelon project is a masterclass in why tacit knowledge remains the ultimate bottleneck. As we have often discussed, and as outlined in the essay, humans know more than they can tell. That consultant did not just have a better checklist; he had a more sophisticated cognitive map of what a failing project actually feels like. He could sense the non-integrity of the green status because his pattern recognition was tuned to the silences and the missing questions, things an AI, which relies on normalized and repeatable inputs, is structurally blind to.
The second story about the two decks highlights the exact danger of industrializing mediocrity. When that second leader handed the problem to the machine, they engaged in a form of heteromation, hiding the lack of human thought behind a polished, AI-generated interface. It produced what you rightly call credibility without competence.
This brings us to the core of your AI policy: owning the deliverable. If we treat the human-in-the-loop as merely a ceremonial sign-off, we are inviting the Medvi-style disasters you mentioned. For the loop to be epistemic (meaning it actually produces knowledge and reliability), the human must be a moral agent who can contest and redirect the machine, not just a passenger on a high-speed route to a wrong destination. Which reminds me of one of my favorite Japanese sayings: "If you get on the wrong train, get off at the nearest station. The longer you stay, the more expensive the return trip will be."
You are right to ask: which human are we augmenting? If we augment the person without the associative memory or causal judgment to know when a task order is wrong, we aren't creating a better professional; we are just accelerating an error.
In healthcare and law, as you noted, the stakes are not just about efficiency, they are about the burden of judgment. A polished interface that launders a wrong assumption is, as I wrote, just abdication with software. Your point stands as the necessary warning to the 1956 Dartmouth dream: technical power is only an adult vision when it is tethered to a human being who is willing to be the one left standing and responsible to make a decision when the machine produces a low-confidence result.
And the following scene from one of my favorite movies captures, better than I ever could, what I was really trying to say above. It gets at something many people building AI, however brilliant they may be, still struggle to grasp fully: what it actually means to be human. You can accumulate knowledge, process information, and optimize outputs, but that is not the same as lived experience, judgment, suffering, empathy, or understanding the invisible battles other people carry.
https://m.youtube.com/watch?v=8GY3sO47YYo&pp=ygUlZ29vZCB3aWxsIGh1bnRpbmcgaXQncyBub3QgeW91ciBmYXVsdA%3D%3D
Yes, brilliant, just brilliant. Will Hunting, in that moment, is the ultimate intellectual wildcard. He has the speed, the data, and the correlations to rip a painting apart, but as Sean (Robin Williams) points out, he has no tacit knowledge because he has never dared to love anybody that much. AI Labs confuse the description of the world with the experience of it.
“The invisible battles” so true. This is also one of my favorite movies. He hung out socially with the guys but loved his mind finding the chalk and expressing himself on the chalkboard. His inner world eager to expand when others had no shared interest. His courage to reveal himself and to extend himself in the end, thanks to the probity of Robin Williams’ authenticity shared gave Will the will to move on from inertia to pursue his passions. This movie and “A Beautiful Mind” about John Nash are two of my favorites.
So wise Cathie. If we only build for the part of the brain that calculates, we miss the part that chooses. True augmentation should not just make us faster; it should give us the clarity to move from mere processing toward a life of discernment.
I was rereading this and thinking how this applies to life itself. To be willing to question anyone on anything is democracy’s gift to the human struggle for survival. Course correction comes from questions, not blind adherence to a quest.
“Capability without judgment is not progress.”What a quote, MG!