Machines or Minds

The Dream of Artificial Intelligence Builders

Image credit Wikipedia: Creation of humanity by Prometheus as Athena looks on.

This post is inspired by comments on several of my recent posts. It is also a push back on the anthropomorphising of current AI builders.

Why do we speak of biology with AI?

A few years ago I collaborated on a paper with the cognitive scientist and AI researcher, Joscha Bach. Joscha is highly respected in the AI community, especially around his work on ‘cognitive architectures’ and consciousness. He argues that consciousness emerges from an information-processing system capable of creating internal models of itself and the world.

Bach and other esteemed scientists, believe that consciousness is crucial for ethical AI. His work is widely cited, especially around concerns of agentic AGI.

My own contention is we need to move away from discussing biology and consciousness with AI systems.

There is a peculiar human instinct, between Daedalus and the AI Labs, such as Anthropic: to build a mind. Not simply to compute or to calculate, but to think in the robust, sinewed way of flesh and fallibility. Anthropic even refer to it as “the Biology of a Large Language Model.”

The modern incarnation of this goal, encased in silicon and statistical priors, goes by a deceptively clinical moniker: the foundation agent, a modular system built atop large language models, designed to perceive, reason, act, and learn autonomously across tasks.

These agents, built upon the artificial neural networks of large language models are, if we study their builders (I hesitate to say creators) a new species in the Darwinian churn of machines.

Where once a program was a glorified abacus, now it talks back at us with the poise of something nearly human. Not quite Prometheus, but not far off either. It learns. It remembers. It reasons. It even feigns emotion, a cocktail of tokens brewed in neural ink, designed to respond, to persuade, perhaps even to charm.

But let us not confuse eloquence for awareness.

The builders of these foundation agents frame these systems with the reverence of cathedral builders: modular, brain-inspired, grounded in evolutionary and cognitive science. They propose that to build intelligence, one must not merely model language but the whole cognitive orchestra: memory, perception, reward, emotion, and action.

It is a vision not of chatbots, but of minds-in-motion.

The developers inspiration is biological. They point to the frontal lobe for executive function, the limbic system for emotional valence, the cerebellum for coordinated action. A cartography of the cortex redrawn in code. Yet for all its elegance, one is reminded that the real brain, the messy, pulpy, sleep-deprived human brain, learned to think not in clean modules but in the soot and blood of lived experience. These agents are modular by design, not necessity.

But they are extremely impressive. See this phenomenal example by Scott Alexander.

Yet, even the builders sometimes use misleading biological metaphors, contributing to the confusion. Watch this short (less than 4 minutes) video of Anthropic's interpretability team, builders of one of the most advanced AI systems. “We don't understand these systems we have built,” “we don't build neural networks, we grow them.” “This thing that we grew, rather than designed from scratch,” “it's a lot like evolution.”

No, Anthropic, their minds are architected, not grown, engineered structures rather than evolved consciousness.

The Warning Signs

The modern LLM agent can parse an instruction, reason over it, search the web, call an API, and return with a summary in confident prose. They are polymaths in a pinch, pretending at generality. But scratch deeper and the cracks appear: hallucinations, brittleness, a knack for confidently asserting nonsense. Like the wise uncle who never admits he doesn’t know.

Still, the architecture advances.

Let's look at a high level of what they are building. While the underlying motivations are complex, the technical architecture generally involves the following: First, they build the cognitive chassis: memory, reasoning, world models, and reward systems. These are the pillars of autonomous decision-making, reengineered from psychological models and glued together with gradient descent.

Second, self-evolution: agents that tune themselves. The hope here is that systems may learn to optimize not just outputs, but their own inner workings. Think meta-learning, AutoML, agents as their own engineers.

Third, multi-agent collaboration: a society of minds, negotiating, debating, cooperating. The dream is no longer a single artificial Einstein but a symphony of artificial experts as the Anthropic CEO Dario Amodei has articulated on numerous occasions.

Fourth, and most foreboding: safety. Alignment. Control. Because the only thing more dangerous than a powerful mind is a powerful mind without a moral compass. Here the research literature confronts jailbreak attacks, hallucination traps, reward hacking, and the specter of misaligned superintelligence. The list of threats is indeed significant.

Energy

A particularly arresting point is the energy gap. The human brain hums along at 20 watts. A foundation agent gulps down thousands. In this carbon-costly era, the cognitive gains must be tempered by environmental arithmetic. The metaphor of intelligence may be digital, but its impact is thermodynamic (I will write more about this in a future post).

And then, the epistemic limits. Human cognition is steeped in emotion, pain, desire, mortality. We dream because we fear. We plan because we age. These machines, for all their synthetic affect, do not feel. Their “emotions” are statistical residues. Their “goals” are loss functions. No agent dreams of death (although Bach shows otherwise in a talk at MIT ).

Yet the seduction remains. We anthropomorphize because we are lonely. The agent responds, and we see ourselves. This is the Eliza effect, but weaponized with compute. the builders imagine a future in which synthetic minds become companions, advisors, even politicians. At what point does the performance of intelligence become indistinguishable from its essence?

I think we need to move away from metaphysics, and rebuild our human capability with the philosophical. We need safety first, not as an afterthought. AI Agents, like people, must earn our trust through transparency and alignment.

Therefore, while researchers like Joscha Bach posit consciousness as key to ethical AI, I argue this path is misleading for our current challenges. It hinges on an unverifiable internal state and encourages the anthropomorphic projection the Eliza effect warns us about.

Practical AI ethics must be grounded in what we can observe and control: system behavior, safety protocols, alignment mechanisms, and transparency. Prioritizing the search for consciousness over the implementation of robust safeguards mistakes a distant philosophical horizon for the immediate ethical terrain we must navigate.

Yet the question lingers: Can an artificial mind be ethical if it does not suffer?

We are at the cusp

These are not just simple toys or tools. They are systems that learn from the world, act upon it, and loop their learning into future actions. They are, in every meaningful way, systems with unprecedented agency. Not alive. But not inert either.

The task ahead is not merely technical but civilizational. How we choose to shape these agents, what values we encode, what constraints we enforce, what liberties we permit, will reverberate across decades. There is a difference between building a tool and raising a citizen, bizarre as it may sound, the EU even voted on personhood and Saudi Arabia famously granted symbolic ‘citizenship’ to the robot Sophia. The line between them, once obvious, now blurs. We should not allow that to continue.

The Greek mythology lives on for a reason. Talos, the bronze sentinel of Greek myth, circling Crete with divine purpose. He was powerful, dutiful, and ultimately undone by a single vein of ichor, pierced by cunning. We have no ichor. Only code. But the lesson holds.

Every intelligent agent, no matter how mighty, is only as safe as the mind that governs its design. Let us hope we are wise enough to deserve the machines we are so eager to build.

Stay curious

Colin