The Birth, Adolescence, and Now Awkward Teen Years of AI

The past, present and future of AI

The story of Artificial Intelligence is the story of human aspiration, and equally, of human hubris. You see, we like to think of ourselves as capable of building anything, especially something that can think. Or replicate the human brain. And maybe, just maybe, we can. But before we leap onto the altar of technological worship, it might be prudent to ask, what exactly are we building, and for whose benefit?

AI, much like us, has had its phases. A quaint infancy in the 1940s and 1950s, marked by aspirations of reasoning like a human, a rebellious adolescence in the 1980s with statistical flamboyance, and now, a kind of awkward teenage dominance filled with deep learning and generative models, the AI equivalent of growing taller than all its parents but still unsure what to do with its lanky limbs (e.g. Trurl’s Machine). The road to this point has been paved with bold promises, from machines playing games to agents generating complex visual art, and yet, alongside every breakthrough has been a series of troubling questions left unanswered.

Where did this journey begin? Let’s time-travel to the 1940’s where a young Alan Turing’s ideas of computability merged with the Martian brain of John von Neumann, then we move quickly to the summer of 1956 at Dartmouth College, and the more pragmatic proposals of John McCarthy and Marvin Minsky created a rallying cry for a new kind of science. They called it artificial intelligence. Here were the big questions, how do you make machines reason, plan, understand language, learn? Could machines one day even reflect?

Mimicking the Brain

Fast-forward through history, and we find ourselves grappling with this peculiar divide, the logical and the connectionist. For decades, AI models stuck to symbolic reasoning, crafting meticulous rule-based expert systems that were, if we’re being honest, the logical equivalent of an obsessive-compulsive librarian armed with a lot of if-then bookshelves. The trouble was, these systems were rigid. In came neural networks with their connectionist approach, promising to mimic brains and do away with handcrafted rules.

Did they solve the problems? Partly, but also raised a few more. Take neural networks, today foundational to “deep learning,” which were once seen as merely one tool in the kit. The story of neural nets becoming a roaring superstar has less to do with method alone and more to do with economics, computation got cheaper, data more abundant, and suddenly, statistical brute force became the preferred strategy. If logic-based AI was an old-school professor in a tweed jacket, today’s neural models are more like weight-lifting teenagers in branded athleisure, flaunting their capacity to flex, albeit with the slight risk of lacking an actual sense of nuance.

Yet, here's the strange twist, for every loud roar of achievement (think AlphaGo, GPT-4, AlphaFold[1]), there are whispers of frailty. These models, no matter how many parameters they boast, can stumble when faced with nuance. They can’t reason beyond the boundaries of statistical correlations. Can they genuinely understand? Can they infer from first principles? When tasked with generating a text, a picture, or an insight, are they merely performing a magic trick, or, as it appears, approximating the complex nuance of human-like creativity?

It is also worth acknowledging the breakthroughs these models have enabled, especially in natural language processing (NLP) and computer vision. They have achieved remarkable feats,transforming language understanding, generating convincing text, and powering real-time translation. In computer vision, deep learning has revolutionized how machines interpret and categorize visual information, pushing boundaries in fields from healthcare imaging to autonomous driving. The reliance on scale has undeniably led to these significant advancements, even if it comes at the cost of an overemphasis on brute-force approaches.

So we come to the current inflection point. Generative AI, the darling of 2023 and beyond, has powers that surprise even its creators. How is it that a Transformer model, which merely trained itself to predict the next word in a text, suddenly demonstrates abstraction, generalization, and the cheekiness to create rhyming mathematical proofs? It’s uncanny, but the answer seems to lie in scaling laws, bigger models, more data, more compute. Growth that is less a breakthrough in intelligence than a brute increase in capacity. It raises a real conundrum, if size is what matters, are we mistaking breadth for depth?

Align and Contain

As we ponder generative AI's role, we must remember, for every application that astonishes us, like predicting protein structures or designing new proteins with RFdiffusion (yes, we are in science-fiction territory here, but real!), there is an ethical parallel in question. Where does control end and chaos begin? Today, your AI might design a better medicine, tomorrow it might mistakenly design a bioweapon, or just confuse its facts enough to become misleading in healthcare. This is the alignment or containment problem! Are we limited to asking it nicely to be good, and poking around individual nodes of its neural network to guess which ones are harmful or dishonest? Do we have built-in loss functions or training data to steer toward true alignment?

The ethical concerns extend beyond healthcare and science. AI algorithms, while powerful, are prone to bias, reflecting the prejudices embedded in the data they are trained on. This has significant implications for decision-making in areas like hiring, law enforcement, and lending, where biased outputs can lead to real-world discrimination and perpetuate inequalities. There is also the looming concern of job displacement, as AI systems become more capable, industries ranging from manufacturing to customer service face significant shifts in workforce needs. The automation of repetitive tasks promises efficiency but comes with societal challenges that need addressing, such as how to retrain and support workers displaced by these advances.

Moreover, the potential for misuse in surveillance and warfare cannot be ignored. AI's power to analyze vast amounts of data in real time makes it an attractive tool for mass surveillance, which, in the wrong hands, could lead to the erosion of privacy and civil liberties. The militarization of AI also poses ethical dilemmas, as autonomous weapons and decision-making systems raise the stakes in warfare, potentially reducing human oversight in life-and-death situations.

What not How

And here lies the dillema, not in whether we should build, but in how we should proceed. It’s a classic first-principles quandary, if we want these technologies to ‘integrate’ with us meaningfully, we have to ask ourselves not just what they can do, but what they should do. Should an AI be persuasive without being right? Should it act as an expert on any field with its “confidence curve,” even when occasionally the emperor is wearing no clothes?

To chart a better path forward, we should shift from seeking marvels of what machines can do, to a foundation of what societies need, in healthcare, education, and scientific discovery, the latter is so wonderfully explained by DeepMind researchers here. Our approach, bizzarely, might require a dose of humility, a return to that teenage awkwardness of AI, one willing to acknowledge its imperfections, one that involves people in its calibration. Imagine AI, not as an oracle, but a student constantly open to the shaping influence of educators, clinicians, policymakers, each turning a knob to calibrate and correct.

Generative AI might be a pivot point, but it should be a pivot toward collective intelligence. Tools, no matter how advanced, are still tools, and in the end, they should bend to the needs of society. Not the other way around.

Stay curious

Colin

---

[1] See I told you they were weight-lifting teenagers

Image generated with ChatGPT