The Architect of Tomorrow
The Architect of Tomorrow
John von Neumann and the Dawn of Artificial Intelligence
This is part of a series, of people, who have contributed significantly to their field. Whilst they are known within the field, rarely are they known outside the field or widely, yet should be.
I ‘met’ John van Neumann thirty-seven years ago whilst sitting on a wooden bench with a view towards the magnificent Bamburgh Castle, which had been a casualty of war as a witness to the War of the Roses (in 1464). Bamburgh Castle is set in a sublime landscape of golden sand dunes and the cold foreboding North Sea in a picturesque part of the east coast of England, near to the border with Scotland. I was reading The Making of the Atomic Bomb, the Pulitzer Prize winning book by Richard Rhodes. I was especially drawn to von Neumann, because Rhodes, in his 886-page encyclopaedic history of the atomic bomb only mentioned him fleetingly and even then, a few perfunctory times, yet it seemed to me, from these passing sentences that von Neumann’s work was decisive to the implosion of the bombs which wrought havoc over those two cities in Japan and ended the Second World War.
Not long after reading Rhodes epic book I happened across another Pulitzer Prize winning masterpiece; Gödel, Escher, Bach by the cognitive scientist (AI pioneer and a personal mentor) Douglas Hofstadter, to my surprise, dismay and utter bafflement, Hofstadter did not mention John von Neumann once. I was previously told that ‘I see things others do not see’, and began a rigorous study of von Neumann, reading, listening to, and watching anything that I could find on his life and work and seeking out people that knew him. What piqued my interest most was the role he played in the history of computing (hardware and software), cognitive science, artificial systems and human psychology. From my earliest research it soon began to become clear that John von Neumann’s contributions to the fields of mathematics, computer science, biology, and physics are monumental, yet underrepresented. Understanding the reasons became a lifetime struggle, one that underscores my now developed belief that the Artificial Intelligent machines we are building replicates human psychology, and John von Neumann played a pivotal role in the foundation of the field of AI and predicated where it could lead.
Discovering the life story of von Neumann is mesmerizing and disturbing at the same time. To those that new him, and many were the most extraordinary minds of the 20th century, including celebrated Nobel Prize winners, mathematicians and physicists, such as Einstein, Teller, Feynman, Bohr, Wigner, Wiener, Oppenheimer, Fermi, Turing, he was considered the most gifted amongst them. The Nobel-winning physicist and mathematician Eugene Wigner said of von Neumann:
“I have known a great many intelligent people in my life. I knew Planck, von Laue and Heisenberg. Paul Dirac was my brother-in-law; Leo Szilard and Edward Teller have been among my closest friends; and Albert Einstein was a good friend, too. But none of them had a mind as quick and acute as Jansci von Neumann. I have often remarked this in the presence of those men, and no one ever disputed me.”
During the 1930s and early 1950s, a time of considerable innovation breakthroughs, the most haunting revelation was not merely that John von Neumann rationally and logically envisioned every possible future, connected to computing machines, climate change, energy and war, but that he pursued the creation of radical, and indeed ominous, ideas and technological change with the fervor of an enthusiast and the eerie calm of a prophet. John von Neumann was capable of all sorts of remarkable things, his diverse developments led to world changing solutions and systems; the creation of the modern computer, with a specific focus on mimicking the human memory and thought process, he understood and identified aspects of messenger RNA, long before they were ‘discovered.’ Sydney Brenner, the Nobel prize winning molecular biologist who hypothesized and then proved the existence of messenger RNA reports with wonder and astonishment that von Neumann had earlier understood from theory alone how any such system must work. John von Neumann unearthed economic utility and the theory of games, and yet he was also in charge of the calculations for nuclear detonation with an assertion to cause the widest possible destruction over Japan and was a proponent of using the H-Bomb to scorch the earth of the Soviet Union at the start of the cold war era.
The Singularity
The first question, with which one is confronted with the genius of John von Neumann, is the elementary question, where to begin the story. Much has been written about his early years as a child prodigy and those formative years will have played an important role in shaping his own logical, or rational thinking – or the psychology behind the human.
According to Johnny’s brother: "non omnis moriar" (not all of me will die) the edict from Horace, was always one of his family’s “fundamental philosophy of life, adopted and reinforced as a guiding principle.”
About ten years after first reading Rhodes book, I had the good fortune of listening to interviews conducted by Rhodes with various scientists who worked on the Manhattan Project. Nicholas Metropolis recalls collaborating with Johhny who he described as:
“Very, very quick, I mean, you have no idea how quickly he would infer things and extrapolate them.” “…very quick and very sharp. He was a universalist.”
From the mid 1940s to his untimely death in 1957, von Neumann articulated ideas that would prove to be unnervingly farsighted. He began a quest to build machines that could initially emulate the human brain, as summarized by his incredibly gifted wife Klara von Neumann, herself a pioneer in computing and acknowledged as one of the very first software coders. Klara wrote:
“To design the machine, Johnny and his co-workers tried to imitate some of the known operations of the live brain. This is the aspect which led him to study neurology, to seek out men in the fields of neurology and psychiatry, to attend meetings on these subjects, and, eventually, to give lectures to groups on the possibilities of copying an extremely simplified model of the living brain for man-made machines.” (~ Klara von Neumann, wife of John von Neumann (Preface to von Neumann 1958, p.viii)).
He went even further and outlined a future where machines could not only match but potentially surpass human intelligence, a concept now recognized as artificial general intelligence (AGI). Johnny (as he was known to his friends) posited that the development of such machines could lead to a ‘singularity’, a point beyond which human affairs, as we know them, could not continue.
According to the esteemed computer pioneer, cryptologist and mathematician Irving J. Good, Johnny’s thoughts on the accelerating progress of technology and its transformative potential, suggested that it would “approach some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”
Similarly, in his obituary for John von Neumann, the pioneering computer scientist and mathematician Stanislaw Ulam also reflected on conversations with von Neumann, noting their discussions about
The ever-accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.
This concept of the technological singularity, later popularized by Vernor Vinge and more recently Ray Kurzweil, and now widely discussed in AI circles, was revolutionary for its time. Von Neumann was essentially describing the potential for an intelligence explosion, a rapid, runaway increase in machine intelligence that could fundamentally alter human civilization. Von Neumann's notion of a technological singularity was a point beyond which technological progress becomes uncontrollable and irreversible which suggests a fundamental shift in the nature of human existence.
Calculating Machines
Von Neumann’s ability to think logically and dispassionately about scenarios involving massive destruction highlighted both his brilliance and the darker aspects of his pragmatic approach to the development and use of technology. His work on the Manhattan Project was pivotal; von Neumann’s mathematical genius was crucial in developing the implosion method used in the plutonium bomb dropped on Nagasaki. He made significant contributions to the explosive lenses used to compress the plutonium core to a supercritical state. His work was not just theoretical; von Neumann was deeply involved in the practical aspects of bomb design and testing. In 1944, while working on the Manhattan Project at Los Alamos, von Neumann encountered a complex implosion calculation problem that was crucial for the development of the atomic bomb. According to Nicholas Metropolis; “Johnny’s great contribution was his role in seeing that the implosion methods would work very well.”
The manual calculations were painfully slow, taking weeks to complete. Frustrated by this bottleneck, von Neumann learned about the ENIAC (Electronic Numerical Integrator and Computer), which was being developed at the University of Pennsylvania. Although the machine was not yet fully operational, von Neumann immediately grasped its potential. During his visit to the facility, von Neumann quickly absorbed the structure of ENIAC, which was not unusual given his immense intellect and familiarity with complex systems. (Note - George Dyson’s fabulous book Turing’s Cathedral challenges computing's creation myth by highlighting the key role played by John von Neumann).
He later returned to Los Alamos and, with remarkable memory recall, devised a set of coding instructions for the nuclear implosion problem. Months later, when the ENIAC was finally completed, von Neumann’s code was fed into the machine. To the astonishment of the operators, it ran successfully on the first try, solving in hours a problem that had previously taken weeks. Convinced of ENIACs power, von Neumann helped persuade others in government, academia, and industry that digital computers would significantly accelerate the solution of many difficult, unsolved scientific problems.
This feat not only accelerated the Manhattan Project but also demonstrated the revolutionary potential of electronic computing. It showcased von Neumann's unique ability to bridge theoretical concepts and practical applications, a talent that would prove crucial in his later work on computer architecture and artificial intelligence. Indeed, the von Neumann Architecture, known for its simplicity, flexibility and legacy computing, is still used by the majority of computers in the 21st century[1].
Von Neumann’s major achievement was shifting away from the constraints of “vacuum tubes, wiring, punch cards, and magnetic core memory,” toward a more abstract understanding of computing. This high-level perspective allowed him to recognize intriguing parallels between computers and the human brain.
Impact
Von Neumann’s intellectual pursuits and groundbreaking publications, spanned an astonishing array of fields, including quantum mechanics, mathematics, computers, physics, economics, and beyond. As one example his 1928/1932 book, Mathematical Foundations of Quantum Mechanics is still required reading after almost 100 years. The foundations for using game theory in economics is another example of his brilliance and foresight, which he introduced, in his influential book Theory of Games and Economic Behavior (1944), co-authored with Oskar Morgenstern, where they outline fundamental advances in the history of many fields, including; decision, economics and cognitive sciences. His contributions in each of these scientific disciplines coupled with his work for the US government, military, the RAND corporation and bellweather companies such as IBM, were so transformative and percipient that they could individually constitute a lifetime’s work for any one esteemed scientist.
Nuclear weapons
In the late 1940s and early 1950s, as part of the Atomic Energy Commission’s General Advisory Committee, he suggested dropping atomic bombs on Soviet cities before they could develop their own nuclear arsenal, a logic that was rooted in his cold and calculated analysis of geopolitical realities.
One chilling story highlights von Neumann’s attitude towards nuclear weapons. At a meeting, when discussing the destructive power of hydrogen bombs, he suggested using them to create a “scorched earth” barrier, effectively a radioactive wasteland that would deter any Soviet advance. This proposal, while allegedly not seriously considered for implementation, demonstrated his readiness to consider extreme measures.
Von Neumann’s detached and clinical view of these devastating weapons is encapsulated in a remark he made about the need for absolute power to ensure peace: “If you say why not bomb them tomorrow, I say, why not today? If you say at five o’clock, I say why not one o’clock?” He believed that a nuclear war with the Soviet Union was inevitable and argued that striking first could be strategically advantageous. As a witness to the atrocities of two world wars, von Neumann was rationally convinced only a nuclear blast could prevent future nuclear wars.
Weather control
Von Neumann's prescient vision extended even to the realm of weather control, a concept that seems lifted from science fiction. In the early 1950s, at the height of the Cold War, he chaired a top-secret committee for the US government codenamed "Project Cirrus." The goal was nothing short of altering weather patterns for military advantage.
In October 1947, the project attempted to modify a hurricane off the coast of Jacksonville, Florida. The team seeded the hurricane with dry ice, hoping to disrupt its structure. To everyone's shock, instead of dissipating, the hurricane abruptly changed course and slammed into Savannah, Georgia, causing significant damage. This audacious experiment, while ethically questionable, showcased von Neumann's willingness to push the boundaries of science and technology. It also highlighted his deep understanding of complex systems, from weather patterns to computational models. The incident remained classified for years, only coming to light in the 1960s.
Others may profess that von Neumann was not the first to devise of the modern-day computer or be an instigator of machine intelligence or indeed stake a claim to foreshadowing the field of Artificial Intelligence and machine learning, but that may be in name and theory alone. As I embarked on my quest to understand the origins of Artificial Intelligence the contribution of Johhny von Neumann came up time and time again, albeit his sizable contribution and foresight is considerably overlooked, this is exemplified by Stigler’s Law as expressed by Mark Twain “It takes a thousand men to invent a telegraph, or a steam engine, or a phonograph, or a photograph, or a telephone or any other important thing, and the last man gets the credit and we forget the others.”
Johnny may not have named the discipline ‘Artificial Intelligence’ but on September 20, 1948, six years before the naming of the field by his former Princeton colleague John McCarthy in 1956, von Neumann gave a talk at the Hixon Symposium, which he had co-organized with Norbert Weiner, Warren McCulloch and Claude Shannon in Pasadena California. In a published paper of his talk, The General and Logical Theory of Automata Johhny outlines his name for the construct “Artificial Automata” and “Artificial Systems”. A term which later influenced the naming of Artificial Intelligence. His work as documented in his posthumous book Theory of self-reproducing Automata, should also be widely read by AI developers.
In his introduction to the third edition of Johnny’s book The Computer and the Brain, Ray Kurzweil, himself a child prodigy and celebrated futurist, Google chief scientist, inventor, and author of several books on AI and the Singularity referred to Johnny’s work as:
“The earliest serious examination of the relationship between our thinking and the computer”.
Digital Revolution
In John Brockman’s highly readable book of essays Possible Minds: 25 ways of Looking at AI, Seth Lloyd quotes von Neumann as saying that “technological progress will become incomprehensively rapid and complicated.” Von Neumann’s work laid the groundwork for the digital revolution, a foundation upon which modern theorists such as Jean Baudrillard would later build their critiques of hyperreality and simulation. Johnny grappled with the implications of a world increasingly mediated by technology, a world where the lines between the real and the artificial blur, and where human affairs are fundamentally altered by the machines we create.
Peter Lax who worked closely with Johnny sums up what I believe to be true:
“He was the most remarkable man. I’m always utterly surprised that his name is not common, household.
It is a name that should be known to every American, in fact, every person in the world, just as the name of Einstein is. I am always utterly surprised how come he’s almost totally unknown. All people who had met him and interacted with him realized that his brain was more powerful than anyone’s they have ever encountered. I remember Hans Bethe even said, only half in jest, that von Neumann’s brain was a new development of the human brain. Only a slight exaggeration. People today have a hard time to imagine how brilliant von Neumann was.”
There is much more to tell, including his role in the foundations of computing and my own specific interest, artificial intelligence. His life story is truly intriguing and yet baffling and indeed ended way too soon.
Stay curious
dr Colin W.P. Lewis
This is part of a series, of people, who have contributed significantly to their field. Whilst they are known within the field, rarely are they known outside the field or widely, yet should be. Others in the series are Alonzo Church Benoît Mandelbrot, Eric Kandel, Hermann Ebbinghaus. Many more to follow.
Recommended reading on his thinking and biography:
The Martian’s Daughter: A Memoir, by Marina von Neumann-Whitman
John von Neumann and the Origins of Modern Computing by William Aspray
John von Neumann: The Giga Brain by dr Leo Lexican
The Computer and the Brain - John von Neumann
Turing’s Cathedral by George Dyson
John Von Neumann and Norbert Weiner. From Mathematics to the Technologies of Life and Death. by Steve Joshua Heims
The Man from the Future: The Visionary Life of John von Neumann by Bhattacharya Ananyo
Can we Survive Technology by John von Neumann
[1] The von Neumann architecture remains central to modern computing, but it has been modified with various techniques to overcome its limitations. Most general-purpose computers, especially consumer devices like laptops, desktops, and many servers, still rely on this architecture, but it’s augmented with multi-core processing, caching, and sometimes elements of parallel computing to meet modern performance demands.