was a machine that could think in ways a human could never understand, let alone achieve. For such a device, to pass Turing’s Test or slip past a Voight-Kampff check machine will be trivial. In his 1950 paper, Turing sensed the possibility of this development - and the crisis that might ensue. Could man handle the crushing sensation that a device was outperforming him? Perhaps dramatically. “We like to believe that Man is in some subtle way superior to the rest of creation,” Turing wrote. “It is best if he can be shown to be necessarily superior, for then there is no danger of him losing his commanding position.” What would have surprised Turing, I suppose, is the speed at which we've acquiesced - and even accelerated - this very loss of our dominance.
Imagine a device poking at the very origins of the universe at a speed of trillions of calculations a second, spinning past Newton’s and Einstein’s laws and into a realm of physics apprehensible only inside its own electronic consciousness. Compacting time - centuries of human scientific labor reproduced in moments - before shooting far ahead, alone to a subtle knowledge we can only envy. Such a machine would not, as Newton had, stand on the shoulders of giants so much as it would muscle its own, unique way ahead. The AI would have disappeared, but to a place very different than where Maes’ AI had gone. Hers had been erased by human design. This new, really “thinking” AI would slip to invisibility because of its own light-speed cognition. It would think itself out of our understanding. No human could follow, limited as we are by our wet, slow, decaying biological software. Humans and computers, after all, deal with information differently. Think of how poor your memory is compared with the perfect fidelity of a machine, or the way people can even “remember” events that never happened. The machines would have more than knowledge, then. They would linger close to a possessing a profound and inscrutable wisdom. They would inhabit an honestly miraculous gateland that no human would ever enter. And this is where the problems would begin.
2.
We've now passed the moment when humans completely train the very best machines. The AI devices can teach themselves, now. Of course there are still decades of adjustment, of leaps in hardware and programming ideas to eliminate the seams between our minds and the fused ideas of a digital system. But the humans in the loop of the best of these designs, the hard working and well meaning geniuses of the New Caste, are as much trainers as engineers now. They resemble shipbuilders of an earlier era, preparing vessels for voyages to lands they will never themselves see. Their digital minds - technically machine learning structures such as deep neural networks - run quickly to the frontiers. Every moment, all around us, machines are educating themselves. About the world. About themselves. About us too. Tuned AI plays games, looks at photos, studies chemical reactions, reads your email and watches you drive - and then it tries to unspool just what is going on, before winding it up into a new instinct. Machine learning systems already produce mathematical proofs that linger beyond what a human mind can understand; next may be a machine-to-machine mathematics that expands the dimensions of thought. (In aman and machine fusion that surely would have unnerved Joseph Weizenbaum
189
HOUSE_OVERSIGHT_018421