This geographical division of authority made logical sense when most of the actors were humans who spent their lives within a single nation state, but now that the actors of importance include geographically distributed hybrid intelligences such as multinational corporations, that logic is less obvious. Today we live in a complex transitional period, when distributed superintelligences still largely rely on the nation states to settle the arguments arising among them. Often, those arguments are resolved differently in different jurisdictions. It is becoming more difficult even to assign individual humans to nation states: International travelers living and working outside their native country, refugees, and immigrants (documented and not) are still dealt with as awkward exceptions. Superintelligences built purely of information technology will prove even more awkward for the territorial system of authority, since there is no reason why they need to be tied to physical resources in a single country—or even to any particular physical resources at all. An artificial intelligence might well exist “in the cloud” rather than at any physical location.
I can imagine at least four scenarios for how machine superintelligences will relate to hybrid superintelligences.
In one obvious scenario, multiple machine intelligences will ultimately be controlled by, and allied with, individual nation states. In this state/AI scenario, one can envision American and Chinese super-AIs wrestling each other for resources on behalf of their state. In some sense, these AIs would be citizens of their nation state in the way that many commercial corporations often act as “corporate citizens” today. In this scenario, the host nation states would presumably give the machine superintelligences the resources they needed to work for the state’s advantage. Or, to the degree that the superintelligences can influence their state governments, they will presumably do so to enhance their own power, for instance by garnering a larger share of the state’s resources. Nation states’ Als might not want competing Als to grow up within their jurisdiction. In this scenario, the superintelligences become an extension of the state, and vice versa.
The state/AI scenario seems plausible, but it is not our current course. Our most powerful and rapidly improving artificial intelligences are controlled by for-profit corporations. This is the corporate/AI scenario, in which the balance of power between nation states and corporations becomes inverted. Today, the most powerful and intelligent collections of machines are probably owned by Google, but companies like Amazon, Baidu, Microsoft, Facebook, Apple, and IBM may not be far behind. These companies all see a business imperative to build artificial intelligences of their own. It is easy to imagine a future in which corporations independently build their own machine intelligences, protected within firewalls preventing the machines from taking advantage of one another’s knowledge. These machines will be designed to have goals aligned with those of the corporation. If this alignment is effective, nation states may continue to lag behind in developing their own artificial-intelligence capability and instead depend on their “corporate citizens” to do it for them. To the extent that corporations successfully control the goals, they will become more powerful and autonomous than nation states.
Another scenario, perhaps the one people fear the most, is that artificial intelligences will not be aligned with either humans or hybrid superintelligences but will act solely in their own interest. They might even merge into a single machine superintelligence, since there may be no technical requirement for machine intelligences to maintain distinct identities. The attitude of a self-interested super-AI toward hybrid
123
HOUSE_OVERSIGHT_016343