Esa Origins 7 February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop Challenges of Artificial Intelligence:
Envisioning and Addressing Adverse Outcomes
ARIZONA STATE UNIVERSITY
colleague and drastically different from the one seen by a member of another nation state, or a supporter of a different political party, or someone ina different consumer profile category.
Al ATTACKS ON SOURCES AND IDENTITY
Messaging and persuasion promises to be amplified by the use of simulated yet believable, realistic, yet synthetic audio, photos, and even video that make believable, persuasive content to the next level. Beyond influencing citizens and affecting democracy, such content, including false signaling, can be injected in sequences with careful timing so as to influence leaders (or machines themselves over time) to create crises, or even escalations to frank warfare. So, messaging and persuasion promises to be assisted and amplified by the use of simulated yet believable, realistic, yet synthetic content, audio, photos, and even video that make believable, persuasive content to the next level. Over the several decades, extrapolations of research we see today lead to the following:
e Generative models that produce audio or video of anyone saying anything. There is already substantial work on “style transfer” as well as photorealistic generative models in many domains. Speech synthesis is becoming similarly competent. It is inevitable that we will be able to make synthetic video and audio that is completely indistinguishable from the real thing.
e Generative models that produce coherent text content that appears as if has been written by a human. Such generative content will be able to appear if the content was written by a particular person. For example, in 2030 it will likely to possible for anyone to write a 4 paragraph email that reads like it was written by your close friend.
e Adaptive botnets, worms, or viruses that use modern machine learning techniques to learn and adapt. Viruses and botnets already cause a huge amount of damage by just copying code across many computers. If they had the ability to design and experiment with new attack strategies, and communicate what they learn to other copies, defending against them could become even more difficult. Similarly ML could be used to make DDoS attacks more effective.
e Automated analysis of software vulnerabilities. People are already using ML to try to detect vulnerabilities (for the purpose of defending against them) -- it is only a matter of time before they start being used for attack (if they aren’t being so used already).
The above capabilities, together with similar powers of synthesis that we are likely to develop in the next 15 years, could potentially combine to make the internet much more vulnerable to attack at much lower cost, and by a wider set of people, than ever before. The first two capabilities would seem to make it much easier to launch automated social engineering attacks with much higher success rates than e.g. current spam email and phishing attacks, while the second two capabilities might make technical attacks much more effective.
Combined, all of these capabilities could conspire to create an internet ecosystem where it is very
difficult to trust the communication that you receive and very easy to intercept, spoof, steal, or alter communication, as well as to improperly gain control of internet resources. This is obviously already
HOUSE_OVERSIGHT_011287