Origins February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop
ARIZONA STATE UNIVERSITY Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes
true today to some extent, but the above advances in ML/Al could make the situation substantially worse, in extreme cases perhaps even rendering useful mass communication on the internet untenable.
The rising capabilities can be used in multiple ways in multiple settings with multiple goals. Some uses may be subtle and employed over time to do important but damaging biasing of sentiment about individuals and groups of people. The capabilities can be combined to enable identity theft or identify distortion for destroying the reputation of people and groups. As such, these abilities could enable small groups to wield great power in multiple arenas and for new forms of blackmail, threats, and control.
SUMMARY
Powerful personalized persuasion technologies are positioned to put massive power in the hands of a few and may even manipulate the owners of the technology. Powerful propaganda and persuasion machines threatens to undermine democracy, free availability of information about the state of the world, and, more generally, freedom of thought. Leaders may increasingly depend upon such propaganda optimization systems for attaining and holding power. Over time, even the potential initial owners of such systems might become unaware or unable to control these systems—and may believe the propaganda themselves.
In the longer-term, there is the possibility that one or multiple systems, or distributed coalitions of systems communicating implicitly or explicitly could autonomously persuade, subjugate, and control populations. Pathways to such situations include the side effects of rise in the large-scale use by people of communicating personalized filters that interpret and pool information with the initial intention of grappling with widespread uses of manipulative information.
SAMPLE TRAJECTORY
e ML-based customized advert placement continues to prove highly successful, generating revenues for large online companies
e Profits from online content (online newspapers behind paywalls, charitable contributions to information sources e.g. Wikipedia) stagnate or decline
e Anincreasing number of information sources enter into collaborations with media brokers who offer "content customization" in exchange for ad-revenue sharing
e Poor oversight of content personalization outcomes (there are, after all, billions of ad versions being shown, and updated on an hourly basis), means that for some ad content (political parties, pharmaceuticals) for some minority of target audiences (especially less privileged) the effect is very harmful.
KEY POINTS e New directions with generation of provocative, believable content, hacking of identity e Algorithmic manipulation of data to optimize desired behavior regardless of content
HOUSE_OVERSIGHT_011288