ways we don’t understand and certainly can’t follow in real time, we face a problem: We don’t know what to tell the machine not to do. So many of the things we’d hope to teach it - be compassionate, fight for liberty, follow a moral code - far transcend what might be achieved by us in math. We haven’t after all, even solved the problem of how program ourselves reliably with these values.
If Bostrom’s paper clip machine seems fantastic, it is easy enough to conjure other and more real dangers lingering at the edge of disappeared human control. Think of health care. To begin, you need to know about an important “game” from the world of research into how humans interact with each other known as the “The Ultimatum Problem.” It runs like this: I tell you that you can have a million dollars, but you have to split it with someone else. How you split it is up to you, but if your partner rejects the formula you propose, neither of you gets a cent. Offer to split the pot with a dollar to your pal and the rest to you. Insulting. But where to settle? You might expect that the smartest offer would be a 50/50 split, but humans are greedy. You want more and can probably get it; your partner does not want to end up with zero. Generally when scientists shake this cocktail of greed and fear they find an offer of $300,000 is nearly always accepted. However, there’s a surprising way to change the outcome: Match the human against a computer in the negotiation. A pal suggesting an 80/20 split to a friend will be rejected. Too greedy. But a computer? Somehow the impersonality, the beeping digital charmlessness of the machine lures biological players to compromise. An offer of $200,000 is usually happily accepted.
It may be, scientists think, that our competitive instinct is muted when we interact with a machine. But researchers have also discovered they can manipulate the split other ways: Sad movies, war chants, hard rock - each bends the emotions of players and changes the result. Increased testosterone produces less compromise. Players primed with family pictures or made to play the game facing a mirror show a warm humanity and a more even split. So imagine this research married to machine- human interaction: A computer has been assigned to review the medical options for your failing liver. It decides that it makes no sense to give you a new one. So it spends the weeks before it delivers this news using its Al to show you certain photos, to play you music it knows is likely to soften you up a bit, generally to manipulate you. It runs off-the-shelf language-analysis neural webs being used today to eavesdrop on customer support calls to track the way you speak to determine what each sentence might reveal about your sophistication.2®” Then it tells you something you'd never accept so easily from a doctor: No liver. Sorry. @. Here’s a machine optimizing not for paperclips - which we could care less about - but for a public good most of us support: More efficient health care. And murdering you in the process.
Optimize Health Care Spending. Just where might such an algorithmic command lead, exactly. Over time, a health-care optimizing AI will surely discover that the greatest risk to human health is humans: Smoking, couch-sitting, driving. Might it begin to
267 It runs: Language Use, Customer Personality, and the Customer Journey (Scott Nowson, Global Innovation Lead, Xerox)
195
HOUSE_OVERSIGHT_018427