When it comes to real robots helping real people, the standard definition of AI fails us, for two fundamental reasons: First, optimizing the robot’s reward function in isolation is different from optimizing it when the robot acts around people, because people take actions too. We make decisions in service of our own interests, and these decisions dictate what actions we execute. Moreover, we reason about the robot—that is, we respond to what we think it’s doing or will do and what we think its capabilities are. Whatever actions the robot decides on need to mesh well with ours. This is the coordination problem.
Second, it is ultimately a human who determines what the robot’s reward function should be in the first place. And they are meant to incentivize robot behavior that matches what the end-user wants, what the designer wants, or what society as a whole wants. I believe that capable robots that go beyond very narrowly defined tasks will need to understand this to achieve compatibility with humans. This is the va/ue-alignment problem.
The Coordination Problem: People are more than objects in the environment.
When we design robots for a particular task, it’s tempting to abstract people away. A robotic personal assistant, for example, needs to know how to move to pick up objects, so we define that problem in isolation from the people for whom the robot is picking these objects up. Still, as the robot moves around, we don’t want it bumping into anything, and that includes people, so we might include the physical location of the person in the definition of the robot’s state. Same for cars: We don’t want them colliding with other cars, so we enable them to track the positions of those other cars and assume that they’ ll be moving consistently in the same direction in the future. A human being, in this sense, is no different to a robot from a ball rolling on a flat surface. The ball will behave in the next few seconds the same way it behaved in the past few; it keeps rolling in the same direction at roughly the same speed. This is of course nothing like real human behavior, but such simplification enables many robots to succeed in their tasks and, for the most part, stay out of people’s way. A robot in your house, for example, might see you coming down the hall, move aside to let you pass, and resume its task once you’ve gone by.
As robots have become more capable, though, treating people as consistently moving obstacles is starting to fall short. A human driver switching lanes won’t continue in the same direction but will move straight ahead once they’ ve made the lane change. When you reach for something, you often reach around other objects and stop when you get to the one you want. When you walk down a hallway, you have a destination in mind: You might take a right into the bedroom or a left into the living room. Relying on the assumption that we’re no different from a rolling ball leads to inefficiency when the robot stays out of the way if it doesn’t need to, and it can imperil the robot when the person’s behavior changes. Even just to stay out of the way, robots have to be somewhat accurate at anticipating human actions. And, unlike the rolling ball, what people will do depends on what they decide to do. So to anticipate human actions, robots need to start understanding human decision making. And that doesn’t mean assuming that human behavior is perfectly optimal; that might be enough for a chess- or Go-playing robot, but in the real world, people’s decisions are less predictable than the optimal move in a board game.
99
HOUSE_OVERSIGHT_016902