The Singularity and Machine Ethics
Luke Muehlhauser and Louie Helm, Singularity Institute of Artificial Intelligence
Many researchers have argued that a self-improving artificial intelligence (AI) could become so vastly more powerful than humans that we would not be able to stop it from achieving its goals. If so, and if the AI’s goals differ from ours, then this could be disastrous for humans. One proposed solution is to program the AI’s goal system to want what we want before the AI self-improves beyond our capacity to control it. Unfortunately, it is difficult to specify what we want. After a brief digression concerning human intuitions about intelligence, we offer a series of "intuition pumps" in moral philosophy for our conclusion that human values are complex and difficult to specify. We then survey the evidence from the psychology of motivation, moral psychology, and neuroeconomics that supports our position. We conclude by recommending ideal preference theories of value as a promising approach for developing a machine ethics suitable for navigating the Singularity.
What are these proposed "intuition pumps"? At first reading, I understood them to be a mechanism by which the AI's morality can evolve.
ReplyDeleteAnd if that's not what you meant, I think it IS what you should mean! What a cool idea!