Utility counterfeitingUtility counterfeiting
TranscriptHi, I'm Tim Tyler, and this is a video about utility counterfeiting.Utility counterfeiting is an umbrella category that refers to problems with generating fake utility such as the wirehead problem and the pornography problem in machine intelligence.
The wirehead problemTo recap, the wirehead problem involves an agent stimulating its own pleasure centres or performing some kind of direct synthesis of utility.
The pornography problemBy contrast, the pornography problem is more indirect. It is associated with manufacturing counterfeit utility by using sensory illusions. It involves an agent fooling itself into thinking that its actions are high in utilily - when in fact they are not - by manipulating its sense data to create a pleasing view, which is then rated as being of high utility.This is pretty similar to what happens when a human is exposed to pornography. A pornography-viewing human receives sensory stimulii that are normally associated with sex - which is in turn linked to reproductive success and so gets rated highly. However, in fact, pornography is not a reliable signal associated with reproductive success at all. Instead it is a pictorial representation of sex. It is a fake representation of something which is usually linked to high utility. For an intelligent machine, the pornography problem might manifest itself by the machine creating an illusion of a peaceful planet filled with happy humans - or whatever else it had been built to value. The reality might be very different. The pornography problem is pretty similar to the wirehead problem. If anything, it is probably harder to defend against. Wireheading can be crudely defended against by making it very difficult for an agent to perform brain surgery on itself. Using the same strategy against the pornography problem seems less likely to be effective.
PrevalenceWirehead behaviour is relatively uncommon - suggesting that there may be practical means of preventing its occurence.Pornography is probably more common - but again, not everyone indulges to the same degree. As before, this suggests that there may be ways of avoiding the problem.
Utility counterfeitingProbably more important than either of these two classes of problem is the umbrella category that contains both of them. That is what we are describing here as utility counterfeiting. It could also be referred fake utility.
Proposed solutionsThe main idea that has been proposed for dealing with the problems associated with utility counterfeiting involves giving the agent an understanding that the things that it wants should be thought of as being states of the world. If the agent really believes that, wireheading and pornography are likely to be percieved as ultimately unsatisfying illusions. However, exactly how best to provide an agent with this kind of strong belief about the intended purpose of its own representation of utility is not yet clear.The two problems have similar underlying causes. If proper preventative measures can be found, then both problems are likely to be fixed by the same type of intervention. Enjoy,
|
