Hi, I'm Tim Tyler, and this is a video about utility counterfeiting.
Utility counterfeiting is an umbrella category that refers to
problems with generating fake utility such as the wirehead
problem and the pornography problem in machine intelligence.
The wirehead problem
To recap, the
wirehead problem involves an agent stimulating its own pleasure centres or
performing some kind of direct synthesis of utility.
The pornography problem
By contrast, the pornography problem is more indirect. It is
associated with manufacturing counterfeit utility by using sensory
illusions. It involves an agent fooling itself into thinking that its actions
are high in utilily - when in fact they are not - by manipulating its sense
data to create a pleasing view, which is then rated as being of high
This is pretty similar to what happens when a human is exposed to
pornography. A pornography-viewing human receives sensory stimulii that are
normally associated with sex - which is in turn linked to
reproductive success and so gets rated highly. However, in fact, pornography
is not a reliable signal associated with reproductive success at all.
Instead it is a pictorial representation of sex. It is a fake
representation of something which is usually linked to high utility.
For an intelligent machine, the pornography problem might manifest
itself by the machine creating an illusion of a peaceful planet
filled with happy humans - or whatever else it had been built to value. The
reality might be very different.
The pornography problem is pretty similar to the wirehead
problem. If anything, it is probably harder to defend against.
Wireheading can be crudely defended against by making it very difficult for an
agent to perform brain surgery on itself. Using the same strategy against
the pornography problem seems less likely to be effective.
Wirehead behaviour is relatively uncommon - suggesting that there may
be practical means of preventing its occurence.
Pornography is probably more common - but again, not everyone indulges to the
same degree. As before, this suggests that there may be ways of avoiding the
Probably more important than either of these two classes of problem is the
umbrella category that contains both of them. That is what we are describing
here as utility counterfeiting. It could also be referred fake
The main idea that has been proposed for dealing with the problems associated
with utility counterfeiting involves giving the agent an
understanding that the things that it wants should be thought of as being
states of the world. If the agent really believes that,
wireheading and pornography are likely to be percieved as
ultimately unsatisfying illusions. However, exactly how best to
provide an agent with this kind of strong belief about the intended purpose of
its own representation of utility is not yet clear.
The two problems have similar underlying causes. If proper
preventative measures can be found, then both problems are likely to be
fixed by the same type of intervention.