Expected Utility Maximisers

Background

• Utilitarianism

  Utilitarianism is a political ethical philosophy in which actions are taken in accordance with the extent to which they help to universally maximise some measure of utility.

  It originated primarily with Jeremy Bentham and John Stuart Mill in the 1800s.

• Microeconomics

  The concept of expected utility was originally developed by Daniel Bernoulli , in a solution to the St. Petersburg paradox.

  It was subsequently developed by John von Neumann and Oskar Morgenstern in the 1940s.

  The concept has subsequently been explored in modern times, resulting in the rebranding of the idea as generalized expected utility and cumulative prospect theory.

  The framework has subsequently been extended to deal with subjective probability estimates, and the surreal utility values used by combinatorial game theory.

Expected Utility Maximisers

For an example, consider a computer program that plays the game of go. Such a program considers its possible moves, calculates their possible consequences, and then performs the move that it thinks gives it the best chance of winning.

Expected utility maximisation is common framework used in the context of modelling intelligent agents and constructing synthetic intelligences.

A utility function can neatly encapsulate many concepts from economics - such as risk aversion and temporal discounting and marginal utility.

If the utility function is expressed as in a Turing-complete lanugage, the framework represents a remarkably-general model of intelligent agents - one which is capable of representing any pattern of behavioural responses that can itself be represented computationally.

The utility function encodes all the agent's preferences - often including:

• Temporal discounting

  Temporal discounting refers to how an agent values utility now, compared to utility later. Is ten dollars now better than twenty dollars tomorrow? An agent's temporal discounting preferences specify such things.

• Risk aversion

  Risk aversion refers to the reluctance of an agent to accept a bargain with an uncertain payoff rather than another bargain with a more certain, but lower, payoff.

Any computable agent may described using a utility function

Some argue that humans have no utility function. However, this makes little sense: all computable agents have utility functions. The human utility function may not be easy to write down - but that doesn't mean that it doesn't exist.

Self-Improving Systems

After a certain point, such systems tend to naturally come to share various traits with living organisms - they will resist death, maintain themselves, absorb resources, grow and/or reproduce, eliminate the competition - and so on. These natural tendencies are not necessarily benign.

Self-improving systems may wish to change their levels of temporal discounting and risk aversion - depending on their capabilities. One obvious way of doing that is to make these factors depend on your percieved self-confidence.

The complex field of utility engineering deals with how to construct utiliy functions which are useful, and don't have too many undesirable side effects.

Pragmatic and ideal goals

Self-improving systems will often make use of the concepts of ideal goals and pragmatic goals.

An ideal goal represents what a system actually wants. Pragmatic goals are a cut-down versions of this - which are faster, easier or cheaper to calculate.

For example, the synthetic intelligence, Deep Blue had a complex utility function with over 8,000 parts, which contained relative piece values, the worth of central control vs castling, heuristics about pawn promotion - and so on. However, its real aim was to increase IBM's stock price by winning games of chess.

A self-improving system will normally only have one ideal utility function - but may derive various pragmatic utility functions from this - depending on the resource constraints it faces.

It is not usually a good idea to encode strategies for dealing with resource constraints into the ideal goals of a system - since resource availability may change as time passes. Strategies such as outcome pruning and temporal discounting normally belong in pragmatic utility functions - and are best kept out of ideal utility functions.

Instrumental and ultimate goals

Often long-term projects can be broken down into numerous short-term ones. For example having a child can be decomposed into learning a trade, getting a job, buying a house, finding a mate - and so on. These short-term targets are known as instrumental goals.

Instrumental and ultimate goals are sometimes referred to as primary goals and secondary goals. The terms instrumental values and terminal values express the same basic concept.

Instrumental goals can sometimes be used as pragmatic goals - but these are really separate concepts.

Ultimate goals and ideal goals however are often rather similar concepts.

Links