Stream Compression Virtues

Stream Compression Virtues


Hi! I'm Tim Tyler, and this is a technical video about the relative virtues of stream compression and file compression when modelling the forecasting component of a modular machine intelligence.

Previous prizes

The previous machine intelligence compression prize - run by Marcus Hutter - focused on file compression.

That competition was oriented around the compression of a single large text file - taken from Wikipedia.

Stream vs file

Think for a moment about the needs of an intelligent agent attempting to understand the consequences of its possible actions.

To make real time predictions such an agent constantly needs to be able to create forecasts about what might happen next, conditional on it taking various actions.

The tool needed to deal with such situations is a lot more like a stream compressor than it is like a file compressor. Stream compressors are constantly concerned with the symbols which will come next in a stream - and their respective probabilities.

Updating a model in real time

Agents often have to process and understand their sensory inputs while more data continues to stream in.

Agents sometimes get an opportunity to compress their sensory data when no more inputs are being received. For example, that can happen when they are asleep - or if they meditate. However, not all machine intelligences may get such opportunities. Even if you take agents offline from time-to-time to allow them to digest their sense data, stream compression is still often a much better model.

In stream compression an agent must balance its resources between using its existing model to make predictions, and developing a better model of the data.

Also agents are often quite focussed on their current circumstances. If you are in a dynamic situation, it might well be important to maintain a good model of what is happening now - but less important to digest material that happened some time ago. Agents should be able to prioritise such things.

Stream compression is better

The main strength of stream compressors, though is simply the ability to continuously make predictions in real time. It is often absolutely critical that agents be able to rapidly make predictions about their immediate future. Stream compression is a natural fit for this application - whereas file compression is not.

I think these features mean that stream compression is almost always a more natural fit - than file compression - to the problem of developing forecasting systems for use in intelligent machines.

Theoretical equivalence

There is a sense in which stream compression and file compression are equivalent to one another. It is possible to convert stream compressors into file compressors - and visa versa. However, if you are resource limited, the operation is often extremely impractical and inefficient - since the resulting construction often operates far too slowly to be of very much use.

Matching pennies

The matching pennies game involves dealing with an incoming sensory stream of data. The match between stream compression and the real-time forecasting problems intelligent agents have to solve is one of the reasons that this game has been selected for use on this site.



  • File vs stream compression

    Tim Tyler | Contact |