I've been wanting to write this post for a while, but never had the courage.  But here goes.

Every so often, I encounter a paper that proposes a new objective function for agent behavior.  Sometimes, something like predictive information is proposed; sometimes, something more like entropy rate is proposed.  In both cases, I have a bit of an issue.

When we try and say maximize predictability while minimizing memory, you end up either flipping coins (when you penalize memory too much) or running in very large circles (when you don't penalize memory enough).  There doesn't usually seem to be an interesting intermediate behavior.

When you maximize entropy rate, you typically end up flipping coins.

The key to making these objective functions interesting, I think, is to add enough constraints that they start doing interesting things.  And since this now impinges upon an old project that I may pick up again in the near future, that's all I'll say!

I've been reading a lot of papers lately in physics, in neuroscience, in biology in general, in which new mechanisms for memory are discovered.  Certain types of inputs are shown, the system's state appears to remember something about past inputs, and victory is declared.

I think that the general trend here is wonderful.  Systems do have memory, and many have memory for a reason; some of them don't, but their memory can actually be used for something.  It's definitely about time that we started cataloguing these phenomena.

However, at the risk of being a Debbie Downer, I want to point out that pretty much any input-dependent dynamical system has memory.  What that means is that if your system's state evolves according to some set of rules than involves the input, then you're pretty much guaranteed to have memory.  Thus, memory in and of itself is not that special.  Both forgetting and remembering are typical.  The real question is, what does the system remember?

What is special is when the system remembers, and only remembers, just the "right" things.  Now, "right" is an unfortunate word because it's user-defined, but what is "right" depends on what the system is used for.  In any particular application, there's some number of necessary things that the system needs to remember in order to do its job, whatever that may be.  And it is usually tricky to design an input-dependent dynamical system that stores these things and only these things.  (Exception: periodic input is exceptionally easy to remember.  It is entirely predictable.)

I've therefore started to look at some of these papers a little more critically, like the grump that I am.  It doesn't make me less excited where this field will go, but it does highlight that more studies on what is "typical" are sorely needed.