The problem with not knowing dynamical systems

I don't usually like to write about stuff like this, but I feel like this might actually do some good.  Myself and Jim Crutchfield (well-known for his work on chaos theory) have papers about new methods for continuous-time, discrete-event process inference and prediction (here) and about how one can view the predictive capabilities of dynamical systems as a function of their attractor type (here).  The reviews-- one from an information theory journal and another from machine learning experts-- unfortunately illustrated a lack of common knowledge on interdisciplinary problems.  So I thought I'd put a few key points here, for those studying recurrent neural networks in any way, shape, or form.

First, if you have a dynamical system, you can classify its behavior qualitatively by attractor type.  There are three types of attractors: fixed points, limit cycles, and beautiful strange attractors.  It turns out that the "qualitative" attractor type is a guide to many computational properties of the dynamical system (again, soon to appear on arXiv).

Second, hidden Markov models-- including unifilar ones, in which the current state and next symbol determine the next state-- are not memoryless or Markovian.

More to come.