The problem seems to be wanting the HTM to have some kind of "meta-standard". An anomaly that regularly occurs (is periodic) - isn't. So by definition, the HTM does exactly as it is supposed to by learning "patterns" which at one point are anomalous (as is every input at first). So conceptually I don't see the problem? Just thinking out loud...
Cheers, David On Thu, Oct 15, 2015 at 10:21 AM, Alex Lavin <ala...@numenta.com> wrote: > Hi Cas, > In your scenario, where you're concerned an HTM model will learn to > recognize a "common" anomaly as normal, a good idea then would be to turn > off learning. It is likely by this point the model has been exposed to a > large history of data, and thus has sufficiently learned temporal patterns > to reliably detect anomalies, both the common ones and others. And as Mark > recommended, it would be good to have a second model that continues to > learn. > > To illustrate what I mean, I've done this in the Hot Gym anomaly example > [1]. Running as is (with learning always on), two anomalies are detected: > at "2010-10-23 10:00:00" and "2010-11-13 08:00:00". By adding `if i == > 2000: model.disableLearning()` before the model runs [2], I turn off > learning for all of the incoming data instances 2000+. This results in 38 > additional anomalies detected. But with i == 3000, the model still flags > only the original two anomalies. So as I described above, disabling > learning can be useful, but this example shows this would only be a > reliable solution if the model has had sufficient time to learn from the > data. > > Additionally, take a look at these plots [3, 4]. The first represents the > model with learning always on, and the second with learning turned off at > record 3000. Notice the difference in the anomaly log-likelihood values > (bottom plots) after 3000. > > [1] > https://github.com/numenta/nupic/tree/master/examples/opf/clients/hotgym/anomaly > [2] > https://github.com/numenta/nupic/blob/master/examples/opf/clients/hotgym/anomaly/hotgym_anomaly.py#L68 > [3] https://plot.ly/~alavin/3236/hot-gym-anomalies/ > [4] https://plot.ly/~alavin/3238/hot-gym-anomalies/ > > Cheers, > Alex > -- *With kind regards,* David Ray Java Solutions Architect *Cortical.io <http://cortical.io/>* Sponsor of: HTM.java <https://github.com/numenta/htm.java> d....@cortical.io http://cortical.io
