Hello Matt,
On 11/02/2015 04:29 PM, Matthew Taylor wrote:
A perfect prediction would predict the **next** value perfectly, not
the current value.
Clear
Basically, what I was trying to say is that when NuPIC is entirely
confused, it guesses that the next value in the data will be the same
as the last value it saw. So if you pass in "5" as the value, it might
return a predicted value of "5". It is fairly obvious for moving data
when this is the case, because you can easily see it when plotted.
Clear
When evaluating prediction models, the bottom line method of measuring
performance is to compare the predicted values NuPIC is returning to
the actual values in the data *when they occur*. So if you are
predicting 10 steps ahead, you'll not know how well the predictions
really are until you get to 10 steps in the future. But at that point
it's fairly easy to compare them and see how far off NuPIC is.
Clear
I generally use the inference shifter anytime I am plotting results.
This assures that the data is lined up properly in time, and that each
timestamp plots the actual result and predicted result on the same
vertical frame. I suggest if you are plotting to also use the
inference shifter. Your misunderstanding of this issue may certainly
be cleared up. Your charts will NOT be correct if you don't use it.
Clear, just one note: without using inference shifter and if my
prediction step is one and theoretically NuPIC would predict perfectly
all values then I will get two identical plots where the predicted plot
will be shifted one step forward on horizontal line. If I'm wrong
correct me please.
I don't know what your PPPS comment means. :/ NuPIC will predict
repeating patterns in the sequence if it has seem them enough times.
I will try to explain following way, hope this will be more clear:
I've assumed that when new data comes to NuPIC (first few rows of csv
input) the NuPIC is in some kind of "repeat state" and simply repeat
what it sees. After enough data NuPIC goes to some kind of "predict
state" and from there it try to make predictions.
I was asking if is OK for NuPIC to start with:
"repeat state" (when you have not enough data)
then go to
"predict state" (when you have enough data to make predictions)
then again go to
"repeat state" (if there are new patterns that was not seen before)
Now after reading your message I know that there are no such states and
it is the feature of HTM memory. And therefore my question:
"How to know if NuPIC is predicting or repeating?" is meaningless and
the correct answer to this should be:
"You cannot, NuPIC is always learning and making predictions based on
what it sees in the past, for example if it sees suddenly bunch of zeros
then probably the prediction will be zero because most probably the next
value will be again zero".
Funny how often I've read/hear this but just now realized the concepts
behind it ;)
If I'm wrong correct me please.
Thank you very much.
