Hello Matt,
On 11/02/2015 06:16 AM, Matthew Taylor wrote:
Hello,
Generally, you can tell when NuPIC is totally confused about input
data when it simply repeats the last value it saw as the prediction.
This is pretty easy to see in a graph, because (assuming you used the
inference shifter) the prediction line is trailing the actual value
line by one step.
OK but, how can I be sure that it is not perfect prediction but rather
repeating?
I don't quite understand the rest of your question. When NuPIC just
repeats the last line, it will not be correct as the input data
changes.
Now I do not understand you. I was thinking about your video "One Hot
Gym Prediction Tutorial" around 45:00 you've made mistake in code and
NuPIC just repeated values. Around 48:00 you've fixed the code mistake
and then NuPIC start to predict values. In your data it is obvious when
NuPIC started to make predictions and when was repeating, but what if I
have data in which this is not so obvious? How can I know if NuPIC is
predicting?
PS:
What do you mean by:
"When NuPIC just repeats the last line, it will not be correct as then
input data changes."
PPS:
I did not use inference shifter. I was predicting one step ahead and it
was pretty clear from graph. I've assumed that inference shifter is
useful with larger steps.
PPPS:
Just for sake of clarity, it is not to possible for NuPIC to repeat then
predict and then again repeat? NuPIC repeat and then predict, am I correct?
You can also look into using the MetricsManager (see the hot
gym source code [1][2]) to get some error metrics out of the results.
[1]
https://github.com/numenta/nupic/blob/master/examples/opf/clients/hotgym/prediction/one_gym/run.py#L53-L66
[2]
https://github.com/numenta/nupic/blob/master/examples/opf/clients/hotgym/prediction/one_gym/run.py#L103-L122
Thank you for that, I will definitively check those metrics, can I find
further info about this?
Thank you very much.
