Hmm, I haven't run that in a while but I hope nothing significant has changed in NuPIC. There is some natural variation in the swarm algorithm from run to run but it shouldn't be that large.
Unfortunately I am out of the country on vacation for 5 more days with limited email access. I probably won't be able to look at it until next week sometime. Hope that's ok. --Subutai On Wed, Oct 8, 2014 at 10:28 AM, John Blackburn <[email protected]> wrote: > Dear Subutai > > I tried to run your "multiple fields example 1" from > > https://github.com/subutai/nupic.subutai/tree/master/swarm_examples > > I ran the command > > run_swarm.py multi1_search_def.json --overwrite --maxWorkers 5 > > using the supplied JSON file and "run_swarm.py" from the "scripts" > directory. I got the result: > > Field Contributions: > { u'metric1': 0.0, > u'metric2': 20.0598347434741, > u'metric3': -63.85677190034707, > u'metric4': -157.77883953004587, > u'metric5': -153.23706619032606} > > Best results on the optimization metric > multiStepBestPredictions:multiStep:errorMetric='altMAPE':steps=[1]:window=1000:field=metric1 > (maximize=False): > [41] Experiment _NupicModelInfo(jobID=1062, modelID=4815, > status=completed, completionReason=eof, updateCounter=22, numRecords=1500) > (modelParams|clParams|alpha_0.055045.modelParams|tpParams|minThreshold_11.modelParams|tpParams|activationThreshold_14.modelParams|tpParams|pamLength_3.modelParams|sensorParams|encoders|metric2:n_296.modelParams|sensorParams|encoders|metric1:n_307.modelParams|spParams|synPermInactiveDec_0.055135): > > multiStepBestPredictions:multiStep:errorMetric='altMAPE':steps=[1]:window=1000:field=metric1: > 1.57090277774 > > So the error was only slightly improved to 1.57 (altMAPE) compared to the > "basic swarm with one field" > > Now in the readme file, you stated you got the result: > > Best results on the optimization metric > multiStepBestPredictions:multiStep:errorMetric='altMAPE':steps=[1]:window=1000:field=metric1 > (maximize=False): [52] Experiment _GrokModelInfo(jobID=1161, modelID=23650, > status=completed, completionReason=eof, updateCounter=22, numRecords=1500) > (modelParams|clParams|alpha_0.0248715879513.modelParams|tpParams|minThreshold_10.modelParams|tpParams|activationThreshold_13.modelParams|tpParams|pamLength_2.modelParams|sensorParams|encoders|metric2:n_271.modelParams|sensorParams|encoders|metric1:n_392.modelParams|spParams|synPermInactiveDec_0.0727958344423): > multiStepBestPredictions:multiStep:errorMetric='altMAPE':steps=[1]:window=1000:field=metric1: > 0.886040768868 > > Field Contributions: > { u'metric1': 0.0, > u'metric2': 54.62889798318686, > u'metric3': -23.71223053273957, > u'metric4': -91.68162623355796, > u'metric5': -25.51553640787998} > > Which gives a considerable improvement to to 0.886 (altMAPE). Note that in > "Field > Contributions" you get a 54.6% improvement from metric2 while in my run I > only got 20.05% improvement. > > Can we explain this discrepancy? I think I ran your code exactly. It's > important because it shows my NUPIC > > is not working as well with multiple fields as yours is which is especially > important for the bridge > project I keep going on about! I notice your output refers to GrokModelInfo, > while mine refers to > > NupicModelInfo. > > John. > > > > >
