Bayes classification result are unstable after classifying non-existing features
--------------------------------------------------------------------------------
Key: MAHOUT-955
URL: https://issues.apache.org/jira/browse/MAHOUT-955
Project: Mahout
Issue Type: Bug
Components: Classification
Affects Versions: 0.5
Environment: JRE 7
Reporter: Michał B
Priority: Critical
Bayes classification results are unstable, and change during runtime!
Sample test:
MyClassifier classifier = new MyClassifier(new ChimeoBayesAlgorithm(),params);
//Custom simple wrapper for classifier
ClassifierResult category = classifier.classify("existing");
double resultA = category.getScore();
category = classifier.classify("nonexisting");
category = classifier.classify("existing");
double resultB = category.getScore();
Assert.assertEquals(resultA,resultB,0.0); // FAIL!!!
Test like the one above will fail. Because nonexisting tokens are added to
InMemoryBayesDatastore->featureDictionary therefore
datastore.getWeight("sumWeight", "vocabCount") change after classification of
unknown feature. Moreover, the featureDictionary fills with not wanted strings
using heapspace.
More on this here
http://www.lucidimagination.com/search/document/7dabe3efec8d136d/issues_with_memory_use_and_inconsistent_or_state_influenced_results_when_using_cbayesalgorit#8853165db260bf75
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
