Now in the trunk we have the tools: $ bin/opennlp DoccatEvaluator Usage: opennlp DoccatEvaluator[.leipzig] [-reportOutputFile outputFile] -model model [-misclassified true|false] -data sampleData [-encoding charsetName]
Arguments description: -reportOutputFile outputFile the path of the fine-grained report file. -model model the model file to be evaluated. -misclassified true|false if true will print false negatives and false positives. -data sampleData data to be used, usually a file name. -encoding charsetName encoding for reading and writing text, if absent the system default is used. n$ bin/opennlp DoccatCrossValidator Usage: opennlp DoccatCrossValidator[.leipzig] [-reportOutputFile outputFile] [-misclassified true|false] [-folds num] [-featureGenerators fg] [-params paramsFile] -lang language -data sampleData [-encoding charsetName] Arguments description: -reportOutputFile outputFile the path of the fine-grained report file. -misclassified true|false if true will print false negatives and false positives. -folds num number of folds, default is 10. -featureGenerators fg Comma separated feature generator classes. Bag of words is used if not specified. -params paramsFile training parameters file. -lang language language which is being processed. -data sampleData data to be used, usually a file name. -encoding charsetName encoding for reading and writing text, if absent the system default is used. If misclassified is true, the evaluator will use the stderr to print the misclassified documents. If reportOutputFile is set, the evaluator will print to it some detailed reports, for example the f-measure for the different outcomes and the confusion matrix. 2014-04-10 19:48 GMT-03:00 William Colen <william.co...@gmail.com>: > Yes, I just finished implementing the confusion matrix report, just like > the one I did for the POS Tagger. I will commit it today. > > I could not test it properly with Leipzig corpus. For some reason to > Doccat never fails with this corpus! > To effectively test it I used the 20news corpus. > > > 2014-04-10 19:37 GMT-03:00 Jörn Kottmann <kottm...@gmail.com>: > > I thought it should be done similar to the way pos tags are measured when >> I implemented that. >> >> A confusion matrix might also be helpful to see which categories are more >> difficult to classify for the system. >> >> Jörn >> >> >> On 04/10/2014 03:00 PM, William Colen wrote: >> >>> Actually, since we always add a tag to each document, accuracy makes >>> sense. >>> We could implement F-1 for the individual categories. >>> >>> 2014-04-09 17:23 GMT-03:00 William Colen <william.co...@gmail.com>: >>> >>> Hello, >>>> >>>> I was checking if there is any open issue related to Doccat, and I found >>>> this one - >>>> >>>> OPENNLP-81: Add a cli tool for the doccat evaluation support >>>> >>>> I noticed that there is already a class >>>> named DocumentCategorizerEvaluator, which is not used anywhere >>>> internally. >>>> This is evaluating performance in terms of accuracy, but I believe it >>>> would >>>> be better do do it in terms of F-Measuare. >>>> >>>> Any thoughts? >>>> >>>> As we are working in a major version, I think it would be OK to change >>>> it. >>>> >>>> >>>> Thank you, >>>> William >>>> >>>> >> >