Hello Richard, thanks for reporting this. For 1.8.0 we replaced a Heap with a SortedSet [1]. In this commit there is one loop [2] which iterates through the parses which will be advanced. The order of the Parsers in the Heap was not so well defined, therefore we decided to sort them by probability. We also noticed that this change is changing the output of the parser with the existing models in our SourceForge model eval test [3].
After running the evaluation on the OntoNotes4 data set I only got very small change and decided it is ok to do this. I am not aware of how big the change is but is was less than the delta in test case [4] of 0.001. What do you think? Should this be rolled back? Anyway, that said, about the parser, I still need to understand what happened with the lemmatizer. Jörn [1] https://github.com/apache/opennlp/commit/3df659b9bfb02084e782f1e8b6ec716f56e0611c [2] https://github.com/apache/opennlp/blob/3df659b9bfb02084e782f1e8b6ec716f56e0611c/opennlp-tools/src/main/java/opennlp/tools/parser/AbstractBottomUpParser.java#L285 [3] https://github.com/apache/opennlp/commit/3df659b9bfb02084e782f1e8b6ec716f56e0611c#diff-a5834f32b8a41b76a336126e4b13d4f7L349 [4] https://github.com/apache/opennlp/blob/3df659b9bfb02084e782f1e8b6ec716f56e0611c/opennlp-tools/src/test/java/opennlp/tools/eval/OntoNotes4ParserEval.java#L70 On Sat, May 13, 2017 at 10:35 PM, Richard Eckart de Castilho <r...@apache.org > wrote: > Hi all, > > > On 11.05.2017, at 18:37, Joern Kottmann <kottm...@gmail.com> wrote: > > > > The Apache OpenNLP PMC would like to call for a Vote on Apache OpenNLP > > 1.8.0 Release Candidate 2. > > Should OpenNLP 1.8.0 yield identical results as 1.7.2 when the same > models are used during classification? > > E.g. the English parser model seems to create different POS tags now > for the sentence "We need a very complicated example sentence , > which contains as many constituents and dependencies as possible .". > "a" is now wrongly tagged as "," whereas 1.7.2 tagged it correctly as "DT". > > Should OpenNLP 1.8.0 yield identical results as 1.7.2 when the same > training data is used during training? > > I have a test that trains a lemmatizer model on GUM 3.0.0. With 1.7.2, > this model reached an f-score of ~0.96. With 1.8.0, I only get ~0.84. > > Cheers, > > -- Richard > > >
