With the issues reported by Richard we should cancel the vote and rollback the release.
I change my vote to -1 (binding) 2017-05-13 19:08 GMT-03:00 Richard Eckart de Castilho <r...@apache.org>: > > > On 13.05.2017, at 22:35, Richard Eckart de Castilho <r...@apache.org> > wrote: > > > > Should OpenNLP 1.8.0 yield identical results as 1.7.2 when the same > > training data is used during training? > > > > I have a test that trains a lemmatizer model on GUM 3.0.0. With 1.7.2, > > this model reached an f-score of ~0.96. With 1.8.0, I only get ~0.84. > > Also, this test which trains and evaluates a lemmatizer model > takes ~8 sec with 1.7.2 and ~170 sec with 1.8.0. Even when only > considering the training phase (no evaluation), the test runs > much faster with 1.7.2 than with 1.8.0. > > Here are some details on the training phase. > > It seems odd that the events, outcomes, and predicates change that much. > > === 1.7.2 > > done. 50697 events > Indexing... done. > Sorting and merging events... done. Reduced 50697 events to 12675. > Done indexing. > Incorporating indexed data for training... > done. > Number of Event Tokens: 12675 > Number of Outcomes: 389 > Number of Predicates: 13488 > ...done. > Computing model parameters ... > Performing 10 iterations. > 1: ... loglikelihood=-302335.58198350534 0.8420616604532812 > 2: ... loglikelihood=-61602.20311717376 0.9492672150225852 > 3: ... loglikelihood=-30747.954089148297 0.9769217113438665 > 4: ... loglikelihood=-19986.853691639506 0.9850484249561118 > 5: ... loglikelihood=-14672.523462458894 0.9881255301102629 > 6: ... loglikelihood=-11572.587093608756 0.9893879322247865 > 7: ... loglikelihood=-9571.242700030467 0.9900783083811665 > 8: ... loglikelihood=-8185.394028944442 0.9906897844053889 > 9: ... loglikelihood=-7174.66904253965 0.9912223602974535 > 10: ... loglikelihood=-6407.4278143846 0.9917746612225575 > > > === 1.8.0 > > done. 50697 events > Indexing... done. > Sorting and merging events... done. Reduced 50697 events to 26026. > Done indexing. > Incorporating indexed data for training... > done. > Number of Event Tokens: 26026 > Number of Outcomes: 7668 > Number of Predicates: 15279 > ...done. > Computing model parameters ... > Performing 10 iterations. > 1: ... loglikelihood=-453475.08854769287 1.972503303943034E-5 > 2: ... loglikelihood=-165718.68620632993 0.9509241177978973 > 3: ... loglikelihood=-85388.42871190465 0.9761327100222893 > 4: ... loglikelihood=-56404.00400621838 0.9892104069274316 > 5: ... loglikelihood=-41004.08840359108 0.9938457896916977 > 6: ... loglikelihood=-31539.64788603799 0.9955421425330887 > 7: ... loglikelihood=-25264.889481438582 0.9964889441189814 > 8: ... loglikelihood=-20883.72059438774 0.9972384953744797 > 9: ... loglikelihood=-17699.228362701586 0.9977710712665444 > 10: ... loglikelihood=-15306.654021266759 0.9980669467621358 > > > I also get some differences in f-score for other tests that train models, > but not as significant as when training a lemmatizer model. > > -- Richard >
