This vote is now cancelled, will push out another RC after fixing the reported issues
-1 binding Sent from my iPhone > On May 13, 2017, at 8:44 PM, William Colen <[email protected]> wrote: > > With the issues reported by Richard we should cancel the vote and rollback > the release. > > I change my vote to -1 (binding) > > 2017-05-13 19:08 GMT-03:00 Richard Eckart de Castilho <[email protected]>: > >> >>> On 13.05.2017, at 22:35, Richard Eckart de Castilho <[email protected]> >> wrote: >>> >>> Should OpenNLP 1.8.0 yield identical results as 1.7.2 when the same >>> training data is used during training? >>> >>> I have a test that trains a lemmatizer model on GUM 3.0.0. With 1.7.2, >>> this model reached an f-score of ~0.96. With 1.8.0, I only get ~0.84. >> >> Also, this test which trains and evaluates a lemmatizer model >> takes ~8 sec with 1.7.2 and ~170 sec with 1.8.0. Even when only >> considering the training phase (no evaluation), the test runs >> much faster with 1.7.2 than with 1.8.0. >> >> Here are some details on the training phase. >> >> It seems odd that the events, outcomes, and predicates change that much. >> >> === 1.7.2 >> >> done. 50697 events >> Indexing... done. >> Sorting and merging events... done. Reduced 50697 events to 12675. >> Done indexing. >> Incorporating indexed data for training... >> done. >> Number of Event Tokens: 12675 >> Number of Outcomes: 389 >> Number of Predicates: 13488 >> ...done. >> Computing model parameters ... >> Performing 10 iterations. >> 1: ... loglikelihood=-302335.58198350534 0.8420616604532812 >> 2: ... loglikelihood=-61602.20311717376 0.9492672150225852 >> 3: ... loglikelihood=-30747.954089148297 0.9769217113438665 >> 4: ... loglikelihood=-19986.853691639506 0.9850484249561118 >> 5: ... loglikelihood=-14672.523462458894 0.9881255301102629 >> 6: ... loglikelihood=-11572.587093608756 0.9893879322247865 >> 7: ... loglikelihood=-9571.242700030467 0.9900783083811665 >> 8: ... loglikelihood=-8185.394028944442 0.9906897844053889 >> 9: ... loglikelihood=-7174.66904253965 0.9912223602974535 >> 10: ... loglikelihood=-6407.4278143846 0.9917746612225575 >> >> >> === 1.8.0 >> >> done. 50697 events >> Indexing... done. >> Sorting and merging events... done. Reduced 50697 events to 26026. >> Done indexing. >> Incorporating indexed data for training... >> done. >> Number of Event Tokens: 26026 >> Number of Outcomes: 7668 >> Number of Predicates: 15279 >> ...done. >> Computing model parameters ... >> Performing 10 iterations. >> 1: ... loglikelihood=-453475.08854769287 1.972503303943034E-5 >> 2: ... loglikelihood=-165718.68620632993 0.9509241177978973 >> 3: ... loglikelihood=-85388.42871190465 0.9761327100222893 >> 4: ... loglikelihood=-56404.00400621838 0.9892104069274316 >> 5: ... loglikelihood=-41004.08840359108 0.9938457896916977 >> 6: ... loglikelihood=-31539.64788603799 0.9955421425330887 >> 7: ... loglikelihood=-25264.889481438582 0.9964889441189814 >> 8: ... loglikelihood=-20883.72059438774 0.9972384953744797 >> 9: ... loglikelihood=-17699.228362701586 0.9977710712665444 >> 10: ... loglikelihood=-15306.654021266759 0.9980669467621358 >> >> >> I also get some differences in f-score for other tests that train models, >> but not as significant as when training a lemmatizer model. >> >> -- Richard >>
