So you still think it's fine that the default would perform at 37 BLEU points less than just selecting the most likely translation of each phrase?
You know I think I would have to try really hard to design a system that performed so poorly. James ________________________________________ From: amittai axelrod <amit...@umiacs.umd.edu> Sent: Wednesday, June 24, 2015 5:36 PM To: Read, James C; Lane Schwartz Cc: moses-support@mit.edu; Philipp Koehn Subject: Re: [Moses-support] Major bug found in Moses what *i* would do is tune my systems. ~amittai On 6/24/15 09:15, Read, James C wrote: > Thank you for such an invitation. Let's see. Given the choice of > > a) reading through thousands of lines of code trying to figure out why the > default behaviour performs considerably worse than merely selecting the most > likely translation of each phrase or > b) spending much less time implementing a simple system that does just that > > which one would you do? > > For all know maybe I've already implemented such a system that does just that > and not only that improves considerably on such a basic benchmark. But given > that on this list we don't seem to be able to accept that there is a problem > with the default behaviour of Moses I can only conclude that nobody would be > interested in access to the code of such a system. > > James > > ________________________________________ > From: amittai axelrod <amit...@umiacs.umd.edu> > Sent: Friday, June 19, 2015 7:52 PM > To: Read, James C; Lane Schwartz > Cc: moses-support@mit.edu; Philipp Koehn > Subject: Re: [Moses-support] Major bug found in Moses > > if we don't understand the problem, how can we possibly fix it? > all the relevant code is open source. go for it! > > ~amittai > > On 6/19/15 12:49, Read, James C wrote: >> So, all I did was filter out the less likely phrase pairs and the BLEU >> score shot up. Was that such a stroke of genius? Was that not blindingly >> obvious? >> >> >> Your telling me that redesigning the search algorithm to prefer higher >> scoring phrase pairs is all we need to do to get a best paper at ACL? >> >> >> James >> >> >> >> ------------------------------------------------------------------------ >> *From:* Lane Schwartz <dowob...@gmail.com> >> *Sent:* Friday, June 19, 2015 7:40 PM >> *To:* Read, James C >> *Cc:* Philipp Koehn; Burger, John D.; moses-support@mit.edu >> *Subject:* Re: [Moses-support] Major bug found in Moses >> On Fri, Jun 19, 2015 at 11:28 AM, Read, James C <jcr...@essex.ac.uk >> <mailto:jcr...@essex.ac.uk>> wrote: >> >> What I take issue with is the en-masse denial that there is a >> problem with the system if it behaves in such a way with no LM + no >> pruning and/or tuning. >> >> >> There is no mass denial taking place. >> >> Regardless of whether or not you tune, the decoder will do its best to >> find translations with the highest model score. That is the expected >> behavior. >> >> What I have tried to tell you, and what other people have tried to tell >> you, is that translations with high model scores are not necessarily >> good translations. >> >> We all want our models to be such that high model scores correspond to >> good translations, and that low model scores correspond with bad >> translations. But unfortunately, our models do not innately have this >> characteristic. We all know this. We also know a good way to deal with >> this shortcoming, namely tuning. Tuning is the process by which we >> attempt to ensure that high model scores correspond to high quality >> translations, and that low model scores correspond to low quality >> translations. >> >> If you can design models that naturally correspond with translation >> quality without tuning, that's great. If you can do that, you've got a >> great shot at winning a Best Paper award at ACL. >> >> In the meantime, you may want to consider an apology for your rude >> behavior and unprofessional attitude. >> >> Goodbye. >> Lane >> >> >> >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
