> On Jun 24, 2015, at 10:47 , Read, James C <jcr...@essex.ac.uk> wrote: > > So you still think it's fine that the default would perform at 37 BLEU points > less than just selecting the most likely translation of each phrase?
Yes, I'm pretty sure we all think that's fine, because one of the steps of building a system is tuning. Is this really the essence of your complaint? That the behavior without tuning is not very good? (Please try to reply without your usual snarkiness.) - John Burger MITRE > You know I think I would have to try really hard to design a system that > performed so poorly. > > James > > ________________________________________ > From: amittai axelrod <amit...@umiacs.umd.edu> > Sent: Wednesday, June 24, 2015 5:36 PM > To: Read, James C; Lane Schwartz > Cc: moses-support@mit.edu; Philipp Koehn > Subject: Re: [Moses-support] Major bug found in Moses > > what *i* would do is tune my systems. > > ~amittai > > On 6/24/15 09:15, Read, James C wrote: >> Thank you for such an invitation. Let's see. Given the choice of >> >> a) reading through thousands of lines of code trying to figure out why the >> default behaviour performs considerably worse than merely selecting the most >> likely translation of each phrase or >> b) spending much less time implementing a simple system that does just that >> >> which one would you do? >> >> For all know maybe I've already implemented such a system that does just >> that and not only that improves considerably on such a basic benchmark. But >> given that on this list we don't seem to be able to accept that there is a >> problem with the default behaviour of Moses I can only conclude that nobody >> would be interested in access to the code of such a system. >> >> James >> >> ________________________________________ >> From: amittai axelrod <amit...@umiacs.umd.edu> >> Sent: Friday, June 19, 2015 7:52 PM >> To: Read, James C; Lane Schwartz >> Cc: moses-support@mit.edu; Philipp Koehn >> Subject: Re: [Moses-support] Major bug found in Moses >> >> if we don't understand the problem, how can we possibly fix it? >> all the relevant code is open source. go for it! >> >> ~amittai >> >> On 6/19/15 12:49, Read, James C wrote: >>> So, all I did was filter out the less likely phrase pairs and the BLEU >>> score shot up. Was that such a stroke of genius? Was that not blindingly >>> obvious? >>> >>> >>> Your telling me that redesigning the search algorithm to prefer higher >>> scoring phrase pairs is all we need to do to get a best paper at ACL? >>> >>> >>> James >>> >>> >>> >>> ------------------------------------------------------------------------ >>> *From:* Lane Schwartz <dowob...@gmail.com> >>> *Sent:* Friday, June 19, 2015 7:40 PM >>> *To:* Read, James C >>> *Cc:* Philipp Koehn; Burger, John D.; moses-support@mit.edu >>> *Subject:* Re: [Moses-support] Major bug found in Moses >>> On Fri, Jun 19, 2015 at 11:28 AM, Read, James C <jcr...@essex.ac.uk >>> <mailto:jcr...@essex.ac.uk>> wrote: >>> >>> What I take issue with is the en-masse denial that there is a >>> problem with the system if it behaves in such a way with no LM + no >>> pruning and/or tuning. >>> >>> >>> There is no mass denial taking place. >>> >>> Regardless of whether or not you tune, the decoder will do its best to >>> find translations with the highest model score. That is the expected >>> behavior. >>> >>> What I have tried to tell you, and what other people have tried to tell >>> you, is that translations with high model scores are not necessarily >>> good translations. >>> >>> We all want our models to be such that high model scores correspond to >>> good translations, and that low model scores correspond with bad >>> translations. But unfortunately, our models do not innately have this >>> characteristic. We all know this. We also know a good way to deal with >>> this shortcoming, namely tuning. Tuning is the process by which we >>> attempt to ensure that high model scores correspond to high quality >>> translations, and that low model scores correspond to low quality >>> translations. >>> >>> If you can design models that naturally correspond with translation >>> quality without tuning, that's great. If you can do that, you've got a >>> great shot at winning a Best Paper award at ACL. >>> >>> In the meantime, you may want to consider an apology for your rude >>> behavior and unprofessional attitude. >>> >>> Goodbye. >>> Lane >>> >>> >>> >>> _______________________________________________ >>> Moses-support mailing list >>> Moses-support@mit.edu >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
