: Re: [Moses-support] Major bug found in Moses
James, did you try the modifications Philip suggested (removing the word
penalty and lowering p(f|e)?
(I doubt it will be enough to get a best paper award, but it would probably
improve your bleu, that's always a good start :) )
On Friday, June 19
; moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
Hi James,
Well, it's pretty straightforward: The decoder's job is to find the
hypothesis with the maximum model score. That's why everybody builds
models which assign high model score to high-quality translations
To: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
[sorry for the garbled message before]
you are right. The idea is pretty obvious. It roughly corresponds to
'Histogram pruning' in this paper:
Zens, R., Stanton, D., Xu, P. (2012). A Systematic Comparison
amit...@umiacs.umd.edu
Sent: Friday, June 19, 2015 7:52 PM
To: Read, James C; Lane Schwartz
Cc: moses-support@mit.edu; Philipp Koehn
Subject: Re: [Moses-support] Major bug found in Moses
if we don't understand the problem, how can we possibly fix it?
all the relevant code is open source. go
: Lane Schwartz dowob...@gmail.com
Sent: Wednesday, June 24, 2015 4:43 PM
To: Read, James C
Cc: Rico Sennrich; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
On Wed, Jun 24, 2015 at 8:11 AM, Read, James C
jcr...@essex.ac.ukmailto:jcr...@essex.ac.uk wrote:
Other than
: amittai axelrod amit...@umiacs.umd.edu
Sent: Wednesday, June 24, 2015 5:36 PM
To: Read, James C; Lane Schwartz
Cc: moses-support@mit.edu; Philipp Koehn
Subject: Re: [Moses-support] Major bug found in Moses
what *i* would do is tune my systems.
~amittai
On 6/24/15 09:15, Read, James C wrote
-support@mit.edu; Philipp Koehn
Subject: Re: [Moses-support] Major bug found in Moses
if we don't understand the problem, how can we possibly fix it?
all the relevant code is open source. go for it!
~amittai
On 6/19/15 12:49, Read, James C wrote:
So, all I did was filter out the less likely phrase
On Wed, Jun 24, 2015 at 8:11 AM, Read, James C jcr...@essex.ac.uk wrote:
Other than that it seems painfully clear that the point I meant to make
has not been understood entirely. If the default behaviour produces BLEU
scores considerably lower than merely selecting the most likely translation
it.
Thanks.
-Original Message-
From: moses-support-boun...@mit.edu [mailto:moses-support-boun...@mit.edu] On
Behalf Of Read, James C
Sent: Wednesday, June 24, 2015 10:29 AM
To: John D. Burger
Cc: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
Please allow me
poorly.
James
From: amittai axelrod amit...@umiacs.umd.edu
Sent: Wednesday, June 24, 2015 5:36 PM
To: Read, James C; Lane Schwartz
Cc: moses-support@mit.edu; Philipp Koehn
Subject: Re: [Moses-support] Major bug found in Moses
what *i* would do
rico.sennr...@gmx.ch
Sent: Friday, June 19, 2015 8:25 PM
To: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
[sorry for the garbled message before]
you are right. The idea is pretty obvious. It roughly corresponds to
'Histogram pruning' in this paper:
Zens, R., Stanton, D
On Wed, Jun 24, 2015 at 9:05 AM, Read, James C jcr...@essex.ac.uk wrote:
As the title of this thread makes clear the purpose of reporting the bug
was not to invite a discussion about conclusions made in my draft paper.
Clearly a community that builds its career around research in SMT is
: [Moses-support] Major bug found in Moses
On Jun 24, 2015, at 10:47 , Read, James C jcr...@essex.ac.uk wrote:
So you still think it's fine that the default would perform at 37 BLEU points
less than just selecting the most likely translation of each phrase?
Yes, I'm pretty sure we all think
PM
To: Read, James C
Cc: Rico Sennrich; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
On Wed, Jun 24, 2015 at 9:05 AM, Read, James C jcr...@essex.ac.uk wrote:
As the title of this thread makes clear the purpose of reporting the bug was
not to invite
: moses-support-boun...@mit.edu on behalf of Read, James C
jcr...@essex.ac.uk
Date: Wednesday 24 June 2015 17:29
To: John D. Burger j...@mitre.org
Cc: moses-support@mit.edu moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
Please allow me to give a synthesis of my
O.
- Original Message -
From: Marcin Junczys-Dowmunt junc...@amu.edu.pl
To: moses-support@mit.edu
Sent: Friday, 19 June, 2015 19:21:45
Subject: Re: [Moses-support] Major bug found in Moses
On that interesting idea that moses should be naturally good at
translating things, just
-support@mit.edu');
*Subject:* Re: [Moses-support] Major bug found in Moses
On Fri, Jun 19, 2015 at 11:28 AM, Read, James C jcr...@essex.ac.uk
javascript:_e(%7B%7D,'cvml','jcr...@essex.ac.uk'); wrote:
What I take issue with is the en-masse denial that there is a problem
with the system
On 19/06/15 19:21, Marcin Junczys-Dowmunt wrote:
So, if anything, Moses is just a very flexible text-rewriting tool.
Tuning (and data) turns into a translator, GEC tool, POS-tagger,
Chunker, Semantic Tagger etc.
that's a good point, and the basis of some criticism that can be
levelled at the
I like the idea very much. I would need to discuss this with my
collegues, but I guess we can publish recipes for the MT engines we use
in production at WIPO and the UN. They are modelled after some of your
WMT systems, but tuned for speed and small size.
On 20.06.2015 15:42, Adam Lopez wrote:
Can and
should we make a wider effort to facilitate the reproduction of systems
by disseminating settings or configuration files? This dissemination is
partially done by system description papers, but they cannot cover all
settings [this would make for a very boring paper]. I put some effort
To: Read, James C
Cc: Marcin Junczys-Dowmunt;
moses-support@mit.edumailto:moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
I think you are misunderstanding how decoding works. The highest-weighted
translation of each source phrase is not necessarily
: Wednesday, June 17, 2015 5:25 PM
To: Read, James C
Cc: Marcin Junczys-Dowmunt; moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
I think you are misunderstanding how decoding works. The
highest-weighted translation of each source phrase
...@mit.edu on behalf
of Burger, John D. j...@mitre.org
Sent: Thursday, June 18, 2015 6:32 PM
To: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
On Jun 17, 2015, at 11:54, Read, James C jcr...@essex.ac.uk wrote:
The question remains why isn't the system capable of finding
Sent: Friday, June 19, 2015 5:08 PM
To: Read, James C
Cc: Hieu Hoang; moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
Hi James,
Yes, he just said that.
The decoder's job is to find the hypothesis with the maximum model
score. That's one reason why your work
, Doug
Subject: Re: [Moses-support] Major bug found in Moses
James,
The underlying questions that you appear to be posing are these: When the
search space is simplified by decoding without a language model, to what extent
is the decoder able to identify hypotheses that have the best model score
Sent: Thursday, June 18, 2015 9:39 PM
To: Burger, John D.
Cc: moses-support@mit.edumailto:moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
Hi,
I am great fan of open source software, but there is a danger to
view its inner workings as a black box - which leads
-support] Major bug found in Moses
James,
You may see the techniques that exist as outdated, wrong-headed, and
inefficient. You have the right to hold that opinion. It may even be that
history proves you right. Progress in science is made by people posing
questions - often questions
:* Re: [Moses-support] Major bug found in Moses
Hi,
I am great fan of open source software, but there is a danger to
view its inner workings as a black box - which leads to the
strange theories of what is going on, instead of real understanding.
But we can try to understand
*Subject:* Re: [Moses-support] Major bug found in Moses
Hi,
I am great fan of open source software, but there is a danger to
view its inner workings as a black box - which leads to the
strange theories of what is going on, instead of real understanding.
But we can try
James,
1) Acknowledging that the search algorithm performs poorly with no LM,
tuning or pruning despite the fact the search space clearly contains high
quality translations
Yes. We all acknowledge this. If you have a better technique, that's great.
Show that it's better. Your paper does not
That's actually presentation-worthy material :) I have to note that down
somewhere.
W dniu 2015-06-19 16:23, amittai axelrod napisał(a):
speaking of cobbling together a good translation from imperfect parts:
google:
A motorist heard on the radio the announcement: Caution Caution On
with this level of denial.
James
From: amittai axelrod amit...@umiacs.umd.edu
Sent: Friday, June 19, 2015 5:33 PM
To: Read, James C; Lane Schwartz
Cc: moses-support@mit.edu; Philipp Koehn
Subject: Re: [Moses-support] Major bug found in Moses
* [i'm] the guy that says
On Fri, Jun 19, 2015 at 11:28 AM, Read, James C jcr...@essex.ac.uk wrote:
What I take issue with is the en-masse denial that there is a problem with
the system if it behaves in such a way with no LM + no pruning and/or
tuning.
There is no mass denial taking place.
Regardless of whether or
...@gmail.com
Sent: Friday, June 19, 2015 6:10 PM
To: Read, James C
Cc: Philipp Koehn; Burger, John D.; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
James,
1) Acknowledging that the search algorithm performs poorly with no LM, tuning
or pruning despite the fact the search
?
James
From: Lane Schwartz dowob...@gmail.com
Sent: Friday, June 19, 2015 7:40 PM
To: Read, James C
Cc: Philipp Koehn; Burger, John D.; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
On Fri, Jun 19, 2015 at 11:28 AM, Read, James C
:* Friday, June 19, 2015 7:40 PM
*To:* Read, James C
*Cc:* Philipp Koehn; Burger, John D.; moses-support@mit.edu
*Subject:* Re: [Moses-support] Major bug found in Moses
On Fri, Jun 19, 2015 at 11:28 AM, Read, James C jcr...@essex.ac.uk
mailto:jcr...@essex.ac.uk wrote:
What I take issue
P.S. Have a good weekend everybody. Be back in action in a couple of days.
James
From: Read, James C
Sent: Friday, June 19, 2015 7:49 PM
To: Lane Schwartz
Cc: Philipp Koehn; Burger, John D.; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found
Hi Rico,
since you are at it, some pointers to the more advanced pruning
techniques that do perform better, please :)
On 19.06.2015 19:25, Rico Sennrich wrote:
[sorry for the garbled message before]
you are right. The idea is pretty obvious. It roughly corresponds to
'Histogram pruning' in
Marcin Junczys-Dowmunt junczys@... writes:
Hi Rico,
since you are at it, some pointers to the more advanced pruning
techniques that do perform better, please :)
On 19.06.2015 19:25, Rico Sennrich wrote:
[sorry for the garbled message before]
you are right. The idea is pretty
On that interesting idea that moses should be naturally good at
translating things, just for general considerations.
Since some said this thread has educational value I would like to share
something that might not be obvious due to the SMT-biased posts here.
Moses is also the _leading_ tool
Read, James C jcread@... writes:
So, all I did was filter out the less likely phrase pairs and the BLEU
score shot up. Was that such a stroke of genius? Was that not blindingly
obvious?
you are right. The idea is pretty obvious. It roughly corresponds to
'Histogram pruning' in this paper:
[sorry for the garbled message before]
you are right. The idea is pretty obvious. It roughly corresponds to
'Histogram pruning' in this paper:
Zens, R., Stanton, D., Xu, P. (2012). A Systematic Comparison of Phrase
Table Pruning Technique. In Proceedings of the 2012 Joint Conference on
Ah OK, I misunderstood, I thought you were talking about more advanced
pruning techniques compared to the significance method from Johnson et
al. while you only referred to the 30-best variant.
Cheers,
Marcin
On 19.06.2015 19:35, Rico Sennrich wrote:
Marcin Junczys-Dowmunt junczys@... writes:
From: Matthias Huck mh...@inf.ed.ac.uk
Sent: Friday, June 19, 2015 5:08 PM
To: Read, James C
Cc: Hieu Hoang; moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
Hi James,
Yes, he just said that.
The decoder's job is to find
...@gmail.com
Sent: Wednesday, June 17, 2015 8:34 PM
To: Read, James C; Kenneth Heafield; moses-support@mit.edu
Cc: Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
4 BLEU is nothing to sniff at :) I was answering Ken's tangent aspersion
that LM are needed for tuning.
I have some sympathy
Cc: Arnold, Doug; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
To paint you a picture:
Imagine you have a rat in a labyrinth (the labyrinth is the TM and the search
space). That rat is quite good at finding the center of that labyrinth. Now you
somehow disable
...@kheafield.com
Sent: Wednesday, June 17, 2015 7:13 PM
To: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
I'll bite.
The moses.ini files ship with bogus feature weights. One is required to
tune the system to discover good weights for their system. You did not
tune
; Hieu Hoang; Kenneth Heafield; moses-support@mit.edu
Cc: Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
hi --
you might not be aware, but your emails sound almost belligerently
confrontational. i can see how you would be frustrated, but starting a
conversation with i have
: Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
4 BLEU is nothing to sniff at :) I was answering Ken's tangent aspersion
that LM are needed for tuning.
I have some sympathy for you. You're looking at ways to improve
translation by reducing the search space. I've bashed my
To: Read, James C; Kenneth Heafield; moses-support@mit.edu
Cc: Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
4 BLEU is nothing to sniff at :) I was answering Ken's tangent aspersion
that LM are needed for tuning.
I have some sympathy for you. You're looking at ways
From: moses-support-boun...@mit.edu moses-support-boun...@mit.edu on behalf
of Hieu Hoang hieuho...@gmail.com
Sent: Wednesday, June 17, 2015 7:35 PM
To: Kenneth Heafield; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
On 17/06/2015 20:13, Kenneth Heafield wrote
Hi all,
I tried unsuccessfully to publish experiments showing this bug in Moses
behaviour. As a result I have lost interest in attempting to have my work
published. Nonetheless I think you all should be aware of an anomaly in Moses'
behaviour which I have thoroughly exposed and should be easy
Junczys-Dowmunt
Cc: Moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
You would expect an improvement of 37 BLEU points?
James
From: Marcin Junczys-Dowmunt junc...@amu.edu.pl
Sent: Wednesday, June 17, 2015 4:32 PM
To: Read
-
FROM: Marcin Junczys-Dowmunt junc...@amu.edu.pl
SENT: Wednesday, June 17, 2015 4:32 PM
TO: Read, James C
CC: Moses-support@mit.edu; Arnold, Doug
SUBJECT: Re: [Moses-support] Major bug found in Moses
Hi James,
there are many more factors involved than just
From: Marcin Junczys-Dowmunt junc...@amu.edu.pl
Sent: Wednesday, June 17, 2015 5:12 PM
To: Read, James C
Cc: Arnold, Doug; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
Hi James
No, not at all. I would say that is expected behaviour. It's how search spaces
Read, James C jcread@... writes:
I have been unable to find a logical explanation for this behaviour other
than to conclude that there must be some kind of bug in Moses which causes a
TM only run of Moses to perform poorly in finding the most likely
translations according to the TM when
there
Hi James,
there are many more factors involved than just probability, for instance
word penalties, phrase penalities etc. To be able to validate your own
claim you would need to set weights for all those non-probabilities to
zero. Otherwise there is no hope that moses will produce anything
To: Marcin Junczys-Dowmunt
Cc: Moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
You would expect an improvement of 37 BLEU points?
James
From: Marcin Junczys-Dowmunt junc...@amu.edu.pl
Sent: Wednesday, June 17, 2015 4:32
...@amu.edu.pl
SENT: Wednesday, June 17, 2015 5:12 PM
TO: Read, James C
CC: Arnold, Doug; moses-support@mit.edu
SUBJECT: Re: [Moses-support] Major bug found in Moses
Hi James
No, not at all. I would say that is expected behaviour. It's how search
spaces and optimization works. If anything
-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
On 17/06/2015 20:13, Kenneth Heafield wrote:
I'll bite.
The moses.ini files ship with bogus feature weights. One is required to
tune the system to discover good weights for their system. You did not
tune. The results
moses-support-boun...@mit.edu on
behalf of Hieu Hoang hieuho...@gmail.com
Sent: Wednesday, June 17, 2015 7:35 PM
To: Kenneth Heafield; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
On 17/06/2015 20:13, Kenneth Heafield wrote:
I'll bite.
The moses.ini files
No.
James
From: Lane Schwartz dowob...@gmail.com
Sent: Wednesday, June 17, 2015 7:58 PM
To: Read, James C
Cc: Hieu Hoang; Kenneth Heafield; moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
James,
Did you run any
Sent: Wednesday, June 17, 2015 7:13 PM
To: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
I'll bite.
The moses.ini files ship with bogus feature weights. One is required to
tune the system to discover good weights for their system. You did not
tune. The results
Read, James C jcread@... writes:
Actually the approximation I expect to be:
p(e|f)=p(f|e)
Why would you expect this to give poor results if the TM is well trained?
Surely the results of my filtering
experiments provve otherwise.
James
I recommend you read the following:
;;
}
}
From: moses-support-boun...@mit.edu moses-support-boun...@mit.edu on behalf
of Rico Sennrich rico.sennr...@gmx.ch
Sent: Wednesday, June 17, 2015 7:17 PM
To: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
Read
Sent: Wednesday, June 17, 2015 5:23 PM
To: Read, James C; Marcin Junczys-Dowmunt
Cc: Moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
Hi,
BLEU scores don't mean much, unless you know what the translations look like.
Marcin's explanation sounds very
-boun...@mit.edu on behalf
of Rico Sennrich rico.sennr...@gmx.ch
Sent: Wednesday, June 17, 2015 5:32 PM
To: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses
Read, James C jcread@... writes:
I have been unable to find a logical explanation for this behaviour other
than
;;
}
}
From: Matt Post p...@cs.jhu.edu
Sent: Wednesday, June 17, 2015 5:25 PM
To: Read, James C
Cc: Marcin Junczys-Dowmunt; moses-support@mit.edu; Arnold, Doug
Subject: Re: [Moses-support] Major bug found in Moses
I think you are misunderstanding how decoding works
provve otherwise.
James
From: moses-support-boun...@mit.edu moses-support-boun...@mit.edu on behalf
of Rico Sennrich rico.sennr...@gmx.ch
Sent: Wednesday, June 17, 2015 5:32 PM
To: moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found
69 matches
Mail list logo
