Re: [Moses-support] Moses on a mac
This seems to be a gcc problems caused by one of my recent commits. I've undone the change until I can find out why. apologies Hieu Hoang www.hoang.co.uk/hieu -Original Message- From: moses-support-boun...@mit.edu [mailto:moses-support-boun...@mit.edu] On Behalf Of Kemal Oflazer Sent: 04 March 2009 07:31 To: moses-support@mit.edu Subject: [Moses-support] Moses on a mac Dear All I just install moses on a large mac system and wanted to test out an earlier setup. Train went just fine but moses dies with Start loading LanguageModel /Users/oflazer/smt/models/lm/smorph-lm-n5.lm : [0.000] seconds pure virtual method called terminate called without an active exception Abort trap this seems to be perhaps related to srilm (does not seem to be loeading the file) which is properly installed. Is there anything special to mac that I need to be careful about? Thanks Kemal - Kemal Oflazer http://people.sabanciuniv.edu/oflazer/ ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Error in running moses with randlm
Yeah, sorry about this- I broke moses, at least for certain compilers. I'll fix it shortly. -Chris On Wed, Mar 4, 2009 at 12:17 PM, Miles Osborne mi...@inf.ed.ac.uk wrote: ok, it seems that the most recent version of Moses had a bad commit and broke the language model interface. so, this is not really anything to do with RandLM as such. Miles 2009/2/26 Michael Zuckerman michael90...@gmail.com: Hi, As you said, I tried again with europarl used for training the language model, but I get the same error: Start loading LanguageModel /home/michez/alfabetic/lm/randlm/test/model.BloomMap : [0.000] seconds pure virtual method called terminate called without an active exception Aborted For creating the language model I ran: $ ../bin/buildlm -struct BloomMap -falsepos 8 -values 8 -output-prefix model -input-path ../../europarl.lower.token.en.gz Thank you for your help, Michael. - Show quoted text - On Tue, Feb 24, 2009 at 8:36 PM, Miles Osborne mi...@inf.ed.ac.uk wrote: can you try it again with a large amount of data for training the language model? in the past i've noticed that it doesn't work very well with minute numbers of sentences. try europarl (i get a different error message, but it might be the same thing) Miles 2009/2/24 Michael Zuckerman michael90...@gmail.com: Hi, I am running moses on a small example containing two german sentences (in file in): das ist ein kleines haus das ist ein kleines haus I am using the attached randlm language model model.BloomMap, and the attached phrase table and moses.ini files. My command line is: $ ../../../../mosesdecoder/moses-cmd/src/moses -f moses.ini in out When loading the language model, moses gives an error: Defined parameters (per moses.ini or switch): config: moses.ini input-factors: 0 lmodel-file: 5 0 3 /home/michez/alfabetic/lm/randlm/test/model.BloomMap mapping: T 0 ttable-file: 0 0 1 phrase-table ttable-limit: 10 weight-d: 1 weight-l: 1 weight-t: 1 weight-w: 0 Added ScoreProducer(0 Distortion) index=0-0 Added ScoreProducer(1 WordPenalty) index=1-1 Added ScoreProducer(2 !UnknownWordPenalty) index=2-2 Loading lexical distortion models... have 0 models Start loading LanguageModel /home/michez/alfabetic/lm/randlm/test/model.BloomMap : [0.000] seconds pure virtual method called terminate called without an active exception Aborted Do you have a clue how to handle this error ? Thanks, Michael. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] word alignment symmetrisation heuristics
one thing to remember is that the link between AER and BLEU is not obvious; in my view at least AER-like scores should be treated with skepticism and the real merit of an alignment approach should be the corresponding translation performance (BLEU etc). can you provide associated BLEU scores for those AER numbers? Miles 2009/3/4 J.Tiedemann j.tiedem...@rug.nl: hi, I'm just wondering if Och's refined heuristics is also implemented in Moses. The grow-diag is not exactly the same as far as I understand. The reason why I'm asking is because I found out that in all of my experiments with europarl data the intersection always produces the best results in terms of AER (for example using the wpt03 data) whereas I see better performances reported for refined compared with intersection in various papers (also for the wpt03 data). However, I cannot believe that the grow-heuristics would perform so much worse than the original refined approach. My AER scores with standard GIZA settings and moses heuristics for wpt03 data are the following: moses.intersect AER = 0.0613 moses.grow-diag AER = 0.0843 moses.grow-diag-final-and AER = 0.0926 moses.grow-diag-final AER = 0.1312 moses.srctotgt AER = 0.1039 moses.tgttosrc AER = 0.1162 moses.union AER = 0.1444 does this sound reasonable? Jorg ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] word alignment symmetrisation heuristics
it depends on what you want to do. I was interested in the word alignment in particular. not necessarily for running MT with moses. for SMT I usually just use the default grow-diag-final-and which probably gives the best input anyway. this is, I guess, because it's better on recall. AER seems to strongly prefer precision. jorg On Wed, 4 Mar 2009 13:46:36 + Miles Osborne mi...@inf.ed.ac.uk wrote: one thing to remember is that the link between AER and BLEU is not obvious; in my view at least AER-like scores should be treated with skepticism and the real merit of an alignment approach should be the corresponding translation performance (BLEU etc). can you provide associated BLEU scores for those AER numbers? Miles 2009/3/4 J.Tiedemann j.tiedem...@rug.nl: hi, I'm just wondering if Och's refined heuristics is also implemented in Moses. The grow-diag is not exactly the same as far as I understand. The reason why I'm asking is because I found out that in all of my experiments with europarl data the intersection always produces the best results in terms of AER (for example using the wpt03 data) whereas I see better performances reported for refined compared with intersection in various papers (also for the wpt03 data). However, I cannot believe that the grow-heuristics would perform so much worse than the original refined approach. My AER scores with standard GIZA settings and moses heuristics for wpt03 data are the following: moses.intersect AER = 0.0613 moses.grow-diag AER = 0.0843 moses.grow-diag-final-and AER = 0.0926 moses.grow-diag-final AER = 0.1312 moses.srctotgt AER = 0.1039 moses.tgttosrc AER = 0.1162 moses.union AER = 0.1444 does this sound reasonable? Jorg ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] word alignment symmetrisation heuristics
yep, that sounds reasonable. in that case it is good to remember that those heuristics are all designed for eventual translation and not for doing well at AER. i can easily imagine some other set of heuristics which will do well at word alignment-like tasks and not necessarily pan-out into good bleu scores etc. Miles 2009/3/4 J.Tiedemann j.tiedem...@rug.nl: it depends on what you want to do. I was interested in the word alignment in particular. not necessarily for running MT with moses. for SMT I usually just use the default grow-diag-final-and which probably gives the best input anyway. this is, I guess, because it's better on recall. AER seems to strongly prefer precision. jorg On Wed, 4 Mar 2009 13:46:36 + Miles Osborne mi...@inf.ed.ac.uk wrote: one thing to remember is that the link between AER and BLEU is not obvious; in my view at least AER-like scores should be treated with skepticism and the real merit of an alignment approach should be the corresponding translation performance (BLEU etc). can you provide associated BLEU scores for those AER numbers? Miles 2009/3/4 J.Tiedemann j.tiedem...@rug.nl: hi, I'm just wondering if Och's refined heuristics is also implemented in Moses. The grow-diag is not exactly the same as far as I understand. The reason why I'm asking is because I found out that in all of my experiments with europarl data the intersection always produces the best results in terms of AER (for example using the wpt03 data) whereas I see better performances reported for refined compared with intersection in various papers (also for the wpt03 data). However, I cannot believe that the grow-heuristics would perform so much worse than the original refined approach. My AER scores with standard GIZA settings and moses heuristics for wpt03 data are the following: moses.intersect AER = 0.0613 moses.grow-diag AER = 0.0843 moses.grow-diag-final-and AER = 0.0926 moses.grow-diag-final AER = 0.1312 moses.srctotgt AER = 0.1039 moses.tgttosrc AER = 0.1162 moses.union AER = 0.1444 does this sound reasonable? Jorg ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Moses-support Digest, Vol 29, Issue 2
yes, i'm working on extending moses to do chart decoding, a la Hiero. it should be ready in a few weeks, fingers crossed. currently doing it offline to avoid annoying everyone with commit emails but if u want a copy, or even chip in with the development, can certainly send u the code. lane - thanks for the heads up on the extractor 2009/3/4 Lane Schwartz dowob...@gmail.com: Date: Wed, 04 Mar 2009 14:39:28 +0100 From: J.Tiedemann j.tiedem...@rug.nl Subject: [Moses-support] hiero To: moses-support@mit.edu Message-ID: web-107598...@mail3.rug.nl Content-Type: text/plain;charset=utf-8;format=flowed one more question: I somehow remember that someone once said that a kind of Hiero system was planned to be integrated in Moses. Is that true? Maybe there is even a test version already available? I did some experiments with the SAMT toolkit but I have problems with the Hiero mode. Is there maybe another tool available? (I know - I'm lazy ...) cheers, Jorg Jorg, I know that Hieu and some others have been working on a hierarchical mode for Moses, but I'm not sure what the state of it is. There is another open source hierarchical decoder - Joshua. It's available on Sourceforge at http://sourceforge.net/projects/joshua. The documentation is somewhat lacking at the moment, but if you want to try it out, feel free to email me directly for instructions on how to run it. BTW, Hieu and whoever else was using the Joshua rule extractor - the version I showed you at the MT marathon had a major bug in one of the lexprobs. You should get the latest version and retry if you haven't. The bug is fixed in recent versions. We're going to try to check in documentation soon. Cheers, Lane ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] word alignment symmetrisation heuristics
Hi Joerg, grow would be the closest to Franz's refined methods, as far as I understand it. The methods vary in the number of alignment points added - so AER may not be the most interesting number, rather look at precision/recall. -phi On Wed, Mar 4, 2009 at 1:29 PM, J.Tiedemann j.tiedem...@rug.nl wrote: hi, I'm just wondering if Och's refined heuristics is also implemented in Moses. The grow-diag is not exactly the same as far as I understand. The reason why I'm asking is because I found out that in all of my experiments with europarl data the intersection always produces the best results in terms of AER (for example using the wpt03 data) whereas I see better performances reported for refined compared with intersection in various papers (also for the wpt03 data). However, I cannot believe that the grow-heuristics would perform so much worse than the original refined approach. My AER scores with standard GIZA settings and moses heuristics for wpt03 data are the following: moses.intersect AER = 0.0613 moses.grow-diag AER = 0.0843 moses.grow-diag-final-and AER = 0.0926 moses.grow-diag-final AER = 0.1312 moses.srctotgt AER = 0.1039 moses.tgttosrc AER = 0.1162 moses.union AER = 0.1444 does this sound reasonable? Jorg ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
