[Moses-support] equal sign (=) in LM path

Tom Hoar Wed, 04 Feb 2015 04:00:22 -0800

Using R-3 and I got the same equal sign exception as below. I attached two moses.ini/moses.log file sets. The first example successfully loads the phrase & reordering tables with paths with equal signs. and a KENLM path without an equal sign.

The second example fails to load after simply changing the KENLM path to a location with an equal sign.

So Hieu, your comment "I think it's fixed" looks true for the phrase/reordering references, but not true for the LM references.


Tom



On 02/04/2015 01:54 PM, Lisa Kim wrote:



Thank you.
It is successful to change the LM name without "=" symbol.

However, decoding still fails with following message.
Could you check the error message?

--------------------
Start loading text phrase table. Moses  format : [0.038] seconds
terminate called recursively
terminate called after throwing an instance of 'util::Exception'
  what():  Aborted (core dumped)
Exit code: 134

The decoder died. CONFIG WAS -weight-overwrite 'LM1= 0.125000 PhrasePenalty0= 0.050000 WordPenalty0= -0.250000 TranslationModel0= 0.050000 0.050000 0.050000 0.050000 TranslationModel1= 0.250000 LM0= 0.125000'

2015. 2. 3. 오후 8:12에 "Hieu Hoang" <hieuho...@gmail.com <mailto:hieuho...@gmail.com>> 님이 작성:


    as ken mentioned, don't use the = sign in the name of your LM
    file. I think it's fixed in the latest version of moses. Which
    version do you have, from when ago?

    moses_chart should handle factors. Again, if you version is from
    long ago, I suggest your update

    On 02/02/15 10:06, ekkim214 wrote:


    I want to use some factor (class) in the hierarchical model.
    I am using the ems script and using KenLM for surface and factor
    both.
    The building of LM is successful with settings
    "-discount_fallback" but
    I got an error during tuning phase like this.

    Exception: moses/LM/Ken.cpp:399 in Moses::LanguageModel*
    Moses::ConstructKenLM(const string&) threw util::Exception
    because `args.size() != 2'.
    Incorrect format of KenLM property:
    
path=/home3/ekkim/working/E2K_1501/hierarchical_class/lm/travel22=mkcls.binlm.5
    Exit code: 1

    Does not "moses_chart" decoder which is distributed in the
    package support factored training?


    _______________________________________________
    Moses-support mailing list
    Moses-support@mit.edu  <mailto:Moses-support@mit.edu>
    http://mailman.mit.edu/mailman/listinfo/moses-support




_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Defined parameters (per moses.ini or switch):
	config: run0.moses.2.ini 
	distortion-limit: 6 
	feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/phrase-table.gz input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/reordering-table.wbe-msd-bidirectional-fe.gz Distortion KENLM lazyken=1 name=LM0 factor=0 order=0 path=/opt/domt/MODELS/smt/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3-l=micro_lm-T=kenlm-n=3/LM0/kenlm.en_gb.bin 
	input-factors: 0 
	mapping: 0 T 0 
	weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3 Distortion0= 0.3 LM0= 0.5 
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/phrase-table.gz input-factor=0 output-factor=0
FeatureFunction: TranslationModel0 start: 3 end: 6
line=LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/reordering-table.wbe-msd-bidirectional-fe.gz
FeatureFunction: LexicalReordering0 start: 7 end: 12
Initializing LexicalReordering..
line=Distortion
FeatureFunction: Distortion0 start: 13 end: 13
line=KENLM lazyken=1 name=LM0 factor=0 order=0 path=/opt/domt/MODELS/smt/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3-l=micro_lm-T=kenlm-n=3/LM0/kenlm.en_gb.bin
Exception: moses/LM/Ken.cpp:451 in Moses::LanguageModel* Moses::ConstructKenLM(const string&) threw util::Exception because `args.size() != 2'.
Incorrect format of KenLM property: path=/opt/domt/MODELS/smt/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3-l=micro_lm-T=kenlm-n=3/LM0/kenlm.en_gb.bin

run0.moses.2.ini
Description: application/wine-extension-ini

Defined parameters (per moses.ini or switch):
	config: run0.moses.1.ini 
	distortion-limit: 6 
	feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/phrase-table.gz input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/reordering-table.wbe-msd-bidirectional-fe.gz Distortion KENLM lazyken=1 name=LM0 factor=0 order=0 path=/home/tahoar/Desktop/domt/LM0/kenlm.en_gb.bin 
	input-factors: 0 
	mapping: 0 T 0 
	weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3 Distortion0= 0.3 LM0= 0.5 
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/phrase-table.gz input-factor=0 output-factor=0
FeatureFunction: TranslationModel0 start: 3 end: 6
line=LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/reordering-table.wbe-msd-bidirectional-fe.gz
FeatureFunction: LexicalReordering0 start: 7 end: 12
Initializing LexicalReordering..
line=Distortion
FeatureFunction: Distortion0 start: 13 end: 13
line=KENLM lazyken=1 name=LM0 factor=0 order=0 path=/home/tahoar/Desktop/domt/LM0/kenlm.en_gb.bin
FeatureFunction: LM0 start: 14 end: 14
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LexicalReordering0
Loading table into memory...done.
Loading Distortion0
Loading LM0
Loading TranslationModel0
Start loading text phrase table. Moses format : [1.756] seconds
Reading /opt/domt/TRAININGS/smt/tm-en_gb-nl_nl-micro_tm/domt-s=nl_nl-t=en_gb-p=micro_tm-a=giza_grow-diag-final-and-g=3/phrase-table.gz
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Created input-output object : [2.843] seconds

run0.moses.1.ini
Description: application/wine-extension-ini

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] equal sign (=) in LM path

Reply via email to