[Moses-support] mkcls - usable for bilingual class creation?

[Allen Smith]

Hi. The original Och paper (at least the one not in, IIRC, German) that's
referenced for mkcls (at http://www.fjoch.com/mkcls.html) includes a
description of using a starting alignment to improve the word
classification. Is this indeed doable with mkcls, and if so, how?

Thanks,

-Allen (Allen W. Smith, Ph.D.)
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Eliminating load times for MOSES phrase table, language model and reordering model

[Dingyuan Wang]

Try Compact Table? http://www.statmt.org/moses/?n=Advanced.RuleTables#ntoc3

2017年4月5日 下午7:33，"RR" 寫道：

> Hi,
>
> I am currently working on a system to explore MOSES decoding parameter
> space - running MOSES in decoding mode with one parameter set, getting a
> BLEU score and rerunning with different parameters.
>
> More than 50% of my time is being spent repeatedly "loading" the phrase
> table, language model and reordering model. For example,
>
> $ ./example.fr | .../moses -f ./run11.moses.ini -threads 31   >
> ./trans_baseline
>
> gives: "Start loading text phrase table. Moses format : [37.296] seconds."
>
> 37 seconds just to load a 590MB phrase table, even if it is on RAMDISK.
> htop indicates that for those 37 seconds, a single core is 100% utilized.
>
> My understanding is that some kind of optimized data structure (a hash
> table of some kind?) is being created in those 37 seconds, which is then
> lost and recreated when I re-run MOSES with a different value of a decoding
> parameter.
>
> If I want to eliminate this loading time, what is the best way forward? Is
> there a phrase table format that avoids a significant loading time (i.e. a
> 500MB phrase table loads in < 1 sec)? Should I try to find a way to run a
> single MOSES server but with different decoding parameters? I have a system
> with 130GB RAM and I am using a small phrase table that has been filtered
> for my decoding set.
>
> Thanks in advance,
>
> Roko
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Eliminating load times for MOSES phrase table, language model and reordering model

[RR]

Hi,

I am currently working on a system to explore MOSES decoding parameter
space - running MOSES in decoding mode with one parameter set, getting a
BLEU score and rerunning with different parameters.

More than 50% of my time is being spent repeatedly "loading" the phrase
table, language model and reordering model. For example,

$ ./example.fr | .../moses -f ./run11.moses.ini -threads 31   >
./trans_baseline

gives: "Start loading text phrase table. Moses format : [37.296] seconds."

37 seconds just to load a 590MB phrase table, even if it is on RAMDISK.
htop indicates that for those 37 seconds, a single core is 100% utilized.

My understanding is that some kind of optimized data structure (a hash
table of some kind?) is being created in those 37 seconds, which is then
lost and recreated when I re-run MOSES with a different value of a decoding
parameter.

If I want to eliminate this loading time, what is the best way forward? Is
there a phrase table format that avoids a significant loading time (i.e. a
500MB phrase table loads in < 1 sec)? Should I try to find a way to run a
single MOSES server but with different decoding parameters? I have a system
with 130GB RAM and I am using a small phrase table that has been filtered
for my decoding set.

Thanks in advance,

Roko
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Support Moses and GPU on cloud

[liling tan]

Dear Marcin and Moses community,

Thanks for the tips!

Yeah, g2.8xlarge is painfully expensive...  Training on separate instances
sounds more reasonable. Now, I've to explain to the devs why I need 2
instances ;P

Regards,
Liling

On Tue, Apr 4, 2017 at 4:25 PM, liling tan  wrote:

> Dear Marcin and Moses community,
>
> Are you running on g2.8xlarge on AWS?
>
> I think I went to the cheap g2.2xlarge and 15GB RAM is a little too low
> for MGIZA++ , taking forever... I think I've got to recreate a new larger
> instance.
>
> Regards,
> Liling
>
>
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support