Hi Amir
Even if you use IRSTLM to build the language model, you can still use
KenLM for decoding. Make sure you create an arpa file with IRSTLM, then
use build_binary to binarise it so that it loads quickly with KenLM.
Then you can use multi-threaded decoding,
cheers - Barry
On 14/02/14 13:01, amir haghighi wrote:
Thank you Barry,
I use IRSTLM to build the language model. Can I use multi-thread for
decoder-setting?
I get "IRST LM is not threadsafe" error.
I want to use IRSTLM, is there any other way to speed up the tuning step ?
Regrads
On Fri, Feb 14, 2014 at 1:53 AM, Barry Haddow
<bhad...@staffmail.ed.ac.uk <mailto:bhad...@staffmail.ed.ac.uk>> wrote:
Hi Amir
You can add
decoder-settings = "-threads 4"
to your TUNING stanza.
Also try
filter-settings = "-MinScore 2:0.0001"
for more aggressive filtering.
Running tuning on a laptop though is always going to be slow,
cheers - Barry
On 14/02/14 09:26, amir haghighi wrote:
Thank you arezki and yohit
I don't know how can I change multi-thread setting in ems config
file.
On Fri, Feb 14, 2014 at 12:36 AM, Arezki Sadoune
<arezkisado...@yahoo.fr <mailto:arezkisado...@yahoo.fr>> wrote:
Hello Amir
I think your tuning process will go faster if you use a
multi-threaded Mert.
/home/mert-moses.pl <http://mert-moses.pl> --threads 4
you have of course tu indicate 8 instead of 4 if your laptop
is equipped with eight cores
Best regards
Le Vendredi 14 février 2014 8h27, amir haghighi
<amir.haghighi...@gmail.com
<mailto:amir.haghighi...@gmail.com>> a écrit :
Hello
I have a corpus with 400'000 sentences for training, 1000
sentences for tuning and 100'000 sentences for test. I
couldn't run ems on my corpus, after 3 days, with my old laptop.
I have bought a new laptop (core i7, cpu 2.40 , 8G Ram) but I
can't still run ems! it is 3 days that it is in the tuning
step and it is not finished yet.
Is it possible that it gets in an endless loop?
How can I check it's process?
regards
Amir
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
