Moses2 doesn't support the Compact phrase-table, only ProbingPT.
You have to rebinarize the phrase-table with CreateProbingPT. It's
better if you also integrate the lexicalised reordering table the
phrase-table. The script
scripts/generic/binarize4moses2.perl
does this all in 1 go
On 13/05/2017 04:14, Steve Braich wrote:
Hello,
I am trying to improve the performance of Moses in a server
environment (not training).
I have tried various things, including the following which is
described on http://www.statmt.org/moses/?n=Moses.Optimize.
It has had no impact:
* *Multi-threading: *Up to 16 Cores on Google's Cloud
I used the option -threads all and -threads 16
* *Memory:*
I have launched Moses with as much as 60 GB of memory
* *Caching the Models*
Note: I only have the following files that I understand to be what
you need to translate, including Phrase Table, Reordering Table,
and target language model.
Am I missing anything? Your documentation suggests that I might
at http://www.statmt.org/moses/?n=Moses.Optimize#ntoc2
<http://www.statmt.org/moses/?n=Moses.Optimize>. I see these other
files in my EMS directory but I get no errors without them.
cat /home/autom8tr8n/mt_publish/model_enzh/phrase-table.minphr >
/dev/null
cat
/home/autom8tr8n/mt_publish/model_enzh/reordering-table.minlexr >
/dev/null
cat /home/autom8tr8n/mt_publish/model_enzh/target.blm.zh > /dev/null
* *Memory:
*I have launched Moses with as much as 60 GB of memory
* *Other:*
I made sure "transparent huge pages" are enabled, and I use the
compact phrase and reordering tables.
So, I am now trying Moses2 and I am following the instructions here
(http://www.statmt.org/moses/?n=Site.Moses2). I am having some issues
with this. Here are my questions:
* *Does the Moses executable have to match for Training and Server?*
In other words, do I have to train the models with Moses2 if I
want to run them with Moses2 Server?
* *Moses2 Error Messages:*
I tried just running it with Moses2 (without server and just basic
options):
~/mosesdecoder/bin/moses2 -f ~/mt_publish/model_enzh/moses.ini
I get this error message:
Starting...
Defined parameters (per moses.ini or switch):
config: /home/autom8tr8n/mt_publish/model_enzh/moses.ini
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/autom8tr8n/mt_publish/model_enzh/phrase-table.minphr
input-factor=0 output-factor=0 LexicalReordering
name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/autom8tr8n/mt_publish/model_enzh/reordering-table
Distortion KENLM name=LM0 factor=0
path=/home/autom8tr8n/mt_publish/model_enzh/target.blm.zh order=3
input-factors: 0
mapping: 0 T 0
mark-unknown:
server:
server-port: 3001
unknown-word-prefix: __UNK__
unknown-word-suffix: __UNK__
weight: LexicalReordering0= 0.0892686 0.0768025 0.0930783
0.0716109 0.00303172 0.0492278 Distortion0= 0.0399769 LM0=
0.107799 WordPenalty0= -0.242343 PhrasePenalty0= 0.0403541
TranslationModel0= 0.0396487 0.0448175 0.0681687 0.0338723
UnknownWordPenalty0= 1
*Feature name PhraseDictionaryCompact is not registered.Aborted
(core dumped)
^This error suggests that I didn't compile with CMPH. I did. See
below.*
* *Compilation*
From the file dates, Moses2 appears to have been compiled when I
initially compiled Moses (1). Here is how I compiled it:
./bjam -a --with-boost=/home/autom8tr8n/mosesdecoder/boost_1_63_0
--with-cmph=/home/autom8tr8n/mosesdecoder/cmph/cmph2.0
--with-xmlrpc-c=/home/autom8tr8n/mosesdecoder/opt
Where is the compilation log file where I can see what Moses 2 was
compiled with?
Please answer my questions about Moses2, look over the other
optimization I did, and please, I am all ears if you have other
suggestions to boost performance. My very unscientific manual testing
has shown about 1.2 seconds to translate a phrase. No improvement at
all has been made with any optimization that I have tried so far. A
VM with a single core and 3.75 GB of memory performs just as good as a
16 core, 60 GB memory VM.
Thanks in advance,
Steve
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://moses-smt.org/
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
