Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Hieu Hoang]

run this 1st
   cat 
/mnt/storage/Common/models/language_models/langmod.5gram.blm/mnt/storage/Common/models/translation_models/moses3_model/train/model/reordering-table.wbe-msd-bidirectional-fe.gz.*
 
 > /dev/null
before running mosesserver. Is there a speedup? If there's speedup, then 
it would suggest that IO is a problem

what happens if you use
moses
rather than
   mosesserver
? if there's a speedup, then the mosesserver is not able to cope with 
the number of connections

On 23/07/2015 18:19, Barry Haddow wrote:
> Hi Oren
>
> You can fit a lot of model in 50G RAM. It's worth looking at the 
> compact phrase and reordering models, pruning options, and kenlm 
> quantisation, and you might fit the models on one machine.
>
> As to the slowdown, a delay of 20s or so suggests that it's waiting on 
> I/O, or perhaps xmlrpc-c is queueing up requests or connections.
>
> cheers - Barry
>
> On 23/07/15 15:06, Oren wrote:
>> More details...
>>
>> Our NFS is mounted at /mnt/storage .
>>
>> The command to run moses server is:
>>
>> /mnt/storage/Common/mosesdecoder3/bin/mosesserver -f 
>> /mnt/storage/Common/models/translation_models/moses3_model/mert-work/moses.ini
>>  
>> mark-unknown -xml-input -threads 18 exclusive
>>
>> Here is the complete moses.ini (excluding comments):
>>
>>> [input-factors]
>>> 0
>>>
>>> [mapping]
>>> 0 T 0
>>>
>>> [distortion-limit]
>>> 6
>>>
>>> [feature]
>>> UnknownWordPenalty
>>> WordPenalty
>>> PhrasePenalty
>>> PhraseDictionaryMemory name=TranslationModel0 num-features=4 
>>> path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/phrase-table.gz
>>>  
>>> input-factor=0 output-factor=0
>>> LexicalReordering name=LexicalReordering0 num-features=6 
>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 
>>> path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/reordering-table.wbe-msd-bidirectional-fe.gz
>>> Distortion
>>> KENLM lazyken=0 name=LM0 
>>> path=/mnt/storage/Common/models/language_models/langmod.5gram.blmorder=5
>>>
>>> [threads]
>>> 6
>>>
>>> [weight]
>>>
>>> LexicalReordering0= 0.0268311 -0.0146878 0.0261305 0.0380759 
>>> 0.0118265 0.07479
>>> Distortion0= 0.074665
>>> LM0= 0.0972206
>>> WordPenalty0= -0.18469
>>> PhrasePenalty0= -0.212528
>>> TranslationModel0= 0.0184105 0.091358 0.112618 0.0161682
>>> UnknownWordPenalty0= 1
>>
>> On Wednesday, July 22, 2015, Oren > > wrote:
>>
>> Yes, we use a lot of RAM in our setup. But the improved response
>> time justifies it.
>>
>> Our language is on a nfs, bit we've been working this way with
>> moses 1 for some time with no problems (ten different machines
>> using the same language model file over nfs). Same for the
>> reordering model. Neither of them is in memory. Raising these
>> models into memory will require raising our already excessive RAM
>> requirements...
>>
>> Thanks again for the help.
>>
>> On Wednesday, July 22, 2015, Barry Haddow
>> > > wrote:
>>
>> Hi Oren
>>
>> I'm not aware of any threading problems with
>> PhraseDictionaryMemory, but not many people use it since it
>> takes up too much memory. Moses command line runs
>> multi-threaded using a thread pool.
>>
>> Is your language model on a local file system? Running it
>> over nfs can be bad. What about your reordering model? Are
>> you using the compact or the memory version?
>>
>> cheers - Barry
>>
>> On 22/07/15 15:09, Oren wrote:
>>> We have no swapping issues. I was asking if the use of
>>> an in-memory translation model might cause multithreading
>>> problems.
>>>
>>>
>>> I'm not sure how to replicate the problem on cmd moses,
>>> since it's purely a multithreading issue. Ca you run cmd
>>> moses multithreaded?
>>>
>>> I can't attach the complete moses.ini because it's on a
>>> separate network... But I copied below the stuff that looked
>>> relevant. I also tried to change the [threads] setting to
>>> 18, with no apparent effect.
>>>
>>> [input-factors]
>>> 0
>>>
>>> [mapping]
>>> 0 T 0
>>>
>>> [distortion-limit]
>>> 6
>>>
>>> [feature]
>>> UnknownWordPenalty
>>> WordPenalty
>>> PhrasePenalty
>>> PhraseDictionaryMemory name=TranslationModel0 num-features=4
>>> path=/phrase-table.gz input-factor=0 output-factor=0
>>> LexicalReordering 
>>> Distortion
>>> KENLM lazyken=0 name=LM0 path= order=5
>>>
>>> [threads]
>>> 6
>>>
>>> [weight]
>>>
>>> 
>>>
>>> On Tuesday, July 21, 2015, Hieu Hoang 
>>> wrote:
>>>
>>> is it possible you can make your moses.ini file
>>> available for us to see?
>>>
>>> do you know if the same problem occurs if you use the
>>> 

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Barry Haddow]

Hi Oren

You can fit a lot of model in 50G RAM. It's worth looking at the compact 
phrase and reordering models, pruning options, and kenlm quantisation, 
and you might fit the models on one machine.


As to the slowdown, a delay of 20s or so suggests that it's waiting on 
I/O, or perhaps xmlrpc-c is queueing up requests or connections.


cheers - Barry

On 23/07/15 15:06, Oren wrote:

More details...

Our NFS is mounted at /mnt/storage .

The command to run moses server is:

/mnt/storage/Common/mosesdecoder3/bin/mosesserver -f 
/mnt/storage/Common/models/translation_models/moses3_model/mert-work/moses.ini 
mark-unknown -xml-input -threads 18 exclusive


Here is the complete moses.ini (excluding comments):


[input-factors]
0

[mapping]
0 T 0

[distortion-limit]
6

[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4 
path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/phrase-table.gz 
input-factor=0 output-factor=0
LexicalReordering name=LexicalReordering0 num-features=6 
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 
path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/reordering-table.wbe-msd-bidirectional-fe.gz

Distortion
KENLM lazyken=0 name=LM0 
path=/mnt/storage/Common/models/language_models/langmod.5gram.blmorder=5


[threads]
6

[weight]

LexicalReordering0= 0.0268311 -0.0146878 0.0261305 0.0380759 
0.0118265 0.07479

Distortion0= 0.074665
LM0= 0.0972206
WordPenalty0= -0.18469
PhrasePenalty0= -0.212528
TranslationModel0= 0.0184105 0.091358 0.112618 0.0161682
UnknownWordPenalty0= 1


On Wednesday, July 22, 2015, Oren > wrote:


Yes, we use a lot of RAM in our setup. But the improved response
time justifies it.

Our language is on a nfs, bit we've been working this way with
moses 1 for some time with no problems (ten different machines
using the same language model file over nfs). Same for the
reordering model. Neither of them is in memory. Raising these
models into memory will require raising our already excessive RAM
requirements...

Thanks again for the help.

On Wednesday, July 22, 2015, Barry Haddow
> wrote:

Hi Oren

I'm not aware of any threading problems with
PhraseDictionaryMemory, but not many people use it since it
takes up too much memory. Moses command line runs
multi-threaded using a thread pool.

Is your language model on a local file system? Running it over
nfs can be bad. What about your reordering model? Are you
using the compact or the memory version?

cheers - Barry

On 22/07/15 15:09, Oren wrote:

We have no swapping issues. I was asking if the use of
an in-memory translation model might cause multithreading
problems.


I'm not sure how to replicate the problem on cmd moses, since
it's purely a multithreading issue. Ca you run cmd moses
multithreaded?

I can't attach the complete moses.ini because it's on a
separate network... But I copied below the stuff that looked
relevant. I also tried to change the [threads] setting to 18,
with no apparent effect.

[input-factors]
0

[mapping]
0 T 0

[distortion-limit]
6

[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/phrase-table.gz input-factor=0 output-factor=0
LexicalReordering 
Distortion
KENLM lazyken=0 name=LM0 path= order=5

[threads]
6

[weight]



On Tuesday, July 21, 2015, Hieu Hoang 
wrote:

is it possible you can make your moses.ini file available
for us to see?

do you know if the same problem occurs if you use the
command line moses, rather than mosesserver?


Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 21 July 2015 at 18:07, Barry Haddow
 wrote:


On 21/07/15 14:51, Oren wrote:

I am using the in-memory mode, using about 50GB of
RAM. (No swap issues as far as I can tell.) Could
that cause issues?


Yes, swapping would definitely cause issues - was
that your question?




I looked at the commit you linked to, but it doesn't
seem to be something configurable beyond the
-threads switch. Am I missing something?


The commit enables you to set the maximum number of
connections to be the same as the maximum number of
threads.



On Tuesday, July 21, 2015, Barry Haddow
   

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Oren]

More details...

Our NFS is mounted at /mnt/storage .

The command to run moses server is:

/mnt/storage/Common/mosesdecoder3/bin/mosesserver -f
/mnt/storage/Common/models/translation_models/moses3_model/mert-work/moses.ini
mark-unknown -xml-input -threads 18 exclusive

Here is the complete moses.ini (excluding comments):

[input-factors]
0

[mapping]
0 T 0

[distortion-limit]
6

[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/phrase-table.gz
input-factor=0 output-factor=0
LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/reordering-table.wbe-msd-bidirectional-fe.gz

Distortion
KENLM lazyken=0 name=LM0
path=/mnt/storage/Common/models/language_models/langmod.5gram.blm order=5


[threads]
6

[weight]

LexicalReordering0= 0.0268311 -0.0146878 0.0261305 0.0380759 0.0118265
0.07479
Distortion0= 0.074665
LM0= 0.0972206
WordPenalty0= -0.18469
PhrasePenalty0= -0.212528
TranslationModel0= 0.0184105 0.091358 0.112618 0.0161682
UnknownWordPenalty0= 1


On Wednesday, July 22, 2015, Oren  wrote:

> Yes, we use a lot of RAM in our setup. But the improved response time
> justifies it.
>
> Our language is on a nfs, bit we've been working this way with moses 1 for
> some time with no problems (ten different machines using the same language
> model file over nfs). Same for the reordering model. Neither of them is in
> memory. Raising these models into memory will require raising our already
> excessive RAM requirements...
>
> Thanks again for the help.
>
> On Wednesday, July 22, 2015, Barry Haddow  > wrote:
>
>>  Hi Oren
>>
>> I'm not aware of any threading problems with PhraseDictionaryMemory, but
>> not many people use it since it takes up too much memory. Moses command
>> line runs multi-threaded using a thread pool.
>>
>> Is your language model on a local file system? Running it over nfs can be
>> bad. What about your reordering model? Are you using the compact or the
>> memory version?
>>
>> cheers - Barry
>>
>> On 22/07/15 15:09, Oren wrote:
>>
>> We have no swapping issues. I was asking if the use of an in-memory
>> translation model might cause multithreading problems.
>>
>>
>>  I'm not sure how to replicate the problem on cmd moses, since it's
>> purely a multithreading issue. Ca you run cmd moses multithreaded?
>>
>>  I can't attach the complete moses.ini because it's on a separate
>> network... But I copied below the stuff that looked relevant. I also tried
>> to change the [threads] setting to 18, with no apparent effect.
>>
>>  [input-factors]
>> 0
>>
>>  [mapping]
>> 0 T 0
>>
>>  [distortion-limit]
>> 6
>>
>>  [feature]
>> UnknownWordPenalty
>> WordPenalty
>> PhrasePenalty
>> PhraseDictionaryMemory name=TranslationModel0 num-features=4
>> path=/phrase-table.gz input-factor=0 output-factor=0
>> LexicalReordering 
>> Distortion
>> KENLM lazyken=0 name=LM0 path= order=5
>>
>>  [threads]
>> 6
>>
>>  [weight]
>>
>>  
>>
>> On Tuesday, July 21, 2015, Hieu Hoang  wrote:
>>
>>> is it possible you can make your moses.ini file available for us to see?
>>>
>>> do you know if the same problem occurs if you use the command line
>>> moses, rather than mosesserver?
>>>
>>>
>>>  Hieu Hoang
>>> Researcher
>>>  New York University, Abu Dhabi
>>>  http://www.hoang.co.uk/hieu
>>>
>>> On 21 July 2015 at 18:07, Barry Haddow 
>>> wrote:
>>>

 On 21/07/15 14:51, Oren wrote:

 I am using the in-memory mode, using about 50GB of RAM. (No swap issues
 as far as I can tell.) Could that cause issues?


 Yes, swapping would definitely cause issues - was that your question?



  I looked at the commit you linked to, but it doesn't seem to be
 something configurable beyond the -threads switch. Am I missing something?


 The commit enables you to set the maximum number of connections to be
 the same as the maximum number of threads.


 On Tuesday, July 21, 2015, Barry Haddow 
 wrote:

>  Hi Oren
>
> Does your host have 18 threads available? It could also be that
> xmlrpc-c is limiting the number of connections - this can now be 
> configured:
>
> https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523
>
> Slowdowns in Moses are often caused by disk access bottlenecks. You
> can use --minphr-memory and --minlexr-memory to make sure that the phrase
> and reordering tables are loaded in to memory, rather than being access
> on-demand. Make sure your host has enough RAM and is not swapping. As I
> mentioned before there are various ways to make your models smaller (
> http://www.statmt.org/moses/?n=Advanced.RuleTables), which can make a
> big difference to speed depending on y

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Oren]

Yes, we use a lot of RAM in our setup. But the improved response time
justifies it.

Our language is on a nfs, bit we've been working this way with moses 1 for
some time with no problems (ten different machines using the same language
model file over nfs). Same for the reordering model. Neither of them is in
memory. Raising these models into memory will require raising our already
excessive RAM requirements...

Thanks again for the help.

On Wednesday, July 22, 2015, Barry Haddow 
wrote:

>  Hi Oren
>
> I'm not aware of any threading problems with PhraseDictionaryMemory, but
> not many people use it since it takes up too much memory. Moses command
> line runs multi-threaded using a thread pool.
>
> Is your language model on a local file system? Running it over nfs can be
> bad. What about your reordering model? Are you using the compact or the
> memory version?
>
> cheers - Barry
>
> On 22/07/15 15:09, Oren wrote:
>
> We have no swapping issues. I was asking if the use of an in-memory
> translation model might cause multithreading problems.
>
>
>  I'm not sure how to replicate the problem on cmd moses, since it's
> purely a multithreading issue. Ca you run cmd moses multithreaded?
>
>  I can't attach the complete moses.ini because it's on a separate
> network... But I copied below the stuff that looked relevant. I also tried
> to change the [threads] setting to 18, with no apparent effect.
>
>  [input-factors]
> 0
>
>  [mapping]
> 0 T 0
>
>  [distortion-limit]
> 6
>
>  [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 num-features=4
> path=/phrase-table.gz input-factor=0 output-factor=0
> LexicalReordering 
> Distortion
> KENLM lazyken=0 name=LM0 path= order=5
>
>  [threads]
> 6
>
>  [weight]
>
>  
>
> On Tuesday, July 21, 2015, Hieu Hoang  wrote:
>
>> is it possible you can make your moses.ini file available for us to see?
>>
>> do you know if the same problem occurs if you use the command line moses,
>> rather than mosesserver?
>>
>>
>>  Hieu Hoang
>> Researcher
>>  New York University, Abu Dhabi
>>  http://www.hoang.co.uk/hieu
>>
>> On 21 July 2015 at 18:07, Barry Haddow 
>> wrote:
>>
>>>
>>> On 21/07/15 14:51, Oren wrote:
>>>
>>> I am using the in-memory mode, using about 50GB of RAM. (No swap issues
>>> as far as I can tell.) Could that cause issues?
>>>
>>>
>>> Yes, swapping would definitely cause issues - was that your question?
>>>
>>>
>>>
>>>  I looked at the commit you linked to, but it doesn't seem to be
>>> something configurable beyond the -threads switch. Am I missing something?
>>>
>>>
>>> The commit enables you to set the maximum number of connections to be
>>> the same as the maximum number of threads.
>>>
>>>
>>> On Tuesday, July 21, 2015, Barry Haddow 
>>> wrote:
>>>
  Hi Oren

 Does your host have 18 threads available? It could also be that
 xmlrpc-c is limiting the number of connections - this can now be 
 configured:

 https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523

 Slowdowns in Moses are often caused by disk access bottlenecks. You can
 use --minphr-memory and --minlexr-memory to make sure that the phrase and
 reordering tables are loaded in to memory, rather than being access
 on-demand. Make sure your host has enough RAM and is not swapping. As I
 mentioned before there are various ways to make your models smaller (
 http://www.statmt.org/moses/?n=Advanced.RuleTables), which can make a
 big difference to speed depending on your setup.

 cheers - Barry

 On 21/07/15 09:30, Oren wrote:

 Hi Barry,

  Thanks for the quick response.

  I added the switch "-threads 18" to the command to raise moses
 server. The slowness issue persists but in a different form. Most requests
 return right away, even under heavy load, but some requests (about 5%) take
 far longer - about 15-20 seconds.

  Perhaps there are other relevant switches?

  Thanks again.

 On Monday, July 20, 2015, Barry Haddow 
 wrote:

>  Hi Oren
>
> The threading model is different. In v1, the server created a new
> thread for every request, v3 uses a thread pool. Try increasing the number
> of threads.
>
> Also, make sure you use the compact phrase table and KenLM as they are
> normally faster, and pre-pruning your phrase table can help,
>
> cheers - Barry
>
> On 20/07/15 12:01, Oren wrote:
>
> Hi all,
>
>  We are in the process of migrating from Moses 1 to Moses 3. We have
> noticed a significant slowdown when sending many requests at once to Moses
> Server. The first request will actually finish about 25% faster that a
> single request using Moses 1, but as more requests accumulate there is a
> marked slowdown, until requests take 5 times longer or more.
>
>  Is this a known issue? Is it spe

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Oren]

I am using the in-memory mode, using about 50GB of RAM. (No swap issues as
far as I can tell.) Could that cause issues?

I looked at the commit you linked to, but it doesn't seem to be something
configurable beyond the -threads switch. Am I missing something?

On Tuesday, July 21, 2015, Barry Haddow  wrote:

>  Hi Oren
>
> Does your host have 18 threads available? It could also be that xmlrpc-c
> is limiting the number of connections - this can now be configured:
>
> https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523
>
> Slowdowns in Moses are often caused by disk access bottlenecks. You can
> use --minphr-memory and --minlexr-memory to make sure that the phrase and
> reordering tables are loaded in to memory, rather than being access
> on-demand. Make sure your host has enough RAM and is not swapping. As I
> mentioned before there are various ways to make your models smaller (
> http://www.statmt.org/moses/?n=Advanced.RuleTables), which can make a big
> difference to speed depending on your setup.
>
> cheers - Barry
>
> On 21/07/15 09:30, Oren wrote:
>
> Hi Barry,
>
>  Thanks for the quick response.
>
>  I added the switch "-threads 18" to the command to raise moses server.
> The slowness issue persists but in a different form. Most requests return
> right away, even under heavy load, but some requests (about 5%) take far
> longer - about 15-20 seconds.
>
>  Perhaps there are other relevant switches?
>
>  Thanks again.
>
> On Monday, July 20, 2015, Barry Haddow  > wrote:
>
>>  Hi Oren
>>
>> The threading model is different. In v1, the server created a new thread
>> for every request, v3 uses a thread pool. Try increasing the number of
>> threads.
>>
>> Also, make sure you use the compact phrase table and KenLM as they are
>> normally faster, and pre-pruning your phrase table can help,
>>
>> cheers - Barry
>>
>> On 20/07/15 12:01, Oren wrote:
>>
>> Hi all,
>>
>>  We are in the process of migrating from Moses 1 to Moses 3. We have
>> noticed a significant slowdown when sending many requests at once to Moses
>> Server. The first request will actually finish about 25% faster that a
>> single request using Moses 1, but as more requests accumulate there is a
>> marked slowdown, until requests take 5 times longer or more.
>>
>>  Is this a known issue? Is it specific to Moses Server? What can we do
>> about it?
>>
>>  Thanks!
>>
>>  Oren.
>>
>>
>> ___
>> Moses-support mailing 
>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>
> ___
> Moses-support mailing listmoses-supp...@mit.edu 
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Performance issues using Moses Server with Moses 3

[Oren]

We have no swapping issues. I was asking if the use of an in-memory
translation model might cause multithreading problems.


I'm not sure how to replicate the problem on cmd moses, since it's purely a
multithreading issue. Ca you run cmd moses multithreaded?

I can't attach the complete moses.ini because it's on a separate network...
But I copied below the stuff that looked relevant. I also tried to change
the [threads] setting to 18, with no apparent effect.

[input-factors]
0

[mapping]
0 T 0

[distortion-limit]
6

[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/phrase-table.gz input-factor=0 output-factor=0
LexicalReordering 
Distortion
KENLM lazyken=0 name=LM0 path= order=5

[threads]
6

[weight]



On Tuesday, July 21, 2015, Hieu Hoang > wrote:

> is it possible you can make your moses.ini file available for us to see?
>
> do you know if the same problem occurs if you use the command line moses,
> rather than mosesserver?
>
>
> Hieu Hoang
> Researcher
> New York University, Abu Dhabi
> http://www.hoang.co.uk/hieu
>
> On 21 July 2015 at 18:07, Barry Haddow  wrote:
>
>>
>> On 21/07/15 14:51, Oren wrote:
>>
>> I am using the in-memory mode, using about 50GB of RAM. (No swap issues
>> as far as I can tell.) Could that cause issues?
>>
>>
>> Yes, swapping would definitely cause issues - was that your question?
>>
>>
>>
>>  I looked at the commit you linked to, but it doesn't seem to be
>> something configurable beyond the -threads switch. Am I missing something?
>>
>>
>> The commit enables you to set the maximum number of connections to be the
>> same as the maximum number of threads.
>>
>>
>> On Tuesday, July 21, 2015, Barry Haddow 
>> wrote:
>>
>>>  Hi Oren
>>>
>>> Does your host have 18 threads available? It could also be that xmlrpc-c
>>> is limiting the number of connections - this can now be configured:
>>>
>>> https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523
>>>
>>> Slowdowns in Moses are often caused by disk access bottlenecks. You can
>>> use --minphr-memory and --minlexr-memory to make sure that the phrase and
>>> reordering tables are loaded in to memory, rather than being access
>>> on-demand. Make sure your host has enough RAM and is not swapping. As I
>>> mentioned before there are various ways to make your models smaller (
>>> http://www.statmt.org/moses/?n=Advanced.RuleTables), which can make a
>>> big difference to speed depending on your setup.
>>>
>>> cheers - Barry
>>>
>>> On 21/07/15 09:30, Oren wrote:
>>>
>>> Hi Barry,
>>>
>>>  Thanks for the quick response.
>>>
>>>  I added the switch "-threads 18" to the command to raise moses server.
>>> The slowness issue persists but in a different form. Most requests return
>>> right away, even under heavy load, but some requests (about 5%) take far
>>> longer - about 15-20 seconds.
>>>
>>>  Perhaps there are other relevant switches?
>>>
>>>  Thanks again.
>>>
>>> On Monday, July 20, 2015, Barry Haddow 
>>> wrote:
>>>
  Hi Oren

 The threading model is different. In v1, the server created a new
 thread for every request, v3 uses a thread pool. Try increasing the number
 of threads.

 Also, make sure you use the compact phrase table and KenLM as they are
 normally faster, and pre-pruning your phrase table can help,

 cheers - Barry

 On 20/07/15 12:01, Oren wrote:

 Hi all,

  We are in the process of migrating from Moses 1 to Moses 3. We have
 noticed a significant slowdown when sending many requests at once to Moses
 Server. The first request will actually finish about 25% faster that a
 single request using Moses 1, but as more requests accumulate there is a
 marked slowdown, until requests take 5 times longer or more.

  Is this a known issue? Is it specific to Moses Server? What can we do
 about it?

  Thanks!

  Oren.


 ___
 Moses-support mailing 
 listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support



>>>
>>> ___
>>> Moses-support mailing 
>>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Barry Haddow]

Hi Oren

I'm not aware of any threading problems with PhraseDictionaryMemory, but 
not many people use it since it takes up too much memory. Moses command 
line runs multi-threaded using a thread pool.


Is your language model on a local file system? Running it over nfs can 
be bad. What about your reordering model? Are you using the compact or 
the memory version?


cheers - Barry

On 22/07/15 15:09, Oren wrote:
We have no swapping issues. I was asking if the use of an in-memory 
translation model might cause multithreading problems.



I'm not sure how to replicate the problem on cmd moses, since it's 
purely a multithreading issue. Ca you run cmd moses multithreaded?


I can't attach the complete moses.ini because it's on a separate 
network... But I copied below the stuff that looked relevant. I also 
tried to change the [threads] setting to 18, with no apparent effect.


[input-factors]
0

[mapping]
0 T 0

[distortion-limit]
6

[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4 
path=/phrase-table.gz input-factor=0 output-factor=0

LexicalReordering 
Distortion
KENLM lazyken=0 name=LM0 path= order=5

[threads]
6

[weight]



On Tuesday, July 21, 2015, Hieu Hoang > wrote:


is it possible you can make your moses.ini file available for us
to see?

do you know if the same problem occurs if you use the command line
moses, rather than mosesserver?


Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 21 July 2015 at 18:07, Barry Haddow
 wrote:


On 21/07/15 14:51, Oren wrote:

I am using the in-memory mode, using about 50GB of RAM. (No
swap issues as far as I can tell.) Could that cause issues?


Yes, swapping would definitely cause issues - was that your
question?




I looked at the commit you linked to, but it doesn't seem to
be something configurable beyond the -threads switch. Am
I missing something?


The commit enables you to set the maximum number of
connections to be the same as the maximum number of threads.



On Tuesday, July 21, 2015, Barry Haddow
 wrote:

Hi Oren

Does your host have 18 threads available? It could also
be that xmlrpc-c is limiting the number of connections -
this can now be configured:

https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523

Slowdowns in Moses are often caused by disk access
bottlenecks. You can use --minphr-memory and
--minlexr-memory to make sure that the phrase and
reordering tables are loaded in to memory, rather than
being access on-demand. Make sure your host has enough
RAM and is not swapping. As I mentioned before there are
various ways to make your models smaller
(http://www.statmt.org/moses/?n=Advanced.RuleTables),
which can make a big difference to speed depending on
your setup.

cheers - Barry

On 21/07/15 09:30, Oren wrote:

Hi Barry,

Thanks for the quick response.

I added the switch "-threads 18" to the command to raise
moses server. The slowness issue persists but in a
different form. Most requests return right away, even
under heavy load, but some requests (about 5%) take far
longer - about 15-20seconds.

Perhaps there are other relevant switches?

Thanks again.

On Monday, July 20, 2015, Barry Haddow
 wrote:

Hi Oren

The threading model is different. In v1, the server
created a new thread for every request, v3 uses a
thread pool. Try increasing the number of threads.

Also, make sure you use the compact phrase table and
KenLM as they are normally faster, and pre-pruning
your phrase table can help,

cheers - Barry

On 20/07/15 12:01, Oren wrote:

Hi all,

We are in the process of migrating from Moses 1 to
Moses 3. We have noticed a significant slowdown
when sending many requests at once to Moses Server.
The first request will actually finish about 25%
faster that a single request using Moses 1, but as
more requests accumulate there is a marked
slowdown, until requests take 5 times longer or more.

Is this a known issue? Is it specific to Moses
Server? What can we do about it?

Thanks!

Oren.


___
Moses-support mailing list
Mo

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Hieu Hoang]

is it possible you can make your moses.ini file available for us to see?

do you know if the same problem occurs if you use the command line moses,
rather than mosesserver?


Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 21 July 2015 at 18:07, Barry Haddow  wrote:

>
> On 21/07/15 14:51, Oren wrote:
>
> I am using the in-memory mode, using about 50GB of RAM. (No swap issues as
> far as I can tell.) Could that cause issues?
>
>
> Yes, swapping would definitely cause issues - was that your question?
>
>
>
>  I looked at the commit you linked to, but it doesn't seem to be
> something configurable beyond the -threads switch. Am I missing something?
>
>
> The commit enables you to set the maximum number of connections to be the
> same as the maximum number of threads.
>
>
> On Tuesday, July 21, 2015, Barry Haddow 
> wrote:
>
>>  Hi Oren
>>
>> Does your host have 18 threads available? It could also be that xmlrpc-c
>> is limiting the number of connections - this can now be configured:
>>
>> https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523
>>
>> Slowdowns in Moses are often caused by disk access bottlenecks. You can
>> use --minphr-memory and --minlexr-memory to make sure that the phrase and
>> reordering tables are loaded in to memory, rather than being access
>> on-demand. Make sure your host has enough RAM and is not swapping. As I
>> mentioned before there are various ways to make your models smaller (
>> http://www.statmt.org/moses/?n=Advanced.RuleTables), which can make a
>> big difference to speed depending on your setup.
>>
>> cheers - Barry
>>
>> On 21/07/15 09:30, Oren wrote:
>>
>> Hi Barry,
>>
>>  Thanks for the quick response.
>>
>>  I added the switch "-threads 18" to the command to raise moses server.
>> The slowness issue persists but in a different form. Most requests return
>> right away, even under heavy load, but some requests (about 5%) take far
>> longer - about 15-20 seconds.
>>
>>  Perhaps there are other relevant switches?
>>
>>  Thanks again.
>>
>> On Monday, July 20, 2015, Barry Haddow 
>> wrote:
>>
>>>  Hi Oren
>>>
>>> The threading model is different. In v1, the server created a new thread
>>> for every request, v3 uses a thread pool. Try increasing the number of
>>> threads.
>>>
>>> Also, make sure you use the compact phrase table and KenLM as they are
>>> normally faster, and pre-pruning your phrase table can help,
>>>
>>> cheers - Barry
>>>
>>> On 20/07/15 12:01, Oren wrote:
>>>
>>> Hi all,
>>>
>>>  We are in the process of migrating from Moses 1 to Moses 3. We have
>>> noticed a significant slowdown when sending many requests at once to Moses
>>> Server. The first request will actually finish about 25% faster that a
>>> single request using Moses 1, but as more requests accumulate there is a
>>> marked slowdown, until requests take 5 times longer or more.
>>>
>>>  Is this a known issue? Is it specific to Moses Server? What can we do
>>> about it?
>>>
>>>  Thanks!
>>>
>>>  Oren.
>>>
>>>
>>> ___
>>> Moses-support mailing 
>>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>
>> ___
>> Moses-support mailing 
>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Barry Haddow]

On 21/07/15 14:51, Oren wrote:
I am using the in-memory mode, using about 50GB of RAM. (No swap 
issues as far as I can tell.) Could that cause issues?


Yes, swapping would definitely cause issues - was that your question?




I looked at the commit you linked to, but it doesn't seem to be 
something configurable beyond the -threads switch. Am I missing something?


The commit enables you to set the maximum number of connections to be 
the same as the maximum number of threads.




On Tuesday, July 21, 2015, Barry Haddow > wrote:


Hi Oren

Does your host have 18 threads available? It could also be that
xmlrpc-c is limiting the number of connections - this can now be
configured:

https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523

Slowdowns in Moses are often caused by disk access bottlenecks.
You can use --minphr-memory and --minlexr-memory to make sure that
the phrase and reordering tables are loaded in to memory, rather
than being access on-demand. Make sure your host has enough RAM
and is not swapping. As I mentioned before there are various ways
to make your models smaller
(http://www.statmt.org/moses/?n=Advanced.RuleTables), which can
make a big difference to speed depending on your setup.

cheers - Barry

On 21/07/15 09:30, Oren wrote:

Hi Barry,

Thanks for the quick response.

I added the switch "-threads 18" to the command to raise moses
server. The slowness issue persists but in a different form. Most
requests return right away, even under heavy load, but some
requests (about 5%) take far longer - about 15-20seconds.

Perhaps there are other relevant switches?

Thanks again.

On Monday, July 20, 2015, Barry Haddow
> wrote:

Hi Oren

The threading model is different. In v1, the server created a
new thread for every request, v3 uses a thread pool. Try
increasing the number of threads.

Also, make sure you use the compact phrase table and KenLM as
they are normally faster, and pre-pruning your phrase table
can help,

cheers - Barry

On 20/07/15 12:01, Oren wrote:

Hi all,

We are in the process of migrating from Moses 1 to Moses 3.
We have noticed a significant slowdown when sending many
requests at once to Moses Server. The first request will
actually finish about 25% faster that a single request using
Moses 1, but as more requests accumulate there is a marked
slowdown, until requests take 5 times longer or more.

Is this a known issue? Is it specific to Moses Server? What
can we do about it?

Thanks!

Oren.


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support




___
Moses-support mailing list
Moses-support@mit.edu  

http://mailman.mit.edu/mailman/listinfo/moses-support




The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Barry Haddow]

Hi Oren

Does your host have 18 threads available? It could also be that xmlrpc-c 
is limiting the number of connections - this can now be configured:

https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523

Slowdowns in Moses are often caused by disk access bottlenecks. You can 
use --minphr-memory and --minlexr-memory to make sure that the phrase 
and reordering tables are loaded in to memory, rather than being access 
on-demand. Make sure your host has enough RAM and is not swapping. As I 
mentioned before there are various ways to make your models smaller 
(http://www.statmt.org/moses/?n=Advanced.RuleTables), which can make a 
big difference to speed depending on your setup.


cheers - Barry

On 21/07/15 09:30, Oren wrote:

Hi Barry,

Thanks for the quick response.

I added the switch "-threads 18" to the command to raise moses server. 
The slowness issue persists but in a different form. Most requests 
return right away, even under heavy load, but some requests (about 5%) 
take far longer - about 15-20seconds.


Perhaps there are other relevant switches?

Thanks again.

On Monday, July 20, 2015, Barry Haddow > wrote:


Hi Oren

The threading model is different. In v1, the server created a new
thread for every request, v3 uses a thread pool. Try increasing
the number of threads.

Also, make sure you use the compact phrase table and KenLM as they
are normally faster, and pre-pruning your phrase table can help,

cheers - Barry

On 20/07/15 12:01, Oren wrote:

Hi all,

We are in the process of migrating from Moses 1 to Moses 3. We
have noticed a significant slowdown when sending many requests at
once to Moses Server. The first request will actually finish
about 25% faster that a single request using Moses 1, but as more
requests accumulate there is a marked slowdown, until requests
take 5 times longer or more.

Is this a known issue? Is it specific to Moses Server? What can
we do about it?

Thanks!

Oren.


___
Moses-support mailing list
Moses-support@mit.edu  

http://mailman.mit.edu/mailman/listinfo/moses-support




___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Oren]

Hi Barry,

Thanks for the quick response.

I added the switch "-threads 18" to the command to raise moses server. The
slowness issue persists but in a different form. Most requests return right
away, even under heavy load, but some requests (about 5%) take far longer -
about 15-20 seconds.

Perhaps there are other relevant switches?

Thanks again.

On Monday, July 20, 2015, Barry Haddow  wrote:

>  Hi Oren
>
> The threading model is different. In v1, the server created a new thread
> for every request, v3 uses a thread pool. Try increasing the number of
> threads.
>
> Also, make sure you use the compact phrase table and KenLM as they are
> normally faster, and pre-pruning your phrase table can help,
>
> cheers - Barry
>
> On 20/07/15 12:01, Oren wrote:
>
> Hi all,
>
>  We are in the process of migrating from Moses 1 to Moses 3. We have
> noticed a significant slowdown when sending many requests at once to Moses
> Server. The first request will actually finish about 25% faster that a
> single request using Moses 1, but as more requests accumulate there is a
> marked slowdown, until requests take 5 times longer or more.
>
>  Is this a known issue? Is it specific to Moses Server? What can we do
> about it?
>
>  Thanks!
>
>  Oren.
>
>
> ___
> Moses-support mailing listmoses-supp...@mit.edu 
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Performance issues using Moses Server with Moses 3

[Barry Haddow]

Hi Oren

The threading model is different. In v1, the server created a new thread 
for every request, v3 uses a thread pool. Try increasing the number of 
threads.


Also, make sure you use the compact phrase table and KenLM as they are 
normally faster, and pre-pruning your phrase table can help,


cheers - Barry

On 20/07/15 12:01, Oren wrote:

Hi all,

We are in the process of migrating from Moses 1 to Moses 3. We have 
noticed a significant slowdown when sending many requests at once to 
Moses Server. The first request will actually finish about 25% faster 
that a single request using Moses 1, but as more requests accumulate 
there is a marked slowdown, until requests take 5 times longer or more.


Is this a known issue? Is it specific to Moses Server? What can we do 
about it?


Thanks!

Oren.


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Performance issues using Moses Server with Moses 3

[Oren]

Hi all,

We are in the process of migrating from Moses 1 to Moses 3. We have noticed
a significant slowdown when sending many requests at once to Moses Server.
The first request will actually finish about 25% faster that a single
request using Moses 1, but as more requests accumulate there is a marked
slowdown, until requests take 5 times longer or more.

Is this a known issue? Is it specific to Moses Server? What can we do about
it?

Thanks!

Oren.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support