Re: [Moses-support] Hardware requirements and translation throughput?

2016-10-19 Thread Hieu Hoang
Definitely use Linux.

RAM needed - create a translation system from your training data and
binarize the models. Add up the sizes of the binary files + (1 or 2GB) and
you'll have a figure for the RAM you need. eg. if your models are
   $ ll -h phrase-table.2.pruned100.NoEnc.minphr
  -rw-rw-r-- 1 hhoang hhoang 2.0G Oct  9  2015
phrase-table.2.pruned100.NoEnc.minphr
   $ ll -h reordering-table.2.wbe-msd-bidirectional-fe.minlexr
  -rw-r--r-- 1 hhoang hhoang 637M Sep 25  2015
reordering-table.2.wbe-msd-bidirectional-fe.minlexr
   $ ll -h europarl.kenlm
   -rw-r--r-- 1 hhoang hhoang 1.8G Sep 25  2015 europarl.kenlm
You'll need 6-7GB RAM. (NB. that free ram required, the OS and other things
also need RAM)

CPU cores - Moses can use around 16 cores. Using anymore won't make it any
faster. Moses2 scales to any number of cores.

Speed is dependent the length of the sentence. For documents with short
sentences (~7 words/sentence), Moses can translate around ~2500 words/sec
with 16 cores. Moses2 can translate ~30,000 words/sec with 32 cores.

For long sentences (~29 words/sentence), Moses can also do ~2500 words/sec
with 16 cores. Moses2 does ~24,000 words/sec with 32 cores

More details here:
https://arxiv.org/abs/1610.04265

Hieu Hoang
http://www.hoang.co.uk/hieu

On 19 October 2016 at 04:25, John Griffith  wrote:

> I know this is an "it depends" question, but what are the hardware
> requirements to run a typical translation service and what kind of
> throughput can I expect? Or maybe what is the range? My customer prefers
> either using either SUSE Linux or Windows Server for an OS. For instance,
> if I'd like to be able to estimate the requirements for translating 10K
> documents per day. What are your experiences?
>
> Thanks,
> -- John
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


[Moses-support] Hardware requirements and translation throughput?

2016-10-18 Thread John Griffith
I know this is an "it depends" question, but what are the hardware
requirements to run a typical translation service and what kind of
throughput can I expect? Or maybe what is the range? My customer prefers
either using either SUSE Linux or Windows Server for an OS. For instance,
if I'd like to be able to estimate the requirements for translating 10K
documents per day. What are your experiences?

Thanks,
-- John
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support