[Moses-support] COLING 2014 Call for System Demonstrations

2014-03-20 Thread 박성배
** Apologies for cross-posting **

COLING 2014 Call for System Demonstrations
The 25th International Conference on Computational Linguistics, August 23 - 29, 
2014, Dublin, Ireland

http://www.coling-2014.org

Important dates
19 May 2014: Paper submission deadline
23 June 2014: Author notification
7 July 2014: Camera-ready paper submission deadline


The COLING 2014 Demonstration Programme Committee invites proposals for system 
demonstrations. The demonstration programme is part of the main conference 
programme and aims at showcasing working systems that apply a wide range of 
conference topics. The session will provide opportunities to exchange ideas 
gained from implementing NLP systems, and to obtain feedback from expert users.

COLING 2014 will be held in Dublin, Ireland from 23-29 August 2014.

The COLING conference has a history that dates back to the 1960s. It is held 
every two years and regularly attracts more than 700 delegates. The conference 
has developed into one of the premier Natural Language Processing (NLP) 
conferences worldwide and is a major international event for the presentation 
of new research results and for the demonstration of new systems and techniques 
in the broad field of Computational Linguistics and NLP.

Topics of interest
COLING 2014 solicits demonstrations on original and unpublished research on the 
following topics, including, but not limited to:
● pragmatics, semantics, syntax, grammars and the lexicon;
● cognitive, mathematical and computational models of language processing;
● models of communication by language;
● lexical semantics and ontologies;
● word segmentation, tagging and chunking;
● parsing, both syntactic and deep;
● generation and summarisation;
● paraphrasing, textual entailment and question answering;
● speech recognition, text-to-speech and spoken language understanding;
● multimodal and natural language interfaces and dialogue systems;
● information retrieval, information extraction and knowledge base linking;
● machine learning for natural language;
● modelling of discourse and dialogue;
● sentiment analysis, opinion mining and social media;
● multilingual processing, machine translation and translation aids;
● applications, tools and language resources;
● system evaluation methodology and metrics.


Submissions
The submissions should address the following questions:
● What is the problem the proposed system addresses?
● Why is the system important and what is its impact?
● What is the novelty of the used approach/technology?
● Who is the target audience?
● How does the system work?
● How does it compare with existing systems?
● How is the system licenced?

The maximum submission length is 4 pages (including references). Papers shall 
be submitted in English and must conform to the official COLING 2014 style 
guidelines available on the conference website. The anonymisation of 
submissions is optional. If authors choose to remain anonymous, it is their 
responsibility to take every measure to conceal potentially identifying 
information.
http://www.coling-2014.org/instructions-for-authors.php

Submission and reviewing will be managed in the START system:
https://www.softconf.com/coling2014/demos/

The only accepted format for submissions is PDF. Accepted papers will appear in 
the conference proceedings in a dedicated volume for demonstration systems.


Demonstration chairs
Lamia Tounsi, CNGL, Dublin City University, Ireland
Rafal Rak, NaCTeM, University of Manchester, UK


Programme Committee
Michiel Bacchiani, Google Inc.
Kay Berkling, Cooperative State University, Karlsruhe
Ann Bies, Linguistic Data Consortium
William Black, University of Manchester
Francis Bond, Nanyang Technological University
Chris Brew, Nuance Communications
Aoife Cahill, Educational Testing Service
Vittorio Castelli, IBM
Md. Faisal Mahbub Chowdhury, IBM
Léa Deleris, IBM
Martin Emms, Trinity College Dublin
Guillaume Gravier, IRISA  INRIA Rennes
Keith Hall, Google Research
Derrick Higgins, Educational Testing Service
Keikichi Hirose, University of Tokyo
Frank Hopfgartner, Technische Universität Berlin
Daxin Jiang, Microsoft STC-A
John Kelleher, Trinity College Dublin
Adam Kilgarriff, Lexical Computing Ltd
BalaKrishna Kolluru, Toshiba
Seamus Lawless, Trinity College Dublin
Saturnino Luz, Trinity College Dublin
Nitin Madnani, Educational Testing Service
Hilary McDonald, Trinity College Dublin
Helen Meng, Chinese University of Hong Kong
Peter Mika, Yahoo Labs
Tony O'Dowd, KantanMT
Florian Pinel, IBM
Johann Roturier, Symantec
Andrew Rowley, University of Manchester
Frédérique Segond, Viseo Research
Swapna Somasundaran, Educational Testing Services
Tomoki Toda, Nara Institute of Science and Technology
Xinglong Wang, Brandwatch
Jason Williams, Microsoft Research

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


[Moses-support] Loading a binary rule table

2014-03-20 Thread Massinissa Ahmim
Dear all,

I've managed to train a hierarchical model using the following command :

nohup /mosesdecoder/scripts/training/train-model.perl --hierarchical
--glue-grammar --score-options=--GoodTuring -root-dir
/disque2/Preparation/syntactic/hierarchical_PSCT -corpus
 /disque2/Preparation/backoff/PSCT.tok.uni.low -f fr -e en  -lm
0:5:/disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
-external-bin-dir /root/external-bin-dir/ -mgiza -mgiza-cpus 30 
training.out 

The resulting model works great but it is very slow.

So I used CreatOnDiskPt to binarise the rule table as follows :

/home/Moses/mosesdecoder/bin/CreateOnDiskPt 1 1 5 20 2 model/rule-table.gz
rules-table

This outputs the following files

-rw-r--r-- 1 root root 85 20 mars  11:35 Misc.dat
-rw-r--r-- 1 root root  781509322 20 mars  11:35 Source.dat
-rw-r--r-- 1 root root 1758588137 20 mars  11:35 TargetColl.dat
-rw-r--r-- 1 root root 1555353459 20 mars  11:35 TargetInd.dat
-rw-r--r-- 1 root root 400714 20 mars  11:35 Vocab.dat

and I've updated my moses.ini as follows :

6 0 0 1 /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
6 0 0 1 /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar

But as I try to use it, I'm getting this :

Defined parameters (per moses.ini or switch):
config: moses.ini
cube-pruning-pop-limit: 1000
input-factors: 0
inputtype: 3
lmodel-file: 0 0 5
/disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
mapping: 0 T 0 1 T 1
max-chart-span: 20 1000
non-terminals: X
search-algorithm: 3
ttable-file: 6 0 0 1
/disque2/Preparation/syntactic/hierarchical_PSCT/rule-table 6 0 0 1
/disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
ttable-limit: 20
weight-l: 0.5000
weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
weight-w: -1
/mosesdecoder/bin
ScoreProducer: WordPenalty start: 0 end: 1
ScoreProducer: !UnknownWordPenalty start: 1 end: 2
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm : [0.000] seconds
/disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm: line 80492:
reached EOF before \end\
ScoreProducer: LM start: 2 end: 3
Finished loading LanguageModels : [0.056] seconds
Using uniform ttable-limit of 20 for all translation tables.
Start loading PhraseTable
/disque2/Preparation/syntactic/hierarchical_PSCT/rule-table : [0.056]
seconds
filePath: /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
ScoreProducer: PhraseModel start: 3 end: 4
Start loading PhraseTable
/disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar :
[0.056] seconds
filePath:
/disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
ScoreProducer: PhraseModel:2 start: 4 end: 5
Finished loading phrase tables : [0.056] seconds
max-chart-span: 20
max-chart-span: 1000
Start loading phrase table from
/disque2/Preparation/syntactic/hierarchical_PSCT/rule-table : [0.056]
seconds
Can't read /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table

So I wondering if I did something wrong with my training
command/binarisation or with the parameters in the moses.ini


Many thanks

Regads

MA




-- 

[image: Description : Description : lingua_custodia_final full logo]

 *The Translation Trustee*

*1, Place Charles de Gaulle*

*78180 Montigny-le-Bretonneux*

*Tel : +33 1 30 44 04 23   Mobile : +33 7 61 44 40 84*

*Email :*  *massinissa.ah...@linguacustodia.com
massinissa.ah...@linguacustodia.com*

*Website :*  *www.linguacustodia.com http://www.linguacustodia.com/ -
www.thetranslationtrustee.com  http://www.thetranslationtrustee.com*

ü Pensez à l'environnement, n'imprimez ce courriel que si nécessaire.

Please do not print this email unless it is absolutely necessary. Spread
environmental awareness.
inline: image001.jpg___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


[Moses-support] Issue about sentence segmentation

2014-03-20 Thread Arnaud Gicquel
Hi all

I am trying to develop a specific segmenter. The goal is to send Moses
decoder sentences instead of large textual flows syntactically incoherent.
In order to integrate this segmenter in an automatic workflow of documents
translation. I would define as a sentence delimiter any character that the
decoder does not take into account in its statistics.

Is there a completely neutral character (I don't want it to be considered
unknown) that I could use as a delimiter?

Thank you for your help

Arnaud Gicquel

-- 
Lingua Custodia
1, Place Charles de Gaulle
78180 Montigny le Bretonneux - France
Tel : 33 1 30 44 04 23
Email :  arnaud.gicq...@linguacustodia.com
Website :  www.linguacustodia.com
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] Problem in Web Based Version of MT System using MOSES - Please help

2014-03-20 Thread Hieu Hoang
maybe try the code in
   contrib/server/Translation-web



On 19 March 2014 17:04, Vishal Goyal(विशाल गोयल) vishal@gmail.comwrote:

 Thanks for kind reply.
 Then kindly suggest how can we make the web translation reality? Which new
 we need to use...



 On Wed, Mar 19, 2014 at 2:07 PM, Barry Haddow 
 bhad...@staffmail.ed.ac.ukwrote:

 Hi Vishal

 The web translation components that you mention are quite old and haven't
 been used much in a while.

 As far as I can see, the main problem is that the translate.cgi expects
 to have many copies of the daemon.pl running, all listening on different
 ports. Each one should wrap a different instance of Moses. So this web
 translation system is not multi-threaded (it was written before Moses had
 threads) it's multi-process,

 cheers - Barry


 On 18/03/14 15:02, Vishal Goyal(विशाल गोयल) wrote:

 Respected All,
 Greetings

 We have installed the moses server and web server on a single machine on
 linux platform. when tested on the local host the system work fine, when we
 installed the system on web server for public use. The system some work
 fine but most of the we are not getting the translation of the input text
 instead the input text is transliterated in the post processing script
 written in transliterate.pl http://transliterate.pl.

 Moses server is running as daemon.pl http://daemon.pl. The index.cgi
 is being used to take user input and translate.cgi start the moses thread
 for tranlation.


 When error log is inspected it show that only one thread of moses is
 running and all other threads are terminated with the error message (error
 log attached.)

 Please also find the attached source code of daemon.pl http://daemon.pl,
 index.cgi and translate.cgi



 --
 *Regards,*

 Vishal Goyal,
 Ph.D., M.Tech., MCA, M.C.S.D.
 Assistant Professor(Stage III) and Placement Coordinator,
 Department of Computer Science,
 Punjabi University Patiala-147002
 [*Online Hindi to Punjabi Machine Translation Tool -*
 http://h2p.learnpunjabi.org ]
 *[Research Cell: An International Journal of Engineering Sciences,
 http://ijoes.vidyapublications.com]*




 ___
 Moses-support mailing list
 Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support



 --
 The University of Edinburgh is a charitable body, registered in
 Scotland, with registration number SC005336.




 --
 *Regards,*
 Vishal Goyal,
 Ph.D., M.Tech., MCA, M.C.S.D.
 Assistant Professor(Stage III) and Placement Coordinator,
 Department of Computer Science,
 Punjabi University Patiala-147002
 [*Online Hindi to Punjabi Machine Translation Tool -*
 http://h2p.learnpunjabi.org ]
 *[Research Cell: An International Journal of Engineering Sciences,
 http://ijoes.vidyapublications.com http://ijoes.vidyapublications.com]*


 ___
 Moses-support mailing list
 Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support




-- 
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


[Moses-support] fail to build Moses

2014-03-20 Thread Tiansi Dong
the command I type is: ./bjam -j8

many thanks for the support

-tiansi dong


build.log.gz
Description: GNU Zip compressed data
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


[Moses-support] KenLM memory usage

2014-03-20 Thread Alexandra Birch
Hi there,

I want to run 4 MT servers at the same time on a machine with limited
memory. Kenlm seems to reserve the amount of memory which the language
model would have taken if it had been loaded into memory. So I don't have
enough memory to run all these servers and the machine grinds to a halt if
I try. Is there any flag I could use which would limit the amount of memory
reserved?

Lexi
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] KenLM memory usage

2014-03-20 Thread Alexandra Birch
I have found the answer on the kenlm web page and it seems to be working:

Full or lazy loading

KenLM supports lazy loading via mmap. This allows you to further reduce
memory usage, especially with trie which has good memory locality. In
Moses, this is controlled by the language model number in moses.ini. Using
language model number 8 will load the full model into memory (MAP_POPULATE
on Linux and read() on other OSes). Language model number 9 will lazily
load the model using mmap. I recommend fully loading if you have the RAM
for it; it actually takes less time to load the full model and use it
because the disk does not have to seek during decoding. Lazy loading works
best with local disk and is not recommended for networked filesystems.



On Thu, Mar 20, 2014 at 2:32 PM, Alexandra Birch lexi.bi...@gmail.comwrote:

 Hi there,

 I want to run 4 MT servers at the same time on a machine with limited
 memory. Kenlm seems to reserve the amount of memory which the language
 model would have taken if it had been loaded into memory. So I don't have
 enough memory to run all these servers and the machine grinds to a halt if
 I try. Is there any flag I could use which would limit the amount of memory
 reserved?

 Lexi

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] Problem in Web Based Version of MT System using MOSES - Please help

2014-03-20 Thread Barry Haddow
Hi Vishal

There's no other code in Moses for web page translation. It just takes a 
bit more work to integrate it with the recent versions of Moses,

cheers - Barry

On 19/03/14 17:04, Vishal Goyal(विशाल गोयल) wrote:
 Thanks for kind reply.
 Then kindly suggest how can we make the web translation reality? Which 
 new we need to use...



 On Wed, Mar 19, 2014 at 2:07 PM, Barry Haddow 
 bhad...@staffmail.ed.ac.uk mailto:bhad...@staffmail.ed.ac.uk wrote:

 Hi Vishal

 The web translation components that you mention are quite old and
 haven't been used much in a while.

 As far as I can see, the main problem is that the translate.cgi
 expects to have many copies of the daemon.pl http://daemon.pl
 running, all listening on different ports. Each one should wrap a
 different instance of Moses. So this web translation system is not
 multi-threaded (it was written before Moses had threads) it's
 multi-process,

 cheers - Barry


 On 18/03/14 15:02, Vishal Goyal(विशाल गोयल) wrote:

 Respected All,
 Greetings

 We have installed the moses server and web server on a single
 machine on linux platform. when tested on the local host the
 system work fine, when we installed the system on web server
 for public use. The system some work fine but most of the we
 are not getting the translation of the input text instead the
 input text is transliterated in the post processing script
 written in transliterate.pl http://transliterate.pl
 http://transliterate.pl.

 Moses server is running as daemon.pl http://daemon.pl
 http://daemon.pl. The index.cgi is being used to take user
 input and translate.cgi start the moses thread for tranlation.


 When error log is inspected it show that only one thread of
 moses is running and all other threads are terminated with the
 error message (error log attached.)

 Please also find the attached source code of daemon.pl
 http://daemon.pl http://daemon.pl, index.cgi and translate.cgi



 -- 
 *Regards,*

 Vishal Goyal,
 Ph.D., M.Tech., MCA, M.C.S.D.
 Assistant Professor(Stage III) and Placement Coordinator,
 Department of Computer Science,
 Punjabi University Patiala-147002
 [*Online Hindi to Punjabi Machine Translation Tool -*
 http://h2p.learnpunjabi.org ]
 *[Research Cell: An International Journal of Engineering
 Sciences, http://ijoes.vidyapublications.com]*




 ___
 Moses-support mailing list
 Moses-support@mit.edu mailto:Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support



 -- 
 The University of Edinburgh is a charitable body, registered in
 Scotland, with registration number SC005336.




 -- 
 *Regards,*
 Vishal Goyal,
 Ph.D., M.Tech., MCA, M.C.S.D.
 Assistant Professor(Stage III) and Placement Coordinator,
 Department of Computer Science,
 Punjabi University Patiala-147002
 [*Online Hindi to Punjabi Machine Translation Tool -* 
 http://h2p.learnpunjabi.org ]
 *[Research Cell: An International Journal of Engineering Sciences, 
 http://ijoes.vidyapublications.com]*



-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] KenLM memory usage

2014-03-20 Thread Marcin Junczys-Dowmunt

Hi,
since KenLM uses shared memory, four instances should take up the same 
amount of memory as only one instance (ran yesterday 8 instances with 8 
threads each with a 99GB LM on a 128 GB machine). If the model fits into 
memory for a single instance it should work if you have enough memory 
left for all the phrase tables and the translation process itself (I 
guess this is actually the problem). Lazy loading was unbearably slow 
for me with the above mentioned configuration, but I was using 64 
threads in total, so a lot of concurrent disk access happing, no wonder 
there.

Best,
Marcin

W dniu 20.03.2014 14:35, Alexandra Birch pisze:

I have found the answer on the kenlm web page and it seems to be working:

Full or lazy loading

KenLM supports lazy loading via mmap. This allows you to further 
reduce memory usage, especially with trie which has good memory 
locality. In Moses, this is controlled by the language model number in 
moses.ini. Using language model number 8 will load the full model into 
memory (MAP_POPULATE on Linux and read() on other OSes). Language 
model number 9 will lazily load the model using mmap. I recommend 
fully loading if you have the RAM for it; it actually takes less time 
to load the full model and use it because the disk does not have to 
seek during decoding. Lazy loading works best with local disk and is 
not recommended for networked filesystems.




On Thu, Mar 20, 2014 at 2:32 PM, Alexandra Birch lexi.bi...@gmail.com 
mailto:lexi.bi...@gmail.com wrote:


Hi there,

I want to run 4 MT servers at the same time on a machine with
limited memory. Kenlm seems to reserve the amount of memory which
the language model would have taken if it had been loaded into
memory. So I don't have enough memory to run all these servers and
the machine grinds to a halt if I try. Is there any flag I could
use which would limit the amount of memory reserved?

Lexi




___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


[Moses-support] tools for factored corpora

2014-03-20 Thread Viktor Pless
Hi, what tools can be used to lemmatize/POS-tag/etc. a corpus in moses
format (with the pipes)? I need them regarding Spanish, English, Hungarian.
Thanks in advance.
Viktor
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] KenLM memory usage

2014-03-20 Thread Kenneth Heafield
Hi Lexi,

I take it that these models are different, not the same model loaded
into each process (in which case they would have shared).  I'd really
recommend trying to compress things more (e.g. trie -a 64 -q 8) before
going to lazy loading.

Kenneth

On 03/20/14 08:13, Marcin Junczys-Dowmunt wrote:
 Hi,
 since KenLM uses shared memory, four instances should take up the same
 amount of memory as only one instance (ran yesterday 8 instances with 8
 threads each with a 99GB LM on a 128 GB machine). If the model fits into
 memory for a single instance it should work if you have enough memory
 left for all the phrase tables and the translation process itself (I
 guess this is actually the problem). Lazy loading was unbearably slow
 for me with the above mentioned configuration, but I was using 64
 threads in total, so a lot of concurrent disk access happing, no wonder
 there.
 Best,
 Marcin
 
 W dniu 20.03.2014 14:35, Alexandra Birch pisze:
 I have found the answer on the kenlm web page and it seems to be working:

 Full or lazy loading

 KenLM supports lazy loading via mmap. This allows you to further
 reduce memory usage, especially with trie which has good memory
 locality. In Moses, this is controlled by the language model number in
 moses.ini. Using language model number 8 will load the full model into
 memory (MAP_POPULATE on Linux and read() on other OSes). Language
 model number 9 will lazily load the model using mmap. I recommend
 fully loading if you have the RAM for it; it actually takes less time
 to load the full model and use it because the disk does not have to
 seek during decoding. Lazy loading works best with local disk and is
 not recommended for networked filesystems.



 On Thu, Mar 20, 2014 at 2:32 PM, Alexandra Birch lexi.bi...@gmail.com
 mailto:lexi.bi...@gmail.com wrote:

 Hi there,

 I want to run 4 MT servers at the same time on a machine with
 limited memory. Kenlm seems to reserve the amount of memory which
 the language model would have taken if it had been loaded into
 memory. So I don't have enough memory to run all these servers and
 the machine grinds to a halt if I try. Is there any flag I could
 use which would limit the amount of memory reserved?

 Lexi




 ___
 Moses-support mailing list
 Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support
 
 
 
 ___
 Moses-support mailing list
 Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support
 
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] KenLM memory usage

2014-03-20 Thread Alexandra Birch
Hi Ken,

Yes, different models, different languages. Thanks! Yes lazy loading is
absolutely dead slow.

Lexi


On Thu, Mar 20, 2014 at 4:53 PM, Kenneth Heafield mo...@kheafield.comwrote:

 Hi Lexi,

 I take it that these models are different, not the same model
 loaded
 into each process (in which case they would have shared).  I'd really
 recommend trying to compress things more (e.g. trie -a 64 -q 8) before
 going to lazy loading.

 Kenneth

 On 03/20/14 08:13, Marcin Junczys-Dowmunt wrote:
  Hi,
  since KenLM uses shared memory, four instances should take up the same
  amount of memory as only one instance (ran yesterday 8 instances with 8
  threads each with a 99GB LM on a 128 GB machine). If the model fits into
  memory for a single instance it should work if you have enough memory
  left for all the phrase tables and the translation process itself (I
  guess this is actually the problem). Lazy loading was unbearably slow
  for me with the above mentioned configuration, but I was using 64
  threads in total, so a lot of concurrent disk access happing, no wonder
  there.
  Best,
  Marcin
 
  W dniu 20.03.2014 14:35, Alexandra Birch pisze:
  I have found the answer on the kenlm web page and it seems to be
 working:
 
  Full or lazy loading
 
  KenLM supports lazy loading via mmap. This allows you to further
  reduce memory usage, especially with trie which has good memory
  locality. In Moses, this is controlled by the language model number in
  moses.ini. Using language model number 8 will load the full model into
  memory (MAP_POPULATE on Linux and read() on other OSes). Language
  model number 9 will lazily load the model using mmap. I recommend
  fully loading if you have the RAM for it; it actually takes less time
  to load the full model and use it because the disk does not have to
  seek during decoding. Lazy loading works best with local disk and is
  not recommended for networked filesystems.
 
 
 
  On Thu, Mar 20, 2014 at 2:32 PM, Alexandra Birch lexi.bi...@gmail.com
  mailto:lexi.bi...@gmail.com wrote:
 
  Hi there,
 
  I want to run 4 MT servers at the same time on a machine with
  limited memory. Kenlm seems to reserve the amount of memory which
  the language model would have taken if it had been loaded into
  memory. So I don't have enough memory to run all these servers and
  the machine grinds to a halt if I try. Is there any flag I could
  use which would limit the amount of memory reserved?
 
  Lexi
 
 
 
 
  ___
  Moses-support mailing list
  Moses-support@mit.edu
  http://mailman.mit.edu/mailman/listinfo/moses-support
 
 
 
  ___
  Moses-support mailing list
  Moses-support@mit.edu
  http://mailman.mit.edu/mailman/listinfo/moses-support
 
 ___
 Moses-support mailing list
 Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] fail to build Moses

2014-03-20 Thread Hieu Hoang
where did you download your source code from, and when did you download it?

There 1 error I see and I think it was fixed a few weeks ago. If you
download the latest version of Moses from github, it should be ok:
   git clone https://github.com/moses-smt/mosesdecoder.git


On 20 March 2014 14:06, Tiansi Dong tian1s...@gmail.com wrote:

 the command I type is: ./bjam -j8

 many thanks for the support

 -tiansi dong

 ___
 Moses-support mailing list
 Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support




-- 
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] Installation issues (Linux mint)

2014-03-20 Thread Hieu Hoang
it looks like boost hasn't been installed properly.

What is the exact command you used to compile Moses? How did you install
boost? What OS are you running on?


On 20 March 2014 17:40, Martin McCaffery mm...@st-andrews.ac.uk wrote:

 Hi all,

 I've been trying to get Moses up and running locally and been running into
 problems. At first I had issues with trying to get Boost working on my
 local machine, but after tweaking Boost the errors have got smaller.

 Now when running ./bjam I seem to simply be getting g++ compile and
 testing errors, but I can't tell much more than that. There are some
 Boost-related undefined references, though: are there any specific
 configuration details I need for Boost that I'm missing?

 Any thoughts would be welcome.

 Thanks!
 Martin

 ___
 Moses-support mailing list
 Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support




-- 
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


Re: [Moses-support] tools for factored corpora

2014-03-20 Thread Massinissa Ahmim
Hi Viktor,

As far I know, you can use the wrapper scripts (treetagger or mxpost)
located at /mosesdecoder/scripts/training/wrappers for this purpose.

Regards

Massinissa


2014-03-20 17:37 GMT+01:00 Viktor Pless viktor.pl...@gmail.com:

 Hi, what tools can be used to lemmatize/POS-tag/etc. a corpus in moses
 format (with the pipes)? I need them regarding Spanish, English, Hungarian.
 Thanks in advance.
 Viktor

 ___
 Moses-support mailing list
 Moses-support@mit.edu
 http://mailman.mit.edu/mailman/listinfo/moses-support




-- 

[image: Description : Description : lingua_custodia_final full logo]

 *The Translation Trustee*

*1, Place Charles de Gaulle*

*78180 Montigny-le-Bretonneux*

*Tel : +33 1 30 44 04 23   Mobile : +33 7 61 44 40 84*

*Email :*  *massinissa.ah...@linguacustodia.com
massinissa.ah...@linguacustodia.com*

*Website :*  *www.linguacustodia.com http://www.linguacustodia.com/ -
www.thetranslationtrustee.com  http://www.thetranslationtrustee.com*

ü Pensez à l'environnement, n'imprimez ce courriel que si nécessaire.

Please do not print this email unless it is absolutely necessary. Spread
environmental awareness.
inline: image001.jpg___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support