Re: [Moses-support] Moses Build with Boost 1.68.0 Fails

[Hieu Hoang]

boost 1.68 worked for me (on Ubuntu 22.04, gcc 11.4). Make sure you use
absolute paths in you bjam command arguments. This is what I use to compile
boost
   ./bootstrap.sh
   ./b2 -j4 --prefix=$PWD --libdir=$PWD/lib64 --layout=system link=static
install
This is what I use to compile moses
   ./bjam -j 6 --with-boost=/home/hihoan/local/workspace/github/boost_1_68_0



Hieu Hoang
https://hieuhoang.github.io/


On Fri, 24 May 2024 at 08:21, Andrew Beaton 
wrote:

> Hello,
> I'm currently trying to build Moses with Boost 1.68.0 (not the latest
> version, but this is for compatibility reasons with a project I'm using
> for
> my thesis). However, the build fails with the following error (full error
> log is attached):
>
> /home/users/abeaton/project/thesis/VTLM2/data/mosesdecoder2/opt/boost_1_68_0/tools/build/src/tools/types/docbook.jam:8:
>
> in load
> ERROR: rule "Copyright" unknown in module "docbook".
>
> And here is the exact command I used to build Moses:
>
> ./bjam --with-boost=opt/boost_1_68_0
>
> Is this a known problem, and is there a fix besides using a more current
> version of Boost? I've also tried with the latest version (1.85.0), but
> then
> I receive another error since the
> boost_1_85_0/tools/build/src/bootstrap.jam
> file seems to be missing in the latest version.
>
> Best,
> Andrew
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> https://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
https://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Error: No such file or directory while opening /home/xyz/working/train/model/extract.o.sorted.gz

[Hieu Hoang]

it may be that a previous step failed, eg due to running out of disk space,
choking on bad data.

Check the logs for the previous steps

Hieu Hoang
http://statmt.org/hieu


On Wed, 4 Jan 2023 at 11:42, Sainik Kumar Mahata 
wrote:

> Dear All,
>
> I have trained my translation models using Moses Toolkit multiple times
> before, but recently, when I tried training a fresh model, I am receiving
> the following error message.
>
> No such file or directory while opening
> /home/sainik/working/train/model/extract.o.sorted.gz
> Aborted
> Exit code: 134
>
> Also, when I am looking into the log file, I can find two more
> warning/error messages.
>
> Use of uninitialized value in substitution (s///) at
> /home/xyx/mosesdecoder/scripts/training/LexicalTranslationModel.pm line 40.
> Use of uninitialized value $a in split at
> /home/xyz/mosesdecoder/scripts/training/LexicalTranslationModel.pm line 109.
>
> This never happened before and it would be very kind of anyone, who can
> guide me on this issue.
>
> Thanks in advance.
>
>
> --
>
> Regards,
>
> Sainik
> Phone/WA: +91-9038141146
>
>
> [image: Mailtrack]
> <https://mailtrack.io?utm_source=gmail_medium=signature_campaign=signaturevirality11;>
>  Sender
> notified by
> Mailtrack
> <https://mailtrack.io?utm_source=gmail_medium=signature_campaign=signaturevirality11;>
>  01/04/23,
> 05:05:11 PM
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> https://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
https://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] hierarchical-to-string translation doesn't produce string output

[Hieu Hoang]

the training does add glue rules for non-syntax models but you're using 
a variant with syntax where you have to do it yourself.


DOwnload the small sample model and look in the tree-to-tree example to 
see how it's done


   http://www.statmt.org/moses/download/sample-models.tgz

Also, I hope you realise that these are research models. They don't 
produce good results in practise and have largely been overtaken by 
neural models


On 3/17/2022 10:16 PM, Oleg Parashchenko wrote:

Hello Hieu,


... You always need a glue rule

I assumed that training should create one. If I need to write one self,
where can I read the details? The page
https://www.statmt.org/moses/?n=Moses.SyntaxTutorial mentions only the
using of glue rules.


Can I have a look at your moses.ini file

I've put the whole model at
http://uucode.com/download/tmp/mini-hie-to-string.tar.gz,
here is the ini file:

```
#
### MOSES CONFIG FILE ###
#

# input factors
[input-factors]
0

# mapping steps
[mapping]
0 T 0

[cube-pruning-pop-limit]
1000

[non-terminals]
X

[search-algorithm]
3

[inputtype]
3

[max-chart-span]
20
1000

# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/share/mini-hie-to-string/model/rule-table.gz input-factor=0
output-factor=0 KENLM name=LM0 factor=0
path=/share/mini-hie-to-string/lm.en order=2

# dense weights for feature functions
[weight]
# The default weights are NOT optimized for translation quality. You MUST tune 
the weights.
# Documentation for tuning is here: 
http://www.statmt.org/moses/?n=FactoredTraining.Tuning
UnknownWordPenalty0= 1
WordPenalty0= -1
PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2
LM0= 0.5
```

Regards,
Oleg

On Thu, 17 Mar 2022 21:09:10 -0700
Hieu Hoang  wrote:


Can I have a look at your moses.ini file. You always need a glue rule
phrase take for scfg models to create the final output

...


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
https://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] hierarchical-to-string translation doesn't produce string output

[Hieu Hoang]

Can I have a look at your moses.ini file. You always need a glue rule
phrase take for scfg models to create the final output

On Thu, Mar 17, 2022, 8:58 PM Oleg Parashchenko  wrote:

> Hello,
>
> I've created and tried a hierarchical model:
>
> ```
> $ echo 'nhoj' | moses -f model/moses.ini
> ...
> Line 0: Initialize search took 0.000 seconds total
> Translating:  nhoj   ||| [0,0]=X (1) [0,1]=X (1) [0,2]=X (1)
> [1,1]=X (1) [1,1]=N (1) [1,2]=X (1) [2,2]=X (1)
>
>   0   1   2
>   0   1   0
> 0   0
>   0
> Line 0: Additional reporting took 0.000 seconds total
> Line 0: Translation took 0.001 seconds total
> Translation took 0.001 seconds
> ...
> ```
>
> Something has happened, but I don't see the expected output "john".
> Adding the parameter "--inputtype 3" to moses doesn't change the result.
>
> What I'm doing wrong?
>
>
> Here is how I've built the model:
>
> ```
> $ cat corpus.en
> john
> mary
> hello
> bye
> hello john
>
> $ cat corpus.ne
> nhoj
> yram
> olleh
> eyb
> olleh  label="N">hnoj
>
> $ lmplz -o 2 --discount_fallback < corpus.en > lm.en
> ...
>
> $ /opt/moses/scripts/training/train-model.perl \
>   -corpus corpus \
>   -f ne -e en \
>   -lm 0:2:$(pwd)/lm.en \
>   -alignment grow-diag-final-and \
>   -external-bin-dir /opt/tools \
>   -mgiza \
>   -hierarchical \
>   2>&1 | tee train-log.txt
>
> $ cat model/rule-table.gz | gunzip -c
> nhoj [X] ||| john [X] ||| 0.5 0.28 1 1 |||
> 1-0 ||| 2 1 1 ||| |||
> ...
> ```
>
> Thanks!
>
> Regards,
> Oleg
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> https://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
https://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Pure-python decoder for Moses?

[Hieu Hoang]

NLTK had something but not sure how good it is
   https://www.nltk.org/_modules/nltk/translate/phrase_based.html
Liling may know more about it

Hieu Hoang
http://statmt.org/hieu


On Mon, 24 Jan 2022 at 09:50, Lane Schwartz  wrote:

> Pretty sure the answer is no. David Chiang had a Python-based system, but
> it was hierarchical phrase-based.
>
>
>
> On Mon, Jan 24, 2022 at 8:50 AM Rachel Bawden 
> wrote:
>
>> Hello,
>>
>> Are there currently any pure-python implementations of mosesdecoder? I am
>> looking to distribute and run pre-trained models (so no tuning) without
>> having to compile Moses, even though decoding will inevitably be less
>> efficient.
>>
>> Thank you very much in advance,
>> All the best,
>>
>> Rachel
>>
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> https://mailman.mit.edu/mailman/listinfo/moses-support
>>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> https://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
https://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Fwd: MOSES v4.0 binaries is forbidden to download

[Hieu Hoang]

fyi

Hieu Hoang
http://statmt.org/hieu


-- Forwarded message -
From: Ulrich Germann 
Date: Fri, 15 Oct 2021 at 14:55
Subject: Re: [Moses-support] MOSES v4.0 binaries is forbidden to download
To: Hieu Hoang 
Cc: Barry Haddow , Kenneth Heafield <
kheaf...@inf.ed.ac.uk>, BOGOYCHEV Nikolay 


Should work now. Was a symlink problem.

Cheers - Uli
-- 
Ulrich Germann
Senior Researcher
School of Informatics
University of Edinburgh
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Problem installing Moses

[Hieu Hoang]

 sudo apt-get install libboost-all-dev


Hieu Hoang
http://statmt.org/hieu


On Mon, 14 Jun 2021 at 22:08, Hieu Hoang  wrote:

> what version of Ubuntu? If it's recent (eg. 20.04, 18.04) then it's easier
> to install boost from ubuntu
>sudo apt-get install
> and compile moses simply:
>./bjam -j4
>
> Hieu Hoang
> http://statmt.org/hieu
>
>
> On Mon, 14 Jun 2021 at 21:55, John Kiddik  wrote:
>
>> Hi, I’m trying to install Moses on my PC.
>> OS is windows 10, but I’m installing it using ubuntu with the help of WSL
>>
>>
>>
>> I installed Boost, and the install was going smoothly then had some
>> errors during the test cases at the end, then it gave me the “build failed”
>> message and email you guys.
>>
>>
>>
>> I’ve attached the build log to this email and the exact command I’ve
>> inputted is this:
>>
>> sudo ./bjam
>> -–with-boost=/mnt/c/Users/mjk32/Desktop/MJK/tests/Machine_Translation/nmt-transfer-learning-xlm-r-master/boost_1_72_0
>>
>>
>>
>> I’m still a beginner in Linux and all of this, so I might have missed
>> some obvious things and I apologize in advance if I did.
>>
>>
>>
>> Thank you for your time,
>> Mujahid Khalifah
>>
>>
>>
>> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Problem installing Moses

[Hieu Hoang]

what version of Ubuntu? If it's recent (eg. 20.04, 18.04) then it's easier
to install boost from ubuntu
   sudo apt-get install
and compile moses simply:
   ./bjam -j4

Hieu Hoang
http://statmt.org/hieu


On Mon, 14 Jun 2021 at 21:55, John Kiddik  wrote:

> Hi, I’m trying to install Moses on my PC.
> OS is windows 10, but I’m installing it using ubuntu with the help of WSL
>
>
>
> I installed Boost, and the install was going smoothly then had some errors
> during the test cases at the end, then it gave me the “build failed”
> message and email you guys.
>
>
>
> I’ve attached the build log to this email and the exact command I’ve
> inputted is this:
>
> sudo ./bjam
> -–with-boost=/mnt/c/Users/mjk32/Desktop/MJK/tests/Machine_Translation/nmt-transfer-learning-xlm-r-master/boost_1_72_0
>
>
>
> I’m still a beginner in Linux and all of this, so I might have missed some
> obvious things and I apologize in advance if I did.
>
>
>
> Thank you for your time,
> Mujahid Khalifah
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Language Model Inquiry

[Hieu Hoang]

On 4/29/2021 5:27 AM, Marwa Gaser wrote:

Hello,

In the baseline training, what do the numbers in the below line 
represent?



3 for the 3-gram?

yes

How about 0 and 8?


0 means that the LM over the surface words. If your output has other 
factors, eg. Je|PRO suis|VB etudiant|ADJ, you can choose to have the LM 
on factor 1


8 means it uses KenLM, as opposed to SRILM or IRSTLM.



-lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:8

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] reg. moses installation

[Hieu Hoang]

cool. it would be good to say why the problem occurred in the first 
place and how you fixed it so others can learn from it



On 4/22/2021 10:32 AM, Namrata Hadimani wrote:

Hi Hieu,

Thanks for the help, I am able to successfully compile the moses ToolKit.

Regards,
Namrata Hadimani

On Thu, 22 Apr 2021 at 17:23, Hieu Hoang <mailto:hieuho...@gmail.com>> wrote:


i've just successfully compiled moses-4.0 on ubuntu 20.04 with
boost 1.71.

there's something wrong with your boost installation

On 4/22/2021 3:09 AM, ram anirudh cherukupally wrote:

There is atleast 60 GB space, so I think it is not space issue.
Has moses-4.0 been tested for compilation using boost 1.71? Do
you recommend using boost 1.64  (as exemplified in Moses manual?)
Thank you

On Thu, Apr 22, 2021 at 1:15 PM Hieu Hoang mailto:hieuho...@gmail.com>> wrote:

there seems to be a problem with the boost library. Is the
disk full? Perhaps you should re-install boost

On 4/21/2021 10:39 PM, ram anirudh cherukupally wrote:

Dear Moses-Support,

Please find the build.log.gz attached as per the
instructions when the build failed.
My system specs:

OS: Ubuntu 20.04
RAM: 8 GB
libboost-dev version: 1.71.0.0ubuntu2

Command used for compiling moses: ./bjam -j4

Thanks and Regards

-- 
CH Ram Anirudh



___
Moses-support mailing list
Moses-support@mit.edu  <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support  
<http://mailman.mit.edu/mailman/listinfo/moses-support>


    -- 
Hieu Hoang

http://statmt.org/hieu  <http://statmt.org/hieu>



-- 
CH Ram Anirudh


-- 
Hieu Hoang

http://statmt.org/hieu  <http://statmt.org/hieu>

___
Moses-support mailing list
Moses-support@mit.edu <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
<http://mailman.mit.edu/mailman/listinfo/moses-support>


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] reg. moses installation

[Hieu Hoang]

i've just successfully compiled moses-4.0 on ubuntu 20.04 with boost 1.71.

there's something wrong with your boost installation

On 4/22/2021 3:09 AM, ram anirudh cherukupally wrote:
There is atleast 60 GB space, so I think it is not space issue. Has 
moses-4.0 been tested for compilation using boost 1.71? Do you 
recommend using boost 1.64  (as exemplified in Moses manual?)

Thank you

On Thu, Apr 22, 2021 at 1:15 PM Hieu Hoang <mailto:hieuho...@gmail.com>> wrote:


there seems to be a problem with the boost library. Is the disk
full? Perhaps you should re-install boost

On 4/21/2021 10:39 PM, ram anirudh cherukupally wrote:

Dear Moses-Support,

Please find the build.log.gz attached as per the instructions
when the build failed.
My system specs:

OS: Ubuntu 20.04
RAM: 8 GB
libboost-dev version: 1.71.0.0ubuntu2

Command used for compiling moses: ./bjam -j4

Thanks and Regards

-- 
CH Ram Anirudh



___
Moses-support mailing list
Moses-support@mit.edu  <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support  
<http://mailman.mit.edu/mailman/listinfo/moses-support>


    -- 
Hieu Hoang

http://statmt.org/hieu  <http://statmt.org/hieu>



--
CH Ram Anirudh


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] reg. moses installation

[Hieu Hoang]

there seems to be a problem with the boost library. Is the disk full? 
Perhaps you should re-install boost


On 4/21/2021 10:39 PM, ram anirudh cherukupally wrote:

Dear Moses-Support,

Please find the build.log.gz attached as per the instructions when the 
build failed.

My system specs:

OS: Ubuntu 20.04
RAM: 8 GB
libboost-dev version: 1.71.0.0ubuntu2

Command used for compiling moses: ./bjam -j4

Thanks and Regards

--
CH Ram Anirudh


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Failed to install the mosesdecoder after installing mgiza++, irstlm and cmph

[Hieu Hoang]

Irstlm and cmph is not compulsory. They are needed for irst on and
compressed phrase take, respectively, but kenlm and probing pt are built
into Moses and are faster and better

Mgiza you will need unless you have another word alignment tool such as
giza++ or fastalign

On Tue, Mar 9, 2021, 4:57 AM L LL  wrote:

> In Ubuntu18.04, I installed the three components in their original
> downloading folder, which I’ m not sure it’ s okay. I didn’ t realize that
> these three components are all compulsory, so I tried for separate
> component to run the bjam command, which without doubt, end up failure. I’
> m not sure the previous try will influence the latter one, or perhaps there
> is something else that may cause this failure?
>
>
>
> This error occurred while I run this command: bjam
> --with-cmph=/home/aaa/cmph --with-irstlm=/home/aaa/irstlm
> --with-mgiza=/home/aaa/mgiza/mgizapp -j 2
>
> And log file is attached to this mail.
>
> Thanks!
>
>
>
> 发送自 Windows 10 版邮件 应用
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Error While Compiling Moses

[Hieu Hoang]

the unit tests are failing, but the compile seems to have succeeded.

I get a similar error compiling on a WSL2 environment but i know the 
compile itself has succeeded:


No such file or directory in 
/mnt/c/Users/hihoan/workspace/github/mosesdecoder/test.binaryDlFBwO 
(deleted) while resizing to 0 bytes Byte: 62


This is because the Windows file system doesn't support some reporting 
functionality,


Check that your disk is not full, check that bin/moses exists. If it 
does, then just ignore the unit test errors.



On Fri Jan 29, 21 12:12 AM, Marwa Gaser wrote:

Dear Moses Support Team,

I am using *wsl *on windows to run mosesdecoder for the first time.

What I did is that I cloned the mosesdecoder repo. Then I opened the 
cmd inside the mosesdecoder repo and ran the following command /*bjam 
-j4* /However, I got the error attached in the file.


Thank you in advance.

Respectfully,
Marwa

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] hierarchical pbmt issue

[Hieu Hoang]

also, the lack of disk space may be in /tmp, not just your output directory.

i'm not sure exactly how you run the training but in the command
extract-parallel.perl, the 'sort' should use a temp directory that has lots
space. So change your command
   ...extract-parallel.perl 40 split "sort"
to
   ...extract-parallel.perl 40 split "sort   -T [directory with lots of
space]"

Hieu Hoang
http://statmt.org/hieu


On Thu, 17 Dec 2020 at 02:31, Hieu Hoang  wrote:

> its likely you have run out of disk space during the extraction of the
> hierarchical phrase table. it requires about 10 times more disk space to
> train than the standard phrase-nased models
> On Thu Dec 17,20 2:10 AM, ram anirudh cherukupally wrote:
> Hi Everyone
>
> I have been trying to train a  hierarchical PBMT (without linguistic
> annotation) system using moses, with the switch --hierarchical to
> train-model.perl (as specified in the manual)
>
> I see that rule-table is not being generated during the training. Phrase
> table is being generated without any problem, but rule table is empty
> (rule-table.gz) turns out to be empty.
>
> Apparently, gzip is failing in Step 5. But I don't know why, because it
> appears to work fine with vanilla PBMT. I am sending the training log
> (stdout), Kindly help me.
>
> --
> CH Ram Anirudh
>
> --
> Hieu Hoanghttp://statmt.org/hieu
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] hierarchical pbmt issue

[Hieu Hoang]

its likely you have run out of disk space during the extraction of the 
hierarchical phrase table. it requires about 10 times more disk space to 
train than the standard phrase-nased models


On Thu Dec 17,20 2:10 AM, ram anirudh cherukupally wrote:
Hi Everyone

I have been trying to train a  hierarchical PBMT (without linguistic 
annotation) system using moses, with the switch --hierarchical to 
train-model.perl (as specified in the manual)


I see that rule-table is not being generated during the training. Phrase 
table is being generated without any problem, but rule table is empty 
(rule-table.gz) turns out to be empty.


Apparently, gzip is failing in Step 5. But I don't know why, because it 
appears to work fine with vanilla PBMT. I am sending the training log 
(stdout), Kindly help me.


--
CH Ram Anirudh

--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] MOSES | Baseline | Can't get similar BLEU

[Hieu Hoang]

That webpage is a good starting point to learn Moses. You should come 
out roughly the same BLEU


double check every command you run. A little thing like not tokenizing 
or truecasing correctly would kill your bleu.


Another example you can try is to reproduce the pre-made models 
available on the website. Download the config file eg


http://www.statmt.org/moses/RELEASE-4.0/models/de-en/steps/1/config.1

Change paths so they point to your executables and data.

These example uses the Experiment Management System, which is described 
here:


   http://www.statmt.org/moses/?n=FactoredTraining.EMS


On Mon Oct 19,20 8:52 PM, Paul Pallaghy wrote:

Hi all
I'm trying to reproduce the Moses baseline training for fr->en
http://www.statmt.org/moses/?n=Moses.Baseline 
<mailto:moses-support@mit.edu>
using around 130K sentences (as an exercise) and it works BUT instead 
of a BLEU of 23.50 like they do on that Moses webpage, I get quite low 
BLEU (7.0-ish by memory).


Anybody else experience this?
Any suggestions?
Any other worked training examples with reproducible BLEU on the web?

Paul Pallaghy


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Can't load text phrase table in Ubuntu 19.10 or 20.04

[Hieu Hoang]

If you are trying to load the text phrase-table it's likely that it runs 
out of memory. You must binarize the phrase table and use that instead 
of the text phrase-table.



On Mon Sep 28,20 7:33 AM, Harold Camacho wrote:

Greetings and apologies for bothering you.

I've been having trouble running the decoder that I've trained with 
the baseline system instructions and also one I've trained myself. The 
decoder used to run fine but after recompiling with XMLRPC support it 
can't load the text phrase table when running moses (even without the 
server argument), the OS automatically kills the process after a 
while. I'm having trouble with both ubuntu versions 19.10 and 20.04 (I 
tried on two different computers with 8 gb of RAM), and also have huge 
pages enabled.


Thank you very much for your help.

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Training The Translation System

[Hieu Hoang]

1. don't use ~, fully specify the path.

2. copy the mgiza or giza++ executables and scripts to 
~/mosesdecoder/tools. This is what i have in my directory


~/workspace/local/training-tools$ll total 12 drwxrwxr-x 2 s0565741 users 
4096 Jul 23 2017 ./ drwxr-xr-x 8 s0565741 users 4096 May 28 09:26 ../ 
lrwxrwxrwx 1 s0565741 users 72 Jul 23 2017 merge_alignment.py-> 
/home/s0565741/workspace/github/mgiza/mgizapp/script 
s/merge_alignment.py* lrwxrwxrwx 1 s0565741 users 55 Jul 23 2017 mgiza-> 
/home/s0565741/workspace/github/mgiza/mgizapp/bin/mgiza* lrwxrwxrwx 1 
s0565741 users 55 Jul 23 2017 mkcls-> 
/home/s0565741/workspace/github/mgiza/mgizapp/bin/mkcls* lrwxrwxrwx 1 
s0565741 users 58 Jul 23 2017 snt2cooc-> 
/home/s0565741/workspace/github/mgiza/mgizapp/bin/snt2cooc*


I recommend using mgiza rather than giza++

On Thu Sep 17,20 6:13 PM, Dhar wrote:


Hello,

This is my first email….so if I ask anything wrong , please treat with 
consideration


I am following moses manual . I have done everything successful upto 
the section 2.3.4 : LANGUAGE MODEL TRAINING  . But in 2.3.5 : TRAINING 
THE TRANSLATION SYSTEM I wrote the following code :


mkdir ~/working

cd ~/working

nohup nice ~/mosesdecoder/scripts/training/train-model.perl -root-dir 
train \


-corpus ~/corpus/news-commentary-v8.fr-en.clean

\

-f fr -e en -alignment grow-diag-final-and -reordering 
msd-bidirectional-fe \


-lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:8

\

-external-bin-dir ~/mosesdecoder/tools >& training.out &

Then I text file named ‘training.out’ created  in /home/dhar/working/ 
and this file gave me following message:


ERROR: Cannot find mkcls, GIZA++/mgiza, & snt2cooc.out/snt2cooc in
/home/rbehnam/mosesdecoder/external-bin-dir.
You MUST specify the parameter -external-bin-dir at
/home/rbehnam/mosesdecoder/scripts/training/train-model.perl line 491.
Best Regards,
Argha Dhar
Bachelor of Science in Computer Science and Engineering,
Khulna University of Engineering & Technology
Bangladesh


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Error when installing Moses in MacOS Catalina

[Hieu Hoang]

the log file doesn't show the errors.

However, there are issues with compiling on MacOS now due to diverging
compilers. See my reply to a previous email
   https://www.mail-archive.com/moses-support@mit.edu/msg16190.html

My suggestion - use Linux

Hieu Hoang
http://statmt.org/hieu


On Thu, 20 Aug 2020 at 13:30, Sainik Kumar Mahata 
wrote:

> Respected Sir,
>
> I have tried many ways to install moses in my macbook but to no avail. I
> have tried manually installing boost, installing boost through macports and
> the latest one being, installing boost using homebrew. But, every time I
> run into an error and the installation does not succeed. Attached herewith
> is the log file, as suggested in the error message. Kindly have a look into
> the same and kindly help me in this regard.
>
> --
>
> Regards,
>
> Mr. Sainik Kumar Mahata
>
> Visvesvaraya Ph.D Fellow
>
> Jadavpur University
> Contact: +91-9038141146
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Failed to get the language model by KenLM

[Hieu Hoang]

use lmplz -T [path]

and set path to somewhere with lots of disk space

On Wed Aug 5,20 12:08 PM, Chen, Y. wrote:

To whom it may concern,

I used Moses to train the language model by KenLM, however, I got the 
following error and I could not get the final model. Could you help me 
with this? Thank you in advance!


The command line I entered:
bzcat 40m_training_data.bz2 | python preprocess.py | 
./mosesdecoder/bin/lmplz -o 3 > 40m_training_data.arpa
Error message I got at the first step of Counting and sorting n-grams 
(out of 5 steps):

terminate called after throwing an instance of 'util::FDException'
  what():  util/file.cc:220 in void util::WriteOrThrow(int, const 
void*, std::size_t) threw FDException because `ret < 1'.
No space left on device in /tmp/lmWazsTd (deleted) while writing 
732919964 bytes

Aborted

Best,
 Yu-Wen

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Failed Moses 4.0 compilation: Attempt to free invalid pointer

[Hieu Hoang]

it works for me with the same library versions as yours. this is the
command i ran
   ./bjam -j11
--with-xmlrpc-c=/home/hihoan/hihoan/workspace/xmlrpc-c/xmlrpc-c-1.51.06
--with-cmph=/mnt/c/Users/hihoan/workspace/cmph/cmph-2.0.2
Perhaps there's something non-standard about your machine, have you
reconfigured it extensively? Or maybe there's a problem with the way you
compiled xmlrpc-c lib

Hieu Hoang
http://statmt.org/hieu


On Sun, 19 Jul 2020 at 22:03, ALISTAIR CHEONG LIANG CHUEN HCI <
1615...@student.hci.edu.sg> wrote:

> Dear Mr Hoang,
>
> I specified libssl-dev explicitly in apt-get install, but no action was
> taken because libssl-dev was apparently preinstalled in Colab machines with
> the latest version (1.1.1-1ubuntu2.1~18.04.6).
>
> The same errors appeared. Attached is the logfile.
>
> Thank you!
>
> Yours sincerely,
> Alistair Cheong
>
> On Sun, 19 Jul 2020 at 23:20, Hieu Hoang  wrote:
>
>> from the errors in build.log.gz, it looks like the compilation neeed the
>> SSL libraries. Try installing
>>libssl-dev
>> I've not seen this error before, please let us know how it goes
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>>
>> On Sun, 19 Jul 2020 at 02:50, ALISTAIR CHEONG LIANG CHUEN HCI <
>> 1615...@student.hci.edu.sg> wrote:
>>
>>> Dear Sir/Mdm,
>>>
>>> I am trying to use this Github library
>>> <https://github.com/artetxem/monoses> on Google Colab to translate
>>> Traditional Chinese to English. However I am facing some issues building
>>> Moses in the Colab environment.
>>>
>>> This Github library needed Moses2 functionality, which meant that I had
>>> to build cmph and xmlrpc myself. Thus, I downloaded the latest stable
>>> builds of both these libraries from their official SourceForge sites, and
>>> built them accordingly.
>>>
>>> To build Moses, I executed this command:
>>> bjam --with-cmph="/content/monoses/third-party/cmph-2.0.2"
>>> --with-xmlrpc-c="/content/monoses/third-party/xmlrpc-c-1.51.06" -j4
>>>
>>> However, I get 2 types of errors. The first one is collect2: error: ld
>>> returned 1 exit status, which happens as a result of 
>>> /content/monoses/third-party/xmlrpc-c-1.51.06/lib/abyss/src/socket_openssl.c:415:
>>> undefined reference to `SSL_get_fd'
>>>
>>> The second error is more frequent:
>>> == BEGIN OUTPUT ==
>>> src/tcmalloc.cc:283] Attempt to free invalid pointer 0x55959718c000
>>> Aborted (core dumped)
>>>
>>> EXIT STATUS: 134
>>> == END OUTPUT ==
>>>
>>> If you would like to see how I built the other libraries, here is my Colab
>>> notebook
>>> <https://colab.research.google.com/drive/1qg4QiT7Zrl-09kKq8Ysk-CTkkrSJS9Qa#scrollTo=xhg02HGYzo6d>
>>> .
>>>
>>> Attached is the logfile.
>>>
>>> Thank you for your time.
>>>
>>> Yours sincerely,
>>> Alistair Cheong
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Failed Moses 4.0 compilation: Attempt to free invalid pointer

[Hieu Hoang]

from the errors in build.log.gz, it looks like the compilation neeed the
SSL libraries. Try installing
   libssl-dev
I've not seen this error before, please let us know how it goes

Hieu Hoang
http://statmt.org/hieu


On Sun, 19 Jul 2020 at 02:50, ALISTAIR CHEONG LIANG CHUEN HCI <
1615...@student.hci.edu.sg> wrote:

> Dear Sir/Mdm,
>
> I am trying to use this Github library
> <https://github.com/artetxem/monoses> on Google Colab to translate
> Traditional Chinese to English. However I am facing some issues building
> Moses in the Colab environment.
>
> This Github library needed Moses2 functionality, which meant that I had to
> build cmph and xmlrpc myself. Thus, I downloaded the latest stable builds
> of both these libraries from their official SourceForge sites, and built
> them accordingly.
>
> To build Moses, I executed this command:
> bjam --with-cmph="/content/monoses/third-party/cmph-2.0.2"
> --with-xmlrpc-c="/content/monoses/third-party/xmlrpc-c-1.51.06" -j4
>
> However, I get 2 types of errors. The first one is collect2: error: ld
> returned 1 exit status, which happens as a result of 
> /content/monoses/third-party/xmlrpc-c-1.51.06/lib/abyss/src/socket_openssl.c:415:
> undefined reference to `SSL_get_fd'
>
> The second error is more frequent:
> == BEGIN OUTPUT ==
> src/tcmalloc.cc:283] Attempt to free invalid pointer 0x55959718c000
> Aborted (core dumped)
>
> EXIT STATUS: 134
> == END OUTPUT ==
>
> If you would like to see how I built the other libraries, here is my Colab
> notebook
> <https://colab.research.google.com/drive/1qg4QiT7Zrl-09kKq8Ysk-CTkkrSJS9Qa#scrollTo=xhg02HGYzo6d>
> .
>
> Attached is the logfile.
>
> Thank you for your time.
>
> Yours sincerely,
> Alistair Cheong
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Compiling failed with --with-nplm

[Hieu Hoang]

they look like linker errors, eg

/usr/bin/ld: util.cpp:(.text+0x1b4): undefined reference to 
`MKL_Set_Num_Threads' collect2: error: ld returned 1 exit status


make sure the environment variables LD_LIBRARY_PATH & LIBRARY_PATH point 
to the directory where the mkl and eigen libraries reside.


the code is quite old and no one has asked about NPLM for a long time. 
Don't be surprised if you encounter more problems using it.


On Wed Jul 15,20 10:15 PM, Googleユーザー wrote:

Dear all,

Compiling failed with the following command:
./bjam -j5 --with-nplm=/home/moses/nplm

After installing the latest version of Eigen and the latest MKL 
(l_mkl_2020.1.217), nplm commands were able to build.


I attached logs. Could you give me some advice?

Best Regards,
Hiroshi Kameya

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] GIZA++ word alignment with paragraphs

[Hieu Hoang]

Hi john

I'm afraid the word alignment tools like Giza++ aren't really designed to
be run against paragraph length input. Probably one reason why you're
getting bad alignments.

I don't know is tweaking the parameters would make it any better, or using
any other word alignment tool

On Wed, Jul 8, 2020, 6:19 PM John Thompson <
john.thompson.jtsoftw...@gmail.com> wrote:

> Hi,
>
> I'm using a 7162 line paragraph-aligned corpus. Unfortunately the
> translation within the paragraph sometimes don't have the sentences
> aligned, i.e. in one language the sentence could be one long sentence, and
> in another language the sentence could have clauses broken up into multiple
> sentences, hence I'm running GIZA++ on paragraphs.
>
> It works partially, but the alignment of words is often wrong or it's
> missing matches that should have been made.
>
> I set the "maxsentencelength" configuration file parameter to 350, though
> most of the paragraphs are around 100 or fewer words.
>
> Q1: What difference do you estimate I should expect between using
> paragraphs vs. sentences?
>
> Q2: Are there GIZA++ parameters I could tune to improve the alignment?
>
> Q3: If I concatenated multiple corpora, would the alignment output likely
> improve?
>
> I could preprocess the corpus, breaking up the paragraphs where the number
> of sentences match, but there may be some cases where the sentences don't
> align, where multiple sentences within the paragraph were joined or split
> differently, such that the sentence count of the paragraph is the same, but
> the sentences don't align.
>
> Q4: How big of effect would these bad sentence alignments have on the rest
> of the alignments?
>
> Q5: Any ideas for how to get better word alignment with these corpora that
> I have, either with GIZA++ or a different tool?
>
> I'm using the word alignment in a language study tool. For example, I have
> the text for a book in both English and Marshallese, but language resources
> for Marshallese are scarce. In my tool I associate the alignment
> information with the text, and also generate a dictionary using the
> alignment output (or optionally the *.dict.actual.ti.final dictionary list
> output). In one page I show the aligned sentences, and in another you can
> click on words to get both the alignment definition, and the dictionary
> definitions.  For each source word dictionary entry, I sort the target
> definitions by descending frequency (or probability if using the
> *.dict.actual.ti.final dictionary list output), and then chop off the list
> after a certain number, as otherwise there will be a lot of bad or spurious
> definitions included.
>
> Thanks!
>
> -John
>
> --
> John Thompson
> john.thompson.jtsoftw...@gmail.com
> https://www.jtlanguage.com
> 1-909-283-4364 (home)
> 1-909-283-5642 (cell)
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Segmentation fault in processPhraseTableMin

[Hieu Hoang]

i recommend using probing pt. It's built into moses and faster when
multithreading

Hieu Hoang
http://statmt.org/hieu


On Wed, 17 Jun 2020 at 14:23, Kenneth Heafield  wrote:

> The CMPH software updated formats at some point but nobody changed
> Moses.  Use a vintage CMPH or have fun hacking...
>
> On 6/17/20 10:08 PM, ser...@prompsit.com wrote:
> > The subject of my previous message is wrong. Actually the problem is
> > with queryPhraseTableMin as the content of the message suggests.
> >
> > El 17 jun. 2020 22:43, Sergio Ortiz Rojas 
> escribió:
> >
> > Hello,
> >
> > Moses on Ubuntu 18.04 with latest versions of cmph and boost
> > compiled from source, no other special compiling options, the program
> >
> > $ queryPhraseTableMin -a -t example.minphr
> >
> > returns a segfault on any non-OOV combination of tokens.
> >
> > The table example.minphr is obtained the usual way
> >
> > $ processPhraseTableMin -in model/phrase-table.gz -out example
> > -nscores 4
> >
> > Moses exacutable (moses) works with that table with no problem, it
> > seems just a problem with queryPhraseTableMin
> >
> > Debugging it a bit with gdb it seems like a problem with an empty
> > element (not NULL) inside a long loop at
> >
> > TargetPhraseVectorPtr PhraseDecoder::DecodeCollection
> >
> > at
> >
> > moses/TranslationModel/CompactPT/PhraseDecoder.cpp
> >
> > What do I need to do to get it working?
> >
> > Thanks
> >
> > Sergio
> >
> >
> >
> > ___
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> > ___
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] [EXT] Re: Error when compiling Moses with xmlrpc-c support

[Hieu Hoang]

i'm not sure why you need to add the ssl library, it shouldn't already be
included if it's needed. I use Ubuntu 18.04 as my main distro and I've
never needed to do this

If you don't need to compile with xmlrpc-c, then add --no-xmlrpc-c. You may
need to also add -a to recompile everything

Hieu Hoang
http://statmt.org/hieu


On Tue, 2 Jun 2020 at 12:48, Hank Kuo  wrote:

> To answer your questions:
>
>1. Both openssl and libssl-dev have already been installed prior to me
>building Moses and generating the build.log.gz output.
>2. That was a typo on my part, I was already specifying --with-cmph in
>my bjam command.
>
>
>
> Within the bjam build system for Moses, is there a way to explicitly link
> ssl during the g++ linking stage? This is the linking command that is
> erroring out during build (extracted from the build.log):
>
>
>
> "g++" -L"/usr/include/lib" -L"/usr/include/lib64" -L"/usr/local/lib"
> -Wl,-rpath-link -Wl,"/usr/include/lib" -Wl,-rpath-link
> -Wl,"/usr/include/lib64" -Wl,-rpath-link -Wl,"/usr/local/lib" -o
> "/home/whkuo/mosesdecoder/bin/moses" -Wl,--start-group
> "moses-cmd/bin/gcc-5.5.0/release/link-static/threading-multi/Main.o"
> "moses/bin/gcc-5.5.0/release/link-static/threading-multi/libmoses.a"
> "probingpt/bin/gcc-5.5.0/release/link-static/threading-multi/libprobingpt.a"
> -Wl,-Bstatic -lboost_filesystem -lboost_iostreams -lz -lbz2 -llzma -lcmph
> -lxmlrpc_xmltok -lxmlrpc_xmlparse -lxmlrpc_util++ -lxmlrpc_util
> -lxmlrpc_server_abyss++ -lxmlrpc_server_abyss -lxmlrpc_server++
> -lxmlrpc_server -lxmlrpc_abyss++ -lboost_program_options
> -lboost_serialization -lboost_thread -lboost_system -ltcmalloc_minimal
> -lxmlrpc -lxmlrpc++ -lxmlrpc_abyss -Wl,-Bdynamic -lSegFault -lrt
> -Wl,--end-group -pthread
>
>
>
> ...failed gcc.link /home/whkuo/mosesdecoder/bin/moses...
>
>
>
> Basically I’m looking for a way to add “-lssl” into the g++ command above…
>
>
>
> Hank
>
>
>
> *From:* Hieu Hoang 
> *Sent:* Friday, May 29, 2020 3:31 PM
> *To:* Hank Kuo 
> *Cc:* moses-support@mit.edu
> *Subject:* [EXT] Re: [Moses-support] Error when compiling Moses with
> xmlrpc-c support
>
>
>
> 2 things:
>
>   1. From the error message, it seems to need the open ssl library. So I
> guess you have to install openssl libssl-dev
>
>  2. it's --with-cmph not --withcmph
>
>
>
> Hieu Hoang
>
> http://statmt.org/hieu
>
>
>
>
>
> On Fri, 29 May 2020 at 12:03, Hank Kuo  wrote:
>
> Hello,
>
> I am encountering a compile error when building Moses with xmlrpc-c
> support.
>
> I downloaded, compiled, and installed xmlrpc-c-1.51.06 into a Ubuntu 18.04
> LTS VM.
>
> After installing xmlrpc-c-1.51.06, the command that I executed to compile
> Moses is:
>
> cd ~/mosesdecoder/
> ./bjam -a -j4 -withcmph=/usr/include
>
> See attached build.log.gz for the build output.
>
> Any help greatly appreciated.
>
> Thanks,
>
> Hank Kuo
> MITRE Corporation
> McLean, VA
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Error when compiling Moses with xmlrpc-c support

[Hieu Hoang]

2 things:
  1. From the error message, it seems to need the open ssl library. So I
guess you have to install openssl libssl-dev
 2. it's --with-cmph not --withcmph

Hieu Hoang
http://statmt.org/hieu


On Fri, 29 May 2020 at 12:03, Hank Kuo  wrote:

> Hello,
>
> I am encountering a compile error when building Moses with xmlrpc-c
> support.
>
> I downloaded, compiled, and installed xmlrpc-c-1.51.06 into a Ubuntu 18.04
> LTS VM.
>
> After installing xmlrpc-c-1.51.06, the command that I executed to compile
> Moses is:
>
> cd ~/mosesdecoder/
> ./bjam -a -j4 -withcmph=/usr/include
>
> See attached build.log.gz for the build output.
>
> Any help greatly appreciated.
>
> Thanks,
>
> Hank Kuo
> MITRE Corporation
> McLean, VA
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses Support Request

[Hieu Hoang]

do you need irstlm? If not, don't compile it with moses. I recommend using
kenlm instead

Hieu Hoang
http://statmt.org/hieu


On Mon, 25 May 2020 at 15:45, Johnas Solomon  wrote:

> Greetings!
> Kindly request your support. Please also find the log file and screenshot
> The last command I run was " ./bjam --with-boost=../boost_1_72_0
> --with-cmph=../cmph-2.0 --with-irstlm=../irstlm"
>
> Thank you!
>
>
>
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Word Lattice Input

[Hieu Hoang]

you might wanna take a look at some of the lattice regression tests, eg

https://github.com/moses-smt/moses-regression-tests/blob/master/tests/phrase.lattice-surface/moses.ini
specifically, add the lines that says  InputFeature

The error is because --weight-i 1 is the old way of specifying the input
feature.

Hieu Hoang
http://statmt.org/hieu


On Wed, 13 May 2020 at 23:00, Moses Visperas 
wrote:

> So I followed the tutorial for the baseline system and I want to try using
> a Lattice input. I made a test.plf file which contains 2 sentences
> formatted as shown in the manual (6.5.1) and was checked by the checkplf
> program.
>
> I used this command :
> ~/mosesdecoder/bin/moses -f moses.ini --inputtype 2 --max-phrase-length 13
> --weight-i 1 < test.plf > PLF13.en 2> PLF.out
> and I got an erro saying:
> Exception: moses/Parameter.cpp:572 in bool
> Moses::Parameter::LoadParam(int, const char**) threw util::Exception.
> Don't mix old and new ini file format
> and when I removed the " --weight-i 1" argument I get this :
> Exception: moses/ConfusionNet.cpp:72 in
> Moses::ConfusionNet::ConfusionNet(const ptr&) threw util::Exception because
> `InputFeature::InstancePtr() == __null'.
> Input feature must be specified
> Anyone can help me on how to set the input feature?
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Problem with moses installing and building

[Hieu Hoang]

you must use the full path to boost, cmph, nplm, irstlm, not relative
paths. Also, only include libraries you need to avoid headaches.

This is the command I use
   bjam -j24 --with-cmph=~/workspace/cmph/cmph-2.0

Hieu Hoang
http://statmt.org/hieu


On Wed, 6 May 2020 at 10:51, nikhil cherian  wrote:

> I have a problem in the build method of Moses. i have tried a couple of
> different solutions
>
> 1. in google colab, I used this command
>
> *!./bjam -j4 --with-cmph=../../cmph-2.0 --with-nplm=../../nplm/nplm
> --max-kenlm-order=9*
>
> but the build failed.
> [image: image.png]
>
> 2. In ubuntu, also the build failed with this command : ./bjam
> --with-boost boost 1_57_0 dir/ --with-irstlm irstlm dir
>
> Here, i had tried the boost to be in a separate directory and one time in
> the /usr/local/ dir. Both times, it did not work. I tried boost 1_57, 1_72
> etc. It still did not work.
>
> Kindly help
> Regards
> Nikhil Cherian
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Question about corpora used for pre-trained models of release 4.0

[Hieu Hoang]

they're trained on Europarl and tuned on wmt08 news test set, I believe.

You can find the exact experimental configuration by looking at the
experiment config file, eg
 http://www.statmt.org/moses/RELEASE-4.0/models/de-en/steps/1/config.1



Hieu Hoang
http://statmt.org/hieu


On Thu, 16 Apr 2020 at 13:46, Felipe Sánchez Martínez 
wrote:

> Hi all,
>
> A student of mine plans to use the phrase table available as a
> pre-trained model at
> http://www.statmt.org/moses/RELEASE-4.0/models/es-en/model/.
>
> Do you know if the test sets in WMT2006 and WMT2007 have been used for
> training those models. According to https://www.statmt.org/europarl/
> (Section Test Sets) they belong to the Q4/2000 portion of the data
> usually reserved for testing.
>
> Thanks a million!
> Regards
> --
> Felipe Sánchez Martínez
> Dept. de Llenguatges i Sistemes Informàtics
> Universitat d'Alacant
> Carretera Sant Vicent del Raspeig s/n
> 03690 Sant Vicent del Raspeig - Alacant (Spain)
> Tel.: +34 965 903 400, ext: 2966
> http://www.dlsi.ua.es/~fsanchez
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] help

[Hieu Hoang]

my advise though - use Linux instead of MacOSX.

The scripts are written for Linux and rarely tested on Mac, you're on you
own if there's a problem

Hieu Hoang
http://statmt.org/hieu


On Thu, 16 Apr 2020 at 09:55, Hieu Hoang  wrote:

> inmert/Jamfile, delete all the unit tests. if necessary, delete all
> unit tests in all Jamfiles you find
>
> Hieu Hoang
> http://statmt.org/hieu
>
>
> On Thu, 16 Apr 2020 at 09:51, Bounyalit Taymany 
> wrote:
>
>> Excuse me when I use the command " find . -name "*Test.cpp" | xargs rm "
>> and I try running bjam again. But It is like this
>>
>> $ ./bjam --with-boost=~/demo/boost_1_72_0 -j4
>>
>> XMLRPC-C: USING VERSION 1.39.12 FROM /usr/local/Cellar/xmlrpc-c/1.39.12
>>
>> fatal: No names found, cannot describe anything.
>>
>> Tip: install tcmalloc for faster threading.  See BUILD-INSTRUCTIONS.txt
>> for more information.
>>
>> BUILDING MOSES SERVER!
>>
>> Performing configuration checks
>>
>>
>> - Shared Boost : no
>>
>> - Static Boost : no
>>
>> error: Unable to find file or target named
>>
>> error: 'ScoreFeatureTest.cpp'
>>
>> error: referred from project at
>>
>> error: 'phrase-extract'
>>
>>
>> On Thu, 16 Apr 2020 at 12:25, Hieu Hoang  wrote:
>>
>>> Try deleting all the unit test file before running the bjam command
>>> again. You can delete the unit test files with the following command
>>>find . -name "*Test.cpp" | xargs rm
>>>
>>> Moses used to compile fine with macOS but macOS's compiler and gcc are
>>> slowly diverging.
>>>
>>> Hieu Hoang
>>> http://statmt.org/hieu
>>>
>>>
>>> On Wed, 15 Apr 2020 at 21:21, Bounyalit Taymany 
>>> wrote:
>>>
>>>> I used command for install Moses "./bjam
>>>> --with-boost=~/workspace/temp/boost_1_64_0 -j4"
>>>> on MacOS Catalina 10.15
>>>> but it failed
>>>>
>>>> ...failed darwin.compile.c++
>>>> mert/bin/darwin-11.0.3/release/link-static/threading-multi/UtilTest.o...
>>>>
>>>> ...skipped
>>>> util_test for
>>>> lack of
>>>> UtilTest.o...
>>>>
>>>> ...skipped
>>>> util_test.passed
>>>> for lack of
>>>> util_test...
>>>>
>>>> darwin.compile.c++
>>>> mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o
>>>>
>>>> clang: error: unknown argument: '-fcoalesce-templates'
>>>>
>>>>
>>>> "g++"  -ftemplate-depth-128 -fcoalesce-templates -O3
>>>> -finline-functions -Wno-inline -Wall -gdwarf-2 -fexceptions
>>>> -Wno-long-double -I/usr/local/Cellar/xmlrpc-c/1.39.12/include  -std=c++0x
>>>> -DBOOST_FILE_SYSTEM_DYN_LINK -DBOOST_PROGRAM_OPTIONS_DYN_LINK
>>>> -DBOOST_SYSTEM_DYN_LINK -DBOOST_TEST_DYN_LINK -DBOOST_THREAD_DYN_DLL
>>>> -DHAVE_XMLRPC_C -DKENLM_MAX_ORDER=6 -DMOSES_VERSION_ID=\"\" -DNDEBUG
>>>> -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES  
>>>> -I"."
>>>> -I"util/double-conversion" -I"~/demo/boost_1_72_0/include" -c -o
>>>> "mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o"
>>>> "mert/VocabularyTest.cpp"
>>>>
>>>>
>>>> ...failed darwin.compile.c++
>>>> mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o...
>>>>
>>>> ...skipped
>>>> vocabulary_test
>>>> for lack of
>>>> VocabularyTest.o...
>>>>
>>>> ...skipped
>>>> vocabulary_test.passed
>>>> for lack of
>>>> vocabulary_test...
>>>>
>>>> ...failed updating 762 targets...
>>>>
>>>> ...skipped 219 targets...
>>>> ___
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] help

[Hieu Hoang]

inmert/Jamfile, delete all the unit tests. if necessary, delete all
unit tests in all Jamfiles you find

Hieu Hoang
http://statmt.org/hieu


On Thu, 16 Apr 2020 at 09:51, Bounyalit Taymany  wrote:

> Excuse me when I use the command " find . -name "*Test.cpp" | xargs rm "
> and I try running bjam again. But It is like this
>
> $ ./bjam --with-boost=~/demo/boost_1_72_0 -j4
>
> XMLRPC-C: USING VERSION 1.39.12 FROM /usr/local/Cellar/xmlrpc-c/1.39.12
>
> fatal: No names found, cannot describe anything.
>
> Tip: install tcmalloc for faster threading.  See BUILD-INSTRUCTIONS.txt
> for more information.
>
> BUILDING MOSES SERVER!
>
> Performing configuration checks
>
>
> - Shared Boost : no
>
> - Static Boost : no
>
> error: Unable to find file or target named
>
> error: 'ScoreFeatureTest.cpp'
>
> error: referred from project at
>
> error: 'phrase-extract'
>
>
> On Thu, 16 Apr 2020 at 12:25, Hieu Hoang  wrote:
>
>> Try deleting all the unit test file before running the bjam command
>> again. You can delete the unit test files with the following command
>>find . -name "*Test.cpp" | xargs rm
>>
>> Moses used to compile fine with macOS but macOS's compiler and gcc are
>> slowly diverging.
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>>
>> On Wed, 15 Apr 2020 at 21:21, Bounyalit Taymany 
>> wrote:
>>
>>> I used command for install Moses "./bjam
>>> --with-boost=~/workspace/temp/boost_1_64_0 -j4"
>>> on MacOS Catalina 10.15
>>> but it failed
>>>
>>> ...failed darwin.compile.c++
>>> mert/bin/darwin-11.0.3/release/link-static/threading-multi/UtilTest.o...
>>>
>>> ...skipped
>>> util_test for
>>> lack of
>>> UtilTest.o...
>>>
>>> ...skipped
>>> util_test.passed
>>> for lack of
>>> util_test...
>>>
>>> darwin.compile.c++
>>> mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o
>>>
>>> clang: error: unknown argument: '-fcoalesce-templates'
>>>
>>>
>>> "g++"  -ftemplate-depth-128 -fcoalesce-templates -O3
>>> -finline-functions -Wno-inline -Wall -gdwarf-2 -fexceptions
>>> -Wno-long-double -I/usr/local/Cellar/xmlrpc-c/1.39.12/include  -std=c++0x
>>> -DBOOST_FILE_SYSTEM_DYN_LINK -DBOOST_PROGRAM_OPTIONS_DYN_LINK
>>> -DBOOST_SYSTEM_DYN_LINK -DBOOST_TEST_DYN_LINK -DBOOST_THREAD_DYN_DLL
>>> -DHAVE_XMLRPC_C -DKENLM_MAX_ORDER=6 -DMOSES_VERSION_ID=\"\" -DNDEBUG
>>> -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES  -I"."
>>> -I"util/double-conversion" -I"~/demo/boost_1_72_0/include" -c -o
>>> "mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o"
>>> "mert/VocabularyTest.cpp"
>>>
>>>
>>> ...failed darwin.compile.c++
>>> mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o...
>>>
>>> ...skipped
>>> vocabulary_test
>>> for lack of
>>> VocabularyTest.o...
>>>
>>> ...skipped
>>> vocabulary_test.passed
>>> for lack of
>>> vocabulary_test...
>>>
>>> ...failed updating 762 targets...
>>>
>>> ...skipped 219 targets...
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] help

[Hieu Hoang]

Try deleting all the unit test file before running the bjam command again.
You can delete the unit test files with the following command
   find . -name "*Test.cpp" | xargs rm

Moses used to compile fine with macOS but macOS's compiler and gcc are
slowly diverging.

Hieu Hoang
http://statmt.org/hieu


On Wed, 15 Apr 2020 at 21:21, Bounyalit Taymany  wrote:

> I used command for install Moses "./bjam
> --with-boost=~/workspace/temp/boost_1_64_0 -j4"
> on MacOS Catalina 10.15
> but it failed
>
> ...failed darwin.compile.c++
> mert/bin/darwin-11.0.3/release/link-static/threading-multi/UtilTest.o...
>
> ...skipped
> util_test for
> lack of
> UtilTest.o...
>
> ...skipped
> util_test.passed
> for lack of
> util_test...
>
> darwin.compile.c++
> mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o
>
> clang: error: unknown argument: '-fcoalesce-templates'
>
>
> "g++"  -ftemplate-depth-128 -fcoalesce-templates -O3
> -finline-functions -Wno-inline -Wall -gdwarf-2 -fexceptions
> -Wno-long-double -I/usr/local/Cellar/xmlrpc-c/1.39.12/include  -std=c++0x
> -DBOOST_FILE_SYSTEM_DYN_LINK -DBOOST_PROGRAM_OPTIONS_DYN_LINK
> -DBOOST_SYSTEM_DYN_LINK -DBOOST_TEST_DYN_LINK -DBOOST_THREAD_DYN_DLL
> -DHAVE_XMLRPC_C -DKENLM_MAX_ORDER=6 -DMOSES_VERSION_ID=\"\" -DNDEBUG
> -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES  -I"."
> -I"util/double-conversion" -I"~/demo/boost_1_72_0/include" -c -o
> "mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o"
> "mert/VocabularyTest.cpp"
>
>
> ...failed darwin.compile.c++
> mert/bin/darwin-11.0.3/release/link-static/threading-multi/VocabularyTest.o...
>
> ...skipped
> vocabulary_test
> for lack of
> VocabularyTest.o...
>
> ...skipped
> vocabulary_test.passed
> for lack of
> vocabulary_test...
>
> ...failed updating 762 targets...
>
> ...skipped 219 targets...
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Trouble building in ubuntu 19.10

[Hieu Hoang]

it compiled successfully on my ubuntu 19.10. My g++ version is 9.2.1. I
installed the boost (v. 1.67) using apt instal.

What are your versions

Hieu Hoang
http://statmt.org/hieu


On Sun, 12 Apr 2020 at 01:42, Harold Camacho 
wrote:

> Hello and apologies for bothering.
>
> I've had trouble trying to build moses SMT
>
> The command I used was:
> ./bjam --with-boost=/home/harold/Downloads/mosesdecoder/boost_1_64_0 -j4
>
> Even tough I specified it to use boost, the build still used gcc by
> default.  I tried many times and always executing "./bjam --clean"
> beforehand.
>
> Thank you very much for your help.
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Tokenization

[Hieu Hoang]

the moses tokenizer expects the input from standard in

Hieu Hoang
http://statmt.org/hieu


On Sun, 12 Apr 2020 at 10:27, Justin Cunningham 
wrote:

> Hi,
>
> I’m currently working on a Neural Machine Translator but I am quite new to
> it all. I am trying to tokenise my files in Linux using the following shell
> script (https://github.com/JustCunn/IrishNMT/blob/master/GaeilgePrepare.sh)
> and these files:
>
> http://opus.nlpl.eu/download.php?f=EUbookshop/v2/moses/en-ga.txt.zip
> <http://opus.nlpl.eu/download.php?f=EUbookshop/v2/moses/de-fr.txt.zip>
> http://opus.nlpl.eu/download.php?f=QED/v2.0a/moses/en-ga.txt.zip
>
> But it just won’t work. Sometimes it will skip it, others it will just be
> stuck on the ‘Tokenizer... number of threads...”. For context, they are all
> plain text files. Am I not formatting the text correctly?
>
> I’d appreciate if someone could help me with this as it would be a huge
> help in my understanding of it all.
>
> Thanks,
> Justin
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] baseline model run with same setting, with same database twice gives different translation output

[Hieu Hoang]

If you mean that you trained twice, it's normal to get slightly different
results. The word alignment and running are stocastic


On Sun, Mar 29, 2020, 3:49 AM rmogla  wrote:

> Hi,
> I ran Moses baseline method twice with the same settings and with the same
> data set, but final translation file generated in both the cases is
> different.
>
> If i am doing something wrong please suggest?
> Thanks and regards
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] where to ask a question? mailing list archive is dead.

[Hieu Hoang]

Hieu Hoang
http://statmt.org/hieu


On Tue, 24 Mar 2020 at 10:30, Artem Shevchenko  wrote:

> Hello,
>
> wow, thank you very much for a so rapid reply!
>
> My idea was first to try with pre-made models to see how good it can be
> and whether I would really need to go all the way for the Baseline
> - or whether I can just incrementally train the pre-made by you model
> using my additional input.
> This would save a lot of time and give a "quick proof of concept" for my
> boss so that he would be then ready to invest more time and effort into
> further teaching the model.
>
> what is confusing is this large number of files in /model/ without any
> explanation which of them is for what.
> why e.g. are there so many moses.ini.1,2,3,...18? Are these different
> experiments?
>
yeah, with different features. They are described in the release notes
   http://www.statmt.org/moses/RELEASE-4.0/Moses-v4.0-release-notes.pdf

I think it would make it much easier for newcomers like me to jump in and
> continue on top of what others have already achieved rather than each time
> invent a wheel.
>
> Thank you vermy much for your work!
>
> Best regards
> Artem Shevchenko
>
>
>
>
>
> вт, 24 мар. 2020 г. в 18:06, Hieu Hoang :
>
>> apologies, the link to the archive is dead, but not the archive or
>> mailing list itself. I've updated the link to
>>https://www.mail-archive.com/moses-support@mit.edu/
>>
>> To answer your question:
>>
>> The pre-made translation models actually gives you all the files that
>> were created during training. If you only want to translate with the
>> existing model, grab 1 of the tuned moses.ini files, eg
>>
>> http://www.statmt.org/moses/RELEASE-4.0/models/de-en/tuning/moses.tuned.ini.1
>> and download only the files it refers to.
>>
>> However, I would say you should train your own models, on your own data.
>> to get good results. If you are new to Moses, you should follow this
>> tutorial to do so
>>http://www.statmt.org/moses/?n=Moses.Baseline
>>
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>>
>> On Tue, 24 Mar 2020 at 09:47, Artem Shevchenko  wrote:
>>
>>> Hello,
>>> Search the mailing list archive
>>> <http://blog.gmane.org/gmane.comp.nlp.moses.user> is dead. So the
>>> question is then:
>>> Who do I ask if my question has not been answered by this FAQ?
>>>
>>> "Search the mailing list archive
>>> <http://blog.gmane.org/gmane.comp.nlp.moses.user>. If you still do not
>>> find the answer, then send questions to the mailing list 'moses-support'.
>>> However, you have to sign up
>>> <http://mailman.mit.edu/mailman/listinfo/moses-support> before
>>> emailing."
>>> I signed up, but still do not know where to ask my question.
>>> Thank you!
>>>
>>> TO my question:
>>> I am trying to use the pre-made translation model of 4.0 verison.
>>> Do I need all those files or only some of them?
>>> Need a description on how to apply the pre-made model.
>>>
>>> Best regards
>>> Artem Shevchenko
>>>
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] where to ask a question? mailing list archive is dead.

[Hieu Hoang]

apologies, the link to the archive is dead, but not the archive or mailing
list itself. I've updated the link to
   https://www.mail-archive.com/moses-support@mit.edu/

To answer your question:

The pre-made translation models actually gives you all the files that were
created during training. If you only want to translate with the existing
model, grab 1 of the tuned moses.ini files, eg

http://www.statmt.org/moses/RELEASE-4.0/models/de-en/tuning/moses.tuned.ini.1
and download only the files it refers to.

However, I would say you should train your own models, on your own data. to
get good results. If you are new to Moses, you should follow this tutorial
to do so
   http://www.statmt.org/moses/?n=Moses.Baseline


Hieu Hoang
http://statmt.org/hieu


On Tue, 24 Mar 2020 at 09:47, Artem Shevchenko  wrote:

> Hello,
> Search the mailing list archive
> <http://blog.gmane.org/gmane.comp.nlp.moses.user> is dead. So the
> question is then:
> Who do I ask if my question has not been answered by this FAQ?
>
> "Search the mailing list archive
> <http://blog.gmane.org/gmane.comp.nlp.moses.user>. If you still do not
> find the answer, then send questions to the mailing list 'moses-support'.
> However, you have to sign up
> <http://mailman.mit.edu/mailman/listinfo/moses-support> before emailing."
> I signed up, but still do not know where to ask my question.
> Thank you!
>
> TO my question:
> I am trying to use the pre-made translation model of 4.0 verison.
> Do I need all those files or only some of them?
> Need a description on how to apply the pre-made model.
>
> Best regards
> Artem Shevchenko
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Tuning a factored model -> crash

[Hieu Hoang]

Hieu Hoang
http://statmt.org/hieu


On Mon, 9 Mar 2020 at 10:08, Haukur Páll Jónsson  wrote:

> Hi all,
>
>
> For the last few weeks, I have been trying to train and tune a factored
> model. I have found it difficult to implement and I now seek some
> assistance.
>
>
> I am trying to build a simple factored model from English to Icelandic:
> T0-0, T0,1-1. Where factor 0 is the surface and factor 1 is the POS. The
> training data (source and target) I have has three factors
> `surface|pos|lemma`, the lemma is ignored for now.
>
>
> When tuning a factored model I run into problems. My first question is,
> what factors should be in tuning data? It seems that I can have all factors
> as the input/source but I'm unsure about the output/target.
>
The tuning data needs to also have factors 0 and 1. You should pre-process
ttraining, tuning and test data in the same way

I run the tuning like so (using 10 threads):

>
>
> "$MOSESDECODER"/scripts/training/mert-moses.pl \
> "$DEV_DATA_IN"."$LANG_FROM" \
> "$DEV_DATA_OUT"."$LANG_TO" \
> "$MOSESDECODER"/bin/moses "$BASE_MOSES_INI" \
> --mertdir "$MOSESDECODER"/bin \
> --working-dir "$TUNE_DIR" \
> --decoder-flags="-threads $THREADS"
>
>
> But then when starting to decode the decoder crashes.
>
>
> Line Line 9: Initialize search took leave|VBP0.103 seconds total
> soft|JJ hands|NNS ,|, Jess|NNP you|PRP very|RB weak|JJ ,|, including|VBG
> 4: Initialize search took  .|.
>  : Collecting options took 0.003severe|JJ ,|, his|PRP$I|PRP loved|VBD
> Sean|NNP .|. you|PRP know|VBP .|.
>   seconds at moses/Manager.cpp Line 0.129 seconds total
> joint|JJ examination|NN Line 2: Initialize search took 141
>  Segmentation fault
> Exit code: 139
>
>
> I am monitoring the memory usage and the decoder is only using about 4GB
> of memory from the 32GB allocated when it crashes. Why the is the decoder
> crashing? Are there some recommendations for settings when training a
> factored model?


>
>
> Haukur Páll Jónsson
>
> Rannsóknarsérfræðingur | Tölvunarfræðideild
>
> Research Specialist | School of Computer Science
>
> Póstfang / E-mail: hauku...@ru.is
>
> [156021669]
>
> Háskólinn í Reykjavík | Reykjavik University
>
> Menntavegur 1 | 101 Reykjavík | Iceland
>
> Sími/Tel: +354 599 6200
>
> www.hr.is
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses Decoder Parameters

[Hieu Hoang]

the weights are in the moses.ini file, eg
[weight]
WordPenalty0= -0.131325
PhrasePenalty0= 0.064328
TranslationModel0= 0.0226604 0.02326 0.211062 0.00701641
LexicalReordering0= 0.12109 0.0194831 0.0563125 0.136496 0.0459526 0.0631051
Distortion0= 0.0218625
LM0= 0.0760463
UnknownWordPenalty0= 1

You can change the max phrase length of the decoder by using
--max-phrase-length when running the decoder

If you are new to Moses, it's best to work through this tutorial
   http://www.statmt.org/moses/?n=Moses.Baseline


Hieu Hoang
http://statmt.org/hieu


On Fri, 28 Feb 2020 at 19:19, Moses Visperas 
wrote:

> Hi all,
>
>
>
> I’ve been seeing a lot of papers where they they changed the decoder
> parameters such as the weights of the translation model, language model,
> distortion model, and the word penalty. Some change the max phrase lengths
> of the decoder. Where do I exactly change this parameters in order to
> achieve a better translation ?
>
>
>
>
>
>
> Thank you!
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Problem downloading cmph file.

[Hieu Hoang]

You need to use
   sudo make install

On Sun, Jan 26, 2020, 8:25 AM Nameet Mankar  wrote:

> May this image help me answer my query can anyone suggest a solution if
> you'll encountered same situation
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses installation failed

[Hieu Hoang]

you need to install the bz2 library, then recompile the boost library from
the beginning. On Ubuntu, do this
   sudo apt install libbz2-dev
See here for install on other linux distributions
  http://www.statmt.org/moses/?n=Development.GetStarted
Alternatively, comment out mentions of bz2 in
   util/Jamfile

Hieu Hoang
http://statmt.org/hieu


On Thu, 26 Dec 2019 at 12:28, Bamba  wrote:

> Hello,
> I am trying to install Moses on ubuntu 18.04, but the installation always
> fails. I have already installed boost successfully. Please find attached
> the build.log.gz file and the command I have used below.
>
> /usr/bin/bjam --with-boost=/Home/siv22/dba007/Tools/boost_1_65_0
> --with-cmph=/Home/siv22/dba007/Tools/NMT/monoses-master/cmph-2.0
> --no-xmlrpc-c --with-mm --with-probing-pt -j16
>
>
>
> Thanks in advance for any help.
>
> Best regards,
> Bamba
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Segmentation fault (core dumped) while translating

[Hieu Hoang]

hi there

the input sentence can't contain <> characters.

If you had tokenized the input with the moses script

     /scripts/tokenizer/tokenizer.perl
then those characters would have been converted to  

If you used your own tokenizer, be sure to

    /scripts/tokenizer/escape-special-chars.perl

before feeding it to the moses decoder.

And of course, you should process your training data the same way you 
process your test and tuning data

On 12/10/2019 2:13 AM, Claudia Matos Veliz wrote:
> Hello everyone
>
> I’m getting a very estrange error while translating using Moses. I’m 
> doing 10-CV experiments and in one of the folds I get a Segmentation 
> Fault error. For the rest everything finished without problem. I hope 
> you can help me with that. Bellow you can see the log. I get the error 
> in the first line of the file I want to translate. I’ve tried removing 
> the line and I got the same error.
>
> $ /opt/mosesdecoder-4.0-prebuilt/moses/bin/moses -config 
> ../model/moses.ini -input-file token.all.fold2.test.clean.ori > 
> token.all.fold2.test.norm
> Defined parameters (per moses.ini or switch):
> config: ../model/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty 
> PhraseDictionaryOnDisk name=TranslationModel0 num-features=4 
> path=/home/claudia/SMT_ext_data/data/mixed/amica/1folds/2/model/phrase-table.table
>  
> input-factor=0 output-factor=0 LexicalReordering 
> name=LexicalReordering0 num-features=6 
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 
> path=/home/claudia/SMT_ext_data/data/mixed/amica/1folds/2/model/reordering-table.wbe-msd-bidirectional-fe.gz
>  
> Distortion KENLM name=LM0 factor=0 
> path=/home/claudia/SMT_ext_data/data/lm/lm/sms.cgn.token.lm order=5
> input-factors: 0
> input-file: token.all.fold2.test.clean.ori
> mapping: 0 T 0
> weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 
> TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 
> 0.3 0.3 Distortion0= 0.3 LM0= 0.5
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryOnDisk name=TranslationModel0 num-features=4 
> path=/home/claudia/SMT_ext_data/data/mixed/amica/1folds/2/model/phrase-table.table
>  
> input-factor=0 output-factor=0
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=LexicalReordering name=LexicalReordering0 num-features=6 
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 
> path=/home/claudia/SMT_ext_data/data/mixed/amica/1folds/2/model/reordering-table.wbe-msd-bidirectional-fe.gz
> Initializing Lexical Reordering Feature..
> FeatureFunction: LexicalReordering0 start: 7 end: 12
> line=Distortion
> FeatureFunction: Distortion0 start: 13 end: 13
> line=KENLM name=LM0 factor=0 
> path=/home/claudia/SMT_ext_data/data/lm/lm/sms.cgn.token.lm order=5
> Loading the LM will be faster if you build a binary file.
> Reading /home/claudia/SMT_ext_data/data/lm/lm/sms.cgn.token.lm
> 5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> 
> FeatureFunction: LM0 start: 14 end: 14
> Loading UnknownWordPenalty0
> Loading WordPenalty0
> Loading PhrasePenalty0
> Loading LexicalReordering0
> Loading table into memory...done.
> Loading Distortion0
> Loading LM0
> Loading TranslationModel0
> Created input-output object : [2.910] seconds
> Translating:  ik vind die zo mooii . ik hou van je 
> Segmentation fault (core dumped)
>
>
>
>
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-- 
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Compact phrase table on-disk format changed

[Hieu Hoang]

Thanks. I'll bear it in mind for next time

On Sun, Dec 8, 2019, 9:19 AM Dingyuan Wang  wrote:

> The problem is caused by the upgrade of cmph library from 2.0 to 2.0.2.
> I don't know which commit of cmph changed its format.
>
> 2019/12/8 3:46, Hieu Hoang:
> > That's strange, I expected that they would work all versions. Another
> > problem might be the encoding
> >
> > Please let me know what you find
> >
> > On Sat, Dec 7, 2019, 9:15 AM Dingyuan Wang  > <mailto:abcdoyle...@gmail.com>> wrote:
> >
> > Dear all,
> >
> > After re-compiling moses binaries, moses can't find all Chinese words
> > (|UNK|UNK|UNK) in the compact phrase tables built by the older
> binary.
> > ASCII characters are not affected. If I rebuild the compact table
> using
> > the new binaries, it works with the new binary but not the old one.
> >
> > The versions are below:
> >
> > (GCC 8.2.0)
> > Moses code version (git tag or commit hash):
> >mmt-mvp-v0.12.1-11198-g3545225c0-dirty
> > Libraries used:
> >   Boost  version 1.62.0
> >Xmlrpc-c  version 1.33.14
> >CMPH (version unknown)
> >
> > (GCC 9.2.1)
> > Moses code version (git tag or commit hash):
> >mmt-mvp-v0.12.1-11270-g41b31167f-dirty
> > Libraries used:
> >   Boost  version 1.71.0
> >Xmlrpc-c  version 1.33.14
> >CMPH (version unknown)
> >
> > I think the difference should be because of the Boost library. I'll
> get
> > the old Boost library for a try.
> >
> > ___
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Compact phrase table on-disk format changed

[Hieu Hoang]

That's strange, I expected that they would work all versions. Another
problem might be the encoding

Please let me know what you find

On Sat, Dec 7, 2019, 9:15 AM Dingyuan Wang  wrote:

> Dear all,
>
> After re-compiling moses binaries, moses can't find all Chinese words
> (|UNK|UNK|UNK) in the compact phrase tables built by the older binary.
> ASCII characters are not affected. If I rebuild the compact table using
> the new binaries, it works with the new binary but not the old one.
>
> The versions are below:
>
> (GCC 8.2.0)
> Moses code version (git tag or commit hash):
>mmt-mvp-v0.12.1-11198-g3545225c0-dirty
> Libraries used:
>   Boost  version 1.62.0
>Xmlrpc-c  version 1.33.14
>CMPH (version unknown)
>
> (GCC 9.2.1)
> Moses code version (git tag or commit hash):
>mmt-mvp-v0.12.1-11270-g41b31167f-dirty
> Libraries used:
>   Boost  version 1.71.0
>Xmlrpc-c  version 1.33.14
>CMPH (version unknown)
>
> I think the difference should be because of the Boost library. I'll get
> the old Boost library for a try.
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses installation fails

[Hieu Hoang]

there may be a problem with the latest version of cmph 2.0.2 so use version
2.0

I have it here if you need it
   http://statmt.org/hieu/downloads/cmph-2.0.tar.gz
The usual commands to build it is
   ./configure --prefix=$PWD
make
make install

Hieu Hoang
http://statmt.org/hieu


On Tue, 26 Nov 2019 at 21:27, Hieu Hoang  wrote:

> searching for the word 'error' reveals that the cmph library hasn't been
> compiled successfully.
>
> Hieu Hoang
> http://statmt.org/hieu
>
>
> On Tue, 26 Nov 2019 at 21:19, Bamba  wrote:
>
>> Hello,
>> I am trying to install Moses on ubuntu 18.04, but the installation always
>> fails. I have already installed boost successfully. Please find attached
>> the build.log.gz file and the command I have used below:
>>
>> /usr/bin/bjam
>> --with-boost=/home/Tools/smt/monoses-master/third-party/moses/opt
>> --with-cmph=/home/Tools/smt/monoses-master/third-party/moses/opt
>> --no-xmlrpc-c --with-mm --with-probing-pt -j16
>>
>>
>>
>> Thanks in advance for any help.
>>
>> Best regards,
>> Bamba
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses installation fails

[Hieu Hoang]

searching for the word 'error' reveals that the cmph library hasn't been
compiled successfully.

Hieu Hoang
http://statmt.org/hieu


On Tue, 26 Nov 2019 at 21:19, Bamba  wrote:

> Hello,
> I am trying to install Moses on ubuntu 18.04, but the installation always
> fails. I have already installed boost successfully. Please find attached
> the build.log.gz file and the command I have used below:
>
> /usr/bin/bjam
> --with-boost=/home/Tools/smt/monoses-master/third-party/moses/opt
> --with-cmph=/home/Tools/smt/monoses-master/third-party/moses/opt
> --no-xmlrpc-c --with-mm --with-probing-pt -j16
>
>
>
> Thanks in advance for any help.
>
> Best regards,
> Bamba
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] mert-moses using parallel mode

[Hieu Hoang]

you should read this:
   http://www.statmt.org/moses/?n=Moses.Optimize

Moses doesn't do tensors operations so GPUs are of no help so no supported

Hieu Hoang
http://statmt.org/hieu


On Tue, 5 Nov 2019 at 04:42, Каргин Тимофей  wrote:

> I started to work with moses and stuck with too long tuning.
> Now I use only one CPU thread, just like it said in the
> http://www.statmt.org/moses/?n=Moses.Baseline .
> Is there any guide how to start tuning on CPU or even better - GPU using
> CUDA?
> I have found mert-moses.pl options such as --jobs=N, --threads=N. But
> this things doesn't work with me - no errors, but only one thread.
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Building mosesdecoder on Mac OS

[Hieu Hoang]

That's odd, it should be using clang rather than clang++ to compile cmd.c

I've created a branch to try and fix it
  https://github.com/moses-smt/mosesdecoder/tree/clang-error
so git checkout clang-error and try to compile again

It looks like clang/osx build is starting to seriously diverge from gcc and
could be a long-term problem. I don't have time to deal with it anymore so
you may have to retreat to gcc on linux if there are lots more issues




On Thu, Oct 31, 2019, 12:55 PM Hrafn Loftsson  wrote:

> Thanks for a quick reply.
>
>
> Yes, I was planning to use moses only for command line experiments.
>
>
> Following your advice, I uninstalled the xmlrpc-c library and now I build
> using:
>
>
> ./bjam --with-boost=/Users/hrafn/nlp/smt/boost_1_71_0/ -j4 toolset=clang
>
>
> Unfortunately, I get a similar error:
>
>
> clang-darwin.compile.c
> symal/bin/clang-darwin-4.2.1/release/link-static/threading-multi/cmd.o
>
> "clang++" -x c -O3 -std=c++0x -O3 -finline-functions -Wno-inline -Wall
> -DKENLM_MAX_ORDER=6 -DMOSES_VERSION_ID=\"mmt-mvp-v0.12.1-2869-g0fef8ebf4\"
> -DNDEBUG -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64
> -D_LARGE_FILES -I"." -I"/Users/hrafn/nlp/smt/boost_1_71_0/include" -c -o
> "symal/bin/clang-darwin-4.2.1/release/link-static/threading-multi/cmd.o"
> "symal/cmd.c"
>
> error: invalid argument '-std=c++0x' not allowed with 'C'
> ...failed clang-darwin.compile.c
> symal/bin/clang-darwin-4.2.1/release/link-static/threading-multi/cmd.o...
>
>
>
> Regards,
>
> Hrafn Loftsson, Ph.D., www.ru.is/~hrafn<http://www.ru.is/~hrafn>
> Dósent | Associate Professor
> Tölvunarfræðideild | School of Computer Science
> Háskólinn í Reykjavík | Reykjavik University
>
>
> 
> Frá: Hieu Hoang 
> Sent: 31. október 2019 19:33
> Til: Hrafn Loftsson
> Afrit: moses-support
> Efni: Re: [Moses-support] Building mosesdecoder on Mac OS
>
> It's been a few years since I've used osx.
>
> My advice is
>   1. Don't include cmph library. It's only used by the compact phrase
> table which crashes on osx
>2. If you're only using the moses command line to do experiments, you
> don't need the server so you don't need to include xmlrpc-c library.
>
> On Thu, Oct 31, 2019, 12:17 PM Hrafn Loftsson  hr...@ru.is>> wrote:
> Hi.
>
>
> I'm trying to build moses on Mac OS Mojave (10.14.6).
>
>
> I have downloaded and built boost with success.
>
>
> According to http://www.statmt.org/moses/?n=Development.GetStarted, I
> need to install xmlrpc-c independently, which I did using brew.
>
>
> This is the exact command I use when trying to build moses:
>
>
> ./bjam --with-boost=/Users/hrafn/nlp/smt/boost_1_71_0/
> --with-cmph=/usr/local/Cellar/xmlrpc-c/1.39.12/ -j4 toolset=clang
>
>
> I have attached the build.log
>
>
> When inspecting the log, I can see the following:
>
>
> "clang++" -x c -O3 -I/usr/local/Cellar/xmlrpc-c/1.39.12/include
> -std=c++0x -O3 -finline-functions -Wno-inline -Wall -DHAVE_CMPH
> -DHAVE_XMLRPC_C -DKENLM_MAX_ORDER=6
> -DMOSES_VERSION_ID=\"mmt-mvp-v0.12.1-2869-g0fef8ebf4\" -DNDEBUG
> -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -I"."
> -I"/Users/hrafn/nlp/smt/boost_1_71_0/include" -c -o
> "symal/bin/clang-darwin-4.2.1/release/link-static/threading-multi/cmd.o"
> "symal/cmd.c"
>
> error: invalid argument '-std=c++0x' not allowed with 'C'
>
>
> Does this mean that C code is being compiled here, with an invalid c++
> option?
>
>
> Regards,
> Hrafn
> ___
> Moses-support mailing list
> Moses-support@mit.edu<mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Building mosesdecoder on Mac OS

[Hieu Hoang]

It's been a few years since I've used osx.

My advice is
  1. Don't include cmph library. It's only used by the compact phrase table
which crashes on osx
   2. If you're only using the moses command line to do experiments, you
don't need the server so you don't need to include xmlrpc-c library.

On Thu, Oct 31, 2019, 12:17 PM Hrafn Loftsson  wrote:

> Hi.
>
>
> I'm trying to build moses on Mac OS Mojave (10.14.6).
>
>
> I have downloaded and built boost with success.
>
>
> According to http://www.statmt.org/moses/?n=Development.GetStarted, I
> need to install xmlrpc-c independently, which I did using brew.
>
>
> This is the exact command I use when trying to build moses:
>
>
> ./bjam --with-boost=/Users/hrafn/nlp/smt/boost_1_71_0/
> --with-cmph=/usr/local/Cellar/xmlrpc-c/1.39.12/ -j4 toolset=clang
>
>
> I have attached the build.log
>
>
> When inspecting the log, I can see the following:
>
>
> "clang++" -x c -O3 -I/usr/local/Cellar/xmlrpc-c/1.39.12/include
> -std=c++0x -O3 -finline-functions -Wno-inline -Wall -DHAVE_CMPH
> -DHAVE_XMLRPC_C -DKENLM_MAX_ORDER=6
> -DMOSES_VERSION_ID=\"mmt-mvp-v0.12.1-2869-g0fef8ebf4\" -DNDEBUG
> -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -I"."
> -I"/Users/hrafn/nlp/smt/boost_1_71_0/include" -c -o
> "symal/bin/clang-darwin-4.2.1/release/link-static/threading-multi/cmd.o"
> "symal/cmd.c"
>
> error: invalid argument '-std=c++0x' not allowed with 'C'
>
>
> Does this mean that C code is being compiled here, with an invalid c++
> option?
>
>
> Regards,
> Hrafn
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses build failed

[Hieu Hoang]

IRSTLM has not been compiled correctly. You don't need it in most instances
- just use KenLM.

exclude this from your bjam command
--with-irstlm=/home/gryffindor/smt1/irstlm

Hieu Hoang
http://statmt.org/hieu


On Thu, 17 Oct 2019 at 12:23, adnan islam  wrote:

> Exact Command:
> ./bjam --with-irstlm=/home/gryffindor/smt1/irstlm
> --with-boost=/home/gryffindor/smt1/boost_1_55_0
> --with-cmph=/home/gryffindor/smt1/cmph-2.0 -j8
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Problem regarding setup mosesdecoder

[Hieu Hoang]

you need the cmph library. See here for how to install it
   http://www.statmt.org/moses/?n=Development.GetStarted
alternatively, don't compile with the --with-cmph flag but then you can't
use the compressed phrase-table

Hieu Hoang
http://statmt.org/hieu


On Tue, 1 Oct 2019 at 19:52, hoonglik wong  wrote:

> Many thanks
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] mosesserver segmentation fault

[Hieu Hoang]

I would personally use Moses2 for the server, of the model is a generic
phrase based or hierarchical model. The server integration is cleaner and
less buggy. And Moses2 is faster.

However you must binarize with CreateProbingPT

More details on Moses2 here
   http://www.statmt.org/moses/?n=Site.Moses2

On Thu, 5 Sep 2019, 11:57 am Itziar Cortes Etxabe, 
wrote:

> Hi all,
>
> I've problems to load translation models using mosesserver command, and
> I'd appreciate any help.
>
> I have problems to load my models with mosesserver using Moses 3.0
> (release). There is no trouble to load them with moses command, and, in
> these case, the translation proccess runs properly. So, it seems that the
> problem resides in mosesserver.
>
> This is the output of the error (a part of it):
>
> ...
> Loading WordPenalty0
> Loading PhrasePenalty0
> Loading LexicalReordering0
> Loading Distortion0
> Loading LM0
> Loading TranslationModel0
> *** Segmentation fault
> Register dump:
>
>  RAX: ffe8   RBX: 5972c590   RCX: 3ff9
>  RDX: 0003   RSI: 00711c1c   RDI: 
>  RBP: 0003   R8 : 5972c630   R9 : 
>  R10: 7f81921479d0   R11: 0202   R12: 5972c5a0
>  R13: 000c   R14: 7ffcd252e3e0   R15: 7ffcd252f662
>  RSP: 7ffcd252e2a0
>
>  RIP: 0030d649dfe0   EFLAGS: 00010297
>
>  CS: 0033   FS:    GS: 
>
>  Trap: 000e   Error: 0005   OldMask:    CR2: ffe8
>
>  FPUCW: 037f   FPUSW:    TAG: 
>  RIP:    RDP: 
>
>  ST(0)  0029   ST(1)  0017
>  ST(2)  0080   ST(3)  0001
>  ST(4)     ST(5)  
>  ST(6)     ST(7)  
>  mxcsr: 1fa4
>  XMM0:   XMM1:
>  
>  XMM2:   XMM3:
>  
>  XMM4:   XMM5:
>  
>  XMM6:   XMM7:
>  
>  XMM8:   XMM9:
>  
>  XMM10:  XMM11:
> 
>  XMM12:  XMM13:
> 
>  XMM14:  XMM15:
> 
>
> Backtrace:
> /usr/lib64/libstdc++.so.6(_ZNSs6assignEPKcm+0x20)[0x30d649dfe0]
> /opt/usuarios/moses/mosesdecoder-RELEASE-3.0/bin/mosesserver[0x40da3d]
> /lib64/libc.so.6(__libc_start_main+0xfd)[0x30cb41ed1d]
> /opt/usuarios/moses/mosesdecoder-RELEASE-3.0/bin/mosesserver[0x40ce49]
> ...
>
>
> I attached mosesserver compilling debug information too in case it helps.
>
> Thanks in advance.
>
> Itziar.
>
>
>
>
>
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Segmentation fault and tcmalloc errors

[Hieu Hoang]

Don't use tcmalloc with Moses2, it has its own memory allocation
optimization.

Can you tell me the exact command you executed, the error it produced and
send me the config file you used

On Fri, 23 Aug 2019, 12:19 pm 周王春澍, 
wrote:

> Dear Hieu,
>
>
>
> Hi, I have successfully trained and tuned the moses model and it trains
> well with moses1.
>
> However, I would like to use optimized multi-thread moses2 to accelerate
> the decoding. I have run probingPT and finally moses2 is able to run. But
> when I feed a relatively large file to moses2 and it start RUN BATCH, it
> throws a segmentation fault after decoder several hundred lines. For
> different files it keeps happening and I am able to run moses1 with these
> files.
>
>
>
> I noticed on the Moses-support that it may because of the tcmalloc.
> However, I tried to download the complied version of moses2 from the moses
> website. This fault is not solved.
>
>
>
> I would really appreciate it if you have any idea about how to solve this
> problem. Thanks very much!
>
>
>
> Wangchunshu Zhou
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] ERROR: Wrong format of --lm. Expected: --lm factor:order:filename //train-model.perl line 605.

[Hieu Hoang]

you're gonna have to do some debugging yourself. Perhaps look inside
train_Moses.job and training.out

Hieu Hoang
http://statmt.org/hieu


On Tue, 30 Jul 2019 at 11:48, Aaron Poet  wrote:

> Hi Hieu
> Thanks for the reply. But, after I added the colon there, it still have
> some problem in running.
> *When I tried to run directly on the note of server it shows below:*
>
> (tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice
> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl
> -root-dir train -corpus
> /home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en
> -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm
> 0:3:$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir
> /home/aaronhan/Moses/mosesdecoder/tools/ >& training.out &
>
> [1] 26177
>
> (tensorflow) aaronhan@grove2:~/Moses/working$
>
> It produced a folder 'train' and sub-folder 'corpus' and the empty files 
> "en.vcb.classes
> en.vcb.classes.cats  zh.vcb.classes  zh.vcb.classes.cats"
>
> (tensorflow) aaronhan@grove2:~/Moses/working$ ls
>
> slurm-23199.out  slurm-23201.out  *train* *train_Moses.job*
>
> slurm-23200.out  slurm-23202.out  training.out
>
> (tensorflow) aaronhan@grove2:~/Moses/working$ cd train/
>
> (tensorflow) aaronhan@grove2:~/Moses/working/train$ ls
>
> *corpus*
>
> (tensorflow) aaronhan@grove2:~/Moses/working/train$ ls corpus/
>
> en.vcb.classes  en.vcb.classes.cats  zh.vcb.classes  zh.vcb.classes.cats
>
> (tensorflow) aaronhan@grove2:~/Moses/working/train$
>
>
> *When I submitted this command as job file to server, it does not appear
> in the job list/ doesn't to look like running.*
>
> Do you have any clue for this, please?
>
> Kind regards,
>
>
>
> On Tue, Jul 30, 2019 at 8:45 AM Hieu Hoang  wrote:
>
>>
>>
>> On Mon, 29 Jul 2019, 19:58 Aaron Poet,  wrote:
>>
>>> Hi Moses-Support
>>>
>>> I firstly build the lm as below:
>>>
>>> (base) aaronhan@grove2:~/Moses$ head -20
>>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en
>>>
>>> \data\
>>>
>>> ngram 1=553697
>>>
>>> ngram 2=8790963
>>>
>>> ngram 3=28838697
>>>
>>>
>>> \1-grams:
>>>
>>> -6.993945  0
>>>
>>> 0-1.6929523
>>>
>>> -2.0611665  0
>>>
>>> -2.3942525  the -0.9461817
>>>
>>> -3.6470015  show-0.6162386
>>>
>>> -4.1808434  stars   -0.5792577
>>>
>>> -4.1823444  X   -0.53191173
>>>
>>> -5.0213814  Girls   -0.33374956
>>>
>>> -2.5064495  -   -0.6326866
>>>
>>> -2.5754578  a   -0.9741243
>>>
>>> -5.1265607  troupe  -0.3527482
>>>
>>> -2.2843323  of  -0.91076595
>>>
>>> -4.663717   talented-0.3983
>>>
>>> -5.2061567  topless -0.27003714
>>>
>>> Then, I binary it as bellow:
>>>
>>> /home/aaronhan/Moses/mosesdecoder/bin/build_binary
>>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en
>>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>>>
>>> (base) aaronhan@grove2:~/Moses$ ls -lh
>>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>>>
>>> -rw-r--r-- 1 aaronhan users 715M Jul 29 18:38
>>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>>>
>>> when I test it, it looks working out:
>>>
>>> (tensorflow) aaronhan@grove2:~/Moses/working$ echo "is this an English
>>> sentence ?"   \
>>>
>>> > | /home/aaronhan/Moses/mosesdecoder/bin/query
>>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>>>
>>> is=23 2 -2.5766807  this=321 3 -1.0483611   an=108 3 -2.1032705
>>> English=1731 2 -2.8970845   sentence=3266 3 -2.8317716  ?=815 2
>>> -2.0824053  =2 3 -0.035550244   Total: -13.575124 OOV: 0
>>>
>>> Perplexity including OOVs:  86.9567697150713
>>>
>>> Perplexity excluding OOVs:  86.9567697150713
>>>
>>> OOVs:   0
>>>
>>> Tokens: 7
>>>
>>> Name:query  VmPeak:748300 kBVmRSS:4052 kB   RSSMax:503644
>>> kBuser:0.008  sys:0.652   CPU:0.66real:61.9797
>>>
>>> However, when I run the overall Moses training, it produced error as
>>> below:  *Wrong format of --lm. Expected: --lm factor:order:filename*
>>>

Re: [Moses-support] ERROR: Wrong format of --lm. Expected: --lm factor:order:filename //train-model.perl line 605.

[Hieu Hoang]

On Mon, 29 Jul 2019, 19:58 Aaron Poet,  wrote:

> Hi Moses-Support
>
> I firstly build the lm as below:
>
> (base) aaronhan@grove2:~/Moses$ head -20
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en
>
> \data\
>
> ngram 1=553697
>
> ngram 2=8790963
>
> ngram 3=28838697
>
>
> \1-grams:
>
> -6.993945  0
>
> 0-1.6929523
>
> -2.0611665  0
>
> -2.3942525  the -0.9461817
>
> -3.6470015  show-0.6162386
>
> -4.1808434  stars   -0.5792577
>
> -4.1823444  X   -0.53191173
>
> -5.0213814  Girls   -0.33374956
>
> -2.5064495  -   -0.6326866
>
> -2.5754578  a   -0.9741243
>
> -5.1265607  troupe  -0.3527482
>
> -2.2843323  of  -0.91076595
>
> -4.663717   talented-0.3983
>
> -5.2061567  topless -0.27003714
>
> Then, I binary it as bellow:
>
> /home/aaronhan/Moses/mosesdecoder/bin/build_binary
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>
> (base) aaronhan@grove2:~/Moses$ ls -lh
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>
> -rw-r--r-- 1 aaronhan users 715M Jul 29 18:38
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>
> when I test it, it looks working out:
>
> (tensorflow) aaronhan@grove2:~/Moses/working$ echo "is this an English
> sentence ?"   \
>
> > | /home/aaronhan/Moses/mosesdecoder/bin/query
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>
> is=23 2 -2.5766807  this=321 3 -1.0483611   an=108 3 -2.1032705
> English=1731 2 -2.8970845   sentence=3266 3 -2.8317716  ?=815 2
> -2.0824053  =2 3 -0.035550244   Total: -13.575124 OOV: 0
>
> Perplexity including OOVs:  86.9567697150713
>
> Perplexity excluding OOVs:  86.9567697150713
>
> OOVs:   0
>
> Tokens: 7
>
> Name:query  VmPeak:748300 kBVmRSS:4052 kB   RSSMax:503644 kB
>   user:0.008  sys:0.652   CPU:0.66real:61.9797
>
> However, when I run the overall Moses training, it produced error as
> below:  *Wrong format of --lm. Expected: --lm factor:order:filename*
>
> (tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice
> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl
> -root-dir train -corpus
> /home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en
> -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm
> 0:3$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir
> /home/aaronhan/Moses/mosesdecoder/tools/ >& training.out &
>
> [1] 23346
>
> You forgot a colon between 3 and $HOME

>
>  GNU nano 2.7.4
> File: training.out
>
>
>
> nohup: ignoring input
>
> Using SCRIPTS_ROOTDIR: /home/aaronhan/Moses/mosesdecoder/scripts
>
> Using single-thread GIZA
>
> using gzip
>
> ERROR: Wrong format of --lm. Expected: --lm factor:order:filename at
> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl line
> 605.
>
>
>
> Can you let me know how to solve this, please?
>
> Kind regards,
> Aaron
>
> --
>
> PhD student, Msc. (excellent), Bsc. (Maths)
>
> Google-Scholar
>  ,
> Presentation -slides,
> Research-Gate 
>
> Google-site  Linkedin
> ,
>
> ADAPT  ResearchCentre &
>
> School of Computing, DCU,
>
> Dublin, Ireland
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses Server

[Hieu Hoang]

There's some documentation about the moses server here:
   http://www.statmt.org/moses/?n=Advanced.Moses

Hieu Hoang
http://statmt.org/hieu


On Tue, 25 Jun 2019 at 13:13, Lucas Pimenta Porto 
wrote:

> Hi
>
> I am a Master student and I started working with Moses soon. I have an
> application developed in Python and I wish, through it, to send text so
> that the moses translates each of the sentences. How will I do this through
> XML-RPC v2?
> I have found little instruction on this subject
>
> thank you
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Compilation Failed with Moses

[Hieu Hoang]

ah great, just a small boost problem. cc'ing to the mailing list to let
other people know

Hieu Hoang
http://statmt.org/hieu


On Sat, 6 Apr 2019 at 22:23, liu shihao  wrote:

> Thanks for your reply!
> After downloading a clean version of source code and installing a older
> version of Boost (1.64.0), it finally works! However, the original version
> of Boost (1.67.0) won’t work.
> Thanks again.
> Shihao
>
> 在 2019年4月6日，21:12，Hieu Hoang  写道：
>
> Moses used to compile with macOS but it may be there's a divergence
> between macOS's compiler and gcc. I don't have a mac anymore and I'm not
> sure who on this mailing list has who can debug this.
>
> Download a clean version of the sourcecode again and try to simplify the
> compilation by giving the minimum minimum argumrents to bjam
>   ./bjam --with-boost=/opt/local/ -j4 toolset=clang
> If that doesn't work, may may need to try:
>
> if you're not interested in syntax-based SMT, you might want to exclude
> the files & programs that requires the offending files.
>1. delete
>  phrase-extract/XmlTree.cpp
>  phrase-extract/SyntaxNodeCollection.h
>  phrase-extract/SyntaxTree.h
>  phrase-extract/syntax-common/tree.h
>  phrase-extract/syntax-common/tree-inl.h
>2. In Jamroot file, comment out the line lines that says
>  phrase-extract//extract-ghkm
>  phrase-extract//pcfg-extract
>  phrase-extract//pcfg-score
>  phrase-extract//extract-mixed-syntax
>  phrase-extract//score-stsg
> there will be some additionally tidying up to do in the code
>
> Hieu Hoang
> http://statmt.org/hieu
>
>
> On Sat, 6 Apr 2019 at 19:20, liu shihao  wrote:
>
>> Hi,
>>
>> I have trouble in building the Moses on the mac(macOS high Sierra 10.13).
>> I have installed Boost successfully (at least I think so) and the log file
>> is as attached.
>> It seems that the namespace of the source code in Xcode is wrong.
>> I would appreciate your help!
>>
>> Regards,
>> Shihao
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Compilation Failed with Moses

[Hieu Hoang]

Moses used to compile with macOS but it may be there's a divergence between
macOS's compiler and gcc. I don't have a mac anymore and I'm not sure who
on this mailing list has who can debug this.

Download a clean version of the sourcecode again and try to simplify the
compilation by giving the minimum minimum argumrents to bjam
  ./bjam --with-boost=/opt/local/ -j4 toolset=clang
If that doesn't work, may may need to try:

if you're not interested in syntax-based SMT, you might want to exclude the
files & programs that requires the offending files.
   1. delete
 phrase-extract/XmlTree.cpp
 phrase-extract/SyntaxNodeCollection.h
 phrase-extract/SyntaxTree.h
 phrase-extract/syntax-common/tree.h
 phrase-extract/syntax-common/tree-inl.h
   2. In Jamroot file, comment out the line lines that says
 phrase-extract//extract-ghkm
 phrase-extract//pcfg-extract
 phrase-extract//pcfg-score
 phrase-extract//extract-mixed-syntax
 phrase-extract//score-stsg
there will be some additionally tidying up to do in the code

Hieu Hoang
http://statmt.org/hieu


On Sat, 6 Apr 2019 at 19:20, liu shihao  wrote:

> Hi,
>
> I have trouble in building the Moses on the mac(macOS high Sierra 10.13).
> I have installed Boost successfully (at least I think so) and the log file
> is as attached.
> It seems that the namespace of the source code in Xcode is wrong.
> I would appreciate your help!
>
> Regards,
> Shihao
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Asking your help

[Hieu Hoang]

I recommend using kenlm

http://www.statmt.org/moses/?n=FactoredTraining.BuildingLanguageModel#ntoc20

It's included with moses and very fast and reliable

Hieu Hoang
Sent while bumping into things

On Wed, 27 Mar 2019, 7:12 pm Zemichael Berihu,  wrote:

> What shall I do? Plz help me but I have done my best and I have followed
> the procedure to follow Irstlm in Ubuntu at least.
>
> Best Regards,
> Zemichael Berihu
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Exit code 127 when tuning the moses baseline system.

[Hieu Hoang]

Can you please supply the exact command you ran. And maybe make the data
you have available for download so the error can be reproduced

On Sat, 23 Mar 2019, 05:14 Anthony Rubin,  wrote:

> Hello,
> I am following the tutorial for the moses baseline system with the
> instructions provided here http://www.statmt.org/moses/?n=Moses.Baseline.
> I have run through this tutorial numerous times now and the program
> continues to crash while tuning.  I get:
> exit code 127:
> ERROR failed to run working/mert-work/extractor.sh at
> mosesdecoder/scripts/training/mert-moses.pl line 1775.
> I am using only the data that is recommended in the tutorial and am
> executing each command exactly as it says to do. Any help would be much
> appreciated.
> Thank you.
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses build failure support

[Hieu Hoang]

the command is

   ./bjam .

not

  ./jam-files/bjam

The instructions to compile are here

   http://www.statmt.org/moses/?n=Development.GetStarted


On 22/03/2019 11:53, ram anirudh cherukupally wrote:

Hi,

I tried installing moses4.0 and it failed while building (bjam), 
showing the following message.


The build failed.  If you need support, run:
  ./jam-files/bjam 
--with-boost=~/project/mt_toolkits/moses/boost_1_69_0 -j4 
--debug-configuration -d2 |gzip >build.log.gz then attach build.log.gz 
to your e-mail.


I have attached the same. Please help me. Thank you.

--
CH Ram Anirudh
Research Scholar,
University of Hyderabad

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Installation error

[Hieu Hoang]

mac doesn't have many of the Linux utilities needed to compile or run
Moses. They need to be installed, I use Macports to do that
   https://www.macports.org/
Please also look at the section on OSX installation in
   http://www.statmt.org/moses/?n=Development.GetStarted

Hieu Hoang
http://statmt.org/hieu


On Thu, 28 Feb 2019 at 14:07, Qiwei Shao 
wrote:

> Hello,
>
>
>
> I tried to install Moses on macOS Mojave, but when I run this line
>
> ./bjam -j4
>
> It tells me no such file or directory. I thought I might need to install
> boost first, but somehow when I ran
>
> wget https://dl.bintray.com/boostorg/release/1.64.0/source/boost_1_64_0.tar.gz
>
> it says wget: command not found. I’m very confused right now. Could you
> tell me what is the problem? Thank you.
>
>
>
> Regards,
>
> Qiwei
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Installation error

[Hieu Hoang]

Hi Yanjun

please use the bjam compile instruction on this webpage

   http://www.statmt.org/moses/?n=Development.GetStarted

The make compile seem to be falling by the wayside.

On 24/02/2019 21:25, Yanjun Gao wrote:

Hi,
I had installation error when I did  "|make -f 
contrib/Makefiles/install-dependencies.gmake|", and "./compile.sh". 
Here attaches with the .gz file.

Any help will be appreciated!

Best,
Yanjun


--
Yanjun Gao
Ph.D Candidate (ABD) in Department of Computer Science and Engineering,
EECS, Pennsylvania State University
http://www.cse.psu.edu/~yug125/


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] German tokenizer may fail with numeric endings

[Hieu Hoang]

I think you have a point. If you change tokenizer.perl to avoid applying
non-breaking prefix to the last word, please send me the change

Hieu Hoang
Sent while bumping into things

On Tue, 6 Nov 2018, 10:46 pm Ozan Çağlayan  Yes the rules are coming from the nonbreaking_prefixes files which are
> text files listing which prefixes, when preceded by a  should not be
> tokenized. But I think this rule should not be applied if the prefix is
> actually a suffix of the sentence. Similar situations arise for French and
> other languages as well. For french, "sec." is a non-breaking prefix which
> is the abbreviation for "seconds" but sec also means "dry". So if a
> sentence ends with the "dry" meaning of "sec." the  is also not
> tokenized.
>
> When the size of the corpora goes to infinity, this means that all
> nonbreaking_prefixes for a language will end up in the model vocabulary for
> NMT.
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] extract.o.sorted.gz file not found error

[Hieu Hoang]

hiya,

can you tell me what the exact commands you ran?

Hieu Hoang
http://statmt.org/hieu


On Tue, 23 Oct 2018 at 19:46, Jamal Laoudi  wrote:

> I am trying to install Moses for the first time, and I am getting an error
> at the translation model building stage as follows:
>  No such file or directory while opening
> /home/jlaoudi/working/train/model/extract.o.sorted.gz
>
> I installed and compiled Moses a few hours ago from
> http://www.statmt.org/moses/?n=Moses.Baseline, so I know I have the
> latest.
>  Your input is appreciated.
> Thanks
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] mert error

[Hieu Hoang]

in your moses.ini, replace the word SRILM with KENLM

what was the exact command you use to create moses.ini? That should be
changed so that KENLM is used instead of SRILM

Hieu Hoang
Sent while bumping into things

On Thu, 6 Sep 2018, 11:28 am Claudia Matos Veliz, <
claudia.matosve...@ugent.be> wrote:

> Hello,
> I have trained a SMT model using Moses, everything was right whit the
> training but when I tried to tunned the model I got the following error
> with SRILM
>
> Using SCRIPTS_ROOTDIR:
> /dependencies/normalisation_demo/moses_kenlm10/scripts
> Assuming --mertdir=/dependencies/normalisation_demo/moses_kenlm10/bin
> Assuming the tables are already filtered, reusing filtered/moses.ini
> Asking moses for feature names and values from filtered/moses.ini
> Executing: /opt/mosesdecoder/bin/moses  -config
> filtered/moses.ini  -inputtype 0 -show-weights > ./features.list
> Defined parameters (per moses.ini or switch):
> config: filtered/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhraseDictionaryMemory
> name=TranslationModel0 table-limit=20 num-features=5
> path=/home/claudia/NeuralMoses/twe_token_test/mert-work/filtered/phrase-table.0-0.1.1.gz
> input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0
> num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0
> output-factor=0
> path=/home/claudia/NeuralMoses/twe_token_test/mert-work/filtered/reordering-table.wbe-msd-bidirectional-fe
> Distortion SRILM name=LM0 factor=0
> path=/home/claudia/NeuralMoses/lm/model.5 order=5
> input-factors: 0
> inputtype: 0
> mapping: 0 T 0
> show-weights:
> weight: UnknownWordPenalty0= 1 WordPenalty0= -1 TranslationModel0= 0.2 0.2
> 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3 Distortion0= 0.3
> LM0= 0.5
> /opt/mosesdecoder/bin
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 1
> WEIGHT UnknownWordPenalty0=1.000,
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 2
> WEIGHT WordPenalty0=-1.000,
> line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20
> num-features=5
> path=/home/claudia/NeuralMoses/twe_token_test/mert-work/filtered/phrase-table.0-0.1.1.gz
> input-factor=0 output-factor=0
> FeatureFunction: TranslationModel0 start: 2 end: 7
> WEIGHT TranslationModel0=0.200,0.200,0.200,0.200,0.200,
> line=LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/claudia/NeuralMoses/twe_token_test/mert-work/filtered/reordering-table.wbe-msd-bidirectional-fe
> FeatureFunction: LexicalReordering0 start: 7 end: 13
> Initializing LexicalReordering..
> Loading table into memory...done.
> WEIGHT LexicalReordering0=0.300,0.300,0.300,0.300,0.300,0.300,
> line=Distortion
> FeatureFunction: Distortion0 start: 13 end: 14
> WEIGHT Distortion0=0.300,
> line=SRILM name=LM0 factor=0 path=/home/claudia/NeuralMoses/lm/model.5
> order=5
> ERROR:Unknown feature function:SRILM
> Exit code: 1
> Failed to run moses with the config filtered/moses.ini at
> /dependencies/normalisation_demo/moses_kenlm10/scripts/training/
> mert-moses.pl line 1271.
>
> The command I used was this:
>
> nice /dependencies/normalisation_demo/moses_kenlm10/scripts/training/
> mert-moses.pl dev/twe_tok.dev.ori dev/twe_tok.dev.tgt
> /opt/mosesdecoder/bin/moses model/moses.ini
>
> Any help???
> Thanks!!
> Claudia
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] terminate called after throwing an instance of 'util::FDException'

[Hieu Hoang]

can you please let me see the file
   /home/arezoo1/emlla/train/model/moses.ini

Hieu Hoang
http://statmt.org/hieu


On Sat, 1 Sep 2018 at 10:24, Arezoo Arjomand 
wrote:

> Hi all,
> I  runthefollowingcommandtotunerule-table
> butthe  erroris   appeared.
> thankyou
>
> error:
> Using SCRIPTS_ROOTDIR:
> /home/arezoo1/mosesdecoder-master/mosesdecoder-master/scripts
> Assuming
> --mertdir=/home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin
> Using cached features list: ./features.list
> MERT starting values and ranges for random generation:
>   WordPenalty0 =  -1.000 ( 0.00 ..  1.00)
>   PhrasePenalty0 =   0.200 ( 0.00 ..  1.00)
>   TranslationModel0 =   0.200 ( 0.00 ..  1.00)
>   TranslationModel0 =   0.200 ( 0.00 ..  1.00)
>   TranslationModel0 =   0.200 ( 0.00 ..  1.00)
>   TranslationModel0 =   0.200 ( 0.00 ..  1.00)
>   TranslationModel1 =   1.000 ( 0.00 ..  1.00)
> LM0 =   0.500 ( 0.00 ..  1.00)
> featlist: WordPenalty0=-1
> featlist: PhrasePenalty0=0.2
> featlist: TranslationModel0=0.2
> featlist: TranslationModel0=0.2
> featlist: TranslationModel0=0.2
> featlist: TranslationModel0=0.2
> featlist: TranslationModel1=1
> featlist: LM0=0.5
> run 1 start at Sat Sep  1 04:57:35 EDT 2018
> Parsing --decoder-flags: |-v 0|
> Saving new config to: ./run1.moses.ini
> Saved: ./run1.moses.ini
> Normalizing lambdas: -1 0.2 0.2 0.2 0.2 0.2 1 0.5
> DECODER_CFG = -weight-overwrite 'LM0= 0.142857 PhrasePenalty0= 0.057143
> WordPenalty0= -0.285714 TranslationModel0= 0.057143 0.057143 0.057143
> 0.057143 TranslationModel1= 0.285714'
> Executing:
> /home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin/moses2 -v 0
> -config /home/arezoo1/emlla/train/model/moses.ini -inputtype 3
> -weight-overwrite 'LM0= 0.142857 PhrasePenalty0= 0.057143 WordPenalty0=
> -0.285714 TranslationModel0= 0.057143 0.057143 0.057143 0.057143
> TranslationModel1= 0.285714'  -n-best-list run1.best100.out 100 distinct
> -input-file /home/arezoo1/emlla/nc-dev2007.true.fr > run1.out
> Executing:
> /home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin/moses2 -v 0
> -config /home/arezoo1/emlla/train/model/moses.ini -inputtype 3
> -weight-overwrite 'LM0= 0.142857 PhrasePenalty0= 0.057143 WordPenalty0=
> -0.285714 TranslationModel0= 0.057143 0.057143 0.057143 0.057143
> TranslationModel1= 0.285714'  -n-best-list run1.best100.out 100 distinct
> -input-file /home/arezoo1/emlla/nc-dev2007.true.fr > run1.out
> 1-10.20.2 0.2 0.2 0.210.5(1) run decoder to produce n-best lists
> params = -v 0
> decoder_config = -weight-overwrite 'LM0= 0.142857 PhrasePenalty0= 0.057143
> WordPenalty0= -0.285714 TranslationModel0= 0.057143 0.057143 0.057143
> 0.057143 TranslationModel1= 0.285714'
> Starting...
> START featureFunctions.Load()
> Loading WordPenalty0
> Finished loading WordPenalty0
> Loading PhrasePenalty0
> Finished loading PhrasePenalty0
> Loading LM0
> terminate called after throwing an instance of 'util::FDException'
>   what():  util/file.cc:175 in std::size_t util::PartialRead(int, void*,
> std::size_t) threw FDException because `ret < 0'.
> Is a directory in /home/arezoo1/emlla while reading 6 bytes in file
> /home/arezoo1/emlla
> Aborted (core dumped)
> Exit code: 134
> The decoder died. CONFIG WAS -weight-overwrite 'LM0= 0.142857
> PhrasePenalty0= 0.057143 WordPenalty0= -0.285714 TranslationModel0=
> 0.057143 0.057143 0.057143 0.057143 TranslationModel1= 0.285714'
>
>
>
> train:
>
> /home/arezoo1/mosesdecoder-master/mosesdecoder-master/scripts/training/train-model.perl
> \
>
> -root-dir train  \-corpus /home/arezoo1/emlla/news-commentary-v8.fr-en.clean  
> \-f fr -e en -alignment grow-diag-final-and  -hierarchical-glue-grammar   
> \  -lm 0:3:/home/arezoo1/emlla/news-commentary-v8.fr-en.blm.en  \-mgiza 
> -mgiza-cpus 4  \*-external-bin-dir 
> */home/arezoo1/mosesdecoder-master/mosesdecoder-master*/tools >*
>
>
>
>
> tune:
> /home/arezoo1/mosesdecoder-master/mosesdecoder-master/scripts/training/
> mert-moses.pl  /home/arezoo1/emlla/nc-dev2007.true.fr
> /home/arezoo1/emlla/nc-dev2007.true.en
> /home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin/moses2
> /home/arezoo1/emlla/train/model/moses.ini  --rootdir
> /home/arezoo1/mosesdecoder-master/mosesdecoder-master/scripts
> --decoder-flags "-v 0"  --no-filter-phrase-table --inputtype 3
> --batch-mira  --return-best-dev --filtercmd
> '/home/arezoo1/mosesdecoder-master/mosesdecoder-master/scripts/training/
> filter-model-given-input.pl'  --batch-mira-args '-J 300' 1>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Some one E-MAIL TO U Re: Fwd: Different translations are obtained from the same decoder without alignment information

[Hieu Hoang]

If you want an email to the mailing list, the archives are here
   https://www.mail-archive.com/moses-support@mit.edu/

Hieu Hoang
http://statmt.org/hieu


On Fri, 24 Aug 2018 at 17:04, Ergun Bicici  wrote:

>
> Dear Hieu,
>
> I did not receive any further message from metaseby...@gmail.com. Can you
> forward to me if you received any other message until now (7:03 pm) from
> him?
>
> In the mean time I am waiting for the new results according to your
> suggestion.
>
> Thanks,
> Ergun
>
> -- Forwarded message -
> From: Ergun Bicici 
> Date: Fri, Aug 24, 2018 at 6:47 PM
> Subject: Re: Some one E-MAIL TO U Re: [Moses-support] Fwd: Different
> translations are obtained from the same decoder without alignment
> information
> To: 
>
>
>
> I received only this:
>
> Bereketab Birhnu 
> 5:53 PM (53 minutes ago)
> to me
>
> Thanks
>
> I am not expecting any more nor I wanted any.
>
> Do you have any further message?
>
> Ergun
>
> On Fri, Aug 24, 2018 at 6:45 PM Ergun Bicici  wrote:
>
>> Why?
>>
>> On Fri, Aug 24, 2018 at 6:42 PM Bereketab Birhnu 
>> wrote:
>>
>>> Check your emails
>>>
>>
>>
>> --
>>
>> Regards,
>> Ergun
>>
>>
>>
>
> --
>
> Regards,
> Ergun
>
>
>
>
> --
>
> Regards,
> Ergun
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Fwd: Different translations are obtained from the same decoder without alignment information

[Hieu Hoang]

could you run with alignments, but WITHOUT -unknown-word-prefix UNK.

alignments shouldn't change the translation but the OOV prefix may do

Hieu Hoang
http://statmt.org/hieu


On Fri, 24 Aug 2018 at 15:29, Ergun Bicici  wrote:

>
> ok, thank you. I'll upload and send you a link.
>
> On Fri, Aug 24, 2018 at 5:27 PM Hieu Hoang  wrote:
>
>> that would be a bug.
>>
>> could you please make the model and input files available for download.
>> I'll check it out
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>>
>> On Fri, 24 Aug 2018 at 15:15, Ergun Bicici  wrote:
>>
>>>
>>> only the evaluation decoding steps are repeated that are steps 10, 9,
>>> and 7 in the following steps in EMS output:
>>> 48 TRAINING:consolidate ->  re-using (1)
>>> 47 TRAINING:prepare-data -> re-using (1)
>>> 46 TRAINING:run-giza -> re-using (1)
>>> 45 TRAINING:run-giza-inverse -> re-using (1)
>>> 44 TRAINING:symmetrize-giza ->  re-using (1)
>>> 43 TRAINING:build-lex-trans ->  re-using (1)
>>> 40 TRAINING:build-osm ->re-using (1)
>>> 39 TRAINING:extract-phrases ->  re-using (1)
>>> 38 TRAINING:build-reordering -> re-using (1)
>>> 37 TRAINING:build-ttable -> re-using (1)
>>> 34 TRAINING:create-config ->re-using (1)
>>> 28 TUNING:truecase-input -> re-using (1)
>>> 24 TUNING:truecase-reference -> re-using (1)
>>> 21 TUNING:filter -> re-using (1)
>>> 20 TUNING:apply-filter ->   re-using (1)
>>> 19 TUNING:tune ->   re-using (1)
>>> 18 TUNING:apply-weights ->  re-using (1)
>>> 15 EVALUATION:test:truecase-input ->re-using (1)
>>> 12 EVALUATION:test:filter ->re-using (1)
>>> 11 EVALUATION:test:apply-filter ->  re-using (1)
>>>
>>>
>>>
>>> *10 EVALUATION:test:decode ->run 9 EVALUATION:test:remove-markup ->
>>>  run 7 EVALUATION:test:detruecase-output ->  run *3
>>> EVALUATION:test:multi-bleu-c ->   run
>>> 2 EVALUATION:test:analysis-coverage ->  re-using (1)
>>> 1 EVALUATION:test:analysis-precision -> run
>>>
>>>
>>> On Fri, Aug 24, 2018 at 4:39 PM Hieu Hoang  wrote:
>>>
>>>> are you rerunning tuning for each case? Or are you using exactly the
>>>> same moses.ini file for the with and with alignment experiments?
>>>>
>>>> Hieu Hoang
>>>> http://statmt.org/hieu
>>>>
>>>>
>>>> On Fri, 24 Aug 2018 at 14:34, Ergun Bicici  wrote:
>>>>
>>>>>
>>>>> Dear Moses maintainers,
>>>>>
>>>>> I discovered that the translations obtained differ when alignment
>>>>> flags (--mark-unknown --unknown-word-prefix UNK --print-alignment-inf)
>>>>> are used. Comparison table is attached (en-ru and ru-en are being
>>>>> recomputed). We expect them to be the same since alignment flags only 
>>>>> print
>>>>> additional information and they are not supposed to alter decoding. In
>>>>> both, the same EMS system was re-run with the alignment information flags
>>>>> or not.
>>>>>
>>>>>- Average of the absolute difference is 0.0094 BLEU (about 1 BLEU
>>>>>points).
>>>>>- Average of the difference is 0.0051 BLEU (about 0.5 BLEU points,
>>>>>results are better with alignment flags).
>>>>>
>>>>> 
>>>>>
>>>>> /opt/Programs/SMT/moses/mosesdecoder/bin/moses --version
>>>>>
>>>>> Moses code version (git tag or commit hash):
>>>>>   mmt-mvp-v0.12.1-2775-g65c75ff07-dirty
>>>>> Libraries used:
>>>>>  Boost  version 1.62.0
>>>>>
>>>>> git status
>>>>> On branch RELEASE-4.0
>>>>> Your branch is up to date with 'origin/RELEASE-4.0'.
>>>>>
>>>>>
>>>>> Note: Using alignment information to recase tokens was tried in [1]
>>>>> for en-fi and en-tr to claim positive results. We tried this method in all
>>>>> translation directions we considered as as can be seen in the align row,
>>>>> this only improves the performance for tr-en and en-tr and for tr-en Moses
>>>>> provides better translations without the alignment flags.
>>>>> [1]The JHU Machine Translation Systems for WMT 2016
>>>>> Shuoyang Ding, Kevin Duh, Huda Khayrallah, Philipp Koehn and Matt Post
>>>>> http://www.statmt.org/wmt16/pdf/W16-2310.pdf
>>>>>
>>>>>
>>>>> Best Regards,
>>>>> Ergun
>>>>>
>>>>> Ergun Biçici
>>>>> http://bicici.github.com/ <http://ergunbicici.blogspot.com/>
>>>>>
>>>>> ___
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>
>>>>
>>>
>>> --
>>>
>>> Regards,
>>> Ergun
>>>
>>>
>>>
>
> --
>
> Regards,
> Ergun
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Fwd: Different translations are obtained from the same decoder without alignment information

[Hieu Hoang]

that would be a bug.

could you please make the model and input files available for download.
I'll check it out

Hieu Hoang
http://statmt.org/hieu


On Fri, 24 Aug 2018 at 15:15, Ergun Bicici  wrote:

>
> only the evaluation decoding steps are repeated that are steps 10, 9, and
> 7 in the following steps in EMS output:
> 48 TRAINING:consolidate ->  re-using (1)
> 47 TRAINING:prepare-data -> re-using (1)
> 46 TRAINING:run-giza -> re-using (1)
> 45 TRAINING:run-giza-inverse -> re-using (1)
> 44 TRAINING:symmetrize-giza ->  re-using (1)
> 43 TRAINING:build-lex-trans ->  re-using (1)
> 40 TRAINING:build-osm ->re-using (1)
> 39 TRAINING:extract-phrases ->  re-using (1)
> 38 TRAINING:build-reordering -> re-using (1)
> 37 TRAINING:build-ttable -> re-using (1)
> 34 TRAINING:create-config ->re-using (1)
> 28 TUNING:truecase-input -> re-using (1)
> 24 TUNING:truecase-reference -> re-using (1)
> 21 TUNING:filter -> re-using (1)
> 20 TUNING:apply-filter ->   re-using (1)
> 19 TUNING:tune ->   re-using (1)
> 18 TUNING:apply-weights ->  re-using (1)
> 15 EVALUATION:test:truecase-input ->re-using (1)
> 12 EVALUATION:test:filter ->re-using (1)
> 11 EVALUATION:test:apply-filter ->  re-using (1)
>
>
>
> *10 EVALUATION:test:decode ->run 9 EVALUATION:test:remove-markup ->
>  run 7 EVALUATION:test:detruecase-output ->  run *3
> EVALUATION:test:multi-bleu-c ->   run
> 2 EVALUATION:test:analysis-coverage ->  re-using (1)
> 1 EVALUATION:test:analysis-precision -> run
>
>
> On Fri, Aug 24, 2018 at 4:39 PM Hieu Hoang  wrote:
>
>> are you rerunning tuning for each case? Or are you using exactly the same
>> moses.ini file for the with and with alignment experiments?
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>>
>> On Fri, 24 Aug 2018 at 14:34, Ergun Bicici  wrote:
>>
>>>
>>> Dear Moses maintainers,
>>>
>>> I discovered that the translations obtained differ when alignment flags 
>>> (--mark-unknown
>>> --unknown-word-prefix UNK --print-alignment-inf) are used. Comparison
>>> table is attached (en-ru and ru-en are being recomputed). We expect them to
>>> be the same since alignment flags only print additional information and
>>> they are not supposed to alter decoding. In both, the same EMS system was
>>> re-run with the alignment information flags or not.
>>>
>>>- Average of the absolute difference is 0.0094 BLEU (about 1 BLEU
>>>points).
>>>- Average of the difference is 0.0051 BLEU (about 0.5 BLEU points,
>>>results are better with alignment flags).
>>>
>>> 
>>>
>>> /opt/Programs/SMT/moses/mosesdecoder/bin/moses --version
>>>
>>> Moses code version (git tag or commit hash):
>>>   mmt-mvp-v0.12.1-2775-g65c75ff07-dirty
>>> Libraries used:
>>>  Boost  version 1.62.0
>>>
>>> git status
>>> On branch RELEASE-4.0
>>> Your branch is up to date with 'origin/RELEASE-4.0'.
>>>
>>>
>>> Note: Using alignment information to recase tokens was tried in [1] for
>>> en-fi and en-tr to claim positive results. We tried this method in all
>>> translation directions we considered as as can be seen in the align row,
>>> this only improves the performance for tr-en and en-tr and for tr-en Moses
>>> provides better translations without the alignment flags.
>>> [1]The JHU Machine Translation Systems for WMT 2016
>>> Shuoyang Ding, Kevin Duh, Huda Khayrallah, Philipp Koehn and Matt Post
>>> http://www.statmt.org/wmt16/pdf/W16-2310.pdf
>>>
>>>
>>> Best Regards,
>>> Ergun
>>>
>>> Ergun Biçici
>>> http://bicici.github.com/ <http://ergunbicici.blogspot.com/>
>>>
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>
> --
>
> Regards,
> Ergun
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Fwd: Different translations are obtained from the same decoder without alignment information

[Hieu Hoang]

are you rerunning tuning for each case? Or are you using exactly the same
moses.ini file for the with and with alignment experiments?

Hieu Hoang
http://statmt.org/hieu


On Fri, 24 Aug 2018 at 14:34, Ergun Bicici  wrote:

>
> Dear Moses maintainers,
>
> I discovered that the translations obtained differ when alignment flags 
> (--mark-unknown
> --unknown-word-prefix UNK --print-alignment-inf) are used. Comparison
> table is attached (en-ru and ru-en are being recomputed). We expect them to
> be the same since alignment flags only print additional information and
> they are not supposed to alter decoding. In both, the same EMS system was
> re-run with the alignment information flags or not.
>
>- Average of the absolute difference is 0.0094 BLEU (about 1 BLEU
>points).
>- Average of the difference is 0.0051 BLEU (about 0.5 BLEU points,
>results are better with alignment flags).
>
> 
>
> /opt/Programs/SMT/moses/mosesdecoder/bin/moses --version
>
> Moses code version (git tag or commit hash):
>   mmt-mvp-v0.12.1-2775-g65c75ff07-dirty
> Libraries used:
>  Boost  version 1.62.0
>
> git status
> On branch RELEASE-4.0
> Your branch is up to date with 'origin/RELEASE-4.0'.
>
>
> Note: Using alignment information to recase tokens was tried in [1] for
> en-fi and en-tr to claim positive results. We tried this method in all
> translation directions we considered as as can be seen in the align row,
> this only improves the performance for tr-en and en-tr and for tr-en Moses
> provides better translations without the alignment flags.
> [1]The JHU Machine Translation Systems for WMT 2016
> Shuoyang Ding, Kevin Duh, Huda Khayrallah, Philipp Koehn and Matt Post
> http://www.statmt.org/wmt16/pdf/W16-2310.pdf
>
>
> Best Regards,
> Ergun
>
> Ergun Biçici
> http://bicici.github.com/ <http://ergunbicici.blogspot.com/>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Decoder died in tunning step!

[Hieu Hoang]

you need to binarize with CreateProbingPT if you use moses2

if you use CreateOnDiskPt then you must use the moses decoder


Hieu Hoang
http://statmt.org/hieu


On Thu, 23 Aug 2018 at 09:02, Arezoo Arjomand 
wrote:

> Hi,
>
> I installed successfully moses decoder on server. I run syntax grammer
> extraction as follow :
>
>
>
> /home/vps/mosesdecoder/scripts/training/train-model.perl  \
>
> -root-dir train  \
>
> -corpus /home/vps/emlla//ttt/news-commentary-v8.fr-en.clean  \
>
> -f fr -e en -alignment grow-diag-final-and  -hierarchical-glue-grammar
>   \  -lm 0:3:/home/vps/emlla//ttt/news-commentary-v8.fr-en.blm.en  \
>
> -mgiza -mgiza-cpus 4  \
>
> *-external-bin-dir * /*home*vps/mosesdecoder/tools >
>
>
>
> the file which created is :
>
>
>
> train → corpus , giza.en-fr, giza.fr-en, model
>
> model → aligned.grow-diag-final-and, extract.sorted.gz, lex.e2f,  moses.ini,
> extract.inv.sorted.gz,glue-grammar, lex.f2e  rule-table.gz
>
>
>
> to tune the model the following command is run:
>
>
>
> /home/vps/mosesdecoder/scripts/training/mert-moses.pl  /home/vps/emlla/
> nc-dev2007.en.true.fr  /home/vps/emlla/nc-dev2007.en.true.en  
> /home/vps/mosesdecoder/bin/moses2
> /home/vps/emlla/train/model/moses.ini --rootdir  
> /home/vps/mosesdecoder/scripts
> --decoder-flags "-v 0" --no-filter-phrase-table --inputtype 3 --batch-mira
> --return-best-dev  --filtercmd '/home/vps/mosesdecoder/scripts/training/
> filter-model-given-input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2" '
> --batch-mira-args '-J 300' --decoder-flags '-threads 8 -v 0' 1>
>
>
> but it dies in following step:
>
>
> Finished loading LM0
>
> Loading UnknownWordPenalty0
>
> Finished loading UnknownWordPenalty0
>
> Loading TranslationModel0
>
> Finished loading TranslationModel0
>
> Loading TranslationModel1
>
> Finished loading TranslationModel1
>
> START LoadMappings()
>
> END LoadMappings()
>
> END LoadDecodeGraphBackoff()
>
> Loaded : [0.0490292] seconds
>
> RUN BATCH
>
> terminate called after throwing an instance of 'util::Exception'
>
>   what():  moses2/SCFG/nbest/KBestExtractor.cpp:42 in void
> Moses2::SCFG::KBestE$
>
> Only suppose to be 1 hypo coll in last stack
>
> Aborted (core dumped)
>
> Exit code: 134
> The decoder died. CONFIG WAS -weight-overwrite 'TranslationModel1=
> 0.285714 Phr$
>
> how can I fix it?
> Thank  you for your attention
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Issues running MGiza on AWS machine

[Hieu Hoang]

redirect stdout and stderr into a file and grep for 'error'

that usually turns up something

Hieu Hoang
http://statmt.org/hieu

On 1 August 2018 at 17:38, James Baker  wrote:

> Thanks Hieu,
>
> I'll give that a go this morning and keep an eye on the disk space and
> RAM, although I would be surprised if that was the problem (I've got <3GB
> of training data, 64GB of RAM, and 100GB of disk space). It also wouldn't
> explain why binaries built on a different machine work, but binaries built
> on the same machine don't.
>
> Any other ideas for things I should be checking?
>
> Cheers,
> James
>
> On Wed, 1 Aug 2018 at 03:03, Hieu Hoang  wrote:
>
>> it's difficult to tell but I would say the mgiza executables isn't the
>> problem. It's probably to do with running out of disk space or memory.
>>
>> the snt2coooc executable in mgiza uses a lot of memory so may have been
>> killed by the OS. The phrase table creation requires a lot of disk space to
>> sort intermediate files.
>>
>> I would monitor those 2 things
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>> On 31 July 2018 at 20:41, James Baker  wrote:
>>
>>> Hi,
>>>
>>> I'm having some peculiar issues with MGiza++. Using MGiza and Moses,
>>> I've successfully built some translation models on my Ubuntu 16.04 desktop
>>> machine. I'd now like to do the same thing, but on a machine hosted in AWS.
>>>
>>> I'm using the same operating system, and as far as I can tell all my
>>> versions are identical. The build of MGiza++ runs fine, reports no errors,
>>> and produces output the same as on my desktop machine. However, when I try
>>> to build the models, I get a whole load of errors and the resultant models
>>> are empty (64 bytes for the reordering model, 0 bytes for the translation
>>> model - the language model builds fine).
>>>
>>> The first "errors" I can see in the log seem to occur on stage 4 of the
>>> Moses training script (train-model.perl):
>>>
>>>(4) generate lexical translation table 0-0 @ Tue Jul 31 10:22:58 UTC
>>> 2018
>>>(/opt/model-builder/training/data.ru,/opt/model-builder/
>>> training/data.en,/opt/model-builder/training/model/lex)
>>>!Argument "anna" isn't numeric in numeric ge (>=) at
>>> /opt/model-builder/mosesdecoder/scripts/training/LexicalTranslationModel.pm
>>> line 112,  line 1.
>>>Use of uninitialized value $ei in numeric ge (>=) at
>>> /opt/model-builder/mosesdecoder/scripts/training/LexicalTranslationModel.pm
>>> line 112,  line 1.
>>>Use of uninitialized value $ei in hash element at /opt/model-builder/
>>> mosesdecoder/scripts/training/LexicalTranslationModel.pm line 118, 
>>> line 1.
>>>Use of uninitialized value $ei in array element at /opt/model-builder/
>>> mosesdecoder/scripts/training/LexicalTranslationModel.pm line 121, 
>>> line 1.
>>>Use of uninitialized value $ei in array element at /opt/model-builder/
>>> mosesdecoder/scripts/training/LexicalTranslationModel.pm line 123, 
>>> line 1.
>>>...
>>>
>>> There are a large number of errors of that nature, and following those
>>> errors there are additional errors but I suspect these are caused by the
>>> fact that this stage is failing.
>>>
>>> It's possible that there are earlier problems, but I'm not really sure
>>> what to be looking for in the logs (for instance - there are some lines
>>> warning about alignments in Model2 being 0 - is that an issue?).
>>>
>>> If I replace the MGiza binaries built on the AWS machine with the
>>> binaries built on my desktop, it runs fine - so I know it's an issue with
>>> MGiza and presumably something to do with my build. The commands I'm
>>> running to build and install are as follows
>>>
>>>git clone https://github.com/moses-smt/mgiza.git
>>>cd mgiza/mgizapp
>>>cmake .
>>>make
>>>make install
>>>cp bin/* ../../mosesdecoder/bin
>>>cp scripts/merge_alignment.py ../../mosesdecoder/bin
>>>
>>> As I mentioned previously, these commands work fine on my desktop
>>> machine which should be a very similar (if not identical) set up.
>>>
>>> Does anyone have any ideas as to what might be causing the problem (or,
>>> more importantly, what I can do to fix it)?
>>>
>>> Thanks in advance,
>>> James
>>>
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Issues running MGiza on AWS machine

[Hieu Hoang]

it's difficult to tell but I would say the mgiza executables isn't the
problem. It's probably to do with running out of disk space or memory.

the snt2coooc executable in mgiza uses a lot of memory so may have been
killed by the OS. The phrase table creation requires a lot of disk space to
sort intermediate files.

I would monitor those 2 things

Hieu Hoang
http://statmt.org/hieu

On 31 July 2018 at 20:41, James Baker  wrote:

> Hi,
>
> I'm having some peculiar issues with MGiza++. Using MGiza and Moses, I've
> successfully built some translation models on my Ubuntu 16.04 desktop
> machine. I'd now like to do the same thing, but on a machine hosted in AWS.
>
> I'm using the same operating system, and as far as I can tell all my
> versions are identical. The build of MGiza++ runs fine, reports no errors,
> and produces output the same as on my desktop machine. However, when I try
> to build the models, I get a whole load of errors and the resultant models
> are empty (64 bytes for the reordering model, 0 bytes for the translation
> model - the language model builds fine).
>
> The first "errors" I can see in the log seem to occur on stage 4 of the
> Moses training script (train-model.perl):
>
>(4) generate lexical translation table 0-0 @ Tue Jul 31 10:22:58 UTC
> 2018
>(/opt/model-builder/training/data.ru,/opt/model-builder/
> training/data.en,/opt/model-builder/training/model/lex)
>!Argument "anna" isn't numeric in numeric ge (>=) at /opt/model-builder/
> mosesdecoder/scripts/training/LexicalTranslationModel.pm line 112, 
> line 1.
>Use of uninitialized value $ei in numeric ge (>=) at /opt/model-builder/
> mosesdecoder/scripts/training/LexicalTranslationModel.pm line 112, 
> line 1.
>Use of uninitialized value $ei in hash element at /opt/model-builder/
> mosesdecoder/scripts/training/LexicalTranslationModel.pm line 118, 
> line 1.
>Use of uninitialized value $ei in array element at /opt/model-builder/
> mosesdecoder/scripts/training/LexicalTranslationModel.pm line 121, 
> line 1.
>Use of uninitialized value $ei in array element at /opt/model-builder/
> mosesdecoder/scripts/training/LexicalTranslationModel.pm line 123, 
> line 1.
>...
>
> There are a large number of errors of that nature, and following those
> errors there are additional errors but I suspect these are caused by the
> fact that this stage is failing.
>
> It's possible that there are earlier problems, but I'm not really sure
> what to be looking for in the logs (for instance - there are some lines
> warning about alignments in Model2 being 0 - is that an issue?).
>
> If I replace the MGiza binaries built on the AWS machine with the binaries
> built on my desktop, it runs fine - so I know it's an issue with MGiza and
> presumably something to do with my build. The commands I'm running to build
> and install are as follows
>
>git clone https://github.com/moses-smt/mgiza.git
>cd mgiza/mgizapp
>cmake .
>make
>make install
>cp bin/* ../../mosesdecoder/bin
>cp scripts/merge_alignment.py ../../mosesdecoder/bin
>
> As I mentioned previously, these commands work fine on my desktop machine
> which should be a very similar (if not identical) set up.
>
> Does anyone have any ideas as to what might be causing the problem (or,
> more importantly, what I can do to fix it)?
>
> Thanks in advance,
> James
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] incomplete phrase table

[Hieu Hoang]

I guess you wanted it to create the following rules
   c -> x
   d -> y
   e -> z
There's no guarantee that it will figure that out. A cause could be there
isn't enough training data.



Hieu Hoang
http://statmt.org/hieu

On 27 July 2018 at 02:06, Janek Amann  wrote:

> Hi all,
>
> I'm pretty new to Moses and I don't think I'm able to figure this out on
> my own. I'm trying to train Moses with this very small data set.
>
> Src:
>
> A C
> B C
> A D
> B E
>
> Tgt:
>
> X
> X
> Y
> Z
>
> And this is my test set:
>
> Src:
>
> A C
> B C
> A D
> B D
> A E
> B E
>
> Tgt:
>
> X
> X
> Y
> Y
> Z
> Z
>
>
> This is the phrase table I'm getting:
>
> A C ||| X ||| 0.5 0.25 1 1 ||| 0-0 1-0 ||| 2 1 1 ||| |||
> A D ||| Y ||| 1 1 1 1 ||| 0-0 1-0 ||| 1 1 1 ||| |||
> B C ||| X ||| 0.5 0.25 1 0.75 ||| 0-0 1-0 ||| 2 1 1 ||| |||
> B E ||| Z ||| 1 1 1 1 ||| 0-0 1-0 ||| 1 1 1 ||| |||
>
> For some reason Moses didn't extract any single tokens which of course
> messes up the translation model.
> These are the commands I used:
>
> for the language model:
>
> /home/janek/mosesdecoder/bin/lmplz \
> -o 3 
> /home/janek/Desktop/Moses/lm/moses_train_4.arpa.tgt \
> --discount_fallback
>
> and the translation model:
>
> /home/janek/mosesdecoder/scripts/training/train-model.perl \
>  -root-dir /home/janek/Desktop/Moses/working \
>  -corpus /home/janek/Desktop/Moses/data/moses_train_4 \
>  -f src \
>  -e tgt \
>  -alignment grow-diag-final-and \
>  -reordering msd-bidirectional-fe \
>  -lm 0:1:/home/janek/Desktop/Moses/lm/moses_train_4.arpa.tgt:8 \
>  -external-bin-dir /home/janek/mosesdecoder/mgiza/mgizapp \
>  -mgiza
>
> Since my dataset is very small I skipped tokenizing and truecasing. I
> didn't do any tuning also.
> I've already tried out all possible options for the alignment but it
> didn't change a thing.
> I'd be really grateful if someone could point me to a solution or at least
> the right direction for solving this.
> This is my first time posting something in a support forum so I don't know
> if you need any more information.
> Just let me know if you do.
>
> Thanks for your help.
>
> Best,
> Janek
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Broken links for Installer for Windows with GUI

[Hieu Hoang]

thanks. Deleting both

Hieu Hoang
http://statmt.org/hieu

On 12 July 2018 at 02:31, Mervyn Yeo  wrote:

> Hi Hieu,
>
> It seems that
>
> http://amittaibox.weebly.com/
>
> is a dead link. Maybe you want to delete that reference as well.
>
> Thanks,
> Mervyn
>
> On Wed, Jul 11, 2018 at 6:10 PM, Hieu Hoang  wrote:
>
>> You're better off compiling from source, using Slate toolkit
>>   https://www.slate.rocks/downloads/slate-toolkit-edition/
>> or Box
>>   http://amittaibox.weebly.com/
>> The Capita package is pretty old now and unmaintained. Will delete
>> reference to it from the Moses website if no-one objects
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>> On 11 July 2018 at 03:01, Mervyn Yeo  wrote:
>>
>>> Hi All,
>>>
>>> I'm trying to download the installer provided by Capita TI but both the
>>> links are broken. Does anyone know where I can get a copy of the builds?
>>>
>>> Thanks,
>>> Mervyn
>>>
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Build failed for moses v4 on macOS 10.13.5

[Hieu Hoang]

try

   ./bjam --with-boost=/Users/johnmiller/moses/opt 
--with-cmph=/Users/johnmiller/moses/opt 
--with-xmlrpc-c=/Users/johnmiller/moses/opt --with-mm --with-probing-pt 
-j8 --debug-configuration -d2 |gzip >build.log.gz


Avoid using compile.sh. It doesn't seem to be supported anymore. I will 
change the webpage accordingly


On 23/06/18 05:26, John Miller wrote:

Hello,

I tried to build Moses v4 from source on macOS 10.13.5.

1. Installed wget via macPorts.

2. Installed dependencies.
bash-3.2$ make -f contrib/Makefiles/install-dependencies.gmake
Lots of warnings.
...updated 14020 targets...
rm -rf /Users/johnmiller/moses/build/boost

3.Tried to build Moses
bash-3.2$./compile.sh
Lots of warnings… deprecated stuff, unused stuff,
at least 1 loop variable incremented both within loop and at loop 
increment.

...failed updating 24 targets...
...skipped 42 targets...
...updated 1205 targets...
The build failed.  If you need support, run:
  ./jam-files/bjam --with-boost=/Users/johnmiller/moses/opt 
--with-cmph=/Users/johnmiller/moses/opt 
--with-xmlrpc-c=/Users/johnmiller/moses/opt --with-mm 
--with-probing-pt -j8 --debug-configuration -d2 |gzip >build.log.gz

then attach build.log.gz to your e-mail.
You MUST do 3 things before sending to the mailing list:
   1. Subscribe to the mailing list at 
http://mailman.mit.edu/mailman/listinfo/moses-support

   2. Attach build.log.gz to your e-mail
   3. Say what is the EXACT command you executed when you got the error
ERROR

4. Ran with indicated command.
bash-3.2$ ./jam-files/bjam --with-boost=/Users/johnmiller/moses/opt 
--with-cmph=/Users/johnmiller/moses/opt 
--with-xmlrpc-c=/Users/johnmiller/moses/opt --with-mm 
--with-probing-pt -j8 --debug-configuration -d2 |gzip >build.log.gz


5. Here is the log.file as text:
Unable to load Boost.Build: could not find "boost-build.jam"
---
Attempted search from /Users/johnmiller/moses up to the root
at /Users/johnmiller/moses/share/boost-build
and in these directories from BOOST_BUILD_PATH and BOOST_ROOT: 
/usr/share/boost-build.

Please consult the documentation at 'http://www.boost.org'.


I’ve attached as well:




6. There is no:
/Users/johnmiller/moses/share/boost-build

but there is:
/Users/johnmiller/moses/jam-files/boost-build
containing:
boost-build.jam


***
From the message it may be looking in the wrong place for the 
boost-build.jam file.


Suggestions?

Install instructions had the install dependencies step for Ubuntu, but 
there was no other place
for installing the dependencies cmph, xmlrpc-c, and boost came along 
for the ride.






John Miller
mille...@me.com <mailto:mille...@me.com>






___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] what is the EXACT command you executed when you got the error

[Hieu Hoang]

looks like the cmph library hasn't been compiled successfully or it's 
not in the directory you specified


   ~/workspace/cmph-2.0


On 23/06/18 16:26, Gebrial Molla wrote:




___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://statmt.org/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses 4 binaries - xmlrpc not loaded - for macOS

[Hieu Hoang]

ah, it was linked to the xmlrpc-c shared library, rather than static
library.

I dont have a mac anymore to be able to create the binaries, apologies.

it is pretty easy to compile on a mac though, work through this web page
   http://www.statmt.org/moses/?n=Development.GetStarted

Hieu Hoang
http://statmt.org/hieu

On 22 June 2018 at 00:39, John Miller  wrote:

> Was delighted to see the possibility to simply download and install
> binaries for Mac OSX, but I get a fail on the ‘Run Moses for the first
> time’ test:
>
> $ ~/moses.bin.macOS/moses/bin/moses -f phrase-model/moses.ini <
> phrase-model/in | tee out
>
> dyld: Library not loaded: /Users/hieu/workspace/xmlrpc-
> c/xmlrpc-c-1.39.07/lib/libxmlrpc_xmltok.3.39.dylib
>   Referenced from: /Users/johnmiller/moses.bin.macOS/moses/bin/moses
>   Reason: image not found
>
> Executable seems to have been linked and tested against the development
> platform - with the result different on my machine for not having the dyld
> libraries of the development platform.
>
> Checking support archives I see items from 2017-11-07 and 1018-01-12
> referencing a similar error.  A response to the 2017-11-07 indicates that
> new binaries are in the offing to solve this problem, but I only see the
> 2017-09-26 binaries that I am using on the download page.
>
>
> Any help would be appreciated. I appreciate your help!
>
> ***
> Note: I have tried to go the download source and build route as well, but
> have run into a problem there as well with respect to xmlrpc.  If there is
> no solution available in the near future for the binaries, then I’ll return
> to the build route. With likely need to ask for help from the community.
>
> Thanks again,
>
>
> John Miller
> mille...@me.com
>
>
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Regarding LM

[Hieu Hoang]

what was the error? Why not try KenLM?

Hieu Hoang
http://statmt.org/hieu

On 21 June 2018 at 11:10, Kamal Deep Garg  wrote:

> Dear sir
>
> I had created arpa file in LM with data of 11Lakh monolingual sentences
> using iRSTLM.
>
>
> when I am converting arpa file to binary file using *build_binary I am
> getting an error.*
>
>
> *Should i change way to develop arpa file?*
> *Please help.*
>
>
> Thank You
>
> Kamal Deep Garg
> 9988493359
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Moses Compilation Error

[Hieu Hoang]

it looks like there is an incompatible version of xmlrpc-c library that
moses is trying to link to.

Add this to your bjam command
  --no-xmlrpc-c


Hieu Hoang

On 14 June 2018 at 09:08, Cheshta Kwatra  wrote:

> EXACT command executed - cd ~/mosesdecoder
>./bjam
> --with-boost=~/boost_1_64_0 -j4
>
> I am working on Ubuntu 14.04 LTS
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Free cloud service to train NMT

[Hieu Hoang]

Laptop gpu are small and expensive. I would personally get a cheap desktop
and a gtx 1080

Hieu Hoang
Sent while bumping into things

On Fri, 8 Jun 2018, 5:52 pm amir haghighi, 
wrote:

> Thanks Hieu.
>
> Is it possible to train the NMT system on the laptop? I mean is there any
> laptop that I can buy for this purpose?
>
> Thanks
>
> On Fri, Jun 8, 2018 at 8:36 PM, Hieu Hoang  wrote:
>
>> try this
>>   https://developer.nvidia.com/academic_gpu_seeding
>> or search the web
>>
>> Hieu Hoang
>>
>> On 8 June 2018 at 14:14, amir haghighi 
>> wrote:
>>
>>> Hello
>>>
>>> I'm going to set up an NMT system using openNMT or Nematus but I can't
>>> run it on my laptop and I don't have access to any cluster.
>>> I was wondering if there is any free cloud computing service which can
>>> be used to setup a full-size state of the art NMT system?
>>>
>>> Thanks
>>>
>>>
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Free cloud service to train NMT

[Hieu Hoang]

try this
  https://developer.nvidia.com/academic_gpu_seeding
or search the web

Hieu Hoang

On 8 June 2018 at 14:14, amir haghighi  wrote:

> Hello
>
> I'm going to set up an NMT system using openNMT or Nematus but I can't run
> it on my laptop and I don't have access to any cluster.
> I was wondering if there is any free cloud computing service which can be
> used to setup a full-size state of the art NMT system?
>
> Thanks
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Unsubscribe me please

[Hieu Hoang]

gladly. However, if you subscribed yourself, you should know how to
unsubscribe yourself.

If not, your should contact the owners at
   moses-support-ow...@mit.edu

Please don't spam everyone on the list with your request


Hieu Hoang
http://moses-smt.org/


On 15 May 2018 at 14:11, blessybjohn94 <blessybjoh...@gmail.com> wrote:

>
>
> Please unsubscribe this email.
> I don't get any email from your mail.
> Please unsubscribe me.
>
>
> Sent from my Samsung Galaxy smartphone.
>
> Please unsubscribe this email.
> I don't get any email from your mail.
> Please unsubscribe me.
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Please fix my error

[Hieu Hoang]

irstlm has not been compiled successfully. Unless you need a specific 
feature from IRSTLM, you should use KenLM which is built into Moses



On 10/05/18 18:17, Abhijeet Ranjan wrote:

I am getting error on running the following command:


./bjam --with-boost=/home/pallavi/smt/boost_1_55_0 
--with-irstlm=/home/pallavi/smt/irstlm 
--with-giza-pp=/home/pallavi/smt/bin -j4



___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://moses-smt.org/less

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Install moses on Windows

[Hieu Hoang]

ah ok. If you're only looking to do decoding on windows, rather training,
then Moses2 compiles with Visual Studio. You can find the Visual Studio
project files in
   contrib/other-builds/moses2

If you want to do training on Windows, there's also the Ubuntu subsystem on
Windows
   https://docs.microsoft.com/en-us/windows/wsl/about
I used this to test training on windows for Moses version 4 release. More
details here
   http://www.statmt.org/moses/RELEASE-4.0/Moses-v4.0-release-notes.pdf

Hieu Hoang
http://moses-smt.org/


On 9 May 2018 at 16:44, Mohamed Amine MENACER <am_mena...@esi.dz> wrote:

> Thank you Hieu for your answer.
> I have already tested Moses on Linux without any problem.
> The issue is that I want to integrate Moses with a Windows application,
> for this I have to install it on Windows and I am not familiar with Cygwin.
> Perhaps I will use a virtual machine running on Windows and I use it as a
> server.
>
> Thank you again.
> Amine.
>
> 2018-05-09 17:30 GMT+02:00 Hieu Hoang <hieuho...@gmail.com>:
>
>> My advise is to use Linux (Ubuntu 16.04). Even if it's inside a virtual
>> machine running on Windows, it will probably be more reliable than running
>> on Cygwin
>>
>> On 09/05/18 14:55, Mohamed Amine MENACER wrote:
>>
>> Dear all,
>>
>> I am trying to install Moses on Windows via Cygwin, here is the exact
>> command that I have used:
>>
>>  ./bjam --with-boost=/usr/lib --with-cmph=/home/am_me/cmph-2.0
>> --with-xmlrpc-c=/home/am_me/xmlrpc-c-1.39.12
>> --with-srilm=/home/am_me/srilm-1.7.2 -j 4
>>
>> However, I met with the attached error.
>>
>> Any help is greatly appreciated.
>> Warm regards.
>> Amine.
>> --
>>
>> Cordialement
>>
>> Mohamed Amine Menacer
>> PhD Student, LORIA -France-
>> Ingénieur d'état en informatique, École nationale Supérieure
>> d'Informatique E.S.I -Algerie-
>>
>>
>> ___
>> Moses-support mailing 
>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> --
>> Hieu Hoanghttp://moses-smt.org/
>>
>>
>
>
> --
>
> Cordialement
>
> Mohamed Amine Menacer
> Thésard, LORIA -France-
> Ingénieur d'état en informatique, École nationale Supérieure
> d'Informatique E.S.I -Algerie-
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Install moses on Windows

[Hieu Hoang]

My advise is to use Linux (Ubuntu 16.04). Even if it's inside a virtual 
machine running on Windows, it will probably be more reliable than 
running on Cygwin



On 09/05/18 14:55, Mohamed Amine MENACER wrote:

Dear all,

I am trying to install Moses on Windows via Cygwin, here is the exact 
command that I have used:


./bjam --with-boost=/usr/lib --with-cmph=/home/am_me/cmph-2.0 
--with-xmlrpc-c=/home/am_me/xmlrpc-c-1.39.12 
--with-srilm=/home/am_me/srilm-1.7.2 -j 4


However, I met with the attached error.

Any help is greatly appreciated.
Warm regards.
Amine.
--

Cordialement


Mohamed Amine Menacer
PhD Student, LORIA -France-
Ingénieur d'état en informatique, École nationale Supérieure 
d'Informatique E.S.I -Algerie-



___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
http://moses-smt.org/

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] https://www.mail-archive.com/moses-support@mit.edu/ is still not reached from Turkey after 3 years

[Hieu Hoang]

Hieu Hoang
http://moses-smt.org/


On 23 April 2018 at 14:02, Ergun Bicici <bic...@gmail.com> wrote:

>
> Dear Hieu,
>
> Thank you for your message. Is there a mirror website to access?
>
external websites mirror the moses-support mailing list archive, eg
   http://moses-support.mit.narkive.com/#slidedown_90


> or could the archives and current group moved to another domain that may
> be more secure against impersonation or internet access cutoff? I found
> that http://www.mail-archive.com/ is blocked for some reason. For
> instance googlegroups might be a possibility as they might be more focused
> on personal communication rather than forums / blogs as the raised issue
> might be sourcing from this. My main concern is that I would like to be
> able to send messages to moses-support and see that they are there at least
> for some time so that even if there is some man-in-the-middle attack, the
> messages can still find their way.
>
> Anyway, I did the reporting part from my side to gain access to
> moses-support from Turkey without internet tunnelling. I don't know the
> exact reason for blocking wikipedia but I heard that it is due to some
> content and my current suggestion is to block that part of wikipedia and it
> is damaging internet search since usually wikipedia entries come up.
>
> Also,​ when I tried to download the CzEng dataset, we encountered a
> difficulty with Onrei Bojar, for that we talked about md5sum usage to
> verify downloaded files. Maybe there was some interception of the network
> download. Therefore, md5sums can be provided with datasets shared.
>
> Regards,
> Ergun
>
> On Mon, Apr 23, 2018 at 1:43 PM, Hieu Hoang <hieuho...@gmail.com> wrote:
>
>> sorry to hear that. I was in Turkey a few weeks ago and saw they blocked
>> innocuous sites such wikipedia, so not surprised mail-archived is also
>> blocked for whatever reason.
>>
>> PM you link to the mit.edu's internal archive. I don't share this
>> publicly as it has people's raw emails, don't want it to be harvested by
>> spammers
>>
>> On 23/04/18 11:28, Ergun Bicici wrote:
>>
>>
>> Dear Moses mailing list,
>>
>> I am not able to reach https://www.mail-archive
>> .com/moses-support@mit.edu/ from Turkey for the last 3 years and why is
>> so is still a mystery to me. I am able to post/send messages but could
>> check them through internet tunneling.
>>
>> Do you know why / have some explanation? Thank you.
>>
>> Here are steps:
>> - I verified that is is blocked
>> - I asked about this to BTK (https://www.btk.gov.tr/, but they did not
>> solve a more specific issue I asked for)
>> - I reported to https://turkeyblocks.org
>>
>> https://www.comparitech.com/privacy-security-tools/blockedinturkey/
>>
>> DOMAIN TO CHECK
>>
>> Istanbul -  www.mail-archive.com/moses-support@mit.edu/ *Not Working* in
>> Turkey.
>>
>> Ankara -  www.mail-archive.com/moses-support@mit.edu/ *Not Working* in
>> Turkey.
>> --
>> This URL appears to be blocked in Turkey.
>>
>>
>>
>> Best Regards,
>> Ergun
>>
>> Ergun Biçici
>> http://bicici.github.com/ <http://ergunbicici.blogspot.com/>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Workshop on Statistical Machine Translation" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to wmt-tasks+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> Hieu Hoanghttp://moses-smt.org/
>>
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
>
> Regards,
> Ergun
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] https://www.mail-archive.com/moses-support@mit.edu/ is still not reached from Turkey after 3 years

[Hieu Hoang]

sorry to hear that. I was in Turkey a few weeks ago and saw they blocked 
innocuous sites such wikipedia, so not surprised mail-archived is also 
blocked for whatever reason.


PM you link to the mit.edu's internal archive. I don't share this 
publicly as it has people's raw emails, don't want it to be harvested by 
spammers


On 23/04/18 11:28, Ergun Bicici wrote:


Dear Moses mailing list,

I am not able to reach 
https://www.mail-archive.com/moses-support@mit.edu/ from Turkey for 
the last 3 years and why is so is still a mystery to me. I am able to 
post/send messages but could check them through internet tunneling.


Do you know why / have some explanation? Thank you.

Here are steps:
- I verified that is is blocked
- I asked about this to BTK (https://www.btk.gov.tr/, but they did not 
solve a more specific issue I asked for)

- I reported to https://turkeyblocks.org

https://www.comparitech.com/privacy-security-tools/blockedinturkey/

DOMAIN TO CHECK

Istanbul -www.mail-archive.com/moses-support@mit.edu/ 
<http://www.mail-archive.com/moses-support@mit.edu/>*Not Working*in 
Turkey.


Ankara -www.mail-archive.com/moses-support@mit.edu/ 
<http://www.mail-archive.com/moses-support@mit.edu/>*Not Working*in 
Turkey.



This URL appears to be blocked in Turkey.



Best Regards,
Ergun

Ergun Biçici
http://bicici.github.com/ <http://ergunbicici.blogspot.com/>

--
You received this message because you are subscribed to the Google 
Groups "Workshop on Statistical Machine Translation" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to wmt-tasks+unsubscr...@googlegroups.com 
<mailto:wmt-tasks+unsubscr...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.


--
Hieu Hoang
http://moses-smt.org/

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Error

[Hieu Hoang]

it looks like the xmlrpc-c library has not been compiled correctly.

If you don't need this library (required by moses server and moses2) then
add the argument
--no-xmlrpc-c
to the bjam command when you compile moses

Hieu Hoang
http://moses-smt.org/


2018-04-20 2:04 GMT+01:00 Gedour Abdoullah <gedou...@gmail.com>:

>
> ./bjam --with-srilm=/home/kader/Bureau/workMoses/srilm
> --with-irstlm=/home/kader/Bureau/workMoses/irstlm/trunk
> --with-giza=/home/kader/Bureau/workMoses/tools
> --
> Sidi Abdellahi Abdelkader
> Master en Systeme d'infomation et nouvelle technologie
> Faculté des Sciences Économiques et de Gestion de Sfax (FSEG)
> Département Informatique
> Université de Sfax
> Tunisie
> Tél : 00216 52 303 538
> Site web : http://sidiabdelghader7.wixsite.com/sidi
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Dual Licensing or relicensing Moses

[Hieu Hoang]

Still the same problem - everyone owns Moses so you need everyone's
permission, not just mine. So no

Hieu Hoang
http://moses-smt.org/


On 10 April 2018 at 17:13, liling tan <alvati...@gmail.com> wrote:

> I understand.
>
> Could we have permission that it's okay to derive work from Moses with
> respect to the (de-)tokenizer and possibly other scripts under an
> MIT/Apache tool?
>
> Legally it's a restriction but I think for what's it worth, having mutual
> agreement between the OSS is sufficient to still keep any port of LGPL work
> until someone starts to enforce legal actions and I think it's safe to back
> off to taking down these functionalities in the Apache/MIT code.
>
> Regards,
> Liling
>
> On Wed, Apr 11, 2018 at 12:09 AM, Hieu Hoang <hieuho...@gmail.com> wrote:
>
>> we can't change the license, or dual license it, without the agreement of
>> everyone who's contributed to Moses. Too much work
>>
>> Hieu Hoang
>> http://moses-smt.org/
>>
>>
>> On 10 April 2018 at 15:47, liling tan <alvati...@gmail.com> wrote:
>>
>>> Dear Moses Dev,
>>>
>>> NLTK has a Python port of the word tokenizer in Moses. The tokenizer
>>> works well in Python and create a good synergy to bridge Python users to
>>> the code that Moses developers have spent years to hone.
>>>
>>> But it seemed to have hit a wall with some licensing issues.
>>> https://github.com/nltk/nltk/issues/2000
>>>
>>> General port of LGPL code is considered derivative and is incompatible
>>> with Apache or MIT license. I understand that LGPL keeps derivative from
>>> being proprietary but it's a little less permissive than non-copyleft
>>> license like Apache and MIT licenses.
>>>
>>> Note that this licensing issue might also affect Marian which is MIT
>>> license and also incompatible with LGPL so although technically users can
>>> chain the code from different libraries, but Marian couldn't have any
>>> dependencies on the Moses components. (But we know do know that none of our
>>> models built with Marian would work without the Moses tokenizer which is in
>>> LGPL).
>>>
>>> Would there be a possibility to dual license the Moses repository with
>>> LGPL and Apache/BSD/MIT license. I'm not sure whether it's allowed to have
>>> dual licenses with LGPL and Apache/BSD/MIT license though. Might have to
>>> check with some proper legal personnel though.
>>>
>>> If dual license is not possible would it be possible relicense the code
>>> under BSD/Apache/MIT license? That way it's more permissive for derivatiive
>>> work?
>>>
>>> I think the last scenario is for NLTK to drop the Python port of Moses
>>> code entirely from Apache license repository but I think that'll remove the
>>> synergy between various OSS.
>>>
>>> Hope to hear from Moses devs soon!
>>>
>>> Regards,
>>> Liling
>>>
>>>
>>>
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Dual Licensing or relicensing Moses

[Hieu Hoang]

we can't change the license, or dual license it, without the agreement of
everyone who's contributed to Moses. Too much work

Hieu Hoang
http://moses-smt.org/


On 10 April 2018 at 15:47, liling tan <alvati...@gmail.com> wrote:

> Dear Moses Dev,
>
> NLTK has a Python port of the word tokenizer in Moses. The tokenizer works
> well in Python and create a good synergy to bridge Python users to the code
> that Moses developers have spent years to hone.
>
> But it seemed to have hit a wall with some licensing issues.
> https://github.com/nltk/nltk/issues/2000
>
> General port of LGPL code is considered derivative and is incompatible
> with Apache or MIT license. I understand that LGPL keeps derivative from
> being proprietary but it's a little less permissive than non-copyleft
> license like Apache and MIT licenses.
>
> Note that this licensing issue might also affect Marian which is MIT
> license and also incompatible with LGPL so although technically users can
> chain the code from different libraries, but Marian couldn't have any
> dependencies on the Moses components. (But we know do know that none of our
> models built with Marian would work without the Moses tokenizer which is in
> LGPL).
>
> Would there be a possibility to dual license the Moses repository with
> LGPL and Apache/BSD/MIT license. I'm not sure whether it's allowed to have
> dual licenses with LGPL and Apache/BSD/MIT license though. Might have to
> check with some proper legal personnel though.
>
> If dual license is not possible would it be possible relicense the code
> under BSD/Apache/MIT license? That way it's more permissive for derivatiive
> work?
>
> I think the last scenario is for NLTK to drop the Python port of Moses
> code entirely from Apache license repository but I think that'll remove the
> synergy between various OSS.
>
> Hope to hear from Moses devs soon!
>
> Regards,
> Liling
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Installation: symlinks created at the wrong place when using --prefix?

[Hieu Hoang]

I would actually advised against using --prefix

The moses scripts are written with the assumption that the directories
   bin
   scripts
are side-by-side. --prefix seems to move bin, but not scripts

Hieu Hoang
http://moses-smt.org/


On 20 February 2018 at 09:20, Hieu Hoang <hieuho...@gmail.com> wrote:

>
>
> Hieu Hoang
> Sent while bumping into things
>
> On 20 Feb 2018 8:25 a.m., "Yves Scherrer" <yves.scher...@helsinki.fi>
> wrote:
>
> Hi all,
>
> My first attempt at testing moses2 failed because CreateProbingPT2 could
> not be found...
>
> On the last line of mosesdecoder/Jamroot,  is created as a symlink to
> CreateProbingPT, but this is done in the $(TOP)/bin directory (same for
> moses_chart BTW). However, $(TOP) refers to the build directory, which may
> differ from the final install directory, namely when --prefix is used and
> set to something else than the current directory. In this case, the
> symlinks are indeed created, but in a directory from which the executables
> have already been moved out of...
>
> Can somebody confirm that behavior? I'm a little surprised that this
> hasn't come up earlier... For me, replacing $(TOP) by $(PREFIX) in the
> last couple of lines of the Jamroot file did the trick both when using
> --prefix and when not using it, but maybe there is a better way to handle
> this.
>
> Do a git pull for your changes.
>
> I don't think a lot of people use --prefix so the bug was missed.
>
> Btw, you can use CreateProbingPT instead of CreateProbingPT2. The soft
> link was added for backyard compatibility when the two used to be different
>
>
> Best,
> Yves
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Installation: symlinks created at the wrong place when using --prefix?

[Hieu Hoang]

Hieu Hoang
Sent while bumping into things

On 20 Feb 2018 8:25 a.m., "Yves Scherrer" <yves.scher...@helsinki.fi> wrote:

Hi all,

My first attempt at testing moses2 failed because CreateProbingPT2 could
not be found...

On the last line of mosesdecoder/Jamroot,  is created as a symlink to
CreateProbingPT, but this is done in the $(TOP)/bin directory (same for
moses_chart BTW). However, $(TOP) refers to the build directory, which may
differ from the final install directory, namely when --prefix is used and
set to something else than the current directory. In this case, the
symlinks are indeed created, but in a directory from which the executables
have already been moved out of...

Can somebody confirm that behavior? I'm a little surprised that this hasn't
come up earlier... For me, replacing $(TOP) by $(PREFIX) in the last couple
of lines of the Jamroot file did the trick both when using --prefix and
when not using it, but maybe there is a better way to handle this.

Do a git pull for your changes.

I don't think a lot of people use --prefix so the bug was missed.

Btw, you can use CreateProbingPT instead of CreateProbingPT2. The soft link
was added for backyard compatibility when the two used to be different


Best,
Yves

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Regarding On-Disk Phrase table

[Hieu Hoang]

after you compile moses, it should be in the bin/ directory

Hieu Hoang
http://moses-smt.org/


On 16 February 2018 at 13:21, Kamal Deep Garg <kamal.1...@gmail.com> wrote:

> Dear Sir
>
> To run this command
>
> ~/CreateOnDiskPt 1 1 4 100 2 phrase-table.1.gz phrase-table.1.folder
>
> how to install CreateOnDiskPt
>
>
>
> Thank You
>
> Kamal Deep Garg
> 9988493359
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Issue in install to Moses 4.0 in Ubuntu 14

[Hieu Hoang]

you might want to delete and do a clean compile from the beginning, and use
master rather than version 4.

It worked for me with ubuntu 14.04

Hieu Hoang
http://moses-smt.org/


On 6 February 2018 at 06:15, Atul Kr. Ojha <shashwatu...@gmail.com> wrote:

> Hi,
>
> Is there any problem to install Moses 4.0 in ubuntu 14 version (because it
> works on my laptop where I am using Ubuntu 16)?
>
> For your reference, I have attached the build.log.gz file.
>
> Please suggest me how to solve or install the above version in Ubuntu 14
>
>
> t
> hanks and regards,
> Atul
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] please help me

[Hieu Hoang]

it runs fine for me. Check that you've spelt everything correctly in the 
moses.ini file


    ./mosesserver --server-port 8080  -f phrase-model/moses.ini
Defined parameters (per moses.ini or switch):
     config: phrase-model/moses.ini
     feature: KENLM name=LM factor=0 order=3 num-features=1 
path=lm/europarl.srilm.gz Distortion WordPenalty UnknownWordPenalty 
PhraseDictionaryMemory input-factor=0 output-factor=0 
path=phrase-model/phrase-table num-features=1 table-limit=10
     input-factors: 0
     mapping: T 0
     n-best-list: nbest.txt 100
     server:
     server-port: 8080
     weight: WordPenalty0= 0 LM= 1 Distortion0= 1 
PhraseDictionaryMemory0= 1
line=KENLM name=LM factor=0 order=3 num-features=1 path=lm/europarl.srilm.gz
Loading the LM will be faster if you build a binary file.
Reading lm/europarl.srilm.gz
5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
**The ARPA file is missing .  Substituting log10 probability -100.000.
**
FeatureFunction: LM start: 0 end: 0
line=Distortion
FeatureFunction: Distortion0 start: 1 end: 1
line=WordPenalty
FeatureFunction: WordPenalty0 start: 2 end: 2
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 3 end: 3
line=PhraseDictionaryMemory input-factor=0 output-factor=0 
path=phrase-model/phrase-table num-features=1 table-limit=10
FeatureFunction: PhraseDictionaryMemory0 start: 4 end: 4
Loading LM
Loading Distortion0
Loading WordPenalty0
Loading UnknownWordPenalty0
Loading PhraseDictionaryMemory0
Start loading text phrase table. Moses format : [0.473] seconds
Reading phrase-model/phrase-table
5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100

RUN SERVER at pid 0
[moses/server/Server.cpp:49] Listening on port 8080


On 31/01/18 05:11, seid mehamed wrote:
> --server-port 8080 

-- 
Hieu Hoang
http://moses-smt.org/

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support