Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-15 Thread Shubham Khandelwal
Hello, I was using http://www.statmt.org/moses/RELEASE-3.0/models/fr- en/tuning/moses.tuned.ini.5 There I had converted phrase table and reordering table to their compact format and also made some changes in this moses.tuned.ini.5 file (attached in this email) to decode fastly. Now, this time, I

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-15 Thread Hieu Hoang
there is no limit to the number of words. if you are using the premade models, make sure you are using tuned moses.ini file in tuning/moses.tuned.ini.? Hieu Hoang http://www.hoang.co.uk/hieu On 13 December 2016 at 09:32, Shubham Khandelwal wrote: > Thanks. Hieu. I understood now. > Also, Is

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-13 Thread Shubham Khandelwal
Hi, This is just the reminder of my previous email. Thanks. Regards, Shubham On Dec 13, 2016 3:02 PM, "Shubham Khandelwal" wrote: > Thanks. Hieu. I understood now. > Also, Is there any limit of number of words for the translation because > when I use this command: *~/mosesdecoder/bin/moses -f

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-13 Thread Shubham Khandelwal
Thanks. Hieu. I understood now. Also, Is there any limit of number of words for the translation because when I use this command: *~/mosesdecoder/bin/moses -f moses.ini* Then, it translates only some words which are in the beginning. I mean, it does not consume all words for the translation after Cr

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-12 Thread Shubham Khandelwal
Okay Thanks Hieu. I will try it with 1TB HD-memory machine. Btw I can see there are 4 pre-made models available for fr-en and de-en ( http://www.statmt.org/moses/RELEASE-3.0/models/fr-en/model/ and http://www.statmt.org/moses/RELEASE-3.0/models/de-en/model/). Can you please tell me among these 4,

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-12 Thread Hieu Hoang
there are actually 7 different configurations. You have to look at the config file in steps/?/config.? For fr-en: 1. phrase-based, truecased 2. phrase-based, lowercased then recased 3. hierarchical model, lowercased then recased 4. phrase-based, lowercased then recased. Using target side

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-12 Thread Hieu Hoang
Hieu Hoang http://www.hoang.co.uk/hieu On 10 December 2016 at 14:06, Shubham Khandelwal wrote: > Yes, CreateOnDiskPt command executed without any error. > > There are 5 files in this phrase-table.3.folder: Misc.dat , Source.dat, > TargetColl.dat, TargetInd.dat, Vocab.dat > *Misc.dat and Vocab.d

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-10 Thread Shubham Khandelwal
Yes, CreateOnDiskPt command executed without any error. There are 5 files in this phrase-table.3.folder: Misc.dat , Source.dat, TargetColl.dat, TargetInd.dat, Vocab.dat *Misc.dat and Vocab.dat files are empty. * I just checked that my hard-disk memory is full as this folder took 165G already. So

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-10 Thread Hieu Hoang
strange, did the CreateOnDiskPt command execute ok, ie. with no error? Does this file exist: /home/shubham/models/fr-en/phrase-table.3.folder/Misc.dat If you do cat Misc.dat what does it say? Hieu Hoang http://www.hoang.co.uk/hieu On 10 December 2016 at 11:30, Shubham Khandelwal wrote: >

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-10 Thread Shubham Khandelwal
Hi, Thanks Hieu for your reply. I have used *CreateOnDiskPt* to binarize the model and stored in *phrase-table.3.folder *using following command: *~/mosesdecoder/bin/CreateOnDiskPt 1 1 4 100 2 phrase-table.3.gz phrase-table.3.folder* Also I have made changes in *moses.ini.3 (*i.e. I have conve

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-09 Thread Hieu Hoang
This is a hierarchical model. You must binarize with CreateOnDiskPt for this model Hieu Hoang http://www.hoang.co.uk/hieu On 9 December 2016 at 08:18, Shubham Khandelwal wrote: > Hello, > > Thanks. It worked now. I have created compact phrase table. > Now, when I am running it using following c

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-09 Thread Shubham Khandelwal
Hello, Thanks. It worked now. I have created compact phrase table. Now, when I am running it using following command: *~/mosesdecoder/bin/moses -f ~/Translate/models/de-en/model/moses.ini.2 -threads all* Then, after creating input-output object, it gave following segmentation fault error: Cre

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-08 Thread Hieu Hoang
maybe try -encoding None On 08/12/2016 19:44, Shubham Khandelwal wrote: Hi Hieu, Thanks for your reply. Yes, I have used the absolute path and also I tried with -T but it did not work. Is there any other solution to this problem. Btw, Can anybody please upload the compact model of all pr

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-08 Thread Shubham Khandelwal
Hi Hieu, Thanks for your reply. Yes, I have used the absolute path and also I tried with -T but it did not work. Is there any other solution to this problem. Btw, Can anybody please upload the compact model of all pre-made models as this will take less space and also it will be very fast during d

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-08 Thread Hieu Hoang
the previous email you referred to says that the directory * binarised-model/* *must exist before you run it, otherwise it will segfault. I would also use absolute path to make sure, ie. not * *binarised-model/phrase-table * *but* * /home/shubham/moses/binarised-model/phrase-table * *The p

Re: [Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-08 Thread Shubham Khandelwal
Hello, This is just the reminder of my previous email. Thanking You. Regards, Shubham On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal wrote: > Hello, > > I have just downloaded phrase-table.2.gz (18GB) de-en model > and phrase-table.3.gz (22GB) fr-en model from the available pre-made > mod

[Moses-support] Conversion of phrase model to PhraseDictionaryCompact

2016-12-07 Thread Shubham Khandelwal
Hello, I have just downloaded phrase-table.2.gz (18GB) de-en model and phrase-table.3.gz (22GB) fr-en model from the available pre-made models. Now, I am converting them to PhraseDictionaryCompact using following command (for exmaple): *~/mosesdecoder/bin/processPhraseTableMin -threads all -in ~