Re: [Moses-support] decoding-graph-backoff

[Saumitra Yadav]

Sir,
As you mentioned adding "-tt" while running decoder for input gives us
feature score for each pair, also a non-zero value for a translation model
(given that there are multiple phrase-table) gives us indication from which
phrase-table translation candidate was used. Sir I'm making a system for
Hindi to Urdu , and after running decoder with "-tt" im getting (small
sample):

|lm=(2:-8.9735,1:-17.0476,2:-0.639163,2:-6.23737,3:-4.51238,3:-1.19594,3:-6.55882,3:-3.23557,3:-2.26149,2:-14.4246,2:-2.78677,2:-7.39963,3:-1.62681,2:-13.4158,1:-13.6408,2:-1.33196,3:-1.25049)|
بغیر |0-0,wa=0-0 ,total=-0.28041, LexicalReordering0= -0.0316486 0 0 0 0 0
Distortion0= 0 LM0= -8.9735 WordPenalty0= -1 PhrasePenalty0= 1
TranslationModel0=
-0.167537 -0.313897 -0.239363 -0.242946 TranslationModel1= 0 0 0 0| ابلی
|1-1,wa=0-0 ,total=-0.463503, LexicalReordering0= -0.510826 0 0 -0.0511291
0 0 Distortion0= 0 LM0= -17.0476 WordPenalty0= -1 PhrasePenalty0= 1
*TranslationModel0=
0 0 0 0 TranslationModel1= 0 0 0 0*|


Red colored TranslationModel0 feature tells us that translation model 0 was
used to get translated phrase in red color
but sir blue color one says that none of the two translation models were
used? Is my assumption correct?
also phrase was translated (blue font)
can you please tell why this happened?


Regards,
Saumitra Yadav
Intern, LTRC
IIIT-Hyderabad

On Thu, Jul 30, 2015 at 7:13 PM, Philipp Koehn  wrote:

> Hi,
>
> yes, that is correct. If there are non-zero valued scores listed with a
> translation model feature, then this translation model was used for the
> phrase pair.
>
> -phi
>
> On Wed, Jul 29, 2015 at 7:57 PM, Saumitra Yadav <
> yadav.saumitr...@gmail.com> wrote:
>
>> Sir,
>> Thank you for that option , it really helped. I just wanted to know if
>> I'm analysing it correctly
>> For initial analysis m just finding how many times which phrase tables
>> were called , so in attached file (formatted just for easy readability )
>> please find the output of one sentence , is it correct to say that
>> TranslationModel0 was used 2 times and TranslationModel1 was used 5 times
>> for given input?
>>
>> Regards,
>> Saumitra Yadav
>> M.Tech.
>> Department Of Computer Science And Technology
>> Goa University
>>
>>
>> On Wed, Jul 29, 2015 at 9:22 PM, Philipp Koehn  wrote:
>>
>>> Hi,
>>>
>>> when you call the decoder with the option "-tt" then you get for
>>> each phrase pairs a list of all feature scores. You can use this
>>> to track down which phrase table was used for each phrase
>>> translation.
>>>
>>> -phi
>>>
>>> On Wed, Jul 29, 2015 at 10:59 AM, Hieu Hoang 
>>> wrote:
>>>
 good question. no. You can try & write it yourself.

 in the TargetPhrase class, there is a method
   GetContainer()
 which points to the phrase-table that a particular rule comes from. You
 can use this.

 On 29/07/2015 18:51, Saumitra Yadav wrote:

 Sir,
 Is there a command or argument which can tell, which phrase in output
 is taken from which phrase-table (incase we have multiple phrase-tables )?

 Regards,
 Saumitra Yadav
 M.Tech.
 Department Of Computer Science And Technology
 Goa University


 On Sun, Jul 26, 2015 at 11:49 AM, Hieu Hoang 
 wrote:

> since you have 3 phrase-tables, you may have to have 3 entries in the
> [decoding-graph-backoff] section, eg
>   [decoding-graph-backoff]
>   0
>   3
>   3
>
>
> Hieu Hoang
> Researcher
> New York University, Abu Dhabi
> http://www.hoang.co.uk/hieu
>
> On 25 July 2015 at 20:23, Saumitra Yadav <
> yadav.saumitr...@gmail.com> wrote:
>
>> Sir,
>> Please find attached , moses.ini file i used and command used
>> was ~/Decoder/mosesdecoder/bin/moses -f moses.ini
>>
>> Regards,
>> Saumitra Yadav
>> M.Tech.
>> Department Of Computer Science And Technology
>> Goa University
>>
>>
>> On Sat, Jul 25, 2015 at 9:21 PM, Hieu Hoang 
>> wrote:
>>
>>> can you please send me the moses.ini file that you used, that cause
>>> the segfault. And send me the exact command you typed
>>>
>>>
>>> On 24/07/2015 14:40, Saumitra Yadav wrote:
>>>
>>> But sir when i did that there was * segmentation fault* while
>>> loading first phrase-table, one walk around i got was giving 
>>> phrase-table
>>> uncompressed to decoder.
>>>
>>> Regards,
>>> Saumitra Yadav
>>> M.Tech.
>>> Department Of Computer Science And Technology
>>> Goa University
>>>
>>>
>>> On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang < 
>>> hieuho...@gmail.com> wrote:
>>>
 i think you have to swap the phrase tables around. The decoder
 always looks at the 1st phrase-table, then backoff to the 2nd if 
 nothing is
 found


 On 22/07/2015 16:59, Saumitra Yadav wrote:

 Sir/Ma'am,
 I'm trying to use mu

Re: [Moses-support] decoding-graph-backoff

[Philipp Koehn]

Hi,

yes, that is correct. If there are non-zero valued scores listed with a
translation model feature, then this translation model was used for the
phrase pair.

-phi

On Wed, Jul 29, 2015 at 7:57 PM, Saumitra Yadav 
wrote:

> Sir,
> Thank you for that option , it really helped. I just wanted to know if I'm
> analysing it correctly
> For initial analysis m just finding how many times which phrase tables
> were called , so in attached file (formatted just for easy readability )
> please find the output of one sentence , is it correct to say that
> TranslationModel0 was used 2 times and TranslationModel1 was used 5 times
> for given input?
>
> Regards,
> Saumitra Yadav
> M.Tech.
> Department Of Computer Science And Technology
> Goa University
>
>
> On Wed, Jul 29, 2015 at 9:22 PM, Philipp Koehn  wrote:
>
>> Hi,
>>
>> when you call the decoder with the option "-tt" then you get for
>> each phrase pairs a list of all feature scores. You can use this
>> to track down which phrase table was used for each phrase
>> translation.
>>
>> -phi
>>
>> On Wed, Jul 29, 2015 at 10:59 AM, Hieu Hoang  wrote:
>>
>>> good question. no. You can try & write it yourself.
>>>
>>> in the TargetPhrase class, there is a method
>>>   GetContainer()
>>> which points to the phrase-table that a particular rule comes from. You
>>> can use this.
>>>
>>> On 29/07/2015 18:51, Saumitra Yadav wrote:
>>>
>>> Sir,
>>> Is there a command or argument which can tell, which phrase in output is
>>> taken from which phrase-table (incase we have multiple phrase-tables )?
>>>
>>> Regards,
>>> Saumitra Yadav
>>> M.Tech.
>>> Department Of Computer Science And Technology
>>> Goa University
>>>
>>>
>>> On Sun, Jul 26, 2015 at 11:49 AM, Hieu Hoang 
>>> wrote:
>>>
 since you have 3 phrase-tables, you may have to have 3 entries in the
 [decoding-graph-backoff] section, eg
   [decoding-graph-backoff]
   0
   3
   3


 Hieu Hoang
 Researcher
 New York University, Abu Dhabi
 http://www.hoang.co.uk/hieu

 On 25 July 2015 at 20:23, Saumitra Yadav < 
 yadav.saumitr...@gmail.com> wrote:

> Sir,
> Please find attached , moses.ini file i used and command used
> was ~/Decoder/mosesdecoder/bin/moses -f moses.ini
>
> Regards,
> Saumitra Yadav
> M.Tech.
> Department Of Computer Science And Technology
> Goa University
>
>
> On Sat, Jul 25, 2015 at 9:21 PM, Hieu Hoang 
> wrote:
>
>> can you please send me the moses.ini file that you used, that cause
>> the segfault. And send me the exact command you typed
>>
>>
>> On 24/07/2015 14:40, Saumitra Yadav wrote:
>>
>> But sir when i did that there was * segmentation fault* while
>> loading first phrase-table, one walk around i got was giving phrase-table
>> uncompressed to decoder.
>>
>> Regards,
>> Saumitra Yadav
>> M.Tech.
>> Department Of Computer Science And Technology
>> Goa University
>>
>>
>> On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang < 
>> hieuho...@gmail.com> wrote:
>>
>>> i think you have to swap the phrase tables around. The decoder
>>> always looks at the 1st phrase-table, then backoff to the 2nd if 
>>> nothing is
>>> found
>>>
>>>
>>> On 22/07/2015 16:59, Saumitra Yadav wrote:
>>>
>>> Sir/Ma'am,
>>> I'm trying to use multiple phrase tables for translation in Moses
>>> decoder, with preference to 1st phrase-table, but was getting a
>>> segmentation fault while loading 1st phrase table, so just switched the
>>> positions of phrase-tables in moses configuration file and it was 
>>> working ,
>>> now the table i want to give preference is 2nd in list , can i use
>>>
>>> [decoding-graph-backoff]
>>>  1
>>>  3
>>> in configuration file for backoff so that moses uses 2nd table and
>>> uses 1st table only for words it couldn't find in 2nd phrase-table?
>>>
>>> Regards,
>>> Saumitra Yadav
>>> M.Tech.
>>> Department Of Computer Science And Technology
>>> Goa University
>>>
>>>
>>>
>>> ___
>>> Moses-support mailing 
>>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>> --
>>> Hieu Hoang
>>> Researcher
>>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>>
>>>
>>
>> --
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>
>>
>

>>>
>>> --
>>> Hieu Hoang
>>> Researcher
>>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>>
>>>
>>> ___
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
> ___
> Moses-support mailing list
> Moses-supp

Re: [Moses-support] decoding-graph-backoff

[Saumitra Yadav]

Sir,
Thank you for that option , it really helped. I just wanted to know if I'm
analysing it correctly
For initial analysis m just finding how many times which phrase tables were
called , so in attached file (formatted just for easy readability ) please
find the output of one sentence , is it correct to say that
TranslationModel0 was used 2 times and TranslationModel1 was used 5 times
for given input?

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University


On Wed, Jul 29, 2015 at 9:22 PM, Philipp Koehn  wrote:

> Hi,
>
> when you call the decoder with the option "-tt" then you get for
> each phrase pairs a list of all feature scores. You can use this
> to track down which phrase table was used for each phrase
> translation.
>
> -phi
>
> On Wed, Jul 29, 2015 at 10:59 AM, Hieu Hoang  wrote:
>
>>  good question. no. You can try & write it yourself.
>>
>> in the TargetPhrase class, there is a method
>>   GetContainer()
>> which points to the phrase-table that a particular rule comes from. You
>> can use this.
>>
>> On 29/07/2015 18:51, Saumitra Yadav wrote:
>>
>>  Sir,
>>  Is there a command or argument which can tell, which phrase in output is
>> taken from which phrase-table (incase we have multiple phrase-tables )?
>>
>>   Regards,
>>  Saumitra Yadav
>>  M.Tech.
>>  Department Of Computer Science And Technology
>>  Goa University
>>
>>
>> On Sun, Jul 26, 2015 at 11:49 AM, Hieu Hoang  wrote:
>>
>>>  since you have 3 phrase-tables, you may have to have 3 entries in the
>>> [decoding-graph-backoff] section, eg
>>>   [decoding-graph-backoff]
>>>   0
>>>   3
>>>   3
>>>
>>>
>>>  Hieu Hoang
>>> Researcher
>>>  New York University, Abu Dhabi
>>>  http://www.hoang.co.uk/hieu
>>>
>>>   On 25 July 2015 at 20:23, Saumitra Yadav <
>>> yadav.saumitr...@gmail.com> wrote:
>>>
 Sir,
 Please find attached , moses.ini file i used and command used
 was ~/Decoder/mosesdecoder/bin/moses -f moses.ini

   Regards,
  Saumitra Yadav
  M.Tech.
  Department Of Computer Science And Technology
  Goa University


   On Sat, Jul 25, 2015 at 9:21 PM, Hieu Hoang 
 wrote:

>  can you please send me the moses.ini file that you used, that cause
> the segfault. And send me the exact command you typed
>
>
> On 24/07/2015 14:40, Saumitra Yadav wrote:
>
> But sir when i did that there was * segmentation fault* while loading
> first phrase-table, one walk around i got was giving phrase-table
> uncompressed to decoder.
>
>   Regards,
>  Saumitra Yadav
>  M.Tech.
>  Department Of Computer Science And Technology
>  Goa University
>
>
> On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang < 
> hieuho...@gmail.com> wrote:
>
>>  i think you have to swap the phrase tables around. The decoder
>> always looks at the 1st phrase-table, then backoff to the 2nd if nothing 
>> is
>> found
>>
>>
>> On 22/07/2015 16:59, Saumitra Yadav wrote:
>>
>>   Sir/Ma'am,
>>  I'm trying to use multiple phrase tables for translation in Moses
>> decoder, with preference to 1st phrase-table, but was getting a
>> segmentation fault while loading 1st phrase table, so just switched the
>> positions of phrase-tables in moses configuration file and it was 
>> working ,
>> now the table i want to give preference is 2nd in list , can i use
>>
>> [decoding-graph-backoff]
>>  1
>>  3
>>  in configuration file for backoff so that moses uses 2nd table and
>> uses 1st table only for words it couldn't find in 2nd phrase-table?
>>
>> Regards,
>>  Saumitra Yadav
>>  M.Tech.
>>  Department Of Computer Science And Technology
>>  Goa University
>>
>>
>>
>>  ___
>> Moses-support mailing 
>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> --
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>
>>
>
> --
> Hieu Hoang
> Researcher
> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>
>

>>>
>>
>> --
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>
>>
>> ___
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>


c1
Description: Binary data
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] decoding-graph-backoff

[Philipp Koehn]

Hi,

when you call the decoder with the option "-tt" then you get for
each phrase pairs a list of all feature scores. You can use this
to track down which phrase table was used for each phrase
translation.

-phi

On Wed, Jul 29, 2015 at 10:59 AM, Hieu Hoang  wrote:

>  good question. no. You can try & write it yourself.
>
> in the TargetPhrase class, there is a method
>   GetContainer()
> which points to the phrase-table that a particular rule comes from. You
> can use this.
>
> On 29/07/2015 18:51, Saumitra Yadav wrote:
>
>  Sir,
>  Is there a command or argument which can tell, which phrase in output is
> taken from which phrase-table (incase we have multiple phrase-tables )?
>
>   Regards,
>  Saumitra Yadav
>  M.Tech.
>  Department Of Computer Science And Technology
>  Goa University
>
>
> On Sun, Jul 26, 2015 at 11:49 AM, Hieu Hoang  wrote:
>
>>  since you have 3 phrase-tables, you may have to have 3 entries in the
>> [decoding-graph-backoff] section, eg
>>   [decoding-graph-backoff]
>>   0
>>   3
>>   3
>>
>>
>>  Hieu Hoang
>> Researcher
>>  New York University, Abu Dhabi
>>  http://www.hoang.co.uk/hieu
>>
>>   On 25 July 2015 at 20:23, Saumitra Yadav < 
>> yadav.saumitr...@gmail.com> wrote:
>>
>>> Sir,
>>> Please find attached , moses.ini file i used and command used
>>> was ~/Decoder/mosesdecoder/bin/moses -f moses.ini
>>>
>>>   Regards,
>>>  Saumitra Yadav
>>>  M.Tech.
>>>  Department Of Computer Science And Technology
>>>  Goa University
>>>
>>>
>>>   On Sat, Jul 25, 2015 at 9:21 PM, Hieu Hoang 
>>> wrote:
>>>
  can you please send me the moses.ini file that you used, that cause
 the segfault. And send me the exact command you typed


 On 24/07/2015 14:40, Saumitra Yadav wrote:

 But sir when i did that there was * segmentation fault* while loading
 first phrase-table, one walk around i got was giving phrase-table
 uncompressed to decoder.

   Regards,
  Saumitra Yadav
  M.Tech.
  Department Of Computer Science And Technology
  Goa University


 On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang < 
 hieuho...@gmail.com> wrote:

>  i think you have to swap the phrase tables around. The decoder always
> looks at the 1st phrase-table, then backoff to the 2nd if nothing is found
>
>
> On 22/07/2015 16:59, Saumitra Yadav wrote:
>
>   Sir/Ma'am,
>  I'm trying to use multiple phrase tables for translation in Moses
> decoder, with preference to 1st phrase-table, but was getting a
> segmentation fault while loading 1st phrase table, so just switched the
> positions of phrase-tables in moses configuration file and it was working 
> ,
> now the table i want to give preference is 2nd in list , can i use
>
> [decoding-graph-backoff]
>  1
>  3
>  in configuration file for backoff so that moses uses 2nd table and
> uses 1st table only for words it couldn't find in 2nd phrase-table?
>
> Regards,
>  Saumitra Yadav
>  M.Tech.
>  Department Of Computer Science And Technology
>  Goa University
>
>
>
>  ___
> Moses-support mailing 
> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> --
> Hieu Hoang
> Researcher
> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>
>

 --
 Hieu Hoang
 Researcher
 New York University, Abu Dhabihttp://www.hoang.co.uk/hieu


>>>
>>
>
> --
> Hieu Hoang
> Researcher
> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] decoding-graph-backoff

[Hieu Hoang]

good question. no. You can try & write it yourself.

in the TargetPhrase class, there is a method
  GetContainer()
which points to the phrase-table that a particular rule comes from. You 
can use this.


On 29/07/2015 18:51, Saumitra Yadav wrote:

Sir,
Is there a command or argument which can tell, which phrase in output 
is taken from which phrase-table (incase we have multiple phrase-tables )?


Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University


On Sun, Jul 26, 2015 at 11:49 AM, Hieu Hoang > wrote:


since you have 3 phrase-tables, you may have to have 3 entries in
the [decoding-graph-backoff] section, eg
  [decoding-graph-backoff]
  0
  3
  3


Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 25 July 2015 at 20:23, Saumitra Yadav
mailto:yadav.saumitr...@gmail.com>>
wrote:

Sir,
Please find attached , moses.ini file i used and command used
was ~/Decoder/mosesdecoder/bin/moses -f moses.ini

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University


On Sat, Jul 25, 2015 at 9:21 PM, Hieu Hoang
mailto:hieuho...@gmail.com>> wrote:

can you please send me the moses.ini file that you used,
that cause the segfault. And send me the exact command you
typed


On 24/07/2015 14:40, Saumitra Yadav wrote:

But sir when i did that there was *segmentation fault*
while loading first phrase-table, one walk around i got
was giving phrase-table uncompressed to decoder.

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University


On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang
mailto:hieuho...@gmail.com>> wrote:

i think you have to swap the phrase tables around.
The decoder always looks at the 1st phrase-table,
then backoff to the 2nd if nothing is found


On 22/07/2015 16:59, Saumitra Yadav wrote:

Sir/Ma'am,
I'm trying to use multiple phrase tables for
translation in Moses decoder, with preference to 1st
phrase-table, but was getting a segmentation fault
while loading 1st phrase table, so just switched the
positions of phrase-tables in moses configuration
file and it was working , now the table i want to
give preference is 2nd in list , can i use

[decoding-graph-backoff]
 1
 3
in configuration file for backoff so that moses uses
2nd table and uses 1st table only for words it
couldn't find in 2nd phrase-table?

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University



___
Moses-support mailing list
Moses-support@mit.edu 
http://mailman.mit.edu/mailman/listinfo/moses-support


-- 
Hieu Hoang

Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu




-- 
Hieu Hoang

Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu






--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] decoding-graph-backoff

[Saumitra Yadav]

Sir,
Is there a command or argument which can tell, which phrase in output is
taken from which phrase-table (incase we have multiple phrase-tables )?

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University


On Sun, Jul 26, 2015 at 11:49 AM, Hieu Hoang  wrote:

> since you have 3 phrase-tables, you may have to have 3 entries in the
> [decoding-graph-backoff] section, eg
>   [decoding-graph-backoff]
>   0
>   3
>   3
>
>
> Hieu Hoang
> Researcher
> New York University, Abu Dhabi
> http://www.hoang.co.uk/hieu
>
> On 25 July 2015 at 20:23, Saumitra Yadav 
> wrote:
>
>> Sir,
>> Please find attached , moses.ini file i used and command used
>> was ~/Decoder/mosesdecoder/bin/moses -f moses.ini
>>
>> Regards,
>> Saumitra Yadav
>> M.Tech.
>> Department Of Computer Science And Technology
>> Goa University
>>
>>
>> On Sat, Jul 25, 2015 at 9:21 PM, Hieu Hoang  wrote:
>>
>>>  can you please send me the moses.ini file that you used, that cause the
>>> segfault. And send me the exact command you typed
>>>
>>>
>>> On 24/07/2015 14:40, Saumitra Yadav wrote:
>>>
>>> But sir when i did that there was * segmentation fault* while loading
>>> first phrase-table, one walk around i got was giving phrase-table
>>> uncompressed to decoder.
>>>
>>>   Regards,
>>>  Saumitra Yadav
>>>  M.Tech.
>>>  Department Of Computer Science And Technology
>>>  Goa University
>>>
>>>
>>> On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang  wrote:
>>>
  i think you have to swap the phrase tables around. The decoder always
 looks at the 1st phrase-table, then backoff to the 2nd if nothing is found


 On 22/07/2015 16:59, Saumitra Yadav wrote:

   Sir/Ma'am,
  I'm trying to use multiple phrase tables for translation in Moses
 decoder, with preference to 1st phrase-table, but was getting a
 segmentation fault while loading 1st phrase table, so just switched the
 positions of phrase-tables in moses configuration file and it was working ,
 now the table i want to give preference is 2nd in list , can i use

 [decoding-graph-backoff]
  1
  3
  in configuration file for backoff so that moses uses 2nd table and
 uses 1st table only for words it couldn't find in 2nd phrase-table?

 Regards,
  Saumitra Yadav
  M.Tech.
  Department Of Computer Science And Technology
  Goa University



  ___
 Moses-support mailing 
 listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support


 --
 Hieu Hoang
 Researcher
 New York University, Abu Dhabihttp://www.hoang.co.uk/hieu


>>>
>>> --
>>> Hieu Hoang
>>> Researcher
>>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>>
>>>
>>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] decoding-graph-backoff

[Hieu Hoang]

since you have 3 phrase-tables, you may have to have 3 entries in the
[decoding-graph-backoff] section, eg
  [decoding-graph-backoff]
  0
  3
  3


Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 25 July 2015 at 20:23, Saumitra Yadav  wrote:

> Sir,
> Please find attached , moses.ini file i used and command used
> was ~/Decoder/mosesdecoder/bin/moses -f moses.ini
>
> Regards,
> Saumitra Yadav
> M.Tech.
> Department Of Computer Science And Technology
> Goa University
>
>
> On Sat, Jul 25, 2015 at 9:21 PM, Hieu Hoang  wrote:
>
>>  can you please send me the moses.ini file that you used, that cause the
>> segfault. And send me the exact command you typed
>>
>>
>> On 24/07/2015 14:40, Saumitra Yadav wrote:
>>
>> But sir when i did that there was * segmentation fault* while loading
>> first phrase-table, one walk around i got was giving phrase-table
>> uncompressed to decoder.
>>
>>   Regards,
>>  Saumitra Yadav
>>  M.Tech.
>>  Department Of Computer Science And Technology
>>  Goa University
>>
>>
>> On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang  wrote:
>>
>>>  i think you have to swap the phrase tables around. The decoder always
>>> looks at the 1st phrase-table, then backoff to the 2nd if nothing is found
>>>
>>>
>>> On 22/07/2015 16:59, Saumitra Yadav wrote:
>>>
>>>   Sir/Ma'am,
>>>  I'm trying to use multiple phrase tables for translation in Moses
>>> decoder, with preference to 1st phrase-table, but was getting a
>>> segmentation fault while loading 1st phrase table, so just switched the
>>> positions of phrase-tables in moses configuration file and it was working ,
>>> now the table i want to give preference is 2nd in list , can i use
>>>
>>> [decoding-graph-backoff]
>>>  1
>>>  3
>>>  in configuration file for backoff so that moses uses 2nd table and
>>> uses 1st table only for words it couldn't find in 2nd phrase-table?
>>>
>>> Regards,
>>>  Saumitra Yadav
>>>  M.Tech.
>>>  Department Of Computer Science And Technology
>>>  Goa University
>>>
>>>
>>>
>>>  ___
>>> Moses-support mailing 
>>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>> --
>>> Hieu Hoang
>>> Researcher
>>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>>
>>>
>>
>> --
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>
>>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] decoding-graph-backoff

[Saumitra Yadav]

Sir,
Please find attached , moses.ini file i used and command used
was ~/Decoder/mosesdecoder/bin/moses -f moses.ini

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University


On Sat, Jul 25, 2015 at 9:21 PM, Hieu Hoang  wrote:

>  can you please send me the moses.ini file that you used, that cause the
> segfault. And send me the exact command you typed
>
>
> On 24/07/2015 14:40, Saumitra Yadav wrote:
>
> But sir when i did that there was * segmentation fault* while loading
> first phrase-table, one walk around i got was giving phrase-table
> uncompressed to decoder.
>
>   Regards,
>  Saumitra Yadav
>  M.Tech.
>  Department Of Computer Science And Technology
>  Goa University
>
>
> On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang  wrote:
>
>>  i think you have to swap the phrase tables around. The decoder always
>> looks at the 1st phrase-table, then backoff to the 2nd if nothing is found
>>
>>
>> On 22/07/2015 16:59, Saumitra Yadav wrote:
>>
>>   Sir/Ma'am,
>>  I'm trying to use multiple phrase tables for translation in Moses
>> decoder, with preference to 1st phrase-table, but was getting a
>> segmentation fault while loading 1st phrase table, so just switched the
>> positions of phrase-tables in moses configuration file and it was working ,
>> now the table i want to give preference is 2nd in list , can i use
>>
>> [decoding-graph-backoff]
>>  1
>>  3
>>  in configuration file for backoff so that moses uses 2nd table and uses
>> 1st table only for words it couldn't find in 2nd phrase-table?
>>
>> Regards,
>>  Saumitra Yadav
>>  M.Tech.
>>  Department Of Computer Science And Technology
>>  Goa University
>>
>>
>>
>>  ___
>> Moses-support mailing 
>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> --
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>
>>
>
> --
> Hieu Hoang
> Researcher
> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>
>


moses.ini
Description: Binary data
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] decoding-graph-backoff

[Hieu Hoang]

can you please send me the moses.ini file that you used, that cause the 
segfault. And send me the exact command you typed


On 24/07/2015 14:40, Saumitra Yadav wrote:
But sir when i did that there was *segmentation fault* while loading 
first phrase-table, one walk around i got was giving phrase-table 
uncompressed to decoder.


Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University


On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang > wrote:


i think you have to swap the phrase tables around. The decoder
always looks at the 1st phrase-table, then backoff to the 2nd if
nothing is found


On 22/07/2015 16:59, Saumitra Yadav wrote:

Sir/Ma'am,
I'm trying to use multiple phrase tables for translation in Moses
decoder, with preference to 1st phrase-table, but was getting a
segmentation fault while loading 1st phrase table, so just
switched the positions of phrase-tables in moses configuration
file and it was working , now the table i want to give preference
is 2nd in list , can i use

[decoding-graph-backoff]
 1
 3
in configuration file for backoff so that moses uses 2nd table
and uses 1st table only for words it couldn't find in 2nd
phrase-table?

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University



___
Moses-support mailing list
Moses-support@mit.edu 
http://mailman.mit.edu/mailman/listinfo/moses-support


-- 
Hieu Hoang

Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu




--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] decoding-graph-backoff

[Saumitra Yadav]

But sir when i did that there was * segmentation fault* while loading first
phrase-table, one walk around i got was giving phrase-table uncompressed to
decoder.

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University


On Thu, Jul 23, 2015 at 8:06 PM, Hieu Hoang  wrote:

>  i think you have to swap the phrase tables around. The decoder always
> looks at the 1st phrase-table, then backoff to the 2nd if nothing is found
>
>
> On 22/07/2015 16:59, Saumitra Yadav wrote:
>
>  Sir/Ma'am,
>  I'm trying to use multiple phrase tables for translation in Moses
> decoder, with preference to 1st phrase-table, but was getting a
> segmentation fault while loading 1st phrase table, so just switched the
> positions of phrase-tables in moses configuration file and it was working ,
> now the table i want to give preference is 2nd in list , can i use
>
> [decoding-graph-backoff]
>  1
>  3
>  in configuration file for backoff so that moses uses 2nd table and uses
> 1st table only for words it couldn't find in 2nd phrase-table?
>
> Regards,
>  Saumitra Yadav
>  M.Tech.
>  Department Of Computer Science And Technology
>  Goa University
>
>
>
> ___
> Moses-support mailing 
> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> --
> Hieu Hoang
> Researcher
> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] decoding-graph-backoff

[Hieu Hoang]

i think you have to swap the phrase tables around. The decoder always 
looks at the 1st phrase-table, then backoff to the 2nd if nothing is found


On 22/07/2015 16:59, Saumitra Yadav wrote:

Sir/Ma'am,
I'm trying to use multiple phrase tables for translation in Moses 
decoder, with preference to 1st phrase-table, but was getting a 
segmentation fault while loading 1st phrase table, so just switched 
the positions of phrase-tables in moses configuration file and it was 
working , now the table i want to give preference is 2nd in list , can 
i use


[decoding-graph-backoff]
 1
 3
in configuration file for backoff so that moses uses 2nd table and 
uses 1st table only for words it couldn't find in 2nd phrase-table?


Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University



___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] decoding-graph-backoff

[Saumitra Yadav]

Sir/Ma'am,
I'm trying to use multiple phrase tables for translation in Moses decoder,
with preference to 1st phrase-table, but was getting a segmentation fault
while loading 1st phrase table, so just switched the positions of
phrase-tables in moses configuration file and it was working , now the
table i want to give preference is 2nd in list , can i use

[decoding-graph-backoff]
 1
 3
in configuration file for backoff so that moses uses 2nd table and uses 1st
table only for words it couldn't find in 2nd phrase-table?

Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] [decoding-graph-backoff]

[Matthias Huck]

Okay fine. So the backoff apparently works even though the code seemed
to suggest something different to me. Got it wrong I guess.

Maybe there's some other problem in my setup, for instance with factored
generation steps.

Thanks anyway for looking into this.


On Sun, 2015-04-19 at 12:52 +0400, Hieu Hoang wrote:
> i used the following simple moses.ini file. the decoder seems to be
> doing what is expected. Models, input and & output attached.
> [input-factors]
> 0
> 
> [mapping]
> 0 T 0
> 1 T 1
> 
> 
> [feature]
> UnknownWordPenalty
> PhraseDictionaryCompact name=pt input-factor=0 output-factor=0
> path=pt.compact num-features=1
> PhraseDictionaryCompact name=pt2 input-factor=0 output-factor=0
> path=pt2.compact num-features=1
> 
> [weight]
> pt= 1
> pt2= 1
> 
> [decoding-graph-backoff]
>  0
>  1
> 
> 
> 
> 
> Hieu Hoang
> Researcher
> 
> New York University, Abu Dhabi
> 
> http://www.hoang.co.uk/hieu
> 
> 
> On 17 April 2015 at 01:26, Matthias Huck  wrote:
> I think your remark in the mail from January was correct, it
> has to be
> ePos-sPos+1 > backoff
> but currently still is
> ePos-sPos+1 <= backoff
> 
> Are you able to somehow test this?
> 
> 
> On Thu, 2015-04-16 at 23:57 +0400, Hieu Hoang wrote:
> > ah yes, I thought the backoff was doing the opposite to what
> it's
> > supposed to do so I changed the comparison around. I checked
> that it
> > backed off, but i didn't run it through tuning.
> >
> > it may still be wrong, or there may be strange interaction
> with the tuning.
> >
> >
> > On 16/04/2015 22:16, Matthias Huck wrote:
> > > Well, what's that business mentioned in your mail from
> January (quoted
> > > below), with the backoff being broken, then being broken
> more, then
> > > possibly been fixed - or not?
> > >
> > >
> 
> https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
> > >
> > >
> > > I set up a system that was supposed to do backoff with
> factored
> > > generation steps, more or less in the manner of what's
> described in this
> > > paper: Interpolated Backoff for Factored Translation
> Models, Philipp
> > > Koehn and Barry Haddow, AMTA 2012.
> > >
> > > MIRA tunes all the weights of the backoff models to 0.
> With exactly the
> > > same configuration, this did not happen last year
> (February 2014). Maybe
> > > the [decoding-graph-backoff] setting didn't have any
> effect prior to
> > > some of your code modifications, and the models were
> actually competing
> > > in older setups? Or it's buggy now. I can't really tell.
> > >
> > > I can show you the two setups if you want.
> > >
> > >
> > >
> > > On Thu, 2015-04-16 at 21:34 +0400, Hieu Hoang wrote:
> > >> Didn't know it has changed. How should it behave and how
> does it
> > >> actually behave?
> > >>
> > >> On 16 Apr 2015 21:04, "Matthias Huck"
>  wrote:
> > >>  Hi Hieu,
> > >>
> > >>  It seems that [decoding-graph-backoff] doesn't
> quite behave
> > >>  like last
> > >>  year any more. Can you briefly explain how its
> behaviour has
> > >>  changed,
> > >>  i.e. what it did before and what it does now?
> Can you please
> > >>  also let me
> > >>  know whether there's a way to reproduce the old
> behaviour via
> > >>  configuration options?
> > >>
> > >>  Cheers,
> > >>  Matthias
> > >>
> > >>
> > >>
> > >>  On Fri, 2015-01-09 at 15:20 +, Hieu Hoang
> wrote:
> > >>  > >From the git history, I think it was slightly
> broken, then
> > >>  I broke it even
> > >>  > more in May 2014.
> > >>  >
> > >>  >
> > >>
> 
> https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
> > >>  >
> > >>  > It was
> > >>  >endPos-startPos+1 >= backoff
> > >>  > then
> > >>  >   endPos-startPos+1 <= backoff
> > >>  > I think it should be
> > >>  >   endPos-startPos+1 > backoff
> > >>  >
> > >>  > I'll change it if it's ok with everyone
> > >>  >
> > >>  >
> > >>  > On 9 January 2015 at 15:11, Marcin
> Junczys-Dowmunt
>

Re: [Moses-support] [decoding-graph-backoff]

[Matthias Huck]

I think your remark in the mail from January was correct, it has to be
ePos-sPos+1 > backoff
but currently still is 
ePos-sPos+1 <= backoff

Are you able to somehow test this?


On Thu, 2015-04-16 at 23:57 +0400, Hieu Hoang wrote:
> ah yes, I thought the backoff was doing the opposite to what it's 
> supposed to do so I changed the comparison around. I checked that it 
> backed off, but i didn't run it through tuning.
> 
> it may still be wrong, or there may be strange interaction with the tuning.
> 
> 
> On 16/04/2015 22:16, Matthias Huck wrote:
> > Well, what's that business mentioned in your mail from January (quoted
> > below), with the backoff being broken, then being broken more, then
> > possibly been fixed - or not?
> >
> > https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
> >
> >
> > I set up a system that was supposed to do backoff with factored
> > generation steps, more or less in the manner of what's described in this
> > paper: Interpolated Backoff for Factored Translation Models, Philipp
> > Koehn and Barry Haddow, AMTA 2012.
> >
> > MIRA tunes all the weights of the backoff models to 0. With exactly the
> > same configuration, this did not happen last year (February 2014). Maybe
> > the [decoding-graph-backoff] setting didn't have any effect prior to
> > some of your code modifications, and the models were actually competing
> > in older setups? Or it's buggy now. I can't really tell.
> >
> > I can show you the two setups if you want.
> >
> >
> >
> > On Thu, 2015-04-16 at 21:34 +0400, Hieu Hoang wrote:
> >> Didn't know it has changed. How should it behave and how does it
> >> actually behave?
> >>
> >> On 16 Apr 2015 21:04, "Matthias Huck"  wrote:
> >>  Hi Hieu,
> >>  
> >>  It seems that [decoding-graph-backoff] doesn't quite behave
> >>  like last
> >>  year any more. Can you briefly explain how its behaviour has
> >>  changed,
> >>  i.e. what it did before and what it does now? Can you please
> >>  also let me
> >>  know whether there's a way to reproduce the old behaviour via
> >>  configuration options?
> >>  
> >>  Cheers,
> >>  Matthias
> >>  
> >>  
> >>  
> >>  On Fri, 2015-01-09 at 15:20 +, Hieu Hoang wrote:
> >>  > >From the git history, I think it was slightly broken, then
> >>  I broke it even
> >>  > more in May 2014.
> >>  >
> >>  >
> >>  
> >> https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
> >>  >
> >>  > It was
> >>  >endPos-startPos+1 >= backoff
> >>  > then
> >>  >   endPos-startPos+1 <= backoff
> >>  > I think it should be
> >>  >   endPos-startPos+1 > backoff
> >>  >
> >>  > I'll change it if it's ok with everyone
> >>  >
> >>  >
> >>  > On 9 January 2015 at 15:11, Marcin Junczys-Dowmunt
> >>  
> >>  > wrote:
> >>  >
> >>  > >  Hm, we have been using it at WIPO, but I have to admit I
> >>  never checked
> >>  > > it _actually_ does anything useful. We sorta believe it
> >>  does.
> >>  > >
> >>  > > W dniu 09.01.2015 o 16:08, Hieu Hoang pisze:
> >>  > >
> >>  > >   Hi All
> >>  > >
> >>  > >  Does anyone use this functionality in Moses when you have
> >>  multiple
> >>  > > phrase-tables?
> >>  > >
> >>  > >  From the code, it doesn't look like it works as described
> >>  in
> >>  > >   http://www.statmt.org/moses/?n=Moses.AdvancedFeatures
> >>  > >
> >>  > >  Maybe I'm missing something
> >>  > >
> >>  > > --
> >>  > > Hieu Hoang
> >>  > > Research Associate
> >>  > > University of Edinburgh
> >>  > > http://www.hoang.co.uk/hieu
> >>  > >
> >>  > >
> >>  > >
> >>  > > ___
> >>  > > Moses-support mailing
> >>  
> >> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
> >>  > >
> >>  > >
> >>  > >
> >>  > > ___
> >>  > > Moses-support mailing list
> >>  > > Moses-support@mit.edu
> >>  > > http://mailman.mit.edu/mailman/listinfo/moses-support
> >>  > >
> >>  > >
> >>  >
> >>  >
> >>  > ___
> >>  > Moses-support mailing list
> >>  > Moses-support@mit.edu
> >>  > http://mailman.mit.edu/mailman/listinfo/moses-support
> >>  > The University of Edinburgh is a charitable body, registered
> >>  in
> >>  > Scotland, with registration number SC0

Re: [Moses-support] [decoding-graph-backoff]

[Hieu Hoang]

ah yes, I thought the backoff was doing the opposite to what it's 
supposed to do so I changed the comparison around. I checked that it 
backed off, but i didn't run it through tuning.

it may still be wrong, or there may be strange interaction with the tuning.


On 16/04/2015 22:16, Matthias Huck wrote:
> Well, what's that business mentioned in your mail from January (quoted
> below), with the backoff being broken, then being broken more, then
> possibly been fixed - or not?
>
> https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
>
>
> I set up a system that was supposed to do backoff with factored
> generation steps, more or less in the manner of what's described in this
> paper: Interpolated Backoff for Factored Translation Models, Philipp
> Koehn and Barry Haddow, AMTA 2012.
>
> MIRA tunes all the weights of the backoff models to 0. With exactly the
> same configuration, this did not happen last year (February 2014). Maybe
> the [decoding-graph-backoff] setting didn't have any effect prior to
> some of your code modifications, and the models were actually competing
> in older setups? Or it's buggy now. I can't really tell.
>
> I can show you the two setups if you want.
>
>
>
> On Thu, 2015-04-16 at 21:34 +0400, Hieu Hoang wrote:
>> Didn't know it has changed. How should it behave and how does it
>> actually behave?
>>
>> On 16 Apr 2015 21:04, "Matthias Huck"  wrote:
>>  Hi Hieu,
>>  
>>  It seems that [decoding-graph-backoff] doesn't quite behave
>>  like last
>>  year any more. Can you briefly explain how its behaviour has
>>  changed,
>>  i.e. what it did before and what it does now? Can you please
>>  also let me
>>  know whether there's a way to reproduce the old behaviour via
>>  configuration options?
>>  
>>  Cheers,
>>  Matthias
>>  
>>  
>>  
>>  On Fri, 2015-01-09 at 15:20 +, Hieu Hoang wrote:
>>  > >From the git history, I think it was slightly broken, then
>>  I broke it even
>>  > more in May 2014.
>>  >
>>  >
>>  
>> https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
>>  >
>>  > It was
>>  >endPos-startPos+1 >= backoff
>>  > then
>>  >   endPos-startPos+1 <= backoff
>>  > I think it should be
>>  >   endPos-startPos+1 > backoff
>>  >
>>  > I'll change it if it's ok with everyone
>>  >
>>  >
>>  > On 9 January 2015 at 15:11, Marcin Junczys-Dowmunt
>>  
>>  > wrote:
>>  >
>>  > >  Hm, we have been using it at WIPO, but I have to admit I
>>  never checked
>>  > > it _actually_ does anything useful. We sorta believe it
>>  does.
>>  > >
>>  > > W dniu 09.01.2015 o 16:08, Hieu Hoang pisze:
>>  > >
>>  > >   Hi All
>>  > >
>>  > >  Does anyone use this functionality in Moses when you have
>>  multiple
>>  > > phrase-tables?
>>  > >
>>  > >  From the code, it doesn't look like it works as described
>>  in
>>  > >   http://www.statmt.org/moses/?n=Moses.AdvancedFeatures
>>  > >
>>  > >  Maybe I'm missing something
>>  > >
>>  > > --
>>  > > Hieu Hoang
>>  > > Research Associate
>>  > > University of Edinburgh
>>  > > http://www.hoang.co.uk/hieu
>>  > >
>>  > >
>>  > >
>>  > > ___
>>  > > Moses-support mailing
>>  
>> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>  > >
>>  > >
>>  > >
>>  > > ___
>>  > > Moses-support mailing list
>>  > > Moses-support@mit.edu
>>  > > http://mailman.mit.edu/mailman/listinfo/moses-support
>>  > >
>>  > >
>>  >
>>  >
>>  > ___
>>  > Moses-support mailing list
>>  > Moses-support@mit.edu
>>  > http://mailman.mit.edu/mailman/listinfo/moses-support
>>  > The University of Edinburgh is a charitable body, registered
>>  in
>>  > Scotland, with registration number SC005336.
>>  
>>  
>>  
>>  --
>>  The University of Edinburgh is a charitable body, registered
>>  in
>>  Scotland, with registration number SC005336.
>>  
>>  
>
>

-- 
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] [decoding-graph-backoff]

[Matthias Huck]

Well, what's that business mentioned in your mail from January (quoted
below), with the backoff being broken, then being broken more, then
possibly been fixed - or not?

https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19


I set up a system that was supposed to do backoff with factored
generation steps, more or less in the manner of what's described in this
paper: Interpolated Backoff for Factored Translation Models, Philipp
Koehn and Barry Haddow, AMTA 2012.

MIRA tunes all the weights of the backoff models to 0. With exactly the
same configuration, this did not happen last year (February 2014). Maybe
the [decoding-graph-backoff] setting didn't have any effect prior to
some of your code modifications, and the models were actually competing
in older setups? Or it's buggy now. I can't really tell.

I can show you the two setups if you want.



On Thu, 2015-04-16 at 21:34 +0400, Hieu Hoang wrote:
> Didn't know it has changed. How should it behave and how does it
> actually behave?
> 
> On 16 Apr 2015 21:04, "Matthias Huck"  wrote:
> Hi Hieu,
> 
> It seems that [decoding-graph-backoff] doesn't quite behave
> like last
> year any more. Can you briefly explain how its behaviour has
> changed,
> i.e. what it did before and what it does now? Can you please
> also let me
> know whether there's a way to reproduce the old behaviour via
> configuration options?
> 
> Cheers,
> Matthias
> 
> 
> 
> On Fri, 2015-01-09 at 15:20 +, Hieu Hoang wrote:
> > >From the git history, I think it was slightly broken, then
> I broke it even
> > more in May 2014.
> >
> >
> 
> https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
> >
> > It was
> >endPos-startPos+1 >= backoff
> > then
> >   endPos-startPos+1 <= backoff
> > I think it should be
> >   endPos-startPos+1 > backoff
> >
> > I'll change it if it's ok with everyone
> >
> >
> > On 9 January 2015 at 15:11, Marcin Junczys-Dowmunt
> 
> > wrote:
> >
> > >  Hm, we have been using it at WIPO, but I have to admit I
> never checked
> > > it _actually_ does anything useful. We sorta believe it
> does.
> > >
> > > W dniu 09.01.2015 o 16:08, Hieu Hoang pisze:
> > >
> > >   Hi All
> > >
> > >  Does anyone use this functionality in Moses when you have
> multiple
> > > phrase-tables?
> > >
> > >  From the code, it doesn't look like it works as described
> in
> > >   http://www.statmt.org/moses/?n=Moses.AdvancedFeatures
> > >
> > >  Maybe I'm missing something
> > >
> > > --
> > > Hieu Hoang
> > > Research Associate
> > > University of Edinburgh
> > > http://www.hoang.co.uk/hieu
> > >
> > >
> > >
> > > ___
> > > Moses-support mailing
> 
> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
> > >
> > >
> > >
> > > ___
> > > Moses-support mailing list
> > > Moses-support@mit.edu
> > > http://mailman.mit.edu/mailman/listinfo/moses-support
> > >
> > >
> >
> >
> > ___
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> > The University of Edinburgh is a charitable body, registered
> in
> > Scotland, with registration number SC005336.
> 
> 
> 
> --
> The University of Edinburgh is a charitable body, registered
> in
> Scotland, with registration number SC005336.
> 
> 



-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] [decoding-graph-backoff]

[Hieu Hoang]

Didn't know it has changed. How should it behave and how does it actually
behave?
On 16 Apr 2015 21:04, "Matthias Huck"  wrote:

> Hi Hieu,
>
> It seems that [decoding-graph-backoff] doesn't quite behave like last
> year any more. Can you briefly explain how its behaviour has changed,
> i.e. what it did before and what it does now? Can you please also let me
> know whether there's a way to reproduce the old behaviour via
> configuration options?
>
> Cheers,
> Matthias
>
>
>
> On Fri, 2015-01-09 at 15:20 +, Hieu Hoang wrote:
> > >From the git history, I think it was slightly broken, then I broke it
> even
> > more in May 2014.
> >
> >
> https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
> >
> > It was
> >endPos-startPos+1 >= backoff
> > then
> >   endPos-startPos+1 <= backoff
> > I think it should be
> >   endPos-startPos+1 > backoff
> >
> > I'll change it if it's ok with everyone
> >
> >
> > On 9 January 2015 at 15:11, Marcin Junczys-Dowmunt 
> > wrote:
> >
> > >  Hm, we have been using it at WIPO, but I have to admit I never checked
> > > it _actually_ does anything useful. We sorta believe it does.
> > >
> > > W dniu 09.01.2015 o 16:08, Hieu Hoang pisze:
> > >
> > >   Hi All
> > >
> > >  Does anyone use this functionality in Moses when you have multiple
> > > phrase-tables?
> > >
> > >  From the code, it doesn't look like it works as described in
> > >   http://www.statmt.org/moses/?n=Moses.AdvancedFeatures
> > >
> > >  Maybe I'm missing something
> > >
> > > --
> > > Hieu Hoang
> > > Research Associate
> > > University of Edinburgh
> > > http://www.hoang.co.uk/hieu
> > >
> > >
> > >
> > > ___
> > > Moses-support mailing listMoses-support@mit.eduhttp://
> mailman.mit.edu/mailman/listinfo/moses-support
> > >
> > >
> > >
> > > ___
> > > Moses-support mailing list
> > > Moses-support@mit.edu
> > > http://mailman.mit.edu/mailman/listinfo/moses-support
> > >
> > >
> >
> >
> > ___
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> > The University of Edinburgh is a charitable body, registered in
> > Scotland, with registration number SC005336.
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] [decoding-graph-backoff]

[Matthias Huck]

Hi Hieu,

It seems that [decoding-graph-backoff] doesn't quite behave like last
year any more. Can you briefly explain how its behaviour has changed,
i.e. what it did before and what it does now? Can you please also let me
know whether there's a way to reproduce the old behaviour via
configuration options?

Cheers,
Matthias



On Fri, 2015-01-09 at 15:20 +, Hieu Hoang wrote:
> >From the git history, I think it was slightly broken, then I broke it even
> more in May 2014.
> 
> https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19
> 
> It was
>endPos-startPos+1 >= backoff
> then
>   endPos-startPos+1 <= backoff
> I think it should be
>   endPos-startPos+1 > backoff
> 
> I'll change it if it's ok with everyone
> 
> 
> On 9 January 2015 at 15:11, Marcin Junczys-Dowmunt 
> wrote:
> 
> >  Hm, we have been using it at WIPO, but I have to admit I never checked
> > it _actually_ does anything useful. We sorta believe it does.
> >
> > W dniu 09.01.2015 o 16:08, Hieu Hoang pisze:
> >
> >   Hi All
> >
> >  Does anyone use this functionality in Moses when you have multiple
> > phrase-tables?
> >
> >  From the code, it doesn't look like it works as described in
> >   http://www.statmt.org/moses/?n=Moses.AdvancedFeatures
> >
> >  Maybe I'm missing something
> >
> > --
> > Hieu Hoang
> > Research Associate
> > University of Edinburgh
> > http://www.hoang.co.uk/hieu
> >
> >
> >
> > ___
> > Moses-support mailing 
> > listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> > ___
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> 
> 
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.



-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] [decoding-graph-backoff]

[Raj Dabre]

Hey,

I have used the multiple decoding paths with and without the backoff
procedures.
I once experimented with using a French-English phrase table as a backoff
table when translating from Creole (Mauritian) to English (I had a small
Creole-English corpus).
I found that I got a decent improvement in BLEU when I used a 4 gram
backoff setting.

In my opinion one should use the multiple Decoding Paths when confident
that the 2nd (and other) table has decent quality. (Domain adaptation
relies on this AFAIK)
But if the 2nd table fulfills the role of a supplementary table (as french
and creole have common words the French-English can provide OOV support)
then backoff is better.

Regards.

On Sat, Jan 10, 2015 at 12:11 AM, Marcin Junczys-Dowmunt  wrote:

>  Hm, we have been using it at WIPO, but I have to admit I never checked
> it _actually_ does anything useful. We sorta believe it does.
>
> W dniu 09.01.2015 o 16:08, Hieu Hoang pisze:
>
>   Hi All
>
>  Does anyone use this functionality in Moses when you have multiple
> phrase-tables?
>
>  From the code, it doesn't look like it works as described in
>   http://www.statmt.org/moses/?n=Moses.AdvancedFeatures
>
>  Maybe I'm missing something
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
> ___
> Moses-support mailing 
> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


-- 
Raj Dabre.
Research Student,
Graduate School of Informatics,
Kyoto University.
CSE MTech, IITB., 2011-2014
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] [decoding-graph-backoff]

[Hieu Hoang]

>From the git history, I think it was slightly broken, then I broke it even
more in May 2014.

https://github.com/moses-smt/mosesdecoder/commit/44fec57c535db2df73ccbb1628d8143a9c728c19

It was
   endPos-startPos+1 >= backoff
then
  endPos-startPos+1 <= backoff
I think it should be
  endPos-startPos+1 > backoff

I'll change it if it's ok with everyone


On 9 January 2015 at 15:11, Marcin Junczys-Dowmunt 
wrote:

>  Hm, we have been using it at WIPO, but I have to admit I never checked
> it _actually_ does anything useful. We sorta believe it does.
>
> W dniu 09.01.2015 o 16:08, Hieu Hoang pisze:
>
>   Hi All
>
>  Does anyone use this functionality in Moses when you have multiple
> phrase-tables?
>
>  From the code, it doesn't look like it works as described in
>   http://www.statmt.org/moses/?n=Moses.AdvancedFeatures
>
>  Maybe I'm missing something
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
> ___
> Moses-support mailing 
> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


-- 
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] [decoding-graph-backoff]

[Marcin Junczys-Dowmunt]

Hm, we have been using it at WIPO, but I have to admit I never checked 
it _actually_ does anything useful. We sorta believe it does.


W dniu 09.01.2015 o 16:08, Hieu Hoang pisze:

Hi All

Does anyone use this functionality in Moses when you have multiple 
phrase-tables?


From the code, it doesn't look like it works as described in
http://www.statmt.org/moses/?n=Moses.AdvancedFeatures

Maybe I'm missing something

--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu



___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] [decoding-graph-backoff]

[Hieu Hoang]

Hi All

Does anyone use this functionality in Moses when you have multiple
phrase-tables?

>From the code, it doesn't look like it works as described in
  http://www.statmt.org/moses/?n=Moses.AdvancedFeatures

Maybe I'm missing something

-- 
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support