Re: [Moses-support] prune phrase table

[Barry Haddow]

And there's prunePhraseTable, which prunes according to weighted TM 
score (as Moses does at runtime).


Some day there will be one pruner to rule them all ...

On 02/10/15 18:39, Philipp Koehn wrote:

Hi,

there is also scripts/training/threshold-filter.perl
which filters out phrase pairs based on minimum scores
- which is not quite what you want but similar enough.

There is also the corresponding
remove-orphan-phrase-pairs-from-reordering-table.perl
which removes phrase pairs from the reordering table
that were removed from the phrase table.

-phi

On Fri, Oct 2, 2015 at 11:43 AM, Marcin Junczys-Dowmunt 
mailto:junc...@amu.edu.pl>> wrote:


You can use filter-pt from contrib/sigtestfilter without the
suffix arrays, needs SALM to compile though. When you only specifg
-n 100 it will prune according to p(t|s)

W dniu 2015-10-02 17:35, Hieu Hoang napisa?(a):


I can't remember, but is there a script that prune the pt,
keeping just the best x rules, according to p(t|s) ?

Hieu Hoang
http://www.hoang.co.uk/hieu

___
Moses-support mailing list
Moses-support@mit.edu  
http://mailman.mit.edu/mailman/listinfo/moses-support



___
Moses-support mailing list
Moses-support@mit.edu 
http://mailman.mit.edu/mailman/listinfo/moses-support




___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] prune phrase table

[Philipp Koehn]

Hi,

there is also scripts/training/threshold-filter.perl
which filters out phrase pairs based on minimum scores
- which is not quite what you want but similar enough.

There is also the corresponding
remove-orphan-phrase-pairs-from-reordering-table.perl
which removes phrase pairs from the reordering table
that were removed from the phrase table.

-phi

On Fri, Oct 2, 2015 at 11:43 AM, Marcin Junczys-Dowmunt 
wrote:

> You can use filter-pt from contrib/sigtestfilter without the suffix
> arrays, needs SALM to compile though. When you only specifg -n 100 it will
> prune according to p(t|s)
>
> W dniu 2015-10-02 17:35, Hieu Hoang napisał(a):
>
> I can't remember, but is there a script that prune the pt, keeping just
> the best x rules, according to p(t|s) ?
>
> Hieu Hoang
> http://www.hoang.co.uk/hieu
>
> ___
> Moses-support mailing 
> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Problem with building Moses on Mint 17 (Ubuntu 14)

[Kenneth Heafield]

Hi,

That's my fault.  If you run git pull and try again, does it work?

Kenneth

On 10/02/2015 03:30 PM, Helene Manis wrote:
> Good afternoon,
> 
> I followed the steps from
> http://www.statmt.org/moses/?n=Development.GetStarted
> 
> including the installation of boost. Here is the exact command that I ran:
> 
> $ ./bjam --with-boost=~/boost_1_55_0 -j4
> 
> and
> 
> $ /usr/bin/bjam --help --debug-configuration -d2 |gzip >build.log.gz
> 
> to produce the error gz.
> 
> Following the message that I got on the command line 'The build failed. 
> If you need supportsending to the mailing list...' , I am sending
> you the error that I got. I would be very grateful for any help/hint/advise.
> 
> Thank you,
> 
> Helene
> 
> 
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
> 
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] prune phrase table

[Hieu Hoang]

I can't remember, but is there a script that prune the pt, keeping just the
best x rules, according to p(t|s) ?

Hieu Hoang
http://www.hoang.co.uk/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] prune phrase table

[Marcin Junczys-Dowmunt]

 

You can use filter-pt from contrib/sigtestfilter without the suffix
arrays, needs SALM to compile though. When you only specifg -n 100 it
will prune according to p(t|s) 

W dniu 2015-10-02 17:35, Hieu Hoang napisał(a): 

> I can't remember, but is there a script that prune the pt, keeping just the 
> best x rules, according to p(t|s) ?
> 
> Hieu Hoang
> http://www.hoang.co.uk/hieu [2] 
> 
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support [1]

 

Links:
--
[1] http://mailman.mit.edu/mailman/listinfo/moses-support
[2] http://www.hoang.co.uk/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] prune phrase table

[DUGAST, LOIC]

Hello


Yes, I guess the sigtest-filter (in the contrib folder) does just that.

It also does significance-based filtering of course.


Loic


From: moses-support-boun...@mit.edu  on behalf 
of Hieu Hoang 
Sent: 02 October 2015 05:35 PM
To: moses-support
Subject: [Moses-support] prune phrase table

I can't remember, but is there a script that prune the pt, keeping just the 
best x rules, according to p(t|s) ?

Hieu Hoang
http://www.hoang.co.uk/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Problem with building Moses on Mint 17 (Ubuntu 14)

[Helene Manis]

Good afternoon,

I followed the steps from
http://www.statmt.org/moses/?n=Development.GetStarted

including the installation of boost. Here is the exact command that I ran:

$ ./bjam --with-boost=~/boost_1_55_0 -j4

and

$ /usr/bin/bjam --help --debug-configuration -d2 |gzip >build.log.gz

to produce the error gz.

Following the message that I got on the command line 'The build failed.  If
you need supportsending to the mailing list...' , I am sending you the
error that I got. I would be very grateful for any help/hint/advise.

Thank you,

Helene


build.log.gz
Description: GNU Zip compressed data
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Re: [Moses-support] Multiple Language Model for training Baseline

[Rico Sennrich]

Hi Sanjanasri,

you can edit the resulting moses.ini file manually to add your models, 
or if you want to do it via command line argument, use the following 
options of train-model.perl: '-config-add-feature-lines' and 
'-config-add-weight-lines'. Unlike the '-lm' option, you have to provide 
a string as it should appear in the moses.ini. The appropriate feature 
line for NPLM is documented here: 
http://www.statmt.org/moses/?n=FactoredTraining.BuildingLanguageModel#ntoc31


NeuralLM factor= order= path=filename

best wishes,
Rico


On 02.10.2015 06:49, Sanjanashree Palanivel wrote:

Dear all,

How to train the baseline system with multiple language 
model. For instance, -lm 0:5:/path/to/LM/mono1binary:8 I know that 8 
stands for KENLM, But how to do for nplm. Any help would be appreciated.


--
Thanks and regards,

Sanjanasri J.P


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support