Re: [Moses-support] Pruning source syntax rule table
The tree-to-string model is faster for decoding, compared to hiero model, and much faster compared to string-to-tree. therefore, imo, you don't need to prune because pruning always causes search errors. if you mean filtering by input sentences to get rid of rules you know will never apply (ie. not pruning bad rules which may apply), then I think you can hack filter-pt to do it. I don't think no-one done it 'cos not a lot of people work on tree-to-string On 23 July 2013 23:11, Marcin Junczys-Dowmunt junc...@amu.edu.pl wrote: Hi list, If I am not mistaken, there is currently no functionality to prune tree-to-string rule tables. Do you think it makes sense to use filter-pt for hierarchical rules instead, for instance by replacing category symbols with the generic [X]. After pruning I would have to reintroduce the syntactic categories. I think in the worst case this should result in under-pruning, keeping more rules than neccessary. Any thoughts why this might be a bad idea before I start implementing this? Thanks, Marcin ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Pruning source syntax rule table
Marcin Junczys-Dowmunt junczys@... writes: Hi list, If I am not mistaken, there is currently no functionality to prune tree-to-string rule tables. Do you think it makes sense to use filter-pt for hierarchical rules instead, for instance by replacing category symbols with the generic [X]. After pruning I would have to reintroduce the syntactic categories. I think in the worst case this should result in under-pruning, keeping more rules than neccessary. Any thoughts why this might be a bad idea before I start implementing this? Thanks, Marcin filter-pt can also prune syntactic models (tree-to-string, string-to-tree and tree-to-tree) if you use the option '-h'. It's using an approximative method to get statistics, however (ignoring nonterminals). I'm currently doing some experiments to compare it to exact pruning, since I'm curious myself if the approximation costs performance. best wishes, Rico ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Pruning source syntax rule table
Ah, perfect. I somehow firmly believed this only works for the non-syntactic hierarchical models. Thanks! Best, Marcin W dniu 24.07.2013 15:19, Rico Sennrich pisze: Marcin Junczys-Dowmunt junczys@... writes: Hi list, If I am not mistaken, there is currently no functionality to prune tree-to-string rule tables. Do you think it makes sense to use filter-pt for hierarchical rules instead, for instance by replacing category symbols with the generic [X]. After pruning I would have to reintroduce the syntactic categories. I think in the worst case this should result in under-pruning, keeping more rules than neccessary. Any thoughts why this might be a bad idea before I start implementing this? Thanks, Marcin filter-pt can also prune syntactic models (tree-to-string, string-to-tree and tree-to-tree) if you use the option '-h'. It's using an approximative method to get statistics, however (ignoring nonterminals). I'm currently doing some experiments to compare it to exact pruning, since I'm curious myself if the approximation costs performance. best wishes, Rico ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
