Hi Jun
It's normal for the phrase table to contain a lot of "noise", and
generally the decoder can cope with this since it looks at various
sources of information, ie different phrase scores, language model,
reordering model. Moses also applies a cutoff when it loads translation
options, so most of them won't be considered.
You can also try pruning the phrase table using the technique of Johnson
et al
http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc17
This makes the phrase table much smaller, but doesn't usually change
translation quality much,
cheers - Barry
On 03/08/12 02:57, jun....@emc.com wrote:
Hi all,
I'm using Moses as the decoder. After the creation of translation
model, i checked with the phrase-table. There are a lot of meaningless
entries, looks l like below:
Can i remove these entries from the phrase-table? Will it have impact
on the translation quality? If so, how about i delete all the
punctuations before the creation of the translation model?
! kadov- > ?? ||| ! kadov- > Network ||| 1 0.341024 1 0.114004
2.718 ||| ||| 4 4
! kadov- > ???? ||| ! kadov- > Network parameters ||| 1 0.32747 1
0.0420637 2.718 ||| ||| 2 2
! kadov- > ????? ||| ! kadov- > Network parameters and ||| 1
0.19872 1 0.0395799 2.718 ||| ||| 2 2
! kadov- > ??????? ||| ! kadov- > Network parameters and security
||| 1 0.129761 1 0.0179069 2.718 ||| ||| 2 2
! kadov- > ???? ||| ! kadov- > Network Time ||| 1 0.225683 1
0.00977698 2.718 ||| ||| 2 2
! kadov- > ?????? ( ||| ! kadov- > Network Time Protocol ( ||| 1
0.0968096 1 0.00137715 2.718 ||| ||| 2 2
! kadov- > ?????? ||| ! kadov- > Network Time Protocol ||| 1
0.208681 1 0.00154011 2.718 ||| ||| 2 2
! kadov- > ?? LUN ||| ! kadov- > Offline LUN ||| 1 0.307783 1
0.03884 2.718 ||| ||| 1 1
! kadov- > ?? LUN ?? ||| ! kadov- > Offline LUN information ||| 1
0.289121 1 0.0252108 2.718 ||| ||| 1 1
! kadov- > ?? LUN ??? ||| ! kadov- > Offline LUN information . |||
1 0.280973 1 0.0243627 2.718 ||| ||| 1 1
! kadov- > ?? ||| ! kadov- > Offline ||| 1 0.313173 1 0.0645311
2.718 ||| ||| 2 2
! kadov- > ???? ||| ! kadov- > Offline state ||| 1 0.266374 1
0.0156563 2.718 ||| ||| 1 1
! kadov- > ?? ||| ! kadov- > Auto ||| 1 0.507327 0.2 0.0200052
2.718 ||| ||| 1 5
! " ||| ! " ||| 0.0185185 0.00821238 1 0.00102938 2.718 ||| ||| 54 1
! # % ' * + - ||| ! # % ' * + - ||| 1 0.00122135 1 0.239671 2.718
||| ||| 2 2
! # % ' * + ||| ! # % ' * + ||| 1 0.00255049 1 0.750455 2.718 |||
||| 2 2
! # % ' * ||| ! # % ' * ||| 1 0.00801901 1 0.773186 2.718 ||| ||| 2 2
! # % ' ||| ! # % ' ||| 1 0.0121474 1 0.784364 2.718 ||| ||| 2 2
! # % ||| ! # % ||| 1 0.0426557 1 0.897899 2.718 ||| ||| 2 2
! # ||| ! # ||| 1 0.0550974 1 0.908339 2.718 ||| ||| 2 2
! #$ % ^&* ||| ! # $ % ^ & * ||| 1 5.14084e-07 1 0.00855668 2.718
||| ||| 2 2
! #$ % ||| ! # $ % ||| 1 6.54489e-05 1 0.23103 2.718 ||| ||| 2 2
! #$ ||| ! # $ ||| 1 8.45389e-05 1 0.233717 2.718 ||| |
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
