Here is what I think about this whole issue. Wouldn’t it be much more beneficial to the whole community to fix the basic beam search issue in Moses that seems to be the root of the problem, rather than write papers investigating it? As far as I understand, the issue most likely arises from the fact that the beam width limit might cut off some hypotheses from a group with equal probability, while leaving some for further processing. The fix would include checking the beam border and including all hypotheses with the least permissible probability, regardless of the actual beam width. This might require some redesign of the data structures that hold the hypotheses (I’m writing this without actually inspecting the source code), but this cannot be an untenable task. Let’s say this takes 8–16 man-hours to code and another 8–16 to test. Any volunteers in academia?
Cheers, Ventzi ––––––– Dr. Ventsislav Zhechev Computational Linguist CMS & Language Technologies Localisation Services Autodesk Development Sàrl Neuchâtel, Switzerland http://VentsislavZhechev.eu tel: +41 32 723 9122 fax: +41 32 723 9399 _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
