Hi
I've discovered that an apparent bug in zlib (specifically version
1.2.3.4 installed in Ubuntu 12.04) can cause the Moses training pipeline
to fail. Upgrading to the latest zlib (1.2.8) fixes the problem.
The problem appears when the extract files are being read in order to
create the reordering tables or translation tables. For the reordering
table, an error message like this
terminate called after throwing an instance of 'util::GZException'
what(): util/read_compressed.cc:163 in virtual std::size_t
util::{anonymous}::GZip::Read(void*, std::size_t, util::ReadCompressed&)
threw GZException'.
zlib encountered invalid distances set code -3
is printed, whilst for the translation table, the Moses scorer swallows
the zlib error but eventually fails in 'consolidate' like this:
ERROR: source phrase does not match in line 289: 'clean .' !=
'definitely recommend to have'
So far I haven't found any reports of this zlib bug, and it seems to be
quite intermittent.
cheers - Barry
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
