Hi, As an addendum:
You can try a manual workaround. Run gunzip on extract.o.sorted.gz and do lexical-reordering-score on the resulting plain text file. It might be inconvenient but would hopefully solve the issue. Cheers, Matthias On Thu, 2015-12-17 at 17:44 +0000, Matthias Huck wrote: > Hi, > > It's a problem that apparently occurs very rarely, and as Guy mentioned, > we were so far assuming that it's caused by a zlib bug. > > However, the zlib bug was (to my knowledge) fixed in zlib v1.2.8. > This seems to be the bug fix: > https://github.com/madler/zlib/commit/51370f365607fe14a6a7a1a27b3bd29d788f5e5b > > I've only encountered the issue once (and I'm training systems > frequently). When I came across it, I executed the same command with a > Moses compile on a different machine that was running an older version > of OpenSuse, rather than Ubuntu 12.04. The problem did not exist on the > old system. > > My guess is that it really is a zlib bug, but it would be worrying if > switching to zlib v1.2.8 doesn't resolve it. > > Cheers, > Matthias > > > On Thu, 2015-12-17 at 09:18 -0800, Varden Wang wrote: > > I seem to be having a very very similar issue. I have the exact same > > lib package as Guy (but I upgraded from lib 1.2.3.4). I'm using the > > commit SHA e211d752f6bc680094520482f190d0f805405c6c of the > > mosesdecoder. The funny thing is that I trained on the very same setup > > on different data sets without encountering this problem. > > > > My error: > > > > Executing: > > /usr/local/google/home/varden/MOSES/mosesdecoder/scripts/../bin/lexical-reordering-score > > /usr/local/google/home/varden/MOSES/FRENCH_moses_HELP_CONTENT_v1/train/model/extract.o.sorted.gz > > 0.5 > > /usr/local/google/home/varden/MOSES/FRENCH_moses_HELP_CONTENT_v1/train/model/reordering-table. > > --model "wbe msd wbe-msd-bidirectional-fe" > > > > Lexical Reordering Scorer > > > > scores lexical reordering models of several types (hierarchical, > > phrase-based and word-based-extraction > > > > terminate called after throwing an instance of 'util::GZException' > > > > what(): zlib encountered invalid distances set code -3 > > > > ERROR: Execution of: > > /usr/local/google/home/varden/MOSES/mosesdecoder/scripts/../bin/lexical-reordering-score > > /usr/local/google/home/varden/MOSES/FRENCH_moses_HELP_CONTENT_v1/train/model/extract.o.sorted.gz > > 0.5 > > /usr/local/google/home/varden/MOSES/FRENCH_moses_HELP_CONTENT_v1/train/model/reordering-table. > > --model "wbe msd wbe-msd-bidirectional-fe" > > > > died with signal 6, with coredump > > > > Thanks, > > > > Varden > > > > On Mon, Dec 7, 2015 at 9:01 AM, <moses-support-requ...@mit.edu> wrote: > > > Send Moses-support mailing list submissions to > > > moses-support@mit.edu > > > > > > To subscribe or unsubscribe via the World Wide Web, visit > > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > or, via email, send a message with subject or body 'help' to > > > moses-support-requ...@mit.edu > > > > > > You can reach the person managing the list at > > > moses-support-ow...@mit.edu > > > > > > When replying, please edit your Subject line so it is more specific > > > than "Re: Contents of Moses-support digest..." > > > > > > > > > Today's Topics: > > > > > > 1. Lexical reordering fails with zlib (Guy) > > > > > > > > > ---------------------------------------------------------------------- > > > > > > Message: 1 > > > Date: Mon, 7 Dec 2015 06:14:47 +0000 (UTC) > > > From: Guy <guy...@yahoo.com> > > > Subject: [Moses-support] Lexical reordering fails with zlib > > > To: moses-support@mit.edu > > > Message-ID: <loom.20151207t071423-...@post.gmane.org> > > > Content-Type: text/plain; charset=us-ascii > > > > > > Hello everyone, > > > > > > I've just recently started to work with Moses and managed to build a > > > couple > > > of models without problems... until now. > > > > > > I was training a new system and I got this error when executing > > > lexical-reordering-score: > > > > > > ../mosesdecoder/scripts/../bin/lexical-reordering-score > > > /local/scratch/train/model/extract.o.sorted.gz 0.5 > > > /local/scratch/train/model/reordering-table. --model "wbe msd > > > wbe-msd-bidirectional-fe" > > > Lexical Reordering Scorer > > > scores lexical reordering models of several types (hierarchical, > > > phrase-based and word-based-extraction > > > terminate called after throwing an instance of 'util::GZException' > > > what(): util/read_compressed.cc:163 in virtual std::size_t > > > util::{anonymous}::GZip::Read(void*, std::size_t, util::ReadCompressed&) > > > threw GZException'. > > > zlib encountered invalid distances set code -3 > > > Aborted > > > > > > I found an old post > > > (http://permalink.gmane.org/gmane.comp.nlp.moses.user/10151) saying this > > > was > > > due to an apparent bug in zlib 1.2.3.4 on Ubuntu 12.04 and that upgrading > > > to > > > zlib 1.2.8 solves the problem. However, I already have zlib 1.2.8 (but on > > > Ubuntu 14.04) and I still get this error. In case it helps, package name > > > is > > > zlib1g:amd64 version 1:1.2.8.dfsg-1ubuntu). > > > > > > It's a bit strange that I didn't stumble upon this problem when I trained > > > previous systems. > > > > > > Any ideas on what to do? > > > > > > Thank you very much, > > > Guy > > > > > > > > > > > > > > > ------------------------------ > > > > > > _______________________________________________ > > > Moses-support mailing list > > > Moses-support@mit.edu > > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > > > End of Moses-support Digest, Vol 110, Issue 16 > > > ********************************************** > > _______________________________________________ > > Moses-support mailing list > > Moses-support@mit.edu > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
