ok i spoke too soon. using STDIN as input seems unreliable. using a named file and reading from it seems ok:
~/projects/randlm/randlm/bin> ./buildlm -struct BloomMap -falsepos 8 -values 8 -output-prefix model -input-type corpus -input-path ~/group/project/statmt9/josh/experiments/multi-europarl/data/lm/europarl-v3.en.gz (in the next release i'll try to fix this, or at least make sure it is documented) Miles 2009/2/19 Miles Osborne <mi...@inf.ed.ac.uk>: > that might be it. but i seem to have it working here, using a > non-gzipped version of Europarl. > > in any case, Michael: tell us if it works when the corpus is gzipped > > Miles > > 2009/2/19 Barry Haddow <bhad...@inf.ed.ac.uk>: >> Hi >> >> I've seen this error before. The short answer is that you need to use a >> gzipped version of the corpus. >> >> The reason is that randlm uses gzip to decompress/compress when you have a >> gzipped corpus, which is fine because gzip takes a -d argument for >> decompressing. If presented with a non-gzipped version of the corpus, randlm >> attempts to fake gzip with cat, which fails because cat doesn't accept -d. >> >> This has come up on the mailing list before, as far as I recall. >> >> regards >> Barry >> >> On Thursday 19 February 2009 13:53, Michael Zuckerman wrote: >>> Hi, >>> >>> We are trying to run RandLM on our files. We use the command: >>> $ ./buildlm -struct BloomMap -falsepos 8 -values 8 -output-prefix model >>> -input-type corpus < ../../europarl.lower.token.en >>> >>> And we get the following errors: >>> cat: invalid option -- d >>> Try `cat --help' for more information. >>> rm ./model.tokens >>> buildlm: RandLMStats.cpp:312: virtual bool >>> randlm::CountStats::observe(const randlm::Word*, randlm::Value, int): >>> Assertion `len > 0' failed. >>> Aborted >>> >>> Are you familiar with these errors ? Do you have an idea about how to solve >>> them ? >>> >>> Thanks, >>> Michael. >> >> -- >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
