Also, just to be nit-picky: lmplz implements Order-Interpolated Modified
Kneser-Ney smoothing -- not Improved Kneser-Ney smoothing, which is a
smoothing technique unique to irstlm -- IKN is referred to as modified
shift beta in recent versions of IRSTLM.
On Wed, Aug 21, 2013 at 3:20 AM, Marcin
Hi Jun, all:
I just released a new version of MultEval (V0.5.1) that does not give the
strange NaN's, but instead prints a warning message telling you that you're
using a single optimizer run so that no value can be calculated and that
any conclusions you draw from these numbers may be unreliable
.
Maybe that's the main problem?
Also, my data sets are not that small, the plain arpa files currently
exceed 20 GB.
Best,
Marcin
W dniu 06.11.2012 22:15, Jonathan Clark pisze:
As far as I know, exact modified Kneser-Ney smoothing (the current
state of the art) is not supported
As far as I know, exact modified Kneser-Ney smoothing (the current
state of the art) is not supported by IRSTLM. IRSTLM instead
implements modified shift-beta smoothing, which isn't quite as
effective -- especially on smaller data sets.
Cheers,
Jon
On Tue, Nov 6, 2012 at 1:08 PM, Marcin
wrote:
Jonathan Clark jhclark@... writes:
Hi all,
The Moses wiki lists the path of the phrase table pruner as
sigtest-filter, but it's now been moved to contrib/sigtest-filter.
1) Could someone change this on the wiki or could I get access to do it
myself?
2) Given
Hi all,
The Moses wiki lists the path of the phrase table pruner as
sigtest-filter, but it's now been moved to contrib/sigtest-filter.
1) Could someone change this on the wiki or could I get access to do it myself?
2) Given that the documentation has collected a bit of dust, should I
still
Hi all,
I just noticed that the moses time reporting is rather misleading.
We see lines:
Collecting options took 13.390 seconds
Search took 13.390 seconds
Translation took 13.390 seconds
However, the Search took X seconds count also includes collecting
options, which seems wrong. I have a
that
reports the total per-sentence processing time, including both
collecting options and search.
I'd just always assumed that the search time reported was that number,
I figured that to get just the search time you could subtract the
Collecting Options time.
On Fri, Aug 10, 2012 at 10:13 AM, Jonathan
always assumed that the search time reported was that number,
I figured that to get just the search time you could subtract the
Collecting Options time.
On Fri, Aug 10, 2012 at 10:13 AM, Jonathan Clark jhcl...@cs.cmu.edu
wrote:
Hi all,
I just noticed that the moses time reporting
, Jonathan Clark wrote:
Also, there is currently no way of relating these stats back to the actual
sentence they came from when using multiple threads as far as I can tell.
Shall I also prefix each of these stats with the line number that the
source sentence came from? This should be useful
Kenneth,
Would it be possible to add a --static flag to the build to explicitly
request static linking or fail behavior? If static linking is truly
desired, having to read lots of debug messages is a painful way of
verifying that you have a static binary.
Jon
On Mon, Aug 6, 2012 at 10:03 AM,
Personally, I would could The sees the process as a small-memory process
and won't be
tempted to kill it when it's running out of memory as a disadvantage
rather than an advantage. If the OOM killer is trying to stabilize the
system, this will potentially prevent it from doing so.
The other
If you need an easy way of calculating several metrics at once
providing only plaintext as input, I have a tool called MultEval at
https://github.com/jhclark/multeval
If you'd like to do a deeper analysis visually comparing the outputs
to the references, I recommend Michael Denkowski's Meteor
That looks like an IRSTLM-format binary file. You might need the --text
yes flag for compile-lm.
You can confirm that the file is in the right format by opening it with a
text viewer such as less. The ARPA format is defined here:
14 matches
Mail list logo
