what is the file size of lm dot kenlm and lm.gz? that will tell you if they built fine.
check that joshua config path to lm is valid. thrown error might be off. matt (from my phone) > Le 23 août 2017 à 15:27, Jeffrey Smith (JIRA) <j...@apache.org> a écrit : > > > [ > https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138334#comment-16138334 > ] > > Jeffrey Smith commented on JOSHUA-277: > -------------------------------------- > > PS. There is a "runs/1/tune/model/lm.kenlm". It is a soft-link to > .../joshua-tutorial/runs/1/lm.kenlm . Perhaps this is not what it is supposed > to be? > > >> UnsatisfiedLinkError: no ken in java.library.path >> ------------------------------------------------- >> >> Key: JOSHUA-277 >> URL: https://issues.apache.org/jira/browse/JOSHUA-277 >> Project: Joshua >> Issue Type: Bug >> Reporter: Thamme Gowda >> >> I followed this guide >> http://joshua.incubator.apache.org/6.0/quick-start.html to test the latest >> build. >> Assuming there few things are broken due to newer maven build system, I >> tried to fix pipeline.pl to get the quick start guide working. >> Which files from kenlm build should I add to JNI path? (I am unable to >> locate the library file in the kenlm build output) >> Here is the full log: >> {code} >> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero >> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en >> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en >> [train-copy-and-filter] cached, skipping... >> [train-vocab-bn] cached, skipping... >> [train-vocab-en] cached, skipping... >> [tune-copy-and-filter] cached, skipping... >> [tune-vocab-bn] cached, skipping... >> [tune-vocab-en.0] cached, skipping... >> [tune-vocab-en.1] cached, skipping... >> [tune-vocab-en.2] cached, skipping... >> [tune-vocab-en.3] cached, skipping... >> [test-copy-and-filter] cached, skipping... >> [test-vocab-bn] cached, skipping... >> [test-vocab-en.0] cached, skipping... >> [test-vocab-en.1] cached, skipping... >> [test-vocab-en.2] cached, skipping... >> [test-vocab-en.3] cached, skipping... >> [source-numlines] cached, skipping... >> [source-numlines] retrieved cached result => 20788 >> [berkeley-aligner-chunk-0] cached, skipping... >> [aligner-combine] cached, skipping... >> [pack-grammar] cached, skipping... >> [lm-sort-uniq] cached, skipping... >> [kenlm] cached, skipping... >> [compile-kenlm] cached, skipping... >> [glue-tune] cached, skipping... >> Error: Could not find or load main class >> joshua.util.encoding.EncoderConfiguration >> [tune-bundle] cached, skipping... >> [mert-1] rebuilding... >> >> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn >> [CHANGED] >> >> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config >> [CHANGED] >> dep=tune/model/grammar.packed/slice_00000.source [CHANGED] >> >> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final >> [NOT FOUND] >> >> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py >> >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn >> >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en >> --tunedir >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune >> --tuner mert --decoder >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command >> --decoder-config >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config >> --decoder-output-file >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest >> --decoder-log-file >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log >> --iterations 10 --metric 'BLEU 4 closest' >> JOB FAILED (return code 1) >> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate >> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'! >> at >> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761) >> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514) >> at org.apache.joshua.decoder.Decoder.<init>(Decoder.java:122) >> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69) >> Caused by: java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) >> at >> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757) >> ... 3 more >> Caused by: java.lang.ExceptionInInitializerError >> at >> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.initializeLM(StateMinimizingLanguageModel.java:75) >> at >> org.apache.joshua.decoder.ff.lm.LanguageModelFF.<init>(LanguageModelFF.java:156) >> at >> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.<init>(StateMinimizingLanguageModel.java:50) >> ... 8 more >> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no >> ken in java.library.path >> at org.apache.joshua.decoder.ff.lm.KenLM.<clinit>(KenLM.java:52) >> ... 11 more >> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path >> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) >> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >> at java.lang.System.loadLibrary(System.java:1122) >> at org.apache.joshua.decoder.ff.lm.KenLM.<clinit>(KenLM.java:43) >> ... 11 more >> Traceback (most recent call last): >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 553, in <module> >> main(sys.argv) >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 536, in main >> run_zmert(opts.tunedir, opts.source, opts.target, opts.decoder, >> opts.decoder_config, opts.decoder_output_file, opts) >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 417, in run_zmert >> opts.metric, opts.iterations or 10) >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 399, in setup_configs >> for feature,weight in get_features(config): >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 351, in get_features >> output = check_output("%s/bin/joshua-decoder -c %s -show-weights -v 0" % >> (JOSHUA, config_file), shell=True) >> File >> "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", >> line 573, in check_output >> raise CalledProcessError(retcode, cmd, output=output) >> subprocess.CalledProcessError: Command >> '/Users/thammegr/work/projects/apache/incubator-joshua/bin/joshua-decoder -c >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config >> -show-weights -v 0' returned non-zero exit status 1 >> {code} > > > > -- > This message was sent by Atlassian JIRA > (v6.4.14#64029)
