Hi, There is no Joshua manual, unfortunately, just the Confluence pages.
I looked at your run and it seems that Thrax is failing. I don't know what your Hadoop configuration is like, but that is likely the problem (see thrax.log in these directories). If you setup Hadoop incorrectly, or don't have enough space, or set it up on a network share instead of local disks, all of these things can cause problems. matt > On Aug 28, 2017, at 2:46 PM, Jeffrey Smith (JIRA) <j...@apache.org> wrote: > > > [ > https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143731#comment-16143731 > ] > > Jeffrey Smith commented on JOSHUA-277: > -------------------------------------- > > Thank you for your help. Here are the two directories in question. Also, do > you know of a manual, similar to the moses manual, for joshua? I couldn't > seem to find one. > Moses does work on this same computer I am running this on. > > joshua-tutorial.tar.gz > <https://drive.google.com/a/macefusion.com/file/d/0ByzVuxx3zna-LTZmZU5EXzA5Mm8/view?usp=drive_web> > > joshua.tar.gz > <https://drive.google.com/a/macefusion.com/file/d/0ByzVuxx3zna-bFBoSXpHYXpFVTA/view?usp=drive_web> > > > > > > > -- > *Jeffrey Smith, PhD* > Chief Systems Engineer and E2 Lead > Multi Agency Collaboration Environment (MACE) > Sierra Nevada Corporation > 3076 Centreville Road, Herndon, VA 20171 > 703-464-6434 (Office) > 603-566-0124 (Cell) > jeff.sm...@macefusion.com > > >> UnsatisfiedLinkError: no ken in java.library.path >> ------------------------------------------------- >> >> Key: JOSHUA-277 >> URL: https://issues.apache.org/jira/browse/JOSHUA-277 >> Project: Joshua >> Issue Type: Bug >> Reporter: Thamme Gowda >> >> I followed this guide >> http://joshua.incubator.apache.org/6.0/quick-start.html to test the latest >> build. >> Assuming there few things are broken due to newer maven build system, I >> tried to fix pipeline.pl to get the quick start guide working. >> Which files from kenlm build should I add to JNI path? (I am unable to >> locate the library file in the kenlm build output) >> Here is the full log: >> {code} >> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero >> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en >> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en >> [train-copy-and-filter] cached, skipping... >> [train-vocab-bn] cached, skipping... >> [train-vocab-en] cached, skipping... >> [tune-copy-and-filter] cached, skipping... >> [tune-vocab-bn] cached, skipping... >> [tune-vocab-en.0] cached, skipping... >> [tune-vocab-en.1] cached, skipping... >> [tune-vocab-en.2] cached, skipping... >> [tune-vocab-en.3] cached, skipping... >> [test-copy-and-filter] cached, skipping... >> [test-vocab-bn] cached, skipping... >> [test-vocab-en.0] cached, skipping... >> [test-vocab-en.1] cached, skipping... >> [test-vocab-en.2] cached, skipping... >> [test-vocab-en.3] cached, skipping... >> [source-numlines] cached, skipping... >> [source-numlines] retrieved cached result => 20788 >> [berkeley-aligner-chunk-0] cached, skipping... >> [aligner-combine] cached, skipping... >> [pack-grammar] cached, skipping... >> [lm-sort-uniq] cached, skipping... >> [kenlm] cached, skipping... >> [compile-kenlm] cached, skipping... >> [glue-tune] cached, skipping... >> Error: Could not find or load main class >> joshua.util.encoding.EncoderConfiguration >> [tune-bundle] cached, skipping... >> [mert-1] rebuilding... >> >> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn >> [CHANGED] >> >> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config >> [CHANGED] >> dep=tune/model/grammar.packed/slice_00000.source [CHANGED] >> >> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final >> [NOT FOUND] >> >> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py >> >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn >> >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en >> --tunedir >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune >> --tuner mert --decoder >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command >> --decoder-config >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config >> --decoder-output-file >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest >> --decoder-log-file >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log >> --iterations 10 --metric 'BLEU 4 closest' >> JOB FAILED (return code 1) >> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate >> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'! >> at >> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761) >> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514) >> at org.apache.joshua.decoder.Decoder.<init>(Decoder.java:122) >> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69) >> Caused by: java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) >> at >> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757) >> ... 3 more >> Caused by: java.lang.ExceptionInInitializerError >> at >> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.initializeLM(StateMinimizingLanguageModel.java:75) >> at >> org.apache.joshua.decoder.ff.lm.LanguageModelFF.<init>(LanguageModelFF.java:156) >> at >> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.<init>(StateMinimizingLanguageModel.java:50) >> ... 8 more >> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no >> ken in java.library.path >> at org.apache.joshua.decoder.ff.lm.KenLM.<clinit>(KenLM.java:52) >> ... 11 more >> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path >> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) >> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >> at java.lang.System.loadLibrary(System.java:1122) >> at org.apache.joshua.decoder.ff.lm.KenLM.<clinit>(KenLM.java:43) >> ... 11 more >> Traceback (most recent call last): >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 553, in <module> >> main(sys.argv) >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 536, in main >> run_zmert(opts.tunedir, opts.source, opts.target, opts.decoder, >> opts.decoder_config, opts.decoder_output_file, opts) >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 417, in run_zmert >> opts.metric, opts.iterations or 10) >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 399, in setup_configs >> for feature,weight in get_features(config): >> File >> "/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py", >> line 351, in get_features >> output = check_output("%s/bin/joshua-decoder -c %s -show-weights -v 0" % >> (JOSHUA, config_file), shell=True) >> File >> "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", >> line 573, in check_output >> raise CalledProcessError(retcode, cmd, output=output) >> subprocess.CalledProcessError: Command >> '/Users/thammegr/work/projects/apache/incubator-joshua/bin/joshua-decoder -c >> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config >> -show-weights -v 0' returned non-zero exit status 1 >> {code} > > > > -- > This message was sent by Atlassian JIRA > (v6.4.14#64029)
