[jira] [Commented] (JOSHUA-277) UnsatisfiedLinkError: no ken in java.library.path
[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137728#comment-16137728
]
Thamme Gowda commented on JOSHUA-277:
-
I (the guy who originally created this issue) had faced this problem while
trying to make Joshua run on OSX.
I can confirm that Joshua pipeline works without any hurdles on Linux. I had
previously built it on Ubuntu and a few days back I had built Joshua and
trained few models on Centos. it worked without any trouble. (I am not sure on
OSX yet)
[~jesmitty] Which is your OS?
Most likely, one of the dependencies (Kenlm) is not built or picked up by
joshua.
Hint: debug why "runs/1/tune/model/lm.kenlm" is not being created when you run
the pipeline.
> UnsatisfiedLinkError: no ken in java.library.path
> -
>
> Key: JOSHUA-277
> URL: https://issues.apache.org/jira/browse/JOSHUA-277
> Project: Joshua
> Issue Type: Bug
>Reporter: Thamme Gowda
>
> I followed this guide http://joshua.incubator.apache.org/6.0/quick-start.html
> to test the latest build.
> Assuming there few things are broken due to newer maven build system, I tried
> to fix pipeline.pl to get the quick start guide working.
> Which files from kenlm build should I add to JNI path? (I am unable to locate
> the library file in the kenlm build output)
> Here is the full log:
> {code}
> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero
> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en
> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
> [train-copy-and-filter] cached, skipping...
> [train-vocab-bn] cached, skipping...
> [train-vocab-en] cached, skipping...
> [tune-copy-and-filter] cached, skipping...
> [tune-vocab-bn] cached, skipping...
> [tune-vocab-en.0] cached, skipping...
> [tune-vocab-en.1] cached, skipping...
> [tune-vocab-en.2] cached, skipping...
> [tune-vocab-en.3] cached, skipping...
> [test-copy-and-filter] cached, skipping...
> [test-vocab-bn] cached, skipping...
> [test-vocab-en.0] cached, skipping...
> [test-vocab-en.1] cached, skipping...
> [test-vocab-en.2] cached, skipping...
> [test-vocab-en.3] cached, skipping...
> [source-numlines] cached, skipping...
> [source-numlines] retrieved cached result =>20788
> [berkeley-aligner-chunk-0] cached, skipping...
> [aligner-combine] cached, skipping...
> [pack-grammar] cached, skipping...
> [lm-sort-uniq] cached, skipping...
> [kenlm] cached, skipping...
> [compile-kenlm] cached, skipping...
> [glue-tune] cached, skipping...
> Error: Could not find or load main class
> joshua.util.encoding.EncoderConfiguration
> [tune-bundle] cached, skipping...
> [mert-1] rebuilding...
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
> [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> [CHANGED]
> dep=tune/model/grammar.packed/slice_0.source [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
> [NOT FOUND]
>
> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en
> --tunedir
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune --tuner
> mert --decoder
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command
> --decoder-config
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> --decoder-output-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest
> --decoder-log-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log
> --iterations 10 --metric 'BLEU 4 closest'
> JOB FAILED (return code 1)
> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate
> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'!
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761)
> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
> at org.apache.joshua.decoder.Decoder.(Decoder.java:122)
> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
>
[jira] [Commented] (JOSHUA-277) UnsatisfiedLinkError: no ken in java.library.path
[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137681#comment-16137681
]
Jeffrey Smith commented on JOSHUA-277:
--
There is a there a file lib/libken.s and bin/lmplz after I ran
download-deps.sh. Perhaps I should give whole output. It is below. it is in
response to the first part of the tutorial (get similar results with other
parts) - $JOSHUA/bin/pipeline.pl \
--rundir 1 \
--readme "Baseline Hiero run" \
--source es \
--target en \
--type hiero \
--corpus $FISHER/corpus/asr/fisher_train \
--tune $FISHER/corpus/asr/fisher_dev \
--test $FISHER/corpus/asr/fisher_dev2 \
--maxlen 11 \
--maxlen-tune 11 \
--maxlen-test 11 \
--tuner-iterations 1 \
--lm-order 3
_
source test1.sh
[train-copy-and-filter] cached, skipping...
[train-tokenize-es] cached, skipping...
[train-tokenize-en] cached, skipping...
[train-trim] cached, skipping...
[train-lowercase-es] cached, skipping...
[train-lowercase-en] cached, skipping...
[train-vocab-es] cached, skipping...
[train-vocab-en] cached, skipping...
[tune-copy-and-filter] cached, skipping...
[tune-tokenize-es] cached, skipping...
[tune-tokenize-en.0] cached, skipping...
[tune-tokenize-en.1] cached, skipping...
[tune-tokenize-en.2] cached, skipping...
[tune-tokenize-en.3] cached, skipping...
[tune-trim] cached, skipping...
[tune-lowercase-es] cached, skipping...
[tune-lowercase-en.0] cached, skipping...
[tune-lowercase-en.1] cached, skipping...
[tune-lowercase-en.2] cached, skipping...
[tune-lowercase-en.3] cached, skipping...
[tune-vocab-es] cached, skipping...
[tune-vocab-en.0] cached, skipping...
[tune-vocab-en.1] cached, skipping...
[tune-vocab-en.2] cached, skipping...
[tune-vocab-en.3] cached, skipping...
[test-copy-and-filter] cached, skipping...
[test-tokenize-es] cached, skipping...
[test-tokenize-en.0] cached, skipping...
[test-tokenize-en.1] cached, skipping...
[test-tokenize-en.2] cached, skipping...
[test-tokenize-en.3] cached, skipping...
[test-trim] cached, skipping...
[test-lowercase-es] cached, skipping...
[test-lowercase-en.0] cached, skipping...
[test-lowercase-en.1] cached, skipping...
[test-lowercase-en.2] cached, skipping...
[test-lowercase-en.3] cached, skipping...
[test-vocab-es] cached, skipping...
[test-vocab-en.0] cached, skipping...
[test-vocab-en.1] cached, skipping...
[test-vocab-en.2] cached, skipping...
[test-vocab-en.3] cached, skipping...
[lm-sort-uniq] cached, skipping...
[kenlm] cached, skipping...
[compile-kenlm] cached, skipping...
[glue-tune] cached, skipping...
[tune-bundle] cached, skipping...
[mert-1] rebuilding...
dep=/data/joshua-tutorial/runs/1/data/tune/corpus.es [CHANGED]
dep=/data/joshua-tutorial/runs/1/tune/joshua.config [CHANGED]
dep=tune/model/grammar.gz.packed/slice_0.source [CHANGED]
dep=/data/joshua-tutorial/runs/1/tune/joshua.config.final [NOT FOUND]
cmd=/data/joshua/scripts/training/run_tuner.py
/data/joshua-tutorial/runs/1/data/tune/corpus.es
/data/joshua-tutorial/runs/1/data/tune/corpus.en --tunedir
/data/joshua-tutorial/runs/1/tune --tuner mert --decoder
/data/joshua-tutorial/runs/1/tune/decoder_command --decoder-config
/data/joshua-tutorial/runs/1/tune/joshua.config --decoder-output-file
/data/joshua-tutorial/runs/1/tune/output.nbest --decoder-log-file
/data/joshua-tutorial/runs/1/tune/joshua.log --iterations 1 --metric 'BLEU 4
closest'
JOB FAILED (return code 1)
/data/joshua/ext/kenlm/util/file.cc:76 in int util::OpenReadOrThrow(const
char*) threw ErrnoException because `-1 == (ret = open(name, 00))'.
No such file or directory while opening
/home/ec2-user/data/joshua-tutorial/runs/1/tune/model/lm.kenlm
Traceback (most recent call last):
File "/data/joshua/scripts/training/run_tuner.py", line 554, in
main(sys.argv)
File "/data/joshua/scripts/training/run_tuner.py", line 537, in main
run_zmert(opts.tunedir, opts.source, opts.target, opts.decoder,
opts.decoder_config, opts.decoder_output_file, opts)
File "/data/joshua/scripts/training/run_tuner.py", line 418, in run_zmert
opts.metric, opts.iterations or 10)
File "/data/joshua/scripts/training/run_tuner.py", line 400, in setup_configs
for feature,weight in get_features(config):
File "/data/joshua/scripts/training/run_tuner.py", line 351, in get_features
output = check_output("%s/bin/joshua-decoder -c %s -show-weights -v 0" %
(JOSHUA, config_file), shell=True)
File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '/data/joshua/bin/joshua-decoder -c
/data/joshua-tutorial/runs/1/tune/joshua.config -show-weights -v 0' returned
non-zero exit status -6
> UnsatisfiedLinkError: no ken in java.library.path
> -
>
> Key: JOSHUA-277
> URL:
[jira] [Commented] (JOSHUA-277) UnsatisfiedLinkError: no ken in java.library.path
[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137288#comment-16137288
]
Matt Post commented on JOSHUA-277:
--
Is there a file lib/libken.so? And bin/lmplz?
> UnsatisfiedLinkError: no ken in java.library.path
> -
>
> Key: JOSHUA-277
> URL: https://issues.apache.org/jira/browse/JOSHUA-277
> Project: Joshua
> Issue Type: Bug
>Reporter: Thamme Gowda
>
> I followed this guide http://joshua.incubator.apache.org/6.0/quick-start.html
> to test the latest build.
> Assuming there few things are broken due to newer maven build system, I tried
> to fix pipeline.pl to get the quick start guide working.
> Which files from kenlm build should I add to JNI path? (I am unable to locate
> the library file in the kenlm build output)
> Here is the full log:
> {code}
> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero
> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en
> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
> [train-copy-and-filter] cached, skipping...
> [train-vocab-bn] cached, skipping...
> [train-vocab-en] cached, skipping...
> [tune-copy-and-filter] cached, skipping...
> [tune-vocab-bn] cached, skipping...
> [tune-vocab-en.0] cached, skipping...
> [tune-vocab-en.1] cached, skipping...
> [tune-vocab-en.2] cached, skipping...
> [tune-vocab-en.3] cached, skipping...
> [test-copy-and-filter] cached, skipping...
> [test-vocab-bn] cached, skipping...
> [test-vocab-en.0] cached, skipping...
> [test-vocab-en.1] cached, skipping...
> [test-vocab-en.2] cached, skipping...
> [test-vocab-en.3] cached, skipping...
> [source-numlines] cached, skipping...
> [source-numlines] retrieved cached result =>20788
> [berkeley-aligner-chunk-0] cached, skipping...
> [aligner-combine] cached, skipping...
> [pack-grammar] cached, skipping...
> [lm-sort-uniq] cached, skipping...
> [kenlm] cached, skipping...
> [compile-kenlm] cached, skipping...
> [glue-tune] cached, skipping...
> Error: Could not find or load main class
> joshua.util.encoding.EncoderConfiguration
> [tune-bundle] cached, skipping...
> [mert-1] rebuilding...
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
> [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> [CHANGED]
> dep=tune/model/grammar.packed/slice_0.source [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
> [NOT FOUND]
>
> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en
> --tunedir
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune --tuner
> mert --decoder
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command
> --decoder-config
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> --decoder-output-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest
> --decoder-log-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log
> --iterations 10 --metric 'BLEU 4 closest'
> JOB FAILED (return code 1)
> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate
> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'!
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761)
> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
> at org.apache.joshua.decoder.Decoder.(Decoder.java:122)
> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757)
> ... 3 more
> Caused by: java.lang.ExceptionInInitializerError
> at
> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.initializeLM(StateMinimizingLanguageModel.java:75)
> at
> org.apache.joshua.decoder.ff.lm.LanguageModelFF.(LanguageModelFF.java:156)
> at
>
[jira] [Commented] (JOSHUA-277) UnsatisfiedLinkError: no ken in java.library.path
[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137279#comment-16137279
]
Jeffrey Smith commented on JOSHUA-277:
--
KenLM did build. I did start on Getting Started and ran download-deps.sh
> UnsatisfiedLinkError: no ken in java.library.path
> -
>
> Key: JOSHUA-277
> URL: https://issues.apache.org/jira/browse/JOSHUA-277
> Project: Joshua
> Issue Type: Bug
>Reporter: Thamme Gowda
>
> I followed this guide http://joshua.incubator.apache.org/6.0/quick-start.html
> to test the latest build.
> Assuming there few things are broken due to newer maven build system, I tried
> to fix pipeline.pl to get the quick start guide working.
> Which files from kenlm build should I add to JNI path? (I am unable to locate
> the library file in the kenlm build output)
> Here is the full log:
> {code}
> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero
> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en
> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
> [train-copy-and-filter] cached, skipping...
> [train-vocab-bn] cached, skipping...
> [train-vocab-en] cached, skipping...
> [tune-copy-and-filter] cached, skipping...
> [tune-vocab-bn] cached, skipping...
> [tune-vocab-en.0] cached, skipping...
> [tune-vocab-en.1] cached, skipping...
> [tune-vocab-en.2] cached, skipping...
> [tune-vocab-en.3] cached, skipping...
> [test-copy-and-filter] cached, skipping...
> [test-vocab-bn] cached, skipping...
> [test-vocab-en.0] cached, skipping...
> [test-vocab-en.1] cached, skipping...
> [test-vocab-en.2] cached, skipping...
> [test-vocab-en.3] cached, skipping...
> [source-numlines] cached, skipping...
> [source-numlines] retrieved cached result =>20788
> [berkeley-aligner-chunk-0] cached, skipping...
> [aligner-combine] cached, skipping...
> [pack-grammar] cached, skipping...
> [lm-sort-uniq] cached, skipping...
> [kenlm] cached, skipping...
> [compile-kenlm] cached, skipping...
> [glue-tune] cached, skipping...
> Error: Could not find or load main class
> joshua.util.encoding.EncoderConfiguration
> [tune-bundle] cached, skipping...
> [mert-1] rebuilding...
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
> [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> [CHANGED]
> dep=tune/model/grammar.packed/slice_0.source [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
> [NOT FOUND]
>
> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en
> --tunedir
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune --tuner
> mert --decoder
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command
> --decoder-config
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> --decoder-output-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest
> --decoder-log-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log
> --iterations 10 --metric 'BLEU 4 closest'
> JOB FAILED (return code 1)
> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate
> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'!
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761)
> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
> at org.apache.joshua.decoder.Decoder.(Decoder.java:122)
> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757)
> ... 3 more
> Caused by: java.lang.ExceptionInInitializerError
> at
> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.initializeLM(StateMinimizingLanguageModel.java:75)
> at
>
[jira] [Commented] (JOSHUA-277) UnsatisfiedLinkError: no ken in java.library.path
[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137271#comment-16137271
]
Matt Post commented on JOSHUA-277:
--
KenLM is not getting built. Did you check out the Getting Started page?
download-deps.sh downloads and builds KenLM. Likely it is failing.
https://cwiki.apache.org/confluence/display/JOSHUA/Getting+Started
> UnsatisfiedLinkError: no ken in java.library.path
> -
>
> Key: JOSHUA-277
> URL: https://issues.apache.org/jira/browse/JOSHUA-277
> Project: Joshua
> Issue Type: Bug
>Reporter: Thamme Gowda
>
> I followed this guide http://joshua.incubator.apache.org/6.0/quick-start.html
> to test the latest build.
> Assuming there few things are broken due to newer maven build system, I tried
> to fix pipeline.pl to get the quick start guide working.
> Which files from kenlm build should I add to JNI path? (I am unable to locate
> the library file in the kenlm build output)
> Here is the full log:
> {code}
> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero
> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en
> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
> [train-copy-and-filter] cached, skipping...
> [train-vocab-bn] cached, skipping...
> [train-vocab-en] cached, skipping...
> [tune-copy-and-filter] cached, skipping...
> [tune-vocab-bn] cached, skipping...
> [tune-vocab-en.0] cached, skipping...
> [tune-vocab-en.1] cached, skipping...
> [tune-vocab-en.2] cached, skipping...
> [tune-vocab-en.3] cached, skipping...
> [test-copy-and-filter] cached, skipping...
> [test-vocab-bn] cached, skipping...
> [test-vocab-en.0] cached, skipping...
> [test-vocab-en.1] cached, skipping...
> [test-vocab-en.2] cached, skipping...
> [test-vocab-en.3] cached, skipping...
> [source-numlines] cached, skipping...
> [source-numlines] retrieved cached result =>20788
> [berkeley-aligner-chunk-0] cached, skipping...
> [aligner-combine] cached, skipping...
> [pack-grammar] cached, skipping...
> [lm-sort-uniq] cached, skipping...
> [kenlm] cached, skipping...
> [compile-kenlm] cached, skipping...
> [glue-tune] cached, skipping...
> Error: Could not find or load main class
> joshua.util.encoding.EncoderConfiguration
> [tune-bundle] cached, skipping...
> [mert-1] rebuilding...
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
> [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> [CHANGED]
> dep=tune/model/grammar.packed/slice_0.source [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
> [NOT FOUND]
>
> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en
> --tunedir
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune --tuner
> mert --decoder
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command
> --decoder-config
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> --decoder-output-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest
> --decoder-log-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log
> --iterations 10 --metric 'BLEU 4 closest'
> JOB FAILED (return code 1)
> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate
> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'!
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761)
> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
> at org.apache.joshua.decoder.Decoder.(Decoder.java:122)
> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757)
> ... 3 more
> Caused by: java.lang.ExceptionInInitializerError
> at
>
[jira] [Commented] (JOSHUA-277) UnsatisfiedLinkError: no ken in java.library.path
[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136879#comment-16136879
]
Jeffrey Smith commented on JOSHUA-277:
--
I seems like, in this example, the problem started at the message:
"dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
[NOT FOUND]". I have the same issue and this issue leads to the same error
messages. Why would joshua.config.final not be found?
> UnsatisfiedLinkError: no ken in java.library.path
> -
>
> Key: JOSHUA-277
> URL: https://issues.apache.org/jira/browse/JOSHUA-277
> Project: Joshua
> Issue Type: Bug
>Reporter: Thamme Gowda
>
> I followed this guide http://joshua.incubator.apache.org/6.0/quick-start.html
> to test the latest build.
> Assuming there few things are broken due to newer maven build system, I tried
> to fix pipeline.pl to get the quick start guide working.
> Which files from kenlm build should I add to JNI path? (I am unable to locate
> the library file in the kenlm build output)
> Here is the full log:
> {code}
> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero
> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en
> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
> [train-copy-and-filter] cached, skipping...
> [train-vocab-bn] cached, skipping...
> [train-vocab-en] cached, skipping...
> [tune-copy-and-filter] cached, skipping...
> [tune-vocab-bn] cached, skipping...
> [tune-vocab-en.0] cached, skipping...
> [tune-vocab-en.1] cached, skipping...
> [tune-vocab-en.2] cached, skipping...
> [tune-vocab-en.3] cached, skipping...
> [test-copy-and-filter] cached, skipping...
> [test-vocab-bn] cached, skipping...
> [test-vocab-en.0] cached, skipping...
> [test-vocab-en.1] cached, skipping...
> [test-vocab-en.2] cached, skipping...
> [test-vocab-en.3] cached, skipping...
> [source-numlines] cached, skipping...
> [source-numlines] retrieved cached result =>20788
> [berkeley-aligner-chunk-0] cached, skipping...
> [aligner-combine] cached, skipping...
> [pack-grammar] cached, skipping...
> [lm-sort-uniq] cached, skipping...
> [kenlm] cached, skipping...
> [compile-kenlm] cached, skipping...
> [glue-tune] cached, skipping...
> Error: Could not find or load main class
> joshua.util.encoding.EncoderConfiguration
> [tune-bundle] cached, skipping...
> [mert-1] rebuilding...
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
> [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> [CHANGED]
> dep=tune/model/grammar.packed/slice_0.source [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
> [NOT FOUND]
>
> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en
> --tunedir
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune --tuner
> mert --decoder
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command
> --decoder-config
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> --decoder-output-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest
> --decoder-log-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log
> --iterations 10 --metric 'BLEU 4 closest'
> JOB FAILED (return code 1)
> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate
> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'!
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761)
> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
> at org.apache.joshua.decoder.Decoder.(Decoder.java:122)
> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757)
> ... 3 more
> Caused by:
[jira] [Commented] (JOSHUA-277) UnsatisfiedLinkError: no ken in java.library.path
[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136869#comment-16136869
]
Jeffrey Smith commented on JOSHUA-277:
--
I don't think it is a good idea to close this ticket.
> UnsatisfiedLinkError: no ken in java.library.path
> -
>
> Key: JOSHUA-277
> URL: https://issues.apache.org/jira/browse/JOSHUA-277
> Project: Joshua
> Issue Type: Bug
>Reporter: Thamme Gowda
>
> I followed this guide http://joshua.incubator.apache.org/6.0/quick-start.html
> to test the latest build.
> Assuming there few things are broken due to newer maven build system, I tried
> to fix pipeline.pl to get the quick start guide working.
> Which files from kenlm build should I add to JNI path? (I am unable to locate
> the library file in the kenlm build output)
> Here is the full log:
> {code}
> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero
> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en
> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
> [train-copy-and-filter] cached, skipping...
> [train-vocab-bn] cached, skipping...
> [train-vocab-en] cached, skipping...
> [tune-copy-and-filter] cached, skipping...
> [tune-vocab-bn] cached, skipping...
> [tune-vocab-en.0] cached, skipping...
> [tune-vocab-en.1] cached, skipping...
> [tune-vocab-en.2] cached, skipping...
> [tune-vocab-en.3] cached, skipping...
> [test-copy-and-filter] cached, skipping...
> [test-vocab-bn] cached, skipping...
> [test-vocab-en.0] cached, skipping...
> [test-vocab-en.1] cached, skipping...
> [test-vocab-en.2] cached, skipping...
> [test-vocab-en.3] cached, skipping...
> [source-numlines] cached, skipping...
> [source-numlines] retrieved cached result =>20788
> [berkeley-aligner-chunk-0] cached, skipping...
> [aligner-combine] cached, skipping...
> [pack-grammar] cached, skipping...
> [lm-sort-uniq] cached, skipping...
> [kenlm] cached, skipping...
> [compile-kenlm] cached, skipping...
> [glue-tune] cached, skipping...
> Error: Could not find or load main class
> joshua.util.encoding.EncoderConfiguration
> [tune-bundle] cached, skipping...
> [mert-1] rebuilding...
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
> [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> [CHANGED]
> dep=tune/model/grammar.packed/slice_0.source [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
> [NOT FOUND]
>
> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en
> --tunedir
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune --tuner
> mert --decoder
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command
> --decoder-config
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> --decoder-output-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest
> --decoder-log-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log
> --iterations 10 --metric 'BLEU 4 closest'
> JOB FAILED (return code 1)
> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate
> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'!
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761)
> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
> at org.apache.joshua.decoder.Decoder.(Decoder.java:122)
> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757)
> ... 3 more
> Caused by: java.lang.ExceptionInInitializerError
> at
> org.apache.joshua.decoder.ff.lm.StateMinimizingLanguageModel.initializeLM(StateMinimizingLanguageModel.java:75)
> at
> org.apache.joshua.decoder.ff.lm.LanguageModelFF.(LanguageModelFF.java:156)
>
[jira] [Commented] (JOSHUA-277) UnsatisfiedLinkError: no ken in java.library.path
[
https://issues.apache.org/jira/browse/JOSHUA-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136867#comment-16136867
]
Jeffrey Smith commented on JOSHUA-277:
--
Somehow the updated tutorial was supposed to have solved this problem. I have
exactly the same problem too, even after going through the new tutorial link.
Which step was supposed to have fixed the problem?
> UnsatisfiedLinkError: no ken in java.library.path
> -
>
> Key: JOSHUA-277
> URL: https://issues.apache.org/jira/browse/JOSHUA-277
> Project: Joshua
> Issue Type: Bug
>Reporter: Thamme Gowda
>
> I followed this guide http://joshua.incubator.apache.org/6.0/quick-start.html
> to test the latest build.
> Assuming there few things are broken due to newer maven build system, I tried
> to fix pipeline.pl to get the quick start guide working.
> Which files from kenlm build should I add to JNI path? (I am unable to locate
> the library file in the kenlm build output)
> Here is the full log:
> {code}
> $JOSHUA/bin/pipeline.pl --source bn --target en --type hiero
> --no-prepare --aligner berkeley --corpus input/bn-en/tok/training.bn-en
> --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
> [train-copy-and-filter] cached, skipping...
> [train-vocab-bn] cached, skipping...
> [train-vocab-en] cached, skipping...
> [tune-copy-and-filter] cached, skipping...
> [tune-vocab-bn] cached, skipping...
> [tune-vocab-en.0] cached, skipping...
> [tune-vocab-en.1] cached, skipping...
> [tune-vocab-en.2] cached, skipping...
> [tune-vocab-en.3] cached, skipping...
> [test-copy-and-filter] cached, skipping...
> [test-vocab-bn] cached, skipping...
> [test-vocab-en.0] cached, skipping...
> [test-vocab-en.1] cached, skipping...
> [test-vocab-en.2] cached, skipping...
> [test-vocab-en.3] cached, skipping...
> [source-numlines] cached, skipping...
> [source-numlines] retrieved cached result =>20788
> [berkeley-aligner-chunk-0] cached, skipping...
> [aligner-combine] cached, skipping...
> [pack-grammar] cached, skipping...
> [lm-sort-uniq] cached, skipping...
> [kenlm] cached, skipping...
> [compile-kenlm] cached, skipping...
> [glue-tune] cached, skipping...
> Error: Could not find or load main class
> joshua.util.encoding.EncoderConfiguration
> [tune-bundle] cached, skipping...
> [mert-1] rebuilding...
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
> [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> [CHANGED]
> dep=tune/model/grammar.packed/slice_0.source [CHANGED]
>
> dep=/Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config.final
> [NOT FOUND]
>
> cmd=/Users/thammegr/work/projects/apache/incubator-joshua/scripts/training/run_tuner.py
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.bn
>
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/data/tune/corpus.en
> --tunedir
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune --tuner
> mert --decoder
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/decoder_command
> --decoder-config
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.config
> --decoder-output-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/output.nbest
> --decoder-log-file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/joshua.log
> --iterations 10 --metric 'BLEU 4 closest'
> JOB FAILED (return code 1)
> Exception in thread "main" java.lang.RuntimeException: Unable to instantiate
> feature function 'StateMinimizingLanguageModel -lm_order 5 -lm_file
> /Users/thammegr/work/projects/apache/incubator-joshua/data/bn-en/tune/model/lm.kenlm'!
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:761)
> at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
> at org.apache.joshua.decoder.Decoder.(Decoder.java:122)
> at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.joshua.decoder.Decoder.initializeFeatureFunctions(Decoder.java:757)
> ... 3 more
> Caused by: java.lang.ExceptionInInitializerError
> at
>
