[jira] [Commented] (JOSHUA-287) KenLM.java catches UnsatisfiedLinkError when attempting to load libken.so (libken.dylib on OSX)
[
https://issues.apache.org/jira/browse/JOSHUA-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15420151#comment-15420151
]
lewis john mcgibbney commented on JOSHUA-287:
-
Brilliant Kellen thank you
> KenLM.java catches UnsatisfiedLinkError when attempting to load libken.so
> (libken.dylib on OSX)
> ---
>
> Key: JOSHUA-287
> URL: https://issues.apache.org/jira/browse/JOSHUA-287
> Project: Joshua
> Issue Type: Bug
> Components: core, kenlm
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Kellen Sunderland
> Fix For: 6.1
>
>
> As explained in
> http://www.mail-archive.com/dev%40joshua.incubator.apache.org/msg01189.html
> currently we have an issue, where, when checked out from master the following
> RuntimeException is thrown.
> {code}
> ---
> T E S T S
> ---
> Running TestSuite
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> {code}
> We need to fix this such that we can run static source code analysis via
> sonar and have our results available on analysis.apache.org.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (JOSHUA-287) KenLM.java catches UnsatisfiedLinkError when attempting to load libken.so (libken.dylib on OSX)
[
https://issues.apache.org/jira/browse/JOSHUA-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399809#comment-15399809
]
Hudson commented on JOSHUA-287:
---
SUCCESS: Integrated in joshua_master #84 (See
[https://builds.apache.org/job/joshua_master/84/])
JOSHUA-287 Refactored unit tests to all use TestNG, removed no longer (kellens:
rev 94e863b810f5caebdaf92f89f1fe723d6f58dc9c)
* src/test/java/org/apache/joshua/decoder/ff/lm/ArpaFileTest.java
* src/test/java/org/apache/joshua/decoder/ff/lm/LanguageModelFFTest.java
* src/test/java/org/apache/joshua/lattice/ArcTest.java
* src/main/java/org/apache/joshua/decoder/ff/lm/KenLM.java
* resources/kenlm/oilers.kenlm
*
src/main/java/org/apache/joshua/decoder/ff/lm/StateMinimizingLanguageModel.java
* src/test/java/org/apache/joshua/decoder/io/TranslationRequestTest.java
* src/test/java/org/apache/joshua/corpus/CorpusArrayTest.java
*
src/test/java/org/apache/joshua/decoder/phrase/constrained/ConstrainedPhraseDecodingTest.java
*
src/test/java/org/apache/joshua/decoder/ff/lm/class_lm/ClassBasedLanguageModelTest.java
* src/test/java/org/apache/joshua/corpus/vocab/VocabularyTest.java
*
src/test/java/org/apache/joshua/decoder/ff/lm/berkeley_lm/LMGrammarBerkeleyTest.java
* src/test/java/org/apache/joshua/decoder/TranslationsTest.java
* src/test/java/org/apache/joshua/decoder/phrase/decode/PhraseDecodingTest.java
* src/test/java/org/apache/joshua/util/FormatUtilsTest.java
* src/test/java/org/apache/joshua/corpus/SpanTest.java
* src/test/java/org/apache/joshua/decoder/ff/ArityPhrasePenaltyFFTest.java
* pom.xml
* src/test/java/org/apache/joshua/system/AlignmentMapTest.java
* src/test/java/org/apache/joshua/decoder/JoshuaDecoderTest.java
* src/test/java/org/apache/joshua/system/MultithreadedTranslationTests.java
* src/test/java/org/apache/joshua/decoder/io/TranslationRequestStreamTest.java
* src/test/java/org/apache/joshua/decoder/phrase/CoverageTest.java
*
src/test/java/org/apache/joshua/decoder/kbest_extraction/KBestExtractionTest.java
* src/test/java/org/apache/joshua/system/KenLmTest.java
* src/test/java/org/apache/joshua/corpus/VocabularyTest.java
* src/test/java/org/apache/joshua/decoder/TestConfigFileCreater.java
* src/test/java/org/apache/joshua/decoder/ArtificialGrammarAndCorpusCreater.java
* src/test/java/org/apache/joshua/decoder/DecoderThreadTest.java
* src/test/java/org/apache/joshua/util/io/KenLmTestUtil.java
* src/test/java/org/apache/joshua/system/StructuredTranslationTest.java
* src/test/java/org/apache/joshua/system/StructuredOutputTest.java
> KenLM.java catches UnsatisfiedLinkError when attempting to load libken.so
> (libken.dylib on OSX)
> ---
>
> Key: JOSHUA-287
> URL: https://issues.apache.org/jira/browse/JOSHUA-287
> Project: Joshua
> Issue Type: Bug
> Components: core, kenlm
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
> Fix For: 6.1
>
>
> As explained in
> http://www.mail-archive.com/dev%40joshua.incubator.apache.org/msg01189.html
> currently we have an issue, where, when checked out from master the following
> RuntimeException is thrown.
> {code}
> ---
> T E S T S
> ---
> Running TestSuite
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> {code}
> We need to fix this such that we can run static source code analysis via
> sonar and have our results available on analysis.apache.org.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (JOSHUA-287) KenLM.java catches UnsatisfiedLinkError when attempting to load libken.so (libken.dylib on OSX)
[
https://issues.apache.org/jira/browse/JOSHUA-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397701#comment-15397701
]
Kellen Sunderland commented on JOSHUA-287:
--
I should have addressed this in my latest PR.
https://github.com/apache/incubator-joshua/pull/33
I think the correct behaviour is to throw a RuntimeException here. There are
two environments to consider, testing, and the general case. At test time my
feeling is that we should simply skip any tests that rely on KenLM. We
shouldn't force users to download/compile KenLM just to run unit tests. In
general if we make a call into the KenLM class and the library is not on our
java.library.path it's a serious error. We should throw a descriptive
exception (now a KenLMLoadException which extends Runtime) and let the caller
deal with it.
Can you provide some details on how this breaks Sonar? Is it still broken
after the PR?
> KenLM.java catches UnsatisfiedLinkError when attempting to load libken.so
> (libken.dylib on OSX)
> ---
>
> Key: JOSHUA-287
> URL: https://issues.apache.org/jira/browse/JOSHUA-287
> Project: Joshua
> Issue Type: Bug
> Components: core, kenlm
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
> Fix For: 6.1
>
>
> As explained in
> http://www.mail-archive.com/dev%40joshua.incubator.apache.org/msg01189.html
> currently we have an issue, where, when checked out from master the following
> RuntimeException is thrown.
> {code}
> ---
> T E S T S
> ---
> Running TestSuite
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> WARN - no grammars supplied! Supplying dummy glue grammar.
> {code}
> We need to fix this such that we can run static source code analysis via
> sonar and have our results available on analysis.apache.org.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
