[jira] [Commented] (JOSHUA-279) Cannot build Joshua master branch

2016-07-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/JOSHUA-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373586#comment-15373586
 ] 

ASF GitHub Bot commented on JOSHUA-279:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-joshua/pull/30


> Cannot build Joshua master branch
> -
>
> Key: JOSHUA-279
> URL: https://issues.apache.org/jira/browse/JOSHUA-279
> Project: Joshua
>  Issue Type: Bug
>  Components: build, documentation, tests
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Hi Folks,
> We need to be cautious of whatever is committed to master branch... the build 
> has been broken for quite some time and there are constant Javadoc issues 
> which make the build unstable as well.
> For example, when i make an attempt to build master branch we have failing 
> tests
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
> ...
> ---
>  T E S T S
> ---
> Running TestSuite
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> %
> %
> %
> %
> %
> %
> %
> %
> %
> Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec 
> <<< FAILURE! - in TestSuite
> setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
> Time elapsed: 0.075 sec  <<< FAILURE!
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
> in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Results :
> Failed tests:
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
>   Run 1: ClassBasedLanguageModelTest.setUp:52 » ExceptionInInitializer
>   Run 2: PASS
> Tests run: 124, Failures: 1, Errors: 0, Skipped: 4
> [INFO] 
> 
> [INFO] BUILD FAILURE
> {code}
> As a workaround I thought I will try to build the project without running the 
> test suite, however now Javadoc issues prevent me from doing so!
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install 
> -DskipTests
> ...
> 1 error
> 14 warnings
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 28.144 s
> [INFO] Finished at: 2016-07-01T14:11:42-07:00
> [INFO] Final Memory: 37M/303M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-javadoc-plugin:2.8:jar (attach-javadocs) on 
> project joshua: MavenReportException: Error while creating archive:
> [ERROR] Exit code: 1 - 
> /usr/local/incubator-joshua/src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java:217:
>  warning: no @param for rule
> [ERROR] public int[] getRuleIds(final Rule rule) {
> [ERROR] ^
> [ERROR] 
>

[jira] [Commented] (JOSHUA-279) Cannot build Joshua master branch

2016-07-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/JOSHUA-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372519#comment-15372519
 ] 

ASF GitHub Bot commented on JOSHUA-279:
---

GitHub user KellenSunderland opened a pull request:

https://github.com/apache/incubator-joshua/pull/30

JOSHUA-279 Skip tests that rely on KenLM when KenLM is not present.

For unit tests that rely on KenLM I'm going to try and convert them all to 
follow this pattern.  We'll initialize the unit of work in the setup, and catch 
the usual exception kenlm throws when the library is not found.  We'll then 
swallow the exception and rethrow an exception indicating to testng that the 
test should be skipped.  This way on systems that don't have the KenLM binaries 
the test will be skipped without failing, and on systems that do have KenLM the 
test will be properly run.

I've run this with 'mvn' and 'mvn clean install' and both are working for 
me with/without KenLM.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/KellenSunderland/incubator-joshua master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/30.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #30


commit 7326befe556f8ab0a744dd2a7718e9af097bfbf4
Author: Kellen Sunderland 
Date:   2016-07-11T16:21:54Z

JOSHUA-279 Skip tests that rely on KenLM when KenLM is not present.




> Cannot build Joshua master branch
> -
>
> Key: JOSHUA-279
> URL: https://issues.apache.org/jira/browse/JOSHUA-279
> Project: Joshua
>  Issue Type: Bug
>  Components: build, documentation, tests
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Hi Folks,
> We need to be cautious of whatever is committed to master branch... the build 
> has been broken for quite some time and there are constant Javadoc issues 
> which make the build unstable as well.
> For example, when i make an attempt to build master branch we have failing 
> tests
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
> ...
> ---
>  T E S T S
> ---
> Running TestSuite
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> %
> %
> %
> %
> %
> %
> %
> %
> %
> Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec 
> <<< FAILURE! - in TestSuite
> setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
> Time elapsed: 0.075 sec  <<< FAILURE!
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
> in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Results :
> Failed tests:
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
>   Run 1: ClassBasedLanguageModelTest.setUp:52

[jira] [Commented] (JOSHUA-279) Cannot build Joshua master branch

2016-07-11 Thread Kellen Sunderland (JIRA) 

[ 
https://issues.apache.org/jira/browse/JOSHUA-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370387#comment-15370387
 ] 

Kellen Sunderland commented on JOSHUA-279:
--

Hey Lewis, sorry about these tests failing. We should not be breaking master 
like this, I'll try to ensure this doesn't happen in the future.  I'd propose 
in the short term we ignore any tests that rely on KenLM.  

There's a few options we can look at in the long term:  

*  Figure out an acceptable way to grab the KenLM binaries as a dependency when 
our project is built.  This would mean downloading them from a reliable source, 
and ensuring the binaries match our platform.
*  Download KenLM source as a dependency and build it locally when Joshua 
builds (but don't include KenLM source in our repository).
*  I was going to look into the feasibility of mocking out the language model 
for these tests.  I'm skeptical that this will work as there's likely millions 
of calls that would need to be mocked, even in the case of a small integration 
test example.  That being said maybe we can perform some kind of simplification 
to allow the tests to be useful and still be mockable.  
*  Using a different, Java based LM for unit tests.  We could for example use 
BerkleyLM, or a very simple Java LM implementation.



> Cannot build Joshua master branch
> -
>
> Key: JOSHUA-279
> URL: https://issues.apache.org/jira/browse/JOSHUA-279
> Project: Joshua
>  Issue Type: Bug
>  Components: build, documentation, tests
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Hi Folks,
> We need to be cautious of whatever is committed to master branch... the build 
> has been broken for quite some time and there are constant Javadoc issues 
> which make the build unstable as well.
> For example, when i make an attempt to build master branch we have failing 
> tests
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
> ...
> ---
>  T E S T S
> ---
> Running TestSuite
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> %
> %
> %
> %
> %
> %
> %
> %
> %
> Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec 
> <<< FAILURE! - in TestSuite
> setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
> Time elapsed: 0.075 sec  <<< FAILURE!
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
> in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Results :
> Failed tests:
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
>   Run 1: ClassBasedLanguageModelTest.setUp:52 » ExceptionInInitializer
>   Run 2: PASS
> Tests run: 124, Failures: 1, Errors: 0, Skipped: 4
> [INFO] 
> 
> [INFO] BUILD FAILURE
> {code}
> As a workaround I thought I will try to build the project without running the 
>

[jira] [Commented] (JOSHUA-279) Cannot build Joshua master branch

2016-07-01 Thread Lewis John McGibbney (JIRA) 

[ 
https://issues.apache.org/jira/browse/JOSHUA-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359676#comment-15359676
 ] 

Lewis John McGibbney commented on JOSHUA-279:
-

commit 342312e309ec1bb9b1074688c1fbd3897783bc49
Author: Lewis John McGibbney 
Date:   Fri Jul 1 14:40:44 2016 -0700

JOSHUA-279 Cannot build Joshua master branch

The above commit fixes the Javadoc and I can now build. The test suite is still 
failing so I am still building with the -DskipTests flag

> Cannot build Joshua master branch
> -
>
> Key: JOSHUA-279
> URL: https://issues.apache.org/jira/browse/JOSHUA-279
> Project: Joshua
>  Issue Type: Bug
>  Components: build, documentation, tests
>Reporter: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Hi Folks,
> We need to be cautious of whatever is committed to master branch... the build 
> has been broken for quite some time and there are constant Javadoc issues 
> which make the build unstable as well.
> For example, when i make an attempt to build master branch we have failing 
> tests
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
> ...
> ---
>  T E S T S
> ---
> Running TestSuite
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> %
> %
> %
> %
> %
> %
> %
> %
> %
> Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec 
> <<< FAILURE! - in TestSuite
> setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
> Time elapsed: 0.075 sec  <<< FAILURE!
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
> in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Results :
> Failed tests:
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
>   Run 1: ClassBasedLanguageModelTest.setUp:52 » ExceptionInInitializer
>   Run 2: PASS
> Tests run: 124, Failures: 1, Errors: 0, Skipped: 4
> [INFO] 
> 
> [INFO] BUILD FAILURE
> {code}
> As a workaround I thought I will try to build the project without running the 
> test suite, however now Javadoc issues prevent me from doing so!
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install 
> -DskipTests
> ...
> 1 error
> 14 warnings
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 28.144 s
> [INFO] Finished at: 2016-07-01T14:11:42-07:00
> [INFO] Final Memory: 37M/303M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-javadoc-plugin:2.8:jar (attach-javadocs) on 
> project joshua: MavenReportException: Error while creating archive:
> [ERROR] Exit code: 1 - 
>