[jira] [Commented] (JOSHUA-280) Existing Language packs not compatible with Joshua master

[Matt Post (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359816#comment-15359816
 ] 

Matt Post commented on JOSHUA-280:
--

The problem is with the phrase-based packed grammar. The decoder used to have 
tons of special hooks to make phrase-based decoding work. A recent fix pushed 
up enabled us to clean all of these out, but it means that we have to repack 
the grammar for phrase-based systems. I'll work on this a minute now.

> Existing Language packs not compatible with Joshua master
> -
>
> Key: JOSHUA-280
> URL: https://issues.apache.org/jira/browse/JOSHUA-280
> Project: Joshua
>  Issue Type: Bug
>  Components: language packs
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Priority: Critical
> Fix For: 6.1
>
>
> When I work with the existing Spanish --> English language pack at 
> http://cs.jhu.edu/~post/language-packs/language-pack-es-en-phrase-2015-03-06.tgz,
>  I get the following error
> {code}
> lmcgibbn@LMC-032857 
> /usr/local/Cellar/joshua/HEAD/libexec/language-pack-es-en-phrase-2015-03-06(NUTCH-2089)
>  $ ./run-joshua-server.sh
> INFO - Parameters read from configuration file: joshua.config
> INFO - tm = 'moses -owner pt -maxspan 0 -path phrase-table.packed 
> -max-source-len 5'
> INFO - defaultnonterminal = 'X'
> INFO - goalsymbol = 'GOAL'
> INFO - featurefunction = 'StateMinimizingLanguageModel -lm_type kenlm 
> -lm_order 5 -lm_file lm.kenlm'
> INFO - markoovs = 'false'
> INFO - search = 'stack'
> INFO - pop-limit: 100
> INFO - poplimit = '100'
> INFO - topn = '0'
> INFO - useuniquenbest = 'true'
> INFO - outputformat = '%s'
> INFO - includealignindex = 'false'
> INFO - featurefunction = 'OOVPenalty'
> INFO - featurefunction = 'WordPenalty'
> INFO - featurefunction = 'Distortion'
> INFO - featurefunction = 'PhrasePenalty'
> INFO - c = 'joshua.config'
> INFO - server-port: 5674
> INFO - serverport = '5674'
> INFO - Read 9 weights (0 of them dense)
> INFO - Reading vocabulary: phrase-table.packed/vocabulary
> INFO - Read 191983 entries from the vocabulary
> INFO - Reading packed config: phrase-table.packed/config
> 102030405060708090.100%
> Exception in thread "main" java.lang.RuntimeException: The grammar at 
> phrase-table.packed was packed with packer version 0, but the earliest 
> supported version is 3
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.readConfig(PackedGrammar.java:1061)
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.(PackedGrammar.java:143)
>   at 
> org.apache.joshua.decoder.phrase.PhraseTable.(PhraseTable.java:65)
>   at 
> org.apache.joshua.decoder.Decoder.initializeTranslationGrammars(Decoder.java:603)
>   at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
>   at org.apache.joshua.decoder.Decoder.(Decoder.java:126)
>   at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-279) Cannot build Joshua master branch

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359739#comment-15359739
 ] 

Lewis John McGibbney commented on JOSHUA-279:
-

I think that 
org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest should be 
improved to 
 1) Access the resource file from the JVM ClassLoader as oppose to an absolute 
or local path
 2) Perform a check to see if kenlm is available, if not then skip the test.
If we perform both of the above then this test will be more useful

> Cannot build Joshua master branch
> -
>
> Key: JOSHUA-279
> URL: https://issues.apache.org/jira/browse/JOSHUA-279
> Project: Joshua
>  Issue Type: Bug
>  Components: build, documentation, tests
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Hi Folks,
> We need to be cautious of whatever is committed to master branch... the build 
> has been broken for quite some time and there are constant Javadoc issues 
> which make the build unstable as well.
> For example, when i make an attempt to build master branch we have failing 
> tests
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
> ...
> ---
>  T E S T S
> ---
> Running TestSuite
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> %
> %
> %
> %
> %
> %
> %
> %
> %
> Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec 
> <<< FAILURE! - in TestSuite
> setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
> Time elapsed: 0.075 sec  <<< FAILURE!
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
> in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Results :
> Failed tests:
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
>   Run 1: ClassBasedLanguageModelTest.setUp:52 » ExceptionInInitializer
>   Run 2: PASS
> Tests run: 124, Failures: 1, Errors: 0, Skipped: 4
> [INFO] 
> 
> [INFO] BUILD FAILURE
> {code}
> As a workaround I thought I will try to build the project without running the 
> test suite, however now Javadoc issues prevent me from doing so!
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install 
> -DskipTests
> ...
> 1 error
> 14 warnings
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 28.144 s
> [INFO] Finished at: 2016-07-01T14:11:42-07:00
> [INFO] Final Memory: 37M/303M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-javadoc-plugin:2.8:jar (attach-javadocs) on 
> project joshua: MavenReportException: Error while creating arch

[jira] [Commented] (JOSHUA-279) Cannot build Joshua master branch

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359728#comment-15359728
 ] 

Hudson commented on JOSHUA-279:
---

FAILURE: Integrated in joshua_master #77 (See 
[https://builds.apache.org/job/joshua_master/77/])
JOSHUA-279 Cannot build Joshua master branch (lewis.mcgibbney: rev 
342312e309ec1bb9b1074688c1fbd3897783bc49)
* src/main/java/org/apache/joshua/decoder/ff/tm/AbstractGrammar.java
* .gitignore
* src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java
* src/main/java/org/apache/joshua/decoder/ff/tm/OwnerMap.java
* 
src/main/java/org/apache/joshua/decoder/ff/tm/hash_based/MemoryBasedBatchGrammar.java


> Cannot build Joshua master branch
> -
>
> Key: JOSHUA-279
> URL: https://issues.apache.org/jira/browse/JOSHUA-279
> Project: Joshua
>  Issue Type: Bug
>  Components: build, documentation, tests
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Hi Folks,
> We need to be cautious of whatever is committed to master branch... the build 
> has been broken for quite some time and there are constant Javadoc issues 
> which make the build unstable as well.
> For example, when i make an attempt to build master branch we have failing 
> tests
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
> ...
> ---
>  T E S T S
> ---
> Running TestSuite
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> %
> %
> %
> %
> %
> %
> %
> %
> %
> Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec 
> <<< FAILURE! - in TestSuite
> setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
> Time elapsed: 0.075 sec  <<< FAILURE!
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
> in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Results :
> Failed tests:
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
>   Run 1: ClassBasedLanguageModelTest.setUp:52 » ExceptionInInitializer
>   Run 2: PASS
> Tests run: 124, Failures: 1, Errors: 0, Skipped: 4
> [INFO] 
> 
> [INFO] BUILD FAILURE
> {code}
> As a workaround I thought I will try to build the project without running the 
> test suite, however now Javadoc issues prevent me from doing so!
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install 
> -DskipTests
> ...
> 1 error
> 14 warnings
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 28.144 s
> [INFO] Finished at: 2016-07-01T14:11:42-07:00
> [INFO] Final Memory: 37M/303M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org

Build failed in Jenkins: joshua_master #77

[Apache Jenkins Server]

See 

Changes:

[lewis.mcgibbney] JOSHUA-279 Cannot build Joshua master branch

--
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on H11 (docker Ubuntu ubuntu yahoo-not-h2) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/incubator-joshua.git # timeout=10
Fetching upstream changes from 
https://git-wip-us.apache.org/repos/asf/incubator-joshua.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/incubator-joshua.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 342312e309ec1bb9b1074688c1fbd3897783bc49 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 342312e309ec1bb9b1074688c1fbd3897783bc49
 > git rev-list 316785c9268f436bbd8b7d527f808945c835c1c4 # timeout=10
[joshua_master] $ /home/jenkins/tools/maven/apache-maven-3.0.4/bin/mvn clean 
deploy javadoc:aggregate
[INFO] Scanning for projects...
[INFO] 
[INFO] 
[INFO] Building Apache Joshua Machine Translation Toolkit 6.0.6-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ joshua ---
[INFO] Deleting 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.2.1:process (default) @ joshua ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ joshua ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ joshua ---
[INFO] Compiling 268 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
joshua ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 321 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
joshua ---
[INFO] Compiling 45 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ joshua ---

---
 T E S T S
---
Running TestSuite
tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 OOVPenalty=-200.000 
| -198.000
ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
ERROR - *This probably means that the KenLM library didn't compile.
ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
ERROR - *installation (it's not /opt/local/, the default), change to
ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
ERROR - *website (joshua-decoder.org).
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
%
%
%
%
%
%
%
%
%
Tests run: 137, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 2.375 sec <<< 
FAILURE! - in TestSuite
setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
Time elapsed: 0.065 sec  <<< FAILURE!
java.lang.ExceptionInInitializerError
at 
org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
in java.library.path
at 
org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
at 
org.apache.josh

Website Branding Issues

[johndament]

Dear podling,

During a recent audit of podling websites, your podling was identified as not 
including the incubating disclaimer.  This disclaimer is required on all 
podling websites, releases and announcements, to clarify that your project may 
not be in compliance with all ASF processes.

Please review the Incubator's branding guide: 
http://incubator.apache.org/guides/branding.html

The full list of observations for all projects can be found at: 
https://wiki.apache.org/incubator/BrandingAuditJune2016

If you have any questions, feel free to respond to this email or email our 
general@ mailing list.  Please note that I am not subscribed to your mailing 
list.

[jira] [Commented] (JOSHUA-280) Existing Language packs not compatible with Joshua master

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359696#comment-15359696
 ] 

Lewis John McGibbney commented on JOSHUA-280:
-

The existing chinese language pack works just fine
{code}
lmcgibbn@LMC-032857 
/usr/local/Cellar/joshua/HEAD/libexec/zh-en-hiero-pack-2016-01(NUTCH-2089) $ 
./run-joshua-server.sh
Parameters read from configuration file:
tm = 'thrax -path grammar.packed -maxspan 20 -owner pt'
tm = 'thrax -path grammar.glue -maxspan -1 -owner glue'
defaultnonterminal = 'X'
goalsymbol = 'GOAL'
featurefunction = 'LanguageModel -lm_order 5 -lm_type berkeleylm -lm_file 
lm.berkeleylm'
markoovs = 'false'
search = 'cky'
poplimit = '100'
topn = '0'
useuniquenbest = 'true'
outputformat = '%S'
includealignindex = 'false'
featurefunction = 'OOVPenalty'
featurefunction = 'WordPenalty'
Parameters overridden from the command line:
server-port: 5674
serverport = '5674'
c = 'joshua.config'
Read 10 weights (0 of them dense)
Reading vocabulary: grammar.packed/vocabulary
Read 300317 entries from the vocabulary
Reading packed config: grammar.packed/config
102030405060708090.100%
Reading encoder configuration: grammar.packed/encoding
Loaded 62685418 rules
Reading grammar from file grammar.glue...
MemoryBasedBatchGrammar: Read 4 rules with 4 distinct source sides from 
'grammar.glue'
Memory used 3447.1 MB
Grammar loading took: 39 seconds.
Stateful object with state index 0
Loading Berkeley LM from binary lm.berkeleylm
FEATURE: tm_pt (weight 0.000)
FEATURE: tm_glue (weight 0.000)
FEATURE: lm_0, order 5 (weight 0.194)
FEATURE: OOVPenalty (weight 0.015)
FEATURE: WordPenalty (weight -0.460)
Grammar sorting happening lazily on-demand.
Model loading took 42 seconds
Memory used 4355.5 MB
** TCP Server running and listening on port 5674.
{code}

> Existing Language packs not compatible with Joshua master
> -
>
> Key: JOSHUA-280
> URL: https://issues.apache.org/jira/browse/JOSHUA-280
> Project: Joshua
>  Issue Type: Bug
>  Components: language packs
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Priority: Critical
> Fix For: 6.1
>
>
> When I work with the existing Spanish --> English language pack at 
> http://cs.jhu.edu/~post/language-packs/language-pack-es-en-phrase-2015-03-06.tgz,
>  I get the following error
> {code}
> lmcgibbn@LMC-032857 
> /usr/local/Cellar/joshua/HEAD/libexec/language-pack-es-en-phrase-2015-03-06(NUTCH-2089)
>  $ ./run-joshua-server.sh
> INFO - Parameters read from configuration file: joshua.config
> INFO - tm = 'moses -owner pt -maxspan 0 -path phrase-table.packed 
> -max-source-len 5'
> INFO - defaultnonterminal = 'X'
> INFO - goalsymbol = 'GOAL'
> INFO - featurefunction = 'StateMinimizingLanguageModel -lm_type kenlm 
> -lm_order 5 -lm_file lm.kenlm'
> INFO - markoovs = 'false'
> INFO - search = 'stack'
> INFO - pop-limit: 100
> INFO - poplimit = '100'
> INFO - topn = '0'
> INFO - useuniquenbest = 'true'
> INFO - outputformat = '%s'
> INFO - includealignindex = 'false'
> INFO - featurefunction = 'OOVPenalty'
> INFO - featurefunction = 'WordPenalty'
> INFO - featurefunction = 'Distortion'
> INFO - featurefunction = 'PhrasePenalty'
> INFO - c = 'joshua.config'
> INFO - server-port: 5674
> INFO - serverport = '5674'
> INFO - Read 9 weights (0 of them dense)
> INFO - Reading vocabulary: phrase-table.packed/vocabulary
> INFO - Read 191983 entries from the vocabulary
> INFO - Reading packed config: phrase-table.packed/config
> 102030405060708090.100%
> Exception in thread "main" java.lang.RuntimeException: The grammar at 
> phrase-table.packed was packed with packer version 0, but the earliest 
> supported version is 3
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.readConfig(PackedGrammar.java:1061)
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.(PackedGrammar.java:143)
>   at 
> org.apache.joshua.decoder.phrase.PhraseTable.(PhraseTable.java:65)
>   at 
> org.apache.joshua.decoder.Decoder.initializeTranslationGrammars(Decoder.java:603)
>   at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
>   at org.apache.joshua.decoder.Decoder.(Decoder.java:126)
>   at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-280) Existing Language packs not compatible with Joshua master

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359691#comment-15359691
 ] 

Lewis John McGibbney commented on JOSHUA-280:
-

Same goes for the Arabic language pack
{code}
lmcgibbn@LMC-032857 
/usr/local/Cellar/joshua/HEAD/libexec/language-pack-ar-en-phrase-2015-03-18(NUTCH-2089)
 $ ./run-joshua-server.sh
INFO - Parameters read from configuration file: joshua.config
INFO - tm = 'moses -owner pt -maxspan 0 -max-source-len 5 -path 
phrase-table.packed'
INFO - defaultnonterminal = 'X'
INFO - goalsymbol = 'GOAL'
INFO - featurefunction = 'StateMinimizingLanguageModel -lm_type kenlm 
-lm_order 5 -lm_file gigaword.kenlm.v5'
INFO - featurefunction = 'StateMinimizingLanguageModel -lm_type kenlm 
-lm_order 5 -lm_file lm.kenlm'
INFO - markoovs = 'false'
INFO - search = 'stack'
INFO - pop-limit: 100
INFO - poplimit = '100'
INFO - topn = '0'
INFO - useuniquenbest = 'true'
INFO - outputformat = '%s'
INFO - includealignindex = 'false'
INFO - featurefunction = 'OOVPenalty'
INFO - featurefunction = 'WordPenalty'
INFO - featurefunction = 'Distortion'
INFO - featurefunction = 'PhrasePenalty'
INFO - c = 'joshua.config'
INFO - server-port: 5674
INFO - serverport = '5674'
INFO - Read 10 weights (0 of them dense)
INFO - Reading vocabulary: phrase-table.packed/vocabulary
INFO - Read 297133 entries from the vocabulary
INFO - Reading packed config: phrase-table.packed/config
102030405060708090.100%
Exception in thread "main" java.lang.RuntimeException: The grammar at 
phrase-table.packed was packed with packer version 0, but the earliest 
supported version is 3
at 
org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.readConfig(PackedGrammar.java:1061)
at 
org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.(PackedGrammar.java:143)
at 
org.apache.joshua.decoder.phrase.PhraseTable.(PhraseTable.java:65)
at 
org.apache.joshua.decoder.Decoder.initializeTranslationGrammars(Decoder.java:603)
at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
at org.apache.joshua.decoder.Decoder.(Decoder.java:126)
at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
{code}

> Existing Language packs not compatible with Joshua master
> -
>
> Key: JOSHUA-280
> URL: https://issues.apache.org/jira/browse/JOSHUA-280
> Project: Joshua
>  Issue Type: Bug
>  Components: language packs
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Priority: Critical
> Fix For: 6.1
>
>
> When I work with the existing Spanish --> English language pack at 
> http://cs.jhu.edu/~post/language-packs/language-pack-es-en-phrase-2015-03-06.tgz,
>  I get the following error
> {code}
> lmcgibbn@LMC-032857 
> /usr/local/Cellar/joshua/HEAD/libexec/language-pack-es-en-phrase-2015-03-06(NUTCH-2089)
>  $ ./run-joshua-server.sh
> INFO - Parameters read from configuration file: joshua.config
> INFO - tm = 'moses -owner pt -maxspan 0 -path phrase-table.packed 
> -max-source-len 5'
> INFO - defaultnonterminal = 'X'
> INFO - goalsymbol = 'GOAL'
> INFO - featurefunction = 'StateMinimizingLanguageModel -lm_type kenlm 
> -lm_order 5 -lm_file lm.kenlm'
> INFO - markoovs = 'false'
> INFO - search = 'stack'
> INFO - pop-limit: 100
> INFO - poplimit = '100'
> INFO - topn = '0'
> INFO - useuniquenbest = 'true'
> INFO - outputformat = '%s'
> INFO - includealignindex = 'false'
> INFO - featurefunction = 'OOVPenalty'
> INFO - featurefunction = 'WordPenalty'
> INFO - featurefunction = 'Distortion'
> INFO - featurefunction = 'PhrasePenalty'
> INFO - c = 'joshua.config'
> INFO - server-port: 5674
> INFO - serverport = '5674'
> INFO - Read 9 weights (0 of them dense)
> INFO - Reading vocabulary: phrase-table.packed/vocabulary
> INFO - Read 191983 entries from the vocabulary
> INFO - Reading packed config: phrase-table.packed/config
> 102030405060708090.100%
> Exception in thread "main" java.lang.RuntimeException: The grammar at 
> phrase-table.packed was packed with packer version 0, but the earliest 
> supported version is 3
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.readConfig(PackedGrammar.java:1061)
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.(PackedGrammar.java:143)
>   at 
> org.apache.joshua.decoder.phrase.PhraseTable.(PhraseTable.java:65)
>   at 
> org.apache.joshua.decoder.Decoder.initializeTranslationGrammars(Decoder.java:603)
>   at org.apache.joshua.decoder.De

[jira] [Updated] (JOSHUA-280) Existing Language packs not compatible with Joshua master

[Lewis John McGibbney (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JOSHUA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lewis John McGibbney updated JOSHUA-280:

Summary: Existing Language packs not compatible with Joshua master  (was: 
Existing Spanish --> English Language pack not compatible with Joshua master)

> Existing Language packs not compatible with Joshua master
> -
>
> Key: JOSHUA-280
> URL: https://issues.apache.org/jira/browse/JOSHUA-280
> Project: Joshua
>  Issue Type: Bug
>  Components: language packs
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Priority: Critical
> Fix For: 6.1
>
>
> When I work with the existing Spanish --> English language pack at 
> http://cs.jhu.edu/~post/language-packs/language-pack-es-en-phrase-2015-03-06.tgz,
>  I get the following error
> {code}
> lmcgibbn@LMC-032857 
> /usr/local/Cellar/joshua/HEAD/libexec/language-pack-es-en-phrase-2015-03-06(NUTCH-2089)
>  $ ./run-joshua-server.sh
> INFO - Parameters read from configuration file: joshua.config
> INFO - tm = 'moses -owner pt -maxspan 0 -path phrase-table.packed 
> -max-source-len 5'
> INFO - defaultnonterminal = 'X'
> INFO - goalsymbol = 'GOAL'
> INFO - featurefunction = 'StateMinimizingLanguageModel -lm_type kenlm 
> -lm_order 5 -lm_file lm.kenlm'
> INFO - markoovs = 'false'
> INFO - search = 'stack'
> INFO - pop-limit: 100
> INFO - poplimit = '100'
> INFO - topn = '0'
> INFO - useuniquenbest = 'true'
> INFO - outputformat = '%s'
> INFO - includealignindex = 'false'
> INFO - featurefunction = 'OOVPenalty'
> INFO - featurefunction = 'WordPenalty'
> INFO - featurefunction = 'Distortion'
> INFO - featurefunction = 'PhrasePenalty'
> INFO - c = 'joshua.config'
> INFO - server-port: 5674
> INFO - serverport = '5674'
> INFO - Read 9 weights (0 of them dense)
> INFO - Reading vocabulary: phrase-table.packed/vocabulary
> INFO - Read 191983 entries from the vocabulary
> INFO - Reading packed config: phrase-table.packed/config
> 102030405060708090.100%
> Exception in thread "main" java.lang.RuntimeException: The grammar at 
> phrase-table.packed was packed with packer version 0, but the earliest 
> supported version is 3
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.readConfig(PackedGrammar.java:1061)
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.(PackedGrammar.java:143)
>   at 
> org.apache.joshua.decoder.phrase.PhraseTable.(PhraseTable.java:65)
>   at 
> org.apache.joshua.decoder.Decoder.initializeTranslationGrammars(Decoder.java:603)
>   at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
>   at org.apache.joshua.decoder.Decoder.(Decoder.java:126)
>   at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (JOSHUA-280) Existing Spanish --> English Language pack not compatible with Joshua master

[Lewis John McGibbney (JIRA)]

Lewis John McGibbney created JOSHUA-280:
---

 Summary: Existing Spanish --> English Language pack not compatible 
with Joshua master
 Key: JOSHUA-280
 URL: https://issues.apache.org/jira/browse/JOSHUA-280
 Project: Joshua
  Issue Type: Bug
  Components: language packs
Affects Versions: 6.0.5
Reporter: Lewis John McGibbney
Priority: Critical
 Fix For: 6.1


When I work with the existing Spanish --> English language pack at 
http://cs.jhu.edu/~post/language-packs/language-pack-es-en-phrase-2015-03-06.tgz,
 I get the following error
{code}
lmcgibbn@LMC-032857 
/usr/local/Cellar/joshua/HEAD/libexec/language-pack-es-en-phrase-2015-03-06(NUTCH-2089)
 $ ./run-joshua-server.sh
INFO - Parameters read from configuration file: joshua.config
INFO - tm = 'moses -owner pt -maxspan 0 -path phrase-table.packed 
-max-source-len 5'
INFO - defaultnonterminal = 'X'
INFO - goalsymbol = 'GOAL'
INFO - featurefunction = 'StateMinimizingLanguageModel -lm_type kenlm 
-lm_order 5 -lm_file lm.kenlm'
INFO - markoovs = 'false'
INFO - search = 'stack'
INFO - pop-limit: 100
INFO - poplimit = '100'
INFO - topn = '0'
INFO - useuniquenbest = 'true'
INFO - outputformat = '%s'
INFO - includealignindex = 'false'
INFO - featurefunction = 'OOVPenalty'
INFO - featurefunction = 'WordPenalty'
INFO - featurefunction = 'Distortion'
INFO - featurefunction = 'PhrasePenalty'
INFO - c = 'joshua.config'
INFO - server-port: 5674
INFO - serverport = '5674'
INFO - Read 9 weights (0 of them dense)
INFO - Reading vocabulary: phrase-table.packed/vocabulary
INFO - Read 191983 entries from the vocabulary
INFO - Reading packed config: phrase-table.packed/config
102030405060708090.100%
Exception in thread "main" java.lang.RuntimeException: The grammar at 
phrase-table.packed was packed with packer version 0, but the earliest 
supported version is 3
at 
org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.readConfig(PackedGrammar.java:1061)
at 
org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.(PackedGrammar.java:143)
at 
org.apache.joshua.decoder.phrase.PhraseTable.(PhraseTable.java:65)
at 
org.apache.joshua.decoder.Decoder.initializeTranslationGrammars(Decoder.java:603)
at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
at org.apache.joshua.decoder.Decoder.(Decoder.java:126)
at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-280) Existing Spanish --> English Language pack not compatible with Joshua master

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359690#comment-15359690
 ] 

Lewis John McGibbney commented on JOSHUA-280:
-

[~post] any idea whats up here? Thanks

> Existing Spanish --> English Language pack not compatible with Joshua master
> 
>
> Key: JOSHUA-280
> URL: https://issues.apache.org/jira/browse/JOSHUA-280
> Project: Joshua
>  Issue Type: Bug
>  Components: language packs
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Priority: Critical
> Fix For: 6.1
>
>
> When I work with the existing Spanish --> English language pack at 
> http://cs.jhu.edu/~post/language-packs/language-pack-es-en-phrase-2015-03-06.tgz,
>  I get the following error
> {code}
> lmcgibbn@LMC-032857 
> /usr/local/Cellar/joshua/HEAD/libexec/language-pack-es-en-phrase-2015-03-06(NUTCH-2089)
>  $ ./run-joshua-server.sh
> INFO - Parameters read from configuration file: joshua.config
> INFO - tm = 'moses -owner pt -maxspan 0 -path phrase-table.packed 
> -max-source-len 5'
> INFO - defaultnonterminal = 'X'
> INFO - goalsymbol = 'GOAL'
> INFO - featurefunction = 'StateMinimizingLanguageModel -lm_type kenlm 
> -lm_order 5 -lm_file lm.kenlm'
> INFO - markoovs = 'false'
> INFO - search = 'stack'
> INFO - pop-limit: 100
> INFO - poplimit = '100'
> INFO - topn = '0'
> INFO - useuniquenbest = 'true'
> INFO - outputformat = '%s'
> INFO - includealignindex = 'false'
> INFO - featurefunction = 'OOVPenalty'
> INFO - featurefunction = 'WordPenalty'
> INFO - featurefunction = 'Distortion'
> INFO - featurefunction = 'PhrasePenalty'
> INFO - c = 'joshua.config'
> INFO - server-port: 5674
> INFO - serverport = '5674'
> INFO - Read 9 weights (0 of them dense)
> INFO - Reading vocabulary: phrase-table.packed/vocabulary
> INFO - Read 191983 entries from the vocabulary
> INFO - Reading packed config: phrase-table.packed/config
> 102030405060708090.100%
> Exception in thread "main" java.lang.RuntimeException: The grammar at 
> phrase-table.packed was packed with packer version 0, but the earliest 
> supported version is 3
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.readConfig(PackedGrammar.java:1061)
>   at 
> org.apache.joshua.decoder.ff.tm.packed.PackedGrammar.(PackedGrammar.java:143)
>   at 
> org.apache.joshua.decoder.phrase.PhraseTable.(PhraseTable.java:65)
>   at 
> org.apache.joshua.decoder.Decoder.initializeTranslationGrammars(Decoder.java:603)
>   at org.apache.joshua.decoder.Decoder.initialize(Decoder.java:514)
>   at org.apache.joshua.decoder.Decoder.(Decoder.java:126)
>   at org.apache.joshua.decoder.JoshuaDecoder.main(JoshuaDecoder.java:69)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (JOSHUA-279) Cannot build Joshua master branch

[Lewis John McGibbney (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JOSHUA-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lewis John McGibbney reassigned JOSHUA-279:
---

Assignee: Lewis John McGibbney

> Cannot build Joshua master branch
> -
>
> Key: JOSHUA-279
> URL: https://issues.apache.org/jira/browse/JOSHUA-279
> Project: Joshua
>  Issue Type: Bug
>  Components: build, documentation, tests
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Hi Folks,
> We need to be cautious of whatever is committed to master branch... the build 
> has been broken for quite some time and there are constant Javadoc issues 
> which make the build unstable as well.
> For example, when i make an attempt to build master branch we have failing 
> tests
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
> ...
> ---
>  T E S T S
> ---
> Running TestSuite
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> %
> %
> %
> %
> %
> %
> %
> %
> %
> Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec 
> <<< FAILURE! - in TestSuite
> setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
> Time elapsed: 0.075 sec  <<< FAILURE!
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
> in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Results :
> Failed tests:
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
>   Run 1: ClassBasedLanguageModelTest.setUp:52 » ExceptionInInitializer
>   Run 2: PASS
> Tests run: 124, Failures: 1, Errors: 0, Skipped: 4
> [INFO] 
> 
> [INFO] BUILD FAILURE
> {code}
> As a workaround I thought I will try to build the project without running the 
> test suite, however now Javadoc issues prevent me from doing so!
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install 
> -DskipTests
> ...
> 1 error
> 14 warnings
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 28.144 s
> [INFO] Finished at: 2016-07-01T14:11:42-07:00
> [INFO] Final Memory: 37M/303M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-javadoc-plugin:2.8:jar (attach-javadocs) on 
> project joshua: MavenReportException: Error while creating archive:
> [ERROR] Exit code: 1 - 
> /usr/local/incubator-joshua/src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java:217:
>  warning: no @param for rule
> [ERROR] public int[] getRuleIds(final Rule rule) {
> [ERROR] ^
> [ERROR] 
> /usr/local/incubator-joshua/src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java:217:
>  warning: no @ret

[jira] [Commented] (JOSHUA-279) Cannot build Joshua master branch

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359676#comment-15359676
 ] 

Lewis John McGibbney commented on JOSHUA-279:
-

commit 342312e309ec1bb9b1074688c1fbd3897783bc49
Author: Lewis John McGibbney 
Date:   Fri Jul 1 14:40:44 2016 -0700

JOSHUA-279 Cannot build Joshua master branch

The above commit fixes the Javadoc and I can now build. The test suite is still 
failing so I am still building with the -DskipTests flag

> Cannot build Joshua master branch
> -
>
> Key: JOSHUA-279
> URL: https://issues.apache.org/jira/browse/JOSHUA-279
> Project: Joshua
>  Issue Type: Bug
>  Components: build, documentation, tests
>Reporter: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Hi Folks,
> We need to be cautious of whatever is committed to master branch... the build 
> has been broken for quite some time and there are constant Javadoc issues 
> which make the build unstable as well.
> For example, when i make an attempt to build master branch we have failing 
> tests
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
> ...
> ---
>  T E S T S
> ---
> Running TestSuite
> tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 
> OOVPenalty=-200.000 | -198.000
> ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
> ERROR - *This probably means that the KenLM library didn't compile.
> ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
> ERROR - *installation (it's not /opt/local/, the default), change to
> ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
> ERROR - *website (joshua-decoder.org).
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - sentence 0 too long 401, truncating to length 200
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> WARN - no grammars supplied!  Supplying dummy glue grammar.
> %
> %
> %
> %
> %
> %
> %
> %
> %
> Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec 
> <<< FAILURE! - in TestSuite
> setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
> Time elapsed: 0.075 sec  <<< FAILURE!
> java.lang.ExceptionInInitializerError
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
> in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
>   at 
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
> Results :
> Failed tests:
> org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
>   Run 1: ClassBasedLanguageModelTest.setUp:52 » ExceptionInInitializer
>   Run 2: PASS
> Tests run: 124, Failures: 1, Errors: 0, Skipped: 4
> [INFO] 
> 
> [INFO] BUILD FAILURE
> {code}
> As a workaround I thought I will try to build the project without running the 
> test suite, however now Javadoc issues prevent me from doing so!
> {code}
> lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install 
> -DskipTests
> ...
> 1 error
> 14 warnings
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 28.144 s
> [INFO] Finished at: 2016-07-01T14:11:42-07:00
> [INFO] Final Memory: 37M/303M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-javadoc-plugin:2.8:jar (attach-javadocs) on 
> project joshua: MavenReportException: Error while creating archive:
> [ERROR] Exit code: 1 - 
> /usr/local/incubator-joshua/src/main/java/org/apach

JOSHUA-279 Cannot build Joshua master branch

[Lewis John Mcgibbney]

Hi Folks,
I've elevated this ticket to blocker... I can't currently build the project
at all.
I will most likely document a committer workflow to try and clarify some of
the steps as master keeps going unstable with it taking a bit of time to
get it back to stable only for it to go unstable again.
For those running off of master this is a tad frustrating but something
which can be easily avoided if we get the contributor workflow documented.
Have a nice weekend troops.
Lewis

-- 
*Lewis*

[jira] [Created] (JOSHUA-279) Cannot build Joshua master branch

[Lewis John McGibbney (JIRA)]

Lewis John McGibbney created JOSHUA-279:
---

 Summary: Cannot build Joshua master branch
 Key: JOSHUA-279
 URL: https://issues.apache.org/jira/browse/JOSHUA-279
 Project: Joshua
  Issue Type: Bug
  Components: tests, build, documentation
Reporter: Lewis John McGibbney
Priority: Blocker
 Fix For: 6.1


Hi Folks,
We need to be cautious of whatever is committed to master branch... the build 
has been broken for quite some time and there are constant Javadoc issues which 
make the build unstable as well.
For example, when i make an attempt to build master branch we have failing tests
{code}
lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install
...
---
 T E S T S
---
Running TestSuite
tm_pt_0=-2.000 tm_glue_0=3.000 lm_0=-206.718 lm_0_oov=2.000 OOVPenalty=-200.000 
| -198.000
ERROR - * FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
ERROR - *This probably means that the KenLM library didn't compile.
ERROR - *Make sure that BOOST_ROOT is set to the root of your boost
ERROR - *installation (it's not /opt/local/, the default), change to
ERROR - *$JOSHUA, and type 'ant kenlm'. If problems persist, see the
ERROR - *website (joshua-decoder.org).
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
%
%
%
%
%
%
%
%
%
Tests run: 126, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 1.818 sec <<< 
FAILURE! - in TestSuite
setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)  
Time elapsed: 0.075 sec  <<< FAILURE!
java.lang.ExceptionInInitializerError
at 
org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
Caused by: java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: no ken 
in java.library.path
at 
org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)
Caused by: java.lang.UnsatisfiedLinkError: no ken in java.library.path
at 
org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(ClassBasedLanguageModelTest.java:52)


Results :

Failed tests:
org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest.setUp(org.apache.joshua.decoder.ff.lm.class_lm.ClassBasedLanguageModelTest)
  Run 1: ClassBasedLanguageModelTest.setUp:52 » ExceptionInInitializer
  Run 2: PASS


Tests run: 124, Failures: 1, Errors: 0, Skipped: 4

[INFO] 
[INFO] BUILD FAILURE
{code}

As a workaround I thought I will try to build the project without running the 
test suite, however now Javadoc issues prevent me from doing so!

{code}
lmcgibbn@LMC-032857 /usr/local/incubator-joshua(master) $ mvn clean install 
-DskipTests
...
1 error
14 warnings
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 28.144 s
[INFO] Finished at: 2016-07-01T14:11:42-07:00
[INFO] Final Memory: 37M/303M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-javadoc-plugin:2.8:jar (attach-javadocs) on 
project joshua: MavenReportException: Error while creating archive:
[ERROR] Exit code: 1 - 
/usr/local/incubator-joshua/src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java:217:
 warning: no @param for rule
[ERROR] public int[] getRuleIds(final Rule rule) {
[ERROR] ^
[ERROR] 
/usr/local/incubator-joshua/src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java:217:
 warning: no @return
[ERROR] public int[] getRuleIds(final Rule rule) {
[ERROR] ^
[ERROR] 
/usr/local/incubator-joshua/src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java:231:
 warning: no @param for words
[ERROR] public int getOovs(final int[] words) {
[ERROR] ^
[ERROR] 
/usr/local/incubator-joshua/src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java:231:
 warning: no @return
[ERROR] public int getOovs(f