[GitHub] incubator-joshua pull request #25: ClassLMs: fixed a bug with class-based lm...

2016-06-20 Thread fhieber 
GitHub user fhieber opened a pull request:

https://github.com/apache/incubator-joshua/pull/25

ClassLMs: fixed a bug with class-based lms not mapping to class ids for 
estimateCost()

Also refactored the code a little bit to have StateMinimizingLanguageModels 
support classes as well. Added some unit tests. The existing regression test 
output was changed to the new output.

@mjpost It is hard to see whether the new regression output for 
test-classlm.sh is 'more' correct than before. If you could test this change 
with some of your models that use class-based lms, that'd be great.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/fhieber/incubator-joshua master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/25.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #25


commit 8fc7544eaaf35f71367b48778eaa1f22772ca390
Author: Felix Hieber 
Date:   2016-06-20T09:21:03Z

ClassLMs: fixed a bug with class-based lms not mapping to class ids when 
estimateCost(). Also refactored the code a little bit to have 
StateMinimizingLanguageModels support classes as well. Added some unit tests. 
The existing regression test output was changed to the new output.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Build failed in Jenkins: joshua_master #67

2016-06-20 Thread Apache Jenkins Server 
See 

Changes:

[post] bugfix on bugfix

--
[...truncated 4949 lines...]
14/14 KB   45/45 KB   29/286 KB   
14/14 KB   45/45 KB   33/286 KB   
14/14 KB   45/45 KB   37/286 KB   
14/14 KB   45/45 KB   41/286 KB   
14/14 KB   45/45 KB   45/286 KB   
14/14 KB   45/45 KB   49/286 KB   
  
14/14 KB   53/286 KB  
Downloaded: 
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/2.19.1/surefire-booter-2.19.1.jar
 (45 KB at 2115.9 KB/sec)
14/14 KB   57/286 KB   
14/14 KB   61/286 KB   
14/14 KB   65/286 KB   
14/14 KB   69/286 KB   
14/14 KB   4/192 KB   69/286 KB   
14/14 KB   4/192 KB   73/286 KB   
14/14 KB   8/192 KB   73/286 KB   
14/14 KB   12/192 KB   73/286 KB   
14/14 KB   12/192 KB   77/286 KB   
14/14 KB   16/192 KB   77/286 KB   
14/14 KB   16/192 KB   81/286 KB   
14/14 KB   20/192 KB   81/286 KB   
14/14 KB   20/192 KB   85/286 KB   
14/14 KB   24/192 KB   85/286 KB   
14/14 KB   24/192 KB   89/286 KB   
14/14 KB   28/192 KB   89/286 KB   
14/14 KB   28/192 KB   93/286 KB   
14/14 KB   32/192 KB   93/286 KB   
14/14 KB   32/192 KB   97/286 KB   
14/14 KB   36/192 KB   97/286 KB   
14/14 KB   36/192 KB   101/286 KB   
14/14 KB   40/192 KB   101/286 KB   
14/14 KB   40/192 KB   105/286 KB   
14/14 KB   44/192 KB   105/286 KB   
14/14 KB   44/192 KB   109/286 KB   
14/14 KB   48/192 KB   109/286 KB   
14/14 KB   48/192 KB   113/286 KB   
14/14 KB   52/192 KB   113/286 KB   
14/14 KB   52/192 KB   117/286 KB   
14/14 KB   56/192 KB   117/286 KB   
14/14 KB   56/192 KB   121/286 KB   
14/14 KB   60/192 KB   121/286 KB   
14/14 KB   60/192 KB   125/286 KB   
14/14 KB   64/192 KB   125/286 KB   
14/14 KB   64/192 KB   129/286 KB   
14/14 KB   68/192 KB   129/286 KB   
14/14 KB   68/192 KB   133/286 KB   
72/192 KB   133/286 KB  
76/192 KB   133/286 KB   
80/192 KB   133/286 KB   
84/192 KB   133/286 KB   
88/192 KB   133/286 KB   
92/192 KB   133/286 KB   
96/192 KB   133/286 KB   
100/192 KB   133/286 KB   
104/192 KB   133/286 KB   
108/192 KB   133/286 KB   
112/192 KB   133/286 KB   
116/192 KB   133/286 KB   
120/192 KB   133/286 KB   
124/192 KB   133/286 KB   
128/192 KB   133/286 KB   
132/192 KB   133/286 KB   
136/192 KB   133/286 KB   
140/192 KB   133/286 KB   
144/192 KB   133/286 KB   
148/192 KB   133/286 KB   
152/192 KB   133/286 KB   
156/192 KB   133/286 KB   
160/192 KB   133/286 KB   
164/192 KB   133/286 KB   
168/192 KB   133/286 KB   
172/192 KB   133/286 KB   
176/192 KB   133/286 KB   
180/192 KB   133/286 KB   
184/192 KB   133/286 KB   
188/192 KB   133/286 KB   

192/192 KB   133/286 KB   
Downloaded: 
http://repo.maven.apache.org/maven2/org/apache/maven/plugin-tools/maven-plugin-annotations/3.3/maven-plugin-annotations-3.3.jar
 (14 KB at 548.4 KB/sec)
192/192 KB   137/286 KB   
192/192 KB   141/286 KB   
192/192 KB   145/286 KB   
192/192 KB   149/286 KB   
192/192 KB   153/286 KB   
192/192 KB   157/286 KB   
192/192 KB   161/286 KB   
192/192 KB   165/286 KB   
192/192 KB   169/286 KB   
192/192 KB   173/286 KB   
192/192 KB   177/286 KB   
192/192 KB   181/286 KB   
192/192 KB   185/286 KB   
192/192 KB   189/286 KB   
192/192 KB   193/286 KB   
192/192 KB   197/286 KB   
192/192 KB   201/286 KB   
192/192 KB   205/286 KB   
192/192 KB   209/286 KB   
192/192 KB   213/286 KB   
192/192 KB   217/286 KB   
192/192 KB   221/286 KB   
192/192 KB   225/286 KB   
192/192 KB   229/286 KB   
192/192 KB   233/286 KB   
192/192 KB   237/286 KB   
192/192 KB   241/286 KB   
192/192 KB   245/286 KB   
192/192 KB   249/286 KB   
192/192 KB   253/286 KB   
192/192 KB   257/286 KB   
192/192 KB   261/286 KB   
192/192 KB   265/286 KB   
192/192 KB   269/286 KB   
192/192 KB   273/286 KB   
192/192 KB   277/286 KB   
192/192 KB   281/286 KB   
192/192 KB   285/286 KB   
192/192 KB   286/286 KB   
192/192 KB   233/286 KB   
  
Downloaded: 
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/2.19.1/surefire-api-2.19.1.jar
 (192 KB at 4255.1 KB/sec)
  
Downloaded: 
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/2.19.1/maven-surefire-common-2.19.1.jar
 (286 KB at 6209.7 KB/sec)
Downloading: 
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-testng/2.19.1/surefire-testng-2.19.1.pom
3/3 KB
 
Downloaded: 
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-testng/2.19.1/surefire-testng-2.19.1.pom
 (3 KB at 113.5 KB/sec)
Downloading: 
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/2.19.1/surefire-providers-2.19.1.pom
3/3 KB   
 
Downloaded: 
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/2.19.1/surefire-providers-2.19.1.pom
 (3 KB at 131.7 KB

Re: [IMPORTANT] Roadmap for 6.1 Release

2016-06-20 Thread Mattmann, Chris A (3980) 
Thanks for doing the yeoman’s work Lewis

++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Director, Information Retrieval and Data Science Group (IRDS)
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
WWW: http://irds.usc.edu/
++










On 6/20/16, 11:34 AM, "Lewis John Mcgibbney"  wrote:

>Hi Folks,
>I've just smartened up Jira a bit with our Roadmap being defined as follows
>
>https://issues.apache.org/jira/browse/joshua/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel
>
>Right now there are only 14/14 issues as RESOLVED for 6.1. This is false as
>I know that many more issues have been addressed however I don't think that
>Jira tickets have been created for all changes to the source code. Maybe
>moving forward we could open Jira issues and link them to the Github
>tickets via commit messages?
>
>Additionally, everything that was currently UNRESOLVED has merely been
>pushed to 6.2. If this is not what is required then please reassign the fix
>version for any ticket(s) to 6.1 and we can fix.
>
>Finally, are there any mitigating factor which would prevent a 6.1 release
>candidate being prepared right now?
>Thanks
>Lewis
>
>-- 
>*Lewis*

Build failed in Jenkins: joshua_master #66

2016-06-20 Thread Apache Jenkins Server 
See 

Changes:

[post] bugfix in argument check

--
[...truncated 1265 lines...]
Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 
<

Build failed in Jenkins: joshua_master #65

2016-06-20 Thread Apache Jenkins Server 
See 

Changes:

[post] synchronizing metadata handling

[post] better rule deletion (deleting rules with nonterminals now works, too)

--
[...truncated 7110 lines...]
112/309 KB   14/14 KB   172/223 KB   
116/309 KB   14/14 KB   172/223 KB   
120/309 KB   14/14 KB   172/223 KB   
124/309 KB   14/14 KB   172/223 KB   
128/309 KB   14/14 KB   172/223 KB   
132/309 KB   14/14 KB   172/223 KB   
136/309 KB   14/14 KB   172/223 KB   
140/309 KB   14/14 KB   172/223 KB   
144/309 KB   14/14 KB   172/223 KB   
144/309 KB   14/14 KB   172/223 KB   4/192 KB   
148/309 KB   14/14 KB   172/223 KB   4/192 KB   
148/309 KB   14/14 KB   172/223 KB   8/192 KB   
152/309 KB   14/14 KB   172/223 KB   12/192 KB   
152/309 KB   14/14 KB   172/223 KB   16/192 KB   
152/309 KB   14/14 KB   172/223 KB   20/192 KB   
152/309 KB   14/14 KB   172/223 KB   8/192 KB   
152/309 KB   14/14 KB   172/223 KB   24/192 KB   
152/309 KB   14/14 KB   172/223 KB   28/192 KB   
156/309 KB   14/14 KB   172/223 KB   28/192 KB   
156/309 KB   14/14 KB   172/223 KB   32/192 KB   
160/309 KB   14/14 KB   172/223 KB   32/192 KB   
160/309 KB   14/14 KB   172/223 KB   36/192 KB   
164/309 KB   14/14 KB   172/223 KB   36/192 KB   
164/309 KB   14/14 KB   172/223 KB   40/192 KB   
168/309 KB   14/14 KB   172/223 KB   40/192 KB   
168/309 KB   14/14 KB   172/223 KB   44/192 KB   
172/309 KB   14/14 KB   172/223 KB   44/192 KB   
172/309 KB   14/14 KB   172/223 KB   48/192 KB   
176/309 KB   14/14 KB   172/223 KB   48/192 KB   
176/309 KB   14/14 KB   172/223 KB   52/192 KB   
180/309 KB   14/14 KB   172/223 KB   52/192 KB   
184/309 KB   14/14 KB   172/223 KB   56/192 KB   
184/309 KB   14/14 KB   172/223 KB   56/192 KB   
184/309 KB   14/14 KB   172/223 KB   60/192 KB   
188/309 KB   14/14 KB   172/223 KB   60/192 KB   
188/309 KB   14/14 KB   172/223 KB   64/192 KB   
192/309 KB   14/14 KB   172/223 KB   64/192 KB   
192/309 KB   14/14 KB   172/223 KB   68/192 KB   
196/309 KB   14/14 KB   172/223 KB   68/192 KB   
196/309 KB   14/14 KB   172/223 KB   72/192 KB   
200/309 KB   14/14 KB   172/223 KB   72/192 KB   
200/309 KB   14/14 KB   172/223 KB   76/192 KB   
204/309 KB   14/14 KB   172/223 KB   76/192 KB   
204/309 KB   14/14 KB   172/223 KB   80/192 KB   
208/309 KB   14/14 KB   172/223 KB   80/192 KB   
208/309 KB   14/14 KB   172/223 KB   84/192 KB   
212/309 KB   14/14 KB   172/223 KB   84/192 KB   
212/309 KB   14/14 KB   172/223 KB   88/192 KB   
216/309 KB   14/14 KB   172/223 KB   88/192 KB   
216/309 KB   14/14 KB   172/223 KB   92/192 KB   
220/309 KB   14/14 KB   172/223 KB   92/192 KB   
220/309 KB   14/14 KB   172/223 KB   96/192 KB   
224/309 KB   14/14 KB   172/223 KB   96/192 KB   
224/309 KB   14/14 KB   172/223 KB   100/192 KB   
228/309 KB   14/14 KB   172/223 KB   100/192 KB   
228/309 KB   14/14 KB   172/223 KB   104/192 KB   
232/309 KB   14/14 KB   172/223 KB   104/192 KB   
232/309 KB   14/14 KB   172/223 KB   108/192 KB   
236/309 KB   14/14 KB   172/223 KB   108/192 KB   
236/309 KB   14/14 KB   172/223 KB   112/192 KB   
240/309 KB   14/14 KB   172/223 KB   112/192 KB   
240/309 KB   14/14 KB   172/223 KB   116/192 KB   
244/309 KB   14/14 KB   172/223 KB   116/192 KB   
244/309 KB   14/14 KB   172/223 KB   120/192 KB   
248/309 KB   14/14 KB   172/223 KB   120/192 KB   
248/309 KB   14/14 KB   172/223 KB   124/192 KB   
252/309 KB   14/14 KB   172/223 KB   124/192 KB   
252/309 KB   14/14 KB   172/223 KB   128/192 KB   
256/309 KB   14/14 KB   172/223 KB   128/192 KB   
256/309 KB   14/14 KB   172/223 KB   132/192 KB   
260/309 KB   14/14 KB   172/223 KB   132/192 KB   
260/309 KB   14/14 KB   172/223 KB   136/192 KB   
264/309 KB   14/14 KB   172/223 KB   136/192 KB   
264/309 KB   14/14 KB   172/223 KB   140/192 KB   
268/309 KB   14/14 KB   172/223 KB   140/192 KB   
268/309 KB   14/14 KB   172/223 KB   144/192 KB   
272/309 KB   14/14 KB   172/223 KB   144/192 KB   
272/309 KB   14/14 KB   172/223 KB   148/192 KB   
276/309 KB   14/14 KB   172/223 KB   148/192 KB   
276/309 KB   14/14 KB   172/223 KB   152/192 KB   
280/309 KB   14/14 KB   172/223 KB   152/192 KB   
280/309 KB   14/14 KB   172/223 KB   156/192 KB   
284/309 KB   14/14 KB   172/223 KB   156/192 KB   
284/309 KB   14/14 KB   172/223 KB   160/192 KB   
288/309 KB   14/14 KB   172/223 KB   160/192 KB   
288/309 KB   14/14 KB   172/223 KB   164/192 KB   
292/309 KB   14/14 KB   172/223 KB   164/192 KB   
292/309 KB   14/14 KB   172/223 KB   168/192 KB   
296/309 KB   14/14 KB   172/223 KB   168/192 KB   
296/309 KB   14/14 KB   172/223 KB   172/192 KB   
300/309 KB   14/14 KB   172/223 KB   172/192 KB   
300/309 KB   14/14 KB   172/223 KB   176/192 KB   
304/309 KB   14/14 KB   172/223 KB   176/192 KB   
304/309 KB   14/14 KB   172/223 KB   180/192 KB   
308/309 KB   14/14 KB   172/223 KB   180/192 KB   
308/309 KB   14/14 KB   172/223 KB   184/192 KB   
309/3

[IMPORTANT] Roadmap for 6.1 Release

2016-06-20 Thread Lewis John Mcgibbney 
Hi Folks,
I've just smartened up Jira a bit with our Roadmap being defined as follows

https://issues.apache.org/jira/browse/joshua/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel

Right now there are only 14/14 issues as RESOLVED for 6.1. This is false as
I know that many more issues have been addressed however I don't think that
Jira tickets have been created for all changes to the source code. Maybe
moving forward we could open Jira issues and link them to the Github
tickets via commit messages?

Additionally, everything that was currently UNRESOLVED has merely been
pushed to 6.2. If this is not what is required then please reassign the fix
version for any ticket(s) to 6.1 and we can fix.

Finally, are there any mitigating factor which would prevent a 6.1 release
candidate being prepared right now?
Thanks
Lewis

-- 
*Lewis*

[jira] [Commented] (JOSHUA-259) Integration tests are failing

2016-06-20 Thread Lewis John McGibbney (JIRA) 

[ 
https://issues.apache.org/jira/browse/JOSHUA-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340116#comment-15340116
 ] 

Lewis John McGibbney commented on JOSHUA-259:
-

[~kellen.sunderland] I don't think this issue is relevant any more as described.

> Integration tests are failing
> -
>
> Key: JOSHUA-259
> URL: https://issues.apache.org/jira/browse/JOSHUA-259
> Project: Joshua
>  Issue Type: Bug
>Reporter: Kellen Sunderland
> Fix For: 6.1
>
>
> Several integration tests are currently failing with Joshua.  I have a quick 
> fix coming for one of the tests but just in case we need more discussion 
> around the failures I'll open a bug.
> The currently failing tests for me:
> test/decoder/too-long
> test/server/http
> test/server/tcp-text
> test/thrax/extraction
> and 
> test/decoder/moses-compat (but this is easy to fix, simple extra space in the 
> expected file)
> These are failing under OS X 10.11.  If working under other environments feel 
> free to post a 'works for me'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (JOSHUA-269) Fix Javadoc in JOSHUA-252 branch to comply with JDK1.8 Spec

2016-06-20 Thread Lewis John McGibbney (JIRA) 

 [ 
https://issues.apache.org/jira/browse/JOSHUA-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lewis John McGibbney resolved JOSHUA-269.
-
Resolution: Fixed

> Fix Javadoc in JOSHUA-252 branch to comply with JDK1.8 Spec
> ---
>
> Key: JOSHUA-269
> URL: https://issues.apache.org/jira/browse/JOSHUA-269
> Project: Joshua
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> When we build the JOSHUA-252 codebase on Jira, we get the following
> {code}
> [INFO] 
> 
> [ERROR] BUILD ERROR
> [INFO] 
> 
> [INFO] An error has occurred in JavaDocs report generation: 
> Exit code: 1 - 
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/OracleExtractionHG.java:629:
>  warning: no @param for tbl
>   public void get_ngrams(HashMap tbl, int order, 
> ArrayList wrds,
>   ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/OracleExtractionHG.java:629:
>  warning: no @param for order
>   public void get_ngrams(HashMap tbl, int order, 
> ArrayList wrds,
>   ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/OracleExtractionHG.java:629:
>  warning: no @param for wrds
>   public void get_ngrams(HashMap tbl, int order, 
> ArrayList wrds,
>   ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/OracleExtractionHG.java:629:
>  warning: no @param for ignore_null_equiv_symbol
>   public void get_ngrams(HashMap tbl, int order, 
> ArrayList wrds,
>   ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/OracleExtractionHG.java:45:
>  error: malformed HTML
>  * @author Zhifei Li,  (Johns Hopkins University)
>   ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/OracleExtractionHG.java:45:
>  error: bad use of '>'
>  * @author Zhifei Li,  (Johns Hopkins University)
> ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/OracleExtractionHG.java:91:
>  warning: no description for @param
>* @param lm_feat_id_
>  ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/SplitHg.java:33:
>  error: malformed HTML
>  * @author Zhifei Li,  (Johns Hopkins University)
>   ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/oracle/SplitHg.java:33:
>  error: bad use of '>'
>  * @author Zhifei Li,  (Johns Hopkins University)
> ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/ui/tree_visualizer/browser/Browser.java:77:
>  error: @param name not found
>* @param args the paths to the source, reference, and n-best files
> ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/ui/tree_visualizer/browser/Browser.java:79:
>  warning: no @param for argv
>   public static void main(String[] argv) throws IOException {
>  ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/ui/tree_visualizer/browser/Browser.java:79:
>  warning: no @throws for java.io.IOException
>   public static void main(String[] argv) throws IOException {
>  ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/ui/tree_visualizer/tree/Tree.java:165:
>  warning: no @return
>   public int size() {
>  ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/ui/tree_visualizer/tree/Tree.java:172:
>  warning: no @return
>   public Node root() {
>   ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/ui/tree_visualizer/tree/Tree.java:51:
>  error: malformed HTML
>  * @author Jonny Weese 
>^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/ui/tree_visualizer/tree/Tree.java:51:
>  error: bad use of '>'
>  * @author Jonny Weese 
> ^
> /home/jenkins/jenkins-slave/workspace/joshua_maven/src/main/java/org/apache/joshua/ui/tree_visualizer/tree/Tree.java:105:
>  warning: no @param for s
>   public Tree(String s) {
>  ^
> /home/jenkins/jenkins-slave/works

[jira] [Updated] (JOSHUA-275) Revamp the Configuration System

2016-06-20 Thread Lewis John McGibbney (JIRA) 

 [ 
https://issues.apache.org/jira/browse/JOSHUA-275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lewis John McGibbney updated JOSHUA-275:

Fix Version/s: (was: 6,2)
   6.2

> Revamp the Configuration System
> ---
>
> Key: JOSHUA-275
> URL: https://issues.apache.org/jira/browse/JOSHUA-275
> Project: Joshua
>  Issue Type: Improvement
>Affects Versions: 6.1, 6.2, 7
>Reporter: Kellen Sunderland
> Fix For: 6.2
>
>
> I'd like to propose we centralize Joshua's configuration system to make use 
> of typesafe/config https://github.com/typesafehub/config .  This config 
> system looks like JSON but with comments so it's easy to read.  Because it's 
> JSON it supports hierarchies of configurations, lists of configuration etc 
> quite easily.  It has some nice features like parsing time automatically.  
> The main advantage here though is that we have a standard config system that 
> doesn't have to be manually parsed.
> Here's a quick example of how we can use it:
> {code:java}
> @Inject
> public PackedGrammar(@TypesafeConfig("PackedGrammar.grammar_dir")
>  String grammar_dir,
>  @TypesafeConfig("PackedGrammar.span_limit")
>  int span_limit, 
>  String owner, 
>  String type) throws FileNotFoundException, 
> IOException ...
> {code}
> and then a config similar to
> \# Joshua configuration file
> {code:javascript}
> config = {
> default-non-terminal = X
> goal-symbol = GOAL
> ...
> 
> PackedGrammar: {
> type: thrax,
> grammar_dir: /local/grammars/...
> span_limit: 50
> }
> ...
> }
> {code}
> Version: TBD, but it's a breaking change so we may consider putting it in 
> Joshua 7.
> Totally open to other config / injection systems if others want to suggest 
> any of their favorites.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (JOSHUA-265) Refactor key interfaces and core code for a future release.

2016-06-20 Thread Lewis John McGibbney (JIRA) 

 [ 
https://issues.apache.org/jira/browse/JOSHUA-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lewis John McGibbney updated JOSHUA-265:

Fix Version/s: 6.2

> Refactor key interfaces and core code for a future release. 
> 
>
> Key: JOSHUA-265
> URL: https://issues.apache.org/jira/browse/JOSHUA-265
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Kellen Sunderland
>Priority: Minor
> Fix For: 6.2
>
>
> We've discussed making some modifications to the key interfaces.  This ticket 
> can focus on making large changes to the codebase for a future release.  This 
> work will likely take some time and some collaboration.  I'd suggest some the 
> code for this be a separate release branch.
> Some issues we can work on:
> *  I'd propose we conform to the SOLID principles for our major interfaces.  
> https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)  . 
> *  We can look at Sparse / Dense feature vectors and how to handle them 
> naturally in Joshua.
> *  Refactor objects that may now be used more broadly than was originally 
> intended (for example Vocabulary class).
> *  We should have a general discussion around what parts of the codebase are 
> responsible for what functions.  We should clearly define what logic should 
> be a part of the Grammar versus the Feature Functions for example, and make 
> sure logic doesn't leak from one of these objects to the others.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (JOSHUA-275) Revamp the Configuration System

2016-06-20 Thread Lewis John McGibbney (JIRA) 

 [ 
https://issues.apache.org/jira/browse/JOSHUA-275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lewis John McGibbney updated JOSHUA-275:

Fix Version/s: 6,2

> Revamp the Configuration System
> ---
>
> Key: JOSHUA-275
> URL: https://issues.apache.org/jira/browse/JOSHUA-275
> Project: Joshua
>  Issue Type: Improvement
>Affects Versions: 6.1, 6.2, 7
>Reporter: Kellen Sunderland
> Fix For: 6,2
>
>
> I'd like to propose we centralize Joshua's configuration system to make use 
> of typesafe/config https://github.com/typesafehub/config .  This config 
> system looks like JSON but with comments so it's easy to read.  Because it's 
> JSON it supports hierarchies of configurations, lists of configuration etc 
> quite easily.  It has some nice features like parsing time automatically.  
> The main advantage here though is that we have a standard config system that 
> doesn't have to be manually parsed.
> Here's a quick example of how we can use it:
> {code:java}
> @Inject
> public PackedGrammar(@TypesafeConfig("PackedGrammar.grammar_dir")
>  String grammar_dir,
>  @TypesafeConfig("PackedGrammar.span_limit")
>  int span_limit, 
>  String owner, 
>  String type) throws FileNotFoundException, 
> IOException ...
> {code}
> and then a config similar to
> \# Joshua configuration file
> {code:javascript}
> config = {
> default-non-terminal = X
> goal-symbol = GOAL
> ...
> 
> PackedGrammar: {
> type: thrax,
> grammar_dir: /local/grammars/...
> span_limit: 50
> }
> ...
> }
> {code}
> Version: TBD, but it's a breaking change so we may consider putting it in 
> Joshua 7.
> Totally open to other config / injection systems if others want to suggest 
> any of their favorites.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (JOSHUA-268) Phrase-based model error (NullPointerException)

2016-06-20 Thread Lewis John McGibbney (JIRA) 

 [ 
https://issues.apache.org/jira/browse/JOSHUA-268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lewis John McGibbney updated JOSHUA-268:

Fix Version/s: 6.2

> Phrase-based model error (NullPointerException)
> ---
>
> Key: JOSHUA-268
> URL: https://issues.apache.org/jira/browse/JOSHUA-268
> Project: Joshua
>  Issue Type: Bug
>  Components: decoders
>Affects Versions: 6.0.5
> Environment: fedora 23
>Reporter: Kyle Richardson
>Priority: Minor
> Fix For: 6.2
>
>
> I'm trying to run the phrase.sh example script (the only modification I made 
> was to take out the --optimizer-runs option, because the system says that 
> this is an "Unknown option"). 
> The error comes at the tuning stage (specifically, it fails at some point in 
> the tuning then complains that it cannot find the "joshua.config.final" 
> file). 
> Looking into the log file (tune/joshua.log), it seems to translate and tune a 
> number of sentences, then it raises the following NullPointerException: 
> Memory used after sentence 7 is 42.5 MB
> Translation 7: -30.617 good how is fine
> Input 2: Collecting options took 0.000 seconds
> Input 8: Collecting options took 0.000 seconds
> Input 2: FATAL UNCAUGHT EXCEPTION: null
> java.lang.NullPointerException
> at joshua.decoder.phrase.Candidate.score(Candidate.java:214)
> at joshua.decoder.phrase.Candidate.compareTo(Candidate.java:136)
> at joshua.decoder.phrase.Candidate.compareTo(Candidate.java:19)
> at java.util.HashMap.compareComparables(HashMap.java:371)
> at java.util.HashMap$TreeNode.treeify(HashMap.java:1920)
> at java.util.HashMap.treeifyBin(HashMap.java:771)
> at java.util.HashMap.putVal(HashMap.java:643)
> at java.util.HashMap.put(HashMap.java:611)
> at java.util.HashSet.add(HashSet.java:219)
> at joshua.decoder.phrase.Stack.addCandidate(Stack.java:125)
> at joshua.decoder.phrase.Stacks.search(Stacks.java:166)
> at joshua.decoder.DecoderThread.translate(DecoderThread.java:113)
> atjoshua.decoder.Decoder$DecoderThreadRunner.run(Decoder.java:218)
> There's nothing informative in the tune/mert.log, it just says that it exited 
> prematurely. The other processes seem to work as expected (although in the 
> giza.log, there are a number of "Sentence mismatch error! Line " warnings). 
> I'm running this on Fedora 23  with Moses.  I had no problems training the 
> hiero model.
> note---
> There appears to be an open ticket for more or less the same problem 
> (JOSHUA-267), the difference however is that in that in this ticket, it 
> appears that the tuner fails on the first input, whereas here, it already 
> decodes/tunes several inputs before failing (see above). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: joshua_master #64

2016-06-20 Thread Apache Jenkins Server 
See 

Changes:

[post] bugfix in invocation

--
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ubuntu-5 (docker Ubuntu ubuntu5 ubuntu yahoo-not-h2) in 
workspace 
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/incubator-joshua.git # timeout=10
Fetching upstream changes from 
https://git-wip-us.apache.org/repos/asf/incubator-joshua.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/incubator-joshua.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 0e4ebe1c69ab5a53c04cd6a25e4b06e340359a82 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 0e4ebe1c69ab5a53c04cd6a25e4b06e340359a82
 > git rev-list da124c2002d44b67485bd5eb138482b56bbc47ef # timeout=10
[joshua_master] $ /home/jenkins/tools/maven/apache-maven-3.0.4/bin/mvn clean 
deploy javadoc:aggregate
[INFO] Scanning for projects...
[INFO] 
[INFO] 
[INFO] Building Apache Joshua Machine Translation Toolkit 6.0.6-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ joshua ---
[INFO] Deleting 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.2.1:process (default) @ joshua ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ joshua ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ joshua ---
[INFO] Compiling 264 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
joshua ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 319 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
joshua ---
[INFO] Compiling 41 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ joshua ---

---
 T E S T S
---
Running TestSuite
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
WARN - sentence 0 too long 401, truncating to length 200
%
%
%
%
%
%
%
%
%
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
WARN - no grammars supplied!  Supplying dummy glue grammar.
Tests run: 122, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 2.001 sec <<< 
FAILURE! - in TestSuite
basicVocabTest(org.apache.joshua.corpus.vocab.VocabularyTest)  Time elapsed: 
0.031 sec  <<< FAILURE!
java.lang.AssertionError: null
at 
org.apache.joshua.corpus.vocab.VocabularyTest.basicVocabTest(VocabularyTest.java:56)


Results :

Failed tests: 
  VocabularyTest.basicVocabTest:56 null

Tests run: 122, Failures: 1, Errors: 0, Skipped: 1

[INFO] 
[INFO] 
[INFO] Skipping Apache Joshua Machine Translation Toolkit
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 14.897s
[INFO] Finished at: Mon Jun 20 16:14:44 UTC 2016
[INFO] Final Memory: 34M/643M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on 
project joshua: There are test failures.
[ERROR] 
[ERROR] Please refer to 
 for 
the individual test results

[GitHub] incubator-joshua pull request #:

2016-06-20 Thread fhieber 
Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/6d2213a20b74432fc7cb131c732f7507b74053e9#commitcomment-17933645
  
In src/main/java/org/apache/joshua/decoder/StructuredTranslation.java:
In src/main/java/org/apache/joshua/decoder/StructuredTranslation.java on 
line 57:
Hi Matt,
 
Thanks, StructuredTranslation is an object to be used in a downstream 
codebase and I think it would be good to have minimal dependencies on Joshua 
objects as possible there :)
 
Regarding your question:
I’d go for a container/value object for the reason stated above if 
possible.
 
Cheers,
Felix


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-273) Joshua API

2016-06-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/JOSHUA-273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339419#comment-15339419
 ] 

ASF GitHub Bot commented on JOSHUA-273:
---

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/6d2213a20b74432fc7cb131c732f7507b74053e9#commitcomment-17933088
  
In src/main/java/org/apache/joshua/decoder/StructuredTranslation.java:
In src/main/java/org/apache/joshua/decoder/StructuredTranslation.java on 
line 57:
Hi Felix — I guess I probably don't. I put this in so as to have the 
convenience functions of querying the maps (that are in FeatureVector), and 
didn't realize why you guys had stuck with just a HashMap. I'll revert it.

In general, this relates to JOSHUA-273: I want to remove all the output 
code from KBestExtractor and place it in StructuredTranslation (and then, in 
turn, to collapse Translation and StructuredTranslation). There are at least 
two ways to do this; do you have a preference?

- StructuredTranslationFactory could be called to create the necessary 
entries in StructuredTranslation, which would then just be stored as strings. 
This approach makes StructuredTranslation basically just a container that is 
operated on.

- StructuredTranslation can do its own computation, formatting the 
"outputFormat" line from the config file. This view makes StructuredTranslation 
itself a bit smarter.

matt


> Joshua API
> --
>
> Key: JOSHUA-273
> URL: https://issues.apache.org/jira/browse/JOSHUA-273
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Matt Post
> Fix For: 7
>
>
> We have a lot of work to do to clean up the decoder's internal object 
> pipeline in order to create a nice, clean API.
> (This is just a stub for this issue; I will return soon with a better 
> description and roadmap. Others feel free to edit, as well).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request #:

2016-06-20 Thread mjpost 
Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/6d2213a20b74432fc7cb131c732f7507b74053e9#commitcomment-17933088
  
In src/main/java/org/apache/joshua/decoder/StructuredTranslation.java:
In src/main/java/org/apache/joshua/decoder/StructuredTranslation.java on 
line 57:
Hi Felix — I guess I probably don't. I put this in so as to have the 
convenience functions of querying the maps (that are in FeatureVector), and 
didn't realize why you guys had stuck with just a HashMap. I'll revert it.

In general, this relates to JOSHUA-273: I want to remove all the output 
code from KBestExtractor and place it in StructuredTranslation (and then, in 
turn, to collapse Translation and StructuredTranslation). There are at least 
two ways to do this; do you have a preference?

- StructuredTranslationFactory could be called to create the necessary 
entries in StructuredTranslation, which would then just be stored as strings. 
This approach makes StructuredTranslation basically just a container that is 
operated on.

- StructuredTranslation can do its own computation, formatting the 
"outputFormat" line from the config file. This view makes StructuredTranslation 
itself a bit smarter.

matt


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---