[GitHub] incubator-joshua pull request: JOSHUA-252 Make it possible to use ...

[thammegowda]

Github user thammegowda commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221782894
  
@lewismc Please merge #17 ( it is a duplicate of #15 )


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301570#comment-15301570
 ] 

ASF GitHub Bot commented on JOSHUA-252:
---

Github user thammegowda commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221782894
  
@lewismc Please merge #17 ( it is a duplicate of #15 )


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301568#comment-15301568
 ] 

ASF GitHub Bot commented on JOSHUA-252:
---

Github user thammegowda commented on the pull request:

https://github.com/apache/incubator-joshua/pull/15#issuecomment-221782749
  
#17 has been created to make the merging job easy by targeting to 
JOSHUA-252.
 So closing this PR.


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: Joshua-262: Slf4j - Log4j bridge

[thammegowda]

Github user thammegowda closed the pull request at:

https://github.com/apache/incubator-joshua/pull/15


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request: Joshua-262: Slf4j - Log4j bridge

[thammegowda]

Github user thammegowda commented on the pull request:

https://github.com/apache/incubator-joshua/pull/15#issuecomment-221782749
  
#17 has been created to make the merging job easy by targeting to 
JOSHUA-252.
 So closing this PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-262) Implement all logging as Slf4j over Log4j

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301565#comment-15301565
 ] 

ASF GitHub Bot commented on JOSHUA-262:
---

GitHub user thammegowda opened a pull request:

https://github.com/apache/incubator-joshua/pull/17

JOSHUA-262 : Replacing System.{out,err}.print* and java.util.log with SLF4j

This is a duplicate of #15 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/thammegowda/incubator-joshua JOSHUA-262

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/17.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #17


commit 09fb6a2d363ac78f091b217a88a8712c47edc5f0
Author: Matt Post 
Date:   2016-05-14T19:27:38Z

don't separately pack the test grammar (done in run bundler)

commit f354c298ff9d1f16b8c034a5d885428d95e43ca3
Author: Matt Post 
Date:   2016-05-16T22:20:45Z

Merge branch 'JOSHUA-264' of 
https://github.com/thammegowda/incubator-joshua into JOSHUA-264

commit 659e464665254050a8f9ed321dcbdd08eef8a3d7
Author: Matt Post 
Date:   2016-05-17T00:14:04Z

Merge branch 'jar-with-dependencies' of 
https://github.com/thammegowda/incubator-joshua into JOSHUA-264

commit c21fa9e82db5b1f784b89ea8109735a3645298f2
Author: Thamme Gowda 
Date:   2016-05-21T07:02:42Z

Log4j - Slf4j bridge 

+ Removed java.util.log statements
+ SLF4j with string format pattern replacement

commit 9114a007ae4a42d97e2218712defafa3a9761560
Author: Thamme Gowda 
Date:   2016-05-21T07:12:24Z

Read me updated

commit 4d04cc2c01669e3b93399758f54aae27e6e2d0ec
Author: Thamme Gowda 
Date:   2016-05-21T18:22:28Z

LOG scope is privatized

commit d6efccbc51260028225652c77bfa0f4bdab8061b
Author: Thamme Gowda 
Date:   2016-05-21T19:06:20Z

Clean LOGs, no redudant if(enabled) checks, no eager toString()s

commit 8652d19d1094cc6329220984d0693e6dcef4
Author: Thamme Gowda 
Date:   2016-05-21T19:14:29Z

Fix spaces

commit d4ac45193450f1c901f23cc938e5981bb64eb8d6
Author: Thamme Gowda 
Date:   2016-05-21T19:32:39Z

Fix log issues such as redundant checks and spaces

commit 158685310332be4166164bce28058a90f1d168d7
Author: Thamme Gowda 
Date:   2016-05-23T04:18:21Z

Replaced System.err.print* with logger api

commit 9d6f84d35754a099123c256b9932a89a2bd316aa
Author: Thamme Gowda 
Date:   2016-05-26T05:34:57Z

Rebased with JOSHUA-252 and resolved merge conflicts




> Implement all logging as Slf4j over Log4j
> -
>
> Key: JOSHUA-262
> URL: https://issues.apache.org/jira/browse/JOSHUA-262
> Project: Joshua
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Thamme Gowda N
> Fix For: 6.1
>
>
> [~hsaputra] suggested that we implement all logging as Slf4j over Log4j. If 
> we use [parameterized logging 
> notation|http://www.slf4j.org/faq.html#logging_performance] we can have good 
> logging in place.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: JOSHUA-262 : Replacing System.{out,...

[thammegowda]

GitHub user thammegowda opened a pull request:

https://github.com/apache/incubator-joshua/pull/17

JOSHUA-262 : Replacing System.{out,err}.print* and java.util.log with SLF4j

This is a duplicate of #15 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/thammegowda/incubator-joshua JOSHUA-262

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/17.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #17


commit 09fb6a2d363ac78f091b217a88a8712c47edc5f0
Author: Matt Post 
Date:   2016-05-14T19:27:38Z

don't separately pack the test grammar (done in run bundler)

commit f354c298ff9d1f16b8c034a5d885428d95e43ca3
Author: Matt Post 
Date:   2016-05-16T22:20:45Z

Merge branch 'JOSHUA-264' of 
https://github.com/thammegowda/incubator-joshua into JOSHUA-264

commit 659e464665254050a8f9ed321dcbdd08eef8a3d7
Author: Matt Post 
Date:   2016-05-17T00:14:04Z

Merge branch 'jar-with-dependencies' of 
https://github.com/thammegowda/incubator-joshua into JOSHUA-264

commit c21fa9e82db5b1f784b89ea8109735a3645298f2
Author: Thamme Gowda 
Date:   2016-05-21T07:02:42Z

Log4j - Slf4j bridge 

+ Removed java.util.log statements
+ SLF4j with string format pattern replacement

commit 9114a007ae4a42d97e2218712defafa3a9761560
Author: Thamme Gowda 
Date:   2016-05-21T07:12:24Z

Read me updated

commit 4d04cc2c01669e3b93399758f54aae27e6e2d0ec
Author: Thamme Gowda 
Date:   2016-05-21T18:22:28Z

LOG scope is privatized

commit d6efccbc51260028225652c77bfa0f4bdab8061b
Author: Thamme Gowda 
Date:   2016-05-21T19:06:20Z

Clean LOGs, no redudant if(enabled) checks, no eager toString()s

commit 8652d19d1094cc6329220984d0693e6dcef4
Author: Thamme Gowda 
Date:   2016-05-21T19:14:29Z

Fix spaces

commit d4ac45193450f1c901f23cc938e5981bb64eb8d6
Author: Thamme Gowda 
Date:   2016-05-21T19:32:39Z

Fix log issues such as redundant checks and spaces

commit 158685310332be4166164bce28058a90f1d168d7
Author: Thamme Gowda 
Date:   2016-05-23T04:18:21Z

Replaced System.err.print* with logger api

commit 9d6f84d35754a099123c256b9932a89a2bd316aa
Author: Thamme Gowda 
Date:   2016-05-26T05:34:57Z

Rebased with JOSHUA-252 and resolved merge conflicts




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301564#comment-15301564
 ] 

Hudson commented on JOSHUA-252:
---

SUCCESS: Integrated in joshua_maven #13 (See 
[https://builds.apache.org/job/joshua_maven/13/])
Pulled JOSHUA-252 changes and Resolved Merge Conflicts (tgowdan: rev 
ef91969a8efcd525f3ff639805ab0bf1147a0616)
* test/pipeline/input/devtest.en.0
* test/bn-en/packed/joshua.config
* src/main/java/org/apache/joshua/decoder/Translation.java
* src/main/java/org/apache/joshua/decoder/ff/TargetBigram.java
* test/decoder/phrase/decode/rules.packed/vocabulary
* test/bn-en/packed/reference.en.0
* src/test/resources/thrax/filtering/grammar.filtered.gz
* test/thrax/filtering/fast.gold
* src/main/java/org/apache/joshua/decoder/ff/tm/PhraseRule.java
* ext/berkeleylm
* src/test/resources/scripts/support/moses_grammar/test.sh
* src/test/resources/lattice/README
* src/main/java/org/apache/joshua/decoder/hypergraph/HyperGraphPruning.java
* src/test/resources/decoder/phrase/decode/config
* test/decoder/source-annotations/joshua.config
* src/test/resources/packed-grammar/test-multiple.sh
* src/test/resources/bn-en/hiero/test-classlm.sh
* test/bn-en/samt/output.gold
* src/test/resources/scripts/normalization/test.sh
* src/main/java/org/apache/joshua/decoder/ff/tm/hash_based/package.html
* src/test/resources/bn-en/hiero/joshua-berkeleylm.config
* test/pipeline/input/devtest.ur
* ext/giza-pp/mkcls-v2/mystl.h
* src/test/resources/decoder/phrase/include-align-index/corpus.es
* test/decoder/oov-list/output.gold
* test/bn-en/packed/.gitignore
* test/decoder/source-annotations/test.sh
* ext/giza-pp/GIZA++-v2/parse.cpp
* test/bn-en/samt/reference.en.3
* ext/giza-pp/GIZA++-v2/dependencies
* test/scripts/support/moses_grammar/output.expected
* ext/giza-pp/GIZA++-v2/model345-peg.cpp
* src/test/resources/decoder/num_translation_options/output.gold
* test/thrax/.gitignore
* src/test/resources/lattice/grammar.test
* test/lattice/output.expected
* src/main/java/org/apache/joshua/decoder/package-info.java
* src/test/resources/grammar/sparse-features/output.gold
* ext/giza-pp/mkcls-v2/general.cpp
* src/main/java/org/apache/joshua/decoder/ff/lm/package.html
* test/decoder/phrase/include-align-index/config
* test/thrax/filtering/fast.log.gold
* ext/giza-pp/GIZA++-v2/logprob.cpp
* test/grammar/sparse-features/test.sh
* test/decoder/num_translation_options/grammar.packed/vocabulary
* ext/giza-pp/GIZA++-v2/Makefile.src
* src/test/resources/bn-en/samt/reference.en.3
* test/decoder/phrase/decode/test.sh
* ext/giza-pp/GIZA++-v2/transpair_model3.cpp
* test/lattice-short/glue-grammar
* src/main/java/org/apache/joshua/decoder/hypergraph/TrivialInsideOutside.java
* src/test/resources/pipeline/input/tune.en.3
* test/bn-en/packed/reference.en.2
* test/packed-grammar/reference.en.3
* src/main/java/org/apache/joshua/decoder/io/DeNormalize.java
* test/bn-en/hiero/output-classlm.gold
* ext/giza-pp/GIZA++-v2/collCounts.h
* ext/giza-pp/mkcls-v2/PopOptimization.h
* test/server/tcp-text/expected
* src/main/java/org/apache/joshua/decoder/ff/tm/SentenceFilteredGrammar.java
* src/main/java/org/apache/joshua/subsample/package-info.java
* test/bn-en/hiero/output.gold
* src/test/resources/decoder/segment-oovs/config
* src/test/resources/thrax/filtering/exact.gold
* src/main/java/org/apache/joshua/decoder/hypergraph/AlignedSourceTokens.java
* ext/giza-pp/GIZA++-v2/TTables.cpp
* Dockerfile
* test/decoder/constrained/test.sh
* src/test/resources/packed-grammar/reference.en.2
* src/test/resources/decoder/num_translation_options/joshua.config
* src/test/resources/decoder/regexp-grammar-both-rule-types/README
* test/decoder/regexp-grammar-both-rule-types/output.gold
* src/test/resources/decoder/rescoring/grammar.gz
* test/decoder/phrase/constrained/config
* test/server/http/expected
* src/main/java/org/apache/joshua/util/io/NullReader.java
* src/main/java/org/apache/joshua/util/FileUtility.java
* src/main/java/org/apache/joshua/decoder/ff/tm/Trie.java
* src/main/java/org/apache/joshua/decoder/ArgsParser.java
* src/main/java/org/apache/joshua/decoder/hypergraph/KBestExtractor.java
* src/test/resources/decoder/phrase/unique-hypotheses/rules.1.gz
* test/scripts/.gitignore
* src/test/resources/thrax/filtering/fast.gold
* test/decoder/phrase/decode/output.gold
* src/test/resources/thrax/filtering/grammar.de
* ext/symal/giza2bal.pl
* test/decoder/denormalization/test.sh
* test/scripts/merge_lms_test.py
* ext/giza-pp/mkcls-v2/TAOptimization.h
* test/pipeline/test.sh
* test/decoder/num_translation_options/test.sh
* src/main/java/org/apache/joshua/decoder/ff/state_maintenance/KenLMState.java
* src/test/resources/decoder/segment-oovs/output.expected
* test/decoder/tree-output/grammar.gz
* test/decoder/fragmentlm/glue
* test/grammar/sparse-features/grammar.packed/slice_0.target.lookup
* ext/giza-pp/GIZA++-v2/Perplexity.h
* ext/giza-pp/mkcls-v2/myleda.

[GitHub] incubator-joshua pull request: JOSHUA-252 Make it possible to use ...

[lewismc]

Github user lewismc commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221779115
  
Cool @thammegowda 


#14 is merged.
Work still to be done
#13 merged
work still to be done
 * merge #15 into JOSHUA-252
 * plan on how we are going to compile and package GIZA++, KenLM, 
berkeleylm, thrax, etc. Right now the ext directory has been completely removed 
as that code should not live but instead be acquired as part of the build 
system.

@mjpost have you made any changes to master which we need to merge into 
JOSHUA-252 branch?
Thanks



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-264) Remove system exits and replace with RuntimeExceptions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301542#comment-15301542
 ] 

ASF GitHub Bot commented on JOSHUA-264:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-joshua/pull/13


> Remove system exits and replace with RuntimeExceptions
> --
>
> Key: JOSHUA-264
> URL: https://issues.apache.org/jira/browse/JOSHUA-264
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Kellen Sunderland
>
> When Joshua is used a library it's much more convenient to get 
> RuntimeExceptions when a fatal error happens.  This way the host process can 
> possibly handle the error or take some appropriate action (alarm, log, etc).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: JOSHUA-264 System.exit() calls are ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-joshua/pull/13


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301535#comment-15301535
 ] 

ASF GitHub Bot commented on JOSHUA-252:
---

Github user thammegowda commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221778184
  
@lewismc thanks for the reply.

rebased #13 , it is ready for merge!


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: too many emails

[Tom Barber]

FWIW I just have a Gmail filter setup to archive them
On 26 May 2016 05:45, "Lewis John Mcgibbney" 
wrote:

> Hi Matt,
>
> As Henry said. Either we get them going to a different list or else you
> subscribe to dev-dig...@joshua.incubator.apache.org (subscribe through
> dev-digest-subscr...@joshua.incubator.apache.org)?
> Which do you prefer?
> Quick reasoning as to why Github convo is shadowed on the Apache lists. If
> Github ever goes away, then we loose all of the conversation. We archive it
> @Apache so we cover our communities.
> Thanks
>
>
> On Wed, May 25, 2016 at 2:11 PM, <
> dev-digest-h...@joshua.incubator.apache.org> wrote:
>
> >
> > From: Matt Post 
> > To: dev@joshua.incubator.apache.org
> > Cc:
> > Date: Wed, 25 May 2016 15:48:24 -0400
> > Subject: too many emails
> > Does someone know how to turn off the mailing of all github comments to
> > dev?
> >
> > The way I see it, we all have to be on dev, so it should be for people,
> > not robots. I am getting every comment about three times.
> >
> > I would just do it but I don't know how.
> >
> >
>

[GitHub] incubator-joshua pull request: JOSHUA-252 Make it possible to use ...

[thammegowda]

Github user thammegowda commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221778184
  
@lewismc thanks for the reply.

rebased #13 , it is ready for merge!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301531#comment-15301531
 ] 

ASF GitHub Bot commented on JOSHUA-252:
---

Github user lewismc commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221777380
  
#14 is merged.
Work still to be done
 * merge #13 into JOSHUA-252 branch
 * merge #15 into JOSHUA-252
 * plan on how we are going to compile and package GIZA++, KenLM, 
berkeleylm, thrax, etc. Right now the ext directory has been completely removed 
as that code should not live but instead be acquired as part of the build 
system.

@mjpost have you made any changes to master which we need to merge into 
JOSHUA-252 branch?
Thanks


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: JOSHUA-252 Make it possible to use ...

[lewismc]

Github user lewismc commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221777380
  
#14 is merged.
Work still to be done
 * merge #13 into JOSHUA-252 branch
 * merge #15 into JOSHUA-252
 * plan on how we are going to compile and package GIZA++, KenLM, 
berkeleylm, thrax, etc. Right now the ext directory has been completely removed 
as that code should not live but instead be acquired as part of the build 
system.

@mjpost have you made any changes to master which we need to merge into 
JOSHUA-252 branch?
Thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request: Maven assembly plugin for executabl...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-joshua/pull/14


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request: JOSHUA-252 Make it possible to use ...

[lewismc]

Github user lewismc commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221776975
  
hi @thammegowda thanks for reply

 *  #13 should be able to merge with this branch (have conflicts now, but i 
can resolve them)
Please rebase the above against this branch

*  #14 can be merged with this, no conflicts
I will merge this

#15 is targetted to branch named maven . Probably I need to raise a new PR 
targeted to this JOSHUA-252 branch ? 

Yes please do. All branches should be named after the associated Jira 
ticket. That way we have full blown provenance tracking for individual issues. 

Thanks, I'll merge #14 now


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301530#comment-15301530
 ] 

ASF GitHub Bot commented on JOSHUA-252:
---

Github user lewismc commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221776975
  
hi @thammegowda thanks for reply

 *  #13 should be able to merge with this branch (have conflicts now, but i 
can resolve them)
Please rebase the above against this branch

*  #14 can be merged with this, no conflicts
I will merge this

#15 is targetted to branch named maven . Probably I need to raise a new PR 
targeted to this JOSHUA-252 branch ? 

Yes please do. All branches should be named after the associated Jira 
ticket. That way we have full blown provenance tracking for individual issues. 

Thanks, I'll merge #14 now


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: JOSHUA-252 Make it possible to use ...

[thammegowda]

Github user thammegowda commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221776421
  
great work @lewismc . 

All my activity is branched off from this branch

* #13  should be able to merge with this branch (have conflicts now, but i 
can resolve them)
* #14 can be merged with this, no conflicts
* #15 is targetted to branch named `maven` . Probably I need to raise a new 
PR targeted to this `JOSHUA-252` branch ? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301519#comment-15301519
 ] 

ASF GitHub Bot commented on JOSHUA-252:
---

Github user thammegowda commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221776421
  
great work @lewismc . 

All my activity is branched off from this branch

* #13  should be able to merge with this branch (have conflicts now, but i 
can resolve them)
* #14 can be merged with this, no conflicts
* #15 is targetted to branch named `maven` . Probably I need to raise a new 
PR targeted to this `JOSHUA-252` branch ? 


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: too many emails

[Lewis John Mcgibbney]

Hi Matt,

As Henry said. Either we get them going to a different list or else you
subscribe to dev-dig...@joshua.incubator.apache.org (subscribe through
dev-digest-subscr...@joshua.incubator.apache.org)?
Which do you prefer?
Quick reasoning as to why Github convo is shadowed on the Apache lists. If
Github ever goes away, then we loose all of the conversation. We archive it
@Apache so we cover our communities.
Thanks


On Wed, May 25, 2016 at 2:11 PM, <
dev-digest-h...@joshua.incubator.apache.org> wrote:

>
> From: Matt Post 
> To: dev@joshua.incubator.apache.org
> Cc:
> Date: Wed, 25 May 2016 15:48:24 -0400
> Subject: too many emails
> Does someone know how to turn off the mailing of all github comments to
> dev?
>
> The way I see it, we all have to be on dev, so it should be for people,
> not robots. I am getting every comment about three times.
>
> I would just do it but I don't know how.
>
>

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301513#comment-15301513
 ] 

ASF GitHub Bot commented on JOSHUA-252:
---

Github user lewismc commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221775479
  
@thammegowda @mjpost  OK the Maven build is now 'stable'. 
We need to list which changes have been made to master branch since this 
work began. These then need to be merged into the JOSHUA-252 branch. 
Additionally, the following needs to be addressed
 * merge all work which has been done to master branch into JOSHUA-252 
branch
 * plan on how we are going to compile and package GIZA++, KenLM, 
berkeleylm, thrax, etc. Right now the ext directory has been completely removed 
as that code should not live but instead be acquired as part of the build 
system.


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: JOSHUA-252 Make it possible to use ...

[lewismc]

Github user lewismc commented on the pull request:

https://github.com/apache/incubator-joshua/pull/12#issuecomment-221775479
  
@thammegowda @mjpost  OK the Maven build is now 'stable'. 
We need to list which changes have been made to master branch since this 
work began. These then need to be merged into the JOSHUA-252 branch. 
Additionally, the following needs to be addressed
 * merge all work which has been done to master branch into JOSHUA-252 
branch
 * plan on how we are going to compile and package GIZA++, KenLM, 
berkeleylm, thrax, etc. Right now the ext directory has been completely removed 
as that code should not live but instead be acquired as part of the build 
system.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Joshua SNAPSHOTS's available on repository.apache.org

[Lewis John Mcgibbney]

These come from the Maven Build

https://builds.apache.org/view/H-L/view/Joshua/job/joshua_maven

and upon every stable successful build will be pushed to
repository.apache.org

https://repository.apache.org/content/repositories/snapshots/org/apache/joshua/joshua/

Lewis

-- 
*Lewis*

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301504#comment-15301504
 ] 

Hudson commented on JOSHUA-252:
---

SUCCESS: Integrated in joshua_maven #12 (See 
[https://builds.apache.org/job/joshua_maven/12/])
JOSHUA-252 Make it possible to use Maven to build Joshua (lewis.mcgibbney: rev 
fdf20e20aebe8d39a9046ffab0c36f1f918818b9)
* src/main/java/org/apache/joshua/decoder/Decoder.java
JOSHUA-252 Make it possible to use Maven to build Joshua (lewis.mcgibbney: rev 
a3a2522f52ace0b9241e3ebfa59cdb80003664ad)
* scripts/training/pipeline.pl
* bin/extract-1best
* bin/meteor
* bin/joshua-decoder
* bin/bleu
* jni/kenlm_wrap.cc
JOSHUA-252 Make it possible to use Maven to build Joshua (lewis.mcgibbney: rev 
9475d9432d6b389748d6103243f5a99ec3433192)
* ext/giza-pp/mkcls-v2/KategProblemKBC.cpp
* ext/giza-pp/GIZA++-v2/ATables.cpp
* ext/giza-pp/mkcls-v2/GDAOptimization.cpp
* ext/giza-pp/GIZA++-v2/snt2plain.cpp
* ext/giza-pp/GIZA++-v2/transpair_model3.h
* ext/giza-pp/GIZA++-v2/hmm.h
* ext/giza-pp/GIZA++-v2/MoveSwapMatrix.h
* ext/giza-pp/GIZA++-v2/model3_viterbi.cpp
* ext/giza-pp/GIZA++-v2/ForwardBackward.cpp
* ext/giza-pp/GIZA++-v2/mystl.h
* ext/giza-pp/mkcls-v2/SAOptimization.h
* ext/giza-pp/mkcls-v2/TAOptimization.cpp
* ext/giza-pp/mkcls-v2/RRTOptimization.h
* ext/giza-pp/GIZA++-v2/Vector.h
* ext/giza-pp/mkcls-v2/IterOptimization.cpp
* examples/README.md
* docker/zh-en-hiero/Dockerfile
* ext/giza-pp/GIZA++-v2/transpair_model3.cpp
* .gitmodules
* ext/giza-pp/GIZA++-v2/myassert.h
* ext/giza-pp/GIZA++-v2/TTables.cpp
* ext/giza-pp/GIZA++-v2/alignment.h
* ext/giza-pp/GIZA++-v2/plain2snt.cpp
* ext/giza-pp/mkcls-v2/KategProblemKBC.h
* ext/giza-pp/mkcls-v2/mystl.h
* ext/giza-pp/GIZA++-v2/logprob.cpp
* ext/giza-pp/mkcls-v2/PopOptimization.h
* ext/giza-pp/GIZA++-v2/getSentence.h
* ext/giza-pp/mkcls-v2/Problem.h
* ext/giza-pp/GIZA++-v2/Makefile.src
* ext/giza-pp/GIZA++-v2/utility.h
* ext/giza-pp/GIZA++-v2/FlexArray.h
* ext/giza-pp/GIZA++-v2/model2.h
* ext/giza-pp/mkcls-v2/RRTOptimization.cpp
* ext/giza-pp/GIZA++-v2/snt2cooc.cpp
* ext/giza-pp/mkcls-v2/ProblemTest.h
* ext/giza-pp/GIZA++-v2/model3.h
* ext/giza-pp/GIZA++-v2/reports.cpp
* examples/docker/ar-en-phrase/Dockerfile
* ext/giza-pp/GIZA++-v2/HMMTables.cpp
* ext/giza-pp/mkcls-v2/makePackage.sh
* ext/giza-pp/mkcls-v2/StatVar.cpp
* ext/giza-pp/mkcls-v2/Makefile
* ext/giza-pp/GIZA++-v2/Parameter.cpp
* ext/giza-pp/mkcls-v2/MSBOptimization.h
* ext/symal/Makefile
* ext/giza-pp/mkcls-v2/myleda.h
* docker/Dockerfile
* ext/giza-pp/GIZA++-v2/collCounts.h
* ext/giza-pp/GIZA++-v2/parse.cpp
* ext/giza-pp/GIZA++-v2/TTables.h
* ext/giza-pp/GIZA++-v2/model345-peg.cpp
* ext/giza-pp/GIZA++-v2/vocab.h
* ext/giza-pp/mkcls-v2/KategProblemTest.h
* ext/giza-pp/GIZA++-v2/Perplexity.cpp
* ext/giza-pp/GIZA++-v2/trainGIZA++.sh
* ext/giza-pp/mkcls-v2/general.h
* Dockerfile
* ext/symal/cmd.h
* ext/giza-pp/GIZA++-v2/vocab.cpp
* examples/README.sp_to_en
* ext/berkeleylm
* ext/giza-pp/mkcls-v2/README
* ext/giza-pp/GIZA++-v2/README
* ext/giza-pp/mkcls-v2/MSBOptimization.cpp
* ext/giza-pp/GIZA++-v2/WordClasses.h
* docker/ar-en-phrase/Dockerfile
* ext/symal/symal.cpp
* ext/giza-pp/mkcls-v2/ProblemTest.cpp
* ext/giza-pp/mkcls-v2/LICENSE
* ext/giza-pp/mkcls-v2/FixedArray.h
* ext/giza-pp/GIZA++-v2/file_spec.h
* ext/giza-pp/mkcls-v2/TAOptimization.h
* ext/giza-pp/GIZA++-v2/D5Tables.h
* ext/giza-pp/GIZA++-v2/AlignTables.cpp
* ext/giza-pp/GIZA++-v2/mymath.h
* ext/giza-pp/README
* ext/giza-pp/mkcls-v2/KategProblemTest.cpp
* ext/giza-pp/mkcls-v2/MYOptimization.h
* ext/giza-pp/mkcls-v2/PopOptimization.cpp
* ext/giza-pp/mkcls-v2/mkcls.cpp
* ext/giza-pp/GIZA++-v2/alignment.cpp
* ext/giza-pp/mkcls-v2/Problem.cpp
* ext/giza-pp/mkcls-v2/KategProblemWBC.cpp
* ext/giza-pp/GIZA++-v2/dependencies
* ext/giza-pp/GIZA++-v2/model3.cpp
* examples/docker/zh-en-hiero/Dockerfile
* ext/giza-pp/GIZA++-v2/Array2.h
* ext/giza-pp/GIZA++-v2/model1.h
* ext/giza-pp/GIZA++-v2/model2to3.cpp
* ext/giza-pp/GIZA++-v2/utility.cpp
* ext/giza-pp/mkcls-v2/GDAOptimization.h
* ext/giza-pp/GIZA++-v2/Array4.h
* ext/giza-pp/GIZA++-v2/Dictionary.h
* ext/giza-pp/GIZA++-v2/model2.cpp
* ext/giza-pp/mkcls-v2/Optimization.h
* ext/giza-pp/GIZA++-v2/D4Tables.h
* ext/giza-pp/GIZA++-v2/Array.h
* ext/giza-pp/GIZA++-v2/model1.cpp
* ext/giza-pp/GIZA++-v2/NTables.cpp
* ext/giza-pp/GIZA++-v2/ForwardBackward.h
* ext/giza-pp/GIZA++-v2/Parameter.h
* ext/giza-pp/GIZA++-v2/Perplexity.h
* ext/giza-pp/mkcls-v2/StatVar.h
* ext/giza-pp/GIZA++-v2/logprob.h
* ext/giza-pp/GIZA++-v2/Makefile
* ext/giza-pp/GIZA++-v2/Makefile.definitions
* ext/giza-pp/GIZA++-v2/transpair_modelhmm.h
* ext/giza-pp/GIZA++-v2/HMMTables.h
* ext/giza-pp/GIZA++-v2/Pointer.h
* ext/giza-pp/GIZA++-v2/transpair_model4.h
* ext/giza-pp/GIZA++-v2/collCounts.cpp
* ext/giza-pp/mkcls-v2/myassert.h
* ext/giza-pp/GIZA++-v2/Globals.h
* ext/giza-pp/GIZA++-v2/ATables.h
* ext/giza-pp/GIZA++-v2/myassert.cp

Jenkins build is back to normal : joshua_maven #12

[Apache Jenkins Server]

See 

Podling Report Reminder - May 2016

[johndament]

Dear podling,

This email was sent by an automated system on behalf of the Apache
Incubator PMC. It is an initial reminder to give you plenty of time to
prepare your quarterly board report.

The board meeting is scheduled for Wed, 15 June 2016, 10:30 am PDT.
The report for your podling will form a part of the Incubator PMC
report. The Incubator PMC requires your report to be submitted 2 weeks
before the board meeting, to allow sufficient time for review and
submission (Wed, June 1st).

Please submit your report with sufficient time to allow the Incubator
PMC, and subsequently board members to review and digest. Again, the
very latest you should submit your report is 2 weeks prior to the board
meeting.

Thanks,

The Apache Incubator PMC

Submitting your Report

--

Your report should contain the following:

*   Your project name
*   A brief description of your project, which assumes no knowledge of
the project or necessarily of its field
*   A list of the three most important issues to address in the move
towards graduation.
*   Any issues that the Incubator PMC or ASF Board might wish/need to be
aware of
*   How has the community developed since the last report
*   How has the project developed since the last report.

This should be appended to the Incubator Wiki page at:

http://wiki.apache.org/incubator/May2016

Note: This is manually populated. You may need to wait a little before
this page is created from a template.

Mentors
---

Mentors should review reports for their project(s) and sign them off on
the Incubator wiki page. Signing off reports shows that you are
following the project - projects that are not signed may raise alarms
for the Incubator PMC.

Incubator PMC

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/868b340949f324b12d810d03c09c25fd5877d3dc#commitcomment-17621082
  
As far as I know, you can add the tst folder to the Eclipse build path and 
then select it and click run. Even if not run, they should be added to the 
Eclipse build to see compile errors. But I am not an Eclipse expert :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/19aadf0e240acb09c5b4336068e6368083f26bef#commitcomment-17621052
  
In src/joshua/decoder/Decoder.java:
In src/joshua/decoder/Decoder.java on line 515:
If one uses Joshua as a library, threading could be handled by the 
surrounding codebase. I think supporting both use cases is important.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17621008
  
In src/joshua/decoder/Translations.java:
In src/joshua/decoder/Translations.java on line 49:
Maybe we can use the TranslationFactory for it.
This is just brainstorming, but lets say at Decoder 
initialization/construction the decoder instantiates its member 
translationFactory with the joshuaConfiguration and the factory configures 
itself in the constructor of what to do. We only would need a single instance 
of factory and it would build for all requests. I am not sure if the pattern 
below is nice, we should get some engineering comments. :)

public TranslationFactory {
 private final boolean buildAlignments;
 public TranslationFactory(JoshuaConfiguration config) {
buildAlignments = config.outputFormat.needsAlignments
 }
 public getTranslation(DerivationState derivation, Sentence sentence) {
  Translation t = new Translation(derivation, Sentence);
  if (buildAlignments) {
   t.setWordAlignments(this.buildWordAlignments(derivation));
  }
  return t;
}
 }
}


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/19aadf0e240acb09c5b4336068e6368083f26bef#commitcomment-17620834
  
In src/joshua/decoder/Decoder.java:
In src/joshua/decoder/Decoder.java on line 515:
Yes, builder is a more appropriate design name, will rename. I also like 
the with*() naming.

And yes, that's more logically sensible and contained. I'll give it a whirl 
and see if it passes the regression tests.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/19aadf0e240acb09c5b4336068e6368083f26bef#commitcomment-17620963
  
In src/joshua/decoder/Decoder.java:
In src/joshua/decoder/Decoder.java on line 515:
As for using decode() versus decodeAll(): you wouldn't get threading that 
way, since decode() blocks. Thought you probably have some way of doing this in 
mind. Do you have some idea of where the entry point is for the API? The way 
decodeAll works is, you pass it an object that can be iterated over, and as it 
receives translations, it outputs them. I'm not sure how to do multithreading 
with the API if the entry point is a blocked call to decode(), but probably 
there's a standard way of doing this that I don't know about?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17620761
  
In src/joshua/decoder/hypergraph/KBestExtractor.java:
In src/joshua/decoder/hypergraph/KBestExtractor.java on line 93:
Uh, yeah, that's how I should have done it before. I even complained about 
the logic being in multiple places and had a bug. Will add that now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/19aadf0e240acb09c5b4336068e6368083f26bef#commitcomment-17620734
  
In src/joshua/decoder/Decoder.java:
In src/joshua/decoder/Decoder.java on line 515:
what if one uses Decoder.decode() and not decodeAll()?
Is there a way to put this into the TranslationFactory by passing it a 
reference to the feature functions? If someone calls 
TranslationFactory().translation() it could execute this KenLM hack since at 
this point one knows that the KenLM states can be destroyed.

Also, if TranslationFactory is used in this pattern I would suggest calling 
it TranslationBuilder and adopt the builder pattern:
TranslationBuilder tb = new TranslationBuilder()
Translation translation = 
tb.withAlignments().withFeatures().buildTranslation();


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17620750
  
In src/joshua/decoder/Translations.java:
In src/joshua/decoder/Translations.java on line 49:
That's a very good thought. So really, we should throw away the hypergraph 
earlier on, even before things get queued up. I think that's the right way, but 
that means you have to push all your parameters down into the function, about 
what you're going to want on the output. Any idea how to do that cleanly? Just 
through a JoshuaConfiguration, or something better?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Wiki access

[Tom Barber]

Hello

Can someone give me(bugg_tb) access to the Joshua wiki please.

Ta

Tom
--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17620605
  
In src/joshua/decoder/hypergraph/KBestExtractor.java:
In src/joshua/decoder/hypergraph/KBestExtractor.java on line 93:
I love the idea of this being an iterator. Could we set k in its 
constructor to make sure it stops producing iterations once k or all possible 
derivations are hit?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17620594
  
In src/joshua/decoder/Translations.java:
In src/joshua/decoder/Translations.java on line 49:
I am not sure I have understood it 100%, but what I worry about with this 
change is that when you have to wait for sentence x very long, and other 
requests x+1x+n are already finished, the decoder will hold on to all these 
hypergraphs until the hypergraph for x is complete, its translation is created 
and all hgs can be destroyed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17620554
  
In src/joshua/decoder/Decoder.java:
In src/joshua/decoder/Decoder.java on line 39:
No, not sure who added it


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17620411
  
In src/joshua/decoder/Decoder.java:
In src/joshua/decoder/Decoder.java on line 193:
where is the right side of this assignment coming from?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17620548
  
In src/joshua/decoder/Decoder.java:
In src/joshua/decoder/Decoder.java on line 193:
This is a bug, needs to be restored or further fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f2f82c38af9aebd28f9d27f685a2e99767a2e575#commitcomment-17620359
  
In src/joshua/decoder/Decoder.java:
In src/joshua/decoder/Decoder.java on line 39:
is this required?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300774#comment-15300774
 ] 

Hudson commented on JOSHUA-252:
---

FAILURE: Integrated in joshua_maven #11 (See 
[https://builds.apache.org/job/joshua_maven/11/])
JOSHUA-252 Make it possible to use Maven to build Joshua (lewis.mcgibbney: rev 
d8a68df396d086c75a26133ea3011e4de6bd54b9)
* src/main/java/org/apache/joshua/util/Regex.java
* src/main/java/org/apache/joshua/tools/TestSetFilter.java
* src/main/java/org/apache/joshua/util/ExtractTopCand.java
* src/main/java/org/apache/joshua/util/io/BinaryOut.java
* src/main/java/org/apache/joshua/ui/tree_visualizer/browser/Browser.java
* src/main/java/org/apache/joshua/util/encoding/EncoderConfiguration.java
* src/main/java/org/apache/joshua/util/ChartSpan.java
* src/main/java/org/apache/joshua/tools/GrammarPacker.java
* src/main/java/org/apache/joshua/util/Ngram.java
* src/main/java/org/apache/joshua/util/io/Reader.java
* src/main/java/org/apache/joshua/util/ListUtil.java
* src/main/java/org/apache/joshua/tools/LabelPhrases.java
* src/main/java/org/apache/joshua/ui/tree_visualizer/tree/Tree.java
* src/main/java/org/apache/joshua/util/Bits.java
* src/main/java/org/apache/joshua/util/Counts.java
* src/main/java/org/apache/joshua/util/FormatUtils.java
* src/main/java/org/apache/joshua/util/io/LineReader.java
* src/main/java/org/apache/joshua/util/io/IndexedReader.java
* src/main/java/org/apache/joshua/util/FileUtility.java
* src/main/java/org/apache/joshua/util/Algorithms.java
* src/main/java/org/apache/joshua/util/CompareGrammars.java
* src/main/java/org/apache/joshua/util/Counted.java


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: joshua_maven #11

[Apache Jenkins Server]

See 

Changes:

[lewis.mcgibbney] JOSHUA-252 Make it possible to use Maven to build Joshua

--
Started by user lewismc
[EnvInject] - Loading node environment variables.
Building remotely on ubuntu-4 (docker Ubuntu ubuntu4 ubuntu yahoo-not-h2) in 
workspace 
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/incubator-joshua.git # timeout=10
Fetching upstream changes from 
https://git-wip-us.apache.org/repos/asf/incubator-joshua.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/incubator-joshua.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/JOSHUA-252^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/JOSHUA-252^{commit} # timeout=10
Checking out Revision d8a68df396d086c75a26133ea3011e4de6bd54b9 
(refs/remotes/origin/JOSHUA-252)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f d8a68df396d086c75a26133ea3011e4de6bd54b9
 > git rev-list aead62094032c6ee9f02144713e68e1602e9bed8 # timeout=10
[joshua_maven] $ /home/jenkins/tools/maven/apache-maven-2.2.1/bin/mvn clean 
deploy javadoc:aggregate
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'javadoc'.
[INFO] 
[INFO] Building Apache Joshua Machine Translation Toolkit
[INFO]task-segment: [clean, deploy]
[INFO] 
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting 
[INFO] [remote-resources:process {execution: default}]
[debug] execute contextualize
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Compiling 251 source files to 

[INFO] -
[ERROR] COMPILATION ERROR : 
[INFO] -
[ERROR] 
:[786,20]
 error: no suitable constructor found for 
PhraseTable(String,String,String,JoshuaConfiguration,int)
[ERROR] constructor 
PhraseTable.PhraseTable(String,String,String,JoshuaConfiguration) is not 
applicable
  (actual and formal argument lists differ in length)
constructor PhraseTable.PhraseTable(String,JoshuaConfiguration) is not 
applicable
  (actual and formal argument lists differ in length)
:[803,29]
 error: no suitable constructor found for 
PhraseTable(,String,String,JoshuaConfiguration,int)
[INFO] 2 errors 
[INFO] -
[INFO] 
[ERROR] BUILD FAILURE
[INFO] 
[INFO] Compilation failure

:[786,20]
 error: no suitable constructor found for 
PhraseTable(String,String,String,JoshuaConfiguration,int)
constructor 
PhraseTable.PhraseTable(String,String,String,JoshuaConfiguration) is not 
applicable
  (actual and formal argument lists differ in length)
constructor PhraseTable.PhraseTable(String,JoshuaConfiguration) is not 
applicable
  (actual and formal argument lists differ in length)
:[803,29]
 error: no suitable constructor found for 
PhraseTable(,String,String,JoshuaConfiguration,int)

[INFO] 
[INFO] For more information, run Maven with the -e switch
[INFO] 
[INFO] Total time: 16 seconds
[INFO] Finished at: Wed May 25 20:18:03 UTC 2016
[INFO] Final Memory: 40M/1291M
[INFO] 
Build step 'Invoke top-level Maven targets' marked build as failure
Publishing Javadoc
Updating JOSHUA-252

Re: too many emails

[Henry Saputra]

We can move the Github announcements to different list if it is too noisy

On Wed, May 25, 2016 at 12:48 PM, Matt Post  wrote:

> Does someone know how to turn off the mailing of all github comments to
> dev?
>
> The way I see it, we all have to be on dev, so it should be for people,
> not robots. I am getting every comment about three times.
>
> I would just do it but I don't know how.

too many emails

[Matt Post]

Does someone know how to turn off the mailing of all github comments to dev?

The way I see it, we all have to be on dev, so it should be for people, not 
robots. I am getting every comment about three times.

I would just do it but I don't know how.

[GitHub] incubator-joshua pull request: New Juju charms for deployment proc...

[buggtb]

Github user buggtb commented on the pull request:

https://github.com/apache/incubator-joshua/pull/16#issuecomment-221677077
  
Thanks chaps. 

Because I'm coming at this from a devops angle, I skipped the build phase 
because outside of the software hacker world you wont find sysadmins compiling 
stuff with Ant. Obviously this is something we've touched on on the mailing 
list and I'm sure will come up again.

There's a bunch of extra stuff that I can do to enhance this charm, but its 
a decent starting point.

As an example as to what a user gets, I dumped a test joshua-full 
deployment into the juju charm store:
https://jujucharms.com/u/apachesoftwarefoundation/joshua-full/trusty/0



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/868b340949f324b12d810d03c09c25fd5877d3dc#commitcomment-17618958
  
Yes, I don't know how to run them. Is it an eclipse plugin or an ant 
target? If someone can tell me, I'll start running them.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/76eb9587a9c321d45a9560481429e2dffd918e4e#commitcomment-17618721
  
In src/joshua/decoder/ff/tm/format/HieroFormatReader.java:
In src/joshua/decoder/ff/tm/format/HieroFormatReader.java on line 83:
I started doing this, but it's touching too many files. I will make a note 
to do it after the 252 merge.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618810
  
In src/joshua/tools/GrammarPacker.java:
In src/joshua/tools/GrammarPacker.java on line 295:
Completely agreed :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/868b340949f324b12d810d03c09c25fd5877d3dc#commitcomment-17618752
  
There is a Vocabulary unit test 
(https://github.com/apache/incubator-joshua/blob/master/tst/joshua/corpus/VocabularyTest.java)
 and a FormatUtil Unit test 
(https://github.com/apache/incubator-joshua/blob/master/tst/joshua/util/FormatUtilsTest.java)

These should be updated accordingly. I guess Unit tests are not run 
currently during build.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618502
  
In src/joshua/decoder/ff/tm/format/MosesFormatReader.java:
In src/joshua/decoder/ff/tm/format/MosesFormatReader.java on line 95:
this should probably go at some point or changed to logging


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/76eb9587a9c321d45a9560481429e2dffd918e4e#commitcomment-17618437
  
Great change!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618536
  
In src/joshua/tools/GrammarPacker.java:
In src/joshua/tools/GrammarPacker.java on line 261:
should probably be removed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618628
  
I love this!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618600
  
In src/joshua/tools/GrammarPacker.java:
In src/joshua/tools/GrammarPacker.java on line 242:
Okay. Currently it's doing it in the GrammarReaders, but it's a redundancy. 
I'll make a comment so it's clearer in the code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618788
  
In src/joshua/tools/GrammarPacker.java:
In src/joshua/tools/GrammarPacker.java on line 295:
These are all fine ideas. I think they're the kind of thing that one of us 
should just pick off sometime we're in the code doing something else.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/366f408672e2d29b69a78531b57056649629e978#commitcomment-17618694
  
In src/joshua/decoder/phrase/PhraseTable.java:
In src/joshua/decoder/phrase/PhraseTable.java on line 99:
"[X]" should be defined as a constant somewhere in the project


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618524
  
In src/joshua/tools/GrammarPacker.java:
In src/joshua/tools/GrammarPacker.java on line 242:
I have to look into this. I think we found a case where it was necessary 
but I cannot remember why.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618617
  
In src/joshua/tools/GrammarPacker.java:
In src/joshua/tools/GrammarPacker.java on line 295:
The "\\s+" string should probably defined as a token separator constant 
somewhere in the project to avoid hard-coding it everywhere.
It might be useful to add a method to Vocabulary.java that returns a list 
of strings instead of the joined version to avoid the splitting here:
public List getSourceWordList(int[] ids)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300621#comment-15300621
 ] 

ASF GitHub Bot commented on JOSHUA-252:
---

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/76eb9587a9c321d45a9560481429e2dffd918e4e#commitcomment-17618558
  
In src/joshua/decoder/ff/tm/Rule.java:
In src/joshua/decoder/ff/tm/Rule.java on line 201:
Yes, definitely, will do. I was originally trying to limit the impact of 
the merge given the huge impending merge with JOSHUA-252, but by now, with the 
other stuff, it's too late :)


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/f5adcdefd3e52c58b36713798c0830d1c42099e3#commitcomment-17618489
  
In src/joshua/decoder/ff/tm/format/MosesFormatReader.java:
In src/joshua/decoder/ff/tm/format/MosesFormatReader.java on line 82:
you can initialize the StringBuffer directly with the following line:
new StringBuffer("[X] ||| [X,1] " + fields[0] + " ||| [X,1] " + fields[1] + 
" |||")


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/76eb9587a9c321d45a9560481429e2dffd918e4e#commitcomment-17618411
  
In src/joshua/decoder/ff/tm/hash_based/MemoryBasedBatchGrammar.java:
In src/joshua/decoder/ff/tm/hash_based/MemoryBasedBatchGrammar.java on line 
130:
We could think about making these type strings an enum at some point to 
clearly define what they are.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[mjpost]

Github user mjpost commented on the pull request:


https://github.com/apache/incubator-joshua/commit/76eb9587a9c321d45a9560481429e2dffd918e4e#commitcomment-17618558
  
In src/joshua/decoder/ff/tm/Rule.java:
In src/joshua/decoder/ff/tm/Rule.java on line 201:
Yes, definitely, will do. I was originally trying to limit the impact of 
the merge given the huge impending merge with JOSHUA-252, but by now, with the 
other stuff, it's too late :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/76eb9587a9c321d45a9560481429e2dffd918e4e#commitcomment-17618391
  
In src/joshua/decoder/ff/tm/format/MosesFormatReader.java:
In src/joshua/decoder/ff/tm/format/MosesFormatReader.java on line 45:
this could be made
private static final int lhs = Vocabulary.id("[X]").
However, ideally all 'dependencies' of a class should be instantiated 
before the class is instantiated. We should think at some point about what 
'contracts' we can define for grammars: i.e. when and how do they modify the 
vocabulary? At construction time? At loading time? Later? etc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: joshua API changes

[Matt Post]

Whoops, the JOSHUA-273 (which does the API) piece wasn't pushed up. Here it is:

https://github.com/apache/incubator-joshua/tree/JOSHUA-273

matt

> On May 25, 2016, at 2:07 PM, Matt Post  wrote:
> 
> Hi folks (especially Felix, Kellen, Tobi) — 
> 
> I made two moderate improvements to Joshua on the way home. The first was to 
> get rid of all the specialized phrase handling; the packer now works as we 
> discussed, packing everything into Hiero format, and the stack-based decoder 
> uses this directly now. Everything should be backwards compatible for hiero, 
> but it's not for phrase-based. I added a "version = 3" line to the packer 
> config to distinguish this, along with a check, so the decoder will throw a 
> runtime exception if you try to load something incompatible. If anything 
> fails, instead of repacking your grammar, just add the line "version = 3" to 
> the packer config. The changes only affect packing for phrase-based models, 
> so I don't think it will matter to you. This is pushed up into master.
> 
> The bigger one is on a JOSHUA-273 branch. I just pushed up a refactoring of 
> the KBestExtraction / structured translation interface, per our discussions 
> this week. However, I wasn't actually sure how to use the API. What is the 
> entry point? Are you calling translate() directly and managing your own 
> thread pools? It doesn't seem like you would be using Decoder.decode() or 
> decodeAll(), since they're not very API-ish.
> 
> If you want to take a look at the changes, I'd welcome feedback, direct 
> changes, etc. Here is a description of the major changes:
> 
> - Large refactor of the Translation output interface
> 
> - Instead of returning Translation objects, the calls to Decoder.translate() 
> now return HyperGraph objects. As before, a HyperGraph represents the 
> complete (pruned) search space the decoder explored. A HyperGraph can then be 
> operated on by KBestExtractors and by the new TranslationFactory object, so 
> that it can be thrown away.
> 
> - KBestExtractor is now an iterator that takes a HyperGraph object and 
> returns DerivationState objects, each representing a single derivation tree
> 
> - Translation and StructuredTranslation are now combined. Translation is 
> effectively a dummy object with a number of fields of interest that get 
> populated by TranslationFactory, per explicit requests. Each request returns 
> the TranslationFactory object, so you can easily chain calls, and then 
> retrieve the Translation object at the end. e.g.,
> 
>   KBestExtractor extractor = new KBestExtractor(hg, ...).
>   for (DerivationState derivation: extractor) {
>   TranslationFactory factory = new TranslationFactory(derivation, 
> ...)
>   Translation translation = factory.alignments()
>   
> .formattedTranslation(config.outputFormat)
>   .features()
>   .translation();
>   }
> 
> - Neither KBestExtractors nor Translation objects do any printing. This 
> improved encapsulation is a big improvement over the past. After building 
> your Translation objects, they will contain only small objects such as 
> strings, feature vectors, and alignments, that can be safely passed 
> downstream while the HyperGraph gets destroyed. Also, code for processing and 
> formatting is all now in one place, the TranslationFactory.
> 
> - Also, I removed the forest rescoring and OracleExtraction classes. These 
> are useful but not used, and are hard to read and should therefore be 
> rewritten. I will do this at some point.
> 
> There are still a few things broken on the branch, but they are small and I 
> am working to fix them. If you have a minute to poke around on the branch, 
> please do, so that the end result is what you imagined when we were chatting 
> the other day.
> 
> matt

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/76eb9587a9c321d45a9560481429e2dffd918e4e#commitcomment-17618301
  
In src/joshua/decoder/ff/tm/format/HieroFormatReader.java:
In src/joshua/decoder/ff/tm/format/HieroFormatReader.java on line 83:
to be consistent with the above changes (English/French -> target/source) 
the comment could be adopted


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request:

[fhieber]

Github user fhieber commented on the pull request:


https://github.com/apache/incubator-joshua/commit/76eb9587a9c321d45a9560481429e2dffd918e4e#commitcomment-17618246
  
In src/joshua/decoder/ff/tm/Rule.java:
In src/joshua/decoder/ff/tm/Rule.java on line 201:
Fantasic commit! Would it make sense to refactor these methods to 
setTarget/setSource as well?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Comment Edited] (JOSHUA-272) Simplify the packing and usage of phrase-based grammars

[Matt Post (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300566#comment-15300566
 ] 

Matt Post edited comment on JOSHUA-272 at 5/25/16 6:15 PM:
---

Fixed with [a recent 
commit|https://github.com/apache/incubator-joshua/commit/aef0b2dbe4555070aec9f15bb2c8d9dcb5671dcd].


was (Author: post):
Fixed with [a recent 
comment|https://github.com/apache/incubator-joshua/commit/aef0b2dbe4555070aec9f15bb2c8d9dcb5671dcd].

> Simplify the packing and usage of phrase-based grammars
> ---
>
> Key: JOSHUA-272
> URL: https://issues.apache.org/jira/browse/JOSHUA-272
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Matt Post
>Assignee: Matt Post
> Fix For: 6.1
>
>
> For historical reasons, phrase-based grammars add some complexity to 
> decoding. The complete tree under each top-level trie node in packed grammars 
> has to fit within a single packed grammars slice, which is limited to 2 GB 
> due to constraints on the size of Java byte[] arrays. We used to sort on just 
> the first item in the trie, which was a problem for phrase-based decoding, 
> since phrase-based rules are implemented as left-branching hierarchical 
> rules. In order to pack large grammars, we packed them without the leading 
> [X,1], and then added it when loading the grammars, both for the packed and 
> memory-based grammars. This was a real mess.
> This was all fixed with a commit a while ago that packs and reads packed 
> grammars based on the first two symbols on the source side. So we should 
> remove all the complexity associated with phrases. They should just be 
> regular rules. There is also a lot of redundancy across the codebase in 
> parsing rules, converting them to different formats, and so on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (JOSHUA-272) Simplify the packing and usage of phrase-based grammars

[Matt Post (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JOSHUA-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Post resolved JOSHUA-272.
--
Resolution: Fixed

Fixed with [a recent 
comment|https://github.com/apache/incubator-joshua/commit/aef0b2dbe4555070aec9f15bb2c8d9dcb5671dcd].

> Simplify the packing and usage of phrase-based grammars
> ---
>
> Key: JOSHUA-272
> URL: https://issues.apache.org/jira/browse/JOSHUA-272
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Matt Post
>Assignee: Matt Post
> Fix For: 6.1
>
>
> For historical reasons, phrase-based grammars add some complexity to 
> decoding. The complete tree under each top-level trie node in packed grammars 
> has to fit within a single packed grammars slice, which is limited to 2 GB 
> due to constraints on the size of Java byte[] arrays. We used to sort on just 
> the first item in the trie, which was a problem for phrase-based decoding, 
> since phrase-based rules are implemented as left-branching hierarchical 
> rules. In order to pack large grammars, we packed them without the leading 
> [X,1], and then added it when loading the grammars, both for the packed and 
> memory-based grammars. This was a real mess.
> This was all fixed with a commit a while ago that packs and reads packed 
> grammars based on the first two symbols on the source side. So we should 
> remove all the complexity associated with phrases. They should just be 
> regular rules. There is also a lot of redundancy across the codebase in 
> parsing rules, converting them to different formats, and so on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (JOSHUA-271) Thrax invocation should not reply upon $HADOOP being set

[Matt Post (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JOSHUA-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Post resolved JOSHUA-271.
--
Resolution: Fixed
  Assignee: Matt Post

> Thrax invocation should not reply upon $HADOOP being set
> 
>
> Key: JOSHUA-271
> URL: https://issues.apache.org/jira/browse/JOSHUA-271
> Project: Joshua
>  Issue Type: Bug
>  Components: pipeline, thrax
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Matt Post
> Fix For: 6.1
>
>
> Right now one cannot run thrax unless the $HADOOP env variable is defined. 
> Every time the hadoop script is invoked it means that the path is coded as 
> $HADOOP/bin/hadoop however what happens if you are using a VM (Vagrant) to 
> connect to a cluster for which no $HADOOP env variable is defined? 
> The hadoop script should be on the path and available to use from there. The 
> only check which should be made is whether it is available from the path or 
> not, if it is not then start_hadoop_cluster subroutine can be called. This 
> reduces code and makes more sense.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: New Juju charms for deployment proc...

[mjpost]

Github user mjpost commented on the pull request:

https://github.com/apache/incubator-joshua/pull/16#issuecomment-221659544
  
Awesome, thanks for the PR! Give me a few days to look it over and I'll get 
back to you.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

joshua API changes

[Matt Post]

Hi folks (especially Felix, Kellen, Tobi) — 

I made two moderate improvements to Joshua on the way home. The first was to 
get rid of all the specialized phrase handling; the packer now works as we 
discussed, packing everything into Hiero format, and the stack-based decoder 
uses this directly now. Everything should be backwards compatible for hiero, 
but it's not for phrase-based. I added a "version = 3" line to the packer 
config to distinguish this, along with a check, so the decoder will throw a 
runtime exception if you try to load something incompatible. If anything fails, 
instead of repacking your grammar, just add the line "version = 3" to the 
packer config. The changes only affect packing for phrase-based models, so I 
don't think it will matter to you. This is pushed up into master.

The bigger one is on a JOSHUA-273 branch. I just pushed up a refactoring of the 
KBestExtraction / structured translation interface, per our discussions this 
week. However, I wasn't actually sure how to use the API. What is the entry 
point? Are you calling translate() directly and managing your own thread pools? 
It doesn't seem like you would be using Decoder.decode() or decodeAll(), since 
they're not very API-ish.

If you want to take a look at the changes, I'd welcome feedback, direct 
changes, etc. Here is a description of the major changes:

- Large refactor of the Translation output interface

- Instead of returning Translation objects, the calls to Decoder.translate() 
now return HyperGraph objects. As before, a HyperGraph represents the complete 
(pruned) search space the decoder explored. A HyperGraph can then be operated 
on by KBestExtractors and by the new TranslationFactory object, so that it can 
be thrown away.

- KBestExtractor is now an iterator that takes a HyperGraph object and returns 
DerivationState objects, each representing a single derivation tree

- Translation and StructuredTranslation are now combined. Translation is 
effectively a dummy object with a number of fields of interest that get 
populated by TranslationFactory, per explicit requests. Each request returns 
the TranslationFactory object, so you can easily chain calls, and then retrieve 
the Translation object at the end. e.g.,

KBestExtractor extractor = new KBestExtractor(hg, ...).
for (DerivationState derivation: extractor) {
TranslationFactory factory = new TranslationFactory(derivation, 
...)
Translation translation = factory.alignments()

.formattedTranslation(config.outputFormat)
.features()
.translation();
}

- Neither KBestExtractors nor Translation objects do any printing. This 
improved encapsulation is a big improvement over the past. After building your 
Translation objects, they will contain only small objects such as strings, 
feature vectors, and alignments, that can be safely passed downstream while the 
HyperGraph gets destroyed. Also, code for processing and formatting is all now 
in one place, the TranslationFactory.

- Also, I removed the forest rescoring and OracleExtraction classes. These are 
useful but not used, and are hard to read and should therefore be rewritten. I 
will do this at some point.

There are still a few things broken on the branch, but they are small and I am 
working to fix them. If you have a minute to poke around on the branch, please 
do, so that the end result is what you imagined when we were chatting the other 
day.

matt

[GitHub] incubator-joshua pull request: New Juju charms for deployment proc...

[chrismattmann]

Github user chrismattmann commented on the pull request:

https://github.com/apache/incubator-joshua/pull/16#issuecomment-221657354
  
+1 LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [jira] [Commented] (JOSHUA-270) pipeline.pl needs major refactoring

[Matt Post]

Having written that, factoring the pipeline would be a good first step to 
replacing the guts of the pipeline. It's worth noting that many of these are 
already done:

- alignment is handled by $JOSHUA/scripts/training/paralign.pl
- tuning is handled by $JOSHUA/scripts/training/run_tuner.py
- there is a script for running Thrax ($JOSHUA/scripts/training/run_thrax.py), 
but it is not pulled into the decoder yet

However, Lewis' basic point stands: the pipeline is a mess, and it would be 
good to have good interfaces to each of the subtasks, as an intermediate step 
to replacing the logic of the pipeline with a more versatile (and readable) 
tool like ducttape.

matt


> On May 24, 2016, at 7:27 PM, Matt Post (JIRA)  wrote:
> 
> 
>  [ 
> https://issues.apache.org/jira/browse/JOSHUA-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299141#comment-15299141
>  ] 
> 
> Matt Post commented on JOSHUA-270:
> --
> 
> The pipeline is a huge mess, probably not worth salvaging. I'm hoping (maybe 
> this year?) to rewrite it, perhaps using this: 
> https://github.com/jhclark/ducttape/
> 
>> pipeline.pl needs major refactoring
>> ---
>> 
>>  Key: JOSHUA-270
>>  URL: https://issues.apache.org/jira/browse/JOSHUA-270
>>  Project: Joshua
>>   Issue Type: Bug
>>   Components: pipeline
>> Affects Versions: 6.0.5
>> Reporter: Lewis John McGibbney
>>  Fix For: 6.1
>> 
>> 
>> Right now 
>> [pipeline.pl|https://github.com/apache/incubator-joshua/blob/master/scripts/training/pipeline.pl]
>>  is well over 2000 lines long and extremely difficult to navigate. 
>> I propose the following
>> * All ENV is refactored into an pipeline_environment file
>> * All Command line parsing and definitions are refactored into a 
>> pipeline_cli file
>> * Sanity checking is refactored into a pipeline_sanity_check file
>> * Dependenct Variable Checking is refactored into 
>> pipeline_dependent_variable_setting file
>> * filter and preprocess corpora is refactored into 
>> pipeline_filter_preprocess_corpora
>> * pipeline_subsampling becomes a file
>> * pipeline_alignment becomes a file
>> * pipeline_parsing becomes a file
>> * pipeline_thrax becomes a file
>> * pipeline_tuning becomes a file
>> * pipeline_testing becomes a file
>> * pipeline_subreoutines becomes a file
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)

Re: [jira] [Commented] (JOSHUA-270) pipeline.pl needs major refactoring

[Matt Post]

Having written that, factoring the pipeline would be a good first step to 
replacing the guts of the pipeline. It's worth noting that many of these are 
already done:

- alignment is handled by $JOSHUA/scripts/training/paralign.pl
- tuning is handled by $JOSHUA/scripts/training/run_tuner.py
- there is a script for running Thrax ($JOSHUA/scripts/training/run_thrax.py), 
but it is not pulled into the decoder yet

However, Lewis' basic point stands: the pipeline is a mess, and it would be 
good to have good interfaces to each of the subtasks, as an intermediate step 
to replacing the logic of the pipeline with a more versatile (and readable) 
tool like ducttape.

matt


> On May 24, 2016, at 7:27 PM, Matt Post (JIRA)  wrote:
> 
> 
>   [ 
> https://issues.apache.org/jira/browse/JOSHUA-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299141#comment-15299141
>  ] 
> 
> Matt Post commented on JOSHUA-270:
> --
> 
> The pipeline is a huge mess, probably not worth salvaging. I'm hoping (maybe 
> this year?) to rewrite it, perhaps using this: 
> https://github.com/jhclark/ducttape/
> 
>> pipeline.pl needs major refactoring
>> ---
>> 
>>   Key: JOSHUA-270
>>   URL: https://issues.apache.org/jira/browse/JOSHUA-270
>>   Project: Joshua
>>Issue Type: Bug
>>Components: pipeline
>>  Affects Versions: 6.0.5
>>  Reporter: Lewis John McGibbney
>>   Fix For: 6.1
>> 
>> 
>> Right now 
>> [pipeline.pl|https://github.com/apache/incubator-joshua/blob/master/scripts/training/pipeline.pl]
>>  is well over 2000 lines long and extremely difficult to navigate. 
>> I propose the following
>> * All ENV is refactored into an pipeline_environment file
>> * All Command line parsing and definitions are refactored into a 
>> pipeline_cli file
>> * Sanity checking is refactored into a pipeline_sanity_check file
>> * Dependenct Variable Checking is refactored into 
>> pipeline_dependent_variable_setting file
>> * filter and preprocess corpora is refactored into 
>> pipeline_filter_preprocess_corpora
>> * pipeline_subsampling becomes a file
>> * pipeline_alignment becomes a file
>> * pipeline_parsing becomes a file
>> * pipeline_thrax becomes a file
>> * pipeline_tuning becomes a file
>> * pipeline_testing becomes a file
>> * pipeline_subreoutines becomes a file
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)

***UNCHECKED*** Re: [jira] [Commented] (JOSHUA-270) pipeline.pl needs major refactoring

[Matt Post]

binCRkioZFHru.bin
Description: PGP/MIME Versions Identification

[GitHub] incubator-joshua pull request: Removed a few unneeded allocations

[hsaputra]

Github user hsaputra commented on the pull request:

https://github.com/apache/incubator-joshua/pull/11#issuecomment-221643871
  
Ah no, I mean ASF has github integration that would allow automatic PR 
close when being merged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-252) Make it possible to use Maven to build Joshua

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300353#comment-15300353
 ] 

Hudson commented on JOSHUA-252:
---

FAILURE: Integrated in joshua_maven #10 (See 
[https://builds.apache.org/job/joshua_maven/10/])
JOSHUA-252 Make it possible to use Maven to build Joshua (lewis.mcgibbney: rev 
aead62094032c6ee9f02144713e68e1602e9bed8)
* src/main/java/org/apache/joshua/decoder/segment_file/ConstraintRule.java
* src/main/java/org/apache/joshua/decoder/phrase/TargetPhrases.java
* src/main/java/org/apache/joshua/lattice/Node.java
* src/main/java/org/apache/joshua/decoder/ff/lm/LanguageModelFF.java
* src/main/java/org/apache/joshua/subsample/BiCorpusFactory.java
* src/main/java/org/apache/joshua/decoder/DecoderThread.java
* src/main/java/org/apache/joshua/decoder/phrase/PhraseChart.java
* src/main/java/org/apache/joshua/corpus/syntax/ArraySyntaxTree.java
* src/main/java/org/apache/joshua/oracle/OracleExtractionHG.java
* src/main/java/org/apache/joshua/decoder/segment_file/Sentence.java
* src/main/java/org/apache/joshua/subsample/SubsamplerCLI.java
* src/main/java/org/apache/joshua/server/ServerThread.java
* src/main/java/org/apache/joshua/subsample/AlignedSubsampler.java
* src/main/java/org/apache/joshua/corpus/Corpus.java
* src/main/java/org/apache/joshua/decoder/io/DeNormalize.java
* src/main/java/org/apache/joshua/metrics/EvaluationMetric.java
* src/main/java/org/apache/joshua/corpus/SymbolTable.java
* src/main/java/org/apache/joshua/subsample/BiCorpus.java
* src/main/java/org/apache/joshua/decoder/segment_file/ConstraintSpan.java
* src/main/java/org/apache/joshua/decoder/phrase/Stacks.java
* src/main/java/org/apache/joshua/decoder/ArgsParser.java
* src/main/java/org/apache/joshua/metrics/BLEU.java
* src/main/java/org/apache/joshua/decoder/Decoder.java
* src/main/java/org/apache/joshua/decoder/segment_file/Token.java
* src/main/java/org/apache/joshua/corpus/ContiguousPhrase.java
* src/main/java/org/apache/joshua/decoder/phrase/Stack.java
* src/main/java/org/apache/joshua/corpus/TerminalIterator.java
* src/main/java/org/apache/joshua/decoder/chart_parser/ComputeNodeResult.java
* src/main/java/org/apache/joshua/lattice/Lattice.java
* src/main/java/org/apache/joshua/corpus/Span.java
* src/main/java/org/apache/joshua/corpus/Vocabulary.java
* src/main/java/org/apache/joshua/decoder/ff/fragmentlm/Tree.java
* src/main/java/org/apache/joshua/decoder/ff/lm/KenLM.java
* src/main/java/org/apache/joshua/decoder/phrase/Hypothesis.java
* src/main/java/org/apache/joshua/decoder/phrase/Future.java
* src/main/java/org/apache/joshua/decoder/phrase/PhraseTable.java
* src/main/java/org/apache/joshua/decoder/hypergraph/ViterbiExtractor.java
* src/main/java/org/apache/joshua/decoder/Support.java
* src/main/java/org/apache/joshua/decoder/phrase/Coverage.java
* src/main/java/org/apache/joshua/subsample/Subsampler.java
* src/main/java/org/apache/joshua/decoder/StructuredTranslation.java


> Make it possible to use Maven to build Joshua
> -
>
> Key: JOSHUA-252
> URL: https://issues.apache.org/jira/browse/JOSHUA-252
> Project: Joshua
>  Issue Type: Improvement
>  Components: build
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
> Fix For: 6.1
>
>
> As per discussion on the dev@ list for now Ant is the official build tool for 
> Joshua however we would like to possibly switch to Maven if / when someone is 
> able to do so.
> Assigning to me for now as I could be able to look into this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: joshua_maven #10

[Apache Jenkins Server]

See 

Changes:

[lewis.mcgibbney] JOSHUA-252 Make it possible to use Maven to build Joshua

--
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ubuntu-4 (docker Ubuntu ubuntu4 ubuntu yahoo-not-h2) in 
workspace 
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/incubator-joshua.git # timeout=10
Fetching upstream changes from 
https://git-wip-us.apache.org/repos/asf/incubator-joshua.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/incubator-joshua.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/JOSHUA-252^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/JOSHUA-252^{commit} # timeout=10
Checking out Revision aead62094032c6ee9f02144713e68e1602e9bed8 
(refs/remotes/origin/JOSHUA-252)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f aead62094032c6ee9f02144713e68e1602e9bed8
 > git rev-list 1fc0590e9f6c77c2ff0efbe6453ddd39364141ad # timeout=10
[joshua_maven] $ /home/jenkins/tools/maven/apache-maven-2.2.1/bin/mvn clean 
deploy javadoc:aggregate
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'javadoc'.
[INFO] 
[INFO] Building Apache Joshua Machine Translation Toolkit
[INFO]task-segment: [clean, deploy]
[INFO] 
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting 
[INFO] [remote-resources:process {execution: default}]
[debug] execute contextualize
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Compiling 251 source files to 

[INFO] -
[ERROR] COMPILATION ERROR : 
[INFO] -
[ERROR] 
:[786,20]
 error: no suitable constructor found for 
PhraseTable(String,String,String,JoshuaConfiguration,int)
[ERROR] constructor 
PhraseTable.PhraseTable(String,String,String,JoshuaConfiguration) is not 
applicable
  (actual and formal argument lists differ in length)
constructor PhraseTable.PhraseTable(String,JoshuaConfiguration) is not 
applicable
  (actual and formal argument lists differ in length)
:[803,29]
 error: no suitable constructor found for 
PhraseTable(,String,String,JoshuaConfiguration,int)
[INFO] 2 errors 
[INFO] -
[INFO] 
[ERROR] BUILD FAILURE
[INFO] 
[INFO] Compilation failure

:[786,20]
 error: no suitable constructor found for 
PhraseTable(String,String,String,JoshuaConfiguration,int)
constructor 
PhraseTable.PhraseTable(String,String,String,JoshuaConfiguration) is not 
applicable
  (actual and formal argument lists differ in length)
constructor PhraseTable.PhraseTable(String,JoshuaConfiguration) is not 
applicable
  (actual and formal argument lists differ in length)
:[803,29]
 error: no suitable constructor found for 
PhraseTable(,String,String,JoshuaConfiguration,int)

[INFO] 
[INFO] For more information, run Maven with the -e switch
[INFO] 
[INFO] Total time: 16 seconds
[INFO] Finished at: Wed May 25 16:25:06 UTC 2016
[INFO] Final Memory: 40M/1291M
[INFO] 
Build step 'Invoke top-level Maven targets' marked build as failure
Publishing Javadoc
Updating JOSHUA-252

[jira] [Commented] (JOSHUA-270) pipeline.pl needs major refactoring

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300224#comment-15300224
 ] 

Lewis John McGibbney commented on JOSHUA-270:
-

Yeah I agree. It's pretty cumbersome and way too much code. 

> pipeline.pl needs major refactoring
> ---
>
> Key: JOSHUA-270
> URL: https://issues.apache.org/jira/browse/JOSHUA-270
> Project: Joshua
>  Issue Type: Bug
>  Components: pipeline
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
> Fix For: 6.1
>
>
> Right now 
> [pipeline.pl|https://github.com/apache/incubator-joshua/blob/master/scripts/training/pipeline.pl]
>  is well over 2000 lines long and extremely difficult to navigate. 
> I propose the following
>  * All ENV is refactored into an pipeline_environment file
>  * All Command line parsing and definitions are refactored into a 
> pipeline_cli file
>  * Sanity checking is refactored into a pipeline_sanity_check file
>  * Dependenct Variable Checking is refactored into 
> pipeline_dependent_variable_setting file
>  * filter and preprocess corpora is refactored into 
> pipeline_filter_preprocess_corpora
>  * pipeline_subsampling becomes a file
>  * pipeline_alignment becomes a file
>  * pipeline_parsing becomes a file
>  * pipeline_thrax becomes a file
>  * pipeline_tuning becomes a file
>  * pipeline_testing becomes a file
>  * pipeline_subreoutines becomes a file



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: New Juju charms for deployment proc...

[buggtb]

GitHub user buggtb opened a pull request:

https://github.com/apache/incubator-joshua/pull/16

New Juju charms for deployment process & move dockerfile

Hi folks

You'll have seen my random mailings about using Juju as a deployment 
process and whilst they aren't cleaned up and fully operational yet, I'd like 
to get these off my dev box and into the repo if possible to get them under VCS 
and into the mainline. 

As we're discussing a number of deployment options I also moved the Docker 
file to better support that rather than it just living under the root directory.

LMKWYT!

Tom

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/buggtb/incubator-joshua master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/16.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #16


commit def263c3ac43916614d52db8d0c8e3b58e0d35bc
Author: Tom Barber 
Date:   2016-05-24T21:20:23Z

move dockerfile so all deployment stuff lives in the same place

commit 7d116e56de42e7c620f43f6466306cbd36f13ce0
Author: Tom Barber 
Date:   2016-05-24T21:20:36Z

joshua runtime juju charm

commit ecf5ee3b98ecd91d8afc15fc8c8a411627d7cb6b
Author: Tom Barber 
Date:   2016-05-24T21:27:48Z

Joshua Full charm

commit cb8b555deadcfbecd88de964b1283a2cf8287c90
Author: Tom Barber 
Date:   2016-05-24T21:29:40Z

fix metadata

commit e67a9e44764c210427e875da7b9ca2d6f51263b6
Author: Tom Barber 
Date:   2016-05-25T12:57:38Z

update layer




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (JOSHUA-261) Remove ext directory from source tree

[Matt Post (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JOSHUA-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Post updated JOSHUA-261:
-
Description: 
Right now we have a bunch of cofe bundled in to the 
[ext|https://github.com/apache/incubator-joshua/tree/master/ext] directory. I 
don't think any of this code can be shipped with an Apache Joshua (Incubating) 
release so we need to think about a mechanism for removing it and making Joshua 
work in other ways.

Here is a partial roadmap:

[ ] remove GIZA++ and symal
[ ] update [the developer 
documentation|https://cwiki.apache.org/confluence/display/JOSHUA/Development] 
to describe how to install them and put them in the path
[ ] update the pipeline scripts to not be hard-coded to $JOSHUA/bin
[ ] update the build files to not try to build them

  was:
Right now we have a bunch of cofe bundled in to the 
[ext|https://github.com/apache/incubator-joshua/tree/master/ext] directory. I 
don't think any of this code can be shipped with an Apache Joshua (Incubating) 
release so we need to think about a mechanism for removing it and making Joshua 
work in other ways.

Here is a partial roadmap:

[ ] remove GIZA++ and symal
[ ] update [the developer 
documentation](https://cwiki.apache.org/confluence/display/JOSHUA/Development) 
to describe how to install them and put them in the path
[ ] update the pipeline scripts to not be hard-coded to $JOSHUA/bin
[ ] update the build files to not try to build them


> Remove ext directory from source tree
> -
>
> Key: JOSHUA-261
> URL: https://issues.apache.org/jira/browse/JOSHUA-261
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Right now we have a bunch of cofe bundled in to the 
> [ext|https://github.com/apache/incubator-joshua/tree/master/ext] directory. I 
> don't think any of this code can be shipped with an Apache Joshua 
> (Incubating) release so we need to think about a mechanism for removing it 
> and making Joshua work in other ways.
> Here is a partial roadmap:
> [ ] remove GIZA++ and symal
> [ ] update [the developer 
> documentation|https://cwiki.apache.org/confluence/display/JOSHUA/Development] 
> to describe how to install them and put them in the path
> [ ] update the pipeline scripts to not be hard-coded to $JOSHUA/bin
> [ ] update the build files to not try to build them



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (JOSHUA-261) Remove ext directory from source tree

[Matt Post (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JOSHUA-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Post updated JOSHUA-261:
-
Description: 
Right now we have a bunch of cofe bundled in to the 
[ext|https://github.com/apache/incubator-joshua/tree/master/ext] directory. I 
don't think any of this code can be shipped with an Apache Joshua (Incubating) 
release so we need to think about a mechanism for removing it and making Joshua 
work in other ways.

Here is a partial roadmap:

[ ] remove GIZA++ and symal
[ ] update [the developer 
documentation](https://cwiki.apache.org/confluence/display/JOSHUA/Development) 
to describe how to install them and put them in the path
[ ] update the pipeline scripts to not be hard-coded to $JOSHUA/bin
[ ] update the build files to not try to build them

  was:Right now we have a bunch of cofe bundled in to the 
[ext|https://github.com/apache/incubator-joshua/tree/master/ext] directory. I 
don't think any of this code can be shipped with an Apache Joshua (Incubating) 
release so we need to think about a mechanism for removing it and making Joshua 
work in other ways.


> Remove ext directory from source tree
> -
>
> Key: JOSHUA-261
> URL: https://issues.apache.org/jira/browse/JOSHUA-261
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Priority: Blocker
> Fix For: 6.1
>
>
> Right now we have a bunch of cofe bundled in to the 
> [ext|https://github.com/apache/incubator-joshua/tree/master/ext] directory. I 
> don't think any of this code can be shipped with an Apache Joshua 
> (Incubating) release so we need to think about a mechanism for removing it 
> and making Joshua work in other ways.
> Here is a partial roadmap:
> [ ] remove GIZA++ and symal
> [ ] update [the developer 
> documentation](https://cwiki.apache.org/confluence/display/JOSHUA/Development)
>  to describe how to install them and put them in the path
> [ ] update the pipeline scripts to not be hard-coded to $JOSHUA/bin
> [ ] update the build files to not try to build them



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (JOSHUA-266) Refactor key interfaces and core code for a future release.

[Matt Post (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JOSHUA-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Post closed JOSHUA-266.

Resolution: Duplicate

> Refactor key interfaces and core code for a future release. 
> 
>
> Key: JOSHUA-266
> URL: https://issues.apache.org/jira/browse/JOSHUA-266
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Kellen Sunderland
>Priority: Minor
>
> We've discussed making some modifications to the key interfaces.  This ticket 
> can focus on making large changes to the codebase for a future release.  This 
> work will likely take some time and some collaboration.  I'd suggest that the 
> code for this be contained in a separate release branch.
> Some issues we can work on:
> *  I'd propose we conform to the SOLID principles for our major interfaces.  
> https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)  . 
> *  We can look at Sparse / Dense feature vectors and how to handle them 
> naturally in Joshua.
> *  Refactor objects that may now be used more broadly than was originally 
> intended (for example Vocabulary class).
> *  We should have a general discussion around what parts of the codebase are 
> responsible for what functions.  We should clearly define what logic should 
> be a part of the Grammar versus the Feature Functions for example, and make 
> sure logic doesn't leak from one of these objects to the others.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-266) Refactor key interfaces and core code for a future release.

[Matt Post (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299609#comment-15299609
 ] 

Matt Post commented on JOSHUA-266:
--

Closing this as a duplicate of JOSHUA-255.

> Refactor key interfaces and core code for a future release. 
> 
>
> Key: JOSHUA-266
> URL: https://issues.apache.org/jira/browse/JOSHUA-266
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Kellen Sunderland
>Priority: Minor
>
> We've discussed making some modifications to the key interfaces.  This ticket 
> can focus on making large changes to the codebase for a future release.  This 
> work will likely take some time and some collaboration.  I'd suggest that the 
> code for this be contained in a separate release branch.
> Some issues we can work on:
> *  I'd propose we conform to the SOLID principles for our major interfaces.  
> https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)  . 
> *  We can look at Sparse / Dense feature vectors and how to handle them 
> naturally in Joshua.
> *  Refactor objects that may now be used more broadly than was originally 
> intended (for example Vocabulary class).
> *  We should have a general discussion around what parts of the codebase are 
> responsible for what functions.  We should clearly define what logic should 
> be a part of the Grammar versus the Feature Functions for example, and make 
> sure logic doesn't leak from one of these objects to the others.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (JOSHUA-272) Simplify the packing and usage of phrase-based grammars

[Matt Post (JIRA)]

Matt Post created JOSHUA-272:


 Summary: Simplify the packing and usage of phrase-based grammars
 Key: JOSHUA-272
 URL: https://issues.apache.org/jira/browse/JOSHUA-272
 Project: Joshua
  Issue Type: Improvement
Reporter: Matt Post
Assignee: Matt Post
 Fix For: 6.1


For historical reasons, phrase-based grammars add some complexity to decoding. 
The complete tree under each top-level trie node in packed grammars has to fit 
within a single packed grammars slice, which is limited to 2 GB due to 
constraints on the size of Java byte[] arrays. We used to sort on just the 
first item in the trie, which was a problem for phrase-based decoding, since 
phrase-based rules are implemented as left-branching hierarchical rules. In 
order to pack large grammars, we packed them without the leading [X,1], and 
then added it when loading the grammars, both for the packed and memory-based 
grammars. This was a real mess.

This was all fixed with a commit a while ago that packs and reads packed 
grammars based on the first two symbols on the source side. So we should remove 
all the complexity associated with phrases. They should just be regular rules. 
There is also a lot of redundancy across the codebase in parsing rules, 
converting them to different formats, and so on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)