[jira] [Commented] (JOSHUA-258) Add back penn-treebank-(de)tokenizer perl scripts

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308852#comment-15308852
 ] 

ASF GitHub Bot commented on JOSHUA-258:
---

Github user lewismc closed the pull request at:

https://github.com/apache/incubator-joshua/pull/8


> Add back penn-treebank-(de)tokenizer perl scripts
> -
>
> Key: JOSHUA-258
> URL: https://issues.apache.org/jira/browse/JOSHUA-258
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> I've been working with the 
> [joshua_translation_engine|https://github.com/joshua-decoder/joshua_translation_engine]
>  (which is friggin excellent, we will definately be standing this up on 
> something more heavyweight in the near future) and recently reported [issue 
> 15|https://github.com/joshua-decoder/joshua_translation_engine/issues/15]
> This issue therefore proposes to add back in penn-treebank-(de)tokenizer perl 
> scripts which were removed between 6.0.4 and 6.0.5 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-258) Add back penn-treebank-(de)tokenizer perl scripts

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273071#comment-15273071
 ] 

Lewis John McGibbney commented on JOSHUA-258:
-

Will keep this issue open for the time being then. Please feel free to close 
off it you have a solution in mind. Thanks

> Add back penn-treebank-(de)tokenizer perl scripts
> -
>
> Key: JOSHUA-258
> URL: https://issues.apache.org/jira/browse/JOSHUA-258
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> I've been working with the 
> [joshua_translation_engine|https://github.com/joshua-decoder/joshua_translation_engine]
>  (which is friggin excellent, we will definately be standing this up on 
> something more heavyweight in the near future) and recently reported [issue 
> 15|https://github.com/joshua-decoder/joshua_translation_engine/issues/15]
> This issue therefore proposes to add back in penn-treebank-(de)tokenizer perl 
> scripts which were removed between 6.0.4 and 6.0.5 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-258) Add back penn-treebank-(de)tokenizer perl scripts

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263260#comment-15263260
 ] 

Lewis John McGibbney commented on JOSHUA-258:
-

Cool

> Add back penn-treebank-(de)tokenizer perl scripts
> -
>
> Key: JOSHUA-258
> URL: https://issues.apache.org/jira/browse/JOSHUA-258
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> I've been working with the 
> [joshua_translation_engine|https://github.com/joshua-decoder/joshua_translation_engine]
>  (which is friggin excellent, we will definately be standing this up on 
> something more heavyweight in the near future) and recently reported [issue 
> 15|https://github.com/joshua-decoder/joshua_translation_engine/issues/15]
> This issue therefore proposes to add back in penn-treebank-(de)tokenizer perl 
> scripts which were removed between 6.0.4 and 6.0.5 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-258) Add back penn-treebank-(de)tokenizer perl scripts

[Matt Post (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263258#comment-15263258
 ] 

Matt Post commented on JOSHUA-258:
--

Yes, it's been too-long neglected! I've been thinking this might be good to 
include in the language packs. I want to update it to implement the Google 
Translate API, with some extensions added by Philipp Koehn that allow it to 
work with CasmaCat, an interactive MT tool.

> Add back penn-treebank-(de)tokenizer perl scripts
> -
>
> Key: JOSHUA-258
> URL: https://issues.apache.org/jira/browse/JOSHUA-258
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> I've been working with the 
> [joshua_translation_engine|https://github.com/joshua-decoder/joshua_translation_engine]
>  (which is friggin excellent, we will definately be standing this up on 
> something more heavyweight in the near future) and recently reported [issue 
> 15|https://github.com/joshua-decoder/joshua_translation_engine/issues/15]
> This issue therefore proposes to add back in penn-treebank-(de)tokenizer perl 
> scripts which were removed between 6.0.4 and 6.0.5 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-258) Add back penn-treebank-(de)tokenizer perl scripts

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263253#comment-15263253
 ] 

ASF GitHub Bot commented on JOSHUA-258:
---

Github user mjpost commented on the pull request:

https://github.com/apache/incubator-joshua/pull/8#issuecomment-215592405
  
As noted there, this was moved to $JOSHUA/scripts/preparation/tokenize.pl 
(detokenize.pl), for the 6.1 release (part of an attempt to impose some 
organization on the scripts under scripts/training, which is getting unwieldy). 
I'll fix it there.


> Add back penn-treebank-(de)tokenizer perl scripts
> -
>
> Key: JOSHUA-258
> URL: https://issues.apache.org/jira/browse/JOSHUA-258
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> I've been working with the 
> [joshua_translation_engine|https://github.com/joshua-decoder/joshua_translation_engine]
>  (which is friggin excellent, we will definately be standing this up on 
> something more heavyweight in the near future) and recently reported [issue 
> 15|https://github.com/joshua-decoder/joshua_translation_engine/issues/15]
> This issue therefore proposes to add back in penn-treebank-(de)tokenizer perl 
> scripts which were removed between 6.0.4 and 6.0.5 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-258) Add back penn-treebank-(de)tokenizer perl scripts

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263079#comment-15263079
 ] 

ASF GitHub Bot commented on JOSHUA-258:
---

Github user lewismc commented on the pull request:

https://github.com/apache/incubator-joshua/pull/8#issuecomment-215571337
  
I also noted this issue over on 
https://github.com/joshua-decoder/joshua_translation_engine/issues/15


> Add back penn-treebank-(de)tokenizer perl scripts
> -
>
> Key: JOSHUA-258
> URL: https://issues.apache.org/jira/browse/JOSHUA-258
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> I've been working with the 
> [joshua_translation_engine|https://github.com/joshua-decoder/joshua_translation_engine]
>  (which is friggin excellent, we will definately be standing this up on 
> something more heavyweight in the near future) and recently reported [issue 
> 15|https://github.com/joshua-decoder/joshua_translation_engine/issues/15]
> This issue therefore proposes to add back in penn-treebank-(de)tokenizer perl 
> scripts which were removed between 6.0.4 and 6.0.5 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-258) Add back penn-treebank-(de)tokenizer perl scripts

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263078#comment-15263078
 ] 

ASF GitHub Bot commented on JOSHUA-258:
---

GitHub user lewismc opened a pull request:

https://github.com/apache/incubator-joshua/pull/8

JOSHUA-258 Add back penn-treebank-(de)tokenizer perl scripts

This issue addresses https://issues.apache.org/jira/browse/JOSHUA-258

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lewismc/incubator-joshua JOSHUA-258

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/8.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #8


commit c43b8b023f4380159812ffa1ab03d6a0c646aaee
Author: Lewis John McGibbney 
Date:   2016-04-28T21:44:02Z

JOSHUA-258 Add back penn-treebank-(de)tokenizer perl scripts




> Add back penn-treebank-(de)tokenizer perl scripts
> -
>
> Key: JOSHUA-258
> URL: https://issues.apache.org/jira/browse/JOSHUA-258
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> I've been working with the 
> [joshua_translation_engine|https://github.com/joshua-decoder/joshua_translation_engine]
>  (which is friggin excellent, we will definately be standing this up on 
> something more heavyweight in the near future) and recently reported [issue 
> 15|https://github.com/joshua-decoder/joshua_translation_engine/issues/15]
> This issue therefore proposes to add back in penn-treebank-(de)tokenizer perl 
> scripts which were removed between 6.0.4 and 6.0.5 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)