[
https://issues.apache.org/jira/browse/JOSHUA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263253#comment-15263253
]
ASF GitHub Bot commented on JOSHUA-258:
---------------------------------------
Github user mjpost commented on the pull request:
https://github.com/apache/incubator-joshua/pull/8#issuecomment-215592405
As noted there, this was moved to $JOSHUA/scripts/preparation/tokenize.pl
(detokenize.pl), for the 6.1 release (part of an attempt to impose some
organization on the scripts under scripts/training, which is getting unwieldy).
I'll fix it there.
> Add back penn-treebank-(de)tokenizer perl scripts
> -------------------------------------------------
>
> Key: JOSHUA-258
> URL: https://issues.apache.org/jira/browse/JOSHUA-258
> Project: Joshua
> Issue Type: Task
> Affects Versions: 6.0.5
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> I've been working with the
> [joshua_translation_engine|https://github.com/joshua-decoder/joshua_translation_engine]
> (which is friggin excellent, we will definately be standing this up on
> something more heavyweight in the near future) and recently reported [issue
> 15|https://github.com/joshua-decoder/joshua_translation_engine/issues/15]
> This issue therefore proposes to add back in penn-treebank-(de)tokenizer perl
> scripts which were removed between 6.0.4 and 6.0.5
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
