----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9511/ -----------------------------------------------------------
Review request for pig, Alan Gates, Prashant Sharma, Jonathan Coveney, and Gunther Hagleitner. Description ------- PIG-3190 adds two 'sane' tokenizers to Pig This addresses bug PIG-3190. https://issues.apache.org/jira/browse/PIG-3190 Diffs ----- ivy.xml 70e8d50 src/org/apache/pig/builtin/LuceneTokenize.java PRE-CREATION src/org/apache/pig/builtin/SnowballTokenize.java PRE-CREATION test/org/apache/pig/test/TestLuceneTokenize.java PRE-CREATION test/org/apache/pig/test/TestSnowballTokenize.java PRE-CREATION test/org/apache/pig/test/data/ExpectedLuceneTokens.txt PRE-CREATION test/org/apache/pig/test/data/ExpectedSnowballTokens.txt PRE-CREATION test/org/apache/pig/test/data/InputFiles/ten_enron_emails.txt PRE-CREATION Diff: https://reviews.apache.org/r/9511/diff/ Testing ------- Runs locally for me, two unit tests pass Thanks, Russell Jurney