[
https://issues.apache.org/jira/browse/PIG-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031012#comment-13031012
]
Dmitriy V. Ryaboy commented on PIG-1825:
----------------------------------------
The patch is really straightforward and the test doesn't actually test the
patch, except to make sure the argument doesn't break parsing. WAL behavior is
not actually verified.
Two things we can do here:
1) make a createPut() method in HBStorage, call it from putNext(), and in a
test create our own HBS, call createPut(), and check that put.getWriteToWal()
returns the right value
2) ignore the trivial test.
Option 1 is the right thing to do, 2 I can probably be convinced of. As is we
shouldn't commit, since the test just adds extra time to unit tests without
doing much useful work.
> ability to turn off the write ahead log for pig's HBaseStorage
> --------------------------------------------------------------
>
> Key: PIG-1825
> URL: https://issues.apache.org/jira/browse/PIG-1825
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.8.0
> Reporter: Corbin Hoenes
> Priority: Minor
> Attachments: HBaseStorage_noWAL.patch, PIG-1825_1.patch
>
>
> Added an option to allow a caller of HBaseStorage to turn off the
> WriteAheadLog feature while doing bulk loads into hbase.
> From the performance tuning wikipage:
> http://wiki.apache.org/hadoop/PerformanceTuning
> "To speed up the inserts in a non critical job (like an import job), you can
> use Put.writeToWAL(false) to bypass writing to the write ahead log."
> We've tested this on HBase 0.20.6 and it helps dramatically.
> The -noWAL options is passed in just like other options for hbase storage:
> STORE myalias INTO 'MyTable' USING
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('mycolumnfamily:field1
> mycolumnfamily:field2','-noWAL');
> This would be my first patch so please educate me with any steps I need to
> do.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
