[ https://issues.apache.org/jira/browse/HBASE-6372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427830#comment-13427830 ]
Zhihong Ted Yu commented on HBASE-6372: --------------------------------------- Lars did some work in HBASE-5659 to improve TestAtomicOperation >From >https://builds.apache.org/job/PreCommit-HBASE-Build/2459//testReport/org.apache.hadoop.hbase.regionserver/TestAtomicOperation/testMultiRowMutationMultiThreads/: {code} 2012-07-31 09:22:36,252 DEBUG [Thread-124] regionserver.TestAtomicOperation$2(398): [rowA/colfamily11:qual1/4069/Put/vlen=6/ts=4278, rowB/colfamily11:qual1/4073/Put/vlen=6/ts=0] Exception in thread "Thread-124" junit.framework.AssertionFailedError at junit.framework.Assert.fail(Assert.java:48) at junit.framework.Assert.fail(Assert.java:56) at org.apache.hadoop.hbase.regionserver.TestAtomicOperation$2.run(TestAtomicOperation.java:400) {code} This corresponds to the following: {code} if (r.size() != 1) { LOG.debug(r); failures.incrementAndGet(); fail(); {code} Maybe Lars has better idea about the above. About the patch: {code} + + " -Dhbase.mapreduce.export.batch=10"); {code} Please use EXPORT_BATCHING in place of duplicate config parameter name above. nit: {code} + final static String EXPORT_BATCHING="hbase.mapreduce.export.batch"; {code} Introduce spaces around = sign. > Add scanner batching to Export job > ---------------------------------- > > Key: HBASE-6372 > URL: https://issues.apache.org/jira/browse/HBASE-6372 > Project: HBase > Issue Type: Improvement > Components: mapreduce > Affects Versions: 0.96.0, 0.94.2 > Reporter: Lars George > Assignee: Shengsheng Huang > Priority: Minor > Labels: newbie > Attachments: HBASE-6372.2.patch, HBASE-6372.patch > > > When a single row is too large for the RS heap then an OOME can take out the > entire RS. Setting scanner batching in custom scans helps avoiding this > scenario, but for the supplied Export job this is not set. > Similar to HBASE-3421 we can set the batching to a low number - or if needed > make it a command line option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira