[ https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695594#comment-13695594 ]
Hudson commented on HBASE-6295: ------------------------------- Integrated in HBase-TRUNK #4201 (See [https://builds.apache.org/job/HBase-TRUNK/4201/]) HBASE-6295 Possible performance improvement in client batch operations: presplit and send in background - addendum (Revision 1497800) Result = SUCCESS nkeywal : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/test/resources/hbase-site.xml > Possible performance improvement in client batch operations: presplit and > send in background > -------------------------------------------------------------------------------------------- > > Key: HBASE-6295 > URL: https://issues.apache.org/jira/browse/HBASE-6295 > Project: HBase > Issue Type: Improvement > Components: Client, Performance > Affects Versions: 0.95.2 > Reporter: Nicolas Liochon > Assignee: Nicolas Liochon > Labels: noob > Fix For: 0.98.0, 0.95.2 > > Attachments: 6295.addendum.patch, 6295.v11.patch, 6295.v12.patch, > 6295.v14.patch, 6295.v15.patch, 6295.v1.patch, 6295.v2.patch, 6295.v3.patch, > 6295.v4.patch, 6295.v5.patch, 6295.v6.patch, 6295.v8.patch, 6295.v9.patch > > > today batch algo is: > {noformat} > for Operation o: List<Op>{ > add o to todolist > if todolist > maxsize or o last in list > split todolist per location > send split lists to region servers > clear todolist > wait > } > {noformat} > We could: > - create immediately the final object instead of an intermediate array > - split per location immediately > - instead of sending when the list as a whole is full, send it when there is > enough data for a single location > It would be: > {noformat} > for Operation o: List<Op>{ > get location > add o to todo location.todolist > if (location.todolist > maxLocationSize) > send location.todolist to region server > clear location.todolist > // don't wait, continue the loop > } > send remaining > wait > {noformat} > It's not trivial to write if you add error management: retried list must be > shared with the operations added in the todolist. But it's doable. > It's interesting mainly for 'big' writes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira