[ https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13685492#comment-13685492 ]
Jean-Marc Spaggiari commented on HBASE-6295: -------------------------------------------- Here are the results for your version: org.apache.hadoop.hbase.PerformanceEvaluation$RandomReadTest 728486.5 org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange100Test 22197.4 org.apache.hadoop.hbase.PerformanceEvaluation$RandomSeekScanTest 137125.4 org.apache.hadoop.hbase.PerformanceEvaluation$RandomWriteTest 69712 org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest 24342.1 Trunk: org.apache.hadoop.hbase.PerformanceEvaluation$RandomReadTest 757343.8 org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange100Test 21856.6 org.apache.hadoop.hbase.PerformanceEvaluation$RandomSeekScanTest 134333 org.apache.hadoop.hbase.PerformanceEvaluation$RandomWriteTest 112591.3 org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest 77897.975 Comparaison (The smaller the better): org,apache,hadoop,hbase,PerformanceEvaluation$RandomReadTest -4% org,apache,hadoop,hbase,PerformanceEvaluation$RandomScanWithRange100Test 2% org,apache,hadoop,hbase,PerformanceEvaluation$RandomSeekScanTest 2% org,apache,hadoop,hbase,PerformanceEvaluation$RandomWriteTest -62% org,apache,hadoop,hbase,PerformanceEvaluation$SequentialWriteTest -220% I now have IntegrationTestBigLinkedList running and will run more. I keep you posted. > Possible performance improvement in client batch operations: presplit and > send in background > -------------------------------------------------------------------------------------------- > > Key: HBASE-6295 > URL: https://issues.apache.org/jira/browse/HBASE-6295 > Project: HBase > Issue Type: Improvement > Components: Client, Performance > Affects Versions: 0.95.2 > Reporter: Nicolas Liochon > Assignee: Nicolas Liochon > Labels: noob > Fix For: 0.98.0 > > Attachments: 6295.v11.patch, 6295.v12.patch, 6295.v1.patch, > 6295.v2.patch, 6295.v3.patch, 6295.v4.patch, 6295.v5.patch, 6295.v6.patch, > 6295.v8.patch, 6295.v9.patch > > > today batch algo is: > {noformat} > for Operation o: List<Op>{ > add o to todolist > if todolist > maxsize or o last in list > split todolist per location > send split lists to region servers > clear todolist > wait > } > {noformat} > We could: > - create immediately the final object instead of an intermediate array > - split per location immediately > - instead of sending when the list as a whole is full, send it when there is > enough data for a single location > It would be: > {noformat} > for Operation o: List<Op>{ > get location > add o to todo location.todolist > if (location.todolist > maxLocationSize) > send location.todolist to region server > clear location.todolist > // don't wait, continue the loop > } > send remaining > wait > {noformat} > It's not trivial to write if you add error management: retried list must be > shared with the operations added in the todolist. But it's doable. > It's interesting mainly for 'big' writes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira