[
https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13696843#comment-13696843
]
Nicolas Liochon commented on HBASE-6295:
----------------------------------------
[~eclark] I'm currently running the tests on a real cluster with 15 nodes. I'm
using this command
{code}
bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver -r
IntegrationTestDataIngestSlowDeterministic
{code}
With an empty config file on the client (hence using the code defaults, as
YCSB).
I reproduced an error, but I had this in the logs (it wasn't the last line in
the logs):
2013-07-01 14:26:05,020 WARN [hbase-table-pool-13-thread-1]
client.AsyncProcess: Attempt #20/20 failed for 1 operations on server
ip-10-191-62-44.ec2.internal,60020,1372259546637 NOT resubmitting.,
tableName=IntegrationTestDataIngestSlowDeterministic,
location=region=IntegrationTestDataIngestSlowDeterministic,6eeeeee8,1372688561696.f12d243ed7420921efe5fa30471c102b.,
hostname=ip-10-191-62-44.ec2.internal,60020,1372259546637, seqNum=1
We're very aggressive with the retries at the beginning: the first retries are
after a 100ms sleeping time, and even after the 16th retry we still wait for
6.4s. May be it's too aggressive. [~sershe], are these back off times to be
expected, or should I get more when I call errorsByServer.calculateBackoffTime ?
> Possible performance improvement in client batch operations: presplit and
> send in background
> --------------------------------------------------------------------------------------------
>
> Key: HBASE-6295
> URL: https://issues.apache.org/jira/browse/HBASE-6295
> Project: HBase
> Issue Type: Improvement
> Components: Client, Performance
> Affects Versions: 0.95.2
> Reporter: Nicolas Liochon
> Assignee: Nicolas Liochon
> Labels: noob
> Fix For: 0.98.0, 0.95.2
>
> Attachments: 6295.addendum.patch, 6295.v11.patch, 6295.v12.patch,
> 6295.v14.patch, 6295.v15.patch, 6295.v1.patch, 6295.v2.patch, 6295.v3.patch,
> 6295.v4.patch, 6295.v5.patch, 6295.v6.patch, 6295.v8.patch, 6295.v9.patch,
> hbase-ycsb-workloads Build time trend.png
>
>
> today batch algo is:
> {noformat}
> for Operation o: List<Op>{
> add o to todolist
> if todolist > maxsize or o last in list
> split todolist per location
> send split lists to region servers
> clear todolist
> wait
> }
> {noformat}
> We could:
> - create immediately the final object instead of an intermediate array
> - split per location immediately
> - instead of sending when the list as a whole is full, send it when there is
> enough data for a single location
> It would be:
> {noformat}
> for Operation o: List<Op>{
> get location
> add o to todo location.todolist
> if (location.todolist > maxLocationSize)
> send location.todolist to region server
> clear location.todolist
> // don't wait, continue the loop
> }
> send remaining
> wait
> {noformat}
> It's not trivial to write if you add error management: retried list must be
> shared with the operations added in the todolist. But it's doable.
> It's interesting mainly for 'big' writes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
