[
https://issues.apache.org/jira/browse/HBASE-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637353#action_12637353
]
stack commented on HBASE-576:
-----------------------------
Thanks J-D. Patch looks good. Pity couldn't be fixed better but yeah, would
need migration script. As is will save a bunch of churn. Let me commit it.
Looking at rpc, I see I broke it a while back; I removed the very reason we
subclass RPC. I replaced all our carefully planted HbaseObjectWritables with
default ObjectWritables. Means we're sending Strings instead of codes for our
parameter names.
So, did a test where a cluster had 1M rows loaded into 11 regions spread over 3
machines. A single client could random-read at ~482/second. Using above patch
and running with 8 threads, was able to read at 1531/second. Basic formula:
throughput can be multiplied by # of threads up to maximum of number of cluster
members: e.g. if 8 threads but only 3 servers, can only see 3X throughput
improvement. If 8 servers hosting regions, should see 8X.
> Investigate IPC performance
> ---------------------------
>
> Key: HBASE-576
> URL: https://issues.apache.org/jira/browse/HBASE-576
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.1.0, 0.1.1, 0.1.2, 0.2.0
> Reporter: Jim Kellerman
> Assignee: stack
> Attachments: htd.patch, pe.patch
>
>
> Turning off all file I/O, and running the PerformanceEvaluation test, of
> 1,048,576 sequential writes to HBase managed to achieve only 7,285 IPCs per
> second.
> Running PerformanceEvaluation sequential write test modified to do an abort
> instead of a commit, it was possible to do 68,337 operations per second. We
> are obviously spending a lot of time doing IPCs.
> We need to investigate to find the bottleneck. Marshalling and unmarshalling?
> Socket setup and teardown?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
