[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15232660#comment-15232660 ]
Yu Li commented on HBASE-15619: ------------------------------- Thanks for the hint [~vrodionov]. Have you done any comparison between 2.0 and 0.98 also? Are hot spots observed in HBASE-15540 only for branch-1/2.0, or maybe existing issues all along? > Performance regression observed: Random read(get) performance of branch-1 > worse than 0.98 > ----------------------------------------------------------------------------------------- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug > Reporter: Yu Li > Assignee: Yu Li > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 (we planed to upgrade to 1.x since Oct. last year, by when > 1.1.2 was the latest stable release in branch-1) > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * case #2: lrucache 100% hit > * case #3: BLOCKCACHE=>false > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||case#2||case#3|| > |0.98.12+|383562|257493|47594| > |1.1.2+|363050|232757|35872| > * AverageLatency(us) > ||HBaseVersion||case#1||case#2||case#3|| > |0.98.12+|2774|4134|22371| > |1.1.2+|2930|4572|29690| > It seems to me each part on the read path has perf regression: RPCServer for > case#1, lrucache read for case#2, and hfile read for case#3... -- This message was sent by Atlassian JIRA (v6.3.4#6332)