[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395137#comment-15395137 ] Yu Li commented on HBASE-15619: --- Here is the improvement on SCC, as promised: HDFS-10690 What's more, HDFS-9668 is not enough to resolve the mentioned problem, so we've done more work but still in testing phase, JFYI. > Performance regression observed: Empty random read(get) performance of > branch-1 worse than 0.98 > --- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: compare.png, flamegraph-108588.098.svg, > flamegraph-1221.branch-1.svg, flamegraph-135684.1.1.svg > > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * -case #2: lrucache 100% hit- > * -case #3: BLOCKCACHE=>false- > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|383562|-257493-|-47594-| > |1.1.2+|363050|-232757-|-35872-| > * AverageLatency(us) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|2774|-4134-|-22371-| > |1.1.2+|2930|-4572-|-29690-| > It seems there's perf regression on RPCServer (we tried 0.98 client against > 1.x server and observed a similar perf to 1.x client) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335287#comment-15335287 ] Yu Li commented on HBASE-15619: --- I see, then HBASE-15971 is more likely to resolve the problem here since I did the test with multiple YCSB instances.:-) I'll for sure try HBASE-15971 out and update the result here (maybe days later though due to online supporting work...). Thanks for the information [~stack] > Performance regression observed: Empty random read(get) performance of > branch-1 worse than 0.98 > --- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: compare.png, flamegraph-108588.098.svg, > flamegraph-1221.branch-1.svg, flamegraph-135684.1.1.svg > > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * -case #2: lrucache 100% hit- > * -case #3: BLOCKCACHE=>false- > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|383562|-257493-|-47594-| > |1.1.2+|363050|-232757-|-35872-| > * AverageLatency(us) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|2774|-4134-|-22371-| > |1.1.2+|2930|-4572-|-29690-| > It seems there's perf regression on RPCServer (we tried 0.98 client against > 1.x server and observed a similar perf to 1.x client) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335276#comment-15335276 ] Yu Li commented on HBASE-15619: --- Yep, the whole work is still not completely done but still something to share in advance: 1. HDFS-9668 - When network-card bandwidth enhanced to 10Gb, there would be much higher IO pressure on disk, and HDD is more frequently exhausted with 100% util. In this case, we found SSD read/write also affected and RT reached as high as ~10s. HDFS-9668 is the correct way to resolve it 2. Improvements on ShortCircuitCache (SCC) - We found some locking problem on SCC and my workmate already made a fix, testing done on our side but still not opening JIRA, will add the link as soon as JIRA created for this. > Performance regression observed: Empty random read(get) performance of > branch-1 worse than 0.98 > --- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: compare.png, flamegraph-108588.098.svg, > flamegraph-1221.branch-1.svg, flamegraph-135684.1.1.svg > > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * -case #2: lrucache 100% hit- > * -case #3: BLOCKCACHE=>false- > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|383562|-257493-|-47594-| > |1.1.2+|363050|-232757-|-35872-| > * AverageLatency(us) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|2774|-4134-|-22371-| > |1.1.2+|2930|-4572-|-29690-| > It seems there's perf regression on RPCServer (we tried 0.98 client against > 1.x server and observed a similar perf to 1.x client) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334835#comment-15334835 ] stack commented on HBASE-15619: --- [~carp84] I think the difference between my tests here and HBASE-15971 are that in HBASE-15971 I was trying to overload the regionserver and had many YCSB instances running whereas on the former, I had a single instance running. Doing the overloading seems to have brought out the issue you saw. Have you tried HBASE-15971 on your setup? Hopefully you'll see an improvement... > Performance regression observed: Empty random read(get) performance of > branch-1 worse than 0.98 > --- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: compare.png, flamegraph-108588.098.svg, > flamegraph-1221.branch-1.svg, flamegraph-135684.1.1.svg > > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * -case #2: lrucache 100% hit- > * -case #3: BLOCKCACHE=>false- > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|383562|-257493-|-47594-| > |1.1.2+|363050|-232757-|-35872-| > * AverageLatency(us) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|2774|-4134-|-22371-| > |1.1.2+|2930|-4572-|-29690-| > It seems there's perf regression on RPCServer (we tried 0.98 client against > 1.x server and observed a similar perf to 1.x client) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334452#comment-15334452 ] Enis Soztutar commented on HBASE-15619: --- bq. We checked the jstack and found some locks on the HDFS layer (which I guess only emerges with PCIe-SSD), and our people already find a way to improve that. Any jiras that you can refer to [~carp84]? > Performance regression observed: Empty random read(get) performance of > branch-1 worse than 0.98 > --- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: compare.png, flamegraph-108588.098.svg, > flamegraph-1221.branch-1.svg, flamegraph-135684.1.1.svg > > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * -case #2: lrucache 100% hit- > * -case #3: BLOCKCACHE=>false- > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|383562|-257493-|-47594-| > |1.1.2+|363050|-232757-|-35872-| > * AverageLatency(us) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|2774|-4134-|-22371-| > |1.1.2+|2930|-4572-|-29690-| > It seems there's perf regression on RPCServer (we tried 0.98 client against > 1.x server and observed a similar perf to 1.x client) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333640#comment-15333640 ] Yu Li commented on HBASE-15619: --- It seems HBASE-15971 has resolved the problem. > Performance regression observed: Empty random read(get) performance of > branch-1 worse than 0.98 > --- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: compare.png, flamegraph-108588.098.svg, > flamegraph-1221.branch-1.svg, flamegraph-135684.1.1.svg > > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * -case #2: lrucache 100% hit- > * -case #3: BLOCKCACHE=>false- > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|383562|-257493-|-47594-| > |1.1.2+|363050|-232757-|-35872-| > * AverageLatency(us) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|2774|-4134-|-22371-| > |1.1.2+|2930|-4572-|-29690-| > It seems there's perf regression on RPCServer (we tried 0.98 client against > 1.x server and observed a similar perf to 1.x client) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236485#comment-15236485 ] Yu Li commented on HBASE-15619: --- Sharp eyes! But it's indeed the case. We checked the jstack and found some locks on the HDFS layer (which I guess only emerges with PCIe-SSD), and our people already find a way to improve that. Will link the issue here as soon as we create any HDFS JIRA (probably days later though). > Performance regression observed: Empty random read(get) performance of > branch-1 worse than 0.98 > --- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: compare.png, flamegraph-108588.098.svg, > flamegraph-1221.branch-1.svg, flamegraph-135684.1.1.svg > > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * -case #2: lrucache 100% hit- > * -case #3: BLOCKCACHE=>false- > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|383562|-257493-|-47594-| > |1.1.2+|363050|-232757-|-35872-| > * AverageLatency(us) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|2774|-4134-|-22371-| > |1.1.2+|2930|-4572-|-29690-| > It seems there's perf regression on RPCServer (we tried 0.98 client against > 1.x server and observed a similar perf to 1.x client) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236482#comment-15236482 ] Yu Li commented on HBASE-15619: --- Thanks for the double check [~stack]! bq. I'm thinking that us being bad at reading non-existent values is a problem but not a critical issue. What you think Yu Li? Agreed, and make sense to knock it down from critical. bq. Seems like 1.1 is about the same as 0.98 otherwise (I thought it was much better). Same feeling, it's a pity, but also means we still have the space to improve. :-) bq. flight recording might give better detail. I didn't do this. Let me know if you want me to (you'd probably be better-off doing it yourself Yu Li if bad performance reading an empty table is important for you). Let me further dig into the empty reading case and get back here if any findings, what you've done is already very helpful, thank you sir! > Performance regression observed: Empty random read(get) performance of > branch-1 worse than 0.98 > --- > > Key: HBASE-15619 > URL: https://issues.apache.org/jira/browse/HBASE-15619 > Project: HBase > Issue Type: Bug >Reporter: Yu Li >Assignee: Yu Li > Attachments: compare.png, flamegraph-108588.098.svg, > flamegraph-1221.branch-1.svg, flamegraph-135684.1.1.svg > > > As titled, I observed the perf regression in the final stress testing before > upgrading our online cluster to 1.x. More details as follows: > 1. HBase version in the comparison test: > * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is > the most important perf-related one (especially under high stress) > * 1.x: checked 3 releases in total > 1) 1.1.2 with important perf fixes/improvements including HBASE-15031 > and HBASE-14465 > 2) 1.1.4 release > 3) 1.2.1RC1 > 2. Test environment > * YCSB: 0.7.0 with > [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied > * Client: 4 physical nodes, each with 8 YCSB instance, each instance with > 100 threads > * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap > * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same > hardware for client and server > 3. Test cases > * -p fieldcount=1 -p fieldlength=128 -p readproportion=1 > * case #1: read against empty table > * -case #2: lrucache 100% hit- > * -case #3: BLOCKCACHE=>false- > 4. Test result > * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so > will only paste comparison data of 0.98.12+ and 1.1.2+ > * per-RS Throughput(ops/s) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|383562|-257493-|-47594-| > |1.1.2+|363050|-232757-|-35872-| > * AverageLatency(us) > ||HBaseVersion||case#1||-case#2-||-case#3-|| > |0.98.12+|2774|-4134-|-22371-| > |1.1.2+|2930|-4572-|-29690-| > It seems there's perf regression on RPCServer (we tried 0.98 client against > 1.x server and observed a similar perf to 1.x client) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15619) Performance regression observed: Empty random read(get) performance of branch-1 worse than 0.98
[
https://issues.apache.org/jira/browse/HBASE-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235702#comment-15235702
]
Vladimir Rodionov commented on HBASE-15619:
---
With hardware:
{quote}
Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same
hardware for client and server
{quote}
only 47K reads from file system? for 3 RS? 15.7K per server? for PCIe-SSD? That
is ridiculously low number.
> Performance regression observed: Empty random read(get) performance of
> branch-1 worse than 0.98
> ---
>
> Key: HBASE-15619
> URL: https://issues.apache.org/jira/browse/HBASE-15619
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
>
> As titled, I observed the perf regression in the final stress testing before
> upgrading our online cluster to 1.x. More details as follows:
> 1. HBase version in the comparison test:
> * 0.98: based on 0.98.12 with some backports, among which HBASE-11297 is
> the most important perf-related one (especially under high stress)
> * 1.x: checked 3 releases in total
> 1) 1.1.2 with important perf fixes/improvements including HBASE-15031
> and HBASE-14465
> 2) 1.1.4 release
> 3) 1.2.1RC1
> 2. Test environment
> * YCSB: 0.7.0 with
> [YCSB-651|https://github.com/brianfrankcooper/YCSB/pull/651] applied
> * Client: 4 physical nodes, each with 8 YCSB instance, each instance with
> 100 threads
> * Server: 1 Master with 3 RS, each RS with 256 handlers and 64G heap
> * Hardware: 64-core CPU, 256GB Mem, 10Gb Net, 1 PCIe-SSD and 11 HDD, same
> hardware for client and server
> 3. Test cases
> * -p fieldcount=1 -p fieldlength=128 -p readproportion=1
> * case #1: read against empty table
> * -case #2: lrucache 100% hit-
> * -case #3: BLOCKCACHE=>false-
> 4. Test result
> * 1.1.4 and 1.2.1 have a similar perf (less than 2% deviation) as 1.1.2+, so
> will only paste comparison data of 0.98.12+ and 1.1.2+
> * per-RS Throughput(ops/s)
> ||HBaseVersion||case#1||-case#2-||-case#3-||
> |0.98.12+|383562|-257493-|-47594-|
> |1.1.2+|363050|-232757-|-35872-|
> * AverageLatency(us)
> ||HBaseVersion||case#1||-case#2-||-case#3-||
> |0.98.12+|2774|-4134-|-22371-|
> |1.1.2+|2930|-4572-|-29690-|
> It seems there's perf regression on RPCServer (we tried 0.98 client against
> 1.x server and observed a similar perf to 1.x client)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
