Thanks for the update, Andrew.
On Fri, Jul 18, 2014 at 2:07 PM, Andrew Purtell <apurt...@apache.org> wrote: > Workload E tests on a new testbed instance with careful attention to > configuration do not produce the same results. We have instead: > > *Workload E* - 0.98.4RC0 > > > > > > [OVERALL] RunTime(ms)1270229 [OVERALL]Throughput(ops/sec) > 7944[INSERT] Operations > 499175 [INSERT]AverageLatency(us) > 18[INSERT]MinLatency(us) > 5[INSERT] MaxLatency(us) > 571160 [INSERT]95thPercentileLatency(ms) > 0[INSERT] 99thPercentileLatency(ms) > 0 [SCAN]Operations > 9500825[SCAN]AverageLatency(us) > > > 21089[SCAN] MinLatency(us) > 772 [SCAN]MaxLatency(us) > 3300020[SCAN]95thPercentileLatency(ms) > 107[SCAN] 99thPercentileLatency(ms) > 152 > > I ran workload E a few times to insure the results were consistent. They > vary a bit due to natural variance but not by 23%. > > > On Thu, Jul 17, 2014 at 5:08 PM, Andrew Purtell <apurt...@apache.org> > wrote: > > > Comparing the relative performance of 0.98.4 RC0 and 0.98.0 on Hadoop > > 2.2.0 using YCSB. > > > > This will be the last report of these from me for a while, as I will be > > losing my current access to EC2 resources tomorrow. > > > > 5 concurrent YCSB clients on 5 servers target 100,000 ops/second in > > aggregate. Reported average values are averages of readings from all > > clients over 3 runs. Min values are the minimum reported by any client on > > any run. Max and percentile values are the maximum reported by any client > > on any run. What is interesting is relative differences, because each EC2 > > testbed has a varying baseline. 0.98.0 and 0.98.4 tests were run on the > > same instance set. > > > > These tests were run with no security coprocessors installed, using HFile > > V2. The workload E results are a concern. *It appears we have a 23% > > decline in measured scan throughput and an 23% increase in average op > time > > from 27 ms to 35 ms. *This does not correspond to any active security > > feature (though that could worsen results potentially, untested) so is > > something changed in core code. Other workloads are not affected so this > is > > something specific to scanning. Perhaps delete tracking. > > > > > > *Hardware and Versions* > > > > Hadoop 2.2.0 > > > > HBase 0.98.0-hadoop2 + HBASE-11277 > > > > HBase 0.98.4-hadoop2 RC0 > > > > YCSB 1.0.4 > > > > > > 11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients > > > > 32 cores > > > > 60 GB RAM > > > > 2 x 320 GB directly attached SSD > > > > NameNode: 4 GB heap > > > > DataNode: 1 GB heap > > > > Master: 1 GB heap > > > > RegionServer: 8 GB heap, 24 GB bucket cache offheap engine > > > > > > *Methodology* > > > > > > Setup: > > > > 0. Start cluster > > 1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" } > > 2. YCSB: Preload 100 million rows into table "seed" > > 3. shell: flush "seed" ; compact "seed" > > 4. Wait for compaction to complete > > 5. shell: create_snapshot "seed", "seed_snap" > > 6. shell: disable "seed" > > > > > > For each test: > > > > 7. shell: clone_snapshot "seed_snap", "test" > > 8. YCSB: On each client (5 clients), run test -p > > operationcount=2000000 -threads 20 -target 20000 > > 9. shell: disable "test" > > 10. shell: drop "test" > > > > > > > > *Workload A* > > *0.98.0* *0.98.4* > > > > > > > > [OVERALL] RunTime(ms) 100743 100693 [OVERALL] Throughput(ops/sec) 99263 > > 99312 [UPDATE] Operations 4997918 4999620 [UPDATE] AverageLatency(us) > > 633 647 [UPDATE] MinLatency(us) 269 268 [UPDATE] MaxLatency(us) 1450432 > > 713191 [UPDATE] 95thPercentileLatency(ms) 0 0 [UPDATE] > > 99thPercentileLatency(ms) 5 4 [READ] Operations 5002242 5000540 [READ] > > AverageLatency(us) 151 144 [READ] MinLatency(us) 0 0 [READ] > > MaxLatency(us) 1104157 952392 [READ] 95thPercentileLatency(ms) 0 0 > > [READ] 99thPercentileLatency(ms) 0 0 > > > > > > > > *Workload B* > > > > > > > > > > > > > > [OVERALL] RunTime(ms) 100465 100458 [OVERALL] Throughput(ops/sec) 99537 > > 99544 [UPDATE] Operations 9499627 9499891 [UPDATE] AverageLatency(us) > > 556 589 [UPDATE] MinLatency(us) 268 264 [UPDATE] MaxLatency(us) 709604 > > 695863 [UPDATE] 95thPercentileLatency(ms) 0 0 [UPDATE] > > 99thPercentileLatency(ms) 1 2 [READ] Operations 500533 500269 [READ] > > AverageLatency(us) 147 144 [READ] MinLatency(us) 0 0 [READ] > > MaxLatency(us) 571294 495148 [READ] 95thPercentileLatency(ms) 0 0 > [READ] > > 99thPercentileLatency(ms) 0 0 > > > > > > > > *Workload C* > > > > > > > > > > > > > > [OVERALL] RunTime(ms) 100091 100022 [OVERALL] Throughput(ops/sec) 99909 > > 99978 [READ] Operations 9916831 10000000 [READ] AverageLatency(us) 524 > > 526 [READ] MinLatency(us) 273 269 [READ] MaxLatency(us) 737108 741634 > > [READ] 95thPercentileLatency(ms) 0 0 [READ] 99thPercentileLatency(ms) 1 > 2 > > > > > > > > *Workload D* > > > > > > > > > > > > > > [OVERALL] RunTime(ms) 114244 103308 [OVERALL] Throughput(ops/sec) 89114 > > 96809 [INSERT] Operations 9499965 9500306 [INSERT] AverageLatency(us) > > 1145 668 [INSERT] MinLatency(us) 270 271 [INSERT] MaxLatency(us) > 4598999 > > 3291540 [INSERT] 95thPercentileLatency(ms) 6 1 [INSERT] > > 99thPercentileLatency(ms) 13 3 [READ] Operations 500035 499694 [READ] > > AverageLatency(us) 14 15 [READ] MinLatency(us) 4 4 [READ] > MaxLatency(us) > > 494730 495198 [READ] 95thPercentileLatency(ms) 0 0 [READ] > > 99thPercentileLatency(ms) 0 0 > > > > > > > > *Workload E* > > > > > > > > > > > > > > [OVERALL] RunTime(ms) 1600910 2078826 [OVERALL] Throughput(ops/sec) > 6308 > > 4835 [INSERT] Operations 499131 500322 [INSERT] AverageLatency(us) 14 > 17 > > [INSERT] MinLatency(us) 5 5 [INSERT] MaxLatency(us) 506079 564468 > > [INSERT] 95thPercentileLatency(ms) 0 0 [INSERT] > 99thPercentileLatency(ms) > > 0 0 [SCAN] Operations 9500869 9499678 [SCAN] AverageLatency(us) > > > > > > 26636 34620 [SCAN] MinLatency(us) 746 755 [SCAN] MaxLatency(us) 8067864 > > 4615914 [SCAN] 95thPercentileLatency(ms) 117 136 [SCAN] > > 99thPercentileLatency(ms) 169 187 > > > > > > > > *Workload F* > > > > > > > > > > > > > > [OVERALL] RunTime(ms) 100876 100820 [OVERALL] Throughput(ops/sec) 99133 > > 99187 [UPDATE] Operations 10000000 10000000 [UPDATE] AverageLatency(us) > > 737 746 [UPDATE] MinLatency(us) 273 272 [UPDATE] MaxLatency(us) 759812 > > 747124 [UPDATE] 95thPercentileLatency(ms) 1 1 [UPDATE] > > 99thPercentileLatency(ms) 5 6 [READ-MODIFY-WRITE] Operations 5000370 > > 5000082 [READ-MODIFY-WRITE] AverageLatency(us) 742 750 > > [READ-MODIFY-WRITE] MinLatency(us) 280 279 [READ-MODIFY-WRITE] > > MaxLatency(us) 756180 747197 [READ-MODIFY-WRITE] > > 95thPercentileLatency(ms) 1 1 [READ-MODIFY-WRITE] > > 99thPercentileLatency(ms) 5 6 [READ] Operations 5000530 5000242 [READ] > > AverageLatency(us) 22 17 [READ] MinLatency(us) 0 0 [READ] > MaxLatency(us) > > 1551953 1097394 [READ] 95thPercentileLatency(ms) 0 0 [READ] > > 99thPercentileLatency(ms) 0 0 > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >