Comparing the relative performance of 0.98.4 RC0 and 0.98.0 on Hadoop 2.2.0
using YCSB.
This will be the last report of these from me for a while, as I will be
losing my current access to EC2 resources tomorrow.
5 concurrent YCSB clients on 5 servers target 100,000 ops/second in
aggregate. Reported average values are averages of readings from all
clients over 3 runs. Min values are the minimum reported by any client on
any run. Max and percentile values are the maximum reported by any client
on any run. What is interesting is relative differences, because each EC2
testbed has a varying baseline. 0.98.0 and 0.98.4 tests were run on the
same instance set.
These tests were run with no security coprocessors installed, using HFile
V2. The workload E results are a concern. *It appears we have a 23% decline
in measured scan throughput and an 23% increase in average op time from 27
ms to 35 ms. *This does not correspond to any active security feature
(though that could worsen results potentially, untested) so is something
changed in core code. Other workloads are not affected so this is something
specific to scanning. Perhaps delete tracking.
*Hardware and Versions*
Hadoop 2.2.0
HBase 0.98.0-hadoop2 + HBASE-11277
HBase 0.98.4-hadoop2 RC0
YCSB 1.0.4
11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients
32 cores
60 GB RAM
2 x 320 GB directly attached SSD
NameNode: 4 GB heap
DataNode: 1 GB heap
Master: 1 GB heap
RegionServer: 8 GB heap, 24 GB bucket cache offheap engine
*Methodology*
Setup:
0. Start cluster
1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" }
2. YCSB: Preload 100 million rows into table "seed"
3. shell: flush "seed" ; compact "seed"
4. Wait for compaction to complete
5. shell: create_snapshot "seed", "seed_snap"
6. shell: disable "seed"
For each test:
7. shell: clone_snapshot "seed_snap", "test"
8. YCSB: On each client (5 clients), run test -p
operationcount=2000000 -threads 20 -target 20000
9. shell: disable "test"
10. shell: drop "test"
*Workload A*
*0.98.0* *0.98.4*
[OVERALL] RunTime(ms) 100743 100693 [OVERALL] Throughput(ops/sec) 99263
99312 [UPDATE] Operations 4997918 4999620 [UPDATE] AverageLatency(us) 633
647 [UPDATE] MinLatency(us) 269 268 [UPDATE] MaxLatency(us) 1450432 713191
[UPDATE] 95thPercentileLatency(ms) 0 0 [UPDATE] 99thPercentileLatency(ms) 5
4 [READ] Operations 5002242 5000540 [READ] AverageLatency(us) 151 144
[READ] MinLatency(us) 0 0 [READ] MaxLatency(us) 1104157 952392 [READ]
95thPercentileLatency(ms) 0 0 [READ] 99thPercentileLatency(ms) 0 0
*Workload B*
[OVERALL] RunTime(ms) 100465 100458 [OVERALL] Throughput(ops/sec) 99537
99544 [UPDATE] Operations 9499627 9499891 [UPDATE] AverageLatency(us) 556
589 [UPDATE] MinLatency(us) 268 264 [UPDATE] MaxLatency(us) 709604 695863
[UPDATE] 95thPercentileLatency(ms) 0 0 [UPDATE] 99thPercentileLatency(ms) 1
2 [READ] Operations 500533 500269 [READ] AverageLatency(us) 147 144
[READ] MinLatency(us) 0 0 [READ] MaxLatency(us) 571294 495148 [READ]
95thPercentileLatency(ms) 0 0 [READ] 99thPercentileLatency(ms) 0 0
*Workload C*
[OVERALL] RunTime(ms) 100091 100022 [OVERALL] Throughput(ops/sec) 99909
99978 [READ] Operations 9916831 10000000 [READ] AverageLatency(us) 524 526
[READ] MinLatency(us) 273 269 [READ] MaxLatency(us) 737108 741634 [READ]
95thPercentileLatency(ms) 0 0 [READ] 99thPercentileLatency(ms) 1 2
*Workload D*
[OVERALL] RunTime(ms) 114244 103308 [OVERALL] Throughput(ops/sec) 89114
96809 [INSERT] Operations 9499965 9500306 [INSERT] AverageLatency(us) 1145
668 [INSERT] MinLatency(us) 270 271 [INSERT] MaxLatency(us) 4598999
3291540 [INSERT] 95thPercentileLatency(ms) 6 1 [INSERT]
99thPercentileLatency(ms) 13 3 [READ] Operations 500035 499694 [READ]
AverageLatency(us) 14 15 [READ] MinLatency(us) 4 4 [READ] MaxLatency(us)
494730 495198 [READ] 95thPercentileLatency(ms) 0 0 [READ]
99thPercentileLatency(ms) 0 0
*Workload E*
[OVERALL] RunTime(ms) 1600910 2078826 [OVERALL] Throughput(ops/sec) 6308
4835 [INSERT] Operations 499131 500322 [INSERT] AverageLatency(us) 14 17
[INSERT] MinLatency(us) 5 5 [INSERT] MaxLatency(us) 506079 564468 [INSERT]
95thPercentileLatency(ms) 0 0 [INSERT] 99thPercentileLatency(ms) 0 0
[SCAN] Operations 9500869 9499678 [SCAN] AverageLatency(us)
26636 34620 [SCAN] MinLatency(us) 746 755 [SCAN] MaxLatency(us) 8067864
4615914 [SCAN] 95thPercentileLatency(ms) 117 136 [SCAN]
99thPercentileLatency(ms) 169 187
*Workload F*
[OVERALL] RunTime(ms) 100876 100820 [OVERALL] Throughput(ops/sec) 99133
99187 [UPDATE] Operations 10000000 10000000 [UPDATE] AverageLatency(us)
737 746 [UPDATE] MinLatency(us) 273 272 [UPDATE] MaxLatency(us) 759812
747124 [UPDATE] 95thPercentileLatency(ms) 1 1 [UPDATE]
99thPercentileLatency(ms) 5 6 [READ-MODIFY-WRITE] Operations 5000370
5000082 [READ-MODIFY-WRITE] AverageLatency(us) 742 750 [READ-MODIFY-WRITE]
MinLatency(us) 280 279 [READ-MODIFY-WRITE] MaxLatency(us) 756180 747197
[READ-MODIFY-WRITE] 95thPercentileLatency(ms) 1 1 [READ-MODIFY-WRITE]
99thPercentileLatency(ms) 5 6 [READ] Operations 5000530 5000242 [READ]
AverageLatency(us) 22 17 [READ] MinLatency(us) 0 0 [READ] MaxLatency(us)
1551953 1097394 [READ] 95thPercentileLatency(ms) 0 0 [READ]
99thPercentileLatency(ms) 0 0
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
