Haha. It explains results. Performance does batch instead of single put with full round trip. I can't do it right now. I create htable per request using single hconnection for the whole app.
четверг, 13 августа 2015 г. пользователь Vladimir Rodionov написал: > --autoFlush=true is ignored in PE. Make sure you run your own tests with > autoFlush= false in HTable. > > -Vlad > > On Thu, Aug 13, 2015 at 1:22 AM, Serega Sheypak <serega.shey...@gmail.com > <javascript:;>> > wrote: > > > Hi. I used PerformanceEvaluation randomWrite for perf measurement. > > > > Here are my metrics: > > > > -- Timers > > ---------------------------------------------------------------------- > > > > .putTimer > > > > count = 3944591 > > > > mean rate = 12389.71 calls/second > > > > 1-minute rate = 8853.79 calls/second > > > > 5-minute rate = 9800.49 calls/second > > > > 15-minute rate = 7816.68 calls/second > > > > min = 1.85 milliseconds > > > > max = 2524.64 milliseconds > > > > mean = 17.36 milliseconds > > > > stddev = 135.73 milliseconds > > > > median = 6.18 milliseconds > > > > 75% <= 10.41 milliseconds > > > > 95% <= 24.38 milliseconds > > > > 98% <= 32.82 milliseconds > > > > 99% <= 40.77 milliseconds > > > > 99.9% <= 2524.64 milliseconds > > > > > > And here are metrics from randomWrite: > > > > > > ## sudo -u hdfs hbase org.apache.hadoop.hbase.PerformanceEvaluation > > --nomapred --flushCommits=true --autoFlush=true --oneCon=true --latency > > randomWrite 10 > > > > 15/08/13 00:30:49 INFO hbase.PerformanceEvaluation: Finished TestClient-4 > > in 461794ms over 1048576 rows > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: > > 9437184/10485700/10485760, latency mean=440.17, min=1.00, max=5525520.00, > > stdDev=20817.84, 95th=2.00, 99th=3.00 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > latency > > log (microseconds), on 1048576 measures > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Min > > = 1.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Avg > > = 440.1488666534424 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > StdDev > > = 20817.257168459604 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 50th > > = 2.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 75th > > = 2.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 95th > > = 2.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 99th > > = 3.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > 99.9th > > = 3884.876000001328 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > 99.99th > > = 922522.8667998798 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > > 99.999th = 2311970.1423621834 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Max > > = 5525520.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > > valueSize after 0 measures > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Min > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Avg > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > StdDev > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 50th > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 75th > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 95th > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest 99th > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > 99.9th > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > 99.99th > > = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest > > 99.999th = 0.0 > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: RandomWriteTest Max > > = 0.0 > > > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: [RandomWriteTest] > > Summary of timings (ms): [458553, 459646, 461004, 458014, 461794, 455945, > > 458489, 457333, 461017, 463250] > > > > 15/08/13 00:30:51 INFO hbase.PerformanceEvaluation: [RandomWriteTest] > Min: > > 455945ms Max: 463250ms Avg: 459504ms > > > > So my 99th percentile is 40MS, randowWrite shows 4ms > > > > What could be the reason for 10 times diff? > > My code does random put (Random string as a key), the size of put is > > hundreds of bytes. > > >
