bharathv edited a comment on pull request #3371: URL: https://github.com/apache/hbase/pull/3371#issuecomment-858820006
> lockless but the likes of #putIfAbsent are not; they lock the 'bucket' the key is in. Nice catch, this coarse grained segment locking thing didn't cross my mind, thanks for the correction. I did run the WAL PE, here are the results, there is like ~1% drop for async WAL but performance was better in FSHLog case. Following are the results for the default AsyncFSWAL implementation. cc: @apurtell **bin/hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation -threads 256 -roll 10000 -verify** **With Patch:** ``` -- Histograms ------------------------------------------------------------------ org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.latencyHistogram.nanos count = 10463419 min = 2490066 max = 69582818 mean = 4095968.77 stddev = 5892795.05 median = 3486862.00 75% <= 3664389.00 95% <= 4136964.00 98% <= 4732630.00 99% <= 33418377.00 99.9% <= 68981951.00 org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncCountHistogram.countPerSync count = 102788 min = 52 max = 103 mean = 101.87 stddev = 2.54 median = 102.00 75% <= 102.00 95% <= 102.00 98% <= 102.00 99% <= 103.00 99.9% <= 103.00 org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncHistogram.nanos-between-syncs count = 102788 min = 113865 max = 62777544 mean = 1628760.64 stddev = 3796429.99 median = 1391037.00 75% <= 1458844.00 95% <= 1647948.00 98% <= 1999632.00 99% <= 2492916.00 99.9% <= 62777544.00 -- Meters ---------------------------------------------------------------------- org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.appendMeter.bytes count = 5828294825 mean rate = 38614011.23 events/second 1-minute rate = 36650337.06 events/second 5-minute rate = 35737367.38 events/second 15-minute rate = 34778685.18 events/second org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncMeter.syncs count = 102789 mean rate = 681.00 events/second 1-minute rate = 646.16 events/second 5-minute rate = 629.76 events/second 15-minute rate = 612.67 events/second ``` **Without patch:** ``` -- Histograms ------------------------------------------------------------------ org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.latencyHistogram.nanos count = 10836273 min = 2468323 max = 66655180 mean = 3893813.44 stddev = 5620979.14 median = 3247994.00 75% <= 3559261.00 95% <= 4026170.00 98% <= 4801295.00 99% <= 5688797.00 99.9% <= 66655180.00 org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncCountHistogram.countPerSync count = 106307 min = 52 max = 103 mean = 101.94 stddev = 1.70 median = 102.00 75% <= 102.00 95% <= 102.00 98% <= 102.00 99% <= 103.00 99.9% <= 103.00 org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncHistogram.nanos-between-syncs count = 106308 min = 126570 max = 54387876 mean = 1423180.52 stddev = 2174336.12 median = 1282380.00 75% <= 1421388.00 95% <= 1610647.00 98% <= 1897988.00 99% <= 2200189.00 99.9% <= 52882012.00 -- Meters ---------------------------------------------------------------------- org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.appendMeter.bytes count = 6035972275 mean rate = 39993180.92 events/second 1-minute rate = 38393898.28 events/second 5-minute rate = 36625545.98 events/second 15-minute rate = 35407434.87 events/second org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncMeter.syncs count = 106308 mean rate = 704.37 events/second 1-minute rate = 676.34 events/second 5-minute rate = 644.42 events/second 15-minute rate = 622.68 events/second ``` It does show up on the profiler though.. doesn't seem like a great idea then? ![image](https://user-images.githubusercontent.com/248754/121570822-19076a00-c9d7-11eb-92dd-457b2d838051.png) ![image](https://user-images.githubusercontent.com/248754/121570909-36d4cf00-c9d7-11eb-8f40-483bf155e0af.png) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org