[ https://issues.apache.org/jira/browse/HBASE-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011717#comment-13011717 ]
stack commented on HBASE-3694: ------------------------------ @Liyin Approx count on get is fine by me. If you need it to be 'exact', go w/ AtomicLong. > high multiput latency due to checking global mem store size in a synchronized > function > -------------------------------------------------------------------------------------- > > Key: HBASE-3694 > URL: https://issues.apache.org/jira/browse/HBASE-3694 > Project: HBase > Issue Type: Improvement > Reporter: Liyin Tang > Assignee: Liyin Tang > Attachments: 3694-cliffs-counter.txt, Hbase-3694[r1085306], > Hbase-3694[r1085306]_2.patch, Hbase-3694[r1085306]_3.patch, > Hbase-3694[r1085508]_4.patch, Hbase-3694[r1085592]_7.patch, > Hbase-3694[r1085593]_5.patch, Hbase-3694[r1085593]_6.patch > > > The problem is we found the multiput latency is very high. > In our case, we have almost 22 Regions in each RS and there are no flush > happened during these puts. > After investigation, we believe that the root cause is the function > getGlobalMemStoreSize, which is to check the high water mark of mem store. > This function takes almost 40% of total execution time of multiput when > instrumenting some metrics in the code. > The actual percentage may be more higher. The execution time is spent on > synchronize contention. > One solution is to keep a static var in HRegion to keep the global MemStore > size instead of calculating them every time. > Why using static variable? > Since all the HRegion objects in the same JVM share the same memory heap, > they need to share fate as well. > The static variable, globalMemStroeSize, naturally shows the total mem usage > in this shared memory heap for this JVM. > If multiple RS need to run in the same JVM, they still need only one > globalMemStroeSize. > If multiple RS run on different JVMs, everything is fine. > After changing, in our cases, the avg multiput latency decrease from 60ms to > 10ms. > I will submit a patch based on the current trunk. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira