[ https://issues.apache.org/jira/browse/HBASE-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011461#comment-13011461 ]
stack commented on HBASE-3694: ------------------------------ bq. In terms of overall design, I would love to see RegionServerServices evolve into something like an IOC container.... Yeah, thats the plan....Need to keep it macro though. Args on why this is not 'metrics' are good. I go along. Just say no to atomic long counters now we have cliff click counters in our CLASSPATH bq. The internal accounting makes sense. I just think MemoryAccountingManager is too specific. We need something more general to reuse it in the future, RegionServerAccountingManager. Agreed. Should be more than just about Memory accounting (and agree w/ Jon that it could be path out of our hairball HRegionServer class). For you Liyin and this patch, I think just make a class named RegionServerAccounting -- drop Manager I'd say, that might be a little megalomanicial -- and put just this one counter in it (as per Jon). Add getRegionServerAccounting to RSS Interface. > high multiput latency due to checking global mem store size in a synchronized > function > -------------------------------------------------------------------------------------- > > Key: HBASE-3694 > URL: https://issues.apache.org/jira/browse/HBASE-3694 > Project: HBase > Issue Type: Improvement > Reporter: Liyin Tang > Assignee: Liyin Tang > Attachments: Hbase-3694[r1085306], Hbase-3694[r1085306]_2.patch, > Hbase-3694[r1085306]_3.patch, Hbase-3694[r1085508]_4.patch > > > The problem is we found the multiput latency is very high. > In our case, we have almost 22 Regions in each RS and there are no flush > happened during these puts. > After investigation, we believe that the root cause is the function > getGlobalMemStoreSize, which is to check the high water mark of mem store. > This function takes almost 40% of total execution time of multiput when > instrumenting some metrics in the code. > The actual percentage may be more higher. The execution time is spent on > synchronize contention. > One solution is to keep a static var in HRegion to keep the global MemStore > size instead of calculating them every time. > Why using static variable? > Since all the HRegion objects in the same JVM share the same memory heap, > they need to share fate as well. > The static variable, globalMemStroeSize, naturally shows the total mem usage > in this shared memory heap for this JVM. > If multiple RS need to run in the same JVM, they still need only one > globalMemStroeSize. > If multiple RS run on different JVMs, everything is fine. > After changing, in our cases, the avg multiput latency decrease from 60ms to > 10ms. > I will submit a patch based on the current trunk. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira