[ https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liyin Tang updated HBASE-6968: ------------------------------ Description: Here are 2 hbase write performance improvements recently: 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when creating a HBaseConfiguraiton object, it would parse the xml configuration files from disk, which is not cheap operation. In HLog.java: orig: {code:title=HLog.java} newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf)); {code} new: {code} newWriter = createWriter(fs, newPath, conf); {code} 2) Change 2 hotspot synchronized functions into double locking pattern. So it shall remove the synchronization overhead in the normal case. orig: {code:title=HBaseRpcMetrics.java} public synchronized void inc(String name, int amt) { MetricsTimeVaryingRate m = get(name); if (m == null) { m = create(name); } m.inc(amt); } {code} new: {code} public void inc(String name, int amt) { MetricsTimeVaryingRate m = get(name); if (m == null) { synchronized (this) { if ((m = get(name)) == null) { m = create(name); } } } m.inc(amt); } {code} ===================== orig: {code:title=MemStoreFlusher.java} public synchronized void reclaimMemStoreMemory() { if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) { flushSomeRegions(); } } {code} new: {code} public void reclaimMemStoreMemory() { if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) { flushSomeRegions(); } } private synchronized void flushSomeRegions() { if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) { return; // double check the global memstore size inside of the synchronized block. } ... } {code} was: Here are 2 hbase write performance improvements recently found out. 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when creating a HBaseConfiguraiton object, it would parse the xml configuration files from disk, which is not cheap operation. In HLog.java: orig: {code:title=HLog.java} newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf)); {code} new: {code} newWriter = createWriter(fs, newPath, conf); {code} 2) Change 2 hotspot synchronized functions into double locking pattern. So it shall remove the synchronization overhead in the normal case. orig: {code:title=HBaseRpcMetrics.java} public synchronized void inc(String name, int amt) { MetricsTimeVaryingRate m = get(name); if (m == null) { m = create(name); } m.inc(amt); } {code} new: {code} public void inc(String name, int amt) { MetricsTimeVaryingRate m = get(name); if (m == null) { synchronized (this) { if ((m = get(name)) == null) { m = create(name); } } } m.inc(amt); } {code} ===================== orig: {code:title=MemStoreFlusher.java} public synchronized void reclaimMemStoreMemory() { if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) { flushSomeRegions(); } } {code} new: {code} public void reclaimMemStoreMemory() { if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) { flushSomeRegions(); } } private synchronized void flushSomeRegions() { if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) { return; // double check the global memstore size inside of the synchronized block. } ... } {code} > Several HBase write perf improvement > ------------------------------------ > > Key: HBASE-6968 > URL: https://issues.apache.org/jira/browse/HBASE-6968 > Project: HBase > Issue Type: Improvement > Reporter: Liyin Tang > > Here are 2 hbase write performance improvements recently: > 1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when > creating a HBaseConfiguraiton object, it would parse the xml configuration > files from disk, which is not cheap operation. > In HLog.java: > orig: > {code:title=HLog.java} > newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf)); > {code} > new: > {code} > newWriter = createWriter(fs, newPath, conf); > {code} > 2) Change 2 hotspot synchronized functions into double locking pattern. So it > shall remove the synchronization overhead in the normal case. > orig: > {code:title=HBaseRpcMetrics.java} > public synchronized void inc(String name, int amt) { > MetricsTimeVaryingRate m = get(name); > if (m == null) { > m = create(name); > } > m.inc(amt); > } > {code} > new: > {code} > public void inc(String name, int amt) { > MetricsTimeVaryingRate m = get(name); > if (m == null) { > synchronized (this) { > if ((m = get(name)) == null) { > m = create(name); > } > } > } > m.inc(amt); > } > {code} > ===================== > orig: > {code:title=MemStoreFlusher.java} > public synchronized void reclaimMemStoreMemory() { > if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) { > flushSomeRegions(); > } > } > {code} > new: > {code} > public void reclaimMemStoreMemory() { > if (this.server.getGlobalMemstoreSize().get() >= globalMemStoreLimit) { > flushSomeRegions(); > } > } > private synchronized void flushSomeRegions() { > if (this.server.getGlobalMemstoreSize().get() < globalMemStoreLimit) { > return; // double check the global memstore size inside of the > synchronized block. > } > ... > } > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira