[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Hu updated HDFS-14541: ---------------------------- Description: Our XiaoMi HBase team are evaluating the performence improvement of HBASE-21879, and we have few CPU flame graph & heap flame graph by using async-profiler, and find that there're some performence issues in DFSClient . See the attached two flame graphs, we can conclude that the try catch block in ShortCircuitCache#trimEvictionMaps has some serious perf problem , we should remove the try catch from DFSClient. {code} /** * Trim the eviction lists. */ private void trimEvictionMaps() { long now = Time.monotonicNow(); demoteOldEvictableMmaped(now); while (true) { long evictableSize = evictable.size(); long evictableMmappedSize = evictableMmapped.size(); if (evictableSize + evictableMmappedSize <= maxTotalSize) { return; } ShortCircuitReplica replica; try { if (evictableSize == 0) { replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped .firstKey()); } else { replica = (ShortCircuitReplica)evictable.get(evictable.firstKey()); } } catch (NoSuchElementException e) { break; } if (LOG.isTraceEnabled()) { LOG.trace(this + ": trimEvictionMaps is purging " + replica + StringUtils.getStackTrace(Thread.currentThread())); } purge(replica); } } {code} Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue. was: Our XiaoMi HBase team are evaluating the performence improvement of HBASE-21879, and we have few CPU flame graph & heap flame graph by using async-profiler, and found that there're some performence issues in DFSClient now . See the attached two flame graph, we can conclude that the try catch block in ShortCircuitCache#trimEvictionMaps has some serious perf problem now, we should remove this from DFSClient. {code} /** * Trim the eviction lists. */ private void trimEvictionMaps() { long now = Time.monotonicNow(); demoteOldEvictableMmaped(now); while (true) { long evictableSize = evictable.size(); long evictableMmappedSize = evictableMmapped.size(); if (evictableSize + evictableMmappedSize <= maxTotalSize) { return; } ShortCircuitReplica replica; try { if (evictableSize == 0) { replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped .firstKey()); } else { replica = (ShortCircuitReplica)evictable.get(evictable.firstKey()); } } catch (NoSuchElementException e) { break; } if (LOG.isTraceEnabled()) { LOG.trace(this + ": trimEvictionMaps is purging " + replica + StringUtils.getStackTrace(Thread.currentThread())); } purge(replica); } } {code} Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue. > ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of > the frequent thrown NoSuchElementException in our HBase benchmark > ------------------------------------------------------------------------------------------------------------------------------------------------ > > Key: HDFS-14541 > URL: https://issues.apache.org/jira/browse/HDFS-14541 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Zheng Hu > Priority: Major > Attachments: async-prof-pid-94152-alloc-2.svg, > async-prof-pid-94152-cpu-1.svg > > > Our XiaoMi HBase team are evaluating the performence improvement of > HBASE-21879, and we have few CPU flame graph & heap flame graph by using > async-profiler, and find that there're some performence issues in DFSClient > . > See the attached two flame graphs, we can conclude that the try catch block > in ShortCircuitCache#trimEvictionMaps has some serious perf problem , we > should remove the try catch from DFSClient. > {code} > /** > * Trim the eviction lists. > */ > private void trimEvictionMaps() { > long now = Time.monotonicNow(); > demoteOldEvictableMmaped(now); > while (true) { > long evictableSize = evictable.size(); > long evictableMmappedSize = evictableMmapped.size(); > if (evictableSize + evictableMmappedSize <= maxTotalSize) { > return; > } > ShortCircuitReplica replica; > try { > if (evictableSize == 0) { > replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped > .firstKey()); > } else { > replica = (ShortCircuitReplica)evictable.get(evictable.firstKey()); > } > } catch (NoSuchElementException e) { > break; > } > if (LOG.isTraceEnabled()) { > LOG.trace(this + ": trimEvictionMaps is purging " + replica + > StringUtils.getStackTrace(Thread.currentThread())); > } > purge(replica); > } > } > {code} > Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org