[ https://issues.apache.org/jira/browse/HDFS-15409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143602#comment-17143602 ]
Danil Lipovoy commented on HDFS-15409: -------------------------------------- About HBase blocks distribution - all tests show the same picture - it is quite evenly distributes. I've tried proposed above the way but unfortunately it wasn't good. {code:java} if (clientShortCircuitNum == 3) { idx = idx % 100; if (idx <= 32) { LOG.info("shortCircuitCache: 0"); return shortCircuitCache[0]; } if (idx > 32 && idx <= 65) { LOG.info("shortCircuitCache: 1"); return shortCircuitCache[1]; } if (idx > 66) { LOG.info("shortCircuitCache: 2"); return shortCircuitCache[2]; } } {code} cat /var/log/hbase/hbase-cmf-hbase-REGIONSERVER-home.com.log.out |grep shortCircuitCache | awk '\{print $8}'| sort | uniq -c | sort -nr | awk '\{printf "%-8s%s\n", $2, $1}'|sort 0 581125 1 621340 2 450377 So, difference bigger and looks like it's not worth the effort. > Optimization Strategy for choosing ShortCircuitCache > ----------------------------------------------------- > > Key: HDFS-15409 > URL: https://issues.apache.org/jira/browse/HDFS-15409 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Lisheng Sun > Assignee: Lisheng Sun > Priority: Major > > When clientShortCircuitNum is 10, the probability of falling into each > ShortCircuitCache is the same, while the probability of other > clientShortCircuitNum is different. > For example if clientShortCircuitNum is 3, when a lot of blockids of SSR are > ***1, ***4, ***7, this situation will fall into a ShortCircuitCache. > Since the real environment blockid is completely unpredictable, i think it is > need to design a strategy which is allocated to a specific ShortCircuitCache. > This should improve performance even more. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org