[jira] [Commented] (HDFS-14953) [Dynamometer] Missing blocks gradually increase after NN starts
[ https://issues.apache.org/jira/browse/HDFS-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992267#comment-16992267 ] Takanobu Asanuma commented on HDFS-14953: - Sorry for late response. Actually, I ported your PR and tested with it, but it didn't fix my problem. Will investigate it further. > [Dynamometer] Missing blocks gradually increase after NN starts > --- > > Key: HDFS-14953 > URL: https://issues.apache.org/jira/browse/HDFS-14953 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: tools >Reporter: Takanobu Asanuma >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14953) [Dynamometer] Missing blocks gradually increase after NN starts
[ https://issues.apache.org/jira/browse/HDFS-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968497#comment-16968497 ] Wei-Chiu Chuang commented on HDFS-14953: Yep. How about porting my PR over here? > [Dynamometer] Missing blocks gradually increase after NN starts > --- > > Key: HDFS-14953 > URL: https://issues.apache.org/jira/browse/HDFS-14953 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: tools >Reporter: Takanobu Asanuma >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14953) [Dynamometer] Missing blocks gradually increase after NN starts
[ https://issues.apache.org/jira/browse/HDFS-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968113#comment-16968113 ] Takanobu Asanuma commented on HDFS-14953: - The case is very similar to [this issue|https://github.com/linkedin/dynamometer/issues/64]. (Thanks for reporting it, [~weichiu].) {quote} After HDFS-9260, NN expects block replicas to be reported in ascending order of block id. If a block id is not in order, NN discards it silently. Because simulated DataNode in Dynamometer uses hash map to store block replicas, the replicas are not reported in order. The Dynamometer cluster would then see missing blocks gradually increase several minutes after NN starts. {quote} > [Dynamometer] Missing blocks gradually increase after NN starts > --- > > Key: HDFS-14953 > URL: https://issues.apache.org/jira/browse/HDFS-14953 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: tools >Reporter: Takanobu Asanuma >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org