[jira] [Commented] (HDFS-13219) RBF: Cluster information on Router is not correct when the Federation shares datanodes
[ https://issues.apache.org/jira/browse/HDFS-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399755#comment-16399755 ] Íñigo Goiri commented on HDFS-13219: Thanks [~maobaolong] for putting examples. In my comment I already agreed that this was an issue and proposed a solution. For solving this correctly, we need to use {{getDatanodeReport()}}; this will give us all the data and then we can aggregate. However, this is pretty expensive so we should: * Cache this information * Have an option to use the old and the new approach > RBF: Cluster information on Router is not correct when the Federation shares > datanodes > -- > > Key: HDFS-13219 > URL: https://issues.apache.org/jira/browse/HDFS-13219 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.9.0 >Reporter: Tao Jie >Priority: Major > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png, screenshot-5.png > > > Now summary information on Router website aggregates summary of each > nameservice. However in a typical federation cluster deployment, datanodes > are shared among nameservices. Consider we have 2 namespaces and 100 > datanodes in one cluster. 100 datanodes are available for each namespace, but > we see 200 datanodes on the router website. So does other information such as > {{Total capacity}}, {{Remaining capacity}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13219) RBF: Cluster information on Router is not correct when the Federation shares datanodes
[ https://issues.apache.org/jira/browse/HDFS-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399738#comment-16399738 ] maobaolong commented on HDFS-13219: --- Maybe we can see something wrong from the following wrong: - ns1-nn1-datanode !screenshot-4.png! ns10-nn20-datanode !screenshot-5.png! - ns-fed-r1-overview !screenshot-3.png! - ns-fed-r1-subcluster !screenshot-2.png! - ns-fed-r1-datanode !screenshot-1.png! We can see something wrong: * Router overview page should not display the live datanode to 8, because we have 4 datanode in all. * Router datanode page should not display the ns1's datanode information, because we have 2 ns (ns1 and ns10). [~linyiqun] [~elgoiri], Please take a look at this issue, maybe we should discuss for it. > RBF: Cluster information on Router is not correct when the Federation shares > datanodes > -- > > Key: HDFS-13219 > URL: https://issues.apache.org/jira/browse/HDFS-13219 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.9.0 >Reporter: Tao Jie >Priority: Major > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png, screenshot-5.png > > > Now summary information on Router website aggregates summary of each > nameservice. However in a typical federation cluster deployment, datanodes > are shared among nameservices. Consider we have 2 namespaces and 100 > datanodes in one cluster. 100 datanodes are available for each namespace, but > we see 200 datanodes on the router website. So does other information such as > {{Total capacity}}, {{Remaining capacity}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org