virajjasani commented on pull request #4107: URL: https://github.com/apache/hadoop/pull/4107#issuecomment-1079369473
@ayushtkn While I agree that JMX metric for slownode is already available, not every downstreamer might have access to it directly, for instance in K8S managed clusters, unless port forward is enabled (not so common case in prod), HDFS downstreamer would not be able to access JMX metrics. We have similar case with `DFS.getDataNodeStats()` API, it provides live/decomm/dead node info, however such node info is already used by JMX metrics, but when it's about downstream or deployment management application trying to use such info, DFS APIs are preferred and not JMX metrics due to similar concerns mentioned above. Moreover, it's not only about downstreamer using the API, we should also provide `dfsadmin -report` option to report slownode info for operators, something that only an API can offer. We only expose slowNode and reportingNodes info for each unique slow peer detection, we do not expose other imp data e.g. how many blocks are currently available, what is the DFS usage etc with the same JMX metric, and we don't even need to. However, providing as much concrete info related to each slow node would be API's responsibility. With API, we also don't need to keep tuning `dfs.datanode.max.nodes.to.report` to adjust how many top N slow nodes we want to get exposed (which is a nice limitation for JMX metrics for sure). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
