[GitHub] [hadoop] xinglin opened a new pull request, #5804: HDFS-17042 Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode (#5730)

xinglin opened a new pull request, #5804: URL: https://github.com/apache/hadoop/pull/5804 … ### Description of PR Backport of HDFS-17042 from trunk to branch-3.3. Almost clean cherry-pick with a small conflict in RpcMetrics.java (_rpcRequeueCalls_ metric has not been backp

[GitHub] [hadoop] xinglin commented on pull request #5803: HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

xinglin commented on PR #5803: URL: https://github.com/apache/hadoop/pull/5803#issuecomment-1619397585 Hi @goiri, In this PR, we basically changed ThreadPoolExecutor to BlockingThreadPoolExecutorService, which comes with some default settings. I am not sure what unit test we should a

[GitHub] [hadoop] hadoop-yetus commented on pull request #5803: HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

hadoop-yetus commented on PR #5803: URL: https://github.com/apache/hadoop/pull/5803#issuecomment-1619316160 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5803: HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

hadoop-yetus commented on PR #5803: URL: https://github.com/apache/hadoop/pull/5803#issuecomment-1619315770 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5783: YARN-11000. Replace queue resource calculation logic in updateClusterResource

hadoop-yetus commented on PR #5783: URL: https://github.com/apache/hadoop/pull/5783#issuecomment-1619313134 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5803: HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

hadoop-yetus commented on PR #5803: URL: https://github.com/apache/hadoop/pull/5803#issuecomment-1619312871 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] xinglin opened a new pull request, #5803: HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

xinglin opened a new pull request, #5803: URL: https://github.com/apache/hadoop/pull/5803 ### Description of PR In [HDFS-17030](https://issues.apache.org/jira/browse/HDFS-17030), we introduced an ExecutorService, to submit getHAServiceState() requests. We constructed the Executor

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5797: YARN-11524. Improve the Policy Description in Federation.md.

slfan1989 commented on code in PR #5797: URL: https://github.com/apache/hadoop/pull/5797#discussion_r1251305309 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md: ## @@ -235,7 +235,112 @@ SQL-Server scripts are located in **sbin/FederationState

[jira] [Commented] (HADOOP-18779) Improve hadoop-function.sh#status script

[ https://issues.apache.org/jira/browse/HADOOP-18779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739738#comment-17739738 ] ASF GitHub Bot commented on HADOOP-18779: - slfan1989 commented on PR #5762: URL

[GitHub] [hadoop] slfan1989 commented on pull request #5782: YARN-11090. [GPG] Support Secure Mode.

slfan1989 commented on PR #5782: URL: https://github.com/apache/hadoop/pull/5782#issuecomment-1619193657 @goiri Thank you very much for your help in reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [hadoop] slfan1989 commented on pull request #5762: HADOOP-18779. Improve hadoop-function.sh#status script.

slfan1989 commented on PR #5762: URL: https://github.com/apache/hadoop/pull/5762#issuecomment-1619193909 @goiri Thank you very much for your help in reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [hadoop] hadoop-yetus commented on pull request #5783: YARN-11000. Replace queue resource calculation logic in updateClusterResource

hadoop-yetus commented on PR #5783: URL: https://github.com/apache/hadoop/pull/5783#issuecomment-1619112661 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] tomicooler commented on a diff in pull request #5293: YARN-11414. ClusterMetricsInfo shows wrong availableMB when node labels enabled

tomicooler commented on code in PR #5293: URL: https://github.com/apache/hadoop/pull/5293#discussion_r1251115513 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetrics

[GitHub] [hadoop] tomicooler commented on pull request #5293: YARN-11414. ClusterMetricsInfo shows wrong availableMB when node labels enabled

tomicooler commented on PR #5293: URL: https://github.com/apache/hadoop/pull/5293#issuecomment-1618880984 Hi, I can help in reviewing the code, but I'm not familiar with ClusterMetrics or FairScheduler. ``` 2. Currently, ClusterMetricsInfo shows available or allocated memory

[GitHub] [hadoop] xinglin commented on pull request #5790: HDFS-17055 Export HAState as a metric from Namenode for monitoring

xinglin commented on PR #5790: URL: https://github.com/apache/hadoop/pull/5790#issuecomment-1618880342 Thanks @goiri for approving/merging this into branch-3.3! Happy July 4th! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [hadoop] hadoop-yetus commented on pull request #5794: HDFS-17064. Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter

hadoop-yetus commented on PR #5794: URL: https://github.com/apache/hadoop/pull/5794#issuecomment-1618810775 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] tomicooler commented on a diff in pull request #5783: YARN-11000. Replace queue resource calculation logic in updateClusterResource

tomicooler commented on code in PR #5783: URL: https://github.com/apache/hadoop/pull/5783#discussion_r1251079277 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/Abstra

[GitHub] [hadoop] tomicooler commented on a diff in pull request #5783: YARN-11000. Replace queue resource calculation logic in updateClusterResource

tomicooler commented on code in PR #5783: URL: https://github.com/apache/hadoop/pull/5783#discussion_r1251071990 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/Absolu

[GitHub] [hadoop] goiri merged pull request #5782: YARN-11090. [GPG] Support Secure Mode.

goiri merged PR #5782: URL: https://github.com/apache/hadoop/pull/5782 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] goiri commented on a diff in pull request #5797: YARN-11524. Improve the Policy Description in Federation.md.

goiri commented on code in PR #5797: URL: https://github.com/apache/hadoop/pull/5797#discussion_r1251062888 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md: ## @@ -235,7 +235,112 @@ SQL-Server scripts are located in **sbin/FederationStateStor

[jira] [Resolved] (HADOOP-18779) Improve hadoop-function.sh#status script

[ https://issues.apache.org/jira/browse/HADOOP-18779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri resolved HADOOP-18779. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve hadoop-fu

[jira] [Commented] (HADOOP-18779) Improve hadoop-function.sh#status script

[ https://issues.apache.org/jira/browse/HADOOP-18779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739668#comment-17739668 ] ASF GitHub Bot commented on HADOOP-18779: - goiri merged PR #5762: URL: https://

[GitHub] [hadoop] goiri merged pull request #5762: HADOOP-18779. Improve hadoop-function.sh#status script.

goiri merged PR #5762: URL: https://github.com/apache/hadoop/pull/5762 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] goiri merged pull request #5790: HDFS-17055 Export HAState as a metric from Namenode for monitoring

goiri merged PR #5790: URL: https://github.com/apache/hadoop/pull/5790 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] goiri commented on a diff in pull request #5802: Hdfs 17066: RBF: The construction of the trash path in the downstream nameservice should be based on src locations.

goiri commented on code in PR #5802: URL: https://github.com/apache/hadoop/pull/5802#discussion_r1251052837 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/MountTableResolver.java: ## @@ -460,9 +465,13 @@ public PathLocation

[GitHub] [hadoop] tomicooler commented on a diff in pull request #5783: YARN-11000. Replace queue resource calculation logic in updateClusterResource

tomicooler commented on code in PR #5783: URL: https://github.com/apache/hadoop/pull/5783#discussion_r1251053956 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestAp

[GitHub] [hadoop] goiri merged pull request #5796: HDFS-17065. Fix typos in hadoop-hdfs-project

goiri merged PR #5796: URL: https://github.com/apache/hadoop/pull/5796 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] tomicooler commented on a diff in pull request #5783: YARN-11000. Replace queue resource calculation logic in updateClusterResource

tomicooler commented on code in PR #5783: URL: https://github.com/apache/hadoop/pull/5783#discussion_r1250921579 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/Abstra

[jira] [Commented] (HADOOP-18781) ABFS Output stream thread pools getting shutdown during GC.

[ https://issues.apache.org/jira/browse/HADOOP-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739637#comment-17739637 ] ASF GitHub Bot commented on HADOOP-18781: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5780: HADOOP-18781. ABFS Output stream thread pools getting shutdown during GC.

steveloughran commented on code in PR #5780: URL: https://github.com/apache/hadoop/pull/5780#discussion_r1250955453 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsOutputStream.java: ## @@ -90,36 +93,51 @@ public void testMaxRequestsAnd

[jira] [Commented] (HADOOP-18790) ITestS3ACopyFromLocalFile: AuditFailureException

[ https://issues.apache.org/jira/browse/HADOOP-18790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739612#comment-17739612 ] Steve Loughran commented on HADOOP-18790: - having reviewed the codepath to get

[GitHub] [hadoop] wzhallright commented on pull request #5796: HDFS-17065. Fix typos in hadoop-hdfs-project

wzhallright commented on PR #5796: URL: https://github.com/apache/hadoop/pull/5796#issuecomment-1617984208 Thanks for review! @ayushtkn @zhangshuyan0 @slfan1989 This PR is ready to be merged. could you help merge this when you have time? -- This is an automated message from the Apach

[GitHub] [hadoop] haiyang1987 commented on a diff in pull request #5794: HDFS-17064. Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter

haiyang1987 commented on code in PR #5794: URL: https://github.com/apache/hadoop/pull/5794#discussion_r1250578517 ## hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md: ## @@ -304,6 +306,7 @@ Usage: | `-idleiterations` \ | Maximum number of idle iterations before

[GitHub] [hadoop] haiyang1987 commented on a diff in pull request #5794: HDFS-17064. Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter

haiyang1987 commented on code in PR #5794: URL: https://github.com/apache/hadoop/pull/5794#discussion_r1250560310 ## hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md: ## @@ -304,6 +306,7 @@ Usage: | `-idleiterations` \ | Maximum number of idle iterations before

[GitHub] [hadoop] zhangshuyan0 commented on a diff in pull request #5794: HDFS-17064. Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter

zhangshuyan0 commented on code in PR #5794: URL: https://github.com/apache/hadoop/pull/5794#discussion_r1250505298 ## hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md: ## @@ -304,6 +306,7 @@ Usage: | `-idleiterations` \ | Maximum number of idle iterations befor

[jira] [Commented] (HADOOP-16822) Provide source artifacts for hadoop-client-api

[ https://issues.apache.org/jira/browse/HADOOP-16822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739493#comment-17739493 ] Cheng Pan commented on HADOOP-16822: [~weichiu] I would suggest reverting it direct

[GitHub] [hadoop] hadoop-yetus commented on pull request #5802: Hdfs 17066: RBF: The construction of the trash path in the downstream nameservice should be based on src locations.

hadoop-yetus commented on PR #5802: URL: https://github.com/apache/hadoop/pull/5802#issuecomment-1617614772 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18781) ABFS Output stream thread pools getting shutdown during GC.

[ https://issues.apache.org/jira/browse/HADOOP-18781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739490#comment-17739490 ] ASF GitHub Bot commented on HADOOP-18781: - hadoop-yetus commented on PR #5780:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5780: HADOOP-18781. ABFS Output stream thread pools getting shutdown during GC.

hadoop-yetus commented on PR #5780: URL: https://github.com/apache/hadoop/pull/5780#issuecomment-1617600548 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex