[GitHub] [hadoop] hchaverri opened a new pull request, #5691: HDFS-17026. Configuring NamenodeHeartbeatService to update JMX on low…

2023-05-25 Thread via GitHub
hchaverri opened a new pull request, #5691: URL: https://github.com/apache/hadoop/pull/5691 …er frequency ### Description of PR Changes to the NamenodeHeartbeatService to update JMX reports on a configurable frequency or to disable JMX updates altogether, as opposed to

[GitHub] [hadoop] mudit-97 commented on a diff in pull request #5681: YARN-11497 : Support removal of only selective node states in untracked removal flow

2023-05-25 Thread via GitHub
mudit-97 commented on code in PR #5681: URL: https://github.com/apache/hadoop/pull/5681#discussion_r1206212258 ##

[GitHub] [hadoop] mudit-97 commented on a diff in pull request #5681: YARN-11497 : Support removal of only selective node states in untracked removal flow

2023-05-25 Thread via GitHub
mudit-97 commented on code in PR #5681: URL: https://github.com/apache/hadoop/pull/5681#discussion_r1206212144 ##

[GitHub] [hadoop] mudit-97 commented on a diff in pull request #5681: YARN-11497 : Support removal of only selective node states in untracked removal flow

2023-05-25 Thread via GitHub
mudit-97 commented on code in PR #5681: URL: https://github.com/apache/hadoop/pull/5681#discussion_r1206211150 ##

[GitHub] [hadoop] slfan1989 commented on pull request #4793: YARN-11276. Add LRU cache for RMWebServices.getApps

2023-05-25 Thread via GitHub
slfan1989 commented on PR #4793: URL: https://github.com/apache/hadoop/pull/4793#issuecomment-1563626222 @leixm Thank you for contribution. Can we rebase? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop] slfan1989 commented on pull request #5681: YARN-11497 : Support removal of only selective node states in untracked removal flow

2023-05-25 Thread via GitHub
slfan1989 commented on PR #5681: URL: https://github.com/apache/hadoop/pull/5681#issuecomment-1563623819 @mudit-97 Thanks for your contribution! there are some indentation issues and need to be fixed. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5681: YARN-11497 : Support removal of only selective node states in untracked removal flow

2023-05-25 Thread via GitHub
slfan1989 commented on code in PR #5681: URL: https://github.com/apache/hadoop/pull/5681#discussion_r1206082428 ##

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5681: YARN-11497 : Support removal of only selective node states in untracked removal flow

2023-05-25 Thread via GitHub
slfan1989 commented on code in PR #5681: URL: https://github.com/apache/hadoop/pull/5681#discussion_r1206082784 ##

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5681: YARN-11497 : Support removal of only selective node states in untracked removal flow

2023-05-25 Thread via GitHub
slfan1989 commented on code in PR #5681: URL: https://github.com/apache/hadoop/pull/5681#discussion_r1206082061 ##

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5681: YARN-11497 : Support removal of only selective node states in untracked removal flow

2023-05-25 Thread via GitHub
slfan1989 commented on code in PR #5681: URL: https://github.com/apache/hadoop/pull/5681#discussion_r1206081314 ##

[jira] [Commented] (HADOOP-18751) Fix incorrect output path in javadoc build phase

2023-05-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726408#comment-17726408 ] ASF GitHub Bot commented on HADOOP-18751: - hadoop-yetus commented on PR #5688: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5688: HADOOP-18751. Fix incorrect output path in javadoc build phase

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #5688: URL: https://github.com/apache/hadoop/pull/5688#issuecomment-1563591775 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5673: YARN-8337. [FederationStateStore - MySql] Improve ApplicationHome Mysql Script.

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #5673: URL: https://github.com/apache/hadoop/pull/5673#issuecomment-1563524594 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #3618: YARN-11000. Replace queue resource calculation logic in updateClusterResource

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #3618: URL: https://github.com/apache/hadoop/pull/3618#issuecomment-1563517982 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5561: HDFS-16983. Whether checking path access permissions should be decided by dfs.permissions.enabled in concat operation

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #5561: URL: https://github.com/apache/hadoop/pull/5561#issuecomment-1563446206 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5672: YARN-7720. Race condition between second app attempt and UAM timeout when first attempt node is down.

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #5672: URL: https://github.com/apache/hadoop/pull/5672#issuecomment-1563443554 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18751) Fix incorrect output path in javadoc build phase

2023-05-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726366#comment-17726366 ] ASF GitHub Bot commented on HADOOP-18751: - GauthamBanasandra commented on code in PR #5688:

[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #5688: HADOOP-18751. Fix incorrect output path in javadoc build phase

2023-05-25 Thread via GitHub
GauthamBanasandra commented on code in PR #5688: URL: https://github.com/apache/hadoop/pull/5688#discussion_r1205934232 ## hadoop-project-dist/pom.xml: ## @@ -106,7 +106,7 @@ ${maven.compile.source} ${maven.compile.encoding}

[jira] [Updated] (HADOOP-18751) Fix incorrect output path in javadoc build phase

2023-05-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18751: Labels: pull-request-available (was: ) > Fix incorrect output path in javadoc build

[GitHub] [hadoop] sreeb-msft closed pull request #5690: Support on Blob Endpoint for Read API

2023-05-25 Thread via GitHub
sreeb-msft closed pull request #5690: Support on Blob Endpoint for Read API URL: https://github.com/apache/hadoop/pull/5690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hadoop] sreeb-msft opened a new pull request, #5690: Support on Blob Endpoint for Read API

2023-05-25 Thread via GitHub
sreeb-msft opened a new pull request, #5690: URL: https://github.com/apache/hadoop/pull/5690 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [hadoop] hadoop-yetus commented on pull request #4793: YARN-11276. Add LRU cache for RMWebServices.getApps

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #4793: URL: https://github.com/apache/hadoop/pull/4793#issuecomment-1563264992 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library

2023-05-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726292#comment-17726292 ] ASF GitHub Bot commented on HADOOP-18709: - hadoop-yetus commented on PR #5638: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5638: HADOOP-18709. Add curator based ZooKeeper communication support over…

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #5638: URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1563142275 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library

2023-05-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726288#comment-17726288 ] ASF GitHub Bot commented on HADOOP-18709: - hadoop-yetus commented on PR #5638: URL:

[jira] [Commented] (HADOOP-18753) S3AFileSystem doesn't consistently handle prefixes that are both files and directories between versions

2023-05-25 Thread Helen Weng (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726289#comment-17726289 ] Helen Weng commented on HADOOP-18753: - Thanks Steve, I completely understand that this is

[GitHub] [hadoop] hadoop-yetus commented on pull request #5638: HADOOP-18709. Add curator based ZooKeeper communication support over…

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #5638: URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1563135627 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5636: YARN-11492. Improve createJerseyClient#setConnectTimeout Code.

2023-05-25 Thread via GitHub
slfan1989 commented on code in PR #5636: URL: https://github.com/apache/hadoop/pull/5636#discussion_r1205701835 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestRouterWebServiceUtil.java:

[GitHub] [hadoop] slfan1989 commented on pull request #5676: YARN-6648. BackPort [GPG] Add SubClusterCleaner in Global Policy Generator.

2023-05-25 Thread via GitHub
slfan1989 commented on PR #5676: URL: https://github.com/apache/hadoop/pull/5676#issuecomment-1563070857 @goiri Can you help review this PR again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5676: YARN-6648. BackPort [GPG] Add SubClusterCleaner in Global Policy Generator.

2023-05-25 Thread via GitHub
slfan1989 commented on code in PR #5676: URL: https://github.com/apache/hadoop/pull/5676#discussion_r1205655642 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml: ## @@ -4370,6 +4370,30 @@ 0.0.0.0:8091 + + + The

[GitHub] [hadoop] lfxy commented on pull request #5561: HDFS-16983. Whether checking path access permissions should be decided by dfs.permissions.enabled in concat operation

2023-05-25 Thread via GitHub
lfxy commented on PR #5561: URL: https://github.com/apache/hadoop/pull/5561#issuecomment-1563020147 @Hexiaoqiao OK, I add both UT in TestHDFSConcat. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Resolved] (HADOOP-18753) S3AFileSystem doesn't consistently handle prefixes that are both files and directories between versions

2023-05-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18753. - Resolution: Won't Fix > S3AFileSystem doesn't consistently handle prefixes that are

[jira] [Commented] (HADOOP-18753) S3AFileSystem doesn't consistently handle prefixes that are both files and directories between versions

2023-05-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726243#comment-17726243 ] Steve Loughran commented on HADOOP-18753: - you've just exceeded the envelope of "sustainable

[jira] [Updated] (HADOOP-18753) S3AFileSystem doesn't consistently handle prefixes that are both files and directories between versions

2023-05-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18753: Component/s: fs/s3 (was: tools) > S3AFileSystem doesn't

[GitHub] [hadoop] hadoop-yetus commented on pull request #4793: YARN-11276. Add LRU cache for RMWebServices.getApps

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #4793: URL: https://github.com/apache/hadoop/pull/4793#issuecomment-1562882955 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4793: YARN-11276. Add LRU cache for RMWebServices.getApps

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #4793: URL: https://github.com/apache/hadoop/pull/4793#issuecomment-1562865100 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-05-25 Thread via GitHub
hfutatzhanghb commented on code in PR #5597: URL: https://github.com/apache/hadoop/pull/5597#discussion_r1205455152 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java: ## @@ -2630,6 +2632,31 @@ public int

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-05-25 Thread via GitHub
hfutatzhanghb commented on code in PR #5597: URL: https://github.com/apache/hadoop/pull/5597#discussion_r1205454941 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java: ## @@ -2630,6 +2632,31 @@ public int

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-05-25 Thread via GitHub
hfutatzhanghb commented on code in PR #5597: URL: https://github.com/apache/hadoop/pull/5597#discussion_r1205453041 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java: ## @@ -2630,6 +2632,31 @@ public int

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-05-25 Thread via GitHub
hfutatzhanghb commented on code in PR #5597: URL: https://github.com/apache/hadoop/pull/5597#discussion_r1205448021 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java: ## @@ -2630,6 +2632,31 @@ public int

[GitHub] [hadoop] hadoop-yetus commented on pull request #4797: YARN-11277. Trigger log-dir deletion by size for NonAggregatingLogHandler

2023-05-25 Thread via GitHub
hadoop-yetus commented on PR #4797: URL: https://github.com/apache/hadoop/pull/4797#issuecomment-1562766851 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5561: HDFS-16983. Whether checking path access permissions should be decided by dfs.permissions.enabled in concat operation

2023-05-25 Thread via GitHub
Hexiaoqiao commented on PR #5561: URL: https://github.com/apache/hadoop/pull/5561#issuecomment-1562699663 > Set dfs.permissions.enabled to false, with the same wrong user to concat, it would not check permission and don't throw exception. I don't think it is necessary. But it

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5643: HDFS-17003. Erasure coding: invalidate wrong block after reporting bad blocks from datanode

2023-05-25 Thread via GitHub
hfutatzhanghb commented on PR #5643: URL: https://github.com/apache/hadoop/pull/5643#issuecomment-1562543360 @ayushtkn Sir, feel so sorry about involving you here too late. could you also please help review this PR ? Thanks a lot. -- This is an automated message from the Apache Git

[GitHub] [hadoop] leixm opened a new pull request, #4793: YARN-11276. Add LRU cache for RMWebServices.getApps

2023-05-25 Thread via GitHub
leixm opened a new pull request, #4793: URL: https://github.com/apache/hadoop/pull/4793 JIRA: YARN-11276. Add lru cache for RMWebServices.getApps. In our YARN cluster, there are thousands of apps running at the same time, the return result of getApps reaches about 10M, and many

[GitHub] [hadoop] ferdelyi commented on pull request #5638: HADOOP-18709. Add curator based ZooKeeper communication support over…

2023-05-25 Thread via GitHub
ferdelyi commented on PR #5638: URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1562493374 Filed bug report on Yetus: [INFRA-24640 ](https://issues.apache.org/jira/browse/INFRA-24640)- "Yetus fails ASF license check on deleted files" -- This is an automated message from the

[GitHub] [hadoop] ferdelyi commented on a diff in pull request #5638: HADOOP-18709. Add curator based ZooKeeper communication support over…

2023-05-25 Thread via GitHub
ferdelyi commented on code in PR #5638: URL: https://github.com/apache/hadoop/pull/5638#discussion_r1205146014 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java: ## @@ -34,31 +39,32 @@ import

[GitHub] [hadoop] ayushtkn commented on pull request #5689: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep"

2023-05-25 Thread via GitHub
ayushtkn commented on PR #5689: URL: https://github.com/apache/hadoop/pull/5689#issuecomment-1562369779 Just Passing by AFAIK Changing configurations default is an incompatible change and can be done only for minor release, so you can do it only for 3.4.0 ` Hadoop-defined

[GitHub] [hadoop] lfxy commented on pull request #5561: HDFS-16983. Whether checking path access permissions should be decided by dfs.permissions.enabled in concat operation

2023-05-25 Thread via GitHub
lfxy commented on PR #5561: URL: https://github.com/apache/hadoop/pull/5561#issuecomment-1562347858 @Hexiaoqiao Ok, I would follow your suggest. But I also have a problem that I think we should test the two case: 1. Set dfs.permissions.enabled to true, with wrong user to concat, it would