[GitHub] [hadoop] xinglin commented on a diff in pull request #5809: HDFS-17068. Datanode should record last directory scan time.

xinglin commented on code in PR #5809: URL: https://github.com/apache/hadoop/pull/5809#discussion_r1256899902 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalDatasetImpl.java: ## @@ -477,4 +482,7 @@ public

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5812: HDFS-17070. Remove unused import in DataNodeMetricHelper.java.

hfutatzhanghb commented on PR #5812: URL: https://github.com/apache/hadoop/pull/5812#issuecomment-1626617062 @ayushtkn Hi, sir. could you please take a look at this minor modification. Thanks a lot ~ -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5809: HDFS-17068. Datanode should record last directory scan time.

hfutatzhanghb commented on code in PR #5809: URL: https://github.com/apache/hadoop/pull/5809#discussion_r1256807585 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java: ## @@ -3811,5 +3818,10 @@ void

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5809: HDFS-17068. Datanode should record last directory scan time.

hfutatzhanghb commented on code in PR #5809: URL: https://github.com/apache/hadoop/pull/5809#discussion_r1256804680 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeMetricHelper.java: ## @@ -72,8 +71,10 @@ public static

[GitHub] [hadoop] slfan1989 commented on pull request #5815: HDFS-17033. Update fsck to display stale state info of blocks accurately

slfan1989 commented on PR #5815: URL: https://github.com/apache/hadoop/pull/5815#issuecomment-1626378352 @YuanbenWang Thanks for your contribution! We need to fix checkstyle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop] hadoop-yetus commented on pull request #5809: HDFS-17068. Datanode should record last directory scan time.

hadoop-yetus commented on PR #5809: URL: https://github.com/apache/hadoop/pull/5809#issuecomment-1626143814 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Comment Edited] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741189#comment-17741189 ] Emanuel Velzi edited comment on HADOOP-18793 at 7/7/23 8:55 PM: Hi

[jira] [Comment Edited] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741189#comment-17741189 ] Emanuel Velzi edited comment on HADOOP-18793 at 7/7/23 8:55 PM: Hi

[jira] [Comment Edited] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741189#comment-17741189 ] Emanuel Velzi edited comment on HADOOP-18793 at 7/7/23 8:54 PM: Hi

[jira] [Comment Edited] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741189#comment-17741189 ] Emanuel Velzi edited comment on HADOOP-18793 at 7/7/23 8:53 PM: Hi

[jira] [Comment Edited] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741189#comment-17741189 ] Emanuel Velzi edited comment on HADOOP-18793 at 7/7/23 8:52 PM: Hi

[jira] [Commented] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741189#comment-17741189 ] Emanuel Velzi commented on HADOOP-18793: Hi [~ste...@apache.org] I have a similar issue related

[jira] [Commented] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741150#comment-17741150 ] ASF GitHub Bot commented on HADOOP-18793: - hadoop-yetus commented on PR #5818: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5818: HADOOP-18793: S3A StagingCommitter does not clean up staging-uploads directory

hadoop-yetus commented on PR #5818: URL: https://github.com/apache/hadoop/pull/5818#issuecomment-1625745185 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] xinglin commented on a diff in pull request #5809: HDFS-17068. Datanode should record last directory scan time.

xinglin commented on code in PR #5809: URL: https://github.com/apache/hadoop/pull/5809#discussion_r1256166496 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java: ## @@ -3811,5 +3818,10 @@ void

[jira] [Commented] (HADOOP-18618) Support custom property for credential provider path

[ https://issues.apache.org/jira/browse/HADOOP-18618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741144#comment-17741144 ] ASF GitHub Bot commented on HADOOP-18618: - hadoop-yetus commented on PR #5352: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5352: HADOOP-18618 : Support custom property for credential provider.

hadoop-yetus commented on PR #5352: URL: https://github.com/apache/hadoop/pull/5352#issuecomment-1625717179 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] xinglin commented on a diff in pull request #5817: HDFS-17073. Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute"

xinglin commented on code in PR #5817: URL: https://github.com/apache/hadoop/pull/5817#discussion_r1256116824 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockChecksumHelper.java: ## @@ -493,7 +493,8 @@ void compute() throws

[GitHub] [hadoop] xinglin commented on a diff in pull request #5817: HDFS-17073. Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute"

xinglin commented on code in PR #5817: URL: https://github.com/apache/hadoop/pull/5817#discussion_r1256116824 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockChecksumHelper.java: ## @@ -493,7 +493,8 @@ void compute() throws

[jira] [Commented] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741101#comment-17741101 ] ASF GitHub Bot commented on HADOOP-18793: - hdaikoku commented on code in PR #5818: URL:

[GitHub] [hadoop] hdaikoku commented on a diff in pull request #5818: HADOOP-18793: S3A StagingCommitter does not clean up staging-uploads directory

hdaikoku commented on code in PR #5818: URL: https://github.com/apache/hadoop/pull/5818#discussion_r1256030064 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/staging/StagingCommitter.java: ## @@ -556,14 +574,6 @@ protected void

[jira] [Updated] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18793: Labels: pull-request-available (was: ) > S3A StagingCommitter does not clean up

[jira] [Commented] (HADOOP-18793) S3A StagingCommitter does not clean up staging-uploads directory

[ https://issues.apache.org/jira/browse/HADOOP-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741095#comment-17741095 ] ASF GitHub Bot commented on HADOOP-18793: - hdaikoku opened a new pull request, #5818: URL:

[GitHub] [hadoop] hdaikoku opened a new pull request, #5818: HADOOP-18793: S3A StagingCommitter does not clean up staging-uploads directory

hdaikoku opened a new pull request, #5818: URL: https://github.com/apache/hadoop/pull/5818 ### Description of PR This PR fixes a bug in StagingCommitter, which leaks the staging uploads directory, by deleting the directory in `StagingCommitter#cleanup()`. ### How was this

[GitHub] [hadoop] hadoop-yetus commented on pull request #5815: HDFS-17033. Update fsck to display stale state info of blocks accurately

hadoop-yetus commented on PR #5815: URL: https://github.com/apache/hadoop/pull/5815#issuecomment-1625581549 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5817: HDFS-17073. Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute"

hadoop-yetus commented on PR #5817: URL: https://github.com/apache/hadoop/pull/5817#issuecomment-1625559843 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5809: HDFS-17068. Datanode should record last directory scan time.

hadoop-yetus commented on PR #5809: URL: https://github.com/apache/hadoop/pull/5809#issuecomment-1625511667 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5809: HDFS-17068. Datanode should record last directory scan time.

hadoop-yetus commented on PR #5809: URL: https://github.com/apache/hadoop/pull/5809#issuecomment-1625472010 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #5747: YARN-7707. BackPort [GPG] Policy generator framework.

slfan1989 commented on PR #5747: URL: https://github.com/apache/hadoop/pull/5747#issuecomment-1625420119 @goiri This PR cannot be triggered by Jenkins and displays the following error message. The error is not caused by our PR. I have resubmitted a new PR(#5810), and it can be compiled

[GitHub] [hadoop] slfan1989 commented on pull request #5752: YARN-11515. [Federation] Improve DefaultRequestInterceptor#init Code.

slfan1989 commented on PR #5752: URL: https://github.com/apache/hadoop/pull/5752#issuecomment-1625403958 @goiri Can you help review this pr? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] slfan1989 commented on pull request #5797: YARN-11524. Improve the Policy Description in Federation.md.

slfan1989 commented on PR #5797: URL: https://github.com/apache/hadoop/pull/5797#issuecomment-1625403214 @goiri Can you help review this pr? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] hadoop-yetus commented on pull request #5816: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues.

hadoop-yetus commented on PR #5816: URL: https://github.com/apache/hadoop/pull/5816#issuecomment-1625329768 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5797: YARN-11524. Improve the Policy Description in Federation.md.

hadoop-yetus commented on PR #5797: URL: https://github.com/apache/hadoop/pull/5797#issuecomment-1625215427 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] haiyang1987 opened a new pull request, #5817: HDFS-17073. Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute"

haiyang1987 opened a new pull request, #5817: URL: https://github.com/apache/hadoop/pull/5817 ### Description of PR https://issues.apache.org/jira/browse/HDFS-17073 Consider improving the log output of the warning messages generated by BlockGroupNonStripedChecksumComputer when

[GitHub] [hadoop] MEILIDEKCL commented on pull request #5808: Add documention for HDFS-17069.

MEILIDEKCL commented on PR #5808: URL: https://github.com/apache/hadoop/pull/5808#issuecomment-1625060321 @ayushtkn @Hexiaoqiao , hello, can you have a look? Thank you very much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] slfan1989 opened a new pull request, #5816: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues.

slfan1989 opened a new pull request, #5816: URL: https://github.com/apache/hadoop/pull/5816 ### Description of PR JIRA: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues. ### How was this patch tested? ### For code

[GitHub] [hadoop] zhtttylz commented on a diff in pull request #5815: HDFS-17033. Update fsck to display stale state info of blocks accurately

zhtttylz commented on code in PR #5815: URL: https://github.com/apache/hadoop/pull/5815#discussion_r1255425461 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java: ## @@ -1681,6 +1682,66 @@ public Boolean get() {

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5809: HDFS-17068. Datanode should record last directory scan time.

hfutatzhanghb commented on code in PR #5809: URL: https://github.com/apache/hadoop/pull/5809#discussion_r1255419327 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/FSDatasetMBean.java: ## @@ -122,4 +122,9 @@ public interface

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5809: HDFS-17068. Datanode should record last directory scan time.

Hexiaoqiao commented on code in PR #5809: URL: https://github.com/apache/hadoop/pull/5809#discussion_r1255390061 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/FSDatasetMBean.java: ## @@ -122,4 +122,9 @@ public interface

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5753: HDFS-17050. Erasure coding: fix bug for invalidating duplicated block when two ec block at the same datanode but different storage.

hfutatzhanghb commented on PR #5753: URL: https://github.com/apache/hadoop/pull/5753#issuecomment-1624899228 Have moved to HDFS-17071. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] hfutatzhanghb closed pull request #5753: HDFS-17050. Erasure coding: fix bug for invalidating duplicated block when two ec block at the same datanode but different storage.

hfutatzhanghb closed pull request #5753: HDFS-17050. Erasure coding: fix bug for invalidating duplicated block when two ec block at the same datanode but different storage. URL: https://github.com/apache/hadoop/pull/5753 -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5815: HDFS-17033. Update fsck to display stale state info of blocks accurately

Hexiaoqiao commented on code in PR #5815: URL: https://github.com/apache/hadoop/pull/5815#discussion_r1255363924 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java: ## @@ -1681,6 +1682,66 @@ public Boolean get() {