[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642320#comment-17642320 ] ASF GitHub Bot commented on HADOOP-18546: - pranavsaxena-microsoft commented on

[GitHub] [hadoop] pranavsaxena-microsoft commented on a diff in pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

pranavsaxena-microsoft commented on code in PR #5176: URL: https://github.com/apache/hadoop/pull/5176#discussion_r1037840780 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsInputStream.java: ## @@ -495,6 +509,199 @@ public void testSucce

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642319#comment-17642319 ] ASF GitHub Bot commented on HADOOP-18546: - pranavsaxena-microsoft commented on

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642318#comment-17642318 ] ASF GitHub Bot commented on HADOOP-18546: - pranavsaxena-microsoft commented on

[GitHub] [hadoop] pranavsaxena-microsoft commented on a diff in pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

pranavsaxena-microsoft commented on code in PR #5176: URL: https://github.com/apache/hadoop/pull/5176#discussion_r1037840233 ## hadoop-common-project/hadoop-common/src/main/resources/core-default.xml: ## @@ -2166,13 +2166,6 @@ The switch to turn S3A auditing on or off. The Ab

[GitHub] [hadoop] pranavsaxena-microsoft commented on a diff in pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

pranavsaxena-microsoft commented on code in PR #5176: URL: https://github.com/apache/hadoop/pull/5176#discussion_r1037839979 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsInputStream.java: ## @@ -82,6 +84,16 @@ public class TestAbfsInp

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642317#comment-17642317 ] ASF GitHub Bot commented on HADOOP-18546: - pranavsaxena-microsoft commented on

[GitHub] [hadoop] pranavsaxena-microsoft commented on pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

pranavsaxena-microsoft commented on PR #5176: URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1334834625 - Test results - [INFO] Results: [INFO] [ERROR] Failures: [ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expecte

[GitHub] [hadoop] dingshun3016 commented on pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

dingshun3016 commented on PR #5050: URL: https://github.com/apache/hadoop/pull/5050#issuecomment-1334807222 @tasanuma Thanks for review, these opinions are very useful. I have fixed them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop] hadoop-yetus commented on pull request #4744: HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer

hadoop-yetus commented on PR #4744: URL: https://github.com/apache/hadoop/pull/4744#issuecomment-1334790364 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642284#comment-17642284 ] ASF GitHub Bot commented on HADOOP-18546: - snvijaya commented on code in PR #51

[GitHub] [hadoop] snvijaya commented on a diff in pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

snvijaya commented on code in PR #5176: URL: https://github.com/apache/hadoop/pull/5176#discussion_r1037798793 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsInputStream.java: ## @@ -495,6 +509,199 @@ public void testSuccessfulReadAhead

[jira] [Assigned] (HADOOP-18548) Hadoop Archive tool (HAR) should acquire delegation tokens from source and destination file systems

[ https://issues.apache.org/jira/browse/HADOOP-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HADOOP-18548: Assignee: Szabolcs Gál (was: Wei-Chiu Chuang) > Hadoop Archive tool (HAR) shoul

[jira] [Commented] (HADOOP-18543) AliyunOSS: AliyunOSSFileSystem#open(Path path, int bufferSize) should use buffer size as its downloadPartSize

[ https://issues.apache.org/jira/browse/HADOOP-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642272#comment-17642272 ] ASF GitHub Bot commented on HADOOP-18543: - masteryhx commented on PR #5172: URL

[GitHub] [hadoop] masteryhx commented on pull request #5172: HADOOP-18543. AliyunOSSFileSystem#open(Path path, int bufferSize) use buffer size as its downloadPartSize

masteryhx commented on PR #5172: URL: https://github.com/apache/hadoop/pull/5172#issuecomment-1334751421 > sorry, but I'm going to say -1 to using the normal IO buffer size as the GET range. The default value of 4k is way too small even for parquet/orc reads, it will break all existing

[jira] [Assigned] (HADOOP-18548) Hadoop Archive tool (HAR) should acquire delegation tokens from source and destination file systems

[ https://issues.apache.org/jira/browse/HADOOP-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HADOOP-18548: Assignee: Wei-Chiu Chuang > Hadoop Archive tool (HAR) should acquire delegation

[jira] [Created] (HADOOP-18548) Hadoop Archive tool (HAR) should acquire delegation tokens from source and destination file systems

Wei-Chiu Chuang created HADOOP-18548: Summary: Hadoop Archive tool (HAR) should acquire delegation tokens from source and destination file systems Key: HADOOP-18548 URL: https://issues.apache.org/jira/browse/H

[GitHub] [hadoop] MingXiangLi commented on pull request #5170: HDFS-16855. Remove the redundant write lock in addBlockPool.

MingXiangLi commented on PR #5170: URL: https://github.com/apache/hadoop/pull/5170#issuecomment-1334745101 `Now that, this case only happen when invoke addBlockPool() and CachingGetSpaceUsed#used < 0, I have an idea, is it possible to add a switch, not add lock when ReplicaCachingGetSpaceUs

[GitHub] [hadoop] hadoop-yetus commented on pull request #5123: HDFS-16837. [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespaces

hadoop-yetus commented on PR #5123: URL: https://github.com/apache/hadoop/pull/5123#issuecomment-1334728452 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] slfan1989 commented on pull request #5179: YARN-11381. Fix hadoop-yarn-common module Java Doc Errors.

slfan1989 commented on PR #5179: URL: https://github.com/apache/hadoop/pull/5179#issuecomment-1334704242 @ayushtkn Can you help review this PR? Thank you very much! We still have some package-info.java files to fix, in this pr, I fixed them. -- This is an automated message from the Apache

[GitHub] [hadoop] hadoop-yetus commented on pull request #5146: YARN-11373. [Federation] Support refreshQueues refreshNodes API's for Federation.

hadoop-yetus commented on PR #5146: URL: https://github.com/apache/hadoop/pull/5146#issuecomment-1334691719 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] tasanuma commented on a diff in pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

tasanuma commented on code in PR #5050: URL: https://github.com/apache/hadoop/pull/5050#discussion_r1037731995 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminManager.java: ## @@ -353,7 +353,7 @@ protected boolean isSuf

[GitHub] [hadoop] tasanuma commented on a diff in pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

tasanuma commented on code in PR #5050: URL: https://github.com/apache/hadoop/pull/5050#discussion_r1037731995 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminManager.java: ## @@ -353,7 +353,7 @@ protected boolean isSuf

[GitHub] [hadoop] hadoop-yetus commented on pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

hadoop-yetus commented on PR #5050: URL: https://github.com/apache/hadoop/pull/5050#issuecomment-1334679750 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] tasanuma commented on a diff in pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

tasanuma commented on code in PR #5050: URL: https://github.com/apache/hadoop/pull/5050#discussion_r1037704437 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -1117,7 +1117,7 @@ public short getMinReplicatio

[GitHub] [hadoop] hadoop-yetus commented on pull request #5180: HDFS-16858. Dynamically adjust max slow disks to exclude.

hadoop-yetus commented on PR #5180: URL: https://github.com/apache/hadoop/pull/5180#issuecomment-1334609666 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] slfan1989 commented on pull request #5104: YARN-11158. Support (Create/Renew/Cancel) DelegationToken API's for Federation.

slfan1989 commented on PR #5104: URL: https://github.com/apache/hadoop/pull/5104#issuecomment-1334584040 @goiri Thank you very much for helping to review the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[jira] [Commented] (HADOOP-18526) Leak of S3AInstrumentation instances via hadoop Metrics references

[ https://issues.apache.org/jira/browse/HADOOP-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642175#comment-17642175 ] ASF GitHub Bot commented on HADOOP-18526: - hadoop-yetus commented on PR #5144:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5144: HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references

hadoop-yetus commented on PR #5144: URL: https://github.com/apache/hadoop/pull/5144#issuecomment-1334467998 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] goiri merged pull request #5104: YARN-11158. Support (Create/Renew/Cancel) DelegationToken API's for Federation.

goiri merged PR #5104: URL: https://github.com/apache/hadoop/pull/5104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] hadoop-yetus commented on pull request #4872: HDFS-16764. ObserverNamenode should throw ObserverRetryOnActiveException instead of FileNotFoundException during processing of addBlock

hadoop-yetus commented on PR #4872: URL: https://github.com/apache/hadoop/pull/4872#issuecomment-1334414322 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #4744: HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer

hadoop-yetus commented on PR #4744: URL: https://github.com/apache/hadoop/pull/4744#issuecomment-1334405016 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #4963: YARN-11326. [Federation] Add RM FederationStateStoreService Metrics.

hadoop-yetus commented on PR #4963: URL: https://github.com/apache/hadoop/pull/4963#issuecomment-1334238015 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5180: HDFS-16858. Dynamically adjust max slow disks to exclude.

hadoop-yetus commented on PR #5180: URL: https://github.com/apache/hadoop/pull/5180#issuecomment-1334219465 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642112#comment-17642112 ] ASF GitHub Bot commented on HADOOP-18546: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

steveloughran commented on code in PR #5176: URL: https://github.com/apache/hadoop/pull/5176#discussion_r1037390997 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsInputStream.java: ## @@ -82,6 +84,16 @@ public class TestAbfsInputStream

[GitHub] [hadoop] hadoop-yetus commented on pull request #4963: YARN-11326. [Federation] Add RM FederationStateStoreService Metrics.

hadoop-yetus commented on PR #4963: URL: https://github.com/apache/hadoop/pull/4963#issuecomment-1334156740 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18543) AliyunOSS: AliyunOSSFileSystem#open(Path path, int bufferSize) should use buffer size as its downloadPartSize

[ https://issues.apache.org/jira/browse/HADOOP-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642065#comment-17642065 ] ASF GitHub Bot commented on HADOOP-18543: - steveloughran commented on PR #5172:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5056: YARN-11358. [Federation] Add FederationInterceptor#allow-partial-result config.

hadoop-yetus commented on PR #5056: URL: https://github.com/apache/hadoop/pull/5056#issuecomment-1334109514 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] steveloughran commented on pull request #5172: HADOOP-18543. AliyunOSSFileSystem#open(Path path, int bufferSize) use buffer size as its downloadPartSize

steveloughran commented on PR #5172: URL: https://github.com/apache/hadoop/pull/5172#issuecomment-1334109192 (note also that includes letting you declare read policy (whole-file, sequential, random, vectoredthat can be used to change default block size too) -- This is an automated me

[jira] [Commented] (HADOOP-18543) AliyunOSS: AliyunOSSFileSystem#open(Path path, int bufferSize) should use buffer size as its downloadPartSize

[ https://issues.apache.org/jira/browse/HADOOP-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642064#comment-17642064 ] ASF GitHub Bot commented on HADOOP-18543: - steveloughran commented on PR #5172:

[GitHub] [hadoop] steveloughran commented on pull request #5172: HADOOP-18543. AliyunOSSFileSystem#open(Path path, int bufferSize) use buffer size as its downloadPartSize

steveloughran commented on PR #5172: URL: https://github.com/apache/hadoop/pull/5172#issuecomment-1334108437 sorry, but I'm going to say -1 to using the normal IO buffer size as the GET range. The default value of 4k is way too small even for parquet/orc reads, it will break all existing ap

[jira] [Commented] (HADOOP-18543) AliyunOSS: AliyunOSSFileSystem#open(Path path, int bufferSize) should use buffer size as its downloadPartSize

[ https://issues.apache.org/jira/browse/HADOOP-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642061#comment-17642061 ] ASF GitHub Bot commented on HADOOP-18543: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5172: HADOOP-18543. AliyunOSSFileSystem#open(Path path, int bufferSize) use buffer size as its downloadPartSize

steveloughran commented on code in PR #5172: URL: https://github.com/apache/hadoop/pull/5172#discussion_r1037378725 ## hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSInputStream.java: ## @@ -57,18 +57,21 @@ public class AliyunOSSInputStream ext

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642057#comment-17642057 ] ASF GitHub Bot commented on HADOOP-18546: - hadoop-yetus commented on PR #5176:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

hadoop-yetus commented on PR #5176: URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1334095330 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5131: YARN-11350. [Federation] Router Support DelegationToken With ZK.

hadoop-yetus commented on PR #5131: URL: https://github.com/apache/hadoop/pull/5131#issuecomment-1334094867 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18526) Leak of S3AInstrumentation instances via hadoop Metrics references

[ https://issues.apache.org/jira/browse/HADOOP-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642055#comment-17642055 ] ASF GitHub Bot commented on HADOOP-18526: - steveloughran commented on PR #5144:

[GitHub] [hadoop] steveloughran commented on pull request #5144: HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references

steveloughran commented on PR #5144: URL: https://github.com/apache/hadoop/pull/5144#issuecomment-1334090863 build vms playing up "Resource temporarily unavailable" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [hadoop] hadoop-yetus commented on pull request #5056: YARN-11358. [Federation] Add FederationInterceptor#allow-partial-result config.

hadoop-yetus commented on PR #5056: URL: https://github.com/apache/hadoop/pull/5056#issuecomment-1334040109 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5146: YARN-11373. [Federation] Support refreshQueues refreshNodes API's for Federation.

hadoop-yetus commented on PR #5146: URL: https://github.com/apache/hadoop/pull/5146#issuecomment-1334009172 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5123: HDFS-16837. [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespaces

hadoop-yetus commented on PR #5123: URL: https://github.com/apache/hadoop/pull/5123#issuecomment-1333976086 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4963: YARN-11326. [Federation] Add RM FederationStateStoreService Metrics.

slfan1989 commented on code in PR #4963: URL: https://github.com/apache/hadoop/pull/4963#discussion_r1037263358 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/federation/TestFederationR

[GitHub] [hadoop] dingshun3016 commented on pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

dingshun3016 commented on PR #5050: URL: https://github.com/apache/hadoop/pull/5050#issuecomment-1333939639 @slfan1989 Thank you for your review，I have fixed it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] hadoop-yetus commented on pull request #5175: YARN-11226. [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router.

hadoop-yetus commented on PR #5175: URL: https://github.com/apache/hadoop/pull/5175#issuecomment-1333924621 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] slfan1989 commented on pull request #5104: YARN-11158. Support (Create/Renew/Cancel) DelegationToken API's for Federation.

slfan1989 commented on PR #5104: URL: https://github.com/apache/hadoop/pull/5104#issuecomment-1333922605 @goiri Thank you very much for helping to review the code, can you help me merge this pr into the trunk branch? I will continue to follow YARN-11225. [Federation] Add postDelegationToken

[GitHub] [hadoop] hadoop-yetus commented on pull request #5170: HDFS-16855. Remove the redundant write lock in addBlockPool.

hadoop-yetus commented on PR #5170: URL: https://github.com/apache/hadoop/pull/5170#issuecomment-1333921434 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642002#comment-17642002 ] ASF GitHub Bot commented on HADOOP-18546: - sreeb-msft commented on PR #5176: UR

[GitHub] [hadoop] sreeb-msft commented on pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

sreeb-msft commented on PR #5176: URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1333917331 Changes and simpler tests look okay to me. Approving once the yetus build results are here. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [hadoop] slfan1989 commented on pull request #5146: YARN-11373. [Federation] Support refreshQueues refreshNodes API's for Federation.

slfan1989 commented on PR #5146: URL: https://github.com/apache/hadoop/pull/5146#issuecomment-1333915412 @goiri Thank you very much for your help in reviewing the code! During the compilation process in this pr, there were java doc errors, but these errors were not caused by our code

[GitHub] [hadoop] dingshun3016 commented on pull request #5180: HDFS-16858. Dynamically adjust max slow disks to exclude.

dingshun3016 commented on PR #5180: URL: https://github.com/apache/hadoop/pull/5180#issuecomment-1333909695 @slfan1989 Thank you for your review，I have fixed it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] slfan1989 commented on pull request #5175: YARN-11226. [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router.

slfan1989 commented on PR #5175: URL: https://github.com/apache/hadoop/pull/5175#issuecomment-1333902261 @goiri Can you help review this PR? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (HADOOP-18526) Leak of S3AInstrumentation instances via hadoop Metrics references

[ https://issues.apache.org/jira/browse/HADOOP-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641988#comment-17641988 ] ASF GitHub Bot commented on HADOOP-18526: - hadoop-yetus commented on PR #5144:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5144: HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references

hadoop-yetus commented on PR #5144: URL: https://github.com/apache/hadoop/pull/5144#issuecomment-1333896168 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5179: YARN-11381. Fix hadoop-yarn-common module Java Doc Errors.

hadoop-yetus commented on PR #5179: URL: https://github.com/apache/hadoop/pull/5179#issuecomment-1333878976 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] slfan1989 commented on pull request #5180: HDFS-16858. Dynamically adjust max slow disks to exclude.

slfan1989 commented on PR #5180: URL: https://github.com/apache/hadoop/pull/5180#issuecomment-1333852890 @dingshun3016 Thank you very much for your contribution, but we need to fix some checkstyle issues. -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5180: HDFS-16858. Dynamically adjust max slow disks to exclude.

slfan1989 commented on code in PR #5180: URL: https://github.com/apache/hadoop/pull/5180#discussion_r1037177179 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeReconfiguration.java: ## @@ -673,12 +654,16 @@ public void testSlow

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5180: HDFS-16858. Dynamically adjust max slow disks to exclude.

slfan1989 commented on code in PR #5180: URL: https://github.com/apache/hadoop/pull/5180#discussion_r1037176907 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeReconfiguration.java: ## @@ -18,33 +18,12 @@ package org.apache.

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5180: HDFS-16858. Dynamically adjust max slow disks to exclude.

slfan1989 commented on code in PR #5180: URL: https://github.com/apache/hadoop/pull/5180#discussion_r1037176541 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java: ## @@ -18,67 +18,12 @@ package org.apache.hadoop.hdfs.server.da

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

slfan1989 commented on code in PR #5050: URL: https://github.com/apache/hadoop/pull/5050#discussion_r1037176029 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMaintenanceWithStriped.java: ## @@ -0,0 +1,281 @@ +/** + * Licensed to the Apache Software

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

slfan1989 commented on code in PR #5050: URL: https://github.com/apache/hadoop/pull/5050#discussion_r1037175690 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMaintenanceWithStriped.java: ## @@ -0,0 +1,281 @@ +/** + * Licensed to the Apache Software

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

slfan1989 commented on code in PR #5050: URL: https://github.com/apache/hadoop/pull/5050#discussion_r1037175063 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMaintenanceWithStriped.java: ## @@ -0,0 +1,281 @@ +/** + * Licensed to the Apache Software

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5050: HDFS-16809. EC striped block is not sufficient when doing in maintenance.

slfan1989 commented on code in PR #5050: URL: https://github.com/apache/hadoop/pull/5050#discussion_r1037174062 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMaintenanceWithStriped.java: ## @@ -0,0 +1,281 @@ +/** + * Licensed to the Apache Software

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5056: YARN-11358. [Federation] Add FederationInterceptor#allow-partial-result config.

slfan1989 commented on code in PR #5056: URL: https://github.com/apache/hadoop/pull/5056#discussion_r1037165134 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestFederationInterceptorRESTRetry.ja

[GitHub] [hadoop] szilard-nemeth closed pull request #5173: YARN-10946. Moved QueueInfo creation to separate class

szilard-nemeth closed pull request #5173: YARN-10946. Moved QueueInfo creation to separate class URL: https://github.com/apache/hadoop/pull/5173 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [hadoop] szilard-nemeth commented on pull request #5173: YARN-10946. Moved QueueInfo creation to separate class

szilard-nemeth commented on PR #5173: URL: https://github.com/apache/hadoop/pull/5173#issuecomment-1333829301 Thanks @p-szucs for working on this. Change looks good to me, committed to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop] ZanderXu commented on a diff in pull request #4872: HDFS-16764. ObserverNamenode should throw ObserverRetryOnActiveException instead of FileNotFoundException during processing of add

ZanderXu commented on code in PR #4872: URL: https://github.com/apache/hadoop/pull/4872#discussion_r1037138461 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -3095,12 +3095,12 @@ LocatedBlock getAdditionalDatanode

[GitHub] [hadoop] ZanderXu commented on a diff in pull request #4744: HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer

ZanderXu commented on code in PR #4744: URL: https://github.com/apache/hadoop/pull/4744#discussion_r1037125839 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java: ## @@ -1666,6 +1670,9 @@ synchronized void recoverUnclosedStream

[GitHub] [hadoop] dingshun3016 opened a new pull request, #5180: HDFS-16858. Dynamically adjust max slow disks to exclude.

dingshun3016 opened a new pull request, #5180: URL: https://github.com/apache/hadoop/pull/5180 JIRA: [HDFS-16858](https://issues.apache.org/jira/browse/HDFS-16858). In large clusters, rolling restart datanodes takes long time. We can make slow disks parameters in datanode reconfigurab

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641903#comment-17641903 ] ASF GitHub Bot commented on HADOOP-18546: - pranavsaxena-microsoft commented on

[GitHub] [hadoop] pranavsaxena-microsoft commented on a diff in pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

pranavsaxena-microsoft commented on code in PR #5176: URL: https://github.com/apache/hadoop/pull/5176#discussion_r1037084565 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsInputStream.java: ## @@ -495,6 +509,199 @@ public void testSucce

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641901#comment-17641901 ] ASF GitHub Bot commented on HADOOP-18546: - pranavsaxena-microsoft commented on

[GitHub] [hadoop] pranavsaxena-microsoft commented on pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

pranavsaxena-microsoft commented on PR #5176: URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1333736063 > ReadAhead feature can reenabled back by default as we are undoing the known problem in the corruption issue reported before. Please include the change into this PR. >

[GitHub] [hadoop] slfan1989 opened a new pull request, #5179: YARN-11381. Fix hadoop-yarn-common module Java Doc Errors.

slfan1989 opened a new pull request, #5179: URL: https://github.com/apache/hadoop/pull/5179 JIRA: YARN-11381. Fix hadoop-yarn-common module Java Doc Errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop] dingshun3016 commented on pull request #5170: HDFS-16855. Remove the redundant write lock in addBlockPool.

dingshun3016 commented on PR #5170: URL: https://github.com/apache/hadoop/pull/5170#issuecomment-1333710803 according to the situation discussed so far, it seems that there are several ways to solve this problem - remove the BLOCK_POOl level write lock in #addBlockPool > but worr

[GitHub] [hadoop] ZanderXu commented on pull request #5123: HDFS-16837. [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespaces

ZanderXu commented on PR #5123: URL: https://github.com/apache/hadoop/pull/5123#issuecomment-1333701800 > @ZanderXu I committed some changes to the tests which are causing merge conflicts for your pull requests. There are just two conflicts though. @simbadzina Sir, thanks. I have upda

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641865#comment-17641865 ] ASF GitHub Bot commented on HADOOP-18546: - snvijaya commented on code in PR #51

[GitHub] [hadoop] snvijaya commented on a diff in pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

snvijaya commented on code in PR #5176: URL: https://github.com/apache/hadoop/pull/5176#discussion_r1037025160 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsInputStream.java: ## @@ -495,6 +509,199 @@ public void testSuccessfulReadAhead

[jira] [Commented] (HADOOP-18526) Leak of S3AInstrumentation instances via hadoop Metrics references

[ https://issues.apache.org/jira/browse/HADOOP-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641839#comment-17641839 ] ASF GitHub Bot commented on HADOOP-18526: - steveloughran commented on PR #5144:

[GitHub] [hadoop] steveloughran commented on pull request #5144: HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references

steveloughran commented on PR #5144: URL: https://github.com/apache/hadoop/pull/5144#issuecomment-1333603252 tested aws london w/ ` -Dparallel-tests -DtestsThreadCount=10 -Dmarkers=delete` s3a fs was set to log at TRACE so print stack traces on all fs creation...just search for "crea

[jira] [Commented] (HADOOP-18546) disable purging list of in progress reads in abfs stream closed

[ https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641836#comment-17641836 ] ASF GitHub Bot commented on HADOOP-18546: - hadoop-yetus commented on PR #5176:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5176: HADOOP-18546. ABFS:disable purging list of in progress reads in abfs stream closed

hadoop-yetus commented on PR #5176: URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1333598564 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5178: 增加 kvstart kvend kvindex 转换方法，转换成kvbuffer对应的坐标；方便理解

hadoop-yetus commented on PR #5178: URL: https://github.com/apache/hadoop/pull/5178#issuecomment-1333449798 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18547) Check if config value is not empty string in AbfsConfiguration.getMandatoryPasswordString()

[ https://issues.apache.org/jira/browse/HADOOP-18547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641766#comment-17641766 ] ASF GitHub Bot commented on HADOOP-18547: - hadoop-yetus commented on PR #5177:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5177: HADOOP-18547. Check if config value is not empty string in AbfsConfiguration.getMandatoryPasswordString()

hadoop-yetus commented on PR #5177: URL: https://github.com/apache/hadoop/pull/5177#issuecomment-133349 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5175: YARN-11226. [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router.

hadoop-yetus commented on PR #5175: URL: https://github.com/apache/hadoop/pull/5175#issuecomment-182387 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] MingXiangLi commented on pull request #5170: HDFS-16855. Remove the redundant write lock in addBlockPool.

MingXiangLi commented on PR #5170: URL: https://github.com/apache/hadoop/pull/5170#issuecomment-172902 The BLOCK_POOl level lock is to protect replica consistency for FsDataSetImpl when read or write operating happend at same time. > forbid refresh() when ReplicaCachingGetSpaceUse