[GitHub] [hadoop] huangzhaobo99 opened a new pull request, #6099: HDFS-17200. Add some datanode related metrics to Metrics.md.

huangzhaobo99 opened a new pull request, #6099: URL: https://github.com/apache/hadoop/pull/6099 https://issues.apache.org/jira/browse/HDFS-17200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Created] (HADOOP-18909) Add some datanode related metrics to Metrics.md

huangzhaobo99 created HADOOP-18909: -- Summary: Add some datanode related metrics to Metrics.md Key: HADOOP-18909 URL: https://issues.apache.org/jira/browse/HADOOP-18909 Project: Hadoop Common

[GitHub] [hadoop] hadoop-yetus commented on pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

hadoop-yetus commented on PR #6055: URL: https://github.com/apache/hadoop/pull/6055#issuecomment-1724914990 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329637115 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringB

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329632148 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringB

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329630792 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringB

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329629701 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringB

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329626965 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -761,6 +763,8 @@ public AbfsRestOperation append(final St

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329626476 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringB

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329626349 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringB

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329625843 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringB

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329625546 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringB

[GitHub] [hadoop] hadoop-yetus commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

hadoop-yetus commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1724875062 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

slfan1989 commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1329599915 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/TestZookeeperFederationStateS

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

slfan1989 commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1329595591 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

slfan1989 commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1329595285 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

slfan1989 commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1329592879 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

slfan1989 commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1329590819 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

slfan1989 commented on code in PR #6083: URL: https://github.com/apache/hadoop/pull/6083#discussion_r1329580078 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/utils/FederationRegistryClient.java: ###

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

slfan1989 commented on code in PR #6083: URL: https://github.com/apache/hadoop/pull/6083#discussion_r1329579878 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator/src/main/java/org/apache/hadoop/yarn/server/globalpolicygenerator/applicationcle

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

slfan1989 commented on code in PR #6083: URL: https://github.com/apache/hadoop/pull/6083#discussion_r1329577079 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/utils/FederationRegistryClient.java: ###

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

saxenapranav commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1329551732 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -315,6 +315,20 @@ public void delete(OperationContext opConte

[GitHub] [hadoop] hadoop-yetus commented on pull request #6098: YARN-11573. Add config option to make container allocation prefer nodes without reserved containers

hadoop-yetus commented on PR #6098: URL: https://github.com/apache/hadoop/pull/6098#issuecomment-1724800045 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

Hexiaoqiao commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1329530784 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java: ## @@ -1468,6 +1475,30 @@ static Collection normalizePaths(

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

Hexiaoqiao commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1329529923 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java: ## @@ -470,17 +475,53 @@ static RenameResult unprotectedRe

[jira] [Resolved] (HADOOP-18906) Increase default batch size of ZKDTSM token seqnum to reduce overflow speed of zonde dataVersion.

[ https://issues.apache.org/jira/browse/HADOOP-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HADOOP-18906. -- Fix Version/s: 3.4.0 3.3.9 Hadoop Flags: Reviewed Resolution: F

[jira] [Commented] (HADOOP-18906) Increase default batch size of ZKDTSM token seqnum to reduce overflow speed of zonde dataVersion.

[ https://issues.apache.org/jira/browse/HADOOP-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766624#comment-17766624 ] Xiaoqiao He commented on HADOOP-18906: -- Sure, will backport to branch-3.3 while ch

[GitHub] [hadoop] szilard-nemeth opened a new pull request, #6098: YARN-11573. Add config option to make container allocation prefer nodes without reserved containers

szilard-nemeth opened a new pull request, #6098: URL: https://github.com/apache/hadoop/pull/6098 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. '

[jira] [Commented] (HADOOP-18906) Increase default batch size of ZKDTSM token seqnum to reduce overflow speed of zonde dataVersion.

[ https://issues.apache.org/jira/browse/HADOOP-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766568#comment-17766568 ] Wei-Chiu Chuang commented on HADOOP-18906: -- Cool I was going to raise this too

[GitHub] [hadoop] hadoop-yetus commented on pull request #2964: HDFS-16000. Rename performance optimization

hadoop-yetus commented on PR #2964: URL: https://github.com/apache/hadoop/pull/2964#issuecomment-1724236459 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766547#comment-17766547 ] Steve Loughran commented on HADOOP-18908: - and a way to turn it off, even for a

[GitHub] [hadoop] fengnanli merged pull request #6097: HADOOP-18906. Increase default batch size of ZKDTSM token seqnum to reduce overflow speed of zonde dataVersion.

fengnanli merged PR #6097: URL: https://github.com/apache/hadoop/pull/6097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had

[GitHub] [hadoop] goiri merged pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…

goiri merged PR #5921: URL: https://github.com/apache/hadoop/pull/5921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] goiri commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…

goiri commented on PR #5921: URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1723920841 Unfortunate that these tests are so flaky. It'd be good to tackle TestDirectoryScanner. I'll go ahead merging this as it is unrelated. -- This is an automated message from the Apache

[GitHub] [hadoop] goiri commented on a diff in pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

goiri commented on code in PR #6096: URL: https://github.com/apache/hadoop/pull/6096#discussion_r1329005745 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java: ## @@ -223,6 +223,57 @@ public void

[GitHub] [hadoop] goiri commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

goiri commented on code in PR #6083: URL: https://github.com/apache/hadoop/pull/6083#discussion_r1328986100 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/utils/FederationRegistryClient.java: ###

[GitHub] [hadoop] hadoop-yetus commented on pull request #6025: Pass down eTag as part of accessCondition to SDK

hadoop-yetus commented on PR #6025: URL: https://github.com/apache/hadoop/pull/6025#issuecomment-1723707862 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6025: Pass down eTag as part of accessCondition to SDK

hadoop-yetus commented on PR #6025: URL: https://github.com/apache/hadoop/pull/6025#issuecomment-1723605099 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5989: YARN-11514. Extend SchedulerResponse with capacityVector

hadoop-yetus commented on PR #5989: URL: https://github.com/apache/hadoop/pull/5989#issuecomment-1723603709 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5832: HADOOP-18184. S3A Prefetching unbuffer.

hadoop-yetus commented on PR #5832: URL: https://github.com/apache/hadoop/pull/5832#issuecomment-1723585006 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #2964: HDFS-16000. Rename performance optimization

hadoop-yetus commented on PR #2964: URL: https://github.com/apache/hadoop/pull/2964#issuecomment-1723570466 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328786375 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java: ## @@ -1468,6 +1475,30 @@ static Collection normalizePaths(

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328789882 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirMkdirOp.java: ## @@ -222,7 +222,8 @@ private static INodesInPath unprotec

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328786375 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java: ## @@ -1468,6 +1475,30 @@ static Collection normalizePaths(

[GitHub] [hadoop] hadoop-yetus commented on pull request #6045: HDFS-17185. Add acquire lock time for DataSetLockManager

hadoop-yetus commented on PR #6045: URL: https://github.com/apache/hadoop/pull/6045#issuecomment-1723438309 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328734421 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java: ## @@ -470,17 +475,53 @@ static RenameResult unprotectedRe

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328709756 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -425,14 +446,18 @@ public CopyState getCopyState() {

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

saxenapranav commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328685087 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -425,14 +446,18 @@ public CopyState getCopyState() { @

[GitHub] [hadoop] Hexiaoqiao merged pull request #6081: HDFS-17192. Add bock info when constructing remote block reader meets IOException.

Hexiaoqiao merged PR #6081: URL: https://github.com/apache/hadoop/pull/6081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@ha

[GitHub] [hadoop] Hexiaoqiao commented on pull request #6097: HADOOP-18906. Increase default batch size of ZKDTSM token seqnum to reduce overflow speed of zonde dataVersion.

Hexiaoqiao commented on PR #6097: URL: https://github.com/apache/hadoop/pull/6097#issuecomment-1723345123 cc @fengnanli -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #6095: HDFS-17197. Show file replication when listing corrupt files.

Hexiaoqiao commented on code in PR #6095: URL: https://github.com/apache/hadoop/pull/6095#discussion_r1328674605 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -6195,7 +6197,11 @@ Collection listCorruptFileBlocks

[jira] [Updated] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18908: Description: s3a now requires the fs.s3a.endpoint.region to be set; and while it can det

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

Hexiaoqiao commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328663679 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java: ## @@ -470,17 +475,53 @@ static RenameResult unprotectedRe

[jira] [Updated] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18908: Affects Version/s: 3.4.0 > Improve s3a region handling, including determining from endpoi

[jira] [Created] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

Steve Loughran created HADOOP-18908: --- Summary: Improve s3a region handling, including determining from endpoint Key: HADOOP-18908 URL: https://issues.apache.org/jira/browse/HADOOP-18908 Project: Had

[jira] [Updated] (HADOOP-18673) AWS SDK V2 - Refactor getS3Region & other follow up items

[ https://issues.apache.org/jira/browse/HADOOP-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18673: Description: * Factor getS3Region into its own ExecutingStoreOperation; * Fix issue with

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328662715 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWr

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328661066 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java: ## @@ -2761,18 +2816,23 @@ public boolean delete(String key

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328660809 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWr

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328660395 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockStorageInterface.java: ## @@ -450,6 +453,22 @@ public void startCopyFromBlob(CloudBlobWr

[GitHub] [hadoop] anmolanmol1234 commented on pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on PR #6025: URL: https://github.com/apache/hadoop/pull/6025#issuecomment-1723322521 > In the new overloaded methods in which etag is introduced as a parameter. Lets, specify if they are srcEtag or dstEtag. Reason being, from reading that particular method, it becom

[jira] [Created] (HADOOP-18907) ITestS3APrefetchingCacheFiles teardown failure if setup() fails

Steve Loughran created HADOOP-18907: --- Summary: ITestS3APrefetchingCacheFiles teardown failure if setup() fails Key: HADOOP-18907 URL: https://issues.apache.org/jira/browse/HADOOP-18907 Project: Hado

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

saxenapranav commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328597522 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWrap

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

saxenapranav commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328582191 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWrap

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

saxenapranav commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328582191 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWrap

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

saxenapranav commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328582191 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWrap

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328574133 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -315,6 +315,20 @@ public void delete(OperationContext opCon

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328573795 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWr

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328572679 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWr

[GitHub] [hadoop] hadoop-yetus commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…

hadoop-yetus commented on PR #5921: URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1723187323 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5939: HDFS-17152. Fix the documentation of count command in FileSystemShell.md.

hfutatzhanghb commented on PR #5939: URL: https://github.com/apache/hadoop/pull/5939#issuecomment-1723168263 @slfan1989 Hi, sir. Could we push this simple modification forward when you have free time? Thanks a lot~ -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hadoop] haiyang1987 commented on pull request #6094: HDFS-17194. Enhance the log message for striped block recovery

haiyang1987 commented on PR #6094: URL: https://github.com/apache/hadoop/pull/6094#issuecomment-1723136576 The failed unit test seems unrelated to the change. Hi Sirs @Hexiaoqiao @ayushtkn @ZanderXu Could you please help me review this minor changes when you have free time ? Thanks a

[jira] [Commented] (HADOOP-18906) Increase default batch size of ZKDTSM token seqnum to reduce overflow speed of zonde dataVersion.

[ https://issues.apache.org/jira/browse/HADOOP-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766312#comment-17766312 ] ASF GitHub Bot commented on HADOOP-18906: - hadoop-yetus commented on PR #6097:

[GitHub] [hadoop] hadoop-yetus commented on pull request #6097: HADOOP-18906. Increase default batch size of ZKDTSM token seqnum to reduce overflow speed of zonde dataVersion.

hadoop-yetus commented on PR #6097: URL: https://github.com/apache/hadoop/pull/6097#issuecomment-1723114748 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6053: YARN-11567 - Aggregate container launch debug artifacts on error

hadoop-yetus commented on PR #6053: URL: https://github.com/apache/hadoop/pull/6053#issuecomment-1723111549 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

saxenapranav commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1328475380 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: ## @@ -435,6 +456,24 @@ public void startCopyFromBlob(CloudBlobWrap

[GitHub] [hadoop] KeeProMise commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

KeeProMise commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1723086155 @goiri Do you have time to help review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [hadoop] zhuxiangyi commented on pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on PR #2964: URL: https://github.com/apache/hadoop/pull/2964#issuecomment-1723082950 > Thanks @zhuxiangyi for your works. It is great idea and improvement. Almost LGTM. Leave some comments inline. Will give my +1 once correct. Thanks. @Hexiaoqiao Thank you ver

[GitHub] [hadoop] hadoop-yetus commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

hadoop-yetus commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1723080068 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328469427 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java: ## @@ -470,17 +475,53 @@ static RenameResult unprotectedRe

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328465474 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java: ## @@ -1468,6 +1475,30 @@ static Collection normalizePaths(

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328464878 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java: ## @@ -1468,6 +1475,30 @@ static Collection normalizePaths(

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328454449 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java: ## @@ -470,17 +475,53 @@ static RenameResult unprotectedRe

[GitHub] [hadoop] zhuxiangyi commented on a diff in pull request #2964: HDFS-16000. Rename performance optimization

zhuxiangyi commented on code in PR #2964: URL: https://github.com/apache/hadoop/pull/2964#discussion_r1328445332 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java: ## @@ -470,17 +475,53 @@ static RenameResult unprotectedRe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

hadoop-yetus commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1723023243 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] zhangshuyan0 commented on a diff in pull request #6095: HDFS-17197. Show file replication when listing corrupt files.

zhangshuyan0 commented on code in PR #6095: URL: https://github.com/apache/hadoop/pull/6095#discussion_r1328403016 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -6195,7 +6197,11 @@ Collection listCorruptFileBloc

[GitHub] [hadoop] hadoop-yetus commented on pull request #6095: HDFS-17197. Show file replication when listing corrupt files.

hadoop-yetus commented on PR #6095: URL: https://github.com/apache/hadoop/pull/6095#issuecomment-1722953186 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6085: YARN-9586. Need more doc for yarn.federation.policy-manager-params when LoadBasedRouterPolicy is used.

hadoop-yetus commented on PR #6085: URL: https://github.com/apache/hadoop/pull/6085#issuecomment-1722949086 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] KeeProMise closed pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

KeeProMise closed pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified URL: https://github.com/apache/hadoop/pull/6096 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use