Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1993626962 Addendum PR: #6624 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825910#comment-17825910 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani opened a new pull request, #6624: URL:

[PR] HADOOP-19066. Run FIPS test for valid bucket locations (ADDENDUM) [hadoop]

2024-03-12 Thread via GitHub
virajjasani opened a new pull request, #6624: URL: https://github.com/apache/hadoop/pull/6624 Jira: HADOOP-19066 Addendum to PR: #6539 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [ABFS] [Backport 3.4] Back Merging PRs from trunk to Branch 3.4 [hadoop]

2024-03-12 Thread via GitHub
anujmodi2021 commented on PR #6611: URL: https://github.com/apache/hadoop/pull/6611#issuecomment-1993585491 > I think it will depend on the fact first commit works as it its and is not dependent on second one. We need to merge these separately such that if we have to revert in future we

Re: [PR] YARN-11660. Fix huge performance regression for SingleConstraintAppPlacementAllocator [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6623: URL: https://github.com/apache/hadoop/pull/6623#issuecomment-1993566244 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825904#comment-17825904 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1993556163 Oh wait, FIPS is only for US and Canada endpoints. The above error is legit. Let me provide an addendum to ignore the test if non-US or Canada endpoints are used. -- This is

Re: [PR] [ABFS] [Backport 3.4] Back Merging PRs from trunk to Branch 3.4 [hadoop]

2024-03-12 Thread via GitHub
mukund-thakur commented on PR #6611: URL: https://github.com/apache/hadoop/pull/6611#issuecomment-1993499010 Cp'ed the first one, compiled and ran azure tests. It went fine. Will do the same for 2nd one and combine the checkstyle in 2nd one. -- This is an automated message from the

Re: [PR] HDFS-17380. FsImageValidation: remove inaccessible nodes. [hadoop]

2024-03-12 Thread via GitHub
szetszwo commented on PR #6549: URL: https://github.com/apache/hadoop/pull/6549#issuecomment-1993371108 > When you talk about inaccessible inode, do you mean NameNode unexpected logic cause some inodes are unreachable? Yes. The inaccessible inodes may be caused by a bug like

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-12 Thread via GitHub
saxenapranav commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1522432071 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -208,6 +240,8 @@ public AbfsClient(final URL baseUrl,

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-12 Thread via GitHub
saxenapranav commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1510679721 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1411,6 +1448,97 @@ protected AccessTokenProvider

Re: [PR] HDFS-17352. Add configuration to control whether DN delete this replica from disk when client requests a missing block [hadoop]

2024-03-12 Thread via GitHub
haiyang1987 commented on PR #6559: URL: https://github.com/apache/hadoop/pull/6559#issuecomment-1993179784 > Thanks involve me here. I think @zhangshuyan0 should be more professional about this improvement. Let's wait her/his feedback. ok, thanks for your comment. -- This is an

Re: [PR] HDFS-17352. Add configuration to control whether DN delete this replica from disk when client requests a missing block [hadoop]

2024-03-12 Thread via GitHub
Hexiaoqiao commented on PR #6559: URL: https://github.com/apache/hadoop/pull/6559#issuecomment-1993154964 Thanks involve me here. I think @zhangshuyan0 should be more professional about this improvement. Let's wait her/his feedback. -- This is an automated message from the Apache Git

Re: [PR] YARN-11660. Fix huge performance regression for SingleConstraintAppPlacementAllocator [hadoop]

2024-03-12 Thread via GitHub
zuston commented on code in PR #6623: URL: https://github.com/apache/hadoop/pull/6623#discussion_r1522391003 ##

Re: [PR] [ABFS] [Backport 3.4] Back Merging PRs from trunk to Branch 3.4 [hadoop]

2024-03-12 Thread via GitHub
mukund-thakur commented on PR #6611: URL: https://github.com/apache/hadoop/pull/6611#issuecomment-1992698068 I think it will depend on the fact first commit works as it its and is not dependent on second one. We need to merge these separately such that if we have to revert in future we can

[jira] [Commented] (HADOOP-19106) [ABFS] All tests of. ITestAzureBlobFileSystemAuthorization fails with NPE

2024-03-12 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825849#comment-17825849 ] Mukund Thakur commented on HADOOP-19106: It fails because

[jira] [Commented] (HADOOP-19106) [ABFS] All tests of. ITestAzureBlobFileSystemAuthorization fails with NPE

2024-03-12 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825838#comment-17825838 ] Mukund Thakur commented on HADOOP-19106: It does fail for me with the same config mentioned in  

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825832#comment-17825832 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992594998 Issue seems with FIPS cases. FIPS enabled and 1. bucket created on oregon, s3 client configured with `us-east-2` region with cross-region access enabled and no endpoint

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825824#comment-17825824 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992532765 Just created a bucket in london and now i can reproduce the failure, checking. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825822#comment-17825822 ] ASF GitHub Bot commented on HADOOP-19050: - steveloughran commented on code in PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on code in PR #6544: URL: https://github.com/apache/hadoop/pull/6544#discussion_r1522069895 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java: ## @@ -401,4 +406,22 @@ private static Region

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825820#comment-17825820 ] ASF GitHub Bot commented on HADOOP-19050: - steveloughran commented on PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on PR #6544: URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1992515204 You're right there is no rename; copy is all there is. So that is not available yet? Hmmm. This isn't ready for production yet is it? Let us keep it in trunk for now. The other

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825819#comment-17825819 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992509237 Something seems odd. This test overrides endpoint/region configs so setting any endpoint/region should have made no difference: ``` @Test public void

[jira] [Assigned] (HADOOP-19088) upgrade to jersey-json 1.22.0

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-19088: --- Assignee: PJ Fanning > upgrade to jersey-json 1.22.0 >

[jira] [Updated] (HADOOP-19088) upgrade to jersey-json 1.22.0

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19088: Fix Version/s: 3.5.0 > upgrade to jersey-json 1.22.0 > - > >

[jira] [Commented] (HADOOP-19088) upgrade to jersey-json 1.22.0

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825814#comment-17825814 ] ASF GitHub Bot commented on HADOOP-19088: - steveloughran commented on PR #6585: URL:

Re: [PR] HADOOP-19088. Use jersey-json 1.22.0 [hadoop]

2024-03-12 Thread via GitHub
steveloughran merged PR #6585: URL: https://github.com/apache/hadoop/pull/6585 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HADOOP-19088. Use jersey-json 1.22.0 [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on PR #6585: URL: https://github.com/apache/hadoop/pull/6585#issuecomment-1992505236 in trunk, backport to 3.4 needed too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HADOOP-19088) upgrade to jersey-json 1.22.0

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825812#comment-17825812 ] ASF GitHub Bot commented on HADOOP-19088: - steveloughran merged PR #6585: URL:

[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825807#comment-17825807 ] ASF GitHub Bot commented on HADOOP-18950: - pjfanning commented on PR #4854: URL:

Re: [PR] HADOOP-18950: shaded avro jar [hadoop]

2024-03-12 Thread via GitHub
pjfanning commented on PR #4854: URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1992503197 I don't know enough about Avro to hack it to work with shaded and non-shaded annotations. I thought all we cared about was how to support the internal Hadoop code and its

Re: [PR] [ABFS] [Backport 3.4] Back Merging PRs from trunk to Branch 3.4 [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on PR #6611: URL: https://github.com/apache/hadoop/pull/6611#issuecomment-1992502048 I think what I will do here is check this branch out then cherrypick each PR in order, so that they are still two separate PRs. Do you see any problem with this? -- This is an

[jira] [Commented] (HADOOP-19102) [ABFS]: FooterReadBufferSize should not be greater than readBufferSize

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825790#comment-17825790 ] Steve Loughran commented on HADOOP-19102: - what is the exception when its wrong and is there a

[jira] [Commented] (HADOOP-19102) [ABFS]: FooterReadBufferSize should not be greater than readBufferSize

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825789#comment-17825789 ] ASF GitHub Bot commented on HADOOP-19102: - steveloughran commented on code in PR #6617: URL:

Re: [PR] HADOOP-19102. FooterReadBufferSize should not be greater than readBufferSize [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on code in PR #6617: URL: https://github.com/apache/hadoop/pull/6617#discussion_r1522047711 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsInputStreamReadFooter.java: ## @@ -54,9 +63,44 @@ public class

[jira] [Commented] (HADOOP-19108) S3 Express: document use

2024-03-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825788#comment-17825788 ] Dongjoon Hyun commented on HADOOP-19108: Is this targeting Apache Hadoop 3.4.1? > S3 Express:

[jira] [Updated] (HADOOP-19102) [ABFS]: FooterReadBufferSize should not be greater than readBufferSize

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19102: Affects Version/s: 3.4.0 > [ABFS]: FooterReadBufferSize should not be greater than

[jira] [Updated] (HADOOP-19102) [ABFS]: FooterReadBufferSize should not be greater than readBufferSize

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19102: Fix Version/s: (was: 3.4.0) (was: 3.5.0) > [ABFS]:

[jira] [Commented] (HADOOP-19102) [ABFS]: FooterReadBufferSize should not be greater than readBufferSize

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825785#comment-17825785 ] Steve Loughran commented on HADOOP-19102: - [~pranavsaxena] can I remind you to set the bus you

Re: [PR] HADOOP-18950: shaded avro jar [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on code in PR #4854: URL: https://github.com/apache/hadoop/pull/4854#discussion_r1522032834 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Path.java: ## @@ -27,7 +27,7 @@ import java.util.Optional; import

[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825784#comment-17825784 ] ASF GitHub Bot commented on HADOOP-18950: - steveloughran commented on code in PR #4854: URL:

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825781#comment-17825781 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992431076 rebasing both trunk and branch-3.4 before re-running the tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Commented] (HADOOP-18950) upgrade avro to 1.11.3 due to CVE

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825778#comment-17825778 ] ASF GitHub Bot commented on HADOOP-18950: - steveloughran commented on PR #4854: URL:

Re: [PR] HADOOP-18950: shaded avro jar [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on PR #4854: URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1992402992 let's do it in 3.4.1 after a 1.3.0 release, and make the "we've tuned the packaging" a key change along with "we've fixed the bits steve broke". -- This is an automated message

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825777#comment-17825777 ] ASF GitHub Bot commented on HADOOP-19066: - steveloughran commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992389645 looking at my current settings I've set endpoint to london but the region is unset; making sure that the classic binding mechanism still works. {code}

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825774#comment-17825774 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992374906 I will re-run the test suite and followup. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825773#comment-17825773 ] ASF GitHub Bot commented on HADOOP-19066: - steveloughran commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992373059 not good on branch-3.4; we need a followup i'm afraid. leaving in trunk rather than reverting for now as the other tests all seem happy. -- This is an automated message from the

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825771#comment-17825771 ] Steve Loughran commented on HADOOP-19066: - afraid things break for me with a test bucket set up

[jira] [Reopened] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-19066: - > AWS SDK V2 - Enabling FIPS should be allowed with central endpoint >

[jira] [Commented] (HADOOP-18910) ABFS: Adding Support for MD5 Hash based integrity verification of the request content during transport

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825768#comment-17825768 ] ASF GitHub Bot commented on HADOOP-18910: - steveloughran commented on PR #6069: URL:

Re: [PR] HADOOP-18910: [ABFS] Adding Support for MD5 Hash based integrity verification of the request content during transport [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on PR #6069: URL: https://github.com/apache/hadoop/pull/6069#issuecomment-1992362377 think its time for the default for namespace.enabled to become true? I'd support that -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825765#comment-17825765 ] ASF GitHub Bot commented on HADOOP-19066: - steveloughran commented on PR #6539: URL:

[jira] [Updated] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19066: Fix Version/s: 3.5.0 Resolution: Fixed Status: Resolved (was: Patch

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
steveloughran commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992329398 (testing cherrypick; if all is good will merge to 3.4.x) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825764#comment-17825764 ] ASF GitHub Bot commented on HADOOP-19066: - steveloughran merged PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-03-12 Thread via GitHub
steveloughran merged PR #6539: URL: https://github.com/apache/hadoop/pull/6539 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Created] (HADOOP-19108) S3 Express: document use

2024-03-12 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19108: --- Summary: S3 Express: document use Key: HADOOP-19108 URL: https://issues.apache.org/jira/browse/HADOOP-19108 Project: Hadoop Common Issue Type:

[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18996: Description: HADOOP-18995 upgrades the SDK version which allows connecting to a s3

Re: [PR] HDFS-17383:Datanode current block token should come from active NameNode in HA mode [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6562: URL: https://github.com/apache/hadoop/pull/6562#issuecomment-1992228065 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-1991971058 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-1991866091 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] MAPREDUCE-7402. fix mapreduce.task.io.sort.factor=1 lead to an infinite loop. [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6622: URL: https://github.com/apache/hadoop/pull/6622#issuecomment-1991831364 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11660. Fix huge performance regression for SingleConstraintAppPlacementAllocator [hadoop]

2024-03-12 Thread via GitHub
slfan1989 commented on code in PR #6623: URL: https://github.com/apache/hadoop/pull/6623#discussion_r1521601149 ##

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1521449729 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1411,6 +1448,97 @@ protected AccessTokenProvider

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1521424714 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1655,7 +1694,7 @@ void setIsNamespaceEnabled(final

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1521405518 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/AzureServiceErrorCode.java: ## @@ -99,24 +104,22 @@ public static

[jira] [Commented] (HADOOP-18854) add options to disable range merging of vectored io

2024-03-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825659#comment-17825659 ] Steve Loughran commented on HADOOP-18854: - thanks. will close as done. > add options to

Re: [PR] MAPREDUCE-7402. fix mapreduce.task.io.sort.factor=1 lead to an infinite loop. [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6622: URL: https://github.com/apache/hadoop/pull/6622#issuecomment-1991492005 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-12 Thread via GitHub
anujmodi2021 commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1521326382 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java: ## @@ -160,6 +162,7 @@ public class AzureBlobFileSystem extends

Re: [PR] YARN-11660. Fix huge performance regression for SingleConstraintAppPlacementAllocator [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6623: URL: https://github.com/apache/hadoop/pull/6623#issuecomment-1991401418 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17352. Add configuration to control whether DN delete this replica from disk when client requests a missing block [hadoop]

2024-03-12 Thread via GitHub
haiyang1987 commented on PR #6559: URL: https://github.com/apache/hadoop/pull/6559#issuecomment-1991339659 Hi @Hexiaoqiao Sir would you mind look it again, thanks~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] HDFS-17391. Adjust the checkpoint io buffer size to the chunk size. [hadoop]

2024-03-12 Thread via GitHub
Hexiaoqiao commented on PR #6594: URL: https://github.com/apache/hadoop/pull/6594#issuecomment-1991324410 Committed to trunk. Thanks @ThinkerLei . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] HDFS-17391. Adjust the checkpoint io buffer size to the chunk size. [hadoop]

2024-03-12 Thread via GitHub
Hexiaoqiao merged PR #6594: URL: https://github.com/apache/hadoop/pull/6594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MAPREDUCE-7402. fix mapreduce.task.io.sort.factor=1 lead to an infinite loop. [hadoop]

2024-03-12 Thread via GitHub
KeeProMise commented on code in PR #6622: URL: https://github.com/apache/hadoop/pull/6622#discussion_r1521225842 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMerger.java: ## @@ -229,6

Re: [PR] HDFS-17422. Enhance the stability of the unit test TestDFSAdmin. [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6621: URL: https://github.com/apache/hadoop/pull/6621#issuecomment-1991182311 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17391:Adjust the checkpoint io buffer size to the chunk size [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6594: URL: https://github.com/apache/hadoop/pull/6594#issuecomment-1991164298 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19107) Drop support for HBase v1

2024-03-12 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825580#comment-17825580 ] Ayush Saxena commented on HADOOP-19107: --- Currently v2 is broken for me {noformat} [INFO] ---

[jira] [Created] (HADOOP-19107) Drop support for HBase v1

2024-03-12 Thread Ayush Saxena (Jira)
Ayush Saxena created HADOOP-19107: - Summary: Drop support for HBase v1 Key: HADOOP-19107 URL: https://issues.apache.org/jira/browse/HADOOP-19107 Project: Hadoop Common Issue Type: Task

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825568#comment-17825568 ] ASF GitHub Bot commented on HADOOP-18516: - hadoop-yetus commented on PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-03-12 Thread via GitHub
hadoop-yetus commented on PR #6552: URL: https://github.com/apache/hadoop/pull/6552#issuecomment-1991044330 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] YARN-11660. Fix huge performance regression for SingleConstraintAppPlacementAllocator [hadoop]

2024-03-12 Thread via GitHub
zuston opened a new pull request, #6623: URL: https://github.com/apache/hadoop/pull/6623 ### Description of PR When using the `SingleConstraintAppPlacementAllocator` with scheduling request in our internal cluster, I found the huge performance regression from the metric of

Re: [PR] MAPREDUCE-7402. fix mapreduce.task.io.sort.factor=1 lead to an infinite loop. [hadoop]

2024-03-12 Thread via GitHub
ayushtkn commented on code in PR #6622: URL: https://github.com/apache/hadoop/pull/6622#discussion_r1521008831 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMerger.java: ## @@ -229,6

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825549#comment-17825549 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-03-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1503721707 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemChooseSAS.java: ## @@ -128,11 +130,18 @@ public void

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825548#comment-17825548 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-03-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1503719618 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemChooseSAS.java: ## @@ -97,16 +99,16 @@ public void

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825547#comment-17825547 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-03-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1520976609 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/FixedSASTokenProvider.java: ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825543#comment-17825543 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-03-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1520974197 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java: ## @@ -1308,10 +1308,9 @@ public void access(final Path path,

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-03-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825542#comment-17825542 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-03-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1520972493 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java: ## @@ -976,33 +977,60 @@ public AccessTokenProvider

  1   2   >