Re: [PR] HDFS-17529. RBF: Improve router state store cache entry deletion [hadoop]

2024-05-23 Thread via GitHub
ZanderXu commented on PR #6833: URL: https://github.com/apache/hadoop/pull/6833#issuecomment-2128331182 Merged. Thanks @kokonguyen191 for your contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] HDFS-17529. RBF: Improve router state store cache entry deletion [hadoop]

2024-05-23 Thread via GitHub
ZanderXu merged PR #6833: URL: https://github.com/apache/hadoop/pull/6833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-19184) TestStagingCommitter.testJobCommitFailure failing

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849127#comment-17849127 ] ASF GitHub Bot commented on HADOOP-19184: - hadoop-yetus commented on PR #6843: URL:

Re: [PR] HADOOP-19184. Fix UT TestStagingCommitter.testJobCommitFailure [hadoop]

2024-05-23 Thread via GitHub
hadoop-yetus commented on PR #6843: URL: https://github.com/apache/hadoop/pull/6843#issuecomment-2128218733 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19184) TestStagingCommitter.testJobCommitFailure failing

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849122#comment-17849122 ] ASF GitHub Bot commented on HADOOP-19184: - mukund-thakur commented on PR #6843: URL:

Re: [PR] HADOOP-19184. Fix UT TestStagingCommitter.testJobCommitFailure [hadoop]

2024-05-23 Thread via GitHub
mukund-thakur commented on PR #6843: URL: https://github.com/apache/hadoop/pull/6843#issuecomment-2128081264 Tested using us-west-1 bucket. All good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] HDFS-17528. FsImageValidation: set txid when saving a new image [hadoop]

2024-05-23 Thread via GitHub
szetszwo commented on PR #6828: URL: https://github.com/apache/hadoop/pull/6828#issuecomment-2127936231 The jenkins builds keep getting stuck for a day and then fail. Not sure if it is a known problem? -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-18610) ABFS OAuth2 Token Provider to support Azure Workload Identity for AKS

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849082#comment-17849082 ] ASF GitHub Bot commented on HADOOP-18610: - rakeshadr commented on code in PR #6787: URL:

Re: [PR] HADOOP-18610: [ABFS] OAuth2 Token Provider support for Azure Workload Identity [hadoop]

2024-05-23 Thread via GitHub
rakeshadr commented on code in PR #6787: URL: https://github.com/apache/hadoop/pull/6787#discussion_r1612126232 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADAuthenticator.java: ## @@ -103,14 +105,55 @@ public static AzureADToken

[jira] [Commented] (HADOOP-19161) S3A: option "fs.s3a.performance.flags" to take list of performance flags

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849067#comment-17849067 ] ASF GitHub Bot commented on HADOOP-19161: - hadoop-yetus commented on PR #6789: URL:

Re: [PR] HADOOP-19161. S3A: option "fs.s3a.performance.flags" to take list of performance flags [hadoop]

2024-05-23 Thread via GitHub
hadoop-yetus commented on PR #6789: URL: https://github.com/apache/hadoop/pull/6789#issuecomment-2127773836 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19137) [ABFS]:Extra getAcl call while calling the very first API of FileSystem

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849047#comment-17849047 ] ASF GitHub Bot commented on HADOOP-19137: - mukund-thakur commented on code in PR #6752: URL:

Re: [PR] HADOOP-19137. [ABFS]Fail fs init if cpk config is given on non-hns account [hadoop]

2024-05-23 Thread via GitHub
mukund-thakur commented on code in PR #6752: URL: https://github.com/apache/hadoop/pull/6752#discussion_r1609025458 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java: ## @@ -223,6 +225,21 @@ public void initialize(URI uri,

[jira] [Commented] (HADOOP-18508) support multiple s3a integration test runs on same bucket in parallel

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849030#comment-17849030 ] ASF GitHub Bot commented on HADOOP-18508: - mukund-thakur commented on PR #5081: URL:

Re: [PR] HADOOP-18508. S3A: Support parallel integration test runs on same bucket [hadoop]

2024-05-23 Thread via GitHub
mukund-thakur commented on PR #5081: URL: https://github.com/apache/hadoop/pull/5081#issuecomment-2127583254 > @mukund-thakur given the failures you saw were just related to you not reading the docs, can we merge this? Haha sure apart from this

Re: [PR] HADOOP-19168. Upgrade kafka-client to 3.4.0 due to CVEs [hadoop]

2024-05-23 Thread via GitHub
rohit-kb commented on PR #6808: URL: https://github.com/apache/hadoop/pull/6808#issuecomment-2127428486 Sure Steve, will look into it soon. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] HADOOP-19168. Upgrade kafka-client to 3.4.0 due to CVEs [hadoop]

2024-05-23 Thread via GitHub
rohit-kb closed pull request #6808: HADOOP-19168. Upgrade kafka-client to 3.4.0 due to CVEs URL: https://github.com/apache/hadoop/pull/6808 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (HADOOP-19168) Upgrade Kafka Clients due to CVEs

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849011#comment-17849011 ] ASF GitHub Bot commented on HADOOP-19168: - rohit-kb commented on PR #6808: URL:

[jira] [Commented] (HADOOP-19168) Upgrade Kafka Clients due to CVEs

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849010#comment-17849010 ] ASF GitHub Bot commented on HADOOP-19168: - rohit-kb closed pull request #6808: HADOOP-19168.

[jira] [Commented] (HADOOP-18508) support multiple s3a integration test runs on same bucket in parallel

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849001#comment-17849001 ] ASF GitHub Bot commented on HADOOP-18508: - steveloughran commented on PR #5081: URL:

Re: [PR] HADOOP-18508. S3A: Support parallel integration test runs on same bucket [hadoop]

2024-05-23 Thread via GitHub
steveloughran commented on PR #5081: URL: https://github.com/apache/hadoop/pull/5081#issuecomment-2127404245 @mukund-thakur given the failures you saw were just related to you not reading the docs, can we merge this? -- This is an automated message from the Apache Git Service. To respond

Re: [PR] HDFS-17518: Sync the editslog if a file is closed in the lease monitor [hadoop]

2024-05-23 Thread via GitHub
ThinkerLei commented on code in PR #6809: URL: https://github.com/apache/hadoop/pull/6809#discussion_r1611850764 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java: ## @@ -626,7 +626,8 @@ private synchronized boolean

[jira] [Commented] (HADOOP-19182) Upgrade kafka to 3.4.0

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848987#comment-17848987 ] ASF GitHub Bot commented on HADOOP-19182: - steveloughran commented on PR #6842: URL:

[jira] [Commented] (HADOOP-18962) Upgrade kafka to 3.4.0

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848986#comment-17848986 ] ASF GitHub Bot commented on HADOOP-18962: - steveloughran commented on code in PR #6247: URL:

Re: [PR] HADOOP-19182. Upgrade kafka to 3.4.0 [hadoop]

2024-05-23 Thread via GitHub
steveloughran commented on PR #6842: URL: https://github.com/apache/hadoop/pull/6842#issuecomment-2127314515 commented on #6247 about this...lets get that pr in but resolve all updated transient dependencies: your PR highlights more than the earlier one -- This is an automated message

Re: [PR] HADOOP-18962. Upgrade kafka to 3.4.0 [hadoop]

2024-05-23 Thread via GitHub
steveloughran commented on code in PR #6247: URL: https://github.com/apache/hadoop/pull/6247#discussion_r1611824352 ## LICENSE-binary: ## @@ -376,7 +376,7 @@ hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/com

[jira] [Resolved] (HADOOP-19168) Upgrade Kafka Clients due to CVEs

2024-05-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19168. - Resolution: Duplicate rohit, dupe of HADOOP-18962. let's focus on that > Upgrade

[jira] [Updated] (HADOOP-19182) Upgrade kafka to 3.4.0

2024-05-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19182: Affects Version/s: 3.4.0 > Upgrade kafka to 3.4.0 > -- > >

[jira] [Updated] (HADOOP-19182) Upgrade kafka to 3.4.0

2024-05-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19182: Issue Type: Improvement (was: Bug) > Upgrade kafka to 3.4.0 > -- >

[jira] [Commented] (HADOOP-19137) [ABFS]:Extra getAcl call while calling the very first API of FileSystem

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848977#comment-17848977 ] ASF GitHub Bot commented on HADOOP-19137: - steveloughran commented on code in PR #6752: URL:

Re: [PR] HADOOP-19137. [ABFS]Fail fs init if cpk config is given on non-hns account [hadoop]

2024-05-23 Thread via GitHub
steveloughran commented on code in PR #6752: URL: https://github.com/apache/hadoop/pull/6752#discussion_r1611784761 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsCustomEncryption.java: ## @@ -390,9 +386,34 @@ private AzureBlobFileSystem

[jira] [Created] (HADOOP-19185) Improve ABFS metric integration with iOStatistics

2024-05-23 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19185: --- Summary: Improve ABFS metric integration with iOStatistics Key: HADOOP-19185 URL: https://issues.apache.org/jira/browse/HADOOP-19185 Project: Hadoop Common

[jira] [Commented] (HADOOP-18325) ABFS: Add correlated metric support for ABFS operations

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848972#comment-17848972 ] ASF GitHub Bot commented on HADOOP-18325: - steveloughran commented on PR #6314: URL:

[jira] [Resolved] (HADOOP-18325) ABFS: Add correlated metric support for ABFS operations

2024-05-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18325. - Fix Version/s: 3.5.0 Resolution: Fixed > ABFS: Add correlated metric support

[jira] [Commented] (HADOOP-18325) ABFS: Add correlated metric support for ABFS operations

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848970#comment-17848970 ] ASF GitHub Bot commented on HADOOP-18325: - steveloughran merged PR #6314: URL:

Re: [PR] HADOOP-18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-05-23 Thread via GitHub
steveloughran commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-2127227834 fixed in 3.5, submit a PR for this against 3.4. regarding my outstanding comments, i will create a followup -- This is an automated message from the Apache Git Service. To

Re: [PR] HADOOP-18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-05-23 Thread via GitHub
steveloughran merged PR #6314: URL: https://github.com/apache/hadoop/pull/6314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged delete of files and objects

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848968#comment-17848968 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran commented on PR #6726: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged delete of files [hadoop]

2024-05-23 Thread via GitHub
steveloughran commented on PR #6726: URL: https://github.com/apache/hadoop/pull/6726#issuecomment-2127219751 I tried to do it yesterday too. I think my solution would have pulled out createStore() into a method and overrode it, or added S3AInternals.setStore() call. -- This is an

Re: [PR] HDFS-17529. RBF: Improve router state store cache entry deletion [hadoop]

2024-05-23 Thread via GitHub
hadoop-yetus commented on PR #6833: URL: https://github.com/apache/hadoop/pull/6833#issuecomment-2126931785 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19184) TestStagingCommitter.testJobCommitFailure failing

2024-05-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848905#comment-17848905 ] ASF GitHub Bot commented on HADOOP-19184: - steveloughran commented on PR #6843: URL:

Re: [PR] HADOOP-19184. Fix UT TestStagingCommitter.testJobCommitFailure [hadoop]

2024-05-23 Thread via GitHub
steveloughran commented on PR #6843: URL: https://github.com/apache/hadoop/pull/6843#issuecomment-2126839751 I can only trigger the failure on trunk in a parallel test run! this explains why yetus missed it. except: I can also trigger it from the IDE, which isn't parallel. I

Re: [PR] HDFS-17529. RBF: Improve router state store cache entry deletion [hadoop]

2024-05-23 Thread via GitHub
ZanderXu commented on code in PR #6833: URL: https://github.com/apache/hadoop/pull/6833#discussion_r1611258872 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/CachedRecordStore.java: ## @@ -198,8 +195,15 @@ public void