[jira] [Commented] (HADOOP-19189) ITestS3ACommitterFactory failing

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853042#comment-17853042 ] ASF GitHub Bot commented on HADOOP-19189: - virajjasani commented on code in PR

Re: [PR] HADOOP-19189. ITestS3ACommitterFactory failing [hadoop]

2024-06-06 Thread via GitHub
virajjasani commented on code in PR #6857: URL: https://github.com/apache/hadoop/pull/6857#discussion_r1630727716 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/ITestS3ACommitterFactory.java: ## @@ -72,121 +85,156 @@ public class ITestS3ACommitterFactor

[jira] [Commented] (HADOOP-19197) S3A: Support AWS KMS Encryption Context

2024-06-06 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853016#comment-17853016 ] Viraj Jasani commented on HADOOP-19197: --- We need to use it at 3 places: CopyObjec

Re: [PR] HADOOP-19116. Update to zookeeper client 3.8.4 due to CVE-2024-23944 [hadoop]

2024-06-06 Thread via GitHub
virajjasani commented on PR #6675: URL: https://github.com/apache/hadoop/pull/6675#issuecomment-2153958403 Also, imp to note that, the last build has not run a single yarn or mapreduce tests https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6675/4/testReport/ Any patch that upda

[jira] [Commented] (HADOOP-18610) ABFS OAuth2 Token Provider to support Azure Workload Identity for AKS

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853012#comment-17853012 ] ASF GitHub Bot commented on HADOOP-18610: - anujmodi2021 commented on code in PR

Re: [PR] HADOOP-18610: [ABFS] OAuth2 Token Provider support for Azure Workload Identity [hadoop]

2024-06-06 Thread via GitHub
anujmodi2021 commented on code in PR #6787: URL: https://github.com/apache/hadoop/pull/6787#discussion_r1630628608 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/WorkloadIdentityTokenProvider.java: ## @@ -63,58 +64,35 @@ protected AzureADToken re

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853010#comment-17853010 ] ASF GitHub Bot commented on HADOOP-18516: - anujmodi2021 commented on PR #6855:

Re: [PR] HADOOP-18516: [Backport to 3.4] [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-06-06 Thread via GitHub
anujmodi2021 commented on PR #6855: URL: https://github.com/apache/hadoop/pull/6855#issuecomment-2153933059 @steveloughran @mukund-thakur This should be good to merge. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Commented] (HADOOP-19178) WASB Driver Deprecation and eventual removal

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853009#comment-17853009 ] ASF GitHub Bot commented on HADOOP-19178: - anujmodi2021 commented on PR #6862:

Re: [PR] HADOOP-19178: [WASB Deprecation] Updating Documentation on Upcoming Plans for Hadoop-Azure [hadoop]

2024-06-06 Thread via GitHub
anujmodi2021 commented on PR #6862: URL: https://github.com/apache/hadoop/pull/6862#issuecomment-2153929132 Hi @steveloughran Please review this WASB Deprecation Announcement PR and let me know if we need to do anything else. Do we need to mark AzureNativeFileSystem as @deprecated

[jira] [Commented] (HADOOP-18610) ABFS OAuth2 Token Provider to support Azure Workload Identity for AKS

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853007#comment-17853007 ] ASF GitHub Bot commented on HADOOP-18610: - anujmodi2021 commented on code in PR

Re: [PR] HADOOP-18610: [ABFS] OAuth2 Token Provider support for Azure Workload Identity [hadoop]

2024-06-06 Thread via GitHub
anujmodi2021 commented on code in PR #6787: URL: https://github.com/apache/hadoop/pull/6787#discussion_r1630628608 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/WorkloadIdentityTokenProvider.java: ## @@ -63,58 +64,35 @@ protected AzureADToken re

Re: [PR] HDFS-17543. [ARR]: AsyncUtil makes asynchronous code more concise and easier. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise commented on PR #6868: URL: https://github.com/apache/hadoop/pull/6868#issuecomment-2153787222 @goiri @simbadzina @Hexiaoqiao @sjlee If you have time, please help to review this PR, thank you! -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] HDFS-17531. [Discuss] RBF: Aynchronous router RPC. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise commented on PR #6838: URL: https://github.com/apache/hadoop/pull/6838#issuecomment-2153746299 Thank you again for your attention. I will split this huge PR into small PRs. You can review the PRs in the subtasks. I will close this huge PR. -- This is an automated message from t

Re: [PR] HDFS-17531. [Discuss] RBF: Aynchronous router RPC. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise closed pull request #6838: HDFS-17531. [Discuss] RBF: Aynchronous router RPC. URL: https://github.com/apache/hadoop/pull/6838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] HDFS-17545. [ARR] router async rpc client. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise commented on PR #6871: URL: https://github.com/apache/hadoop/pull/6871#issuecomment-2153743472 this PR needs to use Asyncutil of HDFS-17543 and async ProtocolPB of HDFS-17544, We should first review HDFS-17543 and HDFS-17544 and then review this PR. Before HDFS-17543、HDFS-17544 i

Re: [PR] HDFS-17544. [ARR] The router client rpc protocol supports asynchrony. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise commented on PR #6870: URL: https://github.com/apache/hadoop/pull/6870#issuecomment-2153741838 > @KeeProMise Hi, sir. The code in this PR contains [HDFS-17543](https://issues.apache.org/jira/browse/HDFS-17543)'s change. Can we seperate them for better reviewing. @hfutatzha

Re: [PR] HDFS-17543. [ARR]: AsyncUtil makes asynchronous code more concise and easier. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise commented on code in PR #6868: URL: https://github.com/apache/hadoop/pull/6868#discussion_r1630547741 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/async/AsyncForEachRun.java: ## @@ -0,0 +1,97 @@ +/** + * Licensed

[jira] [Commented] (HADOOP-19154) upgrade bouncy castle to 1.78.1 due to CVEs

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852993#comment-17852993 ] ASF GitHub Bot commented on HADOOP-19154: - hadoop-yetus commented on PR #6866:

Re: [PR] HADOOP-19154. Upgrade bouncycastle to 1.78.1 due to CVEs (#6755) [hadoop]

2024-06-06 Thread via GitHub
hadoop-yetus commented on PR #6866: URL: https://github.com/apache/hadoop/pull/6866#issuecomment-2153737457 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17544. [ARR] The router client rpc protocol supports asynchrony. [hadoop]

2024-06-06 Thread via GitHub
hfutatzhanghb commented on PR #6870: URL: https://github.com/apache/hadoop/pull/6870#issuecomment-2153737046 @KeeProMise Hi, sir. The code in this PR contains HDFS-17543's change. Can we seperate them for better reviewing. -- This is an automated message from the Apache Git Service.

[jira] [Commented] (HADOOP-19193) Create orphan commit for website deployment

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852990#comment-17852990 ] ASF GitHub Bot commented on HADOOP-19193: - pan3793 commented on PR #6864: URL:

Re: [PR] HADOOP-19193. Create orphan commit for website deployment [hadoop]

2024-06-06 Thread via GitHub
pan3793 commented on PR #6864: URL: https://github.com/apache/hadoop/pull/6864#issuecomment-2153736606 > This is the key cause of the really big downloads, isn't it? should be, although I haven't analyzed the git repo blob, just asserting based on experience. > I still think we

[jira] [Commented] (HADOOP-19196) Bulk delete api doesn't take the path to delete as the base path

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852983#comment-17852983 ] ASF GitHub Bot commented on HADOOP-19196: - hadoop-yetus commented on PR #6872:

Re: [PR] HADOOP-19196. Allow base path to be deleted as well using Bulk Delete. [hadoop]

2024-06-06 Thread via GitHub
hadoop-yetus commented on PR #6872: URL: https://github.com/apache/hadoop/pull/6872#issuecomment-2153726855 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HDFS-17543. [ARR]: AsyncUtil makes asynchronous code more concise and easier. [hadoop]

2024-06-06 Thread via GitHub
hfutatzhanghb commented on code in PR #6868: URL: https://github.com/apache/hadoop/pull/6868#discussion_r1630528668 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/async/AsyncForEachRun.java: ## @@ -0,0 +1,97 @@ +/** + * Licen

[jira] [Created] (HADOOP-19197) S3A: Support AWS KMS Encryption Context

2024-06-06 Thread Raphael Azzolini (Jira)
Raphael Azzolini created HADOOP-19197: - Summary: S3A: Support AWS KMS Encryption Context Key: HADOOP-19197 URL: https://issues.apache.org/jira/browse/HADOOP-19197 Project: Hadoop Common I

[PR] HDFS-17546. Implementing HostsFileReader timeout [hadoop]

2024-06-06 Thread via GitHub
NyteKnight opened a new pull request, #6873: URL: https://github.com/apache/hadoop/pull/6873 ### Description of PR In hadoop environments that rely on NAS to house dfs.hosts file, during FS hangs (for any reason) the refreshNodes call would infinitely hang (causing a blocked thread) unti

[jira] [Commented] (HADOOP-19189) ITestS3ACommitterFactory failing

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852958#comment-17852958 ] ASF GitHub Bot commented on HADOOP-19189: - hadoop-yetus commented on PR #6857:

Re: [PR] HADOOP-19189. ITestS3ACommitterFactory failing [hadoop]

2024-06-06 Thread via GitHub
hadoop-yetus commented on PR #6857: URL: https://github.com/apache/hadoop/pull/6857#issuecomment-2153493109 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-19196) Bulk delete api doesn't take the path to delete as the base path

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852956#comment-17852956 ] ASF GitHub Bot commented on HADOOP-19196: - mukund-thakur opened a new pull requ

[jira] [Updated] (HADOOP-19196) Bulk delete api doesn't take the path to delete as the base path

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19196: Labels: pull-request-available (was: ) > Bulk delete api doesn't take the path to delete

[PR] HADOOP-19196. Allow base path to be deleted as well using Bulk Delete. [hadoop]

2024-06-06 Thread via GitHub
mukund-thakur opened a new pull request, #6872: URL: https://github.com/apache/hadoop/pull/6872 Follow up on HADOOP-18679. ### Description of PR ### How was this patch tested? Re-ran all the implementation of AbstractContractBulkDeleteTest ### For code chan

[jira] [Updated] (HADOOP-18679) Add API for bulk/paged delete of files and objects

2024-06-06 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-18679: --- Description: iceberg and hbase could benefit from being able to give a list of individual

[jira] [Commented] (HADOOP-19196) Bulk delete api doesn't take the path to delete as the base path

2024-06-06 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852954#comment-17852954 ] Mukund Thakur commented on HADOOP-19196: good catch.  > Bulk delete api doesn'

[jira] [Commented] (HADOOP-19116) update to zookeeper client 3.8.4 due to CVE-2024-23944

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852934#comment-17852934 ] ASF GitHub Bot commented on HADOOP-19116: - pjfanning commented on PR #6675: URL

Re: [PR] HADOOP-19116. Update to zookeeper client 3.8.4 due to CVE-2024-23944 [hadoop]

2024-06-06 Thread via GitHub
pjfanning commented on PR #6675: URL: https://github.com/apache/hadoop/pull/6675#issuecomment-2153313881 A lot of these: ``` java.lang.OutOfMemoryError: unable to create new native thread ``` -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] HADOOP-19116. Update to zookeeper client 3.8.4 due to CVE-2024-23944 [hadoop]

2024-06-06 Thread via GitHub
steveloughran commented on PR #6675: URL: https://github.com/apache/hadoop/pull/6675#issuecomment-2153298967 Lot more failures this time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[jira] [Commented] (HADOOP-19193) Create orphan commit for website deployment

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852928#comment-17852928 ] ASF GitHub Bot commented on HADOOP-19193: - steveloughran commented on PR #6864:

Re: [PR] HADOOP-19193. Create orphan commit for website deployment [hadoop]

2024-06-06 Thread via GitHub
steveloughran commented on PR #6864: URL: https://github.com/apache/hadoop/pull/6864#issuecomment-2153282965 thanks. This is the key cause of the really big downloads, isn't it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (HADOOP-19193) Create orphan commit for website deployment

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852927#comment-17852927 ] ASF GitHub Bot commented on HADOOP-19193: - steveloughran commented on PR #6864:

Re: [PR] HADOOP-19193. Create orphan commit for website deployment [hadoop]

2024-06-06 Thread via GitHub
steveloughran commented on PR #6864: URL: https://github.com/apache/hadoop/pull/6864#issuecomment-2153283381 (I still think we should cull old branches, FWIW) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[jira] [Commented] (HADOOP-19093) Improve rate limiting through ABFS in Manifest Committer

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852924#comment-17852924 ] ASF GitHub Bot commented on HADOOP-19093: - steveloughran closed pull request #6

Re: [PR] HADOOP-19093. [ABFS] Improve rate limiting through ABFS in Manifest Committer [hadoop]

2024-06-06 Thread via GitHub
steveloughran closed pull request #6596: HADOOP-19093. [ABFS] Improve rate limiting through ABFS in Manifest Committer URL: https://github.com/apache/hadoop/pull/6596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[jira] [Commented] (HADOOP-19189) ITestS3ACommitterFactory failing

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852922#comment-17852922 ] ASF GitHub Bot commented on HADOOP-19189: - steveloughran commented on PR #6857:

Re: [PR] HADOOP-19189. ITestS3ACommitterFactory failing [hadoop]

2024-06-06 Thread via GitHub
steveloughran commented on PR #6857: URL: https://github.com/apache/hadoop/pull/6857#issuecomment-2153272189 updated version; tested s3 london -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (HADOOP-19189) ITestS3ACommitterFactory failing

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852921#comment-17852921 ] ASF GitHub Bot commented on HADOOP-19189: - steveloughran commented on code in P

Re: [PR] HADOOP-19189. ITestS3ACommitterFactory failing [hadoop]

2024-06-06 Thread via GitHub
steveloughran commented on code in PR #6857: URL: https://github.com/apache/hadoop/pull/6857#discussion_r1630121834 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/ITestS3ACommitterFactory.java: ## @@ -72,121 +85,141 @@ public class ITestS3ACommitterFact

Re: [PR] HADOOP-19194:Add test to find unshaded dependencies in the aws sdk [hadoop]

2024-06-06 Thread via GitHub
mukund-thakur commented on PR #6865: URL: https://github.com/apache/hadoop/pull/6865#issuecomment-2153104078 > There are currently 4362 unshaded classes as per the test, the number reduces to 13 when we migrate to 2.25.53 Have you added a comment in an older SDK issue or created a new

[jira] [Commented] (HADOOP-19194) Add test to find unshaded dependencies in the aws sdk

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852894#comment-17852894 ] ASF GitHub Bot commented on HADOOP-19194: - mukund-thakur commented on PR #6865:

[jira] [Comment Edited] (HADOOP-19196) Bulk delete api doesn't take the path to delete as the base path

2024-06-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852870#comment-17852870 ] Steve Loughran edited comment on HADOOP-19196 at 6/6/24 5:01 PM:

[jira] [Commented] (HADOOP-19196) Bulk delete api doesn't take the path to delete as the base path

2024-06-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852870#comment-17852870 ] Steve Loughran commented on HADOOP-19196: - we need a contract test for this. {c

[jira] [Assigned] (HADOOP-19196) Bulk delete api doesn't take the path to delete as the base path

2024-06-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-19196: --- Assignee: Mukund Thakur > Bulk delete api doesn't take the path to delete as the b

[jira] [Created] (HADOOP-19196) Bulk delete api doesn't take the path to delete as the base path

2024-06-06 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19196: --- Summary: Bulk delete api doesn't take the path to delete as the base path Key: HADOOP-19196 URL: https://issues.apache.org/jira/browse/HADOOP-19196 Project: Had

[jira] [Commented] (HADOOP-19194) Add test to find unshaded dependencies in the aws sdk

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852857#comment-17852857 ] ASF GitHub Bot commented on HADOOP-19194: - steveloughran commented on code in P

Re: [PR] HDFS-17544. [ARR] The router client rpc protocol supports asynchrony. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise closed pull request #6870: HDFS-17544. [ARR] The router client rpc protocol supports asynchrony. URL: https://github.com/apache/hadoop/pull/6870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] HADOOP-19194:Add test to find unshaded dependencies in the aws sdk [hadoop]

2024-06-06 Thread via GitHub
steveloughran commented on code in PR #6865: URL: https://github.com/apache/hadoop/pull/6865#discussion_r1629843102 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/sdk/TestAWSV2SDK.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] HDFS-17543. [ARR]: AsyncUtil makes asynchronous code more concise and easier. [hadoop]

2024-06-06 Thread via GitHub
hadoop-yetus commented on PR #6868: URL: https://github.com/apache/hadoop/pull/6868#issuecomment-2152937859 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[PR] HDFS-17545. [ARR] router async rpc client. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise opened a new pull request, #6871: URL: https://github.com/apache/hadoop/pull/6871 ### Description of PR please see: https://issues.apache.org/jira/browse/HDFS-17545 ### How was this patch tested? ### For code changes: - [ ] Does the title or this

[PR] Hdfs-17544. [ARR] The router client rpc protocol supports asynchrony. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise opened a new pull request, #6870: URL: https://github.com/apache/hadoop/pull/6870 ### Description of PR Please see : https://issues.apache.org/jira/browse/HDFS-17544 ### How was this patch tested? ### For code changes: - [ ] Does the title or th

Re: [PR] Hdfs-17544. [ARR] The router client rpc protocol supports asynchrony. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise closed pull request #6869: Hdfs-17544. [ARR] The router client rpc protocol supports asynchrony. URL: https://github.com/apache/hadoop/pull/6869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] Hdfs-17544. [ARR] The router client rpc protocol supports asynchrony. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise opened a new pull request, #6869: URL: https://github.com/apache/hadoop/pull/6869 ### Description of PR please see: https://issues.apache.org/jira/browse/HDFS-17544 ### How was this patch tested? ### For code changes: - [ ] Does the title or this

[PR] HDFS-17543. [ARR]: AsyncUtil makes asynchronous code more concise and easier. [hadoop]

2024-06-06 Thread via GitHub
KeeProMise opened a new pull request, #6868: URL: https://github.com/apache/hadoop/pull/6868 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADO

[jira] [Commented] (HADOOP-19194) Add test to find unshaded dependencies in the aws sdk

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852807#comment-17852807 ] ASF GitHub Bot commented on HADOOP-19194: - HarshitGupta11 commented on PR #6865

Re: [PR] HADOOP-19194:Add test to find unshaded dependencies in the aws sdk [hadoop]

2024-06-06 Thread via GitHub
HarshitGupta11 commented on PR #6865: URL: https://github.com/apache/hadoop/pull/6865#issuecomment-2152658341 @steveloughran @mukund-thakur can I get a review on this? There are currently 4362 unshaded classes as per the test, the number reduces to 13 when we migrate to 2.25.53 -- This i

[jira] [Created] (HADOOP-19195) Upgrade aws sdk v2 to 2.25.53

2024-06-06 Thread Harshit Gupta (Jira)
Harshit Gupta created HADOOP-19195: -- Summary: Upgrade aws sdk v2 to 2.25.53 Key: HADOOP-19195 URL: https://issues.apache.org/jira/browse/HADOOP-19195 Project: Hadoop Common Issue Type: Impro

[jira] [Commented] (HADOOP-19194) Add test to find unshaded dependencies in the aws sdk

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852777#comment-17852777 ] ASF GitHub Bot commented on HADOOP-19194: - hadoop-yetus commented on PR #6865:

Re: [PR] HADOOP-19194:Add test to find unshaded dependencies in the aws sdk [hadoop]

2024-06-06 Thread via GitHub
hadoop-yetus commented on PR #6865: URL: https://github.com/apache/hadoop/pull/6865#issuecomment-2152382730 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-19114) upgrade to commons-compress 1.26.1 due to cves

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852766#comment-17852766 ] ASF GitHub Bot commented on HADOOP-19114: - pjfanning opened a new pull request,

[PR] HADOOP-19114. Upgrade to commons-compress 1.26.1 due to CVEs. (#6636) [hadoop]

2024-06-06 Thread via GitHub
pjfanning opened a new pull request, #6867: URL: https://github.com/apache/hadoop/pull/6867 This addresses two CVEs triggered by malformed archives Important: Denial of Service CVE-2024-25710 Moderate: Denial of Service CVE-2024-26308 Contributed by PJ Fanning

[jira] [Commented] (HADOOP-19154) upgrade bouncy castle to 1.78.1 due to CVEs

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852763#comment-17852763 ] ASF GitHub Bot commented on HADOOP-19154: - pjfanning opened a new pull request,

[PR] HADOOP-19154. Upgrade bouncycastle to 1.78.1 due to CVEs (#6755) [hadoop]

2024-06-06 Thread via GitHub
pjfanning opened a new pull request, #6866: URL: https://github.com/apache/hadoop/pull/6866 Addresses * CVE-2024-29857 - Importing an EC certificate with specially crafted F2m parameters can cause high CPU usage during parameter evaluation. * CVE-2024-30171 - Possible timing based

[jira] [Commented] (HADOOP-19194) Add test to find unshaded dependencies in the aws sdk

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852743#comment-17852743 ] ASF GitHub Bot commented on HADOOP-19194: - HarshitGupta11 opened a new pull req

[jira] [Updated] (HADOOP-19194) Add test to find unshaded dependencies in the aws sdk

2024-06-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19194: Labels: pull-request-available (was: ) > Add test to find unshaded dependencies in the a

[PR] HADOOP-19194:Add test to find unshaded dependencies in the aws sdk [hadoop]

2024-06-06 Thread via GitHub
HarshitGupta11 opened a new pull request, #6865: URL: https://github.com/apache/hadoop/pull/6865 ### Description of PR ### How was this patch tested? Local testing via test. ### For code changes: - [x] Does the title or this PR starts with the corresponding

[jira] [Created] (HADOOP-19194) Add test to find unshaded dependencies in the aws sdk

2024-06-06 Thread Harshit Gupta (Jira)
Harshit Gupta created HADOOP-19194: -- Summary: Add test to find unshaded dependencies in the aws sdk Key: HADOOP-19194 URL: https://issues.apache.org/jira/browse/HADOOP-19194 Project: Hadoop Common

Re: [PR] HDFS-17528. FsImageValidation: set txid when saving a new image [hadoop]

2024-06-06 Thread via GitHub
szetszwo commented on PR #6828: URL: https://github.com/apache/hadoop/pull/6828#issuecomment-2151810390 The last few lines of the Jenkins build before failure: ``` [2024-06-03T20:14:42.551Z] cd /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-6828/ubuntu-focal/src/hadoop-hdfs

Re: [PR] HDFS-17534 RBF: Support leader follower mode for multiple subclusters [hadoop]

2024-06-06 Thread via GitHub
hadoop-yetus commented on PR #6861: URL: https://github.com/apache/hadoop/pull/6861#issuecomment-2151728419 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18929) Build failure while trying to create apache 3.3.7 release locally.

2024-06-06 Thread Kanaka Kumar Avvaru (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852667#comment-17852667 ] Kanaka Kumar Avvaru commented on HADOOP-18929: -- There are quite few third