steveloughran commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-2242904433
@adnanhemani hadoop branch-3.4 AWS SDK is now at 2.25.53. That has
everything you need, doesn't it?
If you can do a request of this patch back part branch 3.4 I will merge it
steveloughran merged PR #6544:
URL: https://github.com/apache/hadoop/pull/6544
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-2007791751
Yup, this makes sense. I will ensure that Hadoop gets the SDK changes as
soon as the SDK updates are complete. Thank you again for all your time
reviewing this!
--
This is an
steveloughran commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-2007781626
hey, if yetus is unhappy, rebase is the right thing to do, so don't worry
too much. just trying to remember where i was, that's all
--
This is an automated message from the Apache
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-2007775835
Sorry! Yetus was complaining that it could not apply the changes on top of
`trunk` so I instinctively rebased and force pushed - will keep this in mind!
--
This is an automated
steveloughran commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-2007753756
please, please please, no force push once reviews have started unless
there's merge problems or its been neglected for too long...makes it harder to
seee what's changed between
hadoop-yetus commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1995867156
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: |
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1995387593
> You're right there is no rename; copy is all there is. So that is not
available yet? Hmmm. This isn't ready for production yet is it? Let us keep it
in trunk for now. The other
hadoop-yetus commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1995387553
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: |
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1523757683
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AccessGrantConfiguration.java:
##
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1523715465
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:
##
@@ -401,4 +406,22 @@ private static Region
steveloughran commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1522069895
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:
##
@@ -401,4 +406,22 @@ private static Region
steveloughran commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1992515204
You're right there is no rename; copy is all there is. So that is not
available yet? Hmmm. This isn't ready for production yet is it? Let us keep it
in trunk for now. The other
hadoop-yetus commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1985634498
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: |
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1985378218
Hi Steve, thanks for reviewing this as you finished up with the Hadoop
release work, I really appreciate you taking the time to look at this. I'm
re-running the integration and unit
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1517420760
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AccessGrantConfiguration.java:
##
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1517361600
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AccessGrantConfiguration.java:
##
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1517130806
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##
@@ -1624,4 +1624,21 @@ private Constants() {
* Value: {@value}.
*/
steveloughran commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1516477473
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##
@@ -1624,4 +1624,21 @@ private Constants() {
* Value: {@value}.
*/
steveloughran commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1983988376
Test results. thanks for these. you only need failures. (we don't care about
successes unless something has got very slow)
* rebase on trunk; you need "HADOOP-19057. S3A:
steveloughran commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1957375191
really focused purely on 3.4.0 shipping right now, not looking at stuff it
doesn't need. sorry
--
This is an automated message from the Apache Git Service.
To respond to the
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1947538335
Hi @steveloughran, any thoughts on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1936795463
Hi @ahmarsuhail, I've made all changes requested. Please take a look when
you can.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
hadoop-yetus commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1936789090
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: |
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484871939
##
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md:
##
@@ -614,6 +614,38 @@ If the following property is not set or set to `true`, the
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484857819
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AccessGrantConfiguration.java:
##
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484852608
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AccessGrantConfiguration.java:
##
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484851522
##
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md:
##
@@ -614,6 +614,38 @@ If the following property is not set or set to `true`, the
adnanhemani commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484848907
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:
##
@@ -401,4 +411,19 @@ private static Region
ahmarsuhail commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1936082879
@adnanhemani re test failures, just updating the core-site won't be enough
for some of them, you'll also need the code changes in Steve's PR
https://github.com/apache/hadoop/pull/6515
ahmarsuhail commented on code in PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484099683
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java:
##
@@ -401,4 +411,19 @@ private static Region
hadoop-yetus commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1935575917
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: |
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1935468938
WRT ITest failures, this is the `auth-keys.xml` file I'm using as per the
suggestions on #6515. But I'm still getting the above failures - should this
not be the case?
Tested
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1935462396
Hi @ahmarsuhail and @steveloughran - I've opened this with the new code that
has the S3 Access Grants plugin as part of the AWS Java SDK bundle (this is an
evolution of #6507). We
adnanhemani commented on PR #6544:
URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1935460439
Unit tests and Tests results below, per my understanding all failures are
due to various other known reasons:
```
^C88665a0afd7e:hadoop-aws ahemani$ mvn verify -Dparallel-tests
adnanhemani opened a new pull request, #6544:
URL: https://github.com/apache/hadoop/pull/6544
### Description of PR
This adds S3 Access Grants support in S3A using the S3 Access Grants plugin,
which is a part of the AWS Java SDK bundle starting in v2.23.19. As part of
this PR,
37 matches
Mail list logo
