[
https://issues.apache.org/jira/browse/HADOOP-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583733#comment-17583733
]
ASF GitHub Bot commented on HADOOP-16806:
-----------------------------------------
jmahonin commented on PR #4753:
URL: https://github.com/apache/hadoop/pull/4753#issuecomment-1224304355
It looks like the issue is in retrieving `s3a://landsat-pds/scene_list.gz`.
The corresponding role should have full S3 access at this point.
I've enabled the low-level request tracing. Here's what it looks like for
the first parameterized test:
`testJobSubmissionCollectsTokens[0](org.apache.hadoop.fs.s3a.auth.delegation.ITestDelegatedMRJob)`
```
2022-08-23 16:15:25,838 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "HEAD /scene_list.gz
HTTP/1.1[\r][\n]"
2022-08-23 16:15:25,838 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "Host:
landsat-pds.s3.amazonaws.com[\r][\n]"
2022-08-23 16:15:25,838 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "amz-sdk-invocation-id:
<redacted>[\r][\n]"
2022-08-23 16:15:25,838 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "amz-sdk-request:
attempt=1;max=21[\r][\n]"
2022-08-23 16:15:25,838 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "amz-sdk-retry:
0/0/500[\r][\n]"
2022-08-23 16:15:25,838 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "Authorization:
AWS4-HMAC-SHA256 Credential=<redacted>/20220823/us-east-1/s3/aws4_request,
SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;referer;user-agent;x-amz-content-sha256;x-amz-date,
Signature=<redacted>[\r][\n]"
2022-08-23 16:15:25,838 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "Content-Type:
application/octet-stream[\r][\n]"
2022-08-23 16:15:25,838 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "Referer:
https://audit.example.org/hadoop/1/op_get_file_status/<redacted>/?op=op_get_file_status&p1=scene_list.gz&pr=jmahonin/localh...@example.com&ps=<redacted>&id=<redacted>&t0=15&fs=<redacted>&t1=15&ts=1661271325482[\r][\n]"
2022-08-23 16:15:25,839 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "User-Agent: Hadoop
3.4.0-SNAPSHOT, aws-sdk-java/1.12.262 Linux/5.10.104-linuxkit
OpenJDK_64-Bit_Server_VM/25.342-b07 java/1.8.0_342 kotlin/1.4.10
vendor/Private_Build cfg/retry-mode/legacy[\r][\n]"
2022-08-23 16:15:25,839 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "x-amz-content-sha256:
UNSIGNED-PAYLOAD[\r][\n]"
2022-08-23 16:15:25,839 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "X-Amz-Date:
20220823T161525Z[\r][\n]"
2022-08-23 16:15:25,839 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "Connection:
Keep-Alive[\r][\n]"
2022-08-23 16:15:25,839 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 >> "[\r][\n]"
2022-08-23 16:15:25,936 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 << "HTTP/1.1 400 Bad
Request[\r][\n]"
2022-08-23 16:15:25,936 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 << "x-amz-request-id:
ZMY1169S2RC7NEHP[\r][\n]"
2022-08-23 16:15:25,936 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 << "x-amz-id-2:
ENtHr5DR7HX+qkh5FfYQublaU82ykB/SD5fAvR5kC6JZJibFVLH6Rq+F/EutrE3dAL1uTz6yad8=[\r][\n]"
2022-08-23 16:15:25,936 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 << "Content-Type:
application/xml[\r][\n]"
2022-08-23 16:15:25,936 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 << "Date: Tue, 23 Aug 2022
16:15:25 GMT[\r][\n]"
2022-08-23 16:15:25,936 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 << "Server: AmazonS3[\r][\n]"
2022-08-23 16:15:25,937 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 << "Connection: close[\r][\n]"
2022-08-23 16:15:25,937 [JUnit-testJobSubmissionCollectsTokens[0]] DEBUG
http.wire (Wire.java:wire(73)) - http-outgoing-7 << "[\r][\n]"
```
I've also tried adding this to my auth-keys, which likewise does not work:
```
<property>
<name>fs.s3a.bucket.landsat-pds.aws.credentials.provider</name>
<value>org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider</value>
</property>
```
I've seen mention in HADOOP-13551 and HADOOP-18340 that folks have run into
similar issues with this test, although I haven't been able to determine what,
if anything, was done to correct it.
> AWS AssumedRoleCredentialProvider needs ExternalId add
> ------------------------------------------------------
>
> Key: HADOOP-16806
> URL: https://issues.apache.org/jira/browse/HADOOP-16806
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.1
> Reporter: Jon Hartlaub
> Priority: Minor
> Labels: pull-request-available
>
> AWS has added a security feature to the assume-role function in the form of
> the "ExternalId" key in the AWS Java SDK
> {{STSAssumeRoleSessionCredentialsProvider.Builder}} class. To support this
> security feature, the hadoop aws {{AssumedRoleCredentialProvider}} needs a
> patch to include this value from the configuration as well as an added
> Constant to the {{org.apache.hadoop.fs.s3a.Constants}} file.
> The ExternalId is not a required security feature, it is an augmentation of
> the current assume role configuration.
> Proposed:
> * Get the assume-role ExternalId token from the configuration for the
> configuration key {{fs.s3a.assumed.role.externalid}}
> * Use the configured ExternalId value in the
> {{STSAssumeRoleSessionCredentialsProvider.Builder}}
> e.g.
> {{if (StringUtils.isNotEmpty(externalId)) {}}
> {{ builder.withExternalId(externalId); // include the token for
> cross-account assume role}}
> {{}}}
> Tests:
> * +Unit test+ which verifies the ExternalId state value of the
> {{AssumedRoleCredentialProvider}} is consistent with the configured value -
> either empty or populated
> * Question: not sure about how to write the +integration test+ for this
> feature. We have an account configured for this use-case that verifies this
> feature but I don't have much context on the Hadoop project AWS S3
> integration tests, perhaps a pointer could help.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
