[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[ https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=386477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386477 ] ASF GitHub Bot logged work on BEAM-9160: Author: ASF GitHub Bot Created on: 13/Feb/20 09:24 Start Date: 13/Feb/20 09:24 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #10846: [BEAM-9160] Removed WebIdentityTokenCredentialsProvider explicit json (de)serialization in AWS2 module URL: https://github.com/apache/beam/pull/10846 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 386477) Time Spent: 3.5h (was: 3h 20m) > Update AWS SDK to support Kubernetes Pod Level Identity > --- > > Key: BEAM-9160 > URL: https://issues.apache.org/jira/browse/BEAM-9160 > Project: Beam > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.17.0 >Reporter: Mohamed Noah >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Many organizations have started leveraging pod level identity in Kubernetes. > The current version of the AWS SDK packaged with Beam 2.17.0 is out of date > and doesn't provide native support to pod level identity access management. > > It is recommended that we introduce support to access AWS resources such as > S3 using pod level identity. > Current Version of the AWS Java SDK in Beam: > def aws_java_sdk_version = "1.11.519" > Proposed AWS Java SDK Version: > > com.amazonaws > aws-java-sdk > 1.11.710 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[ https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=386476=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386476 ] ASF GitHub Bot logged work on BEAM-9160: Author: ASF GitHub Bot Created on: 13/Feb/20 09:24 Start Date: 13/Feb/20 09:24 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10846: [BEAM-9160] Removed WebIdentityTokenCredentialsProvider explicit json (de)serialization in AWS2 module URL: https://github.com/apache/beam/pull/10846#issuecomment-585631045 Merged manually to squash the commits This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 386476) Time Spent: 3h 20m (was: 3h 10m) > Update AWS SDK to support Kubernetes Pod Level Identity > --- > > Key: BEAM-9160 > URL: https://issues.apache.org/jira/browse/BEAM-9160 > Project: Beam > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.17.0 >Reporter: Mohamed Noah >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Many organizations have started leveraging pod level identity in Kubernetes. > The current version of the AWS SDK packaged with Beam 2.17.0 is out of date > and doesn't provide native support to pod level identity access management. > > It is recommended that we introduce support to access AWS resources such as > S3 using pod level identity. > Current Version of the AWS Java SDK in Beam: > def aws_java_sdk_version = "1.11.519" > Proposed AWS Java SDK Version: > > com.amazonaws > aws-java-sdk > 1.11.710 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[ https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=386274=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386274 ] ASF GitHub Bot logged work on BEAM-9160: Author: ASF GitHub Bot Created on: 12/Feb/20 22:16 Start Date: 12/Feb/20 22:16 Worklog Time Spent: 10m Work Description: ecapoccia commented on pull request #10846: [BEAM-9160] Removed WebIdentityTokenCredentialsProvider explicit json (de)serialization in AWS2 module URL: https://github.com/apache/beam/pull/10846 @iemejia @andeb this mirrors the changes for AWS2 (left behind in the previous PR) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 386274) Time Spent: 3h 10m (was: 3h) > Update AWS SDK to support Kubernetes Pod Level Identity > --- > > Key: BEAM-9160 > URL: https://issues.apache.org/jira/browse/BEAM-9160 > Project: Beam > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.17.0 >Reporter: Mohamed Noah >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Many organizations have started leveraging pod level identity in Kubernetes. > The current version of the AWS SDK packaged with Beam 2.17.0 is out of date > and doesn't provide native support to pod level identity access management. > > It is recommended that we introduce support to access AWS resources such as > S3 using pod level identity. > Current Version of the AWS Java SDK in Beam: > def aws_java_sdk_version = "1.11.519" > Proposed AWS Java SDK Version: > > com.amazonaws > aws-java-sdk > 1.11.710 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[ https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=385745=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385745 ] ASF GitHub Bot logged work on BEAM-9160: Author: ASF GitHub Bot Created on: 12/Feb/20 09:19 Start Date: 12/Feb/20 09:19 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10836: [BEAM-9160] Removed WebIdentityTokenCredentialsProvider explicit json (de)serialization in AWS module URL: https://github.com/apache/beam/pull/10836#issuecomment-585109076 Merged manually to adjust the commit message This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 385745) Time Spent: 2h 50m (was: 2h 40m) > Update AWS SDK to support Kubernetes Pod Level Identity > --- > > Key: BEAM-9160 > URL: https://issues.apache.org/jira/browse/BEAM-9160 > Project: Beam > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.17.0 >Reporter: Mohamed Noah >Priority: Major > Fix For: 2.20.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > Many organizations have started leveraging pod level identity in Kubernetes. > The current version of the AWS SDK packaged with Beam 2.17.0 is out of date > and doesn't provide native support to pod level identity access management. > > It is recommended that we introduce support to access AWS resources such as > S3 using pod level identity. > Current Version of the AWS Java SDK in Beam: > def aws_java_sdk_version = "1.11.519" > Proposed AWS Java SDK Version: > > com.amazonaws > aws-java-sdk > 1.11.710 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[ https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=385746=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385746 ] ASF GitHub Bot logged work on BEAM-9160: Author: ASF GitHub Bot Created on: 12/Feb/20 09:19 Start Date: 12/Feb/20 09:19 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #10836: [BEAM-9160] Removed WebIdentityTokenCredentialsProvider explicit json (de)serialization in AWS module URL: https://github.com/apache/beam/pull/10836 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 385746) Time Spent: 3h (was: 2h 50m) > Update AWS SDK to support Kubernetes Pod Level Identity > --- > > Key: BEAM-9160 > URL: https://issues.apache.org/jira/browse/BEAM-9160 > Project: Beam > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.17.0 >Reporter: Mohamed Noah >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3h > Remaining Estimate: 0h > > Many organizations have started leveraging pod level identity in Kubernetes. > The current version of the AWS SDK packaged with Beam 2.17.0 is out of date > and doesn't provide native support to pod level identity access management. > > It is recommended that we introduce support to access AWS resources such as > S3 using pod level identity. > Current Version of the AWS Java SDK in Beam: > def aws_java_sdk_version = "1.11.519" > Proposed AWS Java SDK Version: > > com.amazonaws > aws-java-sdk > 1.11.710 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[
https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=385564=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385564
]
ASF GitHub Bot logged work on BEAM-9160:
Author: ASF GitHub Bot
Created on: 12/Feb/20 00:16
Start Date: 12/Feb/20 00:16
Worklog Time Spent: 10m
Work Description: ecapoccia commented on pull request #10825: [BEAM-9160]
Update AWS SDK to support Pod Level Identity
URL: https://github.com/apache/beam/pull/10825#discussion_r377976039
##
File path:
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/options/AwsModule.java
##
@@ -122,6 +123,8 @@ public AWSCredentialsProvider deserializeWithType(
return new SystemPropertiesCredentialsProvider();
} else if
(typeName.equals(ProfileCredentialsProvider.class.getSimpleName())) {
return new ProfileCredentialsProvider();
+ } else if
(typeName.equals(WebIdentityTokenCredentialsProvider.class.getSimpleName())) {
+return WebIdentityTokenCredentialsProvider.create();
Review comment:
@iemejia @andeb opened PR https://github.com/apache/beam/pull/10836 as
discussed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 385564)
Time Spent: 2h 40m (was: 2.5h)
> Update AWS SDK to support Kubernetes Pod Level Identity
> ---
>
> Key: BEAM-9160
> URL: https://issues.apache.org/jira/browse/BEAM-9160
> Project: Beam
> Issue Type: Improvement
> Components: dependencies
>Affects Versions: 2.17.0
>Reporter: Mohamed Noah
>Priority: Major
> Fix For: 2.20.0
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Many organizations have started leveraging pod level identity in Kubernetes.
> The current version of the AWS SDK packaged with Beam 2.17.0 is out of date
> and doesn't provide native support to pod level identity access management.
>
> It is recommended that we introduce support to access AWS resources such as
> S3 using pod level identity.
> Current Version of the AWS Java SDK in Beam:
> def aws_java_sdk_version = "1.11.519"
> Proposed AWS Java SDK Version:
>
> com.amazonaws
> aws-java-sdk
> 1.11.710
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[ https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=385559=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385559 ] ASF GitHub Bot logged work on BEAM-9160: Author: ASF GitHub Bot Created on: 12/Feb/20 00:08 Start Date: 12/Feb/20 00:08 Worklog Time Spent: 10m Work Description: ecapoccia commented on pull request #10836: BEAM-9160 URL: https://github.com/apache/beam/pull/10836 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 385559) Time Spent: 2.5h (was: 2h 20m) > Update AWS SDK to support Kubernetes Pod Level Identity > --- > > Key: BEAM-9160 > URL: https://issues.apache.org/jira/browse/BEAM-9160 > Project: Beam > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.17.0 >Reporter: Mohamed Noah >Priority: Major > Fix For: 2.20.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Many organizations have started leveraging pod level identity in Kubernetes. > The current version of the AWS SDK packaged with Beam 2.17.0 is out of date > and doesn't provide native support to pod level identity access management. > > It is recommended that we introduce support to access AWS resources such as > S3 using pod level identity. > Current Version of the AWS Java SDK in Beam: > def aws_java_sdk_version = "1.11.519" > Proposed AWS Java SDK Version: > > com.amazonaws > aws-java-sdk > 1.11.710 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[ https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=385479=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385479 ] ASF GitHub Bot logged work on BEAM-9160: Author: ASF GitHub Bot Created on: 11/Feb/20 21:46 Start Date: 11/Feb/20 21:46 Worklog Time Spent: 10m Work Description: suztomo commented on issue #10825: [BEAM-9160] Update AWS SDK to support Pod Level Identity URL: https://github.com/apache/beam/pull/10825#issuecomment-584869325 The linkage errors from svm is false positives. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 385479) Time Spent: 2h 20m (was: 2h 10m) > Update AWS SDK to support Kubernetes Pod Level Identity > --- > > Key: BEAM-9160 > URL: https://issues.apache.org/jira/browse/BEAM-9160 > Project: Beam > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.17.0 >Reporter: Mohamed Noah >Priority: Major > Fix For: 2.20.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many organizations have started leveraging pod level identity in Kubernetes. > The current version of the AWS SDK packaged with Beam 2.17.0 is out of date > and doesn't provide native support to pod level identity access management. > > It is recommended that we introduce support to access AWS resources such as > S3 using pod level identity. > Current Version of the AWS Java SDK in Beam: > def aws_java_sdk_version = "1.11.519" > Proposed AWS Java SDK Version: > > com.amazonaws > aws-java-sdk > 1.11.710 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9160) Update AWS SDK to support Kubernetes Pod Level Identity
[ https://issues.apache.org/jira/browse/BEAM-9160?focusedWorklogId=385477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385477 ] ASF GitHub Bot logged work on BEAM-9160: Author: ASF GitHub Bot Created on: 11/Feb/20 21:43 Start Date: 11/Feb/20 21:43 Worklog Time Spent: 10m Work Description: andeb commented on issue #10825: [BEAM-9160] Update AWS SDK to support Pod Level Identity URL: https://github.com/apache/beam/pull/10825#issuecomment-584867846 Thanks for reviewing and merging it, @iemejia! With regards to the linkage errors, they may be false positives considering that there are lines of code to ignore classes from GraalVm? https://github.com/GoogleCloudPlatform/cloud-opensource-java/blob/master/dependencies/src/main/java/com/google/cloud/tools/opensource/classpath/LinkageChecker.java#L208-L213 Also, I thought it made sense to expose `WebIdentityTokenCredentialsProvider` directly to be consistent with other exposed credential providers. Happy to do any cleanup though but it seems that @ecapoccia is already on it! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 385477) Time Spent: 2h 10m (was: 2h) > Update AWS SDK to support Kubernetes Pod Level Identity > --- > > Key: BEAM-9160 > URL: https://issues.apache.org/jira/browse/BEAM-9160 > Project: Beam > Issue Type: Improvement > Components: dependencies >Affects Versions: 2.17.0 >Reporter: Mohamed Noah >Priority: Major > Fix For: 2.20.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Many organizations have started leveraging pod level identity in Kubernetes. > The current version of the AWS SDK packaged with Beam 2.17.0 is out of date > and doesn't provide native support to pod level identity access management. > > It is recommended that we introduce support to access AWS resources such as > S3 using pod level identity. > Current Version of the AWS Java SDK in Beam: > def aws_java_sdk_version = "1.11.519" > Proposed AWS Java SDK Version: > > com.amazonaws > aws-java-sdk > 1.11.710 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
