[ https://issues.apache.org/jira/browse/ARROW-10675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Balanca updated ARROW-10675: --------------------------------- Description: It seems to me that Arrow only supports at the moment the "AssumeRole" AWS STS API, but not the other options offered: * [https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html#stsapi_comparison] * [https://sdk.amazonaws.com/cpp/api/LATEST/class_aws_1_1_auth_1_1_s_t_s_assume_role_web_identity_credentials_provider.html] I am clearly no security/infra expert, but it seems that the configuration "AssumeRoleWithWebIdentity" is used commonly in Kubernetes setups, and I believe it would be beneficial for Arrow C++ & Python library to support. At the moment, a workaround is to call directly `aws sts` to generate a temporary session, but it is a fairly painful solution as the session expires, all PyArrow objects with an S3 filesystem (datasets, ...) need to be re-built with new credentials. was: It seems to me that Arrow only supports at the moment the "AssumeRole" AWS STS API, but not the other options offered: * [https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html#stsapi_comparison] * [https://sdk.amazonaws.com/cpp/api/LATEST/class_aws_1_1_auth_1_1_s_t_s_assume_role_web_identity_credentials_provider.html] I am clearly no security/infra expert, but it seems that the configuration "AssumeRoleWithWebIdentity" is used commonly in Kubernetes setups, and I believe it would be beneficial for Arrow C++ & Python library to support. At the moment, a work around is to call directly `aws sts` to generate a temporary session, but it is a fairly paintful as the session expires: all PyArrow objects with an S3 filesystem (datasets, ...) needs to be re-built with new credentials. > [C++][Python] Support AWS S3 Web identity credentials > ----------------------------------------------------- > > Key: ARROW-10675 > URL: https://issues.apache.org/jira/browse/ARROW-10675 > Project: Apache Arrow > Issue Type: Improvement > Affects Versions: 1.0.1, 2.0.0 > Reporter: Paul Balanca > Priority: Major > > It seems to me that Arrow only supports at the moment the "AssumeRole" AWS > STS API, but not the other options offered: > * > [https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html#stsapi_comparison] > * > [https://sdk.amazonaws.com/cpp/api/LATEST/class_aws_1_1_auth_1_1_s_t_s_assume_role_web_identity_credentials_provider.html] > I am clearly no security/infra expert, but it seems that the configuration > "AssumeRoleWithWebIdentity" is used commonly in Kubernetes setups, and I > believe it would be beneficial for Arrow C++ & Python library to support. > At the moment, a workaround is to call directly `aws sts` to generate a > temporary session, but it is a fairly painful solution as the session > expires, all PyArrow objects with an S3 filesystem (datasets, ...) need to be > re-built with new credentials. -- This message was sent by Atlassian Jira (v8.3.4#803005)