Hi Martijn, As I can see FLINK-31095 <https://issues.apache.org/jira/browse/FLINK-31095> is now closed. I will try the things mentioned there on my test systems and share the results.
Thanks for your help. Regards Anuj On Tue, May 23, 2023 at 1:10 PM Martijn Visser <martijnvis...@apache.org> wrote: > Hi Anuj, > > I recalled another ticket on this topic, which had some things to test. I > don't know if that resolved the issue, can you verify it? See > https://issues.apache.org/jira/browse/FLINK-31095 > > Best regards, > > Martijn > > On Tue, May 23, 2023 at 7:04 AM Anuj Jain <anuj...@gmail.com> wrote: > >> Hello, >> Please provide some pointers on this issue. >> >> Thanks !! >> >> Regards >> Anuj >> >> On Fri, May 19, 2023 at 1:34 PM Anuj Jain <anuj...@gmail.com> wrote: >> >>> Hi Community, >>> Looking forward to some advice on the problem. >>> >>> I also found this similar Jira, but not sure if a fix has been done for >>> the Hadoop connector - can someone confirm this. >>> [FLINK-23487] IRSA doesn't work with S3 - ASF JIRA (apache.org) >>> <https://issues.apache.org/jira/browse/FLINK-23487> >>> >>> Is there any other way to integrate Flink source/sink with AWS IAM from >>> EKS ? >>> >>> Regards >>> Anuj >>> >>> On Thu, May 18, 2023 at 12:41 PM Anuj Jain <anuj...@gmail.com> wrote: >>> >>>> Hi, >>>> I have a flink job running on EKS, reading and writing data records to >>>> S3 buckets. >>>> I am trying to set up access credentials via AWS IAM. >>>> I followed this: >>>> https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html >>>> >>>> I have configured: >>>> com.amazonaws.auth.WebIdentityTokenCredentialsProvider as the credential >>>> provider in flink-conf.yaml for hadoop s3a connector, and annotated my >>>> service account with the role. >>>> >>>> When running the job, i am getting access denied error >>>> Exception: >>>> Caused by: >>>> com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: >>>> Not authorized to perform sts:AssumeRoleWithWebIdentity (Service: >>>> AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; >>>> Request ID: 923df33a-802e-47e2-a203-0841aca03dd8; Proxy: null) >>>> >>>> I have tried to access S3 buckets from AWS CLI running in a pod with >>>> the same service account and that works. >>>> >>>> Am I using the correct credential provider for IAM integration, not >>>> sure if Hadoop S3a supports it. >>>> https://issues.apache.org/jira/browse/HADOOP-18154 >>>> >>>> Please advise if I am doing anything wrong in setting up credentials >>>> via IAM. >>>> >>>> Regards >>>> Anuj Jain >>>> >>>
