[
https://issues.apache.org/jira/browse/FLINK-37683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hong Liang Teoh updated FLINK-37683:
------------------------------------
Fix Version/s: aws-connector-5.2.0
(was: aws-connector-5.1.0)
> [Connector][AWS] SSO credentials fail because of shaded dependencies missing
> ----------------------------------------------------------------------------
>
> Key: FLINK-37683
> URL: https://issues.apache.org/jira/browse/FLINK-37683
> Project: Flink
> Issue Type: Bug
> Components: Connectors / AWS
> Affects Versions: aws-connector-3.0.0, aws-connector-4.0.0,
> aws-connector-4.1.0, aws-connector-3.1.0, aws-connector-4.2.0,
> aws-connector-4.3.0, aws-connector-4.4.0, aws-connector-5.0.0,
> aws-connector-5.1.0
> Environment: * Apache Flink (any version using
> {{flink-connector-aws}} based on AWS SDK v2)
> * {{flink-connector-aws}} (using {{{}KinesisStreamsSource{}}})
> * Java application
> * AWS credentials configured via {{~/.aws/config}} using SSO profiles (e.g.,
> {{{}sso-session{}}}, {{{}sso_account_id{}}}, {{{}sso_role_name{}}}).
> * Running locally via {{flink run}} or deploying a job using the shaded
> connector.
> Reporter: Ryan Despain
> Priority: Minor
> Labels: pull-request-available
> Fix For: aws-connector-5.2.0
>
>
> *Problem:* Users attempting to authenticate to AWS services (specifically
> Kinesis via {{{}KinesisStreamsSource{}}}) using AWS SSO profiles configured
> in {{~/.aws/config}} encounter a runtime error when using the standard shaded
> {{{}flink-connector-aws{}}}. The application fails to load credentials. This
> issue prevents users who rely on AWS SSO from using the connector.
> *Steps to Reproduce:*
> # Configure local AWS credentials using an SSO profile in
> {{{}~/.aws/config{}}}. Ensure the profile works with the AWS CLI.
> # Create a simple Flink Java application that uses {{KinesisStreamsSource}}
> to connect to an AWS Kinesis stream.
> # Include the {{org.apache.flink:flink-connector-aws:<version>}} dependency
> (which brings in the shaded AWS SDK).
> # Attempt to run the job using {{flink run}} or by building an application
> jar and running it.
> # _Expected Result:_ The job connects successfully using the SSO credentials.
> # _Actual Result:_ The job fails with an error related to loading
> credentials.
> _(Optional: Link to reproduction repository provided by the user:
> [soisyourface/flink-kinesis-blah|https://www.google.com/search?q=https://github.com/soisyourface/flink-kinesis-blah])_
> *Error Message:*
> {noformat}
> Unable to load credentials from any of the providers in the chain{noformat}
> And specifically:
> {noformat}
> To use Sso related properties in the 'default' profile, the 'sso' servicee
> module must be on the class path.{noformat}
>
> *Root Cause Analysis:* The AWS SDK for Java v2 uses
> {{software.amazon.awssdk.auth.credentials.internal.ProfileCredentialsUtils}}
> which dynamically loads the {{SSO_PROFILE_CREDENTIALS_PROVIDER_FACTORY}}
> using {{{}ClassLoaderHelper.loadClass{}}}.
> The {{flink-connector-aws}} project shades AWS SDK dependencies into
> {{flink-connector-aws-base}} to avoid classpath conflicts. However, the
> necessary AWS SDK modules for SSO functionality
> ({{{}software.amazon.awssdk:sso{}}} and
> {{{}software.amazon.awssdk:ssooidc{}}}) are not included as dependencies in
> the {{{}flink-connector-aws-base/pom.xml{}}}.
> Consequently, when Flink shades the dependencies, these modules are omitted
> from the final shaded jar. At runtime, the {{ProfileCredentialsUtils}}
> attempts to load the SSO factory class, but it cannot be found on the
> classpath because it wasn't included in the shaded artifact, leading to the
> observed error.
> *Proposed Solution:* Add {{software.amazon.awssdk:sso}} and
> {{software.amazon.awssdk:ssooidc}} as {{<dependency>}} entries within the
> {{<dependencies>}} section of the {{flink-connector-aws-base/pom.xml}} file
> in the {{apache/flink-connector-aws}} repository. This will ensure these
> modules are included when the connector is built and shaded.
> *Verification:* The user who reported this issue verified locally that
> building {{flink-connector-aws}} with these added dependencies resolved the
> problem in their Flink application.
> *Related PR:* A Pull Request with the proposed fix has been submitted:
> [apache/flink-connector-aws#197|https://www.google.com/search?q=https://github.com/apache/flink-connector-aws/pull/197]
> *Slack Conversation:*
> https://apache-flink.slack.com/archives/C03G7LJTS2G/p1743713205172909
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
