sunchao commented on code in PR #5872:
URL: https://github.com/apache/datafusion-comet/pull/5872#discussion_r4173866124
##########
native/core/src/parquet/objectstore/s3.rs:
##########
@@ -1000,11 +1173,393 @@ impl CredentialProviderMetadata {
}
}
+/// The STS region the SDK profile provider takes, on its regional host, when
no region is found.
+const STS_FALLBACK_REGION: &str = "us-east-1";
+
+/// The region whose STS endpoint is the global https://sts.amazonaws.com,
signed for us-east-1,
+/// where the Java SDK sends a role profile's request when no region is found.
+const STS_GLOBAL_REGION: &str = "aws-global";
+
+/// Profile properties that make a profile resolve credentials other than its
static keys.
+const CREDENTIAL_PROPERTIES: [&str; 10] = [
+ "role_arn",
+ "credential_source",
+ "web_identity_token_file",
+ "credential_process",
+ "login_session",
+ "sso_session",
+ "sso_account_id",
+ "sso_region",
+ "sso_role_name",
+ "sso_start_url",
+];
+
+/// A role a profile assumes from its `source_profile`.
+#[derive(Debug)]
+#[cfg_attr(test, derive(PartialEq))]
+struct ProfileRole {
+ role_arn: String,
+ external_id: Option<String>,
+ session_name: Option<String>,
+ region: Option<String>,
+}
+
+/// A role profile's chain: the profile whose credentials start it and the
roles assumed from
+/// them, outermost first.
+#[derive(Debug)]
+#[cfg_attr(test, derive(PartialEq))]
+struct ProfileRoleChain {
+ base: String,
+ /// Whether the base is a web identity role, the only base that calls STS.
+ base_needs_region: bool,
+ roles: Vec<ProfileRole>,
+}
+
+fn has_only_static_keys(profile: &Profile) -> bool {
+ profile.get("aws_access_key_id").is_some()
+ && !CREDENTIAL_PROPERTIES
+ .iter()
+ .any(|property| profile.get(property).is_some())
+}
+
+/// Follows `role_arn` and `source_profile` from `selected` the way the SDK's
profile provider
+/// does (aws-config's profile/credentials/repr.rs). That provider assumes
every role with one
+/// STS region and offers no per-role endpoint, while Hadoop's Java SDK gives
each role its own,
+/// so the roles are assumed here instead. A chain this does not mirror
returns the reason, to
+/// stay on the SDK provider.
+fn resolve_role_chain(profiles: &ProfileSet, selected: &str) ->
Result<ProfileRoleChain, String> {
+ let mut name = selected;
+ let mut visited = Vec::new();
+ let mut roles = Vec::new();
+ loop {
+ let profile = profiles
+ .get_profile(name)
+ .ok_or_else(|| format!("profile {name} is not defined"))?;
+ if visited.contains(&name) {
+ return Err(format!("profile {name} is in a source_profile cycle"));
+ }
+ visited.push(name);
+ // The SDK takes a source profile's static keys ahead of its other
settings, which the
+ // base provider, reading the profile as its selected one, would not.
+ if visited.len() > 1
+ && profile.get("aws_access_key_id").is_some()
+ && !has_only_static_keys(profile)
+ {
+ return Err(format!(
Review Comment:
[P2] Preserve Hadoop’s credential precedence for mixed source profiles. With
the newly supported `ProfileAWSCredentialsProvider`, select an assume-role
`[analytics]` profile whose `source_profile=source`, where `[source]` contains
static keys and `credential_process`. Hadoop’s Java SDK gives
`credential_process` precedence. This branch instead rejects the custom chain
and delegates to the Rust SDK, which prioritizes a source profile’s static
keys. Native therefore signs `AssumeRole` with a different identity and fails
when only the process identity can assume the role. Could we resolve mixed
source profiles using Java’s precedence instead of delegating the entire chain,
and add a regression test asserting the signing key?
Evidence: Reproduced with the verbatim current production profile-resolution
block and locked aws-config 1.12.0, aws-runtime 1.9.2 and aws-sdk-sts 1.114.0.
The fixture sets analytics.role_arn, analytics.source_profile=source and
analytics.region=us-west-2. The source contains synthetic-source-key plus a
credential_process returning synthetic-process-key. Native recorded an STS
request signed by synthetic-source-key. Java SDK 2.29.52’s actual
ProfileCredentialsUtils selected synthetic-process-key from the same fixture.
An in-memory STS accepting only the process identity returned AccessDenied to
native. Sources, fixture and outputs:
/tmp/comet5872-c559-review/profile_probe.rs, JavaProfileOracle.java,
java-mixed-process.credentials, profile_probe.log, java-oracle.log and
restricted_profile_probe.log. No external STS requests were made.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
