vgkowski opened a new pull request, #70766:
URL: https://github.com/apache/airflow/pull/70766
Add `EmrServerlessGetSessionEndpointOperator`, which resolves the Spark
Connect
endpoint URL and a short-lived authentication token for a running EMR
Serverless
interactive session so a client can attach to it.
Why:
- Each interactive session has its own endpoint and a token that expires (~1
hour),
so the endpoint must be fetched fresh right before connecting rather than
cached.
- The token is a short-lived credential, so the operator registers it with
Airflow's
secrets masker before returning it via XCom, keeping it out of task logs
and the
rendered XCom value in the UI.
Changes:
- `EmrServerlessHook.get_session_endpoint` returns the raw
`GetSessionEndpoint` boto3
response (thin wrapper, per the provider convention); the operator does
the field
transformation before pushing to XCom.
- Add `mask_secret` to the `common.compat` SDK shim so the masker import
resolves across
supported Airflow versions.
- Docs (howto section), a system-test example step, and unit tests.
The feature is gated on `botocore >= 1.43.0` at runtime (introduced in the
base PR), so
the provider's dependency floor is unchanged.
* related: #69207
> [!NOTE]
> This PR is part of a stack and depends on
> https://github.com/apache/airflow/pull/70763 (adds
> `EmrServerlessStartSessionOperator` and the shared session plumbing).
Please review/merge
> that one first — this PR's base branch is
`vgkowski/amazon-emr-serverless-start-session` until it
> lands, then it retargets to `main`.
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Kiro (AI coding assistant)
Generated-by: Kiro, following the Gen-AI guidelines in
contributing-docs/05_pull_requests.rst
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
