sundeep8967 opened a new pull request, #72285:
URL: https://github.com/apache/airflow/pull/72285
<!--
Thank you for contributing! Please make sure that your PR targets the `main`
branch.
-->
Closes: #71058
### Description
Following the refactoring of `SparkSubmitOperator` into deployment backend
strategies (`_KubernetesSparkSubmitBackend`, `_YarnSparkSubmitBackend`),
several backend methods accessed private attributes and methods on
`SparkSubmitHook` directly (`_conf`, `_kubernetes_driver_pod`,
`_yarn_application_id`, `_poll_k8s_driver_via_api`).
This PR:
1. Exposes public properties on `SparkSubmitHook`:
- `@property def conf(self)`
- `@property def kubernetes_driver_pod(self)` (with getter and setter)
- `@property def yarn_application_id(self)`
2. Promotes `_poll_k8s_driver_via_api()` to public
`poll_k8s_driver_via_api()` (maintaining `_poll_k8s_driver_via_api()` as a
backwards-compatible alias).
3. Updates backend strategy classes in `spark_submit.py` to use these clean
public properties and methods.
4. Adds unit test coverage in
`tests/unit/apache/spark/hooks/test_spark_submit.py`.
Signed-off-by: Sundeep <[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
