onlyarnav opened a new pull request, #71059: URL: https://github.com/apache/airflow/pull/71059
### Description During the refactoring of `SparkSubmitOperator` into backend strategy classes (`_KubernetesSparkSubmitBackend`, `_YarnSparkSubmitBackend`, `_StandaloneSparkSubmitBackend`), the strategy backends reached directly into internal/private attributes and methods of `SparkSubmitHook`: - `hook._conf` - `hook._kubernetes_driver_pod` - `hook._yarn_application_id` - `hook._driver_id` / `hook._driver_status` - `hook._connection` - `hook._poll_k8s_driver_via_api()` - `hook._start_yarn_application_status_tracking()` - `hook._kill_yarn_application()` - `hook._start_driver_status_tracking()` - `hook._run_post_submit_commands()` Reaching into private `_` members across class boundaries breaks object encapsulation. This PR promotes these key internal attributes and methods on `SparkSubmitHook` to clean public interface members: - **Public Properties:** `@property def conf`, `@property def kubernetes_driver_pod` (with setter), `@property def yarn_application_id`, `@property def driver_id` (with setter), `@property def driver_status`, `@property def connection`. - **Public Methods:** `poll_k8s_driver_via_api()`, `start_yarn_application_status_tracking()`, `kill_yarn_application()`, `start_driver_status_tracking()`, `run_post_submit_commands()`. The strategy backend classes in `spark_submit.py` have been updated to use these clean public hook interfaces, and a unit test has been added to `test_spark_submit.py`. ### Related Context Follow-up thought from PR #68679 review. ### fixes: #71058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
