onlyarnav opened a new issue, #71058: URL: https://github.com/apache/airflow/issues/71058
### Under which category would you file this issue? Providers ### Apache Airflow version 3.3.0 ### What happened and how to reproduce it? ### Description Following the refactoring of `SparkSubmitOperator` into backend strategy classes (`_KubernetesSparkSubmitBackend`, `_YarnSparkSubmitBackend`, `_StandaloneSparkSubmitBackend`), several operator backend methods reach directly into private attributes and methods of `SparkSubmitHook`: - `hook._conf` - `hook._kubernetes_driver_pod` - `hook._yarn_application_id` - `hook._poll_k8s_driver_via_api()` Reaching into private `_` members across class boundaries weakens object encapsulation between `SparkSubmitOperator` and `SparkSubmitHook`. ### Proposed Solution Promote these key internal attributes and methods on `SparkSubmitHook` to clean public interface methods/properties: 1. Expose `@property def conf(self)` on `SparkSubmitHook`. 2. Expose `@property def kubernetes_driver_pod(self)` (getter and setter). 3. Expose `@property def yarn_application_id(self)`. 4. Promote `_poll_k8s_driver_via_api()` to public `poll_k8s_driver_via_api()`. Update `_SparkSubmitDeploymentBackend` strategy classes in `spark_submit.py` to use these clean public hook methods. ### Related Context Follow-up thought from PR #68679 review. ### What you think should happen instead? _No response_ ### Operating System _No response_ ### Deployment None ### Apache Airflow Provider(s) apache-spark ### Versions of Apache Airflow Providers _No response_ ### Official Helm Chart version Not Applicable ### Kubernetes Version _No response_ ### Helm Chart configuration _No response_ ### Docker Image customizations _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
