nailo2c commented on code in PR #65991:
URL: https://github.com/apache/airflow/pull/65991#discussion_r3254099649
##########
providers/apache/spark/src/airflow/providers/apache/spark/hooks/spark_submit.py:
##########
@@ -99,13 +99,27 @@ class SparkSubmitHook(BaseHook, LoggingMixin):
job finishes (on both success and on_kill). Useful for cleaning up
sidecars such
as Istio (e.g. ``["curl -X POST localhost:15020/quitquitquit"]``).
Each command
is executed via the shell; failures produce a warning but do not fail
the task.
+ :param yarn_track_via_application_status: If True (when master is YARN and
Review Comment:
I tend to keep `yarn_track_via_application_status`:
1. The spark_submit hook follows a yarn/k8s parallel naming pattern (e.g.
`_yarn_application_id` vs `_kubernetes_application_id`), so this naming follows
the current architecture.
2. A generic name without a k8s implementation would mislead k8s users into
expecting it to work.
3. I think focusing on the current issue's scope might be better. We can
open a follow-up issue to track the k8s pod status.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
