While you wait for a fix on that JIRA ticket, you may be able to add an
intermediary step in your AirFlow graph, calling Spark's REST API after
submitting the job, and dig into the actual status of the application,
and make a success/fail decision accordingly. You can make repeated
calls in a
Hi Team,
My team recently conducted a POC of Kubernetes/Airflow/Spark with great
success. The major concern we have about this system, after the completion of
our POC is a behavior of spark-submit. When called with a Kubernetes API
endpoint as master spark-submit seems to always return exit
Hi Team,
Is there an estimated time for 3.0.0 release? My team recently did a POC with
Spark/Airflow/Kubernetes and we are trying to align our implementation to the
end of the preview. Thank you.
Marshall Markham
Data Engineer
PrecisionLender, a Q2 Company
NOTE: This communication and any
