As mentioned in the PR, that's currently not the case. There is currently a bug in the core of Airflow ( https://issues.apache.org/jira/browse/AIRFLOW-1623). So if you trigger the on_kill from the UI, the on_kill function is not triggered. And we confirm this behavior.
On Mon, Jan 8, 2018 at 2:15 PM, Driesprong, Fokko <fo...@driesprong.frl> wrote: > Yes, for Spark this should work. Depending on the operator and the > implementation: > https://github.com/apache/incubator-airflow/blob/ > 3e6babe8ed8f8f281b67aa3f4e03bf3cfc1bcbaa/airflow/contrib/ > hooks/spark_submit_hook.py#L412-L428 > > However this is a big change in behaviour. I'm curious about the opinion of > others. > > Cheers, > Fokko > > > 2018-01-08 14:12 GMT+01:00 Milan van der Meer < > milan.vanderm...@realimpactanalytics.com>: > > > Any help? :) > > > > On Thu, Dec 14, 2017 at 8:12 PM, Milan van der Meer < > > milan.vanderm...@realimpactanalytics.com> wrote: > > > > > I recently openend the following PR: https://github.com/apache/ > > > incubator-airflow/pull/2877 > > > > > > The problem is that on_kill is not called for operators when you clear > a > > > task from the UI. > > > Thats problematic when working with ex. spark clusters as the jobs on > the > > > cluster need to be killed. > > > > > > The issue is in the core code of Airflow and Im not familiar enough > with > > > the inner workings there. So I could use some directions on this one > from > > > people who are familiar. > > > > > > For more info, check out the PR. > > > > > > Kind regards, > > > Milan > > > > > > > > > > > -- > > > > *Milan van der Meer* > > > > *Real**Impact* Analytics *| *Big Data Consultant > > www.realimpactanalytics.com > > > > *BE *+32 498 45 96 22 <0032498459622>* | Skype *milan.vandermeer.ria > > > -- *Milan van der Meer* *Real**Impact* Analytics *| *Big Data Consultant www.realimpactanalytics.com *BE *+32 498 45 96 22 <0032498459622>* | Skype *milan.vandermeer.ria
