Airflow dependencies were simplified a bit, -i no longer ignores failed state tasks, check out the -A flag which ignores pretty much all dependencies (including the failed state tasks), though depending on the version you are using there is a bug that is being fixed here: https://github.com/apache/incubator-airflow/pull/2327
On Wed, Jul 5, 2017 at 8:45 AM, Weiwei Zhang <vvianzh...@gmail.com> wrote: > I am using airflow 1.8.1 as well. It is able to pick up the rest of the > tasks when using backfill with the only exception which is when there is a > task failed and I had to clear the status to allow the backfill to work. > Any ideas why it is behaving like this? The previous version 1.6.2 didn't > require clearing the failed task before doing backfill. > > Thx a lot, > Viv > > > On Jul 5, 2017, at 7:38 AM, Tobias Feldhaus < > tobias.feldh...@localsearch.ch> wrote: > > > > I’ve just pulled the newest master and build it; the behaviour is the > same. How can it be that “–i” is not honoured and dependencies are checked? > > > > > > On 05.07.2017, 15:49, "Tobias Feldhaus" <tobias.feldh...@localsearch.ch> > wrote: > > > > But nonetheless, is it not possible to backfill and ignore the > upstream dependencies with “-i” ? > > > > On 05.07.2017, 14:34, "Tobias Feldhaus" <Tobias.Feldhaus@localsearch. > ch> wrote: > > > > I meant –i , but I just needed to manually set the upstream > things to success and it worked. Nevermind. > > > > Best, > > Tobi > > > > On 05.07.2017, 14:28, "Tobias Feldhaus" < > tobias.feldh...@localsearch.ch<mailto:tobias.feldh...@localsearch.ch>> > wrote: > > > > Hi, > > > > When running airflow (1.8.1) backfill with –I and –t like: > > > > airflow backfill -t 'nonspider_sessions' -i -I -s 2017-05-30 -e > 2017-05-31 google_pipelines > > > > I would expect it to rerun that specific task and ignoring the > dependencies. Instead I see this: > > > > [2017-07-05 12:23:30,419] {base_task_runner.py:95} INFO - > Subtask: [2017-07-05 12:23:30,419] {models.py:1145} INFO - Dependencies not > met for <TaskInstance: google_pipelines.nonspider_sessions 2017-05-30 > 05:30:00 [queued]>, dependency 'Trigger Rule' FAILED: Task's trigger rule > 'all_success' requires all upstream tasks to have succeeded, but found 3 > non-success(es). upstream_tasks_state={'successes': 0L, 'failed': 0L, > 'upstream_failed': 0L, 'skipped': 0L, 'done': 0L}, > upstream_task_ids=['frontend_sensor', 'log_sensor', 'tracker_pipeline'] > > > > Am I doing it wrong? > > > > > > > > Best, > > Tobi > > > > > > > > >
