[
https://issues.apache.org/jira/browse/AIRFLOW-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15982876#comment-15982876
]
Joe Schmid commented on AIRFLOW-1142:
-------------------------------------
Bolke, thanks for checking this out. Answers to your questions:
* The two successful runs shown in the UI had a different number of tasks (2)
in the Level2 subdag. The failure happened when I added a third task to the
Level2 subdag. However, the issue does seem to be intermittent and you might
have to trigger that test DAG a few times to observe the issue.
* I can definitely rerun with debug logging on and capture logs
The item that sticks out from the existing log is that dependencies are all met
for the task that never runs and it just logs that over and over:
Dependencies all met for <TaskInstance:
Test_Nested_SubDAG.level1_subdag.Level1_Task1 2017-04-24 23:20:38.776547 [None]>
{models.py:4061} INFO - Updating state for <DagRun
Test_Nested_SubDAG.level1_subdag @ 2017-04-24 23:20:38.776547:
backfill_2017-04-24T23:20:38.776547, externally triggered: False> considering 3
task(s)
{jobs.py:1994} INFO - [backfill progress] | finished run 0 of 1 | tasks
waiting: 1 | succeeded: 2 | kicked_off: 0 | failed: 0 | skipped: 0 |
deadlocked: 0 | not ready: 0
{models.py:1126} INFO - Dependencies all met for <TaskInstance:
Test_Nested_SubDAG.level1_subdag.Level1_Task1 2017-04-24 23:20:38.776547 [None]>
{models.py:4061} INFO - Updating state for <DagRun
Test_Nested_SubDAG.level1_subdag @ 2017-04-24 23:20:38.776547:
backfill_2017-04-24T23:20:38.776547, externally triggered: False> considering 3
task(s)
{jobs.py:1994} INFO - [backfill progress] | finished run 0 of 1 | tasks
waiting: 1 | succeeded: 2 | kicked_off: 0 | failed: 0 | skipped: 0 |
deadlocked: 0 | not ready: 0
> SubDAG Tasks Not Executed Even Though All Dependencies Met
> ----------------------------------------------------------
>
> Key: AIRFLOW-1142
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1142
> Project: Apache Airflow
> Issue Type: Bug
> Components: subdag
> Affects Versions: 1.8.1
> Environment: 1.8.1rc1+incubating, Celery
> Reporter: Joe Schmid
> Priority: Blocker
> Attachments: 2017-04-24T23-20-38-776547, Test_Nested_SubDAG_0.png,
> Test_Nested_SubDAG_1-Zoomed.png, test_nested_subdag.py
>
>
> Testing on 1.8.1rc1, we noticed that tasks in subdags were not getting
> executed even though all dependencies had been met.
> We were able to create a simple test DAG that re-creates the issue. Attached
> is a test DAG, the log file of the subdag operator that shows it fails to run
> even though dependencies are met, and screenshots of what the UI looks like.
> This is definitely a regression as we have many similarly constructed DAGs
> that have been running successfully on a pre-v1.8 version (a fork of
> 1.7.1.3+master) for some time.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
