[GitHub] [airflow] kelechi2020 opened a new issue, #29129: Dag runs for dag with dynamic tasks marked completed with empty task details

[via GitHub]

kelechi2020 opened a new issue, #29129:
URL: https://github.com/apache/airflow/issues/29129

   ### Apache Airflow version
   
   2.5.1
   
   ### What happened
   
   Dag run marked as complete but tasks not  run.
   
   ### What you think should happen instead
   
   Dagrun completes and task runs details are available.
   
   ### How to reproduce
   
   Run below dag
   This dag dynamically creates "10" tasks. 
   ```
   from time import sleep
   
   from airflow.models.dag import DAG
   from airflow.operators.python import PythonOperator
   from datetime import datetime, timedelta
   
   def _dummy_task():
       dummy_obj = dict()
       sleep(2)
       for _ in range(0, 500):
           dummy_obj.update({_: _})
       sleep(15)
       return dummy_obj
   
   
   with DAG('dag_with_parrarel_task',  schedule_interval=None,
            start_date=datetime(2023, 4, 25), is_paused_upon_creation=False, 
catchup=False) as example_dag_1_1:
   
       tasks = [
           PythonOperator(
               task_id="__".join(["task", "{}_of_{}".format(i, '10')]), 
python_callable=_dummy_task,
           )
           for i in range(1, 10)
       ]
       tasks[0].set_downstream(list(tasks[1:]))
   ```
   
   Notice the dag run succeeds but with no task run details. 
   See:
   
![image](https://user-images.githubusercontent.com/25426248/214273550-19ba2b00-8451-4fe1-bae7-6270b3909ed4.png)
   
   I went looking for clues in the DB:
   Dagrun table  looks okay:
   
![image](https://user-images.githubusercontent.com/25426248/214273784-352d9291-d5fc-4cd9-ae8b-9a315f6ea31b.png)
   
   Task instances:
   No task instance was created for the dagruns:
   
![image](https://user-images.githubusercontent.com/25426248/214274699-18eac421-8c77-4e3e-aa0b-263a2c485d16.png)
   
   
   
   Went looking further down in scheduler or worker logs at the time dagrun was 
triggered:
   Scheduler:
   <img width="1748" alt="image" 
src="https://user-images.githubusercontent.com/25426248/214277028-0a725005-021b-4b67-9e9d-8c55a614e832.png";>
   
   
   
   
   
   ### Operating System
   
   linux
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   This was observed in  both on 2.5.1 `apache/airflow:2.5.1-python3.10`
    and 2.5.0  `apache/airflow:2.5.0-python3.10`
   airflow versions.
   
   Works good
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org