ephraimbuddy commented on issue #34012:
URL: https://github.com/apache/airflow/issues/34012#issuecomment-1719221982
What I see is deadlock:
```log
[2023-09-14T10:41:55.788+0000] {backfill_job_runner.py:686} WARNING -
Deadlock discovered for ti_status.to_run=dict_values([<TaskInstance:
stale_data_test.insert_new_daily_data.insert
backfill__2023-07-30T06:00:00+00:00 map_index=0 [scheduled]>, <TaskInstance:
stale_data_test.insert_new_daily_data.insert
backfill__2023-07-22T06:00:00+00:00 map_index=0 [scheduled]>, <TaskInstance:
stale_data_test.insert_new_daily_data.insert
backfill__2023-07-19T06:00:00+00:00 map_index=0 [scheduled]>])
[2023-09-14T10:41:55.874+0000] {backfill_job_runner.py:416} INFO - [backfill
progress] | finished run 0 of 31 | tasks waiting: 0 | succeeded: 16 | running:
0 | failed: 0 | skipped: 0 | deadlocked: 3 | not ready: 3
[2023-09-14T10:41:55.889+0000] {local_executor.py:402} INFO - Shutting down
LocalExecutor; waiting for running tasks to finish. Signal again if you don't
want to wait.
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 33, in <module>
sys.exit(load_entry_point('apache-airflow', 'console_scripts',
'airflow')())
File "/opt/airflow/airflow/__main__.py", line 59, in main
args.func(args)
File "/opt/airflow/airflow/cli/cli_config.py", line 49, in command
return func(*args, **kwargs)
File "/opt/airflow/airflow/utils/cli.py", line 114, in wrapper
return f(*args, **kwargs)
File "/opt/airflow/airflow/utils/providers_configuration_loader.py", line
55, in wrapped_function
return func(*args, **kwargs)
File "/opt/airflow/airflow/cli/commands/dag_command.py", line 153, in
dag_backfill
_run_dag_backfill(dags, args)
File "/opt/airflow/airflow/cli/commands/dag_command.py", line 105, in
_run_dag_backfill
dag.run(
File "/opt/airflow/airflow/models/dag.py", line 2671, in run
run_job(job=job, execute_callable=job_runner._execute)
File "/opt/airflow/airflow/utils/session.py", line 79, in wrapper
return func(*args, session=session, **kwargs)
File "/opt/airflow/airflow/jobs/job.py", line 305, in run_job
return execute_job(job, execute_callable=execute_callable)
File "/opt/airflow/airflow/jobs/job.py", line 334, in execute_job
ret = execute_callable()
File "/opt/airflow/airflow/utils/session.py", line 79, in wrapper
return func(*args, session=session, **kwargs)
File "/opt/airflow/airflow/jobs/backfill_job_runner.py", line 949, in
_execute
raise BackfillUnfinished(err, ti_status)
airflow.exceptions.BackfillUnfinished: BackfillJob is deadlocked.
These tasks have succeeded:
DAG ID Task ID Run ID Try
number
--------------- ------------ -----------------------------------
------------
stale_data_test get_run_data backfill__2023-07-16T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-17T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-18T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-19T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-20T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-21T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-22T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-23T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-24T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-25T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-26T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-27T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-28T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-29T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-30T06:00:00+00:00
1
stale_data_test get_run_data backfill__2023-07-31T06:00:00+00:00
1
These tasks are running:
DAG ID Task ID Run ID Try number
-------- --------- -------- ------------
These tasks have failed:
DAG ID Task ID Run ID Try number
-------- --------- -------- ------------
These tasks are skipped:
DAG ID Task ID Run ID Try number
-------- --------- -------- ------------
These tasks are deadlocked:
DAG ID Task ID Run ID
Map Index Try number
--------------- ----------------------------
----------------------------------- ----------- ------------
stale_data_test insert_new_daily_data.insert
backfill__2023-07-19T06:00:00+00:00 1 0
stale_data_test insert_new_daily_data.insert
backfill__2023-07-22T06:00:00+00:00 1 0
stale_data_test insert_new_daily_data.insert
backfill__2023-07-30T06:00:00+00:00 1 0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]