hussein-awala commented on code in PR #37066:
URL: https://github.com/apache/airflow/pull/37066#discussion_r1471747200
##########
airflow/providers/celery/provider.yaml:
##########
@@ -276,6 +276,20 @@ config:
type: float
example: ~
default: "1.0"
+ task_acks_late:
+ description: |
+ If task runs exceed the visibility_timeout, Celery will re-assign
new tasks, even if the
+ originally started task is still running successfully. The newly
created task instance runs
+ then concurrently and the Airflow UI and logs only show an error
message:
+ 'Task Instance Not Running' FAILED: Task is in the running state
+ Setting task_acks_late to True will force Celery to wait until a
task is finished before a
+ new task unstance is assigned. This effectively overrides the
visibility timeout.
+ See also:
+
https://docs.celeryq.dev/en/stable/reference/celery.app.task.html#celery.app.task.Task.acks_late
+ version_added: ~
+ type: boolean
+ example: ~
+ default: "False"
Review Comment:
```suggestion
default: "True"
```
##########
airflow/providers/celery/executors/default_celery.py:
##########
@@ -72,7 +72,7 @@ def _broker_supports_visibility_timeout(url):
"accept_content": ["json"],
"event_serializer": "json",
"worker_prefetch_multiplier": conf.getint("celery",
"worker_prefetch_multiplier", fallback=1),
- "task_acks_late": True,
+ "task_acks_late": conf.getboolean("celery", "task_acks_late",
fallback=False),
Review Comment:
We need to keep the default value True
```suggestion
"task_acks_late": conf.getboolean("celery", "task_acks_late",
fallback=True),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
