theophanevie commented on issue #15799:
URL: https://github.com/apache/airflow/issues/15799#issuecomment-1285605202
Having this issue on airflow `2.4.1` aswell
celery version 5.2.7
celery configuration :
``` python
celery_app.conf.update(
worker_prefetch_multiplier=1,
task_acks_late=True,
task_reject_on_worker_lost=True,
worker_max_tasks_per_child=100,
worker_cancel_long_running_tasks_on_connection_loss=True,
)
```
worker log :
``` shell
[2022-10-20T06:16:28.195699315Z] ssl.SSLEOFError: EOF occurred in violation
of protocol (_ssl.c:2396)
--
| | [2022-10-20T06:16:28.195695965Z] return self._sslobj.write(data)
| | [2022-10-20T06:16:28.195692435Z] File
"/usr/local/lib/python3.10/ssl.py", line 1149, in write
| | [2022-10-20T06:16:28.195689285Z] n = write(s)
| | [2022-10-20T06:16:28.195685375Z] File
"/usr/local/lib/python3.10/site-packages/amqp/transport.py", line 595, in _write
| | [2022-10-20T06:16:28.195681955Z] self._write(s)
| | [2022-10-20T06:16:28.195678275Z] File
"/usr/local/lib/python3.10/site-packages/amqp/transport.py", line 347, in write
| | [2022-10-20T06:16:28.195674215Z] write(buffer_store.view[:offset])
| | [2022-10-20T06:16:28.195670455Z] File
"/usr/local/lib/python3.10/site-packages/amqp/method_framing.py", line 186, in
write_frame
| | [2022-10-20T06:16:28.195667145Z] conn.frame_writer(1,
self.channel_id, sig, args, content)
| | [2022-10-20T06:16:28.195663465Z] File
"/usr/local/lib/python3.10/site-packages/amqp/abstract_channel.py", line 70, in
send_method
| | [2022-10-20T06:16:28.195658615Z] return self.send_method(
| | [2022-10-20T06:16:28.195655015Z] File
"/usr/local/lib/python3.10/site-packages/amqp/channel.py", line 1407, in
basic_ack
| | [2022-10-20T06:16:28.195651455Z]
self.channel.basic_ack(self.delivery_tag, multiple=multiple)
| | [2022-10-20T06:16:28.195643425Z] File
"/usr/local/lib/python3.10/site-packages/kombu/message.py", line 123, in ack
| | [2022-10-20T06:16:28.195640275Z] self.ack(multiple=multiple)
| | [2022-10-20T06:16:28.195636695Z] File
"/usr/local/lib/python3.10/site-packages/kombu/message.py", line 128, in
ack_log_error
| | [2022-10-20T06:16:28.195633284Z] Traceback (most recent call last):
| | [2022-10-20T06:16:28.195630094Z]
| | [2022-10-20T06:16:28.195626734Z] During handling of the above
exception, another exception occurred:
| | [2022-10-20T06:16:28.195623684Z]
| | [2022-10-20T06:16:28.195619844Z] amqp.exceptions.PreconditionFailed:
(0, 0): (406) PRECONDITION_FAILED - delivery acknowledgement on channel 1 timed
out. Timeout value used: 1800000 ms. This timeout value can be configured, see
consumers doc guide to learn more
| | [2022-10-20T06:16:28.195616414Z] raise error_for_code(
| | [2022-10-20T06:16:28.195612414Z] File
"/usr/local/lib/python3.10/site-packages/amqp/channel.py", line 293, in
_on_close
| | [2022-10-20T06:16:28.195609064Z] listener(*args)
| | [2022-10-20T06:16:28.195604304Z] File
"/usr/local/lib/python3.10/site-packages/amqp/abstract_channel.py", line 156,
in dispatch_method
| | [2022-10-20T06:16:28.195600954Z] return
self.channels[channel_id].dispatch_method(
| | [2022-10-20T06:16:28.195597284Z] File
"/usr/local/lib/python3.10/site-packages/amqp/connection.py", line 537, in
on_inbound_method
| | [2022-10-20T06:16:28.195593984Z] callback(channel, method_sig, buf,
None)
| | [2022-10-20T06:16:28.195590474Z] File
"/usr/local/lib/python3.10/site-packages/amqp/method_framing.py", line 53, in
on_frame
| | [2022-10-20T06:16:28.195587124Z] return self.on_inbound_frame(frame)
| | [2022-10-20T06:16:28.195583494Z] File
"/usr/local/lib/python3.10/site-packages/amqp/connection.py", line 531, in
blocking_read
| | [2022-10-20T06:16:28.195580234Z] while not
self.blocking_read(timeout):
| | [2022-10-20T06:16:28.195576514Z] File
"/usr/local/lib/python3.10/site-packages/amqp/connection.py", line 525, in
drain_events
| | [2022-10-20T06:16:28.195573244Z] drain_events(timeout=0)
| | [2022-10-20T06:16:28.195569814Z] File
"/usr/local/lib/python3.10/site-packages/kombu/transport/base.py", line 217, in
_read
| | [2022-10-20T06:16:28.195566844Z] reader(loop)
| | [2022-10-20T06:16:28.195563264Z] File
"/usr/local/lib/python3.10/site-packages/kombu/transport/base.py", line 235, in
on_readable
| | [2022-10-20T06:16:28.195560104Z] cb(*cbargs)
| | [2022-10-20T06:16:28.195556484Z] File
"/usr/local/lib/python3.10/site-packages/kombu/asynchronous/hub.py", line 362,
in create_loop
| | [2022-10-20T06:16:28.195553264Z] next(loop)
| | [2022-10-20T06:16:28.195549584Z] File
"/usr/local/lib/python3.10/site-packages/celery/worker/loops.py", line 97, in
asynloop
| | [2022-10-20T06:16:28.195546274Z] c.loop(*c.loop_args())
| | [2022-10-20T06:16:28.195542404Z] File
"/usr/local/lib/python3.10/site-packages/celery/worker/consumer/consumer.py",
line 628, in start
| | [2022-10-20T06:16:28.195530283Z] step.start(parent)
| | [2022-10-20T06:16:28.195526623Z] File
"/usr/local/lib/python3.10/site-packages/celery/bootsteps.py", line 116, in
start
| | [2022-10-20T06:16:28.195523193Z] blueprint.start(self)
| | [2022-10-20T06:16:28.195519783Z] File
"/usr/local/lib/python3.10/site-packages/celery/worker/consumer/consumer.py",
line 332, in start
| | [2022-10-20T06:16:28.195516343Z] return self.obj.start()
| | [2022-10-20T06:16:28.195512243Z] File
"/usr/local/lib/python3.10/site-packages/celery/bootsteps.py", line 365, in
start
| | [2022-10-20T06:16:28.195488873Z] step.start(parent)
| | [2022-10-20T06:16:28.195485193Z] File
"/usr/local/lib/python3.10/site-packages/celery/bootsteps.py", line 116, in
start
| | [2022-10-20T06:16:28.195481333Z] self.blueprint.start(self)
| | [2022-10-20T06:16:28.195477453Z] File
"/usr/local/lib/python3.10/site-packages/celery/worker/worker.py", line 203, in
start
| | [2022-10-20T06:16:28.195473203Z] Traceback (most recent call last):
| | [2022-10-20T06:16:28.195455983Z] [2022-10-20 06:16:28,193:
CRITICAL/MainProcess] Couldn't ack 6, reason:SSLEOFError(8, 'EOF occurred in
violation of protocol (_ssl.c:2396)')
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
