aspedrosa opened a new issue, #34668:
URL: https://github.com/apache/airflow/issues/34668
### Apache Airflow version
2.7.1
### What happened
When running either `airflow celery worker` or `airflow celery flower`
following error happens
```
2023-09-28T13:10:32+01:00 [2023-09-28 13:10:32,666: INFO/MainProcess]
Connected to sentinel://XXX.XXX.XXX.XXX:XXXX/0
2023-09-28T13:10:32+01:00 [1;33m[2023-09-28 13:10:32,667:
WARNING/MainProcess]
/usr/local/lib/python3.9/site-packages/celery/worker/consumer/consumer.py:507:
CPendingDeprecationWarning: The broker_connection_retry configuration setting
will no longer determine
2023-09-28T13:10:32+01:00 whether broker connection retries are made
during startup in Celery 6.0 and above.
2023-09-28T13:10:32+01:00 If you wish to retain the existing behavior for
retrying connections on startup,
2023-09-28T13:10:32+01:00 you should set
broker_connection_retry_on_startup to True.
2023-09-28T13:10:32+01:00 warnings.warn(
2023-09-28T13:10:32+01:00
2023-09-28T13:10:32+01:00 [2023-09-28 13:10:32,670: INFO/MainProcess]
mingle: searching for neighbors
2023-09-28T13:10:33+01:00 [2023-09-28 13:10:33,681: INFO/MainProcess]
mingle: all alone
2023-09-28T13:10:33+01:00 [2023-09-28 13:10:33,703: INFO/MainProcess]
celery@XXXX-airflow-celery-worker-XXXX-dev-dsv01 ready.
2023-09-28T13:10:33+01:00 [1;35m[2023-09-28 13:10:33,707:
CRITICAL/MainProcess] Unrecoverable error: TypeError("read_response() got an
unexpected keyword argument 'push_request'")
2023-09-28T13:10:33+01:00 Traceback (most recent call last):
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/celery/worker/worker.py", line 202, in
start
2023-09-28T13:10:33+01:00 self.blueprint.start(self)
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/celery/bootsteps.py", line 116, in start
2023-09-28T13:10:33+01:00 step.start(parent)
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/celery/bootsteps.py", line 365, in start
2023-09-28T13:10:33+01:00 return self.obj.start()
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/celery/worker/consumer/consumer.py",
line 340, in start
2023-09-28T13:10:33+01:00 blueprint.start(self)
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/celery/bootsteps.py", line 116, in start
2023-09-28T13:10:33+01:00 step.start(parent)
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/celery/worker/consumer/consumer.py",
line 742, in start
2023-09-28T13:10:33+01:00 c.loop(*c.loop_args())
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/celery/worker/loops.py", line 97, in
asynloop
2023-09-28T13:10:33+01:00 next(loop)
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/kombu/asynchronous/hub.py", line 373,
in create_loop
2023-09-28T13:10:33+01:00 cb(*cbargs)
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/kombu/transport/redis.py", line 1341,
in on_readable
2023-09-28T13:10:33+01:00 self.cycle.on_readable(fileno)
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/kombu/transport/redis.py", line 566, in
on_readable
2023-09-28T13:10:33+01:00 chan.handlers[type]()
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/kombu/transport/redis.py", line 910, in
_receive
2023-09-28T13:10:33+01:00 ret.append(self._receive_one(c))
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/kombu/transport/redis.py", line 920, in
_receive_one
2023-09-28T13:10:33+01:00 response = c.parse_response()
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/redis/client.py", line 796, in
parse_response
2023-09-28T13:10:33+01:00 response = self._execute(conn, try_read)
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/redis/client.py", line 772, in _execute
2023-09-28T13:10:33+01:00 return conn.retry.call_with_retry(
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/redis/retry.py", line 46, in
call_with_retry
2023-09-28T13:10:33+01:00 return do()
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/redis/client.py", line 773, in <lambda>
2023-09-28T13:10:33+01:00 lambda: command(*args, **kwargs),
2023-09-28T13:10:33+01:00 File
"/usr/local/lib/python3.9/site-packages/redis/client.py", line 794, in try_read
2023-09-28T13:10:33+01:00 return
conn.read_response(disconnect_on_error=False, push_request=True)
2023-09-28T13:10:33+01:00 TypeError: read_response() got an unexpected
keyword argument 'push_request'
2023-09-28T13:10:34+01:00 [2023-09-28 13:10:34 +0100] [23] [INFO]
Handling signal: term
2023-09-28T13:10:34+01:00 [2023-09-28 13:10:34 +0100] [25] [INFO] Worker
exiting (pid: 25)
2023-09-28T13:10:34+01:00 [2023-09-28 13:10:34 +0100] [24] [INFO] Worker
exiting (pid: 24)
2023-09-28T13:10:34+01:00 [2023-09-28 13:10:34 +0100] [23] [INFO]
Shutting down: Master
```
### What you think should happen instead
This can be fixed by pinning redis to 5.0.1, which includes the following
fix https://github.com/redis/redis-py/pull/2922, in the 2.7 constraints files.
### How to reproduce
Use redis sentinel as a broker.
I had the following setting
`broker_url =
sentinel://X.X.X.X:26379/0;sentinel://X.X.X.X:26379/0;sentinel://X.X.X.X:26379/0`
### Operating System
Container with Oracle Linux 8
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other Docker-based deployment
### Deployment details
Custom image where we install airflow with `pip install
apache-airflow[redis,celery]==1.7.1 --contrains
https://raw.githubusercontent.com/apache/airflow/constraints-2.7.1/constraints-3.9.txt`
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
