poonnatuch opened a new issue, #51499:
URL: https://github.com/apache/airflow/issues/51499
### Apache Airflow Provider(s)
redis
### Versions of Apache Airflow Providers
airflow@fb8a120ed440:/opt/airflow$ pip freeze | grep redis
apache-airflow-providers-redis==4.0.2
google-cloud-redis==2.18.1
redis==5.2.1
### Apache Airflow version
3.0.1
### Operating System
Ubuntu 24.04.1 LTS
### Deployment
Docker-Compose
### Deployment details
I deploy using `docker-compose.yaml` from the official document. Last week
the code still working fine but It suddenly error.
### What happened
When trying to excute redis client function from `redis_hook.get_conn()` the
redis throw error Error 2 connecting to redis:6379. No such file or directory.
**But when using Redis Client directly it run successfully.** The problem just
occur out of no where, last week I still using the `redis_hook.get_conn()` and
normally call redis function just fine.
`apache-airflow-providers-redis` and `redis` version
```bash
airflow@fb8a120ed440:/opt/airflow$ pip freeze | grep redis
apache-airflow-providers-redis==4.0.2
google-cloud-redis==2.18.1
redis==5.2.1
```
redis connection configs:
connection_id: redis
host: redis
password: None
```json
{
"db": 0,
"ssl": false,
"ssl_cert_reqs": "required",
"ssl_ca_certs": null,
"ssl_keyfile": null,
"ssl_certfile": null,
"ssl_check_hostname": false
}
```
What I have tried:
1) Create a folder `/var/run/redis` inside the container
https://github.com/redis/redis/issues/8081
2) Try different versions of `apache-airflow-providers-redis` and `redis`
- upgrade the `apache-airflow-providers-redis` version to 4.1.0 while
using `redis==5.2.1`
- upgrade the redis version to `redis==6.2.0`
3) As stated earlier, I have tried using Redis Client directly it connect
and excute successfully
The Test code:
```python
# Get Redis connection from Airflow connection
redis_hook = RedisHook(redis_conn_id="redis")
# Get connection parameters for logging
conn_params = redis_hook.get_connection(redis_hook.redis_conn_id)
logger.info(f"Redis connection parameters: host={conn_params.host},
port={conn_params.port}, password {conn_params.password}")
logger.info("Getting redis client using redis_hook.get_conn()")
redis_client = redis_hook.get_conn()
logger.info("Successfully connected to Redis")
# Test connection with ping
logger.info("Pinging Redis to test connection...")
ping_result = redis_client.ping()
logger.info(f"Redis ping result: {ping_result}")
```
The logs:
```bash
[2025-06-07, 19:00:46] INFO - DAG bundles loaded: dags-folder:
source="airflow.dag_processing.bundles.manager.DagBundlesManager"
[2025-06-07, 19:00:46] INFO - Filling up the DagBag from
/opt/airflow/dags/video_processing_dag.py: source="airflow.models.dagbag.DagBag"
[2025-06-07, 19:00:46] WARNING -
/opt/airflow/dags/video_processing_dag.py:174: UserWarning:
`multiple_outputs=True` is not supported in @task.bash tasks. Ignoring.
stream_check_result = check_video_streams(video_path="{{ params.video_path
}}")
: source="py.warnings"
[2025-06-07, 19:00:46] INFO - Initializing Redis state for task
682ddbcf-d18-4b34-9d4b-bd4b2ef11ef0: source="airflow.task"
[2025-06-07, 19:00:46] INFO - Secrets backends loaded for worker: count=1:
backend_classes=["EnvironmentVariablesBackend"]: source="supervisor"
[2025-06-07, 19:00:46] WARNING - empty cryptography key - values will not be
stored encrypted.: source="airflow.models.crypto"
[2025-06-07, 19:00:46] INFO - Connection Retrieved 'redis':
source="airflow.hooks.base"
[2025-06-07, 19:00:46] INFO - Redis connection parameters: host=redis,
port=6379, password=None: source="airflow.task"
[2025-06-07, 19:00:46] INFO - Getting redis client using
redis_hook.get_conn(): source="airflow.task"
[2025-06-07, 19:00:46] INFO - Secrets backends loaded for worker: count=1:
backend_classes=["EnvironmentVariablesBackend"]: source="supervisor"
[2025-06-07, 19:00:46] INFO - Connection Retrieved 'redis':
source="airflow.hooks.base"
[2025-06-07, 19:00:46] INFO - Successfully connected to Redis:
source="airflow.task"
[2025-06-07, 19:00:46] INFO - Pinging Redis to test connection...:
source="airflow.task"
[2025-06-07, 19:00:46] ERROR - Redis connection error: <class
'redis.exceptions.ConnectionError'>: Error 2 connecting to redis:6379. No such
file or directory.: source="airflow.task"
[2025-06-07, 19:00:46] ERROR - Detailed Redis connection error::
source="airflow.task"
ConnectionError: Error 2 connecting to redis:6379. No such file or directory.
File "/opt/airflow/dags/video_processing_dag.py", line 406 in
initialize_redis_state
File
"/home/airflow/.local/lib/python3.12/site-packages/redis/commands/core.py",
line 1212 in ping
File "/home/airflow/.local/lib/python3.12/site-packages/redis/client.py",
line 559 in execute_command
File "/home/airflow/.local/lib/python3.12/site-packages/redis/client.py",
line 565 in _execute_command
File
"/home/airflow/.local/lib/python3.12/site-packages/redis/connection.py", line
1422 in get_connection
File
"/home/airflow/.local/lib/python3.12/site-packages/redis/connection.py", line
363 in connect
FileNotFoundError: [Errno 2] No such file or directory
File
"/home/airflow/.local/lib/python3.12/site-packages/redis/connection.py", line
357 in connect
File "/home/airflow/.local/lib/python3.12/site-packages/redis/retry.py",
line 62 in call_with_retry
File
"/home/airflow/.local/lib/python3.12/site-packages/redis/connection.py", line
35 in <lambda>
File
"/home/airflow/.local/lib/python3.12/site-packages/redis/connection.py", line
1021 in _connect
File
"/home/airflow/.local/lib/python3.12/site-packages/redis/connection.py", line
1050 in _wrap_socket_with_ssl
```
When try using redis client directly it run perfectly fine.
```python
import redis
# use the same params as in the connections
conn_params = redis_hook.get_connection(redis_hook.redis_conn_id)
redis_client = redis.Redis(
host=conn_params.host,
password=conn_params.password,
port=conn_params.port,
)
# this returns turn
redis_client.ping()
```
### What you think should happen instead
_No response_
### How to reproduce
```python
# Get Redis connection from Airflow connection
redis_hook = RedisHook(redis_conn_id="redis")
# Get connection parameters for logging
conn_params = redis_hook.get_connection(redis_hook.redis_conn_id)
logger.info(f"Redis connection parameters: host={conn_params.host},
port={conn_params.port}, password {conn_params.password}")
logger.info("Getting redis client using redis_hook.get_conn()")
redis_client = redis_hook.get_conn()
logger.info("Successfully connected to Redis")
# Test connection with ping
logger.info("Pinging Redis to test connection...")
ping_result = redis_client.ping() # or any other functions of redis (At
first I use hset)
logger.info(f"Redis ping result: {ping_result}")
```
### Anything else
Last week the system still working just fine. The first time it happened, I
didn't update anything or run any docker compose build.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
