202201142 opened a new issue, #65420:
URL: https://github.com/apache/airflow/issues/65420
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.1.8
### What happened and how to reproduce it?
### Summary
In an Airflow 3.x deployment, a worker process still hard-requires database
configuration during bootstrap.
When AIRFLOW__DATABASE__SQL_ALCHEMY_CONN is not set for worker containers,
the worker fails during airflow.settings.initialize() before it can start
serving Task Execution API work.
Airflow 3 workers are intended to execute tasks through the Task Execution
API and should not require direct metadata DB connectivity for normal worker
operation ( as i can not see any workaround where worker needed metadata conn)
### Reproduction steps
Deploy Airflow 3.1.8 with Celery-based workers (Docker Compose environment).
Start worker container without setting AIRFLOW__DATABASE__SQL_ALCHEMY_CONN.
( If in your entrypoint.sh you have `wait_for_airflow_db` method then you also
need to set `CONNECTION_CHECK_MAX_COUNT = 0` for worker)
Observe worker startup failure.
### Observed stack trace
```
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 3, in <module>
from airflow.__main__ import main
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/__init__.py", line
79, in <module>
settings.initialize()
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/settings.py", line
690, in initialize
configure_vars()
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/settings.py", line
243, in configure_vars
SQL_ALCHEMY_CONN_ASYNC =
_get_async_conn_uri_from_sync(sync_uri=SQL_ALCHEMY_CONN)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/settings.py", line
223, in _get_async_conn_uri_from_sync
scheme, rest = sync_uri.split(":", maxsplit=1)
^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)
```
### Relevant code references
https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/settings.py#L239
https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/settings.py#L251
### What you think should happen instead?
For Airflow 3 worker runtime: Worker bootstrap should not fail solely
because `AIRFLOW__DATABASE__SQL_ALCHEMY_CONN` is unset.
If DB config is still intentionally required in current design, then:
startup error should clearly state that requirement for workers, and docs
should explicitly call this out for Airflow 3 worker deployment.
### Operating System
Linux
### Deployment
Docker-Compose
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
This seems related to unconditional DB variable initialization in
airflow.settings.configure_vars() during worker process bootstrap. what's
thoughts about it ?
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
