hanxdatadog opened a new pull request, #68915: URL: https://github.com/apache/airflow/pull/68915
Closes #68912 ## Problem The Celery worker ignores `[logging] json_logs` / `AIRFLOW__LOGGING__JSON_LOGS`. It always calls `configure_logging(output=sys.stdout.buffer)` without passing `json_output`, so structured JSON logging is never enabled on the worker even when configured globally. ## Solution Add a two-level config lookup before calling `configure_logging`: 1. `[celery] json_logs` — worker-specific override, takes precedence. 2. `[logging] json_logs` — global setting, used when the celery key is absent. 3. `False` — default when neither key is configured. This mirrors the existing `[celery] CELERY_LOGGING_LEVEL` / `[logging] LOGGING_LEVEL` fallback pattern already present in the worker startup code. ```ini # Global — affects the API server, scheduler, and workers: [logging] json_logs = True # Or override for the Celery worker only, leaving other components unchanged: [celery] json_logs = True ``` ## Changes - **`providers/celery/src/airflow/providers/celery/cli/celery_command.py`**: read `[celery] json_logs` → `[logging] json_logs` → `False` and pass `json_output` to `configure_logging`. - **`providers/celery/provider.yaml`** + **`get_provider_info.py`**: new `json_logs` config option under `[celery]`. - **`providers/celery/tests/unit/celery/cli/test_celery_command.py`**: `TestWorkerJsonLogs` — 3 tests covering (a) default is False, (b) global `[logging]` setting is inherited, (c) `[celery]` override takes precedence. - **`providers/celery/docs/celery_executor.rst`**: new "Worker logging" section documenting the fallback order and Airflow 3.0/3.2 compatibility note. - **`providers/celery/newsfragments/68912.feature.rst`**: changelog entry. ## Notes - Only active on Airflow 3+ (`AIRFLOW_V_3_0_PLUS` guard is unchanged). - `conf.getboolean(..., fallback=None)` is safe on 3.0/3.1 even when the `[celery] json_logs` key is absent. - `version_added: ~` in `provider.yaml` — version not yet decided; to be set by the release manager. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
