DartVeDroid commented on issue #53442:
URL: https://github.com/apache/airflow/issues/53442#issuecomment-3831156303
As far as I understand it, in Airflow 3.1.6 logging works like that:
* in `task`:
* we make logs
* logs get passed to `supervisor`
* in most of airflow itself (`supervisor`, etc):
* logs get passed from `task`
* to `structlog` bound logger
* to bound logger's wrapper `AirflowBoundLoggerFilteringAt<...>`
* to wrapper's logger factory's logger: `NamedBytesLogger` or
`NamedWriteLogger`
* to logger's output: stdout or log file
* in api-server:
* we make logs
* logs get passed to ???
So, what we have to do is:
* disable output to files (because I wanted to only use stdout and custom
handlers a-la Airflow 2)
* replace logger_factory with one that returns stdlib `logging` loggers,
when we are not just sending JSONs to another parts of Airflow
* modify `airflow.sdk.log.configure_logging()` to load custom logging
config too just for `triggerer` and `worker` (most airflow components load it
already via `airflow.logging_config.configure_logging()`)
* (in progress) deal with:
* log records' `.msg` field being a JSON
* custom logging handlers' `set_context()` method not being called
I'm currently on the last step, but I'm not sure if I would be allowed to
post it here, despite it being only a crutch honestly.
P.S.:
I'm fine with contributors making changes that break `providers`'
functionality, but, IMO, either `providers`' docs should be updated (which is
kinda hard without the main contributor's involvement), or providers should
explicitly declare that they are incompatible with the last "Stable" version of
Airflow. As of now it's just confusing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
