jason810496 commented on PR #69520: URL: https://github.com/apache/airflow/pull/69520#issuecomment-4922789577
> The entire reason this exists is so that if the config changes the old logs are still viewable. This breaks that doesn't it Yes, but based on - a) `Airflow 2.11.1 already disabled per-run template retrieval by default for security reasons (https://github.com/apache/airflow/pull/61880);` - b) the ES / OS provider for Airflow 3 did respect the log_template (more details as below) I thought we can introduce the breaking change in `3.4` to drop this feature. > > a direct metadata-DB dependency that conflicts with the Airflow 3 execution model > > This isn't true - we already handle this correctly since 3.0.0 by passing the log file suffix to the worker in ExecuteTaskWorkload; changes in 2.11 directly aren't available in 3.x either so that linked pr isn't really relevant It's different part, not for the log file suffix, only the ES / OS consumed the LogTemplate DB model. IIRC, the ES / OS provider (mainly https://github.com/apache/airflow/pull/53821) that works with Airflow 3 **didn't respect the log_template model** (both write and read side will fetch the latest config value instead of going through the LogTemplate model) at all (I will double check for this part) this is the reason why I raise the discussion to drop the LogTemplate DB model. Or another direction, I can make the OS and ES provider still respect the LogTemplate DB model. WDYT? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
