n-badtke-cg opened a new issue, #61814: URL: https://github.com/apache/airflow/issues/61814
### Description Hi, thank you for the amazing ecosystem ❤️ We are using remote_logging and want to somehow use delete_local_logs, but with a delay (we are exporting the logs using OpenTelemetry Collector FileLog Receiver) So our current solution is to set `logging.delete_local_logs = "False"` and set the environment variable `AIRFLOW__LOG_RETENTION_DAYS=1`. The env is getting used in https://github.com/apache/airflow/blob/main/scripts/docker/clean-logs.sh Now, even with only 1d retention, there are too many logs in our system, and the OTEL FileLog Receiver is unfortunately working poorly with many files. The https://github.com/apache/airflow/blob/main/scripts/docker/clean-logs.sh script is using the command `find -type f -mtime +${RETENTION}` where `RETENTION` is set to the `AIRFLOW__LOG_RETENTION_DAYS` env. Why not just use (or additionally offer) `AIRFLOW__LOG_RETENTION_MINUTES` with `find -type f -mmin +${RETENTION_MINUTES}`? 🙂 ### Use case/motivation More granular configuration option for log retention and thus better offering for largely scaled systems ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
