zjj commented on PR #54448: URL: https://github.com/apache/airflow/pull/54448#issuecomment-3194444602
> Two comments here: > > * timeout is a bad idea - for reasons mentioned by @Programmer-RD-AI > * find is generally a better solution, especiallly if you add filter to only find and `chown` those files that are not already owned by AIRFLOW_UID/0 - i believe that solution will be faster in general than `chown -R` - but then you have to likely have first find for directories and then for files > * having a lot of small files in logs is quite an indication of a problem on it's own. If your intention is to improve the docker-compose to be more resilient, you could likely pair this init with deleting old files based on some thresholds. That would make more resilent to infinitely growing the logs + you could use the grooming script that we already use in Helm chart to do both - initial and periodical cleanup - but that would require a bit more updates to the docker-compose > > One note to remember (and something you can read in the documentation): Currently the docker-compose is NOT intended to be used in any kinds of production environments so it is simple by definition and we do not intend to make it "drop-in" ready for production - people who choose docker-compose as production kind of deployment are basically responsible to make sure THEIR docker compose (which could be based on ours) implements all necessary features to maintain things better. So there is no actual need to contribute back anything you developed in your own compose files - those enhancements, production-configurations etc. are solely done by those who choose docker-compose to run it for production. > > This is a deliberate choice of ours. Unlike with Helm chart - where we have complete set of tests, tt's extremely difficult and complicated to implement, maintain and test docker-compose that will be usable out-of-the-box as a generic solution - and we opted to NOT do it. Our docker-compose is merely a starting point for those who want to have their own compose. So if you think that for **your case** adding timeout is enough - it's your choice, but I do not agree it's a good solution **in general** that everyone should start with when creating their own docker-compose based on ours. Others migh choose other options. > > But if you come up with a solution that does not complicate the compose file too much and is usable by others in a generic way (using grooming scripts and having it configurable to be run periodically seems like good generic approach) - feel free to contribute it this way. I understand the official disclaimer that docker-compose isn't meant for production use, so I believe there shouldn't focus too much on unnecessary "optimizations". I would close this pr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
