GitHub user potiuk added a comment to the discussion: Airflow ci/cd pipeline to rebuild airflow
I think you need to trace down what happens and what error you have - there is no "one way" - especially that docker-compose is not something we have recommended or have a guildeline to on how to productionize it, so any scripts you do, you need to make sure you test. I guess you mist have some error - say for example your docker compose down will fail (not sure how docker compose works) if there are no containers and that might break your whole command. Not sure also what "endless shutdown means" and why - this is only you can see on what stops your compose from shutting down. That needs you to login to the server and see what's going on when you do "docker compose down" - there might be multiple reasons. Re - pausing and unpausing dags - you could - potentially utilise CLI (as @enchant3dmango mentioned) or - better - REST API and python client we release, to iterate through and pause or maybe stop all the running tasks. Again note that docker compose is not a perfect way of deploying airlfow as it lacks a lot of flexibility of things like K8S executor for example. Where you can shutdown and restart airflow without actually stopping all running pods (Airlfow has "pod adoption" mechanism in this case. So it's a bit of your chose to choose a platform (docker-compose) that has very little support for more "prodution" support you would like to see, and requiring from you to do more custom work to make it happen. GitHub link: https://github.com/apache/airflow/discussions/47149#discussioncomment-12340604 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
