Hello here, I would like to restart a discussion about Postgres in our Helm Chart. We have been discussing it [1] in the past and got to a lazy consensus [2] after Bitnami dropped support for their "community" charts and images.
*Tl;DR; My proposal is that we revert that decision and breathe a bit of new life into our helm chart - by making Postgres "first class citizen" with Zalando Postgres Operator, alongside converting our Helm Chart to use a more modern approach for extendibility -> Kustomize that will help us to make our chart far more maintainable (we need it badly).* There were some attempts to make it happen [3] but it turns out this is quite complex and adds a lot of complexity, also it can't be easily applied to Airflow 2 because of lack of sqlite parallel access in Airflow 2. I think both me and Jens do not really like the added complexity there and it seems it's not an easy one overall I did a bit of research and I found that one of the popular ways of using Postgres on K8S is the Zalando's Postgres Operator: [4] that uses Patroni https://github.com/patroni/patroni template for - potentially - highly scalable and configurable deployment of postgres that can be even used for production. And with "K8S Operator" approach, it is far more flexible and "production deployable" - it is also fully maintained and - given that Zalando is not in a business of selling infrastructure - there is a chance this will be a long-term community-managed and available solution. And it's actually production-proven as I know for a fact it IS used in Zalando. The Postgres operator integrates nicely with Helm Chart and it also could be our way towards starting to use Kustomize (which is a rather modern, standard way of resource templating for K8S) - because the operator itself uses the "Kustomisable" manifest to deploy the operator. I think one of the biggest issues we have with our Helm Chart was that every single option our users would like to add had to be integrated with the Chart and requires PRs and testing. The goal of Kustomize and the way it integrates with Helm is to allow for very flexible customisation of the chart without having to implement anything in the chart upfront. It uses patches and overlays and adds tooling that allows it to easily extend and modify things on the "user" side rather than having everything result in a PR that maintainers have to review and approve. THe quickstart for the Operator and it's integration with chart is super simple [5] - and it seems that we will not be able to just drop-in replace Bitnami chart, but also we will be able to do it in the way (with Kustomie) that people will be able to get our chart and turn it into fully production ready deployments ALSO including Postgres - while we will not have to carry the burden of actually making Postgres production-ready - due to the way how Kustomize works. Generally speaking, when Kustomize is used people who would like to use any new features, annotations etc. - they can easily do it if Kustomize is integrated into the chart - more about it for example in [6] I think switching to Postgres K8S Operator maintained by community, with Kustomize - we can learn how to do it and we can turn our helm chart a little bit upside down - and rather than having to maintain it with every single change, we could just explain our users how they can use kustomize to extend the chart and make it a default way of deploying our chart - with kustomize as a driver. This will turn our Helm Chart not into a black-box that people will demand having new functionality, but into a "Starting Point" with built-in and explained extension capabilities with howtos and best practices. I know i was a big proponent of telling people "our postgres is not for production" - but, maybe - instead of dropping Postgres support we make it more "professional" - i.e. integrate it in the way that whoever would like to use Postgres in their k8s and not outside. K8S has long evolved from a stateless controller, over the last few years it gained the full capabilities of running stateful deployments - and K8S operators made it actually possible to manage even complex deployments in an easy way (especially if we do not have to maintain the operators, and they are proven and production-battle-tested). We really need a bit of fresh air in our Helm Chart if we are to continue maintaining it. It is a long time neglected (not because of some personal neglect, simply the way how maintenance works - it requires a lot of effort from maintainers to review even the smallest changes). We have a lot of people who want to contribute - and even recently some people who seem to be eager to improve the Helm Chart in-general and make it more modern. I think this one is an opportunity to restart a little of our work on it. I think - similarly to the UI work - using more modern technology with better dev-env and "maintainability" might bring more maintainers and have some of "seasoned" maintainers spend a little more time on Chart refresh, knowing that it can improve the regular churn of issues and continuous effort on merging the PRs to existing approach. We tried to maintain the chart and there were a few efforts in the past to "restart" work on it - but it's never been long term and sustainable. And as they say - it's a definition of madness to try exactly the same thing several times and expect different results. I propose we end the madness and do it differently this time. J. [1] Remove Postgres discussion https://lists.apache.org/thread/mgh3342kht0452f3wdygl1sxb0nyrpz2 [2] Remove Posgress lazy consensus https://lists.apache.org/thread/3o4wj76pmd2m13jho7sxcbfkbgpv88h9 [3] Draft PR to use sqlite in our Helm Chart https://github.com/apache/airflow/pull/55962 [4] Zalando Postgres K8S Operator - https://github.com/zalando/postgres-operator [5] Zalando's Postgres K8S Operator quick - start https://opensource.zalando.com/postgres-operator/docs/quickstart.html [6] Example use cases of joining Kustomize and Helm https://medium.com/@brent.gruber77/the-power-of-kustomize-and-helm-5773d0f4d95e
