Hi Alexander, There has been some discussion around multi-tenancy this week actually and a call has been scheduled next week to discuss it further!
https://lists.apache.org/thread.html/r94991343d4fe536d58a4e5097edde378e7c0db76b95bf73008430820%40%3Cdev.airflow.apache.org%3E Just want to make sure you see that thread and can join if you are able. Jed On Fri, Apr 16, 2021 at 4:18 PM Alexander Ursu <[email protected]> wrote: > Hi, I was encouraged to send a message to this mailing list in a > discussion I opened on the Airflow GitHub here > https://github.com/apache/airflow/discussions/14936 > > It's been a while since I first opened it, but I got around to checking up > on it recently. > > I wanted to bring up this idea of multi-tenancy, at least in regards to > accessing certain variables or connections in DAGs. I'm currently running > Airflow at work and it's being shared by several small teams, and each > needs their own variables and connections. It's of course not ideal that > certain teams have access to potentially sensitive information unrelated to > what they should be concerned with. The only solution I came up, given that > all DAGs can read all the same variables, is to have separate Airflow > deployments for each team. While this would work, it makes management more > difficult, and it doesn't make much sense in my scenario anyways, where we > run a lot of things on one big Kubernetes cluster. No reason to have > multiple, highly-available Airflow setups if we can get away with just one. > > I'm curious if there is any plan for this up ahead, since our teams are > enjoying and investing more time into Airflow. > > If it helps provide context to my situation, I am also set up with the > HashiCorp Vault alternative secrets backend, which has been working well > for me. A suggestion in the discussion I linked was to override the > implementation for the backend and add the logic to restrict access, but I > think it would be very helpful to have it, and multi-tenancy in general, be > baked into Airflow by design. > > Thanks in advance for any help! > > - Alexander Ursu >
