Somehow, my last email does not show up on the Apache list web view (https://lists.apache.org/thread/ck8dsj5w82lvr0cpwr4wlptmydqwnsqc). Re-sending it to hopefully add it there. Sorry for the noise.
Hello, I tried to answer to all open questions: > If I understand correctly, the proposition in AIP-56 is to introduce an > interface as generic as possible so that any potential federated identity > protocol can be integrated in Airflow through User Manager (or as proposed in > this thread, AuthManager) You're 100% correct. This is the ultimate goal. Note on this one, based on the last feedbacks, it seems Auth manager makes more sense than User manager, which I agree. Thus, I updated the AIP to replace user manager by auth manager. > What do you think would be the context of "current user" provided by Airflow > to the implementation of AuthManager? When a user is logged in Airflow (through an auth manager), a Flask session will be set as it is today. This will be the context of "current user". In this session, you can find user information such as user ID or OIDC token (if the auth manager used use that kind of token). This is the way I envisioned so that the auth manager can retrieve the OIDC token in order to interact with the auth manager backend. For now we know that the FAB auth manager will need the user ID as context and the KeyCloak auth manager will need a OIDC token. However, this set of information must be extensible so that new auth manager which do not use OIDC token can be added in Airflow in the future. > Do you think it makes sense to document this in the AIP or leave decisions > for later? I think it makes total sense to add it to the AIP, which I will do, I hope it will make it clearer :) > is_authorized should include context so that it becomes possible the > determine whether a user has access to a resource based on for example the > location the user is at. Or in which environment. It is also useful for > auditing I see your points. I agree then, we should come up with a context of the is_authorized API. I have not started to work on that yet but will do in the following days. If you also want to give it a shot and come up with something, that would be very much appreciated :) > Particularly with security, I'd prefer clarity of how we intend to implement > this. If we consider an access_token equivalent to an OIDC access_token that > is fine, but let us be clear on what we base our design on instead of > seemingly inventing our own. I agree with you. Every time I refer to access token in the AIP, I meant OIDC access token, I'll add that clarification, thanks for the callout. However, this is very specific to each auth manager backend. In the case of KeyCloak, the access token I referred to is indeed an OIDC access token but it is impossible to assert this will be true for all auth managers implemented in the future. We need to keep in mind we cannot enforce third party auth services to implement the protocol we want but we need to be flexible/adaptable enough so we can integrate with majority of them. > At this moment, *any* DAG has full access to the database. That means if we > store user_id -> access_token any DAG can access any resource on behalf of > someone else. This means the AIrflow DB is one of the most insecure places to > store this kind of information. That is correct. Though this will be fixed in the future with AIP-44 (or a following on this one) if I am not mistaken. > post_login is not required if authentication is entirely handled by the > AuthManager. If we consider post_login to be equivalent to a callback method > from OIDC (again: on what do we base our security design?) then sure but then > let's call it as such. This is indeed a callback method from OIDC. Again, I did not want to be OIDC specific because we cannot assert all auth managers in the future will use OIDC protocol. That's why I came up with the name "post_login". I wanted something generic. Though, I am happy to change the name if that makes things easier to understand. As usual, thanks for your feedbacks! Vincent On 2023-05-19, 11:35 AM, "Jakub Kośmider" <kosmi...@google.com.inva <mailto:kosmi...@google.com.inva>LID> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hello, If I understand correctly, the proposition in AIP-56 is to introduce an interface as generic as possible so that any potential federated identity protocol can be integrated in Airflow through User Manager (or as proposed in this thread, AuthManager). In light of this, note that the methods in AuthManager in the AIP refer to the "current user", while the example of "is_authorized" used above is based only on specific resources and permissions provided as method parameters. What do you think would be the context of "current user" provided by Airflow to the implementation of AuthManager? Would it be the entire request or maybe just cookies/auth tokens associated with the request or something else? Do you think it makes sense to document this in the AIP or leave decisions for later? Best regards, Jakub On Fri, May 19, 2023 at 11:22 AM Jarek Potiuk <ja...@potiuk.com <mailto:ja...@potiuk.com>> wrote: > > Overall: I strongly suggest basing our security design on something > existing. For AuthN I like the OIDC scheme. Furthermore, I suggest looking > at https://flask-login.readthedocs.io/en/latest/ > <https://flask-login.readthedocs.io/en/latest/> to see how this is > handled > in Flask. > > Very much agree. Good suggestion Bolke. > > > On Fri, May 19, 2023 at 9:44 AM Bolke de Bruin <bdbr...@gmail.com > <mailto:bdbr...@gmail.com>> wrote: > > > Hi, > > > > On Wed, 17 May 2023 at 18:48, Jarek Potiuk <ja...@potiuk.com > > <mailto:ja...@potiuk.com>> wrote: > > > > > > > > > > > > > > I am very open to change the name of "User Manager" but I do not > think > > we > > > > should call it "Security Manager". "Security Manager" is the current > > name > > > > and I think this is a very bad name because: it is very generic and > > does > > > > not say much about what it is actually doing. By reading "Security > > > Manager" > > > > I would not think this is where users and roles are managed. The main > > > goal > > > > of AIP-56 is removing the whole user management part from core > Airflow > > to > > > > providers. Keeping the name "Security Manager" would, I think, leave > a > > > > legacy for all future user managers (or whatever name we use) on > > > > Flask-AppBuilder (FAB) and that's exactly what I want to avoid. I > chose > > > the > > > > name “User Manager” but I think it is simple, the main goal of this > new > > > > module is to manage users and access for them. > > > > > > > > > > > How about *AuthManager* ? (Auth standing for both Authentication and > > > Authorisation). I do agree "Security" is much bigger promise thant what > > we > > > are implementing here :). > > > > > > > > That works better in my mind. Note we are not just managing Users here: > it > > can be non personal accounts etc. > > > > > > > > > > > > > > > is_authorized should include context so that it becomes possible > the > > > > determine whether a user has access to a resource based on for > example > > > the > > > > location the user is at. Or in which environment. It is also useful > for > > > > auditing. Non implementing security managers can ignore it. > > > > > > > > I agree, that can be indeed a good improvement. Although I have to > say > > I > > > > am pretty confident the AIPs I defined in the API will change while > > > > implementing them and over time. Thus, another option would be to add > > > this > > > > context when needed. I do not think (I might be wrong) features such > as > > > " > > > > determine whether a user has access to a resource based on for > example > > > the > > > > location the user is at. Or in which environment" currently exist in > > > > Airflow. The goal of the AIP is not to implement new features but to > > > have > > > > the same features as we have today with this separation from core > > Airflow > > > > to providers. Again, I am not saying this is a wrong idea (and I > > actually > > > > think this is a good one), I just think if this feature does not > exist > > in > > > > Airflow today, we should add that later. > > > > > > > > > > I think Bolke is right: we should add information that we will pass > > context > > > and broadly what will be in. We can leave details for the > implementation > > > time but I think the context should be there. We should add what will > be > > > part of the context: > > > > > > * action > > > * dag_id > > > * task_id > > > * labels for the dag > > > * location in the DAG_FOLDER > > > * connection_id (when we change connection) > > > * variable-id (when we change variable) > > > * ... > > > > > > > > > This should be complete enough to make any kind of decisions. Probably > it > > > will also evolve over time in the future versions, but we need to have > > > something to start with that we think will be comprehensive and > complete > > > enough to implement usable AuthManager(s) (providing that the name will > > > stick). > > > > > > > > I think this should be extensible (= flexible). Because, I would add > > remote_addr, proxy_ips, roles, tag etc. > > > > For context, I would like to be able to setup policies like this: > > > > > https://towardsdatascience.com/integrating-trino-and-apache-ranger-b808f6b96ad8 > > <https://towardsdatascience.com/integrating-trino-and-apache-ranger-b808f6b96ad8> > > > > Or in open policy agent: > > > > # Financial department staff can view any customer dags > > > > allow = true { > > input.method = "GET" > > input.path = ["payments", customer_id] > > finance[input.user]} > > > > finance = {"john","mary","peter","vivian"} > > > > > > > > > > > > > > What is the need of post_login? Why would you want to store the > > access > > > > token? The security manager should, imho, do this on its own: it > > executes > > > > the login flow. > > > > > > > > The access token is needed to call any API from the user manager to > the > > > > third party backend (KeyCloak, ...). How would you call > is_authorized() > > > > without it? > > > > > > > > > > Yeah. we need to map whatever authentication information comes from > > outside > > > (usually a token coming from external authentication information) with > > > (likely?) flask session in Airflow and I **think** the best is that it > > > should be kept in AIrflow's DB the session in Airflow. I consider this > a > > > bit of an implementation detail, but maybe this needs a bit more > hashing > > ou > > > maybe others can also comment here.. > > > > > > > > Particularly with security, I'd prefer clarity of how we intend to > > implement this. If we consider an access_token equivalent to an OIDC > > access_token that is fine, but let us be clear on what we base our design > > on instead of seemingly inventing our own. > > > > At this moment, *any* DAG has full access to the database. That means if > we > > store user_id -> access_token any DAG can access any resource on behalf > of > > someone else. This means the AIrflow DB is one of the most insecure > places > > to store this kind of information. > > > > post_login is not required if authentication is entirely handled by the > > AuthManager. If we consider post_login to be equivalent to a callback > > method from OIDC (again: on what do we base our security design?) then > sure > > but then let's call it as such. > > > > Overall: I strongly suggest basing our security design on something > > existing. For AuthN I like the OIDC scheme. Furthermore, I suggest > looking > > at https://flask-login.readthedocs.io/en/latest/ > > <https://flask-login.readthedocs.io/en/latest/> to see how this is > > handled > > in Flask. > > > > > > > > > > > > > Why is get_tab_configuration special? We have this possibility in > > > > standard “plugins”. I suggest using that framework > > > > > > > > I am not familiar with plugins. I'll take a look at it, thanks for > > > > bringing it up. > > > > > > > > > > Yeah. The plugin mechanism is indeed foreseen for that - and the > minimal > > > FAB implementation should indeed implement plugin interface to add the > > > current Admin menu - that was what I had in mind by the current > > "Security" > > > tab - it can **just** be added using the current plugin mechanism. > > > > > > > > > > > > What is the need for “get_user_name”? Are we going to invent our > own > > > > Framework? Otherwise Flask might work? > > > > > > > > get_user_name is needed for the UI. On the top right corner of > Airflow > > > > console, you can see your name. This is why it is needed. The only > goal > > > of > > > > this API is to retrieve your user name from the user manager. That's > > it. > > > > Flask will then use this API to get the user name and display it in > the > > > UI. > > > > My experience in Flask is pretty limited so I might miss something > but > > > the > > > > way I see it is, you need a way to retrieve your user name from > > whatever > > > > user manager you are using. This is the way. > > > > > > > > Please have a look at flask login ( > > https://flask-login.readthedocs.io/en/latest/ > > <https://flask-login.readthedocs.io/en/latest/>) or even base your design > > upon Flask-Login. This ensures that the right information is available > > across all components in a way that Flask components understand. The > > question then becomes how does the DAG subsystem (which does not rely on > > Flask) deal with this. It might need a kind of wrapper possibly. > > > > > > > > > > > > > > > > > I hope that helps, > > > > Vincent > > > > > > > > On 2023-05-15, 5:30 PM, "Bolke de Bruin" <bdbr...@gmail.com > > > > <mailto:bdbr...@gmail.com> > <mailto: > > > > bdbr...@gmail.com <mailto:bdbr...@gmail.com>>> wrote: > > > > > > > > > > > > > > > > I think think the name “User Manager” is a misnomer and it should > just > > be > > > > called “Security Manager”. It was confusing the hell out of me when > > > reading > > > > the AIP and I was convinced you wanted to fully manage e.g. keycloak > > > > provided users inside Airflow (CRUD). What you are doing but aren’t > > > exactly > > > > saying: > > > > > > > > > > > > 1 - Define API that a Security Plugin should adhere to (AuthZ/AuthN) > > > > 2 - Have a default Security Plugin based on FAB > > > > > > > > > > > > Points of improvement: > > > > > > > > > > > > * is_authorized should include context so that it becomes possible > the > > > > determine whether a user has access to a resource based on for > example > > > the > > > > location the user is at. Or in which environment. It is also useful > for > > > > auditing. Non implementing security managers can ignore it. > > > > * I’d prefer some kind of ’standards’ based API. Like using Flask’s > way > > > of > > > > registering endpoints or something along those lines. See also > > questions > > > > below. > > > > > > > > > > > > Questions: > > > > > > > > > > > > * What is the need of post_login? Why would you want to store the > > access > > > > token? The security manager should, imho, do this on its own: it > > executes > > > > the login flow. > > > > * Why is get_tab_configuration special? We have this possibility in > > > > standard “plugins”. I suggest using that framework > > > > * What is the need for “get_user_name”? Are we going to invent our > own > > > > Framework? Otherwise Flask might work? > > > > > > > > > > > > > > > > > > > > Cheers > > > > Bolke > > > > > > > > > > > > On 9 May 2023 at 19:55:32, Beck, Vincent (vincb...@amazon.com.inva > > > > <mailto:vincb...@amazon.com.inva> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva>>lid) > > > > wrote: > > > > > > > > > > > > Hello all, > > > > > > > > > > > > I would like to start voting on this one so please add your comments > on > > > the > > > > API or by replying here if you're interested, you still have time to > do > > > it > > > > :) > > > > > > > > > > > > AIP: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > > > > > > > > > > Vincent > > > > > > > > > > > > On 2023-05-03, 8:03 AM, "Jarek Potiuk" <ja...@potiuk.com > > > > <mailto:ja...@potiuk.com> <mailto: > > > > ja...@potiuk.com <mailto:ja...@potiuk.com>> <mailto: > > > > ja...@potiuk.com <mailto:ja...@potiuk.com> <mailto:ja...@potiuk.com > > > > <mailto:ja...@potiuk.com>>>> wrote: > > > > > > > > > > > > > > > > > > > > CAUTION: This email originated from outside of the organization. Do > not > > > > click links or open attachments unless you can confirm the sender and > > > know > > > > the content is safe. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Alsoo to add (and make more concrete) to what it could mean at the > > > > implementation level. > > > > > > > > > > > > > > > > > > > > It means the for our API endpoints and views is that instead of > > > > @auth.hasaccess (for example here > > > > > https://github.com/apache/airflow/blob/main/airflow/www/views.py#L680 > <https://github.com/apache/airflow/blob/main/airflow/www/views.py#L680> > > < > > > > > https://github.com/apache/airflow/blob/main/airflow/www/views.py#L680> > <https://github.com/apache/airflow/blob/main/airflow/www/views.py#L680>> > > < > > > > > https://github.com/apache/airflow/blob/main/airflow/www/views.py#L680> > <https://github.com/apache/airflow/blob/main/airflow/www/views.py#L680>> > > < > > > > > > https://github.com/apache/airflow/blob/main/airflow/www/views.py#L680> > > <https://github.com/apache/airflow/blob/main/airflow/www/views.py#L680&gt> > > > > ;>) > > > > where it check for permissions assigned to current role, airflow > would > > > > entirely delegate the decision to the 'user management plugin'. > > > > > > > > > > > > > > > > > > > > We would simply delegate the decision to whatever plugin we have > > > > configured. In case FAB pługin is used, it would do exactly the same > > > checks > > > > as today (and FAB pługin would contribute the 'Security' menu the > same > > > way > > > > other plugins can contribute their own (as well as CLI commands and > > > > user/role APIs - we would have to add a mechanism where plugins could > > > > contribute CLIs and APIs on their own, but that should be easy). So > if > > > you > > > > have FAB pługin chosen as default, nothing changes for the users > > > comparing > > > > to today. It would not require bumping Airflow Major version because > it > > > > would be backwards compatible. However similarly as providers it > would > > > move > > > > the code of FAB plugins and roles out of the core of Airflow. > > > > > > > > > > > > > > > > > > > > We could have keycloak 'reference' implementation where we would come > > up > > > > with very opinionated approach where we have simple admin/user roles > > > > (matching roughly default set of roles and permissions we currently > > have > > > > with FAB). The big difference here will be that the user/role APIs, > > > views, > > > > CLIs will be gone entirely. And we could add there the concept of > > > > 'tenants' or rather 'teams' giving access to 'dag groups' and > implement > > > > decisions ob authorisation based on 'group' assignment for dag and > > user. > > > > This is the main goal of this change to make it possible and easy > > without > > > > having to mess with FAB permission and Role model which is completely > > not > > > > suitable to manage 'groups of resources of the same type' with > separate > > > set > > > > of permissions (think having few separate teams or tenants - each of > > them > > > > wanting access to separate set of DAGs). > > > > > > > > > > > > > > > > > > > > Currently if an SSO/enterprise user wants to plug in their > authn/authz > > > with > > > > group access - they have to deal with all that including cumbersome > > > syncing > > > > of permissions for individual DAGs matching them to users separately > - > > > just > > > > to satisfy FAB permission model lacking this feature and flexibility. > > > > > > > > > > > > > > > > > > > > No more need for that if we go the direction described here. > > > > > > > > > > > > > > > > > > > > And that's basically it for what we would implement in Airflow > > > > out-of-the-box: FAB minimal implementation + be keycloak opinionated > > > > reference implementation with simple teams/tenant > <https://teams.googleplex.com/u/tenant> > <https://teams.googleplex.com/u/tenant>> support > > > > > > > > > > > > > > > > > > > > This also makes it super flexible for those who want more flexibility > > and > > > > do not want to use our opinionated keycloak plugin. They will be > > entirely > > > > free to implement their own ways - as flexible or as opinionated they > > > want > > > > - without having to messwith permission syncing and the role model of > > > FAB. > > > > Nor keycloak integration. They could go with their own. > > > > > > > > > > > > > > > > > > > > And with that - sky is the limit. The decision to access particular > dag > > > > could be made based on folder where the dag is from, id of dag, or > dag > > > > label or whatever other parameter (as long as we pass all that > > > information > > > > to the plugin). You could even - if you really want - make a decision > > to > > > > allow certain users (and only them) to be able to clear individual > > tasks, > > > > or tasks where specific operator types are used. For example you > could > > > > decide if you really want to only allow th google cloud admin users > to > > > > clear tasks that are operators coming from Google Provider (why > not?). > > Or > > > > only allow to that in certain time windows, or .... whatever. > > > > > > > > > > > > > > > > > > > > It's extremely flexible and powerful - but we will not have to deal > > with > > > > all the complexity in the community - we could delegate it to those > who > > > > mange the deployment or manage deployment of Airflow as a service and > > > > decide they want to integrate it with the service authorisation that > is > > > > already there - think direct integratio with IAM ON AWS or GCP. And > if > > we > > > > just provide a reference, opinionated implementation for keycloak > with > > > > tenant support and legacy/minimal FAB as optional plugins, that > should > > > also > > > > 'good enough' for those who do not want to implement their own plugin > > > > implementation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > J. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > śr., 3 maj 2023, 12:34 użytkownik Jarek Potiuk <ja...@potiuk.com > > > > <mailto:ja...@potiuk.com> > > > <mailto: > > > > ja...@potiuk.com <mailto:ja...@potiuk.com>> <mailto: > > > > ja...@potiuk.com <mailto:ja...@potiuk.com> <mailto:ja...@potiuk.com > > > > <mailto:ja...@potiuk.com>>>> napisał: > > > > > > > > > > > > > > > > > > > > > > I don't think we can do that: we still need to consider users > when > > > they > > > > > are just getting started out with Airflow -- if we only cater to > > giant > > > > > enterprises with SSO and a central directory we remove our ability > > for > > > > > individual or small teams/companies > <https://teams.googleplex.com/u/companies> > <https://teams.googleplex.com/u/companies>> to get started with Airflow. > > > > > > > > > > Let me clarify this because I think we are talking about the same > > > thing. > > > > > > > > > > I **think** having the "minimal FAB user manager implementation" as > > the > > > > > "default" option when you install Airflow serves the purpose you > > > mention > > > > > Ash - if we do it the way that User/Role management is coming > > together > > > > with > > > > > FAB implementation (enabled by default for the initial > installation). > > > > What > > > > > I am saying is that it should not be part of the "core airflow" in > > the > > > > > sense that one could live without it (specifically in those big > > > > SSO/Central > > > > > directory case). > > > > > In my vision those two cases are separated, but the "minimal > > > > > implementation" when enabled should act and behave "as if" nothing > > > > changed > > > > > compared to the current way it looks and behaves. > > > > > > > > > > J. > > > > > > > > > > > > > > > On Wed, May 3, 2023 at 12:06 PM Ash Berlin-Taylor <a...@apache.org > > > > > <mailto:a...@apache.org> > > > > <mailto:a...@apache.org <mailto:a...@apache.org>> <mailto: > > > > a...@apache.org <mailto:a...@apache.org> <mailto:a...@apache.org > > > > <mailto:a...@apache.org>>>> wrote: > > > > > > > > > >> > - I am fine with (and actually I like it better) removing User > and > > > > Role > > > > >> related Rest APIs and CLI commands from Airflow. That will indeed > > make > > > > the > > > > >> AIP way simpler and shorter > > > > >> > > > > >> I don't think we can do that: we still need to consider users when > > > they > > > > >> are just getting started out with Airflow -- if we only cater to > > giant > > > > >> enterprises with SSO and a central directory we remove our ability > > for > > > > >> individual or small teams/companies > <https://teams.googleplex.com/u/companies> > <https://teams.googleplex.com/u/companies>> to get started with Airflow. > > > > >> On May 2 2023, at 8:23 pm, Beck, Vincent <vincb...@amazon.com.inva > > > > >> <mailto:vincb...@amazon.com.inva> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva>> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > >>LID> > > > > >> wrote: > > > > >> > Thank you Jarek and Bolke for taking time to read and provide > > > feedback > > > > >> to the AIP, much appreciated! Multiple points here: > > > > >> > > > > > >> > - I am fine with (and actually I like it better) removing User > and > > > > Role > > > > >> related Rest APIs and CLI commands from Airflow. That will indeed > > make > > > > the > > > > >> AIP way simpler and shorter. As I explained in the AIP comments, I > > am > > > > >> suggesting though to leave an option to user managers to define > > > > additional > > > > >> REST APIs and CLI commands if they need. By default no additional > > REST > > > > API > > > > >> and CLI command would be defined but user managers would have the > > > option > > > > to > > > > >> override it. FAB user manager would be an example of user manager > > > > >> overriding such methods. In any case, if additional Rest API or > CLI > > > > command > > > > >> is defined, this would be done in user managers (as opposed to > core > > > > >> Airflow). > > > > >> > - "the whole "Security" menu should be GONE completely". > > > > >> > > I agree with this when the user manager is not the FAB one, > > > however > > > > >> we need to introduce a new one so Admins can easily go the > external > > > user > > > > >> manager console (e.g. KeyCloak, ...). This new tab would only be a > > > > >> link/redirect to the external console. > > > > >> > > > > > >> > - "All those components (and code related to those) should be > > moved > > > to > > > > >> FAB minimalistic user manager (and removed from Airflow Core)". > > > > >> > > 100%. This is the main principle of this AIP, anything related > > to > > > > >> user management will be moved to user managers. > > > > >> > > > > > >> > - "AuthN and AuthZ should be completely removed from Airflow". > > > > >> > > 100%. > > > > >> > > > > > >> > - "So for a nice out-of-the-box experience (pip install > > > > apache-airflow) > > > > >> a lightweight User Manager could exist as a side project and have > > some > > > > >> menus via the plugin system until something better comes along.". > > > > >> > > I agree with you on the long term. Though, as a first > iteration > > I > > > > >> think it is better to define a user manager using FAB as default > > user > > > > >> manager. The benefit of doing so is to have a backward compatible > > > > >> transition between before AIP and after AIP. This also simplifies > > the > > > > AIP > > > > >> since it is easier to move files from core Airflow to user > managers > > > than > > > > >> creating something new. Creating a new security model would be a > > > > challenge > > > > >> too. However, once the AIP implemented and merged, we can easily > > > replace > > > > >> this out-of-the-box FAB user manager by something "better" if the > > > > communty > > > > >> thinks this is the right way to go. > > > > >> > > > > > >> > As always, feel free to give me your thoughts. > > > > >> > Vincent > > > > >> > On 2023-04-25, 2:00 PM, "Bolke de Bruin" <bdbr...@gmail.com > > > > >> > <mailto:bdbr...@gmail.com> > > > <mailto: > > > > bdbr...@gmail.com <mailto:bdbr...@gmail.com>> <mailto: > > > > bdbr...@gmail.com <mailto:bdbr...@gmail.com> <mailto:bdbr...@gmail.com > > > > <mailto:bdbr...@gmail.com>>> <mailto: > > > > >> bdbr...@gmail.com <mailto:bdbr...@gmail.com> > > > > >> <mailto:bdbr...@gmail.com <mailto:bdbr...@gmail.com>> <mailto: > > > bdbr...@gmail.com <mailto:bdbr...@gmail.com> > > > > <mailto:bdbr...@gmail.com <mailto:bdbr...@gmail.com>>>>> wrote: > > > > >> > I'm not sure if I read the AIP correctly. I think I concur with > > > Jarek > > > > >> here. > > > > >> > > > > > >> > AuthN and AuthZ should be completely removed from Airflow. > Access > > > > >> > management should be outsourced to something like Open Policy > > Agent. > > > > In > > > > >> > other words there should be no user management at all within > > > Airflow. > > > > >> This > > > > >> > removes the need to "sync" users (and thus become outdated) and > > > > reduces > > > > >> the > > > > >> > attack surface drastically. So for a nice out-of-the-box > > experience > > > > (pip > > > > >> > install apache-airflow) a lightweight User Manager could exist > as > > a > > > > side > > > > >> > project and have some menus via the plugin system until > something > > > > better > > > > >> > comes along. Keycloak is great, a bit heavy for us to package > > > > >> > unfortunately. > > > > >> > > > > > >> > > > > > >> > FAB's security model is very non-standard and requires > 'unnatural' > > > > >> mappings > > > > >> > in an enterprise context. It has served us well, but I would say > > > good > > > > >> > riddance. > > > > >> > > > > > >> > > > > > >> > Bolke > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > Op ma 24 apr 2023 om 08:39 schreef Jarek Potiuk < > ja...@potiuk.com <mailto:ja...@potiuk.com> > > > > <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com>> > > > > <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com> > > > > <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com>>> > > > > >> <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com> > > > > >> <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com>> <mailto: > > > > ja...@potiuk.com <mailto:ja...@potiuk.com> <mailto:ja...@potiuk.com > > > > <mailto:ja...@potiuk.com>>>>>: > > > > >> > > > > > >> > > I had a look and I liked what I saw for how we should > integrate > > > the > > > > >> new > > > > >> > > user manager for getting and checking the access :). > > > > >> > > > > > > >> > > But I think the AIP proposal is not bold enough when it comes > to > > > the > > > > >> user > > > > >> > > management part. This is one big comment - we should simplify > it > > > > even > > > > >> > > further and cut more deeply. > > > > >> > > > > > > >> > > One of the big changes I think is super important with > > extensible > > > > user > > > > >> > > management is that we should delegate even more to the > external > > > > >> systems > > > > >> > > managing users. The AIP attempts to map the current API/CLI/UI > > for > > > > >> managing > > > > >> > > the users and roles to the new external user management > > services - > > > > >> but it > > > > >> > > is IMHO absolutely not needed. > > > > >> > > > > > > >> > > As I see the extensible user management is that when you > choose > > > > >> something > > > > >> > > else than the legacy FAB minimalist user management. Airflow > > > should > > > > >> become > > > > >> > > just "user" of that user/role information managed elsewhere. > It > > > > >> should not > > > > >> > > provide ANY option to see and manage the users or roles, it > > should > > > > >> merely > > > > >> > > read the information and give access to the users for > > appropriate > > > > >> actions, > > > > >> > > but then all the role/user management should happen outside of > > > > >> Airflow - in > > > > >> > > the system that is dedicated to manage those. > > > > >> > > > > > > >> > > IMHO - when other than the "non FAB minimalistic user manager" > > > > system > > > > >> is > > > > >> > > used. Airflow should just not have a number of functionalities > > at > > > > all: > > > > >> > > > > > > >> > > * not have "users" or "roles" CLI groups at all - they should > be > > > > >> missing > > > > >> > > * the > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User> > > > > < > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User> > > > > > > > > > < > > > > > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User> > > > > > > > > < > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User> > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User&gt> > > > > ;> > > > > > > > > > > > > >> < > > > > >> > > > > > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User> > > > > < > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User> > > > > > > > > > < > > > > > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User> > > > > > > > > < > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User> > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/User&gt> > > > > ;> > > > > > > > > > > > > >> > > > > > >> > > and > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role> > > > > < > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role> > > > > > > > > > < > > > > > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role> > > > > > > > > < > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role> > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role&gt> > > > > ;> > > > > > > > > > > > > >> < > > > > >> > > > > > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role> > > > > < > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role> > > > > > > > > > < > > > > > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role> > > > > > > > > < > > > > > > > > > > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role> > > <https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#tag/Role&gt> > > > > ;> > > > > > > > > > > > > >> > > > > > >> > > should return 404 NOT FOUND > > > > >> > > * the whole "Security" menu should be GONE completely. > > > > >> > > > > > > >> > > All those components (and code related to those) should be > moved > > > to > > > > >> FAB > > > > >> > > minimalistic user manager (and removed from Airflow Core). > > > > Eventually > > > > >> maybe > > > > >> > > even moved out of the Airflow repo altogether and moved to a > FAB > > > > user > > > > >> > > manager add-on to airflow. > > > > >> > > > > > > >> > > This might seem a drastic change, but in fact - it's not > drastic > > > at > > > > >> all. It > > > > >> > > reflects what many of the Airflow users are doing now - they > > never > > > > >> access > > > > >> > > Security/Admin/Roles. Once they integrate with their LDAP or > > other > > > > >> > > corporate directory, the Security UI, CLI and REST API are > > > > essentially > > > > >> > > useless (and not used). > > > > >> > > > > > > >> > > And it is precisely what we want to achieve - we want to > > delegate > > > > the > > > > >> > > user/role management completely out of Airflow. In fact - > > getting > > > > rid > > > > >> of > > > > >> > > those is the only reason we want to implement the change. If > we > > > > >> **just** > > > > >> > > replace the current FAB and keep all the complexity of > managing > > > the > > > > >> users > > > > >> > > /roles in Airflow, I'd say we have not achieved anything but > > > adding > > > > >> > > complexity. > > > > >> > > > > > > >> > > And - eventually - it makes the AIP-56 way *smaller*. We just > > need > > > > to > > > > >> make > > > > >> > > sure that all the stuff for user management (including REST > API, > > > CLI > > > > >> and > > > > >> > > the UI) is moved to the "FAB minimalistic user manager" (and > add > > > > ways > > > > >> to > > > > >> > > plug-it in the core - possibly using existing plugins > mechanisms > > > > where > > > > >> > > needed). We do not need to define new API for user/role > > > management. > > > > >> We only > > > > >> > > need to describe the way how to check if the user has > permission > > > to > > > > do > > > > >> > > stuff. > > > > >> > > > > > > >> > > J. > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > On Mon, Apr 17, 2023 at 12:34 PM Jarek Potiuk < > ja...@potiuk.com <mailto:ja...@potiuk.com> > > > > <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com>> > > > > <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com> > > > > <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com>>> > > > > >> <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com> > > > > >> <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com>> <mailto: > > > > ja...@potiuk.com <mailto:ja...@potiuk.com> <mailto:ja...@potiuk.com > > > > <mailto:ja...@potiuk.com>>>>> wrote: > > > > >> > > > > > > >> > > > I promise to provide my feedback by the end of the week, > sorry > > > for > > > > >> > > > delaying it, but we had some 2.6 preparation, branching, > > feature > > > > >> flags > > > > >> > > etc. > > > > >> > > > also for AIP-44 and I am getting back on track with that one > > > soon. > > > > >> > > > > > > > >> > > > On Fri, Mar 31, 2023 at 9:35 AM Mehta, Shubham > > > > >> <shu...@amazon.com.inva <mailto:shu...@amazon.com.inva> > > > > >> <mailto:shu...@amazon.com.inva <mailto:shu...@amazon.com.inva>> > > > > >> <mailto: > > > > shu...@amazon.com.inva <mailto:shu...@amazon.com.inva> > > > > <mailto:shu...@amazon.com.inva <mailto:shu...@amazon.com.inva>>> > > > > <mailto: > > > > shu...@amazon.com.inva <mailto:shu...@amazon.com.inva> > > > > <mailto:shu...@amazon.com.inva <mailto:shu...@amazon.com.inva>> <mailto: > > > > shu...@amazon.com.inva <mailto:shu...@amazon.com.inva> > > > > <mailto:shu...@amazon.com.inva <mailto:shu...@amazon.com.inva>>>>lid > > > > >> > > > > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > >> Bumping this up for feedback! > > > > >> > > >> > > > > >> > > >> @Vikram, @Kaxil, et al. - I understand that you're quite > > busy, > > > > but > > > > >> we > > > > >> > > >> would greatly appreciate your feedback here. Your inputs > > could > > > > >> help us > > > > >> > > >> improve the proposal and move forward with multi-tenancy > > work. > > > > >> > > >> > > > > >> > > >> Cheers, > > > > >> > > >> Shubham > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> On 2023-03-28, 2:05 PM, "Beck, Vincent" > > > <vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva>> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva>>> > > > > >> <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > >> <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva>>>> > > > > >> > > >> <mailto:vincb...@amazon.com.inva > > > > >> > > >> <mailto:vincb...@amazon.com.inva> <mailto: > > > > vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva>> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > <mailto: > > > > vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva>>> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva>> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > > <mailto:vincb...@amazon.com.inva <mailto:vincb...@amazon.com.inva> > > > >>>>LID> > > > > >> wrote: > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> CAUTION: This email originated from outside of the > > > organization. > > > > >> Do not > > > > >> > > >> click links or open attachments unless you can confirm the > > > sender > > > > >> and > > > > >> > > know > > > > >> > > >> the content is safe. > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> Hi all, > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> I would like to get your feedback on an AIP I wrote: > > > "Extensible > > > > >> user > > > > >> > > >> management". This AIP is a follow-up on a discussion we had > > in > > > > this > > > > >> > > email > > > > >> > > >> list on the multi tenancy topic. I decided to create a new > > > email > > > > >> thread > > > > >> > > >> because I feel the topic had diverged a bit from the > original > > > > topic > > > > >> > > (multi > > > > >> > > >> tenancy). > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> As a summary, the purpose of this AIP is to extract the > user > > > > >> management > > > > >> > > >> from core Airflow by introducing a user manager interface > in > > > the > > > > >> core > > > > >> > > >> Airflow that can be extended by any provider package who > want > > > to > > > > >> support > > > > >> > > >> user management natively. As a result, if this AIP gets > > > approved > > > > >> and go > > > > >> > > >> through, the multi tenancy feature will be implemented as a > > > > second > > > > >> step > > > > >> > > in > > > > >> > > >> a new user manager and not in Airflow directly. > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> As always, feel very free to give your opinion on this > email > > > > >> thread or > > > > >> > > on > > > > >> > > >> the AIP by adding comments. > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> References: > > > > >> > > >> - AIP: > > > > >> > > >> > > > > >> > > > > > > >> > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > > < > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management&gt> > > > > ;> > > > > > > > > > > > > >> < > > > > >> > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > > < > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management&gt> > > > > ;> > > > > > > > > > > > > >> > > > > > >> > > >> < > > > > >> > > >> > > > > >> > > > > > > >> > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > > < > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management&gt> > > > > ;> > > > > > > > > > > > > >> < > > > > >> > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > > < > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > > > > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management> > > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management&gt> > > > > ;> > > > > > > > > > > > > >> > > > > > >> > > >> > > > > > >> > > >> - Initial email list discussion: > > > > >> > > >> > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61 > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61>> < > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61>> < > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt> > ;> > > < > > > > >> https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > >> <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61>> > < > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt> > ;> > > < > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt> > ;> > > < > > > > > > > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt;> > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&amp;gt;&gt> > > > ;> > > > > < > > > > >> > > >> > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61>> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt> > ;> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt> > ;> > > < > > > > > > > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt;> > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&amp;gt;&gt> > > > > ;> > > > > >> < > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61> > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt> > > > ;> > > > > < > > > > > > > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt;> > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&amp;gt;&gt> > > > ;> > > > > < > > > > > > > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&gt;> > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&amp;gt;&gt> > > > ;> > > > > < > > > > > > > > > > https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&amp;gt;&gt;> > > <https://lists.apache.org/thread/lf585xvvxpqtzhfyc6drzrf3rmg37w61&amp;amp;gt;&amp;gt;&gt> > > > > ;> > > > > > > > > > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> Vincent > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > -- > > > > >> > > > > > >> > -- > > > > >> > Bolke de Bruin > > > > >> > bdbr...@gmail.com <mailto:bdbr...@gmail.com> > > > > >> > <mailto:bdbr...@gmail.com <mailto:bdbr...@gmail.com>> <mailto: > > > > bdbr...@gmail.com <mailto:bdbr...@gmail.com> <mailto:bdbr...@gmail.com > > > > <mailto:bdbr...@gmail.com>>> <mailto: > > bdbr...@gmail.com <mailto:bdbr...@gmail.com> > > > > <mailto:bdbr...@gmail.com <mailto:bdbr...@gmail.com>> > > > > <mailto:bdbr...@gmail.com <mailto:bdbr...@gmail.com> > > > > <mailto:bdbr...@gmail.com <mailto:bdbr...@gmail.com>>>> > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > --------------------------------------------------------------------- > > > > >> > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > > >> > <mailto:dev-unsubscr...@airflow.apache.org> > > <mailto: > > > > dev-unsubscr...@airflow.apache.org > > > > <mailto:dev-unsubscr...@airflow.apache.org>> <mailto: > > > > dev-unsubscr...@airflow.apache.org > > > > <mailto:dev-unsubscr...@airflow.apache.org> <mailto: > > > > dev-unsubscr...@airflow.apache.org > > > > <mailto:dev-unsubscr...@airflow.apache.org>>> > > > > >> > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > >> > <mailto:dev-h...@airflow.apache.org> > > > <mailto: > > > > dev-h...@airflow.apache.org <mailto:dev-h...@airflow.apache.org>> > > > > <mailto: > > > > dev-h...@airflow.apache.org <mailto:dev-h...@airflow.apache.org> > > > > <mailto:dev-h...@airflow.apache.org > > > > <mailto:dev-h...@airflow.apache.org>>> > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > > <mailto:dev-unsubscr...@airflow.apache.org> <mailto: > > > > dev-unsubscr...@airflow.apache.org > > > > <mailto:dev-unsubscr...@airflow.apache.org>> > > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > <mailto:dev-h...@airflow.apache.org> > <mailto: > > > > dev-h...@airflow.apache.org <mailto:dev-h...@airflow.apache.org>> > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > -- > > Bolke de Bruin > > bdbr...@gmail.com <mailto:bdbr...@gmail.com> > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
