+1 binding Thank you Jarek for being very patient and working through all the twists and turns on this AIP.
I do believe that the changes are all positive and I really like the evolution towards a more focused multi-team target. This has become an uber-AIP, which is now building on top of AIPs such as the AIP-72 Task Execution Interface, and AIP-73 Data Awareness (only referencing other in-flight AIPs at this time, rather than already completed ones). That's fantastic. I think this may require the "Auth manager" work to be done as well, which you brought up in a different thread. I noticed that you kept it out of scope of this AIP, which I think is right, but may be needed for the overall use case. Based on the dependencies, I also agree that this would be an Airflow 3.1 target, but having a future target use case in mind is also a good thing for the developers of the underlying AIPs to not preclude those scenarios and to flag additional work needed for this use case during implementation. On Fri, Aug 2, 2024 at 1:13 PM Vincent Beck <vincb...@apache.org> wrote: > +1 binding. Very good demonstration of perseverance on this one :) > > On 2024/08/02 20:02:00 "Bishundeo, Rajeshwar" wrote: > > +1 non-binding. > > > > Great job to all the folks getting this feature 1-step closer to reality! > > > > -- Rajesh > > > > > > > > > > > > > > On 2024-08-02, 3:57 PM, "Mehta, Shubham" <shu...@amazon.com.inva > <mailto:shu...@amazon.com.inva>LID> wrote: > > > > > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > > > > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous > ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas > certain que le contenu ne présente aucun risque. > > > > > > > > > > > > > > +1 non-binding. > > > > > > Thanks, Jarek, for continuing to evolve this piece of work based on the > hundreds of comments you received from everyone. While there are > dependencies on other AIPs, I'd love to see this implemented in Airflow > 3.0. I hope other AIP authors also consider this as a reality and plan for > a multi-team setup during implementation. Ideally, users migrating to > Airflow 3 should be able to plan for an optimal team structure and access > permissions from the outset. The AWS team will definitely help in any way > possible to ship this on time. Looking forward to the execution! > > > > > > Shubham > > > > > > On 2024-08-02, 12:46 PM, "Kaxil Naik" <kaxiln...@gmail.com <mailto: > kaxiln...@gmail.com> <mailto:kaxiln...@gmail.com <mailto: > kaxiln...@gmail.com>>> wrote: > > > > > > > > > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > > > > > > > > > > > > > > > > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous > ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas > certain que le contenu ne présente aucun risque. > > > > > > > > > > > > > > > > > > > > > > > > > > +1 binding > > > > > > > > > > Thank you for persisting with the questions & changes, great job. > > > > > > > > > > Realistically, though, I think this would happen in 3.1 due to > > dependencies, as you mentioned in the email. > > > > > > > > > > Looking forward. > > > > > > > > > > Regards, > > Kaxil > > > > > > > > > > On Fri, 2 Aug 2024 at 20:30, Scheffler Jens (XC-AS/EAE-ADA-T) > > <jens.scheff...@de.bosch.com.inva <mailto: > jens.scheff...@de.bosch.com.inva> <mailto:jens.scheff...@de.bosch.com.inva > <mailto:jens.scheff...@de.bosch.com.inva>>lid> wrote: > > > > > > > > > > > +1 binding. > > > > > > Many thougths and discussions went into this AIP. Also User survery > showed > > > a strong demand for this - same we have also in our house. A multi team > > > setup is eagerly needed. > > > Nevertheless the complexity and missing turn-key delivery will be a > > > challenge. I assume it needs to evolve. > > > Besides a bit sceptic words the possibilities rule-out the additional > > > complexity. I think we should commit making it - especially not with > > > options to have some more breaking changes in AF3 is the right point to > > > plan all meede structural changes in! Thanks Jarek for driving this! > > > > > > Sent from Outlook for iOS<https://aka.ms/o0ukef> < > https://aka.ms/o0ukef>> <https://aka.ms/o0ukef>> < > https://aka.ms/o0ukef&gt;>> > > > ________________________________ > > > From: Jarek Potiuk <ja...@potiuk.com <mailto:ja...@potiuk.com> > <mailto:ja...@potiuk.com <mailto:ja...@potiuk.com>>> > > > Sent: Thursday, August 1, 2024 10:40:16 PM > > > To: dev@airflow.apache.org <mailto:dev@airflow.apache.org> <mailto: > dev@airflow.apache.org <mailto:dev@airflow.apache.org>> < > dev@airflow.apache.org <mailto:dev@airflow.apache.org> <mailto: > dev@airflow.apache.org <mailto:dev@airflow.apache.org>>> > > > Subject: Re: [VOTE] AIP-67 - Multi-team deplyment of Airflow Components > > > > > > Voting ends a week from now - i.e Thursday 8th of August 2023 > (midnight). > > > > > > On Thu, Aug 1, 2024 at 10:13 PM Jarek Potiuk <ja...@potiuk.com > <mailto:ja...@potiuk.com> <mailto:ja...@potiuk.com <mailto: > ja...@potiuk.com>>> wrote: > > > > > > > Hello here, > > > > > > > > I reviewed and - I hope - addressed all the comments so far. One most > > > > recent change in the doc was to allow a connection/variable to > belong to > > > > multiple teams. Also I mentioned a potential future change that will > > > allow > > > > us to run "per-team" schedulers - leaving the UI/DB schema as the > main > > > > components that are shared between teams. > > > > > > > > I am also fine if this one's delivery will slip to 3.1 - because it > has > > > > many dependencies on other AIPs - and also makes some assumptions on > > > > the other AIP-s scope when they are not yet fully fleshed out. So > some > > > > implementation details **might** change. the general approach is > going to > > > > stay: > > > > > > > > * not targeting manageable "mutliple tenants" - but more of a > small-ish > > > > set of teams within the same organisation that want to maintain > separate > > > > environments and isolated execution environment > > > > > > > > * common UI and scheduler (potentially per-team schedule) as the main > > > > "common" piece > > > > > > > > * each team being able to manage their team separately - to the > extent of > > > > parsing DAGs and executing tasks > > > > > > > > The AIP proposal with updates: > > > > > > > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAIRFLOW%2FAIP-67%2BMulti-team%2Bdeployment%2Bof%2BAirflow%2Bcomponents&data=05%7C02%7CJens.Scheffler%40de.bosch.com%7C3f883fcfd40c41a9724608dcb26a3538%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638581416424218968%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=LXRWcx%2B0xBpi6jeIb%2BJd4v72y8GojvYhbonctkx1sZ8%3D&reserved=0 > < > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAIRFLOW%2FAIP-67%2BMulti-team%2Bdeployment%2Bof%2BAirflow%2Bcomponents&data=05%7C02%7CJens.Scheffler%40de.bosch.com%7C3f883fcfd40c41a9724608dcb26a3538%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638581416424218968%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=LXRWcx%2B0xBpi6jeIb%2BJd4v72y8GojvYhbonctkx1sZ8%3D&reserved=0> > < > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAIRFLOW%2FAIP-67%2BMulti-team%2Bdeployment%2Bof%2BAirflow%2Bcomponents&data=05%7C02%7CJens.Scheffler%40de.bosch.com%7C3f883fcfd40c41a9724608dcb26a3538%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638581416424218968%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=LXRWcx%2B0xBpi6jeIb%2BJd4v72y8GojvYhbonctkx1sZ8%3D&reserved=0> > < > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAIRFLOW%2FAIP-67%2BMulti-team%2Bdeployment%2Bof%2BAirflow%2Bcomponents&amp;data=05%7C02%7CJens.Scheffler%40de.bosch.com%7C3f883fcfd40c41a9724608dcb26a3538%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638581416424218968%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&amp;sdata=LXRWcx%2B0xBpi6jeIb%2BJd4v72y8GojvYhbonctkx1sZ8%3D&amp;reserved=0> > ;> > > > < > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-67+Multi-team+deployment+of+Airflow+components > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-67+Multi-team+deployment+of+Airflow+components> > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-67+Multi-team+deployment+of+Airflow+components> > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-67+Multi-team+deployment+of+Airflow+components> > ;> > > > > > > > > > > > > > > > > J. > > > > > > > > > > > > > > > > > > > > > > > > > > Т���������������������������������������������������������������������ХF�?V�7V'67&�&R�?R��?�â?FWb�V�7V'67&�&T??�&f��r�???6�R��&pФf�"??FF�F���?�?6���?�G2�?R��?�â?FWbֆV�???�&f��r�???6�R��&p > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
