Great analysis Kamil and it's fantastic it will be automated! I reviewed the whole list and the only comment was that "apache" is now one big bag of hook/operators to different apache products. I believe (and that was the proposal in https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-target_groupsTargetgroupsintheproviderspackages) that each project in 'apache' folder should have their own package. This way when we get to AIP-8 those should be separated out - while legally the projects are owned by "Apache Software Foundation", the stewardship of each project is in the hands of autonomous communities and being able to separate them on package level makes more sense to me.
BTW. Great catches with some "corner cases" like python/bash/weekday etc. :) J. On Mon, Dec 23, 2019 at 6:08 PM Kamil Breguła <kamil.breg...@polidea.com> wrote: > Hello, > > I created a spreadsheet with all the changes > > https://docs.google.com/spreadsheets/d/1Z5jqsGz9QjmVDQAeI6NK8Aa-v6SQv8jfV8QzIZ4-q4w/edit > What do you think about it? The document has active comments if anyone has > questions. I want to move all the files based on it. I have an automatic > script that will do it quickly. > > Best regards, > Kamil Breguła > > On Sat, Nov 16, 2019 at 1:36 PM Jarek Potiuk <jarek.pot...@polidea.com> > wrote: > > > Subtasks in https://issues.apache.org/jira/browse/AIRFLOW-4733 created > and > > the table of operators is now part of the AIP-21 > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-target_groupsTargetgroupingoftheoperators/sensors/hooksintheproviderspackages > > > > > ! Feel free to pick those tasks and make the move happen! > > > > J. > > > > On Sat, Nov 16, 2019 at 12:32 PM Jarek Potiuk <jarek.pot...@polidea.com> > > wrote: > > > > > So we have now 5 (+1) binding votes: Jarek Potiuk, Kaxil Naik, Felix > > > Uelendall, Kamil Breguła, Kevin Yang, One (+1) non-binding vote: > Tomasz > > > Urbaszek. > > > > > > The vote is "Passed" > > > > > > Thank you! It was a long discussion but I think we have a reasonable > > > approach at the end :). > > > > > > I will update the AIP-21 appropriately and we proceed with renaming. A > > > good deal of that had already happened but I will also create a > separate > > > JIRA with subtasks to move all the subpackages. I will also create an > > issue > > > to automate backport package generation (I will likely work on the last > > one > > > unless someone wants to take it :). > > > > > > J. > > > > > > On Fri, Nov 15, 2019 at 7:09 PM Kevin Yang <yrql...@gmail.com> wrote: > > > > > >> +1 (binding) > > >> > > >> Kaxil Naik <kaxiln...@gmail.com>于2019年11月15日 周五上午10:02写道： > > >> > > >> > I am fine with it :) > > >> > > > >> > On Fri, Nov 15, 2019 at 5:25 PM Jarek Potiuk < > > jarek.pot...@polidea.com> > > >> > wrote: > > >> > > > >> > > I think we should leave all the fundamentals (or 'core') operators > > >> hooks > > >> > > etc where they are. I have not even considered moving them. I am > ok > > to > > >> > have > > >> > > 'aieflow.operators', 'airflow.hooks', 'airflow.sensors' for those. > > It > > >> > feels > > >> > > natural and having them 'higher' in the tree hierarchy is a good > way > > >> to > > >> > > show that they are fundamental part of Airflow. > > >> > > > > >> > > J > > >> > > > > >> > > pt., 15 lis 2019, 16:49 użytkownik Kaxil Naik < > kaxiln...@gmail.com> > > >> > > napisał: > > >> > > > > >> > > > I just have 1 comment: > > >> > > > > > >> > > > Should we have a "*airflow.core*" (or airflow.providers) for all > > the > > >> > > > hooks/operators listed under *fundamentals *as I feel we are > going > > >> to > > >> > > have > > >> > > > "airflow.providers.gcp", "airflow.providers.amazon", > > >> > > > "airflow.providers.apache"? > > >> > > > > > >> > > > What do you all think? Maybe it is not necessary but just wanted > > to > > >> see > > >> > > > what you all think? > > >> > > > > > >> > > > In general, I am happy with this: +1 (binding) > > >> > > > > > >> > > > Regards, > > >> > > > Kaxil > > >> > > > > > >> > > > On Fri, Nov 15, 2019 at 12:42 PM Felix Uellendall > > >> > <felue...@pm.me.invalid > > >> > > > > > >> > > > wrote: > > >> > > > > > >> > > > > +1 (binding) > > >> > > > > > > >> > > > > Felix > > >> > > > > > > >> > > > > Sent from ProtonMail Mobile > > >> > > > > > > >> > > > > On Fri, Nov 15, 2019 at 12:46, Kamil Breguła < > > >> > > kamil.breg...@polidea.com> > > >> > > > > wrote: > > >> > > > > > > >> > > > > > +1 (binding) > > >> > > > > > > > >> > > > > > All my comments were taken into account during the > discussion. > > >> I am > > >> > > > > happy now. > > >> > > > > > > > >> > > > > > On Fri, Nov 15, 2019 at 12:37 PM Jarek Potiuk < > > >> > > > jarek.pot...@polidea.com> > > >> > > > > wrote: > > >> > > > > >> > > >> > > > > >> Some binding votes here please :) ? > > >> > > > > >> > > >> > > > > >> J. > > >> > > > > >> > > >> > > > > >> On Tue, Nov 12, 2019 at 5:25 PM Tomasz Urbaszek < > > >> > > > > tomasz.urbas...@polidea.com> > > >> > > > > >> wrote: > > >> > > > > >> > > >> > > > > >> > +1 (non-binding) > > >> > > > > >> > > > >> > > > > >> > Bests, > > >> > > > > >> > Tomek > > >> > > > > >> > > > >> > > > > >> > On Tue, Nov 12, 2019 at 4:57 PM Jarek Potiuk < > > >> > > > > jarek.pot...@polidea.com> > > >> > > > > >> > wrote: > > >> > > > > >> > > > >> > > > > >> > > The email calls for a 2nd attempt to [VOTE] an update > to > > >> > AIP-21 > > >> > > > > Changes > > >> > > > > >> > in > > >> > > > > >> > > import paths > > >> > > > > >> > > < > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths > > >> > > > > >> > > > > > >> > > > > >> > > with > > >> > > > > >> > > the changes described below. The vote will last till > > Friday > > >> > 15th > > >> > > > > 5pm CEST > > >> > > > > >> > > (72 hours). > > >> > > > > >> > > > > >> > > > > >> > > Committers have a binding vote but everyone from the > > >> community > > >> > > is > > >> > > > > >> > > encouraged to cast an advisory vote. > > >> > > > > >> > > > > >> > > > > >> > > *Summary*: > > >> > > > > >> > > > > >> > > > > >> > > The proposal is to update AIP-21 to move all non-core > > >> > > > > >> > > operators/hooks/sensor (and related files) to > "providers" > > >> > > package. > > >> > > > > >> > > > > >> > > > > >> > > Those packages will be separately released > > >> (schedule/process > > >> > > TBD) > > >> > > > > and > > >> > > > > >> > will > > >> > > > > >> > > be backportable to 1.10.* airflow series, so that users > > can > > >> > > > install > > >> > > > > it > > >> > > > > >> > and > > >> > > > > >> > > start using new Airflow2.0 operators in their Python 3 > > >> Airflow > > >> > > > 1.10 > > >> > > > > >> > > environments (only Python 3.5+ is supported). > > >> > > > > >> > > > > >> > > > > >> > > The proposed split is documented in detail in > > >> > > > > https://docs.google.com/ > > >> > > > > >> > > > > >> > > > > > > >> > > spreadsheets/d/17zA5t2JVxnDdg5Cs1Cg_Mb1GXvGctmesfg2L089QSOk/edit#gid=0 > > >> > > > > >> > > < > > >> > > > > >> > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://docs.google.com/spreadsheets/d/17zA5t2JVxnDdg5Cs1Cg_Mb1GXvGctmesfg2L089QSOk/edit#gid=0 > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > Once we get the proposal accepted, I will update AIP-21 > > to > > >> > > reflect > > >> > > > > all > > >> > > > > >> > that > > >> > > > > >> > > and move the proposed split to CWiki. > > >> > > > > >> > > > > >> > > > > >> > > Discussion concerning the first vote (and changes > > discussed > > >> > and > > >> > > > > >> > > implemented) can be found here: > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://lists.apache.org/thread.html/2c9559184045e772acd21cbdd7435f6bf89c76eb9311311d58d16e5f@%3Cdev.airflow.apache.org%3E > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > Consider this my +1 (binding) vote. > > >> > > > > >> > > > > >> > > > > >> > > J. > > >> > > > > >> > > > > >> > > > > >> > > -- > > >> > > > > >> > > > > >> > > > > >> > > Jarek Potiuk > > >> > > > > >> > > Polidea <https://www.polidea.com/> | Principal > Software > > >> > > Engineer > > >> > > > > >> > > > > >> > > > > >> > > M: +48 660 796 129 <+48660796129> > > >> > > > > >> > > [image: Polidea] <https://www.polidea.com/> > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > > >> > > > >> > > > > >> > -- > > >> > > > > >> > > > >> > > > > >> > Tomasz Urbaszek > > >> > > > > >> > Polidea <https://www.polidea.com/> | Junior Software > > >> Engineer > > >> > > > > >> > > > >> > > > > >> > M: +48 505 628 493 <+48505628493> > > >> > > > > >> > E: tomasz.urbas...@polidea.com < > > tomasz.urbasz...@polidea.com > > >> > > > >> > > > > >> > > > >> > > > > >> > Unique Tech > > >> > > > > >> > Check out our projects! < > https://www.polidea.com/our-work> > > >> > > > > >> > > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> -- > > >> > > > > >> > > >> > > > > >> Jarek Potiuk > > >> > > > > >> Polidea <https://www.polidea.com/> | Principal Software > > >> Engineer > > >> > > > > >> > > >> > > > > >> M: +48 660 796 129 <+48660796129> > > >> > > > > >> [image: Polidea] <https://www.polidea.com/> > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > > -- > > > > > > Jarek Potiuk > > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > > > M: +48 660 796 129 <+48660796129> > > > [image: Polidea] <https://www.polidea.com/> > > > > > > > > > > -- > > > > Jarek Potiuk > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > M: +48 660 796 129 <+48660796129> > > [image: Polidea] <https://www.polidea.com/> > > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
