Hey Fokko, Just to alleviate your concerns a bit - we are going to make it backwards compatible for a start. And we will just start with GCP operators together with Kamil and Tomek, to show how it works (and that it's not that difficult in fact - maybe even automate it it if needed - as we usually do). I think with the recent changes on Pylint / Py3 it will be harder and harder to cherry-pick to v1-10 anyway, so it looks like it's a good time to make some bolder moves.
J. On Mon, Aug 5, 2019 at 8:35 PM Driesprong, Fokko <fo...@driesprong.frl> wrote: > Hi Jarek, > > No worries. I must admit that I've lost track a bit when the case 3+4 got > combined. Maybe for the future better to let the current vote run and maybe > reinitiate a new vote if there are new options. > > Currently, it is quite easy to think of a few obvious groups, but it is so > hard to keep this consistent in the future. I'm not saying that one giant > operator package is ideal, but I think it is easy to maintain. Two/three > years ago the guys for Databricks wrote all the operators for their > platform, and it was in such a state that it was eligible to move to the > core (non-contrib), similar with GCP now. But it is hard to actually do it. > Putting everything into one place makes this from my perspective simpler. > If Ali-Cloud comes with some operators, should we put them in a separate > package right away? Prefixing would be a nice compromise for me. > > If I'm the only one against adding the additional namespaces, then I don't > want to stop the rest of the community of making this happen. I don't like > to veto stuff, but I think we should think this through properly :-) > > Cheers, Fokko > > > Op ma 5 aug. 2019 om 17:55 schreef Jarek Potiuk <jarek.pot...@polidea.com > >: > > > Hey Fokko, > > > > Really sorry, to miss this . I see now that you have not voted because > you > > thought it will be transferred automatically -it was thought as not and > > additional option but combined replacement for (3+4) cases. > > > > This was after I realised that there was a veto for namespaces from Ash > and > > I realised from it that namespaces were a bit confusing and overlapping > > with grouping per-cloud-provider (see the voting "chapter > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Voting > > >"). > > I re-fined the choice back then - whether to not group them (A) or > whether > > to group first by type of entity or first by cloud provider (B-E in > various > > ways). Maybe I was not very clear in my message - apologies. > > > > I now re-added your vote and also Ben's vote who also voted on leaving > the > > prefix and not grouping operators (Ben please correct me if I was wrong). > > Still the result is 4:2 (binding), 6:2 (including non-binding) in favour > of > > grouping per cloud provider rather than using prefixes. And no vetoes. > > > > Re - the point of which operators should be grouped. From the > discussions - > > I do not think it's easy to do a general and always applicable rule. But > I > > believe we agreed that we group the "obvious" choices - like "gcp", > > "azure", "aws". It's for pure "Action" type operators - but still we > leave > > the cross-platform "transfer" ones or non-obvious ones in the old place > > (which we keep as bag of "everything else"). Personally - I am all for > > grouping the qubole/databricks operators in packages same as other cloud > > providers. But with the "softness" of that rule this might (and should) > be > > an individual decision of people who maintain it. And it is quite OK - > not > > everything has to be decided upfront. I don't think we have a "hard" rule > > now that covers all the cases and binds us. It would be very complex one > if > > we have it (and we can defer introducing it for later if needed > especially > > if it requires more than just renaming/moving but also architectural > > changes - as Ash proposed for the transfer operators). But at least we > can > > decrease an entropy by grouping the obvious cases, and we can move > forward > > leaving the Airflow world a bit better, one step at a time. > > > > J. > > > > On Mon, Aug 5, 2019 at 3:58 PM Driesprong, Fokko <fo...@driesprong.frl> > > wrote: > > > > > Hi Jarek, > > > > > > Just wondering how the vote is unanimous. On case 3 I've voted for B. I > > > still believe introducing namespaces is hard to maintain, as I > explained > > > earlier: Mostly because it isn't as trivial to decide when it gets its > > own > > > package or not. Besides the cloud providers, there are companies like > > > Databricks and Qubole which might also end up with their own package > > > because their integration is getting richer over time. Moving this is a > > bit > > > painful because we need to deprecate the old path and move everything > > which > > > introduces noise into version control etc. > > > > > > Adding an additional option to the list, does not invalidate the older > > > options, right? > > > > > > Cheers, Fokko > > > > > > > > > Op ma 5 aug. 2019 om 15:43 schreef Jarek Potiuk < > > jarek.pot...@polidea.com > > > >: > > > > > > > Seems that after this months of discussion we got an unanimous voting > > > > result. I summarised it in the AIP-21 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths > > > > > > > > > and > > > > changed its status to "Accepted". Thanks for all your feedback! We > will > > > > start soon moving GCP operators following those rules and we will > > resolve > > > > any initial problems with those - then we might provide some > additional > > > > learnings and see if we can easily (probably semi-automatically) move > > all > > > > other operators. > > > > > > > > Case 1 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#1airflow.contrib.{resources} > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%231airflow.contrib.%7Bresources%7D> > > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%231airflow.contrib.%7Bresources%7D > > > > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%231airflow.contrib.%7Bresources%7D > > > > > > > < > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%231airflow.contrib.%7Bresources%7D > > > > > > > > > > > > > > > > > What to do with the "contrib" folder > > > > > > > > Case 2 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#2git*_{operator/sensor}{/s}.py > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%232git*_%7Boperator/sensor%7D%7B/s%7D.py> > > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%232git*_%7Boperator/sensor%7D%7B/s%7D.py > > > > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%232git*_%7Boperator/sensor%7D%7B/s%7D.py > > > > > > > < > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%232git*_%7Boperator/sensor%7D%7B/s%7D.py > > > > > > > > > > > > > > > > > Drop modules *_operator suffix > > > > > > > > Case 3 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#3{aws/azure/gcp}_* > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%233%7Baws/azure/gcp%7D_*> > > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%233%7Baws/azure/gcp%7D_* > > > > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%233%7Baws/azure/gcp%7D_* > > > > > > > < > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%233%7Baws/azure/gcp%7D_* > > > > > > > > > > > > > + Case 4 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#4Separatenamespaceforresources > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%234Separatenamespaceforresources> > > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%234Separatenamespaceforresources > > > > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%234Separatenamespaceforresources > > > > > > > < > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%234Separatenamespaceforresources > > > > > > > > > > > > > > > > > Grouping Cloud Providers operators/sensors/hooks > > > > > > > > Case 5 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#5*Operator > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%235*Operator> > > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%235*Operator > > > > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%235*Operator > > > > > > > < > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%235*Operator > > > > > > > > > > > > > > > > > *Operator *Sensor *Hook in class name > > > > > > > > Case 6 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#6Otherisolatedcases > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%236Otherisolatedcases> > > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%236Otherisolatedcases > > > > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%236Otherisolatedcases > > > > > > > < > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%236Otherisolatedcases > > > > > > > > > > > > > > > > > Isolated cases > > > > > > > > Case 7 > > > > < > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case#7 > <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%237> > > < > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%237 > > > > > < > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%237 > > > > > > > < > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Case%237 > > > > > > > > > > > > > > > > > Deprecation method > > > > > > > > *A.* Move everything "contrib" → "airflow" > > > > > > > > *A.* Drop the suffix. > > > > > > > > Example: > > > > > > > > - *airflow.operator.gcp_bigtable_operator.py* > > > > becomes *airflow.operator.gcp_bigtable.py > > > > <http://airflow.operator.gcp_bigtable.py>*. > > > > > > > > *D. * Group operators/sensors/hooks in > > > > *airflow/<PROVIDER>*/operators(sensors, > > > > hooks). Non-cloud provider ones are moved to > > > > airflow/operators(sensors/hooks). > > > > *Drop the prefix.* > > > > > > > > - > > > > *airflow/contrib/operators/sns_publish_operator.py > > > > becomes airflow/aws/operators/**sns_publish_operator.py* > > > > > > > > > > > > - *airflow/contrib/operators/dataproc_operator.py* > > > > becomes *airflow/gcp/operators/dataproc_operator.py* > > > > > > > > > > > > > > > > - > > > > *airflow/contrib/operators/gcp_bigtable_operator.py *becomes > > *airflow/* > > > > *gcp/operators/bigtable_operator.py* > > > > > > > > > > > > > > > > - > > > > *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* > > > > *operators/ssh_operator.py* > > > > > > > > *B.* No change - keep Operator/Sensor suffix in class name. > > > > > > > > *A.* Make individual decisions of renames for operators that do not > > > follow > > > > common conventions used for other operators. > > > > > > > > Consistency trumps compatibility. > > > > > > > > Examples: > > > > > > > > *DataProcHadoopOperator* > > > > > > > > renamed to: > > > > > > > > *DataprocHadoopOperator* > > > > *A.* Native python method (with better IDE support and more flexible > > > but a > > > > bit more verbose) > > > > > > > > J. > > > > > > > > On Wed, Jul 31, 2019 at 10:30 AM Jarek Potiuk < > > jarek.pot...@polidea.com> > > > > wrote: > > > > > > > > > Yep. Agree with Ash on it. There are a number of 'action' operators > > > > > specific for cloud providers and these should be our target. The > > > transfer > > > > > ones require another AIP (A lot of that already discussed in AIP-8 > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827303 > > > > > ). > > > > > > > > > > J. > > > > > > > > > > Principal Software Engineer > > > > > Phone: +48660796129 > > > > > > > > > > śr., 31 lip 2019, 10:01 użytkownik Ash Berlin-Taylor < > a...@apache.org > > > > > > > > napisał: > > > > > > > > > >> This is a good idea for now. I'm also not overly concerned about > > these > > > > >> few non-cloud examples - FTPtoS3Operator can stay where it is and > > > > doesn't > > > > >> have to move under 'aws.' to my mind. > > > > >> > > > > >> Longer term I'd like to go back to making the > > > "transfer/copy/transform" > > > > >> operators "composable" so that we can have a single > > DataCopyOperator, > > > > and > > > > >> give it a source and destination, and it uses hooks to do the > work. > > > > This is > > > > >> not a small undertaking though to do well and efficiently though. > > > > >> > > > > >> -ash > > > > >> > > > > >> > On 30 Jul 2019, at 20:54, Tomasz Urbaszek < > > > > tomasz.urbas...@polidea.com> > > > > >> wrote: > > > > >> > > > > > >> > Maybe we can put all those AtoB operators under one name like > > > > >> “transfer”, > > > > >> > then it would be easier to look for such operator? > > > > >> > > > > > >> > Best, > > > > >> > Tomek > > > > >> > > > > > >> > On Tue, 30 Jul 2019 at 21:39, Chen Tong <cix...@gmail.com> > wrote: > > > > >> > > > > > >> >> Daniel mentioned a good point. Such composed operator may also > > > > involves > > > > >> >> both cloud and non-cloud provider saying FTPtoS3Operator. > Should > > it > > > > in > > > > >> AWS > > > > >> >> folder? > > > > >> >> > > > > >> >> Also, saying in the future, another cloud provider is growing > > large > > > > >> enough. > > > > >> >> Will we rename all plugins related to this provider? What's the > > > > >> criteria we > > > > >> >> say it's big enough? And option D gives an impression like > these > > > > tools > > > > >> may > > > > >> >> be maintained and supported by the Cloud provider. I am not > sure > > if > > > > it > > > > >> will > > > > >> >> be a truth or not. > > > > >> >> > > > > >> >> Best, > > > > >> >> > > > > >> >> > > > > >> >> On Tue, Jul 30, 2019 at 11:18 AM Daniel Standish < > > > > dpstand...@gmail.com > > > > >> > > > > > >> >> wrote: > > > > >> >> > > > > >> >>> One wrinkle with case 3+4 option D is inter-provider > operators. > > > > >> Mainly > > > > >> >>> it's storage I think e.g. XToS3Operator or XToGCSOperator > where > > > the > > > > X > > > > >> is > > > > >> >> a > > > > >> >>> service some different provider. > > > > >> >>> > > > > >> >>> Maybe the rule should be to locate the operator according to > the > > > > first > > > > >> >>> provider referenced. So e.g. s3_to_gcs_transfer_operator.py > > would > > > > go > > > > >> to > > > > >> >>> aws. > > > > >> >>> > > > > >> >>> > > > > >> >>> On Tue, Jul 30, 2019 at 6:21 AM Kamil Breguła < > > > > >> kamil.breg...@polidea.com > > > > >> >>> > > > > >> >>> wrote: > > > > >> >>> > > > > >> >>>> Yes. All changes will be backwards compatible. In the case of > > > using > > > > >> >>>> the old path, a message containing a proposal for change will > > be > > > > >> >>>> reported to the user. > > > > >> >>>> > > > > >> >>>> I prepared an example of how to change the name of a class > in a > > > > case > > > > >> >>>> with the use of a native solution. > > > > >> >>>> Source code: > > > > >> >>>> > > > > >> >> > > > > >> > > > > > > https://github.com/mik-laj/airflow-deprecation-sample/tree/master/rename > > > > >> >>>> Preview from IDE: https://imgur.com/a/45NxS5W > > > > >> >>>> > > > > >> >>>> On Tue, Jul 30, 2019 at 2:28 PM Ash Berlin-Taylor < > > > a...@apache.org> > > > > >> >>> wrote: > > > > >> >>>>> > > > > >> >>>>> Just cos I'm not sure it's _explicitly_ stated, but all of > the > > > > moves > > > > >> >>>> will have a deprecation of the old name right? > > > > >> >>>>> > > > > >> >>>>> 3+4 case D gets my vote too. > > > > >> >>>>> > > > > >> >>>>> -a > > > > >> >>>>> > > > > >> >>>>>> On 30 Jul 2019, at 09:58, Jarek Potiuk < > > > jarek.pot...@polidea.com > > > > > > > > > >> >>>> wrote: > > > > >> >>>>>> > > > > >> >>>>>> I went ahead and updated the page (just to speed it up) as > I > > > > think > > > > >> >> it > > > > >> >>>>>> really makes sense to join those two cases (and I do not > see > > > any > > > > >> >>>> drawbacks > > > > >> >>>>>> - I think the options we have cover all possible > approaches) > > > and > > > > we > > > > >> >>> can > > > > >> >>>>>> always go back if we need to. > > > > >> >>>>>> > > > > >> >>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal > > > > >> >>>>>> > > > > >> >>>>>> My vote is *D*. > > > > >> >>>>>> > > > > >> >>>>>> - airflow/contrib/operators/gcp_bigtable_operator.py → > > > > >> >>> airflow/gcp > > > > >> >>>>>> /operators/bigtable_operator.py > > > > >> >>>>>> - airflow/contrib/operators/ssh_operator.py -> > > > > >> >>>>>> airflow/operators/ssh_operator.py > > > > >> >>>>>> > > > > >> >>>>>> I updated the page with my vote. > > > > >> >>>>>> I propose that we vote till Friday on that one (all the > rest > > is > > > > >> >>> already > > > > >> >>>>>> agreed). > > > > >> >>>>>> > > > > >> >>>>>> J. > > > > >> >>>>>> > > > > >> >>>>>> > > > > >> >>>>>> On Tue, Jul 30, 2019 at 9:42 AM Jarek Potiuk < > > > > >> >>> jarek.pot...@polidea.com > > > > >> >>>>> > > > > >> >>>>>> wrote: > > > > >> >>>>>> > > > > >> >>>>>>> I think almost everyone voted and we have almost perfect > > > > >> >> consensus. > > > > >> >>>> We all > > > > >> >>>>>>> agree amongst other on moving all operators out of contrib > > > > >> >> (Great!). > > > > >> >>>>>>> > > > > >> >>>>>>> The only doubts are for *Case 3* (Cloud provider prefix) > and > > > > *Case > > > > >> >>> 4* > > > > >> >>>>>>> (Using Namespaces). > > > > >> >>>>>>> I think there was actually an overlap between those two. > > Also > > > > >> >> Ash's > > > > >> >>>>>>> comment/veto on that is quite clear. > > > > >> >>>>>>> So I have final (I hope!) proposal to join those two cases > > and > > > > >> >> have > > > > >> >>>> one > > > > >> >>>>>>> voting (till Friday) only on that. > > > > >> >>>>>>> > > > > >> >>>>>>> I will update the doc if you all agree with me that makes > > more > > > > >> >> sense > > > > >> >>>> as > > > > >> >>>>>>> single case to vote on: > > > > >> >>>>>>> > > > > >> >>>>>>> *[Case 3 + Case 4]: Grouping Cloud Provider operators.* > > > > >> >>>>>>> > > > > >> >>>>>>> *Option A*. Keep operators/sensors/hooks in > > > > >> >>> airflow/operators(sensors, > > > > >> >>>>>>> hooks) and keep/add prefixes in file names. > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py > becomes > > > > >> >>>>>>> airflow/operators/**aws_sns_publish_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* > > > > >> >>>>>>> becomes *airflow/operators/gcp_dataproc_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/hooks/gcp_bigtable_hook.py *becomes > > > *airflow/* > > > > >> >>>>>>> *hooks/gcp_bigtable_hook.py* > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes > > > *airflow/* > > > > >> >>>>>>> *operators/ssh_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> *Option B*. Group operators/sensors/hooks in > > > > >> >>>> airflow/operators(sensors, > > > > >> >>>>>>> hooks)/*<PROVIDER>.* Non-cloud provider ones are moved to > > > > >> >>>>>>> airflow/operators(sensors/hooks), Drop the prefix. > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py > > > > >> >>>>>>> becomes airflow/operators/**aws/sns_publish_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* > > > > >> >>>>>>> becomes *airflow/operators/gcp/dataproc_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py > > *becomes > > > > >> >>>> *airflow/* > > > > >> >>>>>>> *operators/gcp/bigtable_operator.py* > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes > > > *airflow/* > > > > >> >>>>>>> *operators/ssh_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> *Option C*. Group operators/sensors/hooks in > > > > >> >>>> airflow/operators(sensors, > > > > >> >>>>>>> hooks)*/<PROVIDER>.* Non-cloud provider ones are moved to > > > > >> >>>>>>> airflow/operators(sensors/hooks), Keep the prefix. > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py > > > > >> >>>>>>> becomes > > airflow/operators/**aws/aws_sns_publish_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* > > > > >> >>>>>>> becomes *airflow/operators/gcp/gcp_dataproc_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py > > *becomes > > > > >> >>>> *airflow/* > > > > >> >>>>>>> *operators/gcp/gcp_bigtable_operator.py* > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes > > > *airflow/* > > > > >> >>>>>>> *operators/ssh_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> *Option D.* Group operators/sensors/hooks in > > > > >> >>>> *airflow/<PROVIDER>*/operators(sensors, > > > > >> >>>>>>> hooks). Non-cloud provider ones are moved to > > > > >> >>>>>>> airflow/operators(sensors/hooks). Drop the prefix. > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py > > > > >> >>>>>>> becomes airflow/aws/operators/**sns_publish_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* > > > > >> >>>>>>> becomes *airflow/gcp/operators/dataproc_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py > > *becomes > > > > >> >>>> *airflow/* > > > > >> >>>>>>> *gcp/operators/bigtable_operator.py* > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes > > > *airflow/* > > > > >> >>>>>>> *operators/ssh_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> *Option E.* Group operators/sensors/hooks in > > > > >> >>>> *airflow/<PROVIDER>*/operators(sensors, > > > > >> >>>>>>> hooks). Non-cloud provider ones are moved to > > > > >> >>>>>>> airflow/operators(sensors/hooks).* Keep the prefix*. > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/sns_publish_operator.py > > > > >> >>>>>>> becomes > > airflow/aws/operators/aws_**sns_publish_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* > > > > >> >>>>>>> becomes *airflow/gcp/operators/gcp_dataproc_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py > > *becomes > > > > >> >>>> *airflow/* > > > > >> >>>>>>> *gcp/operators/gcp_bigtable_operator.py* > > > > >> >>>>>>> - > > > > >> >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes > > > *airflow/* > > > > >> >>>>>>> *operators/ssh_operator.py* > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> Shall I proceed with updating the page ? > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> J. > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> On Mon, Jul 29, 2019 at 11:18 AM Kaxil Naik < > > > > kaxiln...@gmail.com> > > > > >> >>>> wrote: > > > > >> >>>>>>> > > > > >> >>>>>>>> Thanks Jarek, adding my vote. > > > > >> >>>>>>>> > > > > >> >>>>>>>> On Mon, Jul 29, 2019 at 2:40 PM Ash Berlin-Taylor < > > > > >> >> a...@apache.org> > > > > >> >>>> wrote: > > > > >> >>>>>>>> > > > > >> >>>>>>>>> Thanks Jarek, much clearer. I have voted there too. > > > > >> >>>>>>>>> > > > > >> >>>>>>>>> -ash > > > > >> >>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>>>> On 29 Jul 2019, at 08:13, Driesprong, Fokko > > > > >> >> <fo...@driesprong.frl > > > > >> >>>> > > > > >> >>>>>>>>> wrote: > > > > >> >>>>>>>>>> > > > > >> >>>>>>>>>> Thanks Jarek, the Wiki works much better for me. I've > > added > > > > my > > > > >> >>> vote > > > > >> >>>>>>>> there > > > > >> >>>>>>>>>> as well. > > > > >> >>>>>>>>>> > > > > >> >>>>>>>>>> You've convinced me on the second case. I've changed my > > > vote > > > > on > > > > >> >>>> Case 2 > > > > >> >>>>>>>>> from > > > > >> >>>>>>>>>> A to B :-) > > > > >> >>>>>>>>>> > > > > >> >>>>>>>>>> Maybe we should leave the vote open a bit longer since > it > > > > >> >>> involves > > > > >> >>>>>>>> quite > > > > >> >>>>>>>>>> some reading, and I would like to get some proper > > feedback > > > > and > > > > >> >>>>>>>> opinions > > > > >> >>>>>>>>> on > > > > >> >>>>>>>>>> this. Plus I only see two votes right now. If we don't > > > think > > > > >> >> this > > > > >> >>>>>>>>> through, > > > > >> >>>>>>>>>> it might be that we're having this vote again in the > near > > > > >> >> future > > > > >> >>>> :-) > > > > >> >>>>>>>>>> > > > > >> >>>>>>>>>> Cheers, Fokko > > > > >> >>>>>>>>>> > > > > >> >>>>>>>>>> Op zo 28 jul. 2019 om 10:49 schreef Jarek Potiuk < > > > > >> >>>>>>>>> jarek.pot...@polidea.com>: > > > > >> >>>>>>>>>> > > > > >> >>>>>>>>>>> I updated the AIP-21 proposal in the way that should > > > answer > > > > >> >> all > > > > >> >>>> the > > > > >> >>>>>>>>>>> concerns in this thread. > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> NOTE! There is an action for all of those who are > > > > interested: > > > > >> >>>> Please > > > > >> >>>>>>>>>>> fill-in your voting in the voting table till Tuesday > > 30th > > > of > > > > >> >>> July > > > > >> >>>> 6pm > > > > >> >>>>>>>>> CEST > > > > >> >>>>>>>>>>> (5pm BST, 9 am PST). > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> I created a summary of the proposal > > > > >> >>>>>>>>>>> < > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>> in table form - which contains also examples for each > > > case. > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> I added a voting table > > > > >> >>>>>>>>>>> < > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Voting > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>> where you should add your votes: > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> I propose this voting mechanism: > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> Voting will take place till *Tuesday* 30 Jul 2019 6pm > > CEST > > > > (5 > > > > >> >>> pm > > > > >> >>>>>>>> BST, > > > > >> >>>>>>>>> 9am > > > > >> >>>>>>>>>>> PST) . > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> After the choice, the final consistent set of choices > > will > > > > be > > > > >> >>>>>>>> announced > > > > >> >>>>>>>>>>> (taking into account majority of binding vote, also > > > > including > > > > >> >>>>>>>> potential > > > > >> >>>>>>>>>>> vetos and consistency between the choices. Non-binding > > > votes > > > > >> >>> will > > > > >> >>>> be > > > > >> >>>>>>>>> taken > > > > >> >>>>>>>>>>> into account in case there is a draw. The final set of > > > > choices > > > > >> >>>> will > > > > >> >>>>>>>> be > > > > >> >>>>>>>>>>> announced at devlist thread > > > > >> >>>>>>>>>>> < > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://lists.apache.org/thread.html/4cedc94bee53ad908eee8333a56b58be8b5641881e73f69b97e436a9@%3Cdev.airflow.apache.org%3E > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>> after > > > > >> >>>>>>>>>>> the voting completes. > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> Unless there is a veto raised to the final proposal, > the > > > > final > > > > >> >>>>>>>> proposal > > > > >> >>>>>>>>> is > > > > >> >>>>>>>>>>> accepted by Lazy Consensus > > > > >> >>>>>>>>>>> < > > > https://community.apache.org/committers/lazyConsensus.html > > > > > > > > > >> >>> on > > > > >> >>>>>>>>> *Friday* > > > > >> >>>>>>>>>>> 02 > > > > >> >>>>>>>>>>> Aug 2019 at 6pm CEST (5 pm BST, 9am PST). > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> Please let me know if what I proposed can be further > > > > improved > > > > >> >> to > > > > >> >>>>>>>> avoid > > > > >> >>>>>>>>>>> chaos. > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> J. > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> On Sat, Jul 27, 2019 at 11:41 AM Jarek Potiuk < > > > > >> >>>>>>>> jarek.pot...@polidea.com > > > > >> >>>>>>>>>> > > > > >> >>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>>> Ok. I see that indeed voting could have been > organised > > a > > > > bit > > > > >> >>>> better > > > > >> >>>>>>>> - > > > > >> >>>>>>>>> dev > > > > >> >>>>>>>>>>>> list does not seem to cope well with such a complex > > case > > > > :). > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>>> As Kamil noticed - we already have a formal AIP-21 > > > > >> >>>>>>>>>>>> < > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>>> in the Wiki - which I should have mentioned before. I > > > have > > > > >> >> not > > > > >> >>>> much > > > > >> >>>>>>>>> time > > > > >> >>>>>>>>>>>> today - but tomorrow I will try to summarise the > > options > > > - > > > > >> >>> based > > > > >> >>>> on > > > > >> >>>>>>>>> some > > > > >> >>>>>>>>>>>> real code examples (to make it easier to digest). I > > think > > > > the > > > > >> >>>> best > > > > >> >>>>>>>>>>> approach > > > > >> >>>>>>>>>>>> will be to make a voting matrix in the AIP-21 Wiki > page > > > > where > > > > >> >>>> people > > > > >> >>>>>>>>> will > > > > >> >>>>>>>>>>>> be able to state their preferences for each case (by > > > > adding a > > > > >> >>>> row to > > > > >> >>>>>>>>> the > > > > >> >>>>>>>>>>>> table). I think that might provide much better > clarity > > > and > > > > a > > > > >> >>>> single > > > > >> >>>>>>>>> place > > > > >> >>>>>>>>>>>> which will show the current aggregated state of > voting. > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>>> However - as Ash also stated - some of the options > are > > > > >> >>>>>>>> inter-dependent > > > > >> >>>>>>>>> so > > > > >> >>>>>>>>>>>> at the end we will have to adjust the results in case > > > they > > > > >> >> are > > > > >> >>>> not > > > > >> >>>>>>>>>>>> consistent - and make final voting on aggregated set > > of > > > > >> >> cases > > > > >> >>> - > > > > >> >>>>>>>> but I > > > > >> >>>>>>>>>>>> think in this case following "lasy consensus" - i,e. > if > > > > noone > > > > >> >>>>>>>> objects > > > > >> >>>>>>>>> to > > > > >> >>>>>>>>>>>> final set of cases, we will proceed with it.. > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>>> Do you think that will work better ? > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>>> J. > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>>> Principal Software Engineer > > > > >> >>>>>>>>>>>> Phone: +48660796129 > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>>> sob., 27 lip 2019, 09:26 użytkownik Kamil Breguła < > > > > >> >>>>>>>>>>>> kamil.breg...@polidea.com> napisał: > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>>>> Document is also available as AIP on wiki: > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>> On Sat, Jul 27, 2019, 9:07 AM Driesprong, Fokko > > > > >> >>>>>>>> <fo...@driesprong.frl > > > > >> >>>>>>>>>> > > > > >> >>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>> I agree with all, this is a bit chaotic. For the > sake > > > of > > > > >> >>>> clarity, > > > > >> >>>>>>>> I > > > > >> >>>>>>>>>>>>> would > > > > >> >>>>>>>>>>>>>> suggest moving the Google Doc to the Wiki as an > AIP. > > > > >> >> Create a > > > > >> >>>>>>>>>>> structural > > > > >> >>>>>>>>>>>>>> code improvement AIP and then and add a matrix of > > > > >> >> users/cases > > > > >> >>>>>>>> where > > > > >> >>>>>>>>>>>>>> everybody can add his vote per case. This gives > also > > > the > > > > >> >>>>>>>> opportunity > > > > >> >>>>>>>>>>> for > > > > >> >>>>>>>>>>>>>> changing your vote on a specific case. WDYT? > > > > >> >>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>> Cheers, Fokko > > > > >> >>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>> Op vr 26 jul. 2019 om 22:28 schreef Chen Tong < > > > > >> >>>> cix...@gmail.com>: > > > > >> >>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>> I agree with Ash. It is clear to vote on each case > > > > rather > > > > >> >>> than > > > > >> >>>>>>>> all > > > > >> >>>>>>>>>>>>>>> together. > > > > >> >>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 3:54 PM Ash Berlin-Taylor > < > > > > >> >>>>>>>> a...@apache.org> > > > > >> >>>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>> A number of proposals overlap or add on to each > > > other, > > > > >> >> so I > > > > >> >>>>>>>> think > > > > >> >>>>>>>>>>>>> (?) a > > > > >> >>>>>>>>>>>>>>>> single vote makes sense. > > > > >> >>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>> To be clear, my vote is -1 until it's clear > exactly > > > > what > > > > >> >> we > > > > >> >>>> are > > > > >> >>>>>>>>>>>>> voting > > > > >> >>>>>>>>>>>>>> on. > > > > >> >>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>> On 26 July 2019 20:39:56 BST, Kaxil Naik < > > > > >> >>>> kaxiln...@gmail.com> > > > > >> >>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>>>>>> I agree with Ash and instead of voting on all 7 > > > Cases > > > > >> >>>> together, > > > > >> >>>>>>>>>>> how > > > > >> >>>>>>>>>>>>>>>>> about > > > > >> >>>>>>>>>>>>>>>>> separate vote emails for all the cases? Each > email > > > can > > > > >> >>> have > > > > >> >>>> the > > > > >> >>>>>>>>>>>>>>>>> description > > > > >> >>>>>>>>>>>>>>>>> copied over from the doc. > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> It would probably be a bit easier to track a > > > decision > > > > in > > > > >> >>> the > > > > >> >>>>>>>>>>> future > > > > >> >>>>>>>>>>>>> as > > > > >> >>>>>>>>>>>>>>>>> well. > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> What do you guys think? @Jarek Potiuk < > > > > >> >>>>>>>> jarek.pot...@polidea.com> > > > > >> >>>>>>>>>>>>>>>>> @Driesprong, > > > > >> >>>>>>>>>>>>>>>>> Fokko <fo...@driesprong.frl> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> On Sat, Jul 27, 2019 at 1:03 AM Kaxil Naik < > > > > >> >>>>>>>> kaxiln...@gmail.com> > > > > >> >>>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> *Case #1* airflow.contrib.{resources} : > *Solution > > > A * > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> *Case #2* git *_{operator/sensor}{/s}.py : > > > *Solution > > > > A > > > > >> >> * > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> *Case #3* {aws/azure/gcp}_* : *Solution C* > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> *Case #4* Separate namespace for resources : > > > > >> >>>>>>>>>>>>>>>>>> *Solution A* > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> *Case #5* *Operator : *Solution A* > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> *Case #7* : *Solution A* > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 11:09 PM Daniel > Standish > > > > >> >>>>>>>>>>>>>>>>> <dpstand...@gmail.com> > > > > >> >>>>>>>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> I have tried to add some clarification to > > Jarek's > > > > >> >>> summary, > > > > >> >>>>>>>> but > > > > >> >>>>>>>>>>> I > > > > >> >>>>>>>>>>>>> am > > > > >> >>>>>>>>>>>>>>>>> a > > > > >> >>>>>>>>>>>>>>>>>>> little fuzzy on exact proposal for case 3 so > > > > hopefully > > > > >> >>>> Jarek > > > > >> >>>>>>>>>>> can > > > > >> >>>>>>>>>>>>>>>>> further > > > > >> >>>>>>>>>>>>>>>>>>> clarify. > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> According to my reading, in this motion cases > > > 4,5,6 > > > > >> >> are > > > > >> >>>> all > > > > >> >>>>>>>>>>>>> either > > > > >> >>>>>>>>>>>>>>>>>>> proposal > > > > >> >>>>>>>>>>>>>>>>>>> *rejections* or otherwise have no effect, so > > > > attention > > > > >> >>>> can be > > > > >> >>>>>>>>>>>>>>>>> focused on > > > > >> >>>>>>>>>>>>>>>>>>> cases 1,2,3 and 7. > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> *Motion is to adopt the following:* > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> Case 1 - Solution A - Get rid of contrib > > > > >> >>>>>>>>>>>>>>>>>>> All objects will be moved out of contrib > folder > > > and > > > > >> >> into > > > > >> >>>>>>>>>>>>> respective > > > > >> >>>>>>>>>>>>>>>>>>> non-contrib folder. > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> Case 2 - Solution A - Remove object type > suffix > > > from > > > > >> >>>> modules > > > > >> >>>>>>>>>>>>>>>>>>> Example: > > > > >> >>>>>>>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator > > becomes > > > > >> >>>>>>>>>>>>>>>>>>> airflow.operators.foo.FooOperator > > > > >> >>>>>>>>>>>>>>>>>>> Example: > > > > >> >>>>>>>>>>>>>>>>>>> Airflow.hooks.foo_hook.FooOperator becomes > > > > >> >>>>>>>>>>>>> airflow.hooks.foo.FooHook > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> Case 3 - conventions for cloud provider etc > > > prefixes > > > > >> >>>>>>>>>>>>>>>>>>> *I am not sure exactly what the proposal is.* > > > > >> >>>>>>>>>>>>>>>>>>> Example case is > > > > >> >>>>>>>>>>>>> airflow/contrib/operators/gcp_bigtable_operator.py > > > > >> >>>>>>>>>>>>>>>>>>> Since motion adopts case 1 solution A, we can > > > assume > > > > >> >> it > > > > >> >>> is > > > > >> >>>>>>>> now > > > > >> >>>>>>>>>>>>> named > > > > >> >>>>>>>>>>>>>>>>> like > > > > >> >>>>>>>>>>>>>>>>>>> so: airflow/operators/gcp_bigtable_operator.py > > > > >> >>>>>>>>>>>>>>>>>>> So what is proposed for this case? > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> Case 4 - Solution B - *no change* > > > > >> >>>>>>>>>>>>>>>>>>> This proposal was about namespaces but the > > motion > > > is > > > > >> >> to > > > > >> >>>>>>>> reject > > > > >> >>>>>>>>>>>>> the > > > > >> >>>>>>>>>>>>>>>>>>> proposal. > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> Case 5 - Solution B - *no change* > > > > >> >>>>>>>>>>>>>>>>>>> The proposal was to remove class type suffix > > e.g. > > > > >> >> remove > > > > >> >>>> the > > > > >> >>>>>>>>>>>>>>>>> Operator in > > > > >> >>>>>>>>>>>>>>>>>>> FooOperator, but the motion is to reject this > > > > proposal > > > > >> >>> -- > > > > >> >>>>>>>> i.e. > > > > >> >>>>>>>>>>>>>>>>> motion is > > > > >> >>>>>>>>>>>>>>>>>>> no > > > > >> >>>>>>>>>>>>>>>>>>> change on this case, i.e. we resolve to keep > > > > >> >> "Operator" > > > > >> >>>> (and > > > > >> >>>>>>>>>>>>>>>>> "Sensor") > > > > >> >>>>>>>>>>>>>>>>>>> suffixes on class names > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> Case 6 - *No change* > > > > >> >>>>>>>>>>>>>>>>>>> This case concerns object naming > > inconsistencies, > > > > but > > > > >> >>>> motion > > > > >> >>>>>>>>>>> does > > > > >> >>>>>>>>>>>>>>>>> not > > > > >> >>>>>>>>>>>>>>>>>>> enact > > > > >> >>>>>>>>>>>>>>>>>>> any specific proposal. > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> Case 7 - Solution A - use warnings.warn > template > > > for > > > > >> >>>>>>>>>>> deprecation > > > > >> >>>>>>>>>>>>>>>>> warnings > > > > >> >>>>>>>>>>>>>>>>>>> (instead of zope) > > > > >> >>>>>>>>>>>>>>>>>>> Example: > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> # in old_package/old_mod.py > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> import warnings > > > > >> >>>>>>>>>>>>>>>>>>> from new_package.new_mod import * > > > > >> >>>>>>>>>>>>>>>>>>> warnings.warn("old_package.old_mod has moved > to > > > > >> >>>>>>>>>>>>> new_package.new_mod. > > > > >> >>>>>>>>>>>>>>>>>>> Import > > > > >> >>>>>>>>>>>>>>>>>>> of " > > > > >> >>>>>>>>>>>>>>>>>>> "old_package.old_mod will become unsupported > in > > > > >> >> version > > > > >> >>>> 2", > > > > >> >>>>>>>>>>>>>>>>>>> DeprecationWarning, 2) > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> Reference implementation here: > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://github.com/mik-laj/airflow-deprecation-sample/tree/master/solution1 > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> FWIW +1 (non-binding) on 1,2,7 -- unsure on 3. > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> I am very happy that this motion now gets rid > of > > > > >> >>> contrib. > > > > >> >>>>>>>>>>> Thank > > > > >> >>>>>>>>>>>>> you > > > > >> >>>>>>>>>>>>>>>>>>> Sergio > > > > >> >>>>>>>>>>>>>>>>>>> for turning the tide with your effective > > > > argumentation > > > > >> >>> ;) > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 3:16 AM Ash > > Berlin-Taylor > > > < > > > > >> >>>>>>>>>>>>> a...@apache.org> > > > > >> >>>>>>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>> Could someone summarise what the proposed > name > > > > >> >> changes > > > > >> >>>> are, > > > > >> >>>>>>>>>>>>> here, > > > > >> >>>>>>>>>>>>>>>>> in > > > > >> >>>>>>>>>>>>>>>>>>> this > > > > >> >>>>>>>>>>>>>>>>>>>> thread? Pointing at a google doc and only > > giving > > > > >> >>> partial > > > > >> >>>>>>>>>>>>> examples > > > > >> >>>>>>>>>>>>>>>>> is 1) > > > > >> >>>>>>>>>>>>>>>>>>> a > > > > >> >>>>>>>>>>>>>>>>>>>> bit difficult to quickly work out what we are > > > > voting > > > > >> >>> on, > > > > >> >>>> and > > > > >> >>>>>>>>>>> 2) > > > > >> >>>>>>>>>>>>>>>>> using a > > > > >> >>>>>>>>>>>>>>>>>>>> google doc isn't great for a longer term > > record. > > > > >> >>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>> Thanks, > > > > >> >>>>>>>>>>>>>>>>>>>> Ash > > > > >> >>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> On 26 Jul 2019, at 09:14, Jarek Potiuk > > > > >> >>>>>>>>>>>>>>>>> <jarek.pot...@polidea.com> > > > > >> >>>>>>>>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> I would like to recast the vote. Let's start > > > from > > > > 0 > > > > >> >> :) > > > > >> >>>> . I > > > > >> >>>>>>>>>>>>> would > > > > >> >>>>>>>>>>>>>>>>> like > > > > >> >>>>>>>>>>>>>>>>>>> to > > > > >> >>>>>>>>>>>>>>>>>>>>> keep the July 30th 6pm CEST date, and at > least > > > > three > > > > >> >>> +1 > > > > >> >>>>>>>>>>>>>>>>> (binding) > > > > >> >>>>>>>>>>>>>>>>>>> votes > > > > >> >>>>>>>>>>>>>>>>>>>> are > > > > >> >>>>>>>>>>>>>>>>>>>>> needed to pass. I marked in red the > > > modifications > > > > to > > > > >> >>> the > > > > >> >>>>>>>>>>>>>>>>> previous > > > > >> >>>>>>>>>>>>>>>>>>>> proposal. > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> Here is the proposal (details here > > > > >> >>>>>>>>>>>>>>>>>>>>> < > > > > >> >>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit#heading=h.j4jc3i70qszo > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> ) > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 1: A: Get rid of contrib,* > > > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 2: A: > > > > >> >>> Airflow.operators.foo_operator.FooOperator > > > > >> >>>>>>>>>>>>> could > > > > >> >>>>>>>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator* > > > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 3: C: > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>> > > > > airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator > > > > >> >>>>>>>>>>>>>>>>>>> could > > > > >> >>>>>>>>>>>>>>>>>>>>> become > > > > >> >>> airflow.gcp.operators.bigtable.BigTableOperator* > > > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction* > > > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor") > > > > >> >> suffixes > > > > >> >>> on > > > > >> >>>>>>>>>>>>> class > > > > >> >>>>>>>>>>>>>>>>>>> names* > > > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on > > > > >> >>> case-by-case > > > > >> >>>>>>>>>>>>> (and > > > > >> >>>>>>>>>>>>>>>>> my team > > > > >> >>>>>>>>>>>>>>>>>>>> can > > > > >> >>>>>>>>>>>>>>>>>>>>> do the job of GCP-related operators)* > > > > >> >>>>>>>>>>>>>>>>>>>>> - *Case 7: Native python solution (with > better > > > IDE > > > > >> >>>>>>>>>>>>>>>>> integration)* > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> This is my binding (+1) vote. > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 10:08 AM Jarek > Potiuk > > < > > > > >> >>>>>>>>>>>>>>>>>>> jarek.pot...@polidea.com> > > > > >> >>>>>>>>>>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> Hey Felix, > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> For 7* -> I am in favour of trying the > native > > > one > > > > >> >> as > > > > >> >>>> well > > > > >> >>>>>>>>>>> (I > > > > >> >>>>>>>>>>>>>>>>> use IDE > > > > >> >>>>>>>>>>>>>>>>>>>> quite > > > > >> >>>>>>>>>>>>>>>>>>>>>> often ;) ) > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> J. > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 24, 2019 at 9:34 AM Driesprong, > > > Fokko > > > > >> >>>>>>>>>>>>>>>>>>> <fo...@driesprong.frl > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> wrote: > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> Thanks Kamil for putting the document > > > together. > > > > I > > > > >> >>>> wasn't > > > > >> >>>>>>>>>>>>> able > > > > >> >>>>>>>>>>>>>>>>> to > > > > >> >>>>>>>>>>>>>>>>>>>> respond > > > > >> >>>>>>>>>>>>>>>>>>>>>>> earlier since I was giving Airflow > workshops > > > > >> >>>> throughout > > > > >> >>>>>>>>>>>>> Europe > > > > >> >>>>>>>>>>>>>>>>> :-) > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 1: *Solution A → Remove everything > > from > > > > >> >>> contrib > > > > >> >>>>>>>>>>> into > > > > >> >>>>>>>>>>>>> a > > > > >> >>>>>>>>>>>>>>>>> single > > > > >> >>>>>>>>>>>>>>>>>>>>>>> package. > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> To make it easier to the end-user, my > > > preference > > > > >> >>>> would be > > > > >> >>>>>>>>>>>>> to > > > > >> >>>>>>>>>>>>>>>>> get > > > > >> >>>>>>>>>>>>>>>>>>> rid of > > > > >> >>>>>>>>>>>>>>>>>>>>>>> the > > > > >> >>>>>>>>>>>>>>>>>>>>>>> contrib package altogether. What's in > > contrib > > > or > > > > >> >> not > > > > >> >>>> is a > > > > >> >>>>>>>>>>>>>>>>> tricky > > > > >> >>>>>>>>>>>>>>>>>>>> question, > > > > >> >>>>>>>>>>>>>>>>>>>>>>> I think this is already reflected in the > > > > document > > > > >> >>>> since > > > > >> >>>>>>>>>>>>>>>>> "maturity" > > > > >> >>>>>>>>>>>>>>>>>>> is > > > > >> >>>>>>>>>>>>>>>>>>>> in > > > > >> >>>>>>>>>>>>>>>>>>>>>>> quotes. What would be the moment to > transfer > > > the > > > > >> >>>> operator > > > > >> >>>>>>>>>>>>> to > > > > >> >>>>>>>>>>>>>>>>> the > > > > >> >>>>>>>>>>>>>>>>>>> core > > > > >> >>>>>>>>>>>>>>>>>>>> or > > > > >> >>>>>>>>>>>>>>>>>>>>>>> vice versa? I'm afraid that renaming > contrib > > > to > > > > >> >>>>>>>>>>> incubating > > > > >> >>>>>>>>>>>>>>>>> will > > > > >> >>>>>>>>>>>>>>>>>>> confuse > > > > >> >>>>>>>>>>>>>>>>>>>>>>> the > > > > >> >>>>>>>>>>>>>>>>>>>>>>> user even more. For people who aren't as > > known > > > > >> >> with > > > > >> >>>>>>>>>>>>> Airflow it > > > > >> >>>>>>>>>>>>>>>>> isn't > > > > >> >>>>>>>>>>>>>>>>>>>> that > > > > >> >>>>>>>>>>>>>>>>>>>>>>> transparent which operator lives where, > > > > especially > > > > >> >>> if > > > > >> >>>> you > > > > >> >>>>>>>>>>>>>>>>> don't use > > > > >> >>>>>>>>>>>>>>>>>>> an > > > > >> >>>>>>>>>>>>>>>>>>>> IDE > > > > >> >>>>>>>>>>>>>>>>>>>>>>> such as Ash ;) Besides that I don't think > it > > > is > > > > >> >>> worth > > > > >> >>>>>>>>>>>>> changing > > > > >> >>>>>>>>>>>>>>>>> the > > > > >> >>>>>>>>>>>>>>>>>>>> public > > > > >> >>>>>>>>>>>>>>>>>>>>>>> API just for a different name. > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> Ok. I am convinced. I will re-cast the vote > > > with > > > > >> >>> this. > > > > >> >>>> It > > > > >> >>>>>>>>>>>>> will > > > > >> >>>>>>>>>>>>>>>>> make > > > > >> >>>>>>>>>>>>>>>>>>>> easier > > > > >> >>>>>>>>>>>>>>>>>>>>>> as we will not have to define other rules > as > > > > those > > > > >> >>>> that we > > > > >> >>>>>>>>>>>>>>>>> already > > > > >> >>>>>>>>>>>>>>>>>>> have. > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 2:* Slight preference to Solution B > > > (let's > > > > >> >>> say a > > > > >> >>>>>>>>>>> +0) > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> I like to search on Github with t, and > then > > > > search > > > > >> >>> for > > > > >> >>>>>>>>>>>>>>>>> BashOperator. > > > > >> >>>>>>>>>>>>>>>>>>>> After > > > > >> >>>>>>>>>>>>>>>>>>>>>>> the change, you should search for > > > Operator/Bash. > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> Jarek, I'm confused with your vote on > this, > > > from > > > > >> >>> your > > > > >> >>>>>>>>>>> mail > > > > >> >>>>>>>>>>>>> you > > > > >> >>>>>>>>>>>>>>>>>>> state: - > > > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 2: B: > > > > >> >>> Airflow.operators.foo_operator.FooOperator > > > > >> >>>>>>>>>>>>> could > > > > >> >>>>>>>>>>>>>>>>> become > > > > >> >>>>>>>>>>>>>>>>>>>>>>> airflow.operators.foo.FooOperator*. But > the > > B > > > > >> >>>> solution is > > > > >> >>>>>>>>>>>>>>>>> keeping it > > > > >> >>>>>>>>>>>>>>>>>>>> the > > > > >> >>>>>>>>>>>>>>>>>>>>>>> same, so that would mean - *Case 2: B: > > > > >> >>>>>>>>>>>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator > > > > *would > > > > >> >>> stay > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > airflow.operators.foo_operator*.FooOperator* > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> My mistake. It was of course A. I think > > there > > > > is a > > > > >> >>>> little > > > > >> >>>>>>>>>>>>>>>>> confusion > > > > >> >>>>>>>>>>>>>>>>>>>> here. > > > > >> >>>>>>>>>>>>>>>>>>>>>> This case is not for changing BashOperator > > into > > > > >> >> Bash > > > > >> >>>> but > > > > >> >>>>>>>>>>> for > > > > >> >>>>>>>>>>>>>>>>> changing > > > > >> >>>>>>>>>>>>>>>>>>>> the > > > > >> >>>>>>>>>>>>>>>>>>>>>> *module* name not the *class* name. So > > > > >> >>>>>>>>>>>>>>>>> bash_operator.BashOperator -> > > > > >> >>>>>>>>>>>>>>>>>>>>>> bash.BashOperator :) . you will still > search > > > for > > > > >> >>>>>>>>>>>>> BashOperator > > > > >> >>>>>>>>>>>>>>>>> in this > > > > >> >>>>>>>>>>>>>>>>>>>> case. > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 3:* I'm leaning towards B > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> Mostly because it isn't as trivial to > decide > > > > when > > > > >> >> it > > > > >> >>>> gets > > > > >> >>>>>>>>>>>>> its > > > > >> >>>>>>>>>>>>>>>>> own > > > > >> >>>>>>>>>>>>>>>>>>>> package > > > > >> >>>>>>>>>>>>>>>>>>>>>>> or not. Besides the cloud providers, there > > are > > > > >> >>>> companies > > > > >> >>>>>>>>>>>>> like > > > > >> >>>>>>>>>>>>>>>>>>>> Databricks > > > > >> >>>>>>>>>>>>>>>>>>>>>>> and Qubole which might also end up with > > their > > > > own > > > > >> >>>> package > > > > >> >>>>>>>>>>>>>>>>> because > > > > >> >>>>>>>>>>>>>>>>>>> their > > > > >> >>>>>>>>>>>>>>>>>>>>>>> integration is getting richer over time. > > > Moving > > > > >> >> this > > > > >> >>>> is a > > > > >> >>>>>>>>>>>>> bit > > > > >> >>>>>>>>>>>>>>>>>>> painful > > > > >> >>>>>>>>>>>>>>>>>>>>>>> because we need to deprecate the old path > > and > > > > move > > > > >> >>>>>>>>>>>>> everything > > > > >> >>>>>>>>>>>>>>>>> which > > > > >> >>>>>>>>>>>>>>>>>>>>>>> introduces noise into version control etc. > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> I'd say, we should go for C. As I proposed. > > > It's > > > > >> >> not > > > > >> >>> as > > > > >> >>>>>>>>>>>>>>>>> difficult as > > > > >> >>>>>>>>>>>>>>>>>>> it > > > > >> >>>>>>>>>>>>>>>>>>>>>> seems to group operators in packages and it > > has > > > > >> >>>> numerous > > > > >> >>>>>>>>>>>>>>>>> advantages. > > > > >> >>>>>>>>>>>>>>>>>>> The > > > > >> >>>>>>>>>>>>>>>>>>>>>> nice thing about deprecations - we can do > > them > > > in > > > > >> >> the > > > > >> >>>>>>>>>>>>>>>>> __init__.py of > > > > >> >>>>>>>>>>>>>>>>>>> the > > > > >> >>>>>>>>>>>>>>>>>>>>>> packages which causes the noise fairly > small > > > (Git > > > > >> >>> will > > > > >> >>>>>>>>>>>>> actually > > > > >> >>>>>>>>>>>>>>>>>>> realise > > > > >> >>>>>>>>>>>>>>>>>>>>>> that the file has been moved and you can > > follow > > > > >> >> that. > > > > >> >>>>>>>>>>> Maybe > > > > >> >>>>>>>>>>>>> not > > > > >> >>>>>>>>>>>>>>>>> as > > > > >> >>>>>>>>>>>>>>>>>>> super > > > > >> >>>>>>>>>>>>>>>>>>>>>> easy to merge changes later to 1.10.4 but > > it's > > > > >> >> not a > > > > >> >>>>>>>>>>> rocket > > > > >> >>>>>>>>>>>>>>>>> science > > > > >> >>>>>>>>>>>>>>>>>>>> either. > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> *Case 7:* Python native solution > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> Yes. > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> --- > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> Apart from all, great work! > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> Cheers, Fokko > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> Op di 23 jul. 2019 om 21:27 schreef Felix > > > > >> >> Uellendall > > > > >> >>>>>>>>>>>>>>>>>>>>>>> <felue...@pm.me.invalid > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> : > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> I have no strong "No" against any > proposed > > > > change > > > > >> >>> of > > > > >> >>>>>>>>>>> these > > > > >> >>>>>>>>>>>>>>>>> cases. > > > > >> >>>>>>>>>>>>>>>>>>> So I > > > > >> >>>>>>>>>>>>>>>>>>>>>>> go > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> with +1 (non-binding). > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> P.S. Thanks Jarek for bringing this up > > again > > > > and > > > > >> >>> your > > > > >> >>>>>>>>>>>>> intense > > > > >> >>>>>>>>>>>>>>>>> work > > > > >> >>>>>>>>>>>>>>>>>>>>>>> towards > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> airflow currently :) and thanks to Kamil > > for > > > > even > > > > >> >>>>>>>>>>> creating > > > > >> >>>>>>>>>>>>>>>>> this > > > > >> >>>>>>>>>>>>>>>>>>>>>>> document. I > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> like how the code is getting more and > more > > > > >> >>> consistent > > > > >> >>>>>>>>>>> and > > > > >> >>>>>>>>>>>>>>>>> clean :) > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> Kind regards, > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> Felix > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> Sent from ProtonMail mobile > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> -------- Original Message -------- > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> On Jul 23, 2019, 18:34, Jarek Potiuk > wrote: > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone, > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> This email is calling a vote on the > > changes > > > in > > > > >> >>>> import > > > > >> >>>>>>>>>>>>> paths. > > > > >> >>>>>>>>>>>>>>>>> It's > > > > >> >>>>>>>>>>>>>>>>>>>> been > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> discussed in > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://lists.apache.org/thread.html/4e648d9421c792d4537f5ac66f1a16dce468f816fc5221a9f9db9433@%3Cdev.airflow.apache.org%3E > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> The vote will last for at least 1 week > > (July > > > > >> >> 30th > > > > >> >>>> 6pm > > > > >> >>>>>>>>>>>>> CEST), > > > > >> >>>>>>>>>>>>>>>>> and > > > > >> >>>>>>>>>>>>>>>>>>> at > > > > >> >>>>>>>>>>>>>>>>>>>>>>> least > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> three +1 (binding) votes have been cast. > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> The proposal to vote is based on the > > > document > > > > >> >> from > > > > >> >>>>>>>>>>> Kamil > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> < > > > > >> >>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit# > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> The proposed solution is: > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 1: B: Contrib vs not: we move > all > > > that > > > > >> >> are > > > > >> >>>>>>>>>>> "well" > > > > >> >>>>>>>>>>>>>>>>> tested > > > > >> >>>>>>>>>>>>>>>>>>> and > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> rename contrib to "incubating" or > > similar.* > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 2: B: > > > > >> >>>>>>>>>>> Airflow.operators.foo_operator.FooOperator > > > > >> >>>>>>>>>>>>>>>>> could > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> become > airflow.operators.foo.FooOperator* > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 3: C: > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>> > > > > airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator > > > > >> >>>>>>>>>>>>>>>>>>>> could > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> become > > > > >> >>>> airflow.gcp.operators.bigtable.BigTableOperator* > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction* > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and > > "Sensor") > > > > >> >>>> suffixes > > > > >> >>>>>>>>>>> on > > > > >> >>>>>>>>>>>>>>>>> class > > > > >> >>>>>>>>>>>>>>>>>>> names* > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases > on > > > > >> >>>> case-by-case > > > > >> >>>>>>>>>>>>> (and > > > > >> >>>>>>>>>>>>>>>>> my > > > > >> >>>>>>>>>>>>>>>>>>> team > > > > >> >>>>>>>>>>>>>>>>>>>>>>> can > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> do the job of GCP-related operators)* > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> This is my (binding) +1 vote. > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> Best regards, > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> J. > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> -- > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> Jarek Potiuk > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | > > > > Principal > > > > >> >>>>>>>>>>> Software > > > > >> >>>>>>>>>>>>>>>>> Engineer > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> M: [+48 660 796 129](tel:+48660796129) > > > > >> >>>>>>>>>>>>>>>>>>>>>>> <[+48660796129](tel:+48660796129)> > > > > >> >>>>>>>>>>>>>>>>>>>>>>>>> [image: Polidea] < > > https://www.polidea.com/> > > > > >> >>>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> -- > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> Jarek Potiuk > > > > >> >>>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | > > Principal > > > > >> >>>> Software > > > > >> >>>>>>>>>>>>>>>>> Engineer > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> M: +48 660 796 129 <+48660796129> > > > > >> >>>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/ > > > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> -- > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> Jarek Potiuk > > > > >> >>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | > > Principal > > > > >> >>> Software > > > > >> >>>>>>>>>>>>>> Engineer > > > > >> >>>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>>> M: +48 660 796 129 <+48660796129> > > > > >> >>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> > > > > >> >>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>>> -- > > > > >> >>>>>>>>>>>>>>>>>> *Kaxil Naik* > > > > >> >>>>>>>>>>>>>>>>>> *Big Data Consultant | DevOps Data Engineer* > > > > >> >>>>>>>>>>>>>>>>>> *Certified *Google Cloud Data Engineer | > > > *Certified* > > > > >> >>> Apache > > > > >> >>>>>>>>>>> Spark > > > > >> >>>>>>>>>>>>> & > > > > >> >>>>>>>>>>>>>>>>> Neo4j > > > > >> >>>>>>>>>>>>>>>>>> Developer > > > > >> >>>>>>>>>>>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil > > > > >> >>>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>>>> -- > > > > >> >>>>>>>>>>>>>>>>> *Kaxil Naik* > > > > >> >>>>>>>>>>>>>>>>> *Big Data Consultant | DevOps Data Engineer* > > > > >> >>>>>>>>>>>>>>>>> *Certified *Google Cloud Data Engineer | > > *Certified* > > > > >> >>> Apache > > > > >> >>>>>>>> Spark > > > > >> >>>>>>>>>>> & > > > > >> >>>>>>>>>>>>>>>>> Neo4j > > > > >> >>>>>>>>>>>>>>>>> Developer > > > > >> >>>>>>>>>>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil > > > > >> >>>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>>> > > > > >> >>>>>>>>>>>>> > > > > >> >>>>>>>>>>>> > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> -- > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> Jarek Potiuk > > > > >> >>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal > Software > > > > >> >>> Engineer > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>>>> M: +48 660 796 129 <+48660796129> > > > > >> >>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> > > > > >> >>>>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>>> > > > > >> >>>>>>>> > > > > >> >>>>>>>> -- > > > > >> >>>>>>>> *Kaxil Naik* > > > > >> >>>>>>>> *Big Data Consultant | DevOps Data Engineer* > > > > >> >>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* > Apache > > > > >> >> Spark & > > > > >> >>>> Neo4j > > > > >> >>>>>>>> Developer > > > > >> >>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil > > > > >> >>>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>>> -- > > > > >> >>>>>>> > > > > >> >>>>>>> Jarek Potiuk > > > > >> >>>>>>> Polidea <https://www.polidea.com/> | Principal Software > > > > Engineer > > > > >> >>>>>>> > > > > >> >>>>>>> M: +48 660 796 129 <+48660796129> > > > > >> >>>>>>> [image: Polidea] <https://www.polidea.com/> > > > > >> >>>>>>> > > > > >> >>>>>>> > > > > >> >>>>>> > > > > >> >>>>>> -- > > > > >> >>>>>> > > > > >> >>>>>> Jarek Potiuk > > > > >> >>>>>> Polidea <https://www.polidea.com/> | Principal Software > > > Engineer > > > > >> >>>>>> > > > > >> >>>>>> M: +48 660 796 129 <+48660796129> > > > > >> >>>>>> [image: Polidea] <https://www.polidea.com/> > > > > >> >>>>> > > > > >> >>>> > > > > >> >>> > > > > >> >> > > > > >> > > > > >> > > > > > > > > -- > > > > > > > > Jarek Potiuk > > > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > > > > > M: +48 660 796 129 <+48660796129> > > > > [image: Polidea] <https://www.polidea.com/> > > > > > > > > > > > > > -- > > > > Jarek Potiuk > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > M: +48 660 796 129 <+48660796129> > > [image: Polidea] <https://www.polidea.com/> > > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
