This is a good idea for now. I'm also not overly concerned about these few non-cloud examples - FTPtoS3Operator can stay where it is and doesn't have to move under 'aws.' to my mind.
Longer term I'd like to go back to making the "transfer/copy/transform" operators "composable" so that we can have a single DataCopyOperator, and give it a source and destination, and it uses hooks to do the work. This is not a small undertaking though to do well and efficiently though. -ash > On 30 Jul 2019, at 20:54, Tomasz Urbaszek <tomasz.urbas...@polidea.com> wrote: > > Maybe we can put all those AtoB operators under one name like “transfer”, > then it would be easier to look for such operator? > > Best, > Tomek > > On Tue, 30 Jul 2019 at 21:39, Chen Tong <cix...@gmail.com> wrote: > >> Daniel mentioned a good point. Such composed operator may also involves >> both cloud and non-cloud provider saying FTPtoS3Operator. Should it in AWS >> folder? >> >> Also, saying in the future, another cloud provider is growing large enough. >> Will we rename all plugins related to this provider? What's the criteria we >> say it's big enough? And option D gives an impression like these tools may >> be maintained and supported by the Cloud provider. I am not sure if it will >> be a truth or not. >> >> Best, >> >> >> On Tue, Jul 30, 2019 at 11:18 AM Daniel Standish <dpstand...@gmail.com> >> wrote: >> >>> One wrinkle with case 3+4 option D is inter-provider operators. Mainly >>> it's storage I think e.g. XToS3Operator or XToGCSOperator where the X is >> a >>> service some different provider. >>> >>> Maybe the rule should be to locate the operator according to the first >>> provider referenced. So e.g. s3_to_gcs_transfer_operator.py would go to >>> aws. >>> >>> >>> On Tue, Jul 30, 2019 at 6:21 AM Kamil Breguła <kamil.breg...@polidea.com >>> >>> wrote: >>> >>>> Yes. All changes will be backwards compatible. In the case of using >>>> the old path, a message containing a proposal for change will be >>>> reported to the user. >>>> >>>> I prepared an example of how to change the name of a class in a case >>>> with the use of a native solution. >>>> Source code: >>>> >> https://github.com/mik-laj/airflow-deprecation-sample/tree/master/rename >>>> Preview from IDE: https://imgur.com/a/45NxS5W >>>> >>>> On Tue, Jul 30, 2019 at 2:28 PM Ash Berlin-Taylor <a...@apache.org> >>> wrote: >>>>> >>>>> Just cos I'm not sure it's _explicitly_ stated, but all of the moves >>>> will have a deprecation of the old name right? >>>>> >>>>> 3+4 case D gets my vote too. >>>>> >>>>> -a >>>>> >>>>>> On 30 Jul 2019, at 09:58, Jarek Potiuk <jarek.pot...@polidea.com> >>>> wrote: >>>>>> >>>>>> I went ahead and updated the page (just to speed it up) as I think >> it >>>>>> really makes sense to join those two cases (and I do not see any >>>> drawbacks >>>>>> - I think the options we have cover all possible approaches) and we >>> can >>>>>> always go back if we need to. >>>>>> >>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal >>>>>> >>>>>> My vote is *D*. >>>>>> >>>>>> - airflow/contrib/operators/gcp_bigtable_operator.py → >>> airflow/gcp >>>>>> /operators/bigtable_operator.py >>>>>> - airflow/contrib/operators/ssh_operator.py -> >>>>>> airflow/operators/ssh_operator.py >>>>>> >>>>>> I updated the page with my vote. >>>>>> I propose that we vote till Friday on that one (all the rest is >>> already >>>>>> agreed). >>>>>> >>>>>> J. >>>>>> >>>>>> >>>>>> On Tue, Jul 30, 2019 at 9:42 AM Jarek Potiuk < >>> jarek.pot...@polidea.com >>>>> >>>>>> wrote: >>>>>> >>>>>>> I think almost everyone voted and we have almost perfect >> consensus. >>>> We all >>>>>>> agree amongst other on moving all operators out of contrib >> (Great!). >>>>>>> >>>>>>> The only doubts are for *Case 3* (Cloud provider prefix) and *Case >>> 4* >>>>>>> (Using Namespaces). >>>>>>> I think there was actually an overlap between those two. Also >> Ash's >>>>>>> comment/veto on that is quite clear. >>>>>>> So I have final (I hope!) proposal to join those two cases and >> have >>>> one >>>>>>> voting (till Friday) only on that. >>>>>>> >>>>>>> I will update the doc if you all agree with me that makes more >> sense >>>> as >>>>>>> single case to vote on: >>>>>>> >>>>>>> *[Case 3 + Case 4]: Grouping Cloud Provider operators.* >>>>>>> >>>>>>> *Option A*. Keep operators/sensors/hooks in >>> airflow/operators(sensors, >>>>>>> hooks) and keep/add prefixes in file names. >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/sns_publish_operator.py becomes >>>>>>> airflow/operators/**aws_sns_publish_operator.py* >>>>>>> >>>>>>> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* >>>>>>> becomes *airflow/operators/gcp_dataproc_operator.py* >>>>>>> >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/hooks/gcp_bigtable_hook.py *becomes *airflow/* >>>>>>> *hooks/gcp_bigtable_hook.py* >>>>>>> - >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* >>>>>>> *operators/ssh_operator.py* >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Option B*. Group operators/sensors/hooks in >>>> airflow/operators(sensors, >>>>>>> hooks)/*<PROVIDER>.* Non-cloud provider ones are moved to >>>>>>> airflow/operators(sensors/hooks), Drop the prefix. >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/sns_publish_operator.py >>>>>>> becomes airflow/operators/**aws/sns_publish_operator.py* >>>>>>> >>>>>>> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* >>>>>>> becomes *airflow/operators/gcp/dataproc_operator.py* >>>>>>> >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py *becomes >>>> *airflow/* >>>>>>> *operators/gcp/bigtable_operator.py* >>>>>>> - >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* >>>>>>> *operators/ssh_operator.py* >>>>>>> >>>>>>> >>>>>>> *Option C*. Group operators/sensors/hooks in >>>> airflow/operators(sensors, >>>>>>> hooks)*/<PROVIDER>.* Non-cloud provider ones are moved to >>>>>>> airflow/operators(sensors/hooks), Keep the prefix. >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/sns_publish_operator.py >>>>>>> becomes airflow/operators/**aws/aws_sns_publish_operator.py* >>>>>>> >>>>>>> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* >>>>>>> becomes *airflow/operators/gcp/gcp_dataproc_operator.py* >>>>>>> >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py *becomes >>>> *airflow/* >>>>>>> *operators/gcp/gcp_bigtable_operator.py* >>>>>>> - >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* >>>>>>> *operators/ssh_operator.py* >>>>>>> >>>>>>> >>>>>>> *Option D.* Group operators/sensors/hooks in >>>> *airflow/<PROVIDER>*/operators(sensors, >>>>>>> hooks). Non-cloud provider ones are moved to >>>>>>> airflow/operators(sensors/hooks). Drop the prefix. >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/sns_publish_operator.py >>>>>>> becomes airflow/aws/operators/**sns_publish_operator.py* >>>>>>> >>>>>>> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* >>>>>>> becomes *airflow/gcp/operators/dataproc_operator.py* >>>>>>> >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py *becomes >>>> *airflow/* >>>>>>> *gcp/operators/bigtable_operator.py* >>>>>>> - >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* >>>>>>> *operators/ssh_operator.py* >>>>>>> >>>>>>> >>>>>>> *Option E.* Group operators/sensors/hooks in >>>> *airflow/<PROVIDER>*/operators(sensors, >>>>>>> hooks). Non-cloud provider ones are moved to >>>>>>> airflow/operators(sensors/hooks).* Keep the prefix*. >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/sns_publish_operator.py >>>>>>> becomes airflow/aws/operators/aws_**sns_publish_operator.py* >>>>>>> >>>>>>> >>>>>>> - *airflow/contrib/operators/dataproc_operator.py* >>>>>>> becomes *airflow/gcp/operators/gcp_dataproc_operator.py* >>>>>>> >>>>>>> >>>>>>> - >>>>>>> *airflow/contrib/operators/gcp_bigtable_operator.py *becomes >>>> *airflow/* >>>>>>> *gcp/operators/gcp_bigtable_operator.py* >>>>>>> - >>>>>>> *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* >>>>>>> *operators/ssh_operator.py* >>>>>>> >>>>>>> >>>>>>> Shall I proceed with updating the page ? >>>>>>> >>>>>>> >>>>>>> J. >>>>>>> >>>>>>> >>>>>>> On Mon, Jul 29, 2019 at 11:18 AM Kaxil Naik <kaxiln...@gmail.com> >>>> wrote: >>>>>>> >>>>>>>> Thanks Jarek, adding my vote. >>>>>>>> >>>>>>>> On Mon, Jul 29, 2019 at 2:40 PM Ash Berlin-Taylor < >> a...@apache.org> >>>> wrote: >>>>>>>> >>>>>>>>> Thanks Jarek, much clearer. I have voted there too. >>>>>>>>> >>>>>>>>> -ash >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 29 Jul 2019, at 08:13, Driesprong, Fokko >> <fo...@driesprong.frl >>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Thanks Jarek, the Wiki works much better for me. I've added my >>> vote >>>>>>>> there >>>>>>>>>> as well. >>>>>>>>>> >>>>>>>>>> You've convinced me on the second case. I've changed my vote on >>>> Case 2 >>>>>>>>> from >>>>>>>>>> A to B :-) >>>>>>>>>> >>>>>>>>>> Maybe we should leave the vote open a bit longer since it >>> involves >>>>>>>> quite >>>>>>>>>> some reading, and I would like to get some proper feedback and >>>>>>>> opinions >>>>>>>>> on >>>>>>>>>> this. Plus I only see two votes right now. If we don't think >> this >>>>>>>>> through, >>>>>>>>>> it might be that we're having this vote again in the near >> future >>>> :-) >>>>>>>>>> >>>>>>>>>> Cheers, Fokko >>>>>>>>>> >>>>>>>>>> Op zo 28 jul. 2019 om 10:49 schreef Jarek Potiuk < >>>>>>>>> jarek.pot...@polidea.com>: >>>>>>>>>> >>>>>>>>>>> I updated the AIP-21 proposal in the way that should answer >> all >>>> the >>>>>>>>>>> concerns in this thread. >>>>>>>>>>> >>>>>>>>>>> NOTE! There is an action for all of those who are interested: >>>> Please >>>>>>>>>>> fill-in your voting in the voting table till Tuesday 30th of >>> July >>>> 6pm >>>>>>>>> CEST >>>>>>>>>>> (5pm BST, 9 am PST). >>>>>>>>>>> >>>>>>>>>>> I created a summary of the proposal >>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal >>>>>>>>>>>> >>>>>>>>>>> in table form - which contains also examples for each case. >>>>>>>>>>> >>>>>>>>>>> I added a voting table >>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Voting >>>>>>>>>>>> >>>>>>>>>>> where you should add your votes: >>>>>>>>>>> >>>>>>>>>>> I propose this voting mechanism: >>>>>>>>>>> >>>>>>>>>>> Voting will take place till *Tuesday* 30 Jul 2019 6pm CEST (5 >>> pm >>>>>>>> BST, >>>>>>>>> 9am >>>>>>>>>>> PST) . >>>>>>>>>>> >>>>>>>>>>> After the choice, the final consistent set of choices will be >>>>>>>> announced >>>>>>>>>>> (taking into account majority of binding vote, also including >>>>>>>> potential >>>>>>>>>>> vetos and consistency between the choices. Non-binding votes >>> will >>>> be >>>>>>>>> taken >>>>>>>>>>> into account in case there is a draw. The final set of choices >>>> will >>>>>>>> be >>>>>>>>>>> announced at devlist thread >>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://lists.apache.org/thread.html/4cedc94bee53ad908eee8333a56b58be8b5641881e73f69b97e436a9@%3Cdev.airflow.apache.org%3E >>>>>>>>>>>> >>>>>>>>>>> after >>>>>>>>>>> the voting completes. >>>>>>>>>>> >>>>>>>>>>> Unless there is a veto raised to the final proposal, the final >>>>>>>> proposal >>>>>>>>> is >>>>>>>>>>> accepted by Lazy Consensus >>>>>>>>>>> <https://community.apache.org/committers/lazyConsensus.html> >>> on >>>>>>>>> *Friday* >>>>>>>>>>> 02 >>>>>>>>>>> Aug 2019 at 6pm CEST (5 pm BST, 9am PST). >>>>>>>>>>> >>>>>>>>>>> Please let me know if what I proposed can be further improved >> to >>>>>>>> avoid >>>>>>>>>>> chaos. >>>>>>>>>>> >>>>>>>>>>> J. >>>>>>>>>>> >>>>>>>>>>> On Sat, Jul 27, 2019 at 11:41 AM Jarek Potiuk < >>>>>>>> jarek.pot...@polidea.com >>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Ok. I see that indeed voting could have been organised a bit >>>> better >>>>>>>> - >>>>>>>>> dev >>>>>>>>>>>> list does not seem to cope well with such a complex case :). >>>>>>>>>>>> >>>>>>>>>>>> As Kamil noticed - we already have a formal AIP-21 >>>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths >>>>>>>>>>>> >>>>>>>>>>>> in the Wiki - which I should have mentioned before. I have >> not >>>> much >>>>>>>>> time >>>>>>>>>>>> today - but tomorrow I will try to summarise the options - >>> based >>>> on >>>>>>>>> some >>>>>>>>>>>> real code examples (to make it easier to digest). I think the >>>> best >>>>>>>>>>> approach >>>>>>>>>>>> will be to make a voting matrix in the AIP-21 Wiki page where >>>> people >>>>>>>>> will >>>>>>>>>>>> be able to state their preferences for each case (by adding a >>>> row to >>>>>>>>> the >>>>>>>>>>>> table). I think that might provide much better clarity and a >>>> single >>>>>>>>> place >>>>>>>>>>>> which will show the current aggregated state of voting. >>>>>>>>>>>> >>>>>>>>>>>> However - as Ash also stated - some of the options are >>>>>>>> inter-dependent >>>>>>>>> so >>>>>>>>>>>> at the end we will have to adjust the results in case they >> are >>>> not >>>>>>>>>>>> consistent - and make final voting on aggregated set of >> cases >>> - >>>>>>>> but I >>>>>>>>>>>> think in this case following "lasy consensus" - i,e. if noone >>>>>>>> objects >>>>>>>>> to >>>>>>>>>>>> final set of cases, we will proceed with it.. >>>>>>>>>>>> >>>>>>>>>>>> Do you think that will work better ? >>>>>>>>>>>> >>>>>>>>>>>> J. >>>>>>>>>>>> >>>>>>>>>>>> Principal Software Engineer >>>>>>>>>>>> Phone: +48660796129 >>>>>>>>>>>> >>>>>>>>>>>> sob., 27 lip 2019, 09:26 użytkownik Kamil Breguła < >>>>>>>>>>>> kamil.breg...@polidea.com> napisał: >>>>>>>>>>>> >>>>>>>>>>>>> Document is also available as AIP on wiki: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths >>>>>>>>>>>>> >>>>>>>>>>>>> On Sat, Jul 27, 2019, 9:07 AM Driesprong, Fokko >>>>>>>> <fo...@driesprong.frl >>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I agree with all, this is a bit chaotic. For the sake of >>>> clarity, >>>>>>>> I >>>>>>>>>>>>> would >>>>>>>>>>>>>> suggest moving the Google Doc to the Wiki as an AIP. >> Create a >>>>>>>>>>> structural >>>>>>>>>>>>>> code improvement AIP and then and add a matrix of >> users/cases >>>>>>>> where >>>>>>>>>>>>>> everybody can add his vote per case. This gives also the >>>>>>>> opportunity >>>>>>>>>>> for >>>>>>>>>>>>>> changing your vote on a specific case. WDYT? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Cheers, Fokko >>>>>>>>>>>>>> >>>>>>>>>>>>>> Op vr 26 jul. 2019 om 22:28 schreef Chen Tong < >>>> cix...@gmail.com>: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree with Ash. It is clear to vote on each case rather >>> than >>>>>>>> all >>>>>>>>>>>>>>> together. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 3:54 PM Ash Berlin-Taylor < >>>>>>>> a...@apache.org> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> A number of proposals overlap or add on to each other, >> so I >>>>>>>> think >>>>>>>>>>>>> (?) a >>>>>>>>>>>>>>>> single vote makes sense. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> To be clear, my vote is -1 until it's clear exactly what >> we >>>> are >>>>>>>>>>>>> voting >>>>>>>>>>>>>> on. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 26 July 2019 20:39:56 BST, Kaxil Naik < >>>> kaxiln...@gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> I agree with Ash and instead of voting on all 7 Cases >>>> together, >>>>>>>>>>> how >>>>>>>>>>>>>>>>> about >>>>>>>>>>>>>>>>> separate vote emails for all the cases? Each email can >>> have >>>> the >>>>>>>>>>>>>>>>> description >>>>>>>>>>>>>>>>> copied over from the doc. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> It would probably be a bit easier to track a decision in >>> the >>>>>>>>>>> future >>>>>>>>>>>>> as >>>>>>>>>>>>>>>>> well. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> What do you guys think? @Jarek Potiuk < >>>>>>>> jarek.pot...@polidea.com> >>>>>>>>>>>>>>>>> @Driesprong, >>>>>>>>>>>>>>>>> Fokko <fo...@driesprong.frl> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Sat, Jul 27, 2019 at 1:03 AM Kaxil Naik < >>>>>>>> kaxiln...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> *Case #1* airflow.contrib.{resources} : *Solution A * >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> *Case #2* git *_{operator/sensor}{/s}.py : *Solution A >> * >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> *Case #3* {aws/azure/gcp}_* : *Solution C* >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> *Case #4* Separate namespace for resources : >>>>>>>>>>>>>>>>>> *Solution A* >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> *Case #5* *Operator : *Solution A* >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> *Case #7* : *Solution A* >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 11:09 PM Daniel Standish >>>>>>>>>>>>>>>>> <dpstand...@gmail.com> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I have tried to add some clarification to Jarek's >>> summary, >>>>>>>> but >>>>>>>>>>> I >>>>>>>>>>>>> am >>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>> little fuzzy on exact proposal for case 3 so hopefully >>>> Jarek >>>>>>>>>>> can >>>>>>>>>>>>>>>>> further >>>>>>>>>>>>>>>>>>> clarify. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> According to my reading, in this motion cases 4,5,6 >> are >>>> all >>>>>>>>>>>>> either >>>>>>>>>>>>>>>>>>> proposal >>>>>>>>>>>>>>>>>>> *rejections* or otherwise have no effect, so attention >>>> can be >>>>>>>>>>>>>>>>> focused on >>>>>>>>>>>>>>>>>>> cases 1,2,3 and 7. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> *Motion is to adopt the following:* >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Case 1 - Solution A - Get rid of contrib >>>>>>>>>>>>>>>>>>> All objects will be moved out of contrib folder and >> into >>>>>>>>>>>>> respective >>>>>>>>>>>>>>>>>>> non-contrib folder. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Case 2 - Solution A - Remove object type suffix from >>>> modules >>>>>>>>>>>>>>>>>>> Example: >>>>>>>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator becomes >>>>>>>>>>>>>>>>>>> airflow.operators.foo.FooOperator >>>>>>>>>>>>>>>>>>> Example: >>>>>>>>>>>>>>>>>>> Airflow.hooks.foo_hook.FooOperator becomes >>>>>>>>>>>>> airflow.hooks.foo.FooHook >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Case 3 - conventions for cloud provider etc prefixes >>>>>>>>>>>>>>>>>>> *I am not sure exactly what the proposal is.* >>>>>>>>>>>>>>>>>>> Example case is >>>>>>>>>>>>> airflow/contrib/operators/gcp_bigtable_operator.py >>>>>>>>>>>>>>>>>>> Since motion adopts case 1 solution A, we can assume >> it >>> is >>>>>>>> now >>>>>>>>>>>>> named >>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>> so: airflow/operators/gcp_bigtable_operator.py >>>>>>>>>>>>>>>>>>> So what is proposed for this case? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Case 4 - Solution B - *no change* >>>>>>>>>>>>>>>>>>> This proposal was about namespaces but the motion is >> to >>>>>>>> reject >>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Case 5 - Solution B - *no change* >>>>>>>>>>>>>>>>>>> The proposal was to remove class type suffix e.g. >> remove >>>> the >>>>>>>>>>>>>>>>> Operator in >>>>>>>>>>>>>>>>>>> FooOperator, but the motion is to reject this proposal >>> -- >>>>>>>> i.e. >>>>>>>>>>>>>>>>> motion is >>>>>>>>>>>>>>>>>>> no >>>>>>>>>>>>>>>>>>> change on this case, i.e. we resolve to keep >> "Operator" >>>> (and >>>>>>>>>>>>>>>>> "Sensor") >>>>>>>>>>>>>>>>>>> suffixes on class names >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Case 6 - *No change* >>>>>>>>>>>>>>>>>>> This case concerns object naming inconsistencies, but >>>> motion >>>>>>>>>>> does >>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>> enact >>>>>>>>>>>>>>>>>>> any specific proposal. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Case 7 - Solution A - use warnings.warn template for >>>>>>>>>>> deprecation >>>>>>>>>>>>>>>>> warnings >>>>>>>>>>>>>>>>>>> (instead of zope) >>>>>>>>>>>>>>>>>>> Example: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> # in old_package/old_mod.py >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> import warnings >>>>>>>>>>>>>>>>>>> from new_package.new_mod import * >>>>>>>>>>>>>>>>>>> warnings.warn("old_package.old_mod has moved to >>>>>>>>>>>>> new_package.new_mod. >>>>>>>>>>>>>>>>>>> Import >>>>>>>>>>>>>>>>>>> of " >>>>>>>>>>>>>>>>>>> "old_package.old_mod will become unsupported in >> version >>>> 2", >>>>>>>>>>>>>>>>>>> DeprecationWarning, 2) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Reference implementation here: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://github.com/mik-laj/airflow-deprecation-sample/tree/master/solution1 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> FWIW +1 (non-binding) on 1,2,7 -- unsure on 3. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I am very happy that this motion now gets rid of >>> contrib. >>>>>>>>>>> Thank >>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>> Sergio >>>>>>>>>>>>>>>>>>> for turning the tide with your effective argumentation >>> ;) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 3:16 AM Ash Berlin-Taylor < >>>>>>>>>>>>> a...@apache.org> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Could someone summarise what the proposed name >> changes >>>> are, >>>>>>>>>>>>> here, >>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>> thread? Pointing at a google doc and only giving >>> partial >>>>>>>>>>>>> examples >>>>>>>>>>>>>>>>> is 1) >>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>> bit difficult to quickly work out what we are voting >>> on, >>>> and >>>>>>>>>>> 2) >>>>>>>>>>>>>>>>> using a >>>>>>>>>>>>>>>>>>>> google doc isn't great for a longer term record. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Ash >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On 26 Jul 2019, at 09:14, Jarek Potiuk >>>>>>>>>>>>>>>>> <jarek.pot...@polidea.com> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I would like to recast the vote. Let's start from 0 >> :) >>>> . I >>>>>>>>>>>>> would >>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>> keep the July 30th 6pm CEST date, and at least three >>> +1 >>>>>>>>>>>>>>>>> (binding) >>>>>>>>>>>>>>>>>>> votes >>>>>>>>>>>>>>>>>>>> are >>>>>>>>>>>>>>>>>>>>> needed to pass. I marked in red the modifications to >>> the >>>>>>>>>>>>>>>>> previous >>>>>>>>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Here is the proposal (details here >>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit#heading=h.j4jc3i70qszo >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ) >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> - *Case 1: A: Get rid of contrib,* >>>>>>>>>>>>>>>>>>>>> - *Case 2: A: >>> Airflow.operators.foo_operator.FooOperator >>>>>>>>>>>>> could >>>>>>>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator* >>>>>>>>>>>>>>>>>>>>> - *Case 3: C: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>> airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator >>>>>>>>>>>>>>>>>>> could >>>>>>>>>>>>>>>>>>>>> become >>> airflow.gcp.operators.bigtable.BigTableOperator* >>>>>>>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction* >>>>>>>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor") >> suffixes >>> on >>>>>>>>>>>>> class >>>>>>>>>>>>>>>>>>> names* >>>>>>>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on >>> case-by-case >>>>>>>>>>>>> (and >>>>>>>>>>>>>>>>> my team >>>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>> do the job of GCP-related operators)* >>>>>>>>>>>>>>>>>>>>> - *Case 7: Native python solution (with better IDE >>>>>>>>>>>>>>>>> integration)* >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This is my binding (+1) vote. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Fri, Jul 26, 2019 at 10:08 AM Jarek Potiuk < >>>>>>>>>>>>>>>>>>> jarek.pot...@polidea.com> >>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hey Felix, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> For 7* -> I am in favour of trying the native one >> as >>>> well >>>>>>>>>>> (I >>>>>>>>>>>>>>>>> use IDE >>>>>>>>>>>>>>>>>>>> quite >>>>>>>>>>>>>>>>>>>>>> often ;) ) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> J. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 24, 2019 at 9:34 AM Driesprong, Fokko >>>>>>>>>>>>>>>>>>> <fo...@driesprong.frl >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks Kamil for putting the document together. I >>>> wasn't >>>>>>>>>>>>> able >>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>> respond >>>>>>>>>>>>>>>>>>>>>>> earlier since I was giving Airflow workshops >>>> throughout >>>>>>>>>>>>> Europe >>>>>>>>>>>>>>>>> :-) >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> *Case 1: *Solution A → Remove everything from >>> contrib >>>>>>>>>>> into >>>>>>>>>>>>> a >>>>>>>>>>>>>>>>> single >>>>>>>>>>>>>>>>>>>>>>> package. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> To make it easier to the end-user, my preference >>>> would be >>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> get >>>>>>>>>>>>>>>>>>> rid of >>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> contrib package altogether. What's in contrib or >> not >>>> is a >>>>>>>>>>>>>>>>> tricky >>>>>>>>>>>>>>>>>>>> question, >>>>>>>>>>>>>>>>>>>>>>> I think this is already reflected in the document >>>> since >>>>>>>>>>>>>>>>> "maturity" >>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>> quotes. What would be the moment to transfer the >>>> operator >>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> core >>>>>>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>>>>>> vice versa? I'm afraid that renaming contrib to >>>>>>>>>>> incubating >>>>>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>>>> confuse >>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> user even more. For people who aren't as known >> with >>>>>>>>>>>>> Airflow it >>>>>>>>>>>>>>>>> isn't >>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>> transparent which operator lives where, especially >>> if >>>> you >>>>>>>>>>>>>>>>> don't use >>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>> IDE >>>>>>>>>>>>>>>>>>>>>>> such as Ash ;) Besides that I don't think it is >>> worth >>>>>>>>>>>>> changing >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>> public >>>>>>>>>>>>>>>>>>>>>>> API just for a different name. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Ok. I am convinced. I will re-cast the vote with >>> this. >>>> It >>>>>>>>>>>>> will >>>>>>>>>>>>>>>>> make >>>>>>>>>>>>>>>>>>>> easier >>>>>>>>>>>>>>>>>>>>>> as we will not have to define other rules as those >>>> that we >>>>>>>>>>>>>>>>> already >>>>>>>>>>>>>>>>>>> have. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> *Case 2:* Slight preference to Solution B (let's >>> say a >>>>>>>>>>> +0) >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I like to search on Github with t, and then search >>> for >>>>>>>>>>>>>>>>> BashOperator. >>>>>>>>>>>>>>>>>>>> After >>>>>>>>>>>>>>>>>>>>>>> the change, you should search for Operator/Bash. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Jarek, I'm confused with your vote on this, from >>> your >>>>>>>>>>> mail >>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>> state: - >>>>>>>>>>>>>>>>>>>>>>> *Case 2: B: >>> Airflow.operators.foo_operator.FooOperator >>>>>>>>>>>>> could >>>>>>>>>>>>>>>>> become >>>>>>>>>>>>>>>>>>>>>>> airflow.operators.foo.FooOperator*. But the B >>>> solution is >>>>>>>>>>>>>>>>> keeping it >>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> same, so that would mean - *Case 2: B: >>>>>>>>>>>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator *would >>> stay >>>>>>>>>>>>>>>>>>>>>>> airflow.operators.foo_operator*.FooOperator* >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> My mistake. It was of course A. I think there is a >>>> little >>>>>>>>>>>>>>>>> confusion >>>>>>>>>>>>>>>>>>>> here. >>>>>>>>>>>>>>>>>>>>>> This case is not for changing BashOperator into >> Bash >>>> but >>>>>>>>>>> for >>>>>>>>>>>>>>>>> changing >>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> *module* name not the *class* name. So >>>>>>>>>>>>>>>>> bash_operator.BashOperator -> >>>>>>>>>>>>>>>>>>>>>> bash.BashOperator :) . you will still search for >>>>>>>>>>>>> BashOperator >>>>>>>>>>>>>>>>> in this >>>>>>>>>>>>>>>>>>>> case. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> *Case 3:* I'm leaning towards B >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Mostly because it isn't as trivial to decide when >> it >>>> gets >>>>>>>>>>>>> its >>>>>>>>>>>>>>>>> own >>>>>>>>>>>>>>>>>>>> package >>>>>>>>>>>>>>>>>>>>>>> or not. Besides the cloud providers, there are >>>> companies >>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>>> Databricks >>>>>>>>>>>>>>>>>>>>>>> and Qubole which might also end up with their own >>>> package >>>>>>>>>>>>>>>>> because >>>>>>>>>>>>>>>>>>> their >>>>>>>>>>>>>>>>>>>>>>> integration is getting richer over time. Moving >> this >>>> is a >>>>>>>>>>>>> bit >>>>>>>>>>>>>>>>>>> painful >>>>>>>>>>>>>>>>>>>>>>> because we need to deprecate the old path and move >>>>>>>>>>>>> everything >>>>>>>>>>>>>>>>> which >>>>>>>>>>>>>>>>>>>>>>> introduces noise into version control etc. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'd say, we should go for C. As I proposed. It's >> not >>> as >>>>>>>>>>>>>>>>> difficult as >>>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>>> seems to group operators in packages and it has >>>> numerous >>>>>>>>>>>>>>>>> advantages. >>>>>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>>>>>>> nice thing about deprecations - we can do them in >> the >>>>>>>>>>>>>>>>> __init__.py of >>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> packages which causes the noise fairly small (Git >>> will >>>>>>>>>>>>> actually >>>>>>>>>>>>>>>>>>> realise >>>>>>>>>>>>>>>>>>>>>> that the file has been moved and you can follow >> that. >>>>>>>>>>> Maybe >>>>>>>>>>>>> not >>>>>>>>>>>>>>>>> as >>>>>>>>>>>>>>>>>>> super >>>>>>>>>>>>>>>>>>>>>> easy to merge changes later to 1.10.4 but it's >> not a >>>>>>>>>>> rocket >>>>>>>>>>>>>>>>> science >>>>>>>>>>>>>>>>>>>> either. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> *Case 7:* Python native solution >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Yes. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Apart from all, great work! >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Cheers, Fokko >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Op di 23 jul. 2019 om 21:27 schreef Felix >> Uellendall >>>>>>>>>>>>>>>>>>>>>>> <felue...@pm.me.invalid >>>>>>>>>>>>>>>>>>>>>>>> : >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I have no strong "No" against any proposed change >>> of >>>>>>>>>>> these >>>>>>>>>>>>>>>>> cases. >>>>>>>>>>>>>>>>>>> So I >>>>>>>>>>>>>>>>>>>>>>> go >>>>>>>>>>>>>>>>>>>>>>>> with +1 (non-binding). >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> P.S. Thanks Jarek for bringing this up again and >>> your >>>>>>>>>>>>> intense >>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>>>>>> towards >>>>>>>>>>>>>>>>>>>>>>>> airflow currently :) and thanks to Kamil for even >>>>>>>>>>> creating >>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>> document. I >>>>>>>>>>>>>>>>>>>>>>>> like how the code is getting more and more >>> consistent >>>>>>>>>>> and >>>>>>>>>>>>>>>>> clean :) >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Kind regards, >>>>>>>>>>>>>>>>>>>>>>>> Felix >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Sent from ProtonMail mobile >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> -------- Original Message -------- >>>>>>>>>>>>>>>>>>>>>>>> On Jul 23, 2019, 18:34, Jarek Potiuk wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This email is calling a vote on the changes in >>>> import >>>>>>>>>>>>> paths. >>>>>>>>>>>>>>>>> It's >>>>>>>>>>>>>>>>>>>> been >>>>>>>>>>>>>>>>>>>>>>>>> discussed in >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://lists.apache.org/thread.html/4e648d9421c792d4537f5ac66f1a16dce468f816fc5221a9f9db9433@%3Cdev.airflow.apache.org%3E >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The vote will last for at least 1 week (July >> 30th >>>> 6pm >>>>>>>>>>>>> CEST), >>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>>>>> least >>>>>>>>>>>>>>>>>>>>>>>>> three +1 (binding) votes have been cast. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The proposal to vote is based on the document >> from >>>>>>>>>>> Kamil >>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>> >> https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit# >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> The proposed solution is: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 1: B: Contrib vs not: we move all that >> are >>>>>>>>>>> "well" >>>>>>>>>>>>>>>>> tested >>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>> rename contrib to "incubating" or similar.* >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 2: B: >>>>>>>>>>> Airflow.operators.foo_operator.FooOperator >>>>>>>>>>>>>>>>> could >>>>>>>>>>>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator* >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 3: C: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>> airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator >>>>>>>>>>>>>>>>>>>> could >>>>>>>>>>>>>>>>>>>>>>>>> become >>>> airflow.gcp.operators.bigtable.BigTableOperator* >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction* >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor") >>>> suffixes >>>>>>>>>>> on >>>>>>>>>>>>>>>>> class >>>>>>>>>>>>>>>>>>> names* >>>>>>>>>>>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on >>>> case-by-case >>>>>>>>>>>>> (and >>>>>>>>>>>>>>>>> my >>>>>>>>>>>>>>>>>>> team >>>>>>>>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>>>>>> do the job of GCP-related operators)* >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> This is my (binding) +1 vote. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> J. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Jarek Potiuk >>>>>>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal >>>>>>>>>>> Software >>>>>>>>>>>>>>>>> Engineer >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> M: [+48 660 796 129](tel:+48660796129) >>>>>>>>>>>>>>>>>>>>>>> <[+48660796129](tel:+48660796129)> >>>>>>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Jarek Potiuk >>>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal >>>> Software >>>>>>>>>>>>>>>>> Engineer >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> M: +48 660 796 129 <+48660796129> >>>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Jarek Potiuk >>>>>>>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal >>> Software >>>>>>>>>>>>>> Engineer >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> M: +48 660 796 129 <+48660796129> >>>>>>>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>> *Kaxil Naik* >>>>>>>>>>>>>>>>>> *Big Data Consultant | DevOps Data Engineer* >>>>>>>>>>>>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* >>> Apache >>>>>>>>>>> Spark >>>>>>>>>>>>> & >>>>>>>>>>>>>>>>> Neo4j >>>>>>>>>>>>>>>>>> Developer >>>>>>>>>>>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> *Kaxil Naik* >>>>>>>>>>>>>>>>> *Big Data Consultant | DevOps Data Engineer* >>>>>>>>>>>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* >>> Apache >>>>>>>> Spark >>>>>>>>>>> & >>>>>>>>>>>>>>>>> Neo4j >>>>>>>>>>>>>>>>> Developer >>>>>>>>>>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> Jarek Potiuk >>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software >>> Engineer >>>>>>>>>>> >>>>>>>>>>> M: +48 660 796 129 <+48660796129> >>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Kaxil Naik* >>>>>>>> *Big Data Consultant | DevOps Data Engineer* >>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* Apache >> Spark & >>>> Neo4j >>>>>>>> Developer >>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Jarek Potiuk >>>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>>>>>> >>>>>>> M: +48 660 796 129 <+48660796129> >>>>>>> [image: Polidea] <https://www.polidea.com/> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Jarek Potiuk >>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>>>>> >>>>>> M: +48 660 796 129 <+48660796129> >>>>>> [image: Polidea] <https://www.polidea.com/> >>>>> >>>> >>> >>
