I think almost everyone voted and we have almost perfect consensus. We all agree amongst other on moving all operators out of contrib (Great!).
The only doubts are for *Case 3* (Cloud provider prefix) and *Case 4* (Using Namespaces). I think there was actually an overlap between those two. Also Ash's comment/veto on that is quite clear. So I have final (I hope!) proposal to join those two cases and have one voting (till Friday) only on that. I will update the doc if you all agree with me that makes more sense as single case to vote on: *[Case 3 + Case 4]: Grouping Cloud Provider operators.* *Option A*. Keep operators/sensors/hooks in airflow/operators(sensors, hooks) and keep/add prefixes in file names. - *airflow/contrib/operators/sns_publish_operator.py becomes airflow/operators/**aws_sns_publish_operator.py* - *airflow/contrib/operators/dataproc_operator.py* becomes *airflow/operators/gcp_dataproc_operator.py* - *airflow/contrib/hooks/gcp_bigtable_hook.py *becomes *airflow/* *hooks/gcp_bigtable_hook.py* - *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* *operators/ssh_operator.py* *Option B*. Group operators/sensors/hooks in airflow/operators(sensors, hooks)/*<PROVIDER>.* Non-cloud provider ones are moved to airflow/operators(sensors/hooks), Drop the prefix. - *airflow/contrib/operators/sns_publish_operator.py becomes airflow/operators/**aws/sns_publish_operator.py* - *airflow/contrib/operators/dataproc_operator.py* becomes *airflow/operators/gcp/dataproc_operator.py* - *airflow/contrib/operators/gcp_bigtable_operator.py *becomes *airflow/* *operators/gcp/bigtable_operator.py* - *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* *operators/ssh_operator.py* *Option C*. Group operators/sensors/hooks in airflow/operators(sensors, hooks)*/<PROVIDER>.* Non-cloud provider ones are moved to airflow/operators(sensors/hooks), Keep the prefix. - *airflow/contrib/operators/sns_publish_operator.py becomes airflow/operators/**aws/aws_sns_publish_operator.py* - *airflow/contrib/operators/dataproc_operator.py* becomes *airflow/operators/gcp/gcp_dataproc_operator.py* - *airflow/contrib/operators/gcp_bigtable_operator.py *becomes *airflow/* *operators/gcp/gcp_bigtable_operator.py* - *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* *operators/ssh_operator.py* *Option D.* Group operators/sensors/hooks in *airflow/<PROVIDER>*/operators(sensors, hooks). Non-cloud provider ones are moved to airflow/operators(sensors/hooks). Drop the prefix. - *airflow/contrib/operators/sns_publish_operator.py becomes airflow/aws/operators/**sns_publish_operator.py* - *airflow/contrib/operators/dataproc_operator.py* becomes *airflow/gcp/operators/dataproc_operator.py* - *airflow/contrib/operators/gcp_bigtable_operator.py *becomes *airflow/* *gcp/operators/bigtable_operator.py* - *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* *operators/ssh_operator.py* *Option E.* Group operators/sensors/hooks in *airflow/<PROVIDER>*/operators(sensors, hooks). Non-cloud provider ones are moved to airflow/operators(sensors/hooks).* Keep the prefix*. - *airflow/contrib/operators/sns_publish_operator.py becomes airflow/aws/operators/aws_**sns_publish_operator.py* - *airflow/contrib/operators/dataproc_operator.py* becomes *airflow/gcp/operators/gcp_dataproc_operator.py* - *airflow/contrib/operators/gcp_bigtable_operator.py *becomes *airflow/* *gcp/operators/gcp_bigtable_operator.py* - *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* *operators/ssh_operator.py* Shall I proceed with updating the page ? J. On Mon, Jul 29, 2019 at 11:18 AM Kaxil Naik <kaxiln...@gmail.com> wrote: > Thanks Jarek, adding my vote. > > On Mon, Jul 29, 2019 at 2:40 PM Ash Berlin-Taylor <a...@apache.org> wrote: > > > Thanks Jarek, much clearer. I have voted there too. > > > > -ash > > > > > > > On 29 Jul 2019, at 08:13, Driesprong, Fokko <fo...@driesprong.frl> > > wrote: > > > > > > Thanks Jarek, the Wiki works much better for me. I've added my vote > there > > > as well. > > > > > > You've convinced me on the second case. I've changed my vote on Case 2 > > from > > > A to B :-) > > > > > > Maybe we should leave the vote open a bit longer since it involves > quite > > > some reading, and I would like to get some proper feedback and opinions > > on > > > this. Plus I only see two votes right now. If we don't think this > > through, > > > it might be that we're having this vote again in the near future :-) > > > > > > Cheers, Fokko > > > > > > Op zo 28 jul. 2019 om 10:49 schreef Jarek Potiuk < > > jarek.pot...@polidea.com>: > > > > > >> I updated the AIP-21 proposal in the way that should answer all the > > >> concerns in this thread. > > >> > > >> NOTE! There is an action for all of those who are interested: Please > > >> fill-in your voting in the voting table till Tuesday 30th of July 6pm > > CEST > > >> (5pm BST, 9 am PST). > > >> > > >> I created a summary of the proposal > > >> < > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal > > >>> > > >> in table form - which contains also examples for each case. > > >> > > >> I added a voting table > > >> < > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Voting > > >>> > > >> where you should add your votes: > > >> > > >> I propose this voting mechanism: > > >> > > >> Voting will take place till *Tuesday* 30 Jul 2019 6pm CEST (5 pm BST, > > 9am > > >> PST) . > > >> > > >> After the choice, the final consistent set of choices will be > announced > > >> (taking into account majority of binding vote, also including > potential > > >> vetos and consistency between the choices. Non-binding votes will be > > taken > > >> into account in case there is a draw. The final set of choices will be > > >> announced at devlist thread > > >> < > > >> > > > https://lists.apache.org/thread.html/4cedc94bee53ad908eee8333a56b58be8b5641881e73f69b97e436a9@%3Cdev.airflow.apache.org%3E > > >>> > > >> after > > >> the voting completes. > > >> > > >> Unless there is a veto raised to the final proposal, the final > proposal > > is > > >> accepted by Lazy Consensus > > >> <https://community.apache.org/committers/lazyConsensus.html> on > > *Friday* > > >> 02 > > >> Aug 2019 at 6pm CEST (5 pm BST, 9am PST). > > >> > > >> Please let me know if what I proposed can be further improved to avoid > > >> chaos. > > >> > > >> J. > > >> > > >> On Sat, Jul 27, 2019 at 11:41 AM Jarek Potiuk < > jarek.pot...@polidea.com > > > > > >> wrote: > > >> > > >>> Ok. I see that indeed voting could have been organised a bit better - > > dev > > >>> list does not seem to cope well with such a complex case :). > > >>> > > >>> As Kamil noticed - we already have a formal AIP-21 > > >>> < > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths > > >>> > > >>> in the Wiki - which I should have mentioned before. I have not much > > time > > >>> today - but tomorrow I will try to summarise the options - based on > > some > > >>> real code examples (to make it easier to digest). I think the best > > >> approach > > >>> will be to make a voting matrix in the AIP-21 Wiki page where people > > will > > >>> be able to state their preferences for each case (by adding a row to > > the > > >>> table). I think that might provide much better clarity and a single > > place > > >>> which will show the current aggregated state of voting. > > >>> > > >>> However - as Ash also stated - some of the options are > inter-dependent > > so > > >>> at the end we will have to adjust the results in case they are not > > >>> consistent - and make final voting on aggregated set of cases - but > I > > >>> think in this case following "lasy consensus" - i,e. if noone objects > > to > > >>> final set of cases, we will proceed with it.. > > >>> > > >>> Do you think that will work better ? > > >>> > > >>> J. > > >>> > > >>> Principal Software Engineer > > >>> Phone: +48660796129 > > >>> > > >>> sob., 27 lip 2019, 09:26 użytkownik Kamil Breguła < > > >>> kamil.breg...@polidea.com> napisał: > > >>> > > >>>> Document is also available as AIP on wiki: > > >>>> > > >>>> > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths > > >>>> > > >>>> On Sat, Jul 27, 2019, 9:07 AM Driesprong, Fokko > <fo...@driesprong.frl > > > > > >>>> wrote: > > >>>> > > >>>>> I agree with all, this is a bit chaotic. For the sake of clarity, I > > >>>> would > > >>>>> suggest moving the Google Doc to the Wiki as an AIP. Create a > > >> structural > > >>>>> code improvement AIP and then and add a matrix of users/cases where > > >>>>> everybody can add his vote per case. This gives also the > opportunity > > >> for > > >>>>> changing your vote on a specific case. WDYT? > > >>>>> > > >>>>> Cheers, Fokko > > >>>>> > > >>>>> Op vr 26 jul. 2019 om 22:28 schreef Chen Tong <cix...@gmail.com>: > > >>>>> > > >>>>>> I agree with Ash. It is clear to vote on each case rather than all > > >>>>>> together. > > >>>>>> > > >>>>>> On Fri, Jul 26, 2019 at 3:54 PM Ash Berlin-Taylor <a...@apache.org > > > > >>>>> wrote: > > >>>>>> > > >>>>>>> A number of proposals overlap or add on to each other, so I think > > >>>> (?) a > > >>>>>>> single vote makes sense. > > >>>>>>> > > >>>>>>> To be clear, my vote is -1 until it's clear exactly what we are > > >>>> voting > > >>>>> on. > > >>>>>>> > > >>>>>>> On 26 July 2019 20:39:56 BST, Kaxil Naik <kaxiln...@gmail.com> > > >>>> wrote: > > >>>>>>>> I agree with Ash and instead of voting on all 7 Cases together, > > >> how > > >>>>>>>> about > > >>>>>>>> separate vote emails for all the cases? Each email can have the > > >>>>>>>> description > > >>>>>>>> copied over from the doc. > > >>>>>>>> > > >>>>>>>> It would probably be a bit easier to track a decision in the > > >> future > > >>>> as > > >>>>>>>> well. > > >>>>>>>> > > >>>>>>>> What do you guys think? @Jarek Potiuk <jarek.pot...@polidea.com > > > > >>>>>>>> @Driesprong, > > >>>>>>>> Fokko <fo...@driesprong.frl> > > >>>>>>>> > > >>>>>>>> On Sat, Jul 27, 2019 at 1:03 AM Kaxil Naik <kaxiln...@gmail.com > > > > >>>>> wrote: > > >>>>>>>> > > >>>>>>>>> *Case #1* airflow.contrib.{resources} : *Solution A * > > >>>>>>>>> > > >>>>>>>>> *Case #2* git *_{operator/sensor}{/s}.py : *Solution A * > > >>>>>>>>> > > >>>>>>>>> *Case #3* {aws/azure/gcp}_* : *Solution C* > > >>>>>>>>> > > >>>>>>>>> *Case #4* Separate namespace for resources : > > >>>>>>>>> *Solution A* > > >>>>>>>>> > > >>>>>>>>> *Case #5* *Operator : *Solution A* > > >>>>>>>>> > > >>>>>>>>> *Case #7* : *Solution A* > > >>>>>>>>> > > >>>>>>>>> On Fri, Jul 26, 2019 at 11:09 PM Daniel Standish > > >>>>>>>> <dpstand...@gmail.com> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> I have tried to add some clarification to Jarek's summary, but > > >> I > > >>>> am > > >>>>>>>> a > > >>>>>>>>>> little fuzzy on exact proposal for case 3 so hopefully Jarek > > >> can > > >>>>>>>> further > > >>>>>>>>>> clarify. > > >>>>>>>>>> > > >>>>>>>>>> According to my reading, in this motion cases 4,5,6 are all > > >>>> either > > >>>>>>>>>> proposal > > >>>>>>>>>> *rejections* or otherwise have no effect, so attention can be > > >>>>>>>> focused on > > >>>>>>>>>> cases 1,2,3 and 7. > > >>>>>>>>>> > > >>>>>>>>>> *Motion is to adopt the following:* > > >>>>>>>>>> > > >>>>>>>>>> Case 1 - Solution A - Get rid of contrib > > >>>>>>>>>> All objects will be moved out of contrib folder and into > > >>>> respective > > >>>>>>>>>> non-contrib folder. > > >>>>>>>>>> > > >>>>>>>>>> Case 2 - Solution A - Remove object type suffix from modules > > >>>>>>>>>> Example: > > >>>>>>>>>> Airflow.operators.foo_operator.FooOperator becomes > > >>>>>>>>>> airflow.operators.foo.FooOperator > > >>>>>>>>>> Example: > > >>>>>>>>>> Airflow.hooks.foo_hook.FooOperator becomes > > >>>> airflow.hooks.foo.FooHook > > >>>>>>>>>> > > >>>>>>>>>> Case 3 - conventions for cloud provider etc prefixes > > >>>>>>>>>> *I am not sure exactly what the proposal is.* > > >>>>>>>>>> Example case is > > >>>> airflow/contrib/operators/gcp_bigtable_operator.py > > >>>>>>>>>> Since motion adopts case 1 solution A, we can assume it is now > > >>>> named > > >>>>>>>> like > > >>>>>>>>>> so: airflow/operators/gcp_bigtable_operator.py > > >>>>>>>>>> So what is proposed for this case? > > >>>>>>>>>> > > >>>>>>>>>> Case 4 - Solution B - *no change* > > >>>>>>>>>> This proposal was about namespaces but the motion is to reject > > >>>> the > > >>>>>>>>>> proposal. > > >>>>>>>>>> > > >>>>>>>>>> Case 5 - Solution B - *no change* > > >>>>>>>>>> The proposal was to remove class type suffix e.g. remove the > > >>>>>>>> Operator in > > >>>>>>>>>> FooOperator, but the motion is to reject this proposal -- i.e. > > >>>>>>>> motion is > > >>>>>>>>>> no > > >>>>>>>>>> change on this case, i.e. we resolve to keep "Operator" (and > > >>>>>>>> "Sensor") > > >>>>>>>>>> suffixes on class names > > >>>>>>>>>> > > >>>>>>>>>> Case 6 - *No change* > > >>>>>>>>>> This case concerns object naming inconsistencies, but motion > > >> does > > >>>>>>>> not > > >>>>>>>>>> enact > > >>>>>>>>>> any specific proposal. > > >>>>>>>>>> > > >>>>>>>>>> Case 7 - Solution A - use warnings.warn template for > > >> deprecation > > >>>>>>>> warnings > > >>>>>>>>>> (instead of zope) > > >>>>>>>>>> Example: > > >>>>>>>>>> > > >>>>>>>>>> # in old_package/old_mod.py > > >>>>>>>>>> > > >>>>>>>>>> import warnings > > >>>>>>>>>> from new_package.new_mod import * > > >>>>>>>>>> warnings.warn("old_package.old_mod has moved to > > >>>> new_package.new_mod. > > >>>>>>>>>> Import > > >>>>>>>>>> of " > > >>>>>>>>>> "old_package.old_mod will become unsupported in version 2", > > >>>>>>>>>> DeprecationWarning, 2) > > >>>>>>>>>> > > >>>>>>>>>> Reference implementation here: > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>> > > >>>> > > >> > > > https://github.com/mik-laj/airflow-deprecation-sample/tree/master/solution1 > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> FWIW +1 (non-binding) on 1,2,7 -- unsure on 3. > > >>>>>>>>>> > > >>>>>>>>>> I am very happy that this motion now gets rid of contrib. > > >> Thank > > >>>> you > > >>>>>>>>>> Sergio > > >>>>>>>>>> for turning the tide with your effective argumentation ;) > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> On Fri, Jul 26, 2019 at 3:16 AM Ash Berlin-Taylor < > > >>>> a...@apache.org> > > >>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> Could someone summarise what the proposed name changes are, > > >>>> here, > > >>>>>>>> in > > >>>>>>>>>> this > > >>>>>>>>>>> thread? Pointing at a google doc and only giving partial > > >>>> examples > > >>>>>>>> is 1) > > >>>>>>>>>> a > > >>>>>>>>>>> bit difficult to quickly work out what we are voting on, and > > >> 2) > > >>>>>>>> using a > > >>>>>>>>>>> google doc isn't great for a longer term record. > > >>>>>>>>>>> > > >>>>>>>>>>> Thanks, > > >>>>>>>>>>> Ash > > >>>>>>>>>>> > > >>>>>>>>>>>> On 26 Jul 2019, at 09:14, Jarek Potiuk > > >>>>>>>> <jarek.pot...@polidea.com> > > >>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>> I would like to recast the vote. Let's start from 0 :) . I > > >>>> would > > >>>>>>>> like > > >>>>>>>>>> to > > >>>>>>>>>>>> keep the July 30th 6pm CEST date, and at least three +1 > > >>>>>>>> (binding) > > >>>>>>>>>> votes > > >>>>>>>>>>> are > > >>>>>>>>>>>> needed to pass. I marked in red the modifications to the > > >>>>>>>> previous > > >>>>>>>>>>> proposal. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Here is the proposal (details here > > >>>>>>>>>>>> < > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>> > > >>>> > > >> > > > https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit#heading=h.j4jc3i70qszo > > >>>>>>>>>>>> > > >>>>>>>>>>>> ) > > >>>>>>>>>>>> > > >>>>>>>>>>>> - *Case 1: A: Get rid of contrib,* > > >>>>>>>>>>>> - *Case 2: A: Airflow.operators.foo_operator.FooOperator > > >>>> could > > >>>>>>>>>>>> become airflow.operators.foo.FooOperator* > > >>>>>>>>>>>> - *Case 3: C: > > >>>>>>>>>>>> > > >>>>>>>> airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator > > >>>>>>>>>> could > > >>>>>>>>>>>> become airflow.gcp.operators.bigtable.BigTableOperator* > > >>>>>>>>>>>> - *Case 4: B: no namespace introduction* > > >>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor") suffixes on > > >>>> class > > >>>>>>>>>> names* > > >>>>>>>>>>>> - *Case 6: We will treat isolated cases on case-by-case > > >>>> (and > > >>>>>>>> my team > > >>>>>>>>>>> can > > >>>>>>>>>>>> do the job of GCP-related operators)* > > >>>>>>>>>>>> - *Case 7: Native python solution (with better IDE > > >>>>>>>> integration)* > > >>>>>>>>>>>> > > >>>>>>>>>>>> This is my binding (+1) vote. > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Fri, Jul 26, 2019 at 10:08 AM Jarek Potiuk < > > >>>>>>>>>> jarek.pot...@polidea.com> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Hey Felix, > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> For 7* -> I am in favour of trying the native one as well > > >> (I > > >>>>>>>> use IDE > > >>>>>>>>>>> quite > > >>>>>>>>>>>>> often ;) ) > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> J. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On Wed, Jul 24, 2019 at 9:34 AM Driesprong, Fokko > > >>>>>>>>>> <fo...@driesprong.frl > > >>>>>>>>>>>> > > >>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> Thanks Kamil for putting the document together. I wasn't > > >>>> able > > >>>>>>>> to > > >>>>>>>>>>> respond > > >>>>>>>>>>>>>> earlier since I was giving Airflow workshops throughout > > >>>> Europe > > >>>>>>>> :-) > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> *Case 1: *Solution A → Remove everything from contrib > > >> into > > >>>> a > > >>>>>>>> single > > >>>>>>>>>>>>>> package. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> To make it easier to the end-user, my preference would be > > >>>> to > > >>>>>>>> get > > >>>>>>>>>> rid of > > >>>>>>>>>>>>>> the > > >>>>>>>>>>>>>> contrib package altogether. What's in contrib or not is a > > >>>>>>>> tricky > > >>>>>>>>>>> question, > > >>>>>>>>>>>>>> I think this is already reflected in the document since > > >>>>>>>> "maturity" > > >>>>>>>>>> is > > >>>>>>>>>>> in > > >>>>>>>>>>>>>> quotes. What would be the moment to transfer the operator > > >>>> to > > >>>>>>>> the > > >>>>>>>>>> core > > >>>>>>>>>>> or > > >>>>>>>>>>>>>> vice versa? I'm afraid that renaming contrib to > > >> incubating > > >>>>>>>> will > > >>>>>>>>>> confuse > > >>>>>>>>>>>>>> the > > >>>>>>>>>>>>>> user even more. For people who aren't as known with > > >>>> Airflow it > > >>>>>>>> isn't > > >>>>>>>>>>> that > > >>>>>>>>>>>>>> transparent which operator lives where, especially if you > > >>>>>>>> don't use > > >>>>>>>>>> an > > >>>>>>>>>>> IDE > > >>>>>>>>>>>>>> such as Ash ;) Besides that I don't think it is worth > > >>>> changing > > >>>>>>>> the > > >>>>>>>>>>> public > > >>>>>>>>>>>>>> API just for a different name. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Ok. I am convinced. I will re-cast the vote with this. It > > >>>> will > > >>>>>>>> make > > >>>>>>>>>>> easier > > >>>>>>>>>>>>> as we will not have to define other rules as those that we > > >>>>>>>> already > > >>>>>>>>>> have. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> *Case 2:* Slight preference to Solution B (let's say a > > >> +0) > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> I like to search on Github with t, and then search for > > >>>>>>>> BashOperator. > > >>>>>>>>>>> After > > >>>>>>>>>>>>>> the change, you should search for Operator/Bash. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Jarek, I'm confused with your vote on this, from your > > >> mail > > >>>> you > > >>>>>>>>>> state: - > > >>>>>>>>>>>>>> *Case 2: B: Airflow.operators.foo_operator.FooOperator > > >>>> could > > >>>>>>>> become > > >>>>>>>>>>>>>> airflow.operators.foo.FooOperator*. But the B solution is > > >>>>>>>> keeping it > > >>>>>>>>>>> the > > >>>>>>>>>>>>>> same, so that would mean - *Case 2: B: > > >>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator *would stay > > >>>>>>>>>>>>>> airflow.operators.foo_operator*.FooOperator* > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> My mistake. It was of course A. I think there is a little > > >>>>>>>> confusion > > >>>>>>>>>>> here. > > >>>>>>>>>>>>> This case is not for changing BashOperator into Bash but > > >> for > > >>>>>>>> changing > > >>>>>>>>>>> the > > >>>>>>>>>>>>> *module* name not the *class* name. So > > >>>>>>>> bash_operator.BashOperator -> > > >>>>>>>>>>>>> bash.BashOperator :) . you will still search for > > >>>> BashOperator > > >>>>>>>> in this > > >>>>>>>>>>> case. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> *Case 3:* I'm leaning towards B > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Mostly because it isn't as trivial to decide when it gets > > >>>> its > > >>>>>>>> own > > >>>>>>>>>>> package > > >>>>>>>>>>>>>> or not. Besides the cloud providers, there are companies > > >>>> like > > >>>>>>>>>>> Databricks > > >>>>>>>>>>>>>> and Qubole which might also end up with their own package > > >>>>>>>> because > > >>>>>>>>>> their > > >>>>>>>>>>>>>> integration is getting richer over time. Moving this is a > > >>>> bit > > >>>>>>>>>> painful > > >>>>>>>>>>>>>> because we need to deprecate the old path and move > > >>>> everything > > >>>>>>>> which > > >>>>>>>>>>>>>> introduces noise into version control etc. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I'd say, we should go for C. As I proposed. It's not as > > >>>>>>>> difficult as > > >>>>>>>>>> it > > >>>>>>>>>>>>> seems to group operators in packages and it has numerous > > >>>>>>>> advantages. > > >>>>>>>>>> The > > >>>>>>>>>>>>> nice thing about deprecations - we can do them in the > > >>>>>>>> __init__.py of > > >>>>>>>>>> the > > >>>>>>>>>>>>> packages which causes the noise fairly small (Git will > > >>>> actually > > >>>>>>>>>> realise > > >>>>>>>>>>>>> that the file has been moved and you can follow that. > > >> Maybe > > >>>> not > > >>>>>>>> as > > >>>>>>>>>> super > > >>>>>>>>>>>>> easy to merge changes later to 1.10.4 but it's not a > > >> rocket > > >>>>>>>> science > > >>>>>>>>>>> either. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> *Case 7:* Python native solution > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Yes. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> --- > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Apart from all, great work! > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Cheers, Fokko > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Op di 23 jul. 2019 om 21:27 schreef Felix Uellendall > > >>>>>>>>>>>>>> <felue...@pm.me.invalid > > >>>>>>>>>>>>>>> : > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> I have no strong "No" against any proposed change of > > >> these > > >>>>>>>> cases. > > >>>>>>>>>> So I > > >>>>>>>>>>>>>> go > > >>>>>>>>>>>>>>> with +1 (non-binding). > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> P.S. Thanks Jarek for bringing this up again and your > > >>>> intense > > >>>>>>>> work > > >>>>>>>>>>>>>> towards > > >>>>>>>>>>>>>>> airflow currently :) and thanks to Kamil for even > > >> creating > > >>>>>>>> this > > >>>>>>>>>>>>>> document. I > > >>>>>>>>>>>>>>> like how the code is getting more and more consistent > > >> and > > >>>>>>>> clean :) > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Kind regards, > > >>>>>>>>>>>>>>> Felix > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Sent from ProtonMail mobile > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> -------- Original Message -------- > > >>>>>>>>>>>>>>> On Jul 23, 2019, 18:34, Jarek Potiuk wrote: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Hello everyone, > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> This email is calling a vote on the changes in import > > >>>> paths. > > >>>>>>>> It's > > >>>>>>>>>>> been > > >>>>>>>>>>>>>>>> discussed in > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>> > > >>>> > > >> > > > https://lists.apache.org/thread.html/4e648d9421c792d4537f5ac66f1a16dce468f816fc5221a9f9db9433@%3Cdev.airflow.apache.org%3E > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> The vote will last for at least 1 week (July 30th 6pm > > >>>> CEST), > > >>>>>>>> and > > >>>>>>>>>> at > > >>>>>>>>>>>>>> least > > >>>>>>>>>>>>>>>> three +1 (binding) votes have been cast. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> The proposal to vote is based on the document from > > >> Kamil > > >>>>>>>>>>>>>>>> < > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>> > > >>>> > > >> > > > https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit# > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> The proposed solution is: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> - *Case 1: B: Contrib vs not: we move all that are > > >> "well" > > >>>>>>>> tested > > >>>>>>>>>> and > > >>>>>>>>>>>>>>>> rename contrib to "incubating" or similar.* > > >>>>>>>>>>>>>>>> - *Case 2: B: > > >> Airflow.operators.foo_operator.FooOperator > > >>>>>>>> could > > >>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator* > > >>>>>>>>>>>>>>>> - *Case 3: C: > > >>>>>>>>>>>>>>>> > > >>>>>>>> airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator > > >>>>>>>>>>> could > > >>>>>>>>>>>>>>>> become airflow.gcp.operators.bigtable.BigTableOperator* > > >>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction* > > >>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor") suffixes > > >> on > > >>>>>>>> class > > >>>>>>>>>> names* > > >>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on case-by-case > > >>>> (and > > >>>>>>>> my > > >>>>>>>>>> team > > >>>>>>>>>>>>>> can > > >>>>>>>>>>>>>>>> do the job of GCP-related operators)* > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> This is my (binding) +1 vote. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Best regards, > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> J. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Jarek Potiuk > > >>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal > > >> Software > > >>>>>>>> Engineer > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> M: [+48 660 796 129](tel:+48660796129) > > >>>>>>>>>>>>>> <[+48660796129](tel:+48660796129)> > > >>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> -- > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Jarek Potiuk > > >>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software > > >>>>>>>> Engineer > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> M: +48 660 796 129 <+48660796129> > > >>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> -- > > >>>>>>>>>>>> > > >>>>>>>>>>>> Jarek Potiuk > > >>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software > > >>>>> Engineer > > >>>>>>>>>>>> > > >>>>>>>>>>>> M: +48 660 796 129 <+48660796129> > > >>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> -- > > >>>>>>>>> *Kaxil Naik* > > >>>>>>>>> *Big Data Consultant | DevOps Data Engineer* > > >>>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* Apache > > >> Spark > > >>>> & > > >>>>>>>> Neo4j > > >>>>>>>>> Developer > > >>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> -- > > >>>>>>>> *Kaxil Naik* > > >>>>>>>> *Big Data Consultant | DevOps Data Engineer* > > >>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* Apache > Spark > > >> & > > >>>>>>>> Neo4j > > >>>>>>>> Developer > > >>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > >> -- > > >> > > >> Jarek Potiuk > > >> Polidea <https://www.polidea.com/> | Principal Software Engineer > > >> > > >> M: +48 660 796 129 <+48660796129> > > >> [image: Polidea] <https://www.polidea.com/> > > >> > > > > > > -- > *Kaxil Naik* > *Big Data Consultant | DevOps Data Engineer* > *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j > Developer > *LinkedIn*: https://www.linkedin.com/in/kaxil > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
