I went ahead and updated the page (just to speed it up) as I think it really makes sense to join those two cases (and I do not see any drawbacks - I think the options we have cover all possible approaches) and we can always go back if we need to.
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal My vote is *D*. - airflow/contrib/operators/gcp_bigtable_operator.py → airflow/gcp /operators/bigtable_operator.py - airflow/contrib/operators/ssh_operator.py -> airflow/operators/ssh_operator.py I updated the page with my vote. I propose that we vote till Friday on that one (all the rest is already agreed). J. On Tue, Jul 30, 2019 at 9:42 AM Jarek Potiuk <jarek.pot...@polidea.com> wrote: > I think almost everyone voted and we have almost perfect consensus. We all > agree amongst other on moving all operators out of contrib (Great!). > > The only doubts are for *Case 3* (Cloud provider prefix) and *Case 4* > (Using Namespaces). > I think there was actually an overlap between those two. Also Ash's > comment/veto on that is quite clear. > So I have final (I hope!) proposal to join those two cases and have one > voting (till Friday) only on that. > > I will update the doc if you all agree with me that makes more sense as > single case to vote on: > > *[Case 3 + Case 4]: Grouping Cloud Provider operators.* > > *Option A*. Keep operators/sensors/hooks in airflow/operators(sensors, > hooks) and keep/add prefixes in file names. > > - > *airflow/contrib/operators/sns_publish_operator.py becomes > airflow/operators/**aws_sns_publish_operator.py* > > > - *airflow/contrib/operators/dataproc_operator.py* > becomes *airflow/operators/gcp_dataproc_operator.py* > > > - > *airflow/contrib/hooks/gcp_bigtable_hook.py *becomes *airflow/* > *hooks/gcp_bigtable_hook.py* > - > *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* > *operators/ssh_operator.py* > > > > *Option B*. Group operators/sensors/hooks in airflow/operators(sensors, > hooks)/*<PROVIDER>.* Non-cloud provider ones are moved to > airflow/operators(sensors/hooks), Drop the prefix. > > - > *airflow/contrib/operators/sns_publish_operator.py > becomes airflow/operators/**aws/sns_publish_operator.py* > > > - *airflow/contrib/operators/dataproc_operator.py* > becomes *airflow/operators/gcp/dataproc_operator.py* > > > - > *airflow/contrib/operators/gcp_bigtable_operator.py *becomes *airflow/* > *operators/gcp/bigtable_operator.py* > - > *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* > *operators/ssh_operator.py* > > > *Option C*. Group operators/sensors/hooks in airflow/operators(sensors, > hooks)*/<PROVIDER>.* Non-cloud provider ones are moved to > airflow/operators(sensors/hooks), Keep the prefix. > > - > *airflow/contrib/operators/sns_publish_operator.py > becomes airflow/operators/**aws/aws_sns_publish_operator.py* > > > - *airflow/contrib/operators/dataproc_operator.py* > becomes *airflow/operators/gcp/gcp_dataproc_operator.py* > > > - > *airflow/contrib/operators/gcp_bigtable_operator.py *becomes *airflow/* > *operators/gcp/gcp_bigtable_operator.py* > - > *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* > *operators/ssh_operator.py* > > > *Option D.* Group operators/sensors/hooks in > *airflow/<PROVIDER>*/operators(sensors, > hooks). Non-cloud provider ones are moved to > airflow/operators(sensors/hooks). Drop the prefix. > > - > *airflow/contrib/operators/sns_publish_operator.py > becomes airflow/aws/operators/**sns_publish_operator.py* > > > - *airflow/contrib/operators/dataproc_operator.py* > becomes *airflow/gcp/operators/dataproc_operator.py* > > > - > *airflow/contrib/operators/gcp_bigtable_operator.py *becomes *airflow/* > *gcp/operators/bigtable_operator.py* > - > *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* > *operators/ssh_operator.py* > > > *Option E.* Group operators/sensors/hooks in > *airflow/<PROVIDER>*/operators(sensors, > hooks). Non-cloud provider ones are moved to > airflow/operators(sensors/hooks).* Keep the prefix*. > > - > *airflow/contrib/operators/sns_publish_operator.py > becomes airflow/aws/operators/aws_**sns_publish_operator.py* > > > - *airflow/contrib/operators/dataproc_operator.py* > becomes *airflow/gcp/operators/gcp_dataproc_operator.py* > > > - > *airflow/contrib/operators/gcp_bigtable_operator.py *becomes *airflow/* > *gcp/operators/gcp_bigtable_operator.py* > - > *airflow/contrib/operators/ssh_operator.py *becomes *airflow/* > *operators/ssh_operator.py* > > > Shall I proceed with updating the page ? > > > J. > > > On Mon, Jul 29, 2019 at 11:18 AM Kaxil Naik <kaxiln...@gmail.com> wrote: > >> Thanks Jarek, adding my vote. >> >> On Mon, Jul 29, 2019 at 2:40 PM Ash Berlin-Taylor <a...@apache.org> wrote: >> >> > Thanks Jarek, much clearer. I have voted there too. >> > >> > -ash >> > >> > >> > > On 29 Jul 2019, at 08:13, Driesprong, Fokko <fo...@driesprong.frl> >> > wrote: >> > > >> > > Thanks Jarek, the Wiki works much better for me. I've added my vote >> there >> > > as well. >> > > >> > > You've convinced me on the second case. I've changed my vote on Case 2 >> > from >> > > A to B :-) >> > > >> > > Maybe we should leave the vote open a bit longer since it involves >> quite >> > > some reading, and I would like to get some proper feedback and >> opinions >> > on >> > > this. Plus I only see two votes right now. If we don't think this >> > through, >> > > it might be that we're having this vote again in the near future :-) >> > > >> > > Cheers, Fokko >> > > >> > > Op zo 28 jul. 2019 om 10:49 schreef Jarek Potiuk < >> > jarek.pot...@polidea.com>: >> > > >> > >> I updated the AIP-21 proposal in the way that should answer all the >> > >> concerns in this thread. >> > >> >> > >> NOTE! There is an action for all of those who are interested: Please >> > >> fill-in your voting in the voting table till Tuesday 30th of July 6pm >> > CEST >> > >> (5pm BST, 9 am PST). >> > >> >> > >> I created a summary of the proposal >> > >> < >> > >> >> > >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Summaryoftheproposal >> > >>> >> > >> in table form - which contains also examples for each case. >> > >> >> > >> I added a voting table >> > >> < >> > >> >> > >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP-21:Changesinimportpaths-Voting >> > >>> >> > >> where you should add your votes: >> > >> >> > >> I propose this voting mechanism: >> > >> >> > >> Voting will take place till *Tuesday* 30 Jul 2019 6pm CEST (5 pm >> BST, >> > 9am >> > >> PST) . >> > >> >> > >> After the choice, the final consistent set of choices will be >> announced >> > >> (taking into account majority of binding vote, also including >> potential >> > >> vetos and consistency between the choices. Non-binding votes will be >> > taken >> > >> into account in case there is a draw. The final set of choices will >> be >> > >> announced at devlist thread >> > >> < >> > >> >> > >> https://lists.apache.org/thread.html/4cedc94bee53ad908eee8333a56b58be8b5641881e73f69b97e436a9@%3Cdev.airflow.apache.org%3E >> > >>> >> > >> after >> > >> the voting completes. >> > >> >> > >> Unless there is a veto raised to the final proposal, the final >> proposal >> > is >> > >> accepted by Lazy Consensus >> > >> <https://community.apache.org/committers/lazyConsensus.html> on >> > *Friday* >> > >> 02 >> > >> Aug 2019 at 6pm CEST (5 pm BST, 9am PST). >> > >> >> > >> Please let me know if what I proposed can be further improved to >> avoid >> > >> chaos. >> > >> >> > >> J. >> > >> >> > >> On Sat, Jul 27, 2019 at 11:41 AM Jarek Potiuk < >> jarek.pot...@polidea.com >> > > >> > >> wrote: >> > >> >> > >>> Ok. I see that indeed voting could have been organised a bit better >> - >> > dev >> > >>> list does not seem to cope well with such a complex case :). >> > >>> >> > >>> As Kamil noticed - we already have a formal AIP-21 >> > >>> < >> > >> >> > >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths >> > >>> >> > >>> in the Wiki - which I should have mentioned before. I have not much >> > time >> > >>> today - but tomorrow I will try to summarise the options - based on >> > some >> > >>> real code examples (to make it easier to digest). I think the best >> > >> approach >> > >>> will be to make a voting matrix in the AIP-21 Wiki page where people >> > will >> > >>> be able to state their preferences for each case (by adding a row to >> > the >> > >>> table). I think that might provide much better clarity and a single >> > place >> > >>> which will show the current aggregated state of voting. >> > >>> >> > >>> However - as Ash also stated - some of the options are >> inter-dependent >> > so >> > >>> at the end we will have to adjust the results in case they are not >> > >>> consistent - and make final voting on aggregated set of cases - >> but I >> > >>> think in this case following "lasy consensus" - i,e. if noone >> objects >> > to >> > >>> final set of cases, we will proceed with it.. >> > >>> >> > >>> Do you think that will work better ? >> > >>> >> > >>> J. >> > >>> >> > >>> Principal Software Engineer >> > >>> Phone: +48660796129 >> > >>> >> > >>> sob., 27 lip 2019, 09:26 użytkownik Kamil Breguła < >> > >>> kamil.breg...@polidea.com> napisał: >> > >>> >> > >>>> Document is also available as AIP on wiki: >> > >>>> >> > >>>> >> > >> >> > >> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths >> > >>>> >> > >>>> On Sat, Jul 27, 2019, 9:07 AM Driesprong, Fokko >> <fo...@driesprong.frl >> > > >> > >>>> wrote: >> > >>>> >> > >>>>> I agree with all, this is a bit chaotic. For the sake of clarity, >> I >> > >>>> would >> > >>>>> suggest moving the Google Doc to the Wiki as an AIP. Create a >> > >> structural >> > >>>>> code improvement AIP and then and add a matrix of users/cases >> where >> > >>>>> everybody can add his vote per case. This gives also the >> opportunity >> > >> for >> > >>>>> changing your vote on a specific case. WDYT? >> > >>>>> >> > >>>>> Cheers, Fokko >> > >>>>> >> > >>>>> Op vr 26 jul. 2019 om 22:28 schreef Chen Tong <cix...@gmail.com>: >> > >>>>> >> > >>>>>> I agree with Ash. It is clear to vote on each case rather than >> all >> > >>>>>> together. >> > >>>>>> >> > >>>>>> On Fri, Jul 26, 2019 at 3:54 PM Ash Berlin-Taylor < >> a...@apache.org> >> > >>>>> wrote: >> > >>>>>> >> > >>>>>>> A number of proposals overlap or add on to each other, so I >> think >> > >>>> (?) a >> > >>>>>>> single vote makes sense. >> > >>>>>>> >> > >>>>>>> To be clear, my vote is -1 until it's clear exactly what we are >> > >>>> voting >> > >>>>> on. >> > >>>>>>> >> > >>>>>>> On 26 July 2019 20:39:56 BST, Kaxil Naik <kaxiln...@gmail.com> >> > >>>> wrote: >> > >>>>>>>> I agree with Ash and instead of voting on all 7 Cases together, >> > >> how >> > >>>>>>>> about >> > >>>>>>>> separate vote emails for all the cases? Each email can have the >> > >>>>>>>> description >> > >>>>>>>> copied over from the doc. >> > >>>>>>>> >> > >>>>>>>> It would probably be a bit easier to track a decision in the >> > >> future >> > >>>> as >> > >>>>>>>> well. >> > >>>>>>>> >> > >>>>>>>> What do you guys think? @Jarek Potiuk < >> jarek.pot...@polidea.com> >> > >>>>>>>> @Driesprong, >> > >>>>>>>> Fokko <fo...@driesprong.frl> >> > >>>>>>>> >> > >>>>>>>> On Sat, Jul 27, 2019 at 1:03 AM Kaxil Naik < >> kaxiln...@gmail.com> >> > >>>>> wrote: >> > >>>>>>>> >> > >>>>>>>>> *Case #1* airflow.contrib.{resources} : *Solution A * >> > >>>>>>>>> >> > >>>>>>>>> *Case #2* git *_{operator/sensor}{/s}.py : *Solution A * >> > >>>>>>>>> >> > >>>>>>>>> *Case #3* {aws/azure/gcp}_* : *Solution C* >> > >>>>>>>>> >> > >>>>>>>>> *Case #4* Separate namespace for resources : >> > >>>>>>>>> *Solution A* >> > >>>>>>>>> >> > >>>>>>>>> *Case #5* *Operator : *Solution A* >> > >>>>>>>>> >> > >>>>>>>>> *Case #7* : *Solution A* >> > >>>>>>>>> >> > >>>>>>>>> On Fri, Jul 26, 2019 at 11:09 PM Daniel Standish >> > >>>>>>>> <dpstand...@gmail.com> >> > >>>>>>>>> wrote: >> > >>>>>>>>> >> > >>>>>>>>>> I have tried to add some clarification to Jarek's summary, >> but >> > >> I >> > >>>> am >> > >>>>>>>> a >> > >>>>>>>>>> little fuzzy on exact proposal for case 3 so hopefully Jarek >> > >> can >> > >>>>>>>> further >> > >>>>>>>>>> clarify. >> > >>>>>>>>>> >> > >>>>>>>>>> According to my reading, in this motion cases 4,5,6 are all >> > >>>> either >> > >>>>>>>>>> proposal >> > >>>>>>>>>> *rejections* or otherwise have no effect, so attention can be >> > >>>>>>>> focused on >> > >>>>>>>>>> cases 1,2,3 and 7. >> > >>>>>>>>>> >> > >>>>>>>>>> *Motion is to adopt the following:* >> > >>>>>>>>>> >> > >>>>>>>>>> Case 1 - Solution A - Get rid of contrib >> > >>>>>>>>>> All objects will be moved out of contrib folder and into >> > >>>> respective >> > >>>>>>>>>> non-contrib folder. >> > >>>>>>>>>> >> > >>>>>>>>>> Case 2 - Solution A - Remove object type suffix from modules >> > >>>>>>>>>> Example: >> > >>>>>>>>>> Airflow.operators.foo_operator.FooOperator becomes >> > >>>>>>>>>> airflow.operators.foo.FooOperator >> > >>>>>>>>>> Example: >> > >>>>>>>>>> Airflow.hooks.foo_hook.FooOperator becomes >> > >>>> airflow.hooks.foo.FooHook >> > >>>>>>>>>> >> > >>>>>>>>>> Case 3 - conventions for cloud provider etc prefixes >> > >>>>>>>>>> *I am not sure exactly what the proposal is.* >> > >>>>>>>>>> Example case is >> > >>>> airflow/contrib/operators/gcp_bigtable_operator.py >> > >>>>>>>>>> Since motion adopts case 1 solution A, we can assume it is >> now >> > >>>> named >> > >>>>>>>> like >> > >>>>>>>>>> so: airflow/operators/gcp_bigtable_operator.py >> > >>>>>>>>>> So what is proposed for this case? >> > >>>>>>>>>> >> > >>>>>>>>>> Case 4 - Solution B - *no change* >> > >>>>>>>>>> This proposal was about namespaces but the motion is to >> reject >> > >>>> the >> > >>>>>>>>>> proposal. >> > >>>>>>>>>> >> > >>>>>>>>>> Case 5 - Solution B - *no change* >> > >>>>>>>>>> The proposal was to remove class type suffix e.g. remove the >> > >>>>>>>> Operator in >> > >>>>>>>>>> FooOperator, but the motion is to reject this proposal -- >> i.e. >> > >>>>>>>> motion is >> > >>>>>>>>>> no >> > >>>>>>>>>> change on this case, i.e. we resolve to keep "Operator" (and >> > >>>>>>>> "Sensor") >> > >>>>>>>>>> suffixes on class names >> > >>>>>>>>>> >> > >>>>>>>>>> Case 6 - *No change* >> > >>>>>>>>>> This case concerns object naming inconsistencies, but motion >> > >> does >> > >>>>>>>> not >> > >>>>>>>>>> enact >> > >>>>>>>>>> any specific proposal. >> > >>>>>>>>>> >> > >>>>>>>>>> Case 7 - Solution A - use warnings.warn template for >> > >> deprecation >> > >>>>>>>> warnings >> > >>>>>>>>>> (instead of zope) >> > >>>>>>>>>> Example: >> > >>>>>>>>>> >> > >>>>>>>>>> # in old_package/old_mod.py >> > >>>>>>>>>> >> > >>>>>>>>>> import warnings >> > >>>>>>>>>> from new_package.new_mod import * >> > >>>>>>>>>> warnings.warn("old_package.old_mod has moved to >> > >>>> new_package.new_mod. >> > >>>>>>>>>> Import >> > >>>>>>>>>> of " >> > >>>>>>>>>> "old_package.old_mod will become unsupported in version 2", >> > >>>>>>>>>> DeprecationWarning, 2) >> > >>>>>>>>>> >> > >>>>>>>>>> Reference implementation here: >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>> >> > >>>>>>> >> > >>>>> >> > >>>> >> > >> >> > >> https://github.com/mik-laj/airflow-deprecation-sample/tree/master/solution1 >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> FWIW +1 (non-binding) on 1,2,7 -- unsure on 3. >> > >>>>>>>>>> >> > >>>>>>>>>> I am very happy that this motion now gets rid of contrib. >> > >> Thank >> > >>>> you >> > >>>>>>>>>> Sergio >> > >>>>>>>>>> for turning the tide with your effective argumentation ;) >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> On Fri, Jul 26, 2019 at 3:16 AM Ash Berlin-Taylor < >> > >>>> a...@apache.org> >> > >>>>>>>> wrote: >> > >>>>>>>>>> >> > >>>>>>>>>>> Could someone summarise what the proposed name changes are, >> > >>>> here, >> > >>>>>>>> in >> > >>>>>>>>>> this >> > >>>>>>>>>>> thread? Pointing at a google doc and only giving partial >> > >>>> examples >> > >>>>>>>> is 1) >> > >>>>>>>>>> a >> > >>>>>>>>>>> bit difficult to quickly work out what we are voting on, and >> > >> 2) >> > >>>>>>>> using a >> > >>>>>>>>>>> google doc isn't great for a longer term record. >> > >>>>>>>>>>> >> > >>>>>>>>>>> Thanks, >> > >>>>>>>>>>> Ash >> > >>>>>>>>>>> >> > >>>>>>>>>>>> On 26 Jul 2019, at 09:14, Jarek Potiuk >> > >>>>>>>> <jarek.pot...@polidea.com> >> > >>>>>>>>>> wrote: >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> I would like to recast the vote. Let's start from 0 :) . I >> > >>>> would >> > >>>>>>>> like >> > >>>>>>>>>> to >> > >>>>>>>>>>>> keep the July 30th 6pm CEST date, and at least three +1 >> > >>>>>>>> (binding) >> > >>>>>>>>>> votes >> > >>>>>>>>>>> are >> > >>>>>>>>>>>> needed to pass. I marked in red the modifications to the >> > >>>>>>>> previous >> > >>>>>>>>>>> proposal. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> Here is the proposal (details here >> > >>>>>>>>>>>> < >> > >>>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>> >> > >>>>>>> >> > >>>>> >> > >>>> >> > >> >> > >> https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit#heading=h.j4jc3i70qszo >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> ) >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> - *Case 1: A: Get rid of contrib,* >> > >>>>>>>>>>>> - *Case 2: A: Airflow.operators.foo_operator.FooOperator >> > >>>> could >> > >>>>>>>>>>>> become airflow.operators.foo.FooOperator* >> > >>>>>>>>>>>> - *Case 3: C: >> > >>>>>>>>>>>> >> > >>>>>>>> >> airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator >> > >>>>>>>>>> could >> > >>>>>>>>>>>> become airflow.gcp.operators.bigtable.BigTableOperator* >> > >>>>>>>>>>>> - *Case 4: B: no namespace introduction* >> > >>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor") suffixes on >> > >>>> class >> > >>>>>>>>>> names* >> > >>>>>>>>>>>> - *Case 6: We will treat isolated cases on case-by-case >> > >>>> (and >> > >>>>>>>> my team >> > >>>>>>>>>>> can >> > >>>>>>>>>>>> do the job of GCP-related operators)* >> > >>>>>>>>>>>> - *Case 7: Native python solution (with better IDE >> > >>>>>>>> integration)* >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> This is my binding (+1) vote. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> On Fri, Jul 26, 2019 at 10:08 AM Jarek Potiuk < >> > >>>>>>>>>> jarek.pot...@polidea.com> >> > >>>>>>>>>>>> wrote: >> > >>>>>>>>>>>> >> > >>>>>>>>>>>>> Hey Felix, >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> For 7* -> I am in favour of trying the native one as well >> > >> (I >> > >>>>>>>> use IDE >> > >>>>>>>>>>> quite >> > >>>>>>>>>>>>> often ;) ) >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> J. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> On Wed, Jul 24, 2019 at 9:34 AM Driesprong, Fokko >> > >>>>>>>>>> <fo...@driesprong.frl >> > >>>>>>>>>>>> >> > >>>>>>>>>>>>> wrote: >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>>> Thanks Kamil for putting the document together. I wasn't >> > >>>> able >> > >>>>>>>> to >> > >>>>>>>>>>> respond >> > >>>>>>>>>>>>>> earlier since I was giving Airflow workshops throughout >> > >>>> Europe >> > >>>>>>>> :-) >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> *Case 1: *Solution A → Remove everything from contrib >> > >> into >> > >>>> a >> > >>>>>>>> single >> > >>>>>>>>>>>>>> package. >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> To make it easier to the end-user, my preference would be >> > >>>> to >> > >>>>>>>> get >> > >>>>>>>>>> rid of >> > >>>>>>>>>>>>>> the >> > >>>>>>>>>>>>>> contrib package altogether. What's in contrib or not is a >> > >>>>>>>> tricky >> > >>>>>>>>>>> question, >> > >>>>>>>>>>>>>> I think this is already reflected in the document since >> > >>>>>>>> "maturity" >> > >>>>>>>>>> is >> > >>>>>>>>>>> in >> > >>>>>>>>>>>>>> quotes. What would be the moment to transfer the operator >> > >>>> to >> > >>>>>>>> the >> > >>>>>>>>>> core >> > >>>>>>>>>>> or >> > >>>>>>>>>>>>>> vice versa? I'm afraid that renaming contrib to >> > >> incubating >> > >>>>>>>> will >> > >>>>>>>>>> confuse >> > >>>>>>>>>>>>>> the >> > >>>>>>>>>>>>>> user even more. For people who aren't as known with >> > >>>> Airflow it >> > >>>>>>>> isn't >> > >>>>>>>>>>> that >> > >>>>>>>>>>>>>> transparent which operator lives where, especially if you >> > >>>>>>>> don't use >> > >>>>>>>>>> an >> > >>>>>>>>>>> IDE >> > >>>>>>>>>>>>>> such as Ash ;) Besides that I don't think it is worth >> > >>>> changing >> > >>>>>>>> the >> > >>>>>>>>>>> public >> > >>>>>>>>>>>>>> API just for a different name. >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> Ok. I am convinced. I will re-cast the vote with this. It >> > >>>> will >> > >>>>>>>> make >> > >>>>>>>>>>> easier >> > >>>>>>>>>>>>> as we will not have to define other rules as those that we >> > >>>>>>>> already >> > >>>>>>>>>> have. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>>> *Case 2:* Slight preference to Solution B (let's say a >> > >> +0) >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> I like to search on Github with t, and then search for >> > >>>>>>>> BashOperator. >> > >>>>>>>>>>> After >> > >>>>>>>>>>>>>> the change, you should search for Operator/Bash. >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> Jarek, I'm confused with your vote on this, from your >> > >> mail >> > >>>> you >> > >>>>>>>>>> state: - >> > >>>>>>>>>>>>>> *Case 2: B: Airflow.operators.foo_operator.FooOperator >> > >>>> could >> > >>>>>>>> become >> > >>>>>>>>>>>>>> airflow.operators.foo.FooOperator*. But the B solution is >> > >>>>>>>> keeping it >> > >>>>>>>>>>> the >> > >>>>>>>>>>>>>> same, so that would mean - *Case 2: B: >> > >>>>>>>>>>>>>> Airflow.operators.foo_operator.FooOperator *would stay >> > >>>>>>>>>>>>>> airflow.operators.foo_operator*.FooOperator* >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> My mistake. It was of course A. I think there is a little >> > >>>>>>>> confusion >> > >>>>>>>>>>> here. >> > >>>>>>>>>>>>> This case is not for changing BashOperator into Bash but >> > >> for >> > >>>>>>>> changing >> > >>>>>>>>>>> the >> > >>>>>>>>>>>>> *module* name not the *class* name. So >> > >>>>>>>> bash_operator.BashOperator -> >> > >>>>>>>>>>>>> bash.BashOperator :) . you will still search for >> > >>>> BashOperator >> > >>>>>>>> in this >> > >>>>>>>>>>> case. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>>> *Case 3:* I'm leaning towards B >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> Mostly because it isn't as trivial to decide when it gets >> > >>>> its >> > >>>>>>>> own >> > >>>>>>>>>>> package >> > >>>>>>>>>>>>>> or not. Besides the cloud providers, there are companies >> > >>>> like >> > >>>>>>>>>>> Databricks >> > >>>>>>>>>>>>>> and Qubole which might also end up with their own package >> > >>>>>>>> because >> > >>>>>>>>>> their >> > >>>>>>>>>>>>>> integration is getting richer over time. Moving this is a >> > >>>> bit >> > >>>>>>>>>> painful >> > >>>>>>>>>>>>>> because we need to deprecate the old path and move >> > >>>> everything >> > >>>>>>>> which >> > >>>>>>>>>>>>>> introduces noise into version control etc. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> I'd say, we should go for C. As I proposed. It's not as >> > >>>>>>>> difficult as >> > >>>>>>>>>> it >> > >>>>>>>>>>>>> seems to group operators in packages and it has numerous >> > >>>>>>>> advantages. >> > >>>>>>>>>> The >> > >>>>>>>>>>>>> nice thing about deprecations - we can do them in the >> > >>>>>>>> __init__.py of >> > >>>>>>>>>> the >> > >>>>>>>>>>>>> packages which causes the noise fairly small (Git will >> > >>>> actually >> > >>>>>>>>>> realise >> > >>>>>>>>>>>>> that the file has been moved and you can follow that. >> > >> Maybe >> > >>>> not >> > >>>>>>>> as >> > >>>>>>>>>> super >> > >>>>>>>>>>>>> easy to merge changes later to 1.10.4 but it's not a >> > >> rocket >> > >>>>>>>> science >> > >>>>>>>>>>> either. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>>> *Case 7:* Python native solution >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> Yes. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> --- >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> Apart from all, great work! >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> Cheers, Fokko >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> Op di 23 jul. 2019 om 21:27 schreef Felix Uellendall >> > >>>>>>>>>>>>>> <felue...@pm.me.invalid >> > >>>>>>>>>>>>>>> : >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> I have no strong "No" against any proposed change of >> > >> these >> > >>>>>>>> cases. >> > >>>>>>>>>> So I >> > >>>>>>>>>>>>>> go >> > >>>>>>>>>>>>>>> with +1 (non-binding). >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> P.S. Thanks Jarek for bringing this up again and your >> > >>>> intense >> > >>>>>>>> work >> > >>>>>>>>>>>>>> towards >> > >>>>>>>>>>>>>>> airflow currently :) and thanks to Kamil for even >> > >> creating >> > >>>>>>>> this >> > >>>>>>>>>>>>>> document. I >> > >>>>>>>>>>>>>>> like how the code is getting more and more consistent >> > >> and >> > >>>>>>>> clean :) >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Kind regards, >> > >>>>>>>>>>>>>>> Felix >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Sent from ProtonMail mobile >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> -------- Original Message -------- >> > >>>>>>>>>>>>>>> On Jul 23, 2019, 18:34, Jarek Potiuk wrote: >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> Hello everyone, >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> This email is calling a vote on the changes in import >> > >>>> paths. >> > >>>>>>>> It's >> > >>>>>>>>>>> been >> > >>>>>>>>>>>>>>>> discussed in >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>> >> > >>>>>>> >> > >>>>> >> > >>>> >> > >> >> > >> https://lists.apache.org/thread.html/4e648d9421c792d4537f5ac66f1a16dce468f816fc5221a9f9db9433@%3Cdev.airflow.apache.org%3E >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> The vote will last for at least 1 week (July 30th 6pm >> > >>>> CEST), >> > >>>>>>>> and >> > >>>>>>>>>> at >> > >>>>>>>>>>>>>> least >> > >>>>>>>>>>>>>>>> three +1 (binding) votes have been cast. >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> The proposal to vote is based on the document from >> > >> Kamil >> > >>>>>>>>>>>>>>>> < >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>> >> > >>>>>>> >> > >>>>> >> > >>>> >> > >> >> > >> https://docs.google.com/document/d/1F8zve5S78DXcjpPttW89HnqT0M0iKjT6fo9jX57Ef6A/edit# >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> The proposed solution is: >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> - *Case 1: B: Contrib vs not: we move all that are >> > >> "well" >> > >>>>>>>> tested >> > >>>>>>>>>> and >> > >>>>>>>>>>>>>>>> rename contrib to "incubating" or similar.* >> > >>>>>>>>>>>>>>>> - *Case 2: B: >> > >> Airflow.operators.foo_operator.FooOperator >> > >>>>>>>> could >> > >>>>>>>>>>>>>>>> become airflow.operators.foo.FooOperator* >> > >>>>>>>>>>>>>>>> - *Case 3: C: >> > >>>>>>>>>>>>>>>> >> > >>>>>>>> >> airflow.contrib.operators.gcp_bigtable_operator.BigTableOperator >> > >>>>>>>>>>> could >> > >>>>>>>>>>>>>>>> become airflow.gcp.operators.bigtable.BigTableOperator* >> > >>>>>>>>>>>>>>>> - *Case 4: B: no namespace introduction* >> > >>>>>>>>>>>>>>>> - *Case 5: B: Keep "Operator" (and "Sensor") suffixes >> > >> on >> > >>>>>>>> class >> > >>>>>>>>>> names* >> > >>>>>>>>>>>>>>>> - *Case 6: We will treat isolated cases on case-by-case >> > >>>> (and >> > >>>>>>>> my >> > >>>>>>>>>> team >> > >>>>>>>>>>>>>> can >> > >>>>>>>>>>>>>>>> do the job of GCP-related operators)* >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> This is my (binding) +1 vote. >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> Best regards, >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> J. >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> -- >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> Jarek Potiuk >> > >>>>>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal >> > >> Software >> > >>>>>>>> Engineer >> > >>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>> M: [+48 660 796 129](tel:+48660796129) >> > >>>>>>>>>>>>>> <[+48660796129](tel:+48660796129)> >> > >>>>>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> -- >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> Jarek Potiuk >> > >>>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software >> > >>>>>>>> Engineer >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> M: +48 660 796 129 <+48660796129> >> > >>>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> -- >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> Jarek Potiuk >> > >>>>>>>>>>>> Polidea <https://www.polidea.com/> | Principal Software >> > >>>>> Engineer >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> M: +48 660 796 129 <+48660796129> >> > >>>>>>>>>>>> [image: Polidea] <https://www.polidea.com/> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> -- >> > >>>>>>>>> *Kaxil Naik* >> > >>>>>>>>> *Big Data Consultant | DevOps Data Engineer* >> > >>>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* Apache >> > >> Spark >> > >>>> & >> > >>>>>>>> Neo4j >> > >>>>>>>>> Developer >> > >>>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil >> > >>>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> -- >> > >>>>>>>> *Kaxil Naik* >> > >>>>>>>> *Big Data Consultant | DevOps Data Engineer* >> > >>>>>>>> *Certified *Google Cloud Data Engineer | *Certified* Apache >> Spark >> > >> & >> > >>>>>>>> Neo4j >> > >>>>>>>> Developer >> > >>>>>>>> *LinkedIn*: https://www.linkedin.com/in/kaxil >> > >>>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>> >> > >> >> > >> -- >> > >> >> > >> Jarek Potiuk >> > >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> > >> >> > >> M: +48 660 796 129 <+48660796129> >> > >> [image: Polidea] <https://www.polidea.com/> >> > >> >> > >> > >> >> -- >> *Kaxil Naik* >> *Big Data Consultant | DevOps Data Engineer* >> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j >> Developer >> *LinkedIn*: https://www.linkedin.com/in/kaxil >> > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
