+1 for providers
Having google, azure, etc. right next to hooks, operators modules just does not
fit in the context of this module level imo. Except for the kubernetes module
all module names are more generic.
Felix
Sent from ProtonMail Mobile
On Fri, Oct 4, 2019 at 21:05, Driesprong, Fokko <fo...@driesprong.frl> wrote:
> I agree with Jarek's suggestion to add another module. Personally I don't
> like having too many on the airflow._ level, there are quite a few already.
>
> Also, I would also go for providers. Mostly because we have hook*s* and
> operator*s*. When there are a lot of {operators,hooks,sensors} it makes
> sense to package these in a separate module. I could imagine that other
> companies that provide managed services
> <https://airflow.readthedocs.io/en/latest/operators-and-hooks-ref.html#service-integrations>
> might
> evolve to a separate package, but maybe we should come up with some
> criteria.
>
> Cheers, Fokko
>
> Op vr 4 okt. 2019 om 19:00 schreef Jarek Potiuk <jarek.pot...@polidea.com>:
>
>> Not that I want to open Pandora's box :). But I think the longer name - the
>> worse. Provider is quite nice and short.
>>
>> I agree with plural 'providers' (as we have hooks and operators). So for
>> consistency it should be indeed plural.
>>
>> J.
>>
>> pt., 4 paź 2019, 18:48 użytkownik Chris Palmer <ch...@crpalmer.com>
>> napisał:
>>
>> > My question about Oracle/MySql wasn't a serious one, but I forget
>> sometimes
>> > that sarcasm doesn't come across well on email.
>> >
>> > I guess my objection is that I don't think that 'provider' adds anything
>> of
>> > value. I'm not convinced that there needs to be a level between 'airflow'
>> > and 'google' but if going that route I would advocate for at least the
>> > plural form 'providers' or the more descriptive 'cloud_providers'.
>> >
>> > Chris
>> >
>> > On Fri, Oct 4, 2019 at 11:57 AM Kamil Breguła <kamil.breg...@polidea.com
>> >
>> > wrote:
>> >
>> > > Hello,
>> > >
>> > > We think that only cloud providers should be separated from others,
>> > > because these services are integrated with each other. Very often,
>> > > when you use one cloud provider, you use many services of a given
>> > > provider. Using a single provider solution provides a uniform way of
>> > > authorization, etc. A large number of problems and mechanisms are
>> > > common to one provider. You can see the amount of integration from
>> > > Microsoft, Azure, Google on the reference list.
>> > > https://airflow.readthedocs.io/en/latest/operators-and-hooks-ref.html
>> > > In this list, cloud reference providers have been placed in separate
>> > > tables because they have a very large number of services. If Oracle
>> > > will have many integrations, it is worth emphasizing this fact and
>> > > moving these integrations to a separate place, so that it is easier to
>> > > find them. and use. Keeping all possible files in one place makes it
>> > > very difficult to use them.
>> > >
>> > > Best regards,
>> > >
>> > > On Fri, Oct 4, 2019 at 5:32 PM Chris Palmer <ch...@crpalmer.com>
>> wrote:
>> > > >
>> > > > This seems unnecessary to me.
>> > > >
>> > > > Is everything going to be under some 'provider' or just certain sets
>> of
>> > > > operators, and if so what differentiates when something should be
>> > under a
>> > > > provider or not? For example, are the mysql operators going to go
>> under
>> > > > 'provider/oracle/'?
>> > > >
>> > > > Chris
>> > > >
>> > > > On Fri, Oct 4, 2019 at 9:21 AM Jarek Potiuk <
>> jarek.pot...@polidea.com>
>> > > > wrote:
>> > > >
>> > > > > Agree with Ash.
>> > > > >
>> > > > > After doing the gcp move and seeing the result we agreed that
>> > > 'provider' is
>> > > > > better as additional prefix.
>> > > > >
>> > > > > If no-one objects (Lazy Consensus
>> > > > > <https://community.apache.org/committers/lazyConsensus.html>) till
>> > > Monday
>> > > > > 3.20 CEST, we will update AIP-21 and move the gcp operators to
>> > > > > *provider/google/[gcp,gsuite]*.
>> > > > >
>> > > > > J.
>> > > > >
>> > >
>> >
>>
