On Sat, May 30, 2020 at 2:31 AM Jarek Potiuk wrote:
>
> Ash - QP: also - as Kamil mentioned he already generated multiple clients
> in https://github.com/mik-laj/airflow-api-clients/tree/master/out/object
> so if you have time , maybe you can also take a look in parallel.
This is a good idea. I t
1 looks nice and clean
Op za 30 mei 2020 om 11:37 schreef Jarek Potiuk
> That's a completely new thing/idea.
>
> We might have separate vote for that. Kamil - can you please lead the vote
> and renaming if it happens?
>
> I am for the change. I think it would make sense to separate them.
>
> I w
+1 for [1] XToYOperator
On Sat, May 30, 2020 at 7:33 AM Felix Uellendall
wrote:
> +1 for [1] XToYOperator
>
> Best Regards,
> Felix
>
> Sent from ProtonMail Mobile
>
> On Sat, May 30, 2020 at 13:58, Kaxil Naik wrote:
>
> > +1 for [1] XToYOperator
> >
> > On Sat, May 30, 2020, 12:56 Tomasz Urbas
Hey QP - maybe you can help. Seems that we currently can't even use the
generated code because it simply fails to compile with missing class (same
for Ash):
import org.openapitools.client.model.OneOfScheduleInterval;
Seems that it comes from this bug in the OpenApi generator :
https://github.com/
@Ash - Another example - the talk we gave with Tomek in Mexico for one (but
we gave it at least 5 times already) - link to the very place we talk about
it: https://youtu.be/06tYjcQ12i4?t=1210 :).
@Kaxil-> I think it will be better to make a decision before we make
Backport Operators release if we
Ah https://airflow.apache.org/docs/1.10.10/_api/index.html#operators --
I haven't looked at that content closely in a long while.
On May 30 2020, at 5:40 pm, Ash Berlin-Taylor wrote:
> Oh really, can you point at any talks? I've not seen them referrred to
> as a first class concept before.
>
>
I am in favor of the idea and have a BaseTransferOperator similar to
BaseSensorOperator. There was a PR at some point to implement a common
interface for this, will check it once I am back to Laptoo.
Although this should not block the release of Backport Packages and can
happen in the near future.
Oh really, can you point at any talks? I've not seen them referrred to
as a first class concept before.
(Sensors are also different because they can be rescheduled -
TransferOperators aren't currently in any way more special to Airflow
than any other operator.)
-a
On May 30 2020, at 5:31 pm, Jar
Well. Sensors are based on BaseSensorOperator which is based on
BaseOperator. So they are technically also operators (just
grand-children). In the future maybe we will have BaseTransferOperator
as well (as per our discussion in Slack). I think this might make
sense to have transfer at the same leve
I understand what you mean, and some separation could make sense, but transfer
operators are still operators (unlike sensors which are a different
interface/base class).
How would people feel about airflow.providers.*. operators.transfer.X?
On 30 May 2020 15:36:42 BST, Felix Uellendall wrote:
I really like the idea, too. It makes the users search for transfers a lot
easier.
Best Regards,
Felix
Sent from ProtonMail Mobile
On Sat, May 30, 2020 at 15:59, Jarek Potiuk wrote:
> I definitely like the idea - if we can discussion and voting on it before
> we prepare backport package final
+1 for [1] XToYOperator
Best Regards,
Felix
Sent from ProtonMail Mobile
On Sat, May 30, 2020 at 13:58, Kaxil Naik wrote:
> +1 for [1] XToYOperator
>
> On Sat, May 30, 2020, 12:56 Tomasz Urbaszek wrote:
>
>> +1 for XToYOperator
>>
>> On Sat, May 30, 2020 at 1:30 PM Bas Harenslak
>> wrote:
>>
I definitely like the idea - if we can discussion and voting on it before
we prepare backport package final (???) RC candidate, that would be great.
On Sat, May 30, 2020 at 3:31 PM Tomasz Urbaszek
wrote:
> Oh, makes even more sense. I like the idea.
>
> T.
>
> On Sat, May 30, 2020 at 3:14 PM Kam
Oh, makes even more sense. I like the idea.
T.
On Sat, May 30, 2020 at 3:14 PM Kamil Breguła
wrote:
> I don't want to create package
> airflow.providers.transfer
> but package
> airflow.providers.*.transfer
> All classes will be in the same provider package but in a different nested
> package.
I don't want to create package
airflow.providers.transfer
but package
airflow.providers.*.transfer
All classes will be in the same provider package but in a different nested
package. You will need to import packages from the transfer package, not
the operator package.
On Sat, May 30, 2020, 15:05
I am in favor of this change. However, do I correctly understand that
providers.transfer will need many other providers (GCS=gcp, S3=aws,
etc)?
T.
On Sat, May 30, 2020 at 2:28 PM Kamil Breguła wrote:
>
> Hello,
>
> We currently have transfer operators and other operators in one
> package - oper
Hello,
We currently have transfer operators and other operators in one
package - operators.
This causes some packages to be large and will grow all the time. For
example: airflow.providers.google.cloud.operators have 52 modules
For this reason, the idea arose to move these operators to the new pac
+1 for [1] XToYOperator
On Sat, May 30, 2020, 12:56 Tomasz Urbaszek wrote:
> +1 for XToYOperator
>
> On Sat, May 30, 2020 at 1:30 PM Bas Harenslak
> wrote:
>
> > #1 -> XToYOperator
> >
> > > On 30 May 2020, at 12:09, Ash Berlin-Taylor wrote:
> > >
> > > +1 for [1] XToYOperator
> > >
> > > On M
+1 for XToYOperator
On Sat, May 30, 2020 at 1:30 PM Bas Harenslak
wrote:
> #1 -> XToYOperator
>
> > On 30 May 2020, at 12:09, Ash Berlin-Taylor wrote:
> >
> > +1 for [1] XToYOperator
> >
> > On May 30 2020, at 10:03 am, Jarek Potiuk
> wrote:
> >
> >> Seems this one will be easier to settle, th
#1 -> XToYOperator
> On 30 May 2020, at 12:09, Ash Berlin-Taylor wrote:
>
> +1 for [1] XToYOperator
>
> On May 30 2020, at 10:03 am, Jarek Potiuk wrote:
>
>> Seems this one will be easier to settle, there is not much controversy :)
>> Following the discussion
>> https://lists.apache.org/threa
+1 for [1] XToYOperator
On May 30 2020, at 10:03 am, Jarek Potiuk wrote:
> Seems this one will be easier to settle, there is not much controversy :)
> Following the discussion
> https://lists.apache.org/thread.html/r456565d372dcd7199d5ca34881d43911bf2c6ce623e515252a3aec73%40%3Cdev.airflow.apache
That's a completely new thing/idea.
We might have separate vote for that. Kamil - can you please lead the vote
and renaming if it happens?
I am for the change. I think it would make sense to separate them.
I wonder what others think.
J.
On Sat, May 30, 2020 at 11:11 AM Kamil Breguła
wrote:
Ash - QP: also - as Kamil mentioned he already generated multiple clients
in https://github.com/mik-laj/airflow-api-clients/tree/master/out/object
so if you have time , maybe you can also take a look in parallel.
On Sat, May 30, 2020 at 11:29 AM Jarek Potiuk
wrote:
> Kamil - I believe you alread
Kamil - I believe you already have those clients generated - so i will use
them from there.
J,.
On Sat, May 30, 2020 at 11:26 AM Jarek Potiuk
wrote:
> OK. Indeed there seem to be quite some ambiguities and inconsistencies
> and the knowledge we are basing our discussion on is a bit fragmented
>
OK. Indeed there seem to be quite some ambiguities and inconsistencies
and the knowledge we are basing our discussion on is a bit fragmented
and "abstract".
Sorry for being a bit pushy but I think we are not doing enough as a
community to make life our users easier, and we sometimes seem to
procee
I would like to move transfer operators to the new package - transfer
airflow.providers.*.transfer.*
airflow.providers.*.operator.*
airflow.providers.*.sensor.*
Transfer operators are very often separated in the documentation and it
would be helpful if this division was included in the code.
https
HAH Again canceling... seems like a routine now :).
Seems that enough people (everyone who spoke) care for consistency and
since (as discussed in
https://lists.apache.org/thread.html/r456565d372dcd7199d5ca34881d43911bf2c6ce623e515252a3aec73%40%3Cdev.airflow.apache.org%3E
)
we found an inconsis
Seems this one will be easier to settle, there is not much controversy :)
Following the discussion
https://lists.apache.org/thread.html/r456565d372dcd7199d5ca34881d43911bf2c6ce623e515252a3aec73%40%3Cdev.airflow.apache.org%3E
I would like to cast a vote on the standard of naming the transfer
operat
28 matches
Mail list logo
