>
> @Kaxil-> I think it will be better to make a decision before we make
> Backport Operators release if we think it is a good direction. The change
> is relatively easy to do and automate, and I think the main idea of
> backport operators is to make it easy for people to switch to 2.0 operators
>
Should we start voting on it? I think if we do it now and we agree on
moving - we have the opportunity to release RC4 on Wednesday and release
final version on Monday next week.
I'd love to have all the moves already in place with the final Backport
release. That would make it really easy for peop
@Ash - Another example - the talk we gave with Tomek in Mexico for one (but
we gave it at least 5 times already) - link to the very place we talk about
it: https://youtu.be/06tYjcQ12i4?t=1210 :).
@Kaxil-> I think it will be better to make a decision before we make
Backport Operators release if we
Ah https://airflow.apache.org/docs/1.10.10/_api/index.html#operators --
I haven't looked at that content closely in a long while.
On May 30 2020, at 5:40 pm, Ash Berlin-Taylor wrote:
> Oh really, can you point at any talks? I've not seen them referrred to
> as a first class concept before.
>
>
I am in favor of the idea and have a BaseTransferOperator similar to
BaseSensorOperator. There was a PR at some point to implement a common
interface for this, will check it once I am back to Laptoo.
Although this should not block the release of Backport Packages and can
happen in the near future.
Oh really, can you point at any talks? I've not seen them referrred to
as a first class concept before.
(Sensors are also different because they can be rescheduled -
TransferOperators aren't currently in any way more special to Airflow
than any other operator.)
-a
On May 30 2020, at 5:31 pm, Jar
Well. Sensors are based on BaseSensorOperator which is based on
BaseOperator. So they are technically also operators (just
grand-children). In the future maybe we will have BaseTransferOperator
as well (as per our discussion in Slack). I think this might make
sense to have transfer at the same leve
I understand what you mean, and some separation could make sense, but transfer
operators are still operators (unlike sensors which are a different
interface/base class).
How would people feel about airflow.providers.*. operators.transfer.X?
On 30 May 2020 15:36:42 BST, Felix Uellendall wrote:
I really like the idea, too. It makes the users search for transfers a lot
easier.
Best Regards,
Felix
Sent from ProtonMail Mobile
On Sat, May 30, 2020 at 15:59, Jarek Potiuk wrote:
> I definitely like the idea - if we can discussion and voting on it before
> we prepare backport package final
I definitely like the idea - if we can discussion and voting on it before
we prepare backport package final (???) RC candidate, that would be great.
On Sat, May 30, 2020 at 3:31 PM Tomasz Urbaszek
wrote:
> Oh, makes even more sense. I like the idea.
>
> T.
>
> On Sat, May 30, 2020 at 3:14 PM Kam
Oh, makes even more sense. I like the idea.
T.
On Sat, May 30, 2020 at 3:14 PM Kamil Breguła
wrote:
> I don't want to create package
> airflow.providers.transfer
> but package
> airflow.providers.*.transfer
> All classes will be in the same provider package but in a different nested
> package.
I don't want to create package
airflow.providers.transfer
but package
airflow.providers.*.transfer
All classes will be in the same provider package but in a different nested
package. You will need to import packages from the transfer package, not
the operator package.
On Sat, May 30, 2020, 15:05
I am in favor of this change. However, do I correctly understand that
providers.transfer will need many other providers (GCS=gcp, S3=aws,
etc)?
T.
On Sat, May 30, 2020 at 2:28 PM Kamil Breguła wrote:
>
> Hello,
>
> We currently have transfer operators and other operators in one
> package - oper
Hello,
We currently have transfer operators and other operators in one
package - operators.
This causes some packages to be large and will grow all the time. For
example: airflow.providers.google.cloud.operators have 52 modules
For this reason, the idea arose to move these operators to the new pac
14 matches
Mail list logo
