+1 from my side. Sounds reasonable and there have not been a better idea
spreading in my head besides SDK seems to be settled.
For the rework besides docs, of course all examples need a clean/tidy-up
as well.
On 17.09.24 13:04, Ash Berlin-Taylor wrote:
Coming back to this after the Summit, I ha
Regarding the warning, maybe a PendingDeprecationWarning would be appropriate?
It is arguably not too useful in general, but does somewhat express the
intention here.
--
Sent from my iPhone
> On 17 Sep 2024, at 19:05, Ash Berlin-Taylor wrote:
>
> Coming back to this after the Summit, I have
Coming back to this after the Summit, I have refined my thoughts some.
As a reminder, The things we currently allow to be imported from the top-level
Airflow module is this:
https://github.com/apache/airflow/blob/8fe286108720ba1f9a303c066022349d6656f735/airflow/__init__.py#L77-L83
```
"DAG"
To add another DAG Author perspective, I'd vote for:
#1 (from airflow ...) but without side-effects
or #2 (from airflow.sdk ...).
To compare with other X-as-code tools:
- Luigi has top-level *Luigi.task* (is class-based)
- Prefect has top-level *from prefect import flow, task* (and seems to
refer
Chiming in here mostly from the DAG author perspective!
I like `airflow.sdk` best. It makes it super clear what the user is supposed to
interact with and what Airflow’s “public” interface is. Importing from
`airflow.models` has always felt weird because it feels like you’re going into
Airflow’s
Yep so. If we do not have side-effects from import airflow -> my vote would
be "airflow.sdk" :)
On Mon, Sep 2, 2024 at 10:29 AM Ash Berlin-Taylor wrote:
> Yes, strongly agreed on the “no side-effects form `import airflow`”.
>
> To summarise the options so far:
>
> 1. `from airflow import DAG, Ta
Yes, strongly agreed on the “no side-effects form `import airflow`”.
To summarise the options so far:
1. `from airflow import DAG, TaskGroup` — have the imports be from the top
level airflow module
2. `from airflow.definitions import DAG, TaskGroup`
3. `from airflow.sdk import DAG, TaskGroup`
>
Should be:
```
@configure_settings
@configure_worker_plugins
def cli_worker():
pass
```
On Sun, Sep 1, 2024 at 12:05 AM Jarek Potiuk wrote:
> Personally for me "airflow.sdk" is best and very straightforward. And we
> have not yet used that for other things before, so it's free to use.
>
> "
Personally for me "airflow.sdk" is best and very straightforward. And we
have not yet used that for other things before, so it's free to use.
"Models" and similar carried more (often misleading) information - they
were sometimes database models, sometimes they were not. This caused a lot
of confus
Hi Ash,
I was thinking hard... was setting the email aside and still have no
real _good_ ideas. I am still good with "models" and "sdk".
Actually what we want to define is an "execution interface" to which the
structual model as API in Python/or other language gives bindings and
helper methods.
> As a side note, I wonder if we should do the user-internal separation better
> for DagRun and TaskInstance
Yes, that is a somewhat inevitable side effect of making it be behind an API,
and one I am looking forward to. There are almost just plain-data classes (but
not using data classes per se
> On 30 Aug 2024, at 17:48, Ash Berlin-Taylor wrote:
>
> Where should DAG, TaskGroup, Labels, decorators etc for authoring be imported
> from inside the DAG files? Similarly for DagRun, TaskInstance (these two
> likely won’t be created directly by users, but just used for reference
> docs/t
I'm partial to everything that we expect users to use to be importable from
`airflow`, but would love to hear other people's thoughts.
On Fri, Aug 30, 2024 at 5:48 AM Ash Berlin-Taylor wrote:
> Hi everyone,
>
> It’s time to have a another discussion about everyone's favourite
> discussion - nami
Hi everyone,
It’s time to have a another discussion about everyone's favourite discussion -
naming things!
Tl;dr if you have all of AIP-72 and its implications loaded in your head
already:
##
Where should DAG, TaskGroup, Labels, decorators etc for authoring be imported
from inside the DAG fil
14 matches
Mail list logo
