mpgreg commented on issue #34206:
URL: https://github.com/apache/airflow/issues/34206#issuecomment-1837468294
This is something that will be necessary for MLops use cases.
@jscheffl if I'm understanding your suggestion correctly couldn't we just
use extras to store dynamic content?
jaklan commented on issue #34206:
URL: https://github.com/apache/airflow/issues/34206#issuecomment-2303625639
Are there any updates on that? I was just reading docs for Dataset Aliases,
but they don't seem to resolve the issue.
Our use-case is - we are triggering a dbt Cloud job using
Lee-W commented on issue #34206:
URL: https://github.com/apache/airflow/issues/34206#issuecomment-2342600710
how about using it with dynamic task mapping?
```python
from __future__ import annotations
from pendulum import datetime
from airflow.datasets import Dataset, D
jaklan commented on issue #34206:
URL: https://github.com/apache/airflow/issues/34206#issuecomment-2342435749
@Lee-W it doesn't seem to work - only the second Dataset was updated:
https://github.com/user-attachments/assets/891c560f-7913-4de5-a43b-699f2f5ae9e8";>
for such DAG:
```
jaklan commented on issue #34206:
URL: https://github.com/apache/airflow/issues/34206#issuecomment-2342574661
@Lee-W but the point is - one dbt Cloud job can be responsible for building
e.g. 30 tables (they can be e.g. all tables with a specific tag: `--select
tag:foobar`). Each table shoul
uranusjr commented on issue #34206:
URL: https://github.com/apache/airflow/issues/34206#issuecomment-2344108361
> ```python
> @task(outlets=[DatasetAlias("example-alias")])
> def produce_dataset_events(*, outlet_events):
> outlet_events["example-alias"].add(Dataset("dbt-cloud-tab
jaklan commented on issue #34206:
URL: https://github.com/apache/airflow/issues/34206#issuecomment-2344722532
@Lee-W thanks for the suggestion. I will test it later, but it looks like an
interesting workaround.
@uranusjr fully agree, that would simplify a lot and make alias much, much
Lee-W commented on issue #34206:
URL: https://github.com/apache/airflow/issues/34206#issuecomment-2345294073
PR created https://github.com/apache/airflow/pull/42189
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
U
