Lee-W commented on issue #44146:
URL: https://github.com/apache/airflow/issues/44146#issuecomment-4249087851
For indicating that a Dag run requires runtime partitioning, I think we
still need something like `PartitionAtRuntime` — but simpler than before. It
would just be `schedule=PartitionAtRuntime()` with no name or other arguments.
For the fan-out case (the AIP's motivating example of looping over a
discovered set and emitting one event per partition), I think the right shape
is:
```python
context["outlet_events"][my_asset].partition_keys = ["key1", "key2", ...]
```
(renaming the existing partition_key to partition_keys)
If we want to carry extra metadata, we could go one step further:
```
context["outlet_events"][my_asset].partition_keys = [
PartitionKey(key=..., extra=...),
...
]
```
This keeps extra attached to the asset event rather than `Asset.extra (which
is deprecated anyway), so we avoid that conflict entirely.
On add_partition vs. direct assignment — making the API feel as Pythonic as
possible seems preferable to me. My inclination is to support both:
`partition_keys = [PartitionKey(...)]` / `parition_keys = [...]` as the
idiomatic setter, with `add_partition` as the underlying implementation (or a
convenience alias).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
