Lee-W commented on issue #57695:
URL: https://github.com/apache/airflow/issues/57695#issuecomment-4676638249
dynamic segement based partition can be achieved by
```python
# Producer emits per-segment partitions (observability) + one completion
signal:
@asset(schedule=PartitionAtRuntime())
def regional_sales(self, outlet_events):
for region in active_regions(): # "what's active" = business
logic, on the worker
process(region)
outlet_events[self].add_partitions(region)
outlet_events[batch_done].add(...) # emit a completion event
when the loop ends
# Downstream triggers on the completion signal — plain asset scheduling:
with DAG("global_report", schedule=batch_done):
...
```
and since we don't want to allow user code in scheduler, we won't add a new
timetable or partition mapper for that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
