Lee-W commented on PR #66004:
URL: https://github.com/apache/airflow/pull/66004#issuecomment-4351024118
> Do we want to make the old dag run disappear (basically what this does)
I feel this is what we want — like clearing task instances. But probably
worth waiting for @cmarteepants's take before we lock the semantics.
> or do we simply want to mark the partitions as unfulfilled? If the latter,
I wonder if it’d be better to have `airflow partitions clear` instead.
What should happen if we mark a partition as unfulfilled — I'm imagining
we'd null out partition_date and partition_key on the existing DagRun (its
logical_date is already None on partitioned runs, so the run becomes
effectively detached / orphaned from any partition while preserving its
terminal state), then let the scheduler emit a fresh DagRun for that partition
slot.
> Implementation-wise, I also wonder if it’d make sense to disassociate a
dag run with a partition (to “clear” the partition), instead of clearing the
run directly.
partition_date / partition_key are columns on DagRun, not a separate entity
— so "disassociate" means setting them NULL on the existing run. not sure
whether it's what we want
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
