GitHub user Leondon9 added a comment to the discussion: Support waiting for asset partition readiness from time-scheduled DAGs
Update — since I opened this, the building blocks landed, so I'll answer my own questions and link the in-flight work. **Q1 (covered by asset partitions in 3.2+?)** Partially. Asset partitioning — `partition_key` on asset events and `PartitionedAssetTimetable` — is there from 3.2. But *querying* asset events by `partition_key` from a task needed Execution API filters that landed later in #64610 / #64611 (3.4). So the pieces a task-level sensor needs only became available recently. **Q2 (does a task-level readiness sensor fit Airflow's direction?)** Looks like yes — it's being implemented as a standard-provider sensor rather than pushed into core scheduling. **Q3 (standard provider, core, or just a doc pattern?)** Standard provider. The sensor and its trigger live in `providers/standard`; the only core touch is wiring in the triggerer so the trigger can read asset events while deferred (a trigger has no DB/Execution-API access of its own). **Q4 (matching semantics?)** Event-existence, with an optional time bound. Waiting on "any event with this partition key" is enough when the key is unique per event (a timestamp), but for keys that repeat across runs (a region code) an unbounded lookup matches a stale event immediately — so an `after` bound scoped to the run's interval is needed. A more general form also filters on event `extra`, time range, and a required event count. Two implementations are in progress, both on top of #64610: - #67941 — `AssetPartitionSensor`, focused on the partition-readiness case above (plus the triggerer wiring for deferrable mode). - #70225 — `AssetEventSensor`, a more general asset-event sensor (`extra` / time-range filters, `expected_count`, `process_result`). They overlap heavily and will likely converge into one; tracking both here so anyone following this thread can find the actual work. --- Drafted-by: Claude Code (Opus 4.8); reviewed by @Leondon9 before posting GitHub link: https://github.com/apache/airflow/discussions/67375#discussioncomment-17869372 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
