The GitHub Actions job "Tests (AMD)" on airflow.git/main has failed. Run started by GitHub user potiuk (triggered by potiuk).
Head commit for run: 365ea553add6b94cec1510d435f1bc0d5561911b / Wei Lee <[email protected]> AIP-76: Hold Dag run until all upstream partitions arrive (#64571) * AIP-76: Hold Dag run until all upstream partitions arrive - Introduces `RollupMapper(upstream_mapper=..., window=...)` so a partitioned Dag run waits until the full set of upstream partitions for one downstream period has arrived (e.g. 24 hourly events for a daily rollup). - Ships 6 temporal `Window` built-ins (Hour / Day / Week / Month / Quarter / Year); custom windows are rejected at serialization time. - Surfaces frozen / mapper-error / partial-rollup state on the next-run-assets UI; `pending_partition_count` and rollup-aware totals stay symmetric across list/detail routes. See `airflow-core/newsfragments/64571.significant.rst` for the full breakdown, including the documented DST edge case for `DayWindow` + local-tz mappers. * refactor(scheduler): drop partition audit-Log rows in favour of scheduler logs The misconfigured-mapper and stale-APDR-cleanup paths persisted audit Log rows on independent scoped=False sessions plus a per-process dedup set so the rows survived an outer rollback. For advisory records this is heavier than the problem warrants in v1 — the misconfig path already logs the exception every tick, and stale cleanup now emits a structured info log. Re-add the UI-visible audit rows if operators report needing them. * refactor(scheduler): narrow rollup mapper with TypeGuard instead of cast Add an ``is_rollup`` TypeGuard helper next to ``RollupMapper`` (mirroring ``is_mapped``) and use it in the partitioned-asset readiness check so the mapper narrows to ``RollupMapper`` without a ``cast``. * fix(scheduler): clear partition Dag runs only when the rollup definition changes Stale-cleanup previously dropped a pending AssetPartitionDagRun whenever the Dag's serialized version changed, so any unrelated structural edit discarded in-flight partition accumulation and could leave a rollup held forever. Stamp a rollup-definition fingerprint (the serialized partition mappers of the Dag's partitioned assets) on the APDR at creation and compare that against the latest definition instead of the Dag version, so only a genuine mapper/window change clears the run. Replaces the unreleased ``dag_version_id`` column on AssetPartitionDagRun with ``rollup_fingerprint`` (migration 0119 amended). * refactor(scheduler): make partition per-loop cap an internal constant The per-tick cap on pending AssetPartitionDagRun rows was exposed as the ``[scheduler] max_partition_dag_runs_to_create_per_loop`` setting, but it is a performance safety bound (keeping the per-tick transaction from starving executor heartbeats and regular scheduling), not a knob operators can meaningfully tune. Drop the unreleased setting and keep the bound as a module-level constant. The per-loop query LIMIT is retained. * docs: cite Starlette compile_path in output-format regex helper Point the ``{name}`` placeholder handling at the Starlette routing prior art it mirrors, so the parsing approach is discoverable. * docs: avoid coined "APDR" in rollup fingerprint docstring The docs spellchecker rejects the coined abbreviation; spell it out as a partition Dag run and drop the double spaces. * docs: explain rollup mapper pairing and link partition mapper classes Spell out why a temporal upstream mapper is needed (it normalizes each upstream key to the window's granularity), clarify that an identity mapper paired with a temporal window fails as a type mismatch at construction, and cross-reference RollupMapper / Window classes to the API docs. * test: use core partition-mapper types when computing expected fingerprint The fingerprint tests built a core PartitionedAssetTimetable from Task SDK Asset / RollupMapper objects, which tripped mypy arg-type checks. Construct it with serialized assets and the core mapper/window types (matching the existing test convention) imported at module top level; the serialized fingerprint is unchanged. Report URL: https://github.com/apache/airflow/actions/runs/26992732592 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
