zozo123 commented on PR #74173: URL: https://github.com/apache/airflow/pull/74173#issuecomment-5992319807
@shahar1 — following up on your critical-path objection with a deeper timing analysis of the current `f53486a4` design. The initial measurement above is encouraging, but it does **not yet establish a pure wall-clock win**. Comparing [treatment attempt 1](https://github.com/apache/airflow/actions/runs/37236836509) with [one successful baseline attempt with the same active job names](https://github.com/apache/airflow/actions/runs/37267385329): | Question | Observed result | |---|---| | Are consumers held behind a separate snapshot producer? | The current builder includes snapshot creation/upload. Median consumer scheduling gap after it finishes is 3s in both runs. | | Does that mean consumer delay is eliminated? | No. Median consumer start from workflow creation is 652s versus 583s: **69s later**. Builder duration is 55s longer, and initial workflow dispatch is 19s longer. These are overlapping timeline observations, not additive causal estimates. | | When are consumers actually ready to test? | Pairing the same 79 jobs and comparing the end of the image-preparation step relative to workflow creation gives a median shift of **16s earlier**. 49 consumers become ready earlier and 30 later; individual shifts range from 175s earlier to 155s later. This is a more direct measure of the tradeoff than the post-builder scheduling gap. | | What is the snapshot's measured cost? | Creation 73s + upload 23s = **96s of builder execution**. All 79 consumers successfully unpacked the snapshot and skipped the legacy image-load action. | | What saving is local to consumer preparation? | Summing paired prep durations gives **110.6 fewer runner-minutes**. Subtracting the 1.6-minute snapshot creation/upload cost leaves **109.0 minutes** for this limited prep-plus-snapshot accounting. It excludes other builder and job changes. | | What happened across the whole workflow? | Total observed runner time fell **69.6 minutes (about 5.3%)**, while workflow wall time fell **87s (about 2.8%)**. The local prep accounting exceeds the total saving by about 39.4 minutes, showing that changes elsewhere offset some of it. | | What finished last? | The same provider compatibility test job finished last in both runs. Its completion determines the measured all-job workflow finish; faster prep across many parallel consumers does not automatically translate into the same wall-clock saving. | The important distinction is **consumer start → image-ready → workflow finish**. Moving creation into the builder removes the separate-producer structure, but retains a 96s snapshot cost on the builder path. In this comparison, faster preparation offsets the later start for the median consumer, with mixed outcomes across jobs. The earlier reported ~236s delay and the current observed 69s are from different comparisons; their difference should not be presented as a controlled measurement of the fix. Limits: this is **one treatment run versus one exact-matrix baseline**, not 79 independent experiments. Source changes, runner performance, cache state, dispatch and test variance can explain part of the differences. The 3s post-builder gap does not prove that every consumer's entire dependency chain is unchanged. Workflow finish covers all executed jobs in this AMD workflow, not verified branch-required checks or every workflow on the PR. Runner-minutes are execution time, not a billing estimate. The dependency preinstallation stage rebuilt in 28.7s; this run does not demonstrate warm-cache dependency-layer reuse. A stronger acceptance experiment would interleave baseline and treatment on the same source/test matrix, keep runner labels and cache conditions comparable, and collect at least 3–5 full attempts per variant. Report per-run paired image-ready shifts, all-job wall time, total runner time and snapshot restore outcomes; analyze warm-cache reruns separately. Test dependency reuse separately with a warm-cache source-only change followed by a lockfile change. The present evidence supports the preparation saving and successful restores; the net wall-clock/weekly ROI claim still needs replication. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
