zozo123 opened a new pull request, #74176: URL: https://github.com/apache/airflow/pull/74176
## Summary This is a follow-up to #74173. It adds a small compatibility-manifest preflight before downloading the multi-gigabyte Docker data-directory snapshot. The producer uploads the existing snapshot metadata as a separate, tiny artifact. Consumers download and validate that metadata first (checkout revision, Docker daemon fingerprint, storage driver/root, and empty-daemon safety conditions). The large snapshot is downloaded only after the manifest matches. Any missing artifact, unknown metadata, mismatch, or validation failure follows the existing stash restore/load path. ## Why A snapshot that cannot be used should not incur the transfer cost of the full archive. This is an acceleration gate, not a correctness shortcut: the complete snapshot validation still runs after download, and the current image-stash path remains the fallback. ## Validation - `pytest -q scripts/tests/ci/test_docker_data_root_snapshot.py` — 25 passed - `bash -n scripts/ci/docker_data_root_snapshot.sh` - `git diff --check` ## Measurement after merge Please compare snapshot metadata-match rate, avoided large-artifact downloads, fallback rate, snapshot transfer/restore time, and end-to-end image-preparation time against #74173. No cache-hit or time-saving rate is assumed. ## Dependency This branch is based on #74173 and should be reviewed/merged after that implementation, or rebased when the snapshot producer/consumer changes land. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
