The GitHub Actions job "Backport Checks" on texera.git/feat/reproducible-builds has succeeded. Run started by GitHub user parshimers (triggered by parshimers).
Head commit for run: 889a6b047b2b46a1c8df6768fa8205e8fa747bc3 / Ian Maxon <[email protected]> feat: make release artifacts and images reproducible Building the same commit twice now gives byte-identical release artifacts: the source tarball, the Docker Compose bundle, the jars and dist zips, the frontend bundle, and the container images. Before, each of them picked up the build machine's clock, user, umask, or whatever the package archives held that day, so a voter or downstream packager could not check a release against its source. | Artifact | What leaked in | Now | | --- | --- | --- | | Source tarball, compose bundle | builder uid/username, umask, file mtimes (bundle changed every run) | `.github/scripts/create-release-tarballs.sh`: sorted, 0:0, commit-time mtimes, `gzip -n`; the RC workflow and voters run the tag's copy | | Jars, dist zips | native-packager stamps zip entries with the wall clock | `SOURCE_DATE_EPOCH` passed to every `sbt dist` | | Frontend bundle | build number from the wall clock; Sass `random(100)` | build number from `SOURCE_DATE_EPOCH` (UTC); fixed `top: 50px` | | Web image yarn | repo's `.yarnrc.yml` deleted, Yarn 4.5.1 swapped in, mutable install | the pinned Yarn 4.14.1 with `--immutable`, as in CI | | Base images | movable tags | pinned by digest; Renovate's `dockerfile` manager bumps them weekly | | apt / pip in images | live archives; unpinned transitive deps; dated logs and caches | `bin/dockerfiles/snapshot/install.sh` installs as of 00:00 UTC on the commit's day (snapshot.ubuntu.com / snapshot.debian.org, pip `--uploaded-prior-to`) | | `/etc/shadow`, layer mtimes | build day; build time | commit day; `rewrite-timestamp=true` | | `.git` in six runtime images | pack files, index, refs | removed | The day, not the commit time, keys the package snapshot so commits from the same day still share cached apt/pip layers. The PyTorch index publishes no upload times, so pip refuses a cutoff while it is in play; the helper installs the one pin that needs it (`torch==2.13.0+cpu`, a local version PyPI cannot host) exactly and without dependencies, then resolves everything else, torch's dependencies included, from PyPI under the cutoff. `.git` shipped only for `OPVersion`, which has looked up `amber/src/main/scala/...` since the module moved to `common/workflow-operator` in #3878. No commit ever touched that path, so every operator version has been `"N/A"` in every deployment; it stays `"N/A"` without `.git`. The worker image also dropped a fallback that, when the operator requirements failed to install, silently shipped `wordcloud==1.9.2` and nothing else; such a build now fails. Non-stable versions are replaced: play-json `3.1.0-M1` -> `3.0.6` (same runtime dependencies; neither service imports it), RustFS `1.0.0-rc.6` -> `1.0.0` (chart unchanged but for its version), and `alpine:latest`, `nginx:alpine`, `curlimages/curl:latest` and `oven/bun:1-alpine` -> exact versions. y-websocket-server gains a lockfile and `npm ci`. `betterproto==2.0.0b7` stays: betterproto 2 has no stable release. Closes #8809. Discussion: #8807. `bin/README.md` documents what is pinned and how to rebuild an image; the vote email gives voters the command that rebuilds both tarballs. Each artifact was built twice from clean and compared byte for byte. | Artifact | Result | | --- | --- | | Release tarballs | `bash .github/scripts/test_create_release_tarballs.sh` (14 checks): identical across umask, clock, TZ, cwd and tag kind; the old recipe differed run to run and leaked uid/username | | `amber` (212 MB) and `config-service` dists | identical after two `sbt clean ... dist` runs with `SOURCE_DATE_EPOCH` | | Frontend `dist/` (1,989 files) | identical across TZ, umask and locale at a fixed path, as in the image (`/frontend`) | | Frontend build number | `ng test --include src/build-version.spec.ts`: 24 pass, incl. UTC, day boundaries and rejected malformed epochs | | `snapshot/install.sh` | `bash bin/dockerfiles/snapshot/test_install.sh` (32 checks) | | jOOQ codegen | 223 files identical from two fresh databases | | play-json 3.0.6 | `check_binary_deps.py jar` (strict) and NOTICE-binary regeneration pass for both services | | Images, podman/buildah `--no-cache --timestamp` | identical: mounter, agent-service, jupyter, config, access-control, file, notebook-migration, workflow-compiling, computing-unit-managing services | | Images, BuildKit v0.33.1 `rewrite-timestamp=true` | identical: mounter, agent-service, jupyter | | Computing-unit Python layers | apt + pip run twice: all 102,313 files identical; torch resolves to `+cpu` | The original mounter Dockerfile, rebuilt eight seconds apart with the same flags, gave different images. Not verified end to end: the web-application, computing-unit-master and -worker images, which exceed the test host's memory and disk, and arm64. Their parts (bundle, dist, Python layers) are each covered above. Yes, Claude Opus 5.5 Report URL: https://github.com/apache/texera/actions/runs/36949382592 With regards, GitHub Actions via GitBox
