Copilot commented on code in PR #13425: URL: https://github.com/apache/gluten/pull/13425#discussion_r4365414267
########## docs/backends/velox/developers/velox-backend-docker-images.md: ########## @@ -0,0 +1,152 @@ +--- +layout: page +title: Pre-built Docker Images +nav_order: 22 +parent: Developer Overview +--- +# Pre-built Docker Images + +Gluten publishes pre-built images to Docker Hub under +[`apache/gluten`](https://hub.docker.com/r/apache/gluten/tags). CI jobs use them as build and test +environments, and they also work as local development environments. Every image is built from a +Dockerfile under [`dev/docker/`](https://github.com/apache/gluten/tree/main/dev/docker) by the +[`docker_image.yml`](https://github.com/apache/gluten/blob/main/.github/workflows/docker_image.yml) +workflow. + +## Images + +Sizes are the compressed size of one architecture, as reported by Docker Hub for the 2026-10-18 +build. Pulled images take about twice that on disk. + +| Tag | Dockerfile | Arch | Size | Used by | +|---|---|---|---|---| +| [`vcpkg-almalinux-8-gcc13`](https://hub.docker.com/r/apache/gluten/tags?name=vcpkg-almalinux-8-gcc13) | [`Dockerfile.almalinux8-gcc13-static-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.almalinux8-gcc13-static-build) | amd64, arm64 | 1.7 GB | Static native builds in `velox_backend_x86`, `delta_spark_ut`, `velox_kafka`, `velox_native_build` (Iceberg), `velox_backend_ansi`, `velox_backend_enhanced`, `velox_nightly`, `velox_backend_cache`, `build_bundle_package` and `build_release`; [release builds](../../../developers/HowToRelease.md); [static build in Docker](velox-backend-build-in-docker.md) | +| [`vcpkg-centos-8-gcc13`](https://hub.docker.com/r/apache/gluten/tags?name=vcpkg-centos-8-gcc13) | [`Dockerfile.centos8-gcc13-static-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.centos8-gcc13-static-build) | amd64, arm64 | 2.7 GB | Not used by any workflow | +| [`vcpkg-centos-9`](https://hub.docker.com/r/apache/gluten/tags?name=vcpkg-centos-9) | [`Dockerfile.centos9-static-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.centos9-static-build) | amd64, arm64 | 2.1 GB | Static native build in `velox_backend_arm`; `velox_backend_cache`; the static [dev container](../../../developers/dev-container.md) | +| [`centos-8-jdk8`](https://hub.docker.com/r/apache/gluten/tags?name=centos-8-jdk8) | [`Dockerfile.centos8-dynamic-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.centos8-dynamic-build) | amd64, arm64 | 5.5 GB | Spark 3.4 / 3.5 unit tests and the Uniffle TPC job in `velox_backend_x86`; `velox_backend_enhanced`; `velox_nightly`; [dynamic build in Docker](velox-backend-build-in-docker.md) | +| [`centos-8-jdk11`](https://hub.docker.com/r/apache/gluten/tags?name=centos-8-jdk11) | same, `JAVA_VERSION=11` | amd64, arm64 | 5.5 GB | Not used by any workflow | +| [`centos-8-jdk17`](https://hub.docker.com/r/apache/gluten/tags?name=centos-8-jdk17) | same, `JAVA_VERSION=17` | amd64, arm64 | 5.5 GB | Spark 4.0 unit tests and build checks in `velox_backend_x86`; `velox_backend_enhanced`; `flink` | Review Comment: Non-blocking correction: the `centos-8-jdk17` reference in `velox_backend_enhanced.yml:253` belongs to a commented-out job, so that workflow does not currently use this tag. Please omit it from the active usage inventory. ########## docs/backends/velox/developers/velox-backend-docker-images.md: ########## @@ -0,0 +1,152 @@ +--- +layout: page +title: Pre-built Docker Images +nav_order: 22 +parent: Developer Overview +--- +# Pre-built Docker Images + +Gluten publishes pre-built images to Docker Hub under +[`apache/gluten`](https://hub.docker.com/r/apache/gluten/tags). CI jobs use them as build and test +environments, and they also work as local development environments. Every image is built from a +Dockerfile under [`dev/docker/`](https://github.com/apache/gluten/tree/main/dev/docker) by the +[`docker_image.yml`](https://github.com/apache/gluten/blob/main/.github/workflows/docker_image.yml) +workflow. + +## Images + +Sizes are the compressed size of one architecture, as reported by Docker Hub for the 2026-10-18 +build. Pulled images take about twice that on disk. + +| Tag | Dockerfile | Arch | Size | Used by | +|---|---|---|---|---| +| [`vcpkg-almalinux-8-gcc13`](https://hub.docker.com/r/apache/gluten/tags?name=vcpkg-almalinux-8-gcc13) | [`Dockerfile.almalinux8-gcc13-static-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.almalinux8-gcc13-static-build) | amd64, arm64 | 1.7 GB | Static native builds in `velox_backend_x86`, `delta_spark_ut`, `velox_kafka`, `velox_native_build` (Iceberg), `velox_backend_ansi`, `velox_backend_enhanced`, `velox_nightly`, `velox_backend_cache`, `build_bundle_package` and `build_release`; [release builds](../../../developers/HowToRelease.md); [static build in Docker](velox-backend-build-in-docker.md) | +| [`vcpkg-centos-8-gcc13`](https://hub.docker.com/r/apache/gluten/tags?name=vcpkg-centos-8-gcc13) | [`Dockerfile.centos8-gcc13-static-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.centos8-gcc13-static-build) | amd64, arm64 | 2.7 GB | Not used by any workflow | +| [`vcpkg-centos-9`](https://hub.docker.com/r/apache/gluten/tags?name=vcpkg-centos-9) | [`Dockerfile.centos9-static-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.centos9-static-build) | amd64, arm64 | 2.1 GB | Static native build in `velox_backend_arm`; `velox_backend_cache`; the static [dev container](../../../developers/dev-container.md) | +| [`centos-8-jdk8`](https://hub.docker.com/r/apache/gluten/tags?name=centos-8-jdk8) | [`Dockerfile.centos8-dynamic-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.centos8-dynamic-build) | amd64, arm64 | 5.5 GB | Spark 3.4 / 3.5 unit tests and the Uniffle TPC job in `velox_backend_x86`; `velox_backend_enhanced`; `velox_nightly`; [dynamic build in Docker](velox-backend-build-in-docker.md) | +| [`centos-8-jdk11`](https://hub.docker.com/r/apache/gluten/tags?name=centos-8-jdk11) | same, `JAVA_VERSION=11` | amd64, arm64 | 5.5 GB | Not used by any workflow | +| [`centos-8-jdk17`](https://hub.docker.com/r/apache/gluten/tags?name=centos-8-jdk17) | same, `JAVA_VERSION=17` | amd64, arm64 | 5.5 GB | Spark 4.0 unit tests and build checks in `velox_backend_x86`; `velox_backend_enhanced`; `flink` | +| [`centos-9-jdk8`](https://hub.docker.com/r/apache/gluten/tags?name=centos-9-jdk8) | [`Dockerfile.centos9-dynamic-build`](https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.centos9-dynamic-build) | amd64, arm64 | 5.1 GB | `cpp-test-udf-test` (shared build, C++ tests, clang-tidy) in `velox_backend_x86` and `velox_backend_arm`; `velox_backend_cache`; the default [dev container](../../../developers/dev-container.md) | +| [`centos-9-jdk11`](https://hub.docker.com/r/apache/gluten/tags?name=centos-9-jdk11) | same, `JAVA_VERSION=11` | amd64, arm64 | 5.1 GB | Not used by any workflow | +| [`centos-9-jdk17`](https://hub.docker.com/r/apache/gluten/tags?name=centos-9-jdk17) | same, `JAVA_VERSION=17` | amd64, arm64 | 5.1 GB | Spark 4.1 / 4.2 unit tests in `velox_backend_x86`; `delta_spark_ut`; `velox_backend_ansi`; the Iceberg test shards (`iceberg_upstream_revision`) | +| [`centos-9-jdk17-cuda12.9-cudf`](https://hub.docker.com/r/apache/gluten/tags?name=centos-9-jdk17-cuda12.9-cudf) | [`cudf/Dockerfile.centos-9-jdk17-cuda12.9-cudf`](https://github.com/apache/gluten/blob/main/dev/docker/cudf/Dockerfile.centos-9-jdk17-cuda12.9-cudf) | amd64 | 5.1 GB | Not used by any workflow | +| [`centos-9-jdk17-cuda13.1-cudf`](https://hub.docker.com/r/apache/gluten/tags?name=centos-9-jdk17-cuda13.1-cudf) | [`cudf/Dockerfile.centos-9-jdk17-cuda13.1-cudf`](https://github.com/apache/gluten/blob/main/dev/docker/cudf/Dockerfile.centos-9-jdk17-cuda13.1-cudf) | amd64 | 13.2 GB | `build-cudf-centos-9` in `velox_backend_x86`; `velox_backend_cache`; [GPU support](../get-started/VeloxGPU.md) | +| [`ubuntu22-m2-cache`](https://hub.docker.com/r/apache/gluten/tags?name=ubuntu22-m2-cache) | [`ubuntu/Dockerfile.ubuntu22-m2-cache`](https://github.com/apache/gluten/blob/main/dev/docker/ubuntu/Dockerfile.ubuntu22-m2-cache) | amd64 | 5.9 GB | Not used by any workflow | + +Pull an image with `docker pull apache/gluten:<tag>`, for example: + +```bash +docker pull apache/gluten:vcpkg-almalinux-8-gcc13 +docker run -it --rm -v "$PWD":/work -w /work apache/gluten:vcpkg-almalinux-8-gcc13 bash +``` + +Docker picks the matching architecture of a multi-arch tag automatically. + +## What the images contain + +Each Dockerfile clones Gluten to pre-populate dependencies, then deletes the clone and the ccache, +so the images contain no Gluten source or binaries. The exception is `ubuntu22-m2-cache`, which +keeps the clone in `/opt/gluten` without its build outputs. Review Comment: The CUDA 12.9 image retains Gluten artifacts in `/root/.m2`: its bundle build runs Maven `clean install`, but cleanup removes only `/opt/gluten` and ccache. Please document this exception to the “no Gluten binaries” claim so developers do not mistake its Maven repository for a dependency-only cache. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
