MartijnVisser opened a new pull request, #28357:
URL: https://github.com/apache/flink/pull/28357
## What is the purpose of the change
The Azure-provided CI machines have ~85GB total disk with only ~17GB free,
while Flink's end-to-end and nightly builds download and generate well over
17GB of data. The CI has been running dangerously close to the disk limit
(notably the release-1.20 nightly builds), causing unpredictable behaviour and
build failures.
`tools/azure-pipelines/free_disk_space.sh` already frees ~28GB, but several
large toolchains shipped with the Azure Ubuntu runner image are left untouched.
This PR removes the ones that are unused by a Flink Java/e2e build, reclaiming
an additional ~12-15GB and giving the compile/test/e2e jobs more headroom. The
script runs on `target: host` and is shared by all three job families, so no
pipeline YAML changes are needed.
## Brief change log
- Remove unused hosted tool caches: `CodeQL`, `go`, `node`, `Ruby`,
`PyPy`. The Python tool cache (`/opt/hostedtoolcache/Python`) is intentionally
**kept**, because the e2e job's `UsePythonVersion` task runs after this script
and relies on it.
- Remove other large language/SDK directories: `/usr/share/swift`,
`/usr/share/miniconda`, `/opt/microsoft` (MS Edge), `/opt/ghc`.
- Prune preloaded Docker base images via `docker image prune --all
--force` (guarded by a `command -v docker` check). Flink restores its cached
testcontainers images later in the pipeline, so nothing the build needs is
evicted.
- The JDKs under `/usr/lib/jvm` are left untouched, since the build reads
`JAVA_HOME_<jdk>_X64`.
## Verifying this change
This change is a CI tooling change without test coverage; it only runs on
Azure CI hosts. It can be verified from the CI logs by comparing the `df -h`
output at the start vs. end of the "Free up disk space" step (expect ~12-15GB
more freed than before), and by confirming the downstream `UsePythonVersion`
and `Set JDK` steps still succeed (proving the Python tool cache and JDKs were
preserved).
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (Claude Code (Opus 4.8))
Generated-by: Claude Code (Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
