This is an automated email from the ASF dual-hosted git repository.
cloud-fan pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 95f4940bd4a7 [SPARK-57930][BUILD] Respect MAVEN_MIRROR_URL for
Coursier-based Maven plugins in build/mvn
95f4940bd4a7 is described below
commit 95f4940bd4a7f288ee53961bc4ad77d81a6f27f6
Author: Wenchen Fan <[email protected]>
AuthorDate: Sat Jul 4 11:38:15 2026 +0800
[SPARK-57930][BUILD] Respect MAVEN_MIRROR_URL for Coursier-based Maven
plugins in build/mvn
### What changes were proposed in this pull request?
`build/mvn` now bridges the `MAVEN_MIRROR_URL` environment variable into
`COURSIER_REPOSITORIES` (as `ivy2Local|${MAVEN_MIRROR_URL}`) just before
invoking `mvn`, when the
caller has not already set `COURSIER_REPOSITORIES`.
### Why are the changes needed?
Some Maven plugins resolve their own dependencies through
[Coursier](https://get-coursier.io/)
rather than through Maven's normal repository resolution. The most visible
example is the
`mvn-scalafmt` plugin invoked by `dev/lint-scala`, which downloads the
scalafmt toolchain
(`scalafmt-core_2.13`, `scala-reflect`) via Coursier.
Coursier does not read `MAVEN_MIRROR_URL`, nor the repositories declared in
the POM (including the
`maven-mirror` profile). In an environment that can only reach an internal
Maven mirror (no direct
access to Maven Central / Sonatype), `dev/lint-scala` therefore fails while
trying to download the
scalafmt toolchain:
```
org.scalafmt.interfaces.ScalafmtException: [v3.8.6] failed to download
download error: Caught java.net.UnknownHostException: repo1.maven.org ...
```
SBT (via `build/sbt-launch-lib.bash`) and Maven (via the `maven-mirror` POM
profile) already honor
`MAVEN_MIRROR_URL`. This change makes Coursier-based plugins honor the same
variable, so mirror-only
environments work out of the box without each developer having to
hand-write a Coursier config
(`~/.config/coursier/mirror.properties`) or export `COURSIER_REPOSITORIES`
themselves. An existing
`COURSIER_REPOSITORIES` set by the caller is respected and not overridden.
### Does this PR introduce _any_ user-facing change?
No. This only affects the local/CI build tooling. When `MAVEN_MIRROR_URL`
is unset (the default),
behavior is unchanged.
### How was this patch tested?
Manually, in a mirror-only environment (Maven Central unreachable):
- Baseline (before the change), running scalafmt via the raw `mvn` binary
fails with
`UnknownHostException: repo1.maven.org` while downloading the scalafmt
toolchain.
- With the change, `./build/mvn scalafmt:format ...` and the full
`./dev/lint-scala` both succeed
("Scalastyle checks passed." / "Scalafmt checks passed."), resolving the
toolchain through the
mirror, with `COURSIER_REPOSITORIES` unset in the environment.
- Verified the guard branches: an existing `COURSIER_REPOSITORIES` is left
untouched, and when
`MAVEN_MIRROR_URL` is unset nothing is exported.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #56989 from cloud-fan/cloud-fan/coursier-respect-maven-mirror.
Authored-by: Wenchen Fan <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 3f2ac2cc4c2c1932ace3f286c04226bdb38100a6)
Signed-off-by: Wenchen Fan <[email protected]>
---
build/mvn | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/build/mvn b/build/mvn
index aa3ba96f1349..0afff30a7e93 100755
--- a/build/mvn
+++ b/build/mvn
@@ -159,6 +159,14 @@ cd "${_CALLING_DIR}"
# Set any `mvn` options if not already present
export MAVEN_OPTS=${MAVEN_OPTS:-"$_COMPILE_JVM_OPTS"}
+# Some Maven plugins (e.g. the mvn-scalafmt plugin used by `dev/lint-scala`)
resolve their own
+# dependencies through Coursier, which ignores MAVEN_MIRROR_URL and the POM
repositories. Bridge
+# MAVEN_MIRROR_URL into COURSIER_REPOSITORIES so mirror-only environments work
out of the box.
+# Respect an existing COURSIER_REPOSITORIES if the caller already set one.
+if [ -n "${MAVEN_MIRROR_URL}" ] && [ -z "${COURSIER_REPOSITORIES}" ]; then
+ export COURSIER_REPOSITORIES="ivy2Local|${MAVEN_MIRROR_URL}"
+fi
+
echo "Using \`mvn\` from path: $MVN_BIN" 1>&2
if [ ! -z "${SPARK_LOCAL_HOSTNAME}" ]; then
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
