Jungtaek Lim created SPARK-58537:
------------------------------------
Summary: Scheduled Maven workflows on branch-4.x build master
instead of the branch
Key: SPARK-58537
URL: https://issues.apache.org/jira/browse/SPARK-58537
Project: Spark
Issue Type: Bug
Components: Project Infra
Affects Versions: 5.0.0
Reporter: Jungtaek Lim
The scheduled Maven workflows on {{branch-4.x}} are named for {{branch-4.x}}
but actually build {{master}}, because they invoke the reusable
{{maven_test.yml}} without passing the {{branch}} input, and that workflow's
default is {{master}}.
h3. The bug
{{.github/workflows/build_maven.yml}} on {{branch-4.x}} is titled:
{code}
name: "Build / Maven (branch-4.x, Scala 2.13, Hadoop 3, JDK 17)"
{code}
but its job body supplies no inputs:
{code}
jobs:
run-build:
permissions:
packages: write
name: Run
uses: ./.github/workflows/maven_test.yml
if: github.repository == 'apache/spark'
{code}
So {{maven_test.yml}} falls back to its declared default:
{code}
branch:
description: Branch to run the build against
required: false
type: string
default: master
{code}
and it then checks out {{ref: ${{ inputs.branch }}}} — i.e. {{master}}.
h3. Affected workflows
Only the three consumers of {{maven_test.yml}} on {{branch-4.x}}:
* {{build_maven.yml}}
* {{build_maven_java21.yml}}
* {{build_maven_java25.yml}}
The six SBT/Python workflows on the branch ({{build_java17/21/25}},
{{build_non_ansi}}, {{build_python_3.11/3.14}}) also omit {{branch:}}, but they
use {{build_and_test.yml}}, whose default on this branch is already
{{branch-4.x}}. They are correct and need no change.
The cleanest fix is therefore to change the {{branch}} default in
{{maven_test.yml}} on {{branch-4.x}} from {{master}} to {{branch-4.x}},
matching {{build_and_test.yml}} on the same branch. Alternatively, pass {{with:
branch: branch-4.x}} explicitly from each of the three callers.
h3. Evidence
The most recent green "Build / Maven (branch-4.x, ...)" run (30840288268,
2026-08-03) produced {{master}}'s artifacts. Its Precompile log contains:
{code}
spark-connect-client-jdbc_2.13-5.0.0-SNAPSHOT.jar, unused-1.0.0.jar define 1
overlapping resource
{code}
At the time, {{branch-4.x}} was at {{4.3.0-SNAPSHOT}} and {{master}} at
{{5.0.0-SNAPSHOT}}, so those coordinates show {{master}} had been checked out.
For contrast, the "Build (branch-4.x, ...)" JDK 17 run (30818354111), which
goes through {{build_and_test.yml}}, logs only {{4.3.0-SNAPSHOT}} — correctly
building the branch.
h3. Impact
{{branch-4.x}} has had no scheduled Maven coverage, and the green checks from
these three workflows are misleading.
This masked SPARK-57897, where {{branch-4.x}} could not load its Maven project
model at all — {{mvn}} failed with {{'dependencies.dependency.version' for
io.grpc:grpc-api:jar is missing}} before any test could run — while the
branch's Maven workflow kept reporting success. The breakage was found only by
building the branch locally.
Note the release branches are handled correctly elsewhere: the
{{build_branch42_*.yml}} / {{build_branch4x_*.yml}} workflows on {{master}} do
pass {{branch:}} explicitly. The bug is specific to the copies of these
workflows that live on {{branch-4.x}} itself, and would be inherited by any
newly cut branch that copies them (e.g. {{branch-4.3}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
