[
https://issues.apache.org/jira/browse/SPARK-58538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58538:
-----------------------------------
Labels: pull-request-available (was: )
> Add branch-4.3 CI scheduler and release integration
> ---------------------------------------------------
>
> Key: SPARK-58538
> URL: https://issues.apache.org/jira/browse/SPARK-58538
> Project: Spark
> Issue Type: Improvement
> Components: Project Infra
> Affects Versions: 4.3.0
> Reporter: Jungtaek Lim
> Priority: Major
> Labels: pull-request-available
>
> {{branch-4.3}} was cut from {{branch-4.x}} for the 4.3.0 release, but the CI
> and release plumbing that {{master}} maintains for each active release branch
> has not been added for it yet. This mirrors SPARK-56710 (which set up
> {{branch-4.2}} and {{branch-4.x}}) and SPARK-54143 (which set up
> {{branch-4.1}}).
> h3. Background
> The per-branch build workflows live on the branch itself, and {{branch-4.3}}
> already inherited a full set of them from {{branch-4.x}} when it was cut
> ({{build_maven.yml}}, {{build_java17/21/25.yml}}, {{build_non_ansi.yml}},
> {{build_python_*.yml}}, etc.). What is missing is on {{master}}: the
> scheduler that dispatches those workflows against {{--ref branch-4.3}}, plus
> the places that enumerate active branches.
> h3. Work items
> On {{master}}:
> * Add {{.github/workflows/branch43_scheduler.yml}}, modelled on
> {{branch42_scheduler.yml}}: cron entries plus a {{workflow_dispatch}}
> {{target}} choice, dispatching each build workflow with {{gh workflow run
> <workflow>.yml --repo ${{ github.repository }} --ref branch-4.3}}. Cron times
> should be staggered against the existing schedulers to spread runner load.
> * Add {{branch-4.3}} to the branch list in
> {{.github/workflows/publish_snapshot.yml}} (both the {{workflow_dispatch}}
> default and the matrix default), which is currently {{'["master",
> "branch-4.x", "branch-4.2", "branch-4.1", "branch-4.0", "branch-3.5"]'}}.
> * Add a {{branch-4.3}} badge row to {{README.md}}, alongside the existing
> {{branch-4.x}} / {{branch-4.2}} / {{branch-4.1}} rows.
> * Check whether {{dev/create-release/release-util.sh}} needs any update; it
> currently excludes {{master}} and {{branch-4.x}} when enumerating release
> branches, and {{branch-4.3}} should be picked up automatically, but this is
> worth confirming.
> On {{branch-4.3}}:
> * The inherited workflows are still titled for {{branch-4.x}} — e.g.
> {{build_maven.yml}} reads {{name: "Build / Maven (branch-4.x, Scala 2.13,
> Hadoop 3, JDK 17)"}} and {{build_java17.yml}} reads {{name: "Build / Java17
> (branch-4.x, ...)"}}. These names appear in the Actions UI and check-run
> names, so they should be retitled to {{branch-4.3}} to avoid confusion with
> the {{branch-4.x}} runs.
> h3. Dependency
> The Maven workflows inherited by {{branch-4.3}} carry the bug tracked
> separately: they call {{maven_test.yml}} without passing {{branch:}}, and
> that workflow's default is {{master}}, so they build {{master}} instead of
> the branch. That should be fixed first (or at the same time), otherwise
> {{branch-4.3}}'s new scheduler will faithfully dispatch three Maven workflows
> that silently test the wrong branch.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
