voonhous opened a new pull request, #20397: URL: https://github.com/apache/hudi/pull/20397
### Describe the issue this Pull Request addresses Closes #20384. Azure CI has not run since 2026-10-06, when hudi-bot stopped mirroring PRs. We plan to retire it rather than revive the bot. Azure is the only CI that runs the Spark datasource suites on Spark 3.5 / Scala 2.12 / JDK 11; GitHub Actions runs them on Spark 4.2 only. Comparison: https://github.com/apache/hudi/issues/20384#issuecomment-6314584346 ### Summary and Changelog - Add `.github/workflows/spark35_ci.yml`: the five JDK 11 Spark datasource jobs trimmed from `bot.yml` in #19514, kept to the Spark 3.5 entry, with `hudi-spark3-common` added to part 1. It runs on every push to master, and on PRs that change the workflow file (this PR included). Ordinary PRs do not run it. - Remove Azure CI: `azure-pipelines-20230430.yml`, `azure_ci_check.yml`, `azure_ci.js`, and the Azure status step in `scheduled_workflow.yml` with its `statuses: write` permission. - `TestSparkSqlHudiPackageStructure` checked that every Scala SQL test class is run by some Azure step. It now checks the same against `spark35_ci.yml`. <details> <summary>Details</summary> **Why master pushes only.** Running Spark 3.5 on every PR roughly doubles the Spark datasource lanes, which works against the runner-minute work in #19910. Azure was never a required check, so a per-merge run keeps the protection it actually gave. **No Codecov upload.** Coverage uploaded only for master commits would make every PR's comparison against its master base show a false drop. **Azure job to new job.** | Azure job | Replaced by | |---|---| | UT_FT_2 (FTA hudi-spark) | `test-spark-java-tests-part2` | | UT_FT_3, UT_FT_4 (Java UT spark-datasource) | `test-spark-java-tests-part1`, `-part2` | | UT_FT_5 (Scala DML 1) | `test-spark-scala-dml-tests` | | UT_FT_6 (Scala DDL & Others) | `test-spark-scala-other-tests` | | UT_FT_8 (Scala feature, DML 2, FTC) | `test-spark-scala-other-tests`, `-scala-dml-tests`, `-java-tests-part3` | | UT_FT_9 (FTB hudi-spark) | `test-spark-java-tests-part3` | | UT_FT_1, UT_FT_7, UT_FT_10 | already in `bot.yml` with the same profile | **Left for after the #19910 stack**, which edits the same files: the Azure badge in `README.md`, and the Azure notes in `scripts/jacoco/README.md` and `packaging/bundle-validation/README.md`. The root `Dockerfile` keeps its `azure_ci_test_base_java11` base image, since `test-common-and-other-modules` builds from it. **Outside the repo:** disable the `apachehudi` Azure DevOps pipelines, stop the hudi-bot VM, and archive the `apachehudi-ci` repos. </details> ### Impact CI only. PRs no longer get an `Azure CI` status. A Spark 3.5 regression is caught on the master commit that introduces it rather than on the PR. ### Risk Level low. CI configuration only; this PR triggers the new workflow once. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
