voonhous opened a new pull request, #19909: URL: https://github.com/apache/hudi/pull/19909
### Describe the issue this Pull Request addresses Part of the CI improvement epic #19524 (#19528). Stacked on #19892; the commits before "ci: stop flink-2 from running hudi-flink's unit tests twice" are that PR's and drop out on rebase. Three jobs spend time on work whose result nothing in the run consumes: flink-2 runs hudi-flink's unit tests a second time by accident, the docker job shades fifteen bundles it never tests, and eight Spark jobs build hudi-examples-spark for a quickstart test that only a ninth job runs. ### Summary and Changelog One commit each, workflow files only, no test or build change: 1. **flink-2 filter.** `-Dit.test=!ITTestHoodieDataSource` is an exclusion with no include, which replaces failsafe's `IT*` pattern with every test class; the job ran 170 unit-test classes (15.5 of its 35 minutes) that test-common-and-other-modules already runs. Now `ITTest*,!ITTestHoodieDataSource`, verified against surefire's own TestListResolver; all 19 IT classes in hudi-flink are named that way. 2. **Docker job build.** Its `mvn install` covers the whole reactor, 7.8 minutes, of which the fifteen bundles are 4 and the examples 2. The bundles have no tests and validate-bundles builds and exercises every one; the examples are on the job's own exclusion list. The build now excludes those twenty modules, each checked to exist in the reactor under the job's profiles. 3. **hudi-examples-spark.** Nine JDK 17 jobs built it, pulling hudi-utilities, hudi-aws and hudi-gcp along, about 90 s each. Only ut-other runs the quickstart; the other eight drop it. The module and its dependencies still get one full build per lane: ut-other on Scala 2.13, and the utilities, spark-client and docker jobs on Scala 2.12. ### Impact Expected against #19892's run (577 runner-minutes over 19 jobs, longest job flink-2 at 39): | lever | runner-minutes per run | wall clock | |---|---|---| | flink-2 filter | about -16 | flink-2 39 to about 23; longest job becomes ut-fat-and-ftc at 37 | | docker build | about -6 | that job 36 to about 30 | | examples-spark | about -12 | every JDK 17 job about 1.5 min shorter, so about 35 | | together | about -34 (6%) | 39 to about 35 | Measured numbers go here once this PR has run. Coverage is unchanged: every module still goes through its full build, checkstyle, scalastyle and RAT included, at least once per Scala lane in the same run; every unit test still runs once. What goes away is the second, third and ninth copy of the same result. ### Risk Level low Workflow files only. If the flink-2 filter matched too little, the job would report fewer integration tests than before; the count to compare is 18 IT classes. A wrong module path in the docker exclusion list fails that build immediately, which the first run shows. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
