Abacn opened a new pull request, #29336: URL: https://github.com/apache/beam/pull/29336
It turns out that each Java PR runs 38 Dataflow jobs. testE2EWordCount is run 11 times --- with different configurations. This PR removes the intermediate Java version (11/17) from precommit, similar to what we already do for Python precommits, where dataflow is only run for lowest and highest supported py version. It will reduce the number of dataflow jobs by 16. The ordinary wordcount/windowedwordcount is still executed by https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_Examples_Dataflow_Java.yml (actually duplicate tests), though not for LegacyWorkerImpersonate, which should be fine as they are still exercised by Java8 and 21. Java8 PreCommit: ``` 14 tests executed in 8 test classes taking 1h 16m 39.951s :runners:google-cloud-dataflow-java:examplesJavaRunnerV2PreCommit PASSED 17m 49.265s 2.361s 22m 32.399s org.apache.beam.examples.WindowedWordCountIT PASSED 17m 46.889s 3.154s 17m 46.889s testWindowedWordCountInStreamingStaticSharding PASSED 7m 29.144s testWindowedWordCountInBatchDynamicSharding PASSED 5m 15.147s testWindowedWordCountInBatchStaticSharding PASSED 4m 59.444s org.apache.beam.examples.WordCountIT PASSED 4m 43.118s 3.052s 4m 43.118s testE2EWordCount PASSED 4m 40.066s :runners:google-cloud-dataflow-java:examples:preCommitLegacyWorker PASSED 16m 18.293s 1.857s 21m 31.052s org.apache.beam.examples.WindowedWordCountIT PASSED 16m 16.416s 3.763s 16m 16.416s testWindowedWordCountInStreamingStaticSharding PASSED 5m 55.697s testWindowedWordCountInBatchDynamicSharding PASSED 5m 21.312s testWindowedWordCountInBatchStaticSharding PASSED 4m 55.644s org.apache.beam.examples.WordCountIT PASSED 5m 12.743s 3.699s 5m 12.743s testE2EWordCount PASSED 5m 9.044s :runners:google-cloud-dataflow-java:examples:preCommitLegacyWorkerImpersonate PASSED 15m 43.016s 1.334s 20m 23.725s org.apache.beam.examples.WindowedWordCountIT PASSED 15m 41.672s 2.895s 15m 41.672s testWindowedWordCountInStreamingStaticSharding PASSED 6m 24.080s testWindowedWordCountInBatchStaticSharding PASSED 4m 39.904s testWindowedWordCountInBatchDynamicSharding PASSED 4m 34.793s org.apache.beam.examples.WordCountIT PASSED 4m 40.695s 2.786s 4m 40.695s testE2EWordCount PASSED 4m 37.909s :runners:google-cloud-dataflow-java:examples-streaming:windmillPreCommit PASSED 6m 46.570s 1.381s 6m 46.570s org.apache.beam.examples.WordCountIT PASSED 6m 45.180s 2.412s 6m 45.180s testE2EWordCount PASSED 6m 42.768s :runners:google-cloud-dataflow-java:examples-streaming:appliancePreCommit PASSED 6m 0.340s 1.611s 6m 0.340s org.apache.beam.examples.WordCountIT PASSED 5m 58.715s 3.716s 5m 58.715s testE2EWordCount ``` 8 tests executed in 4 test classes taking 40m 41.784s Java11/17/21 PreCommit ``` :runners:google-cloud-dataflow-java:examples:preCommitLegacyWorkerImpersonate PASSED 15m 28.398s 1.142s 20m 40.689s org.apache.beam.examples.WindowedWordCountIT PASSED 15m 27.232s 2.365s 15m 27.232s testWindowedWordCountInStreamingStaticSharding PASSED 5m 47.370s testWindowedWordCountInBatchDynamicSharding PASSED 4m 54.508s testWindowedWordCountInBatchStaticSharding PASSED 4m 42.989s org.apache.beam.examples.WordCountIT PASSED 5m 12.282s 2.189s 5m 12.282s testE2EWordCount PASSED 5m 10.093s :runners:google-cloud-dataflow-java:examples:preCommitLegacyWorker PASSED 15m 26.914s 1.674s 20m 14.880s org.apache.beam.examples.WindowedWordCountIT PASSED 15m 25.221s 3.098s 15m 25.221s testWindowedWordCountInStreamingStaticSharding PASSED 5m 46.552s testWindowedWordCountInBatchStaticSharding PASSED 4m 50.387s testWindowedWordCountInBatchDynamicSharding PASSED 4m 45.184s org.apache.beam.examples.WordCountIT PASSED 4m 47.932s 3.231s 4m 47.932s testE2EWordCount PASSED 4m 44.701s ``` **Please** add a meaningful description for your change here ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier). To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md) GitHub Actions Tests Status (on master branch) ------------------------------------------------------------------------------------------------ [](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule) [](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule) [](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule) [](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule) See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
