junaiddshaukat opened a new pull request, #39762: URL: https://github.com/apache/beam/pull/39762
## Summary Part of #18479. Puts the Kafka Streams runner behind an opt-in build flag, so that it can live in `master` without being built for, or released to, anyone who has not asked for it. ``` ./gradlew -Pwith-kafka-streams-runner :runners:kafka-streams:build ``` Without the flag the runner's subprojects are not included in the build at all. ## Why The runner is worth having somewhere people can build it, use it, and work on it — the point being to give it somewhere to be developed and maintained rather than to ship it. It is not ready to be released, and it has known bugs rather than only missing features: bundles are not yet closed after a bounded time (#39633), and other cases are still being looked at. Merging that as part of the normal build would be hard to defend. An opt-in flag separates the two questions. Nothing reaches a user who did not ask for it, and no release contains it, so if the runner becomes stable enough the flag comes off and it is built like any other; and if it does not, it can be dropped again without affecting anyone. ## What changed `settings.gradle.kts` only includes `:runners:kafka-streams` and its subprojects when the property is set. Three places assumed they were always there: - `javaPreCommit` had a hard `dependsOn` on the runner's build. - The Python portable suite registered `kafkaStreamsValidatesRunner` unconditionally, and its job server is only a project when the flag is set. - Both Kafka Streams CI workflows now pass the flag, since otherwise there is nothing for them to build. Documentation says what the flag is and why it exists, and every command on the runner's page now carries it. The options table on that page had also fallen behind and is corrected here: `maxBundleSize` no longer doubles as the source's per-poll limit, which #39748 separated, and `readMaxElementsPerPoll`, `readMaxPollTimeMs` and `sessionTimeoutMs` were missing. ## Testing Checked in both directions, since the point of the change is what happens when the flag is absent: ``` ./gradlew projects # no Kafka Streams projects ./gradlew projects -Pwith-kafka-streams-runner # all of them ./gradlew javaPreCommit --dry-run # configures, no Kafka Streams tasks ./gradlew -Pwith-kafka-streams-runner :runners:kafka-streams:build ``` `kafkaStreamsValidatesRunner` resolves with the flag and does not exist without it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
