Hi all, I'm working on a Kafka Streams portable runner as a GSoC 2026 project ( https://github.com/apache/beam/issues/18479, mentored by Jan Lukavský). The runner can already run global-window pipelines end to end through the Fn API, and I'm now adding real windowing and triggering. Before I commit to an approach I'd like a sanity check from people who've built this before.
My question is general enough that it shouldn't need much Kafka Streams context: for a portable runner, what's the recommended way to execute a windowed GroupByKey today? What I'm planning is to run GroupAlsoByWindow on the runner side, the way the Flink portable runner does it -- feed the post-shuffle elements into GroupAlsoByWindowViaWindowSetNewDoFn + SystemReduceFn.buffering, backed by runner-core's ReduceFnRunner, and provide StateInternals and TimerInternals implementations over the Kafka Streams state store and timers (modelled on FlinkStateInternals). That gets windowing, the trigger state machine, panes, allowed lateness, timestamp combiners and window merging from one place. What I'd like to confirm: 1. Is runner-side ReduceFnRunner still the recommended path for a new portable runner, or is there a preferred/newer approach -- e.g. delegating GroupAlsoByWindow back to the SDK harness, or a shared portable utility I've missed? 2. Are there known pitfalls when backing StateInternals / TimerInternals with a new state+timer backend (state namespacing per window, watermark holds, merging-window state) that you'd steer a newcomer around? 3. Anything about event-time timer firing driven by an aggregated input watermark that tends to bite people the first time? Links for context: - Design doc (overall project): https://docs.google.com/document/d/1BBMURhSG4SxPcvvnKMTrmnKCr_jhXL6R4TBDBW7zsy8/edit - Windowing + triggering proposal (the specific approach above): https://gist.github.com/junaiddshaukat/100fab49cb1e51d67e7928123aa6a982 - Feature branch (code so far): https://github.com/apache/beam/tree/feat/18479-kafka-streams-runner-skeleton Thanks a lot for any pointers. Junaid
