junaiddshaukat opened a new issue, #39633: URL: https://github.com/apache/beam/issues/39633
`--maxBundleTimeMs` is accepted but has no effect. A bundle is closed when it reaches `--maxBundleSize` elements or when a watermark arrives, so on a sparse stream the elements already fed to a bundle are not emitted until the next watermark, and the watermark cannot advance past them. The natural implementation — closing the bundle from a wall-clock punctuator — produces duplicate output against a real broker. With it enabled, an integration test running two chained GroupByKeys across four partitions emits its single group six times, reproducibly, and the count keeps climbing after the input has stopped. What has been ruled out: - **Metrics folding.** Splitting the same input across many bundles does not change a user counter (`MetricsAcrossBundlesTest`). - **`ProcessorContext.commit()`.** The duplication happens with the commit request removed, and does not happen with the punctuator disabled but the commit still requested. So it appears to be closing a Fn-API bundle from a punctuator rather than from record processing. Note that Kafka Streams sets `commitOffsetNeeded` inside `StreamTask#process()`, so data produced from a punctuator sits outside that accounting; KAFKA-6906 was a bug of that shape, though it was fixed long before the version in use here. Part of #18479. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
