MartijnVisser commented on PR #313: URL: https://github.com/apache/flink-connector-kafka/pull/313#issuecomment-5623700914
This isn't fixing FLINK-40585. Runs 34473915461 and 30181683657 both show a precommitted id still ONGOING, so they're FLINK-40626 and #314 is what closes them. Run 31916828021 is the only one without that signature, and it's the v4.0 cell: v4.0 still has `ThrottleUntilFirstCheckpointStrategy`, which FLINK-38696's `a90e8f48` replaced on main with `perCheckpoint(10)`, so the third job writes the rest in one transaction there and loses 966 records where main would lose ten. I can't tell from the log whether that last transaction was cut off by a late marker or hadn't finished committing. The direction is right, the condition isn't. Wait on transaction state, not on the LSO catching up with the high watermark. I'd take #314 first and then move this to v4.0 under its own ticket, and I wouldn't carry it on main: the marker lands inside 2 ms and the drain spends over 3 ms building the consumer before it reads the offsets, so I couldn't hit the race here at all. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
