DanielLeens commented on PR #11835: URL: https://github.com/apache/seatunnel/pull/11835#issuecomment-5322680296
Thanks for the PR. Before suggesting a `dev` sync for this branch (it's currently behind), I checked whether the CI failure on the current head is actually unrelated to this diff — it isn't, so a sync + rerun would not help here. The failing job is `kafka-connector-it (8, ubuntu-latest)`, with 14 failures in `KafkaIT`, deterministic across all 7 `TestContainer` variants: `testSourceKafka[1-7]` and `testKafkaToKafkaExactlyOnceOnBatch[1-7]`. `testSourceKafka` fails at `testKafkaSpecificOffsetsToConsole` (`kafkasource_specific_offsets_to_console.conf`, which hardcodes `test_topic_source-0 = 50` and asserts `id` in `[50, 99]`). CI shows it reading `id=49` at offset 50 instead of `id=50`. That traces back to `warmUpKafkaTopics` in this PR: it writes one warm-up record to partition 0 of `test_topic_source` *before* `startUp()` writes the real 100 records (`generateTestData(..., 0, 100)`), then calls `deleteRecords(beforeOffset(1))`. `deleteRecords` only advances the log-start-offset for future reads — it does not renumber the remaining messages' offsets. So the warm-up record permanently consumes offset 0, and the real records land at offsets 1..100 instead of 0..99, breaking every test that assumes `offset == id` on `test_topic_source` (like the hardcoded `test_topic_source-0 = 50` specific-offsets config). `testKafkaToKafkaExactlyOnceOnBatch` is also failing deterministically across all 7 variants, though on a different topic pair (`kafka_topic_exactly_batch_once_1/2`) not touched by the warm-up — I haven't traced that one to a specific cause, so it may be worth checking separately. Given the first failure is self-inflicted by the warm-up write against `test_topic_source` rather than a pre-existing flake, the fix likely needs the warm-up to avoid writing to (or shifting offsets on) topics that other tests read by absolute offset — e.g. warm up on a dedicated scratch topic instead of the topics under test, or restrict the specific-offsets test's expectations to account for the warm-up record. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
