kirktrue commented on PR #14670: URL: https://github.com/apache/kafka/pull/14670#issuecomment-1800305244
`KafkaConsumerTest` is a test suite that exercised the functionality of the "existing" `KafkaConsumer` with 100 tests. With this change, the suite was refactored to allow each of those tests to be parameterized, such that it now tests both `LegacyKafkaConsumer` and `AsyncKafkaConsumer`. Ideally, those 100 tests would pass for both implementations, and we would have 200 passing tests 🎉 However, though the `LegacyKafkaConsumer` still passes all 100 tests in `KafkaConsumerTest`, the `AsyncKafkaConsumer` fails on 56 tests. The failures can be classified into the following reasons: 1. The test is only applicable for the old group protocol 2. The rebalancing logic is not yet implemented 3. The topic metadata logic is not yet implemented 4. The low-level CTR logic is incorrect To the first point—some of the tests directly reference RPC calls from the old protocol and thus those tests won't pass without some effort in refactoring the tests. It may make sense to leave those alone versus bending things around to make them apply to the new protocol. As for the second and third points, at this point in development, we don't expect tests that exercise rebalance or topic metadata logic to work. For the fourth point, we are "gaining" new tests that stress the CTR layer in ways that we haven't tickled with our existing unit tests or integration tests. Given this situation, I see two options forward: 1. Fix failing tests as part of this PR 2. Note which tests are failing, file Jiras, and fix them in future PRs The first option allows us to merge in the underlying delegate change sooner, preventing it from blocking end-to-end testing. The second option is more holistic and doesn't attempt to shoehorn a half-baked test suite into `trunk`. I take the view that we gained 44 additional passing tests with this change and have a followup task of gaining upward of 56 more soon. @dajac @AndrewJSchofield @philipnee @lianetm—any opinions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
