Bruno Cadonna created KAFKA-10481:
-------------------------------------
Summary: Consider Improving the Success Criteria of Streams'
System Test StreamsBrokerBounceTest
Key: KAFKA-10481
URL: https://issues.apache.org/jira/browse/KAFKA-10481
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 2.6.0
Reporter: Bruno Cadonna
The test StreamsBrokerBounceTest.test_all_brokers_bounce() failed on 2.5
because in the last stage of the test there is only one broker left and the
offset commit could succeed because the min.insync.replicas of
__consumer_offsets is set to 2 and acks is set to all. This causes a time out
and extends the closing of the Kafka Streams client to beyond the duration
passed to the close method of the client.
This affects especially the 2.5 branch since there Kafka Streams commits
offsets for each task, i.e., close() needs to wait for the timeout for each
task. In 2.6 and trunk the offset commit is done per thread, so close() does
only need to wait for one time out per
stream thread.
We fixed this failure by setting min.insync.replicas of __consumer_offsets to
1. However, it would be better to specify a better success criteria. For
example, instead of declaring success if the Kafka Streams client closed
completely and if any amount of records were delivered, it would be more
precise, more realistic, and probably less flaky to verify whether all records
were delivered up to the last committed offset at least once (the test does
only consider at-least-once atm).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
