This is an automated email from the ASF dual-hosted git repository.
bbejeck pushed a commit to branch 2.4
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/2.4 by this push:
new e60eb69 throttle consumer timeout increase (#8188)
e60eb69 is described below
commit e60eb69afa6ebc4ff1f9c51ab364f8774f2c4ca0
Author: Matthew Wong <[email protected]>
AuthorDate: Thu Feb 27 14:46:55 2020 -0800
throttle consumer timeout increase (#8188)
The test_throttled_reassignment test fails because the consumer that is
used to validate reassignment does not start on time to consume all messages.
This does not seem like an issue with the throttling of the reassignment, since
increasing the timeout allowed the test to pass multiple consecutive runs
locally.
This test seemed to rely on the default JmxTool for the console consumer
that was removed in this commit: 179d0d7
The console consumer would check to see if it had partitions assigned to it
before beginning to consume. Although the test occasionally failed with the
JmxTool, it began to fail much more after the removal.
Error messages of failures followed the below format with varying numbers
of missed messages. They are the first messages by the producer.
535 acked message did not make it to the Consumer. They are: 0, 1, 2, 3, 4,
5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19...plus 515 more. Total
Acked: 192792, Total Consumed: 192259. We validated that the first 535 of these
missing messages correctly made it into Kafka's data files. This suggests they
were lost on their way to the consumer.
In the scope of the test, this error suggests that the test is falling into
the race condition described in produce_consume_validate.py, which has the
timeout to prevent the consumer from missing initial messages.
This can serve as a temporary fix until the logic of consumer startup is
addressed further.
Reviewers: Jason Gustafson <[email protected]>, Bill Bejeck
<[email protected]>
---
tests/kafkatest/tests/core/throttling_test.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/kafkatest/tests/core/throttling_test.py
b/tests/kafkatest/tests/core/throttling_test.py
index 586bac9..4a8327e 100644
--- a/tests/kafkatest/tests/core/throttling_test.py
+++ b/tests/kafkatest/tests/core/throttling_test.py
@@ -53,7 +53,7 @@ class ThrottlingTest(ProduceConsumeValidateTest):
# ensure that the consumer is fully started before the producer starts
# so that we don't miss any messages. This timeout ensures the
sufficient
# condition.
- self.consumer_init_timeout_sec = 20
+ self.consumer_init_timeout_sec = 60
self.num_brokers = 6
self.num_partitions = 3
self.kafka = KafkaService(test_context,
