[
https://issues.apache.org/jira/browse/KAFKA-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson updated KAFKA-1589:
-----------------------------------
Fix Version/s: (was: 0.10.1.0)
> Strengthen System Tests
> -----------------------
>
> Key: KAFKA-1589
> URL: https://issues.apache.org/jira/browse/KAFKA-1589
> Project: Kafka
> Issue Type: Bug
> Reporter: Guozhang Wang
> Assignee: Ewen Cheslack-Postava
> Labels: newbie
>
> Although the system test code is also part of the open source repository, not
> too much attention is paid to this module today. The incurred results is that
> we keep breaking the system tests with either changes on the admin tools, or
> library upgrades that change the APIs like Zookeeper. And when the system
> tests breaks / hangs / etc, it is also hard to debug the issue. We need to
> treat the system test suite just as part of the open source code.
> Based on my personal experience trouble shooting system tests, I would
> propose doing at least the follow enhancement around system tests.
> 1. Add unit tests for all system util test tools, for example:
> kafka_system_test_utils.get_controller_attributes
> kafka_system_test_utils.get_leader_for
> 2. Add exception handling logic in the python test framework to clean-up the
> testbed upon failures, so that the subsequent test cases will not be affected.
> 3. Remove timing based mechanism such as "sleep(5000) to wait for metadata to
> be propagated" as much as possible to avoid transient failures.
> After those enhancements, we should probably also pick a very small subset
> (say one from each suite) of the system test cases into the patch reviewing
> process along with the unit tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
