[
https://issues.apache.org/jira/browse/KAFKA-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465896#comment-13465896
]
John Fung edited comment on KAFKA-502 at 10/2/12 5:05 AM:
----------------------------------------------------------
kafka-502-v1.patch
This patch contains changes for the following JIRA:
KAFKA-[441-448]
KAFKA-488
KAFKA-502
KAFKA-503
KAFKA-507
KAFKA-526
A. Adding ReplicaBasicTest test cases to cover KAFKA-[441~448]. There are 10
cases with failure and 10 cases without failure. These 10 cases are running in
the same ReplicaBasicTest class with varying parameters to get different
combinations of arguments such as compression, request-num-acks in Producer,
no. of partition when creating topic, broker log segment size, ... etc.
A.1. Add 10 test cases for ReplicaBasicTest without failure: testcase_0001 ~
testcase_0010
A.2. Add 10 test cases for ReplicaBasicTest with failure: testcase_0101 ~
testcases_0110
B. Adding features:
B.1. Support multi topics. Currently, ProducerPerformance doesn't support
multiple topics. This multi topics testing is achieved by creating one thread
of ProducerPerformance on each topic and let them run in the background.
B.2. Support cluster_config.json at the following levels:
B.2.1. global level : system_test/cluster_config.json
This global level of cluster_config.json should always exist by default and the
config will be saved in the list
“SystemTestEnv.clusterEntityConfigDictListInSystemTestLevel”
B.2.2. testsuite level : system_test/xxxx_testsuite/cluster_config.json
The following function:
system_test_utils.setup_remote_hosts_with_testsuite_level_cluster_config
will look up if there is a testsuite level cluster config file. If it exists,
the config will be saved in the list
“SystemTestEnv.clusterEntityConfigDictListLastFoundInTestSuite” and this config
will override the system test level.
B.2.3. testcase level :
system_test/xxxx_testsuite/testcase_xxxx/cluster_config.json
The following function:
system_test_utils.setup_remote_hosts_with_testcase_level_cluster_config
will look up if there is a testcase level cluster config file. If it exists,
the config will be saved in the list
“SystemTestEnv.clusterEntityConfigDictListLastFoundInTestCase”
B.2.4. Assuming there is a cluster_config.json in
replication_testsuite/testcase_0001, this config will be effective in
testcase_0001. If there is no cluster_config.json in testcase_0002, it is going
to restore the config found in replication_testsuite. If there is no cluster
config in the testsuite level, it is going to restore the config in the system
test level (global).
B.3. Validate checksum matching across all replicas – this is validated by
comparing the checksums of the log segment by a combination of
topic-partition-offset as a key such as:
{u'broker-1': {'test_1-0/00000000000000000000.kafka': '3284084293',
'test_1-0/00000000000000010271.kafka': '853280790',
'test_1-0/00000000000000020555.kafka': '3106018734',
. . .
u'broker-2': {'test_1-0/00000000000000000000.kafka': '3284084293',
'test_1-0/00000000000000010271.kafka': '853280790',
'test_1-0/00000000000000020555.kafka': '3106018734',
. . .
u'broker-3': {'test_1-0/00000000000000000000.kafka': '3284084293',
'test_1-0/00000000000000010271.kafka': '853280790',
'test_1-0/00000000000000020555.kafka': '3106018734',
. . .
B.4. User may specify test cases to run in system_test/testcase_to_run.json
(KAFKA-503)
{
"MirrorMakerTest" : [
"testcase_0001"
],
"ReplicaBasicTest" : [
"testcase_0001",
"testcase_0002",
]
}
B.5. User may specify test cases to skip in system_test/testcase_to_skip.json
(KAFKA-503)
Similar to B.4.
B.6. System Test has a command line argument '-p' to print test case
description only (don't run any test)
To display the testcase description, run: “python -B system_test_runner.py -p”
B.7. System Test has a command line argument '-n' for not cleaning remote hosts
kafka home directories to save time or preserve the previous version binaries
C. Refactoring:
C.1. Move these log messages to DEBUG level “Found the log line” & “Unix
timestamp”
C.2. Move logging config in system_test_runner.py to an external file:
system_file/logging.conf
C.3. At the end of each test case, stop ZK at last to avoid hanging brokers
(KAFKA-507)
C.4. In the beginning of each test case, the zookeeper and broker data
directories are removed to clean up any hidden files under them (KAFKA-526)
C.5. In the test suite class (eg. ReplicaBasicTest) move the logic of setup and
initialization to base classes (ReplicationUtils & SetupUtils) (KAFKA-502)
was (Author: jfung):
kafka-502-v1.patch
This patch contains changes for the following JIRA:
KAFKA-[441-448]
KAFKA-502
KAFKA-503
KAFKA-507
KAFKA-526
A. Adding ReplicaBasicTest test cases to cover KAFKA-[441~448]. There are 10
cases with failure and 10 cases without failure. These 10 cases are running in
the same ReplicaBasicTest class with varying parameters to get different
combinations of arguments such as compression, request-num-acks in Producer,
no. of partition when creating topic, broker log segment size, ... etc.
A.1. Add 10 test cases for ReplicaBasicTest without failure: testcase_0001 ~
testcase_0010
A.2. Add 10 test cases for ReplicaBasicTest with failure: testcase_0101 ~
testcases_0110
B. Adding features:
B.1. Support multi topics. Currently, ProducerPerformance doesn't support
multiple topics. This multi topics testing is achieved by creating one thread
of ProducerPerformance on each topic and let them run in the background.
B.2. Support cluster_config.json at the following levels:
B.2.1. global level : system_test/cluster_config.json
This global level of cluster_config.json should always exist by default and the
config will be saved in the list
“SystemTestEnv.clusterEntityConfigDictListInSystemTestLevel”
B.2.2. testsuite level : system_test/xxxx_testsuite/cluster_config.json
The following function:
system_test_utils.setup_remote_hosts_with_testsuite_level_cluster_config
will look up if there is a testsuite level cluster config file. If it exists,
the config will be saved in the list
“SystemTestEnv.clusterEntityConfigDictListLastFoundInTestSuite” and this config
will override the system test level.
B.2.3. testcase level :
system_test/xxxx_testsuite/testcase_xxxx/cluster_config.json
The following function:
system_test_utils.setup_remote_hosts_with_testcase_level_cluster_config
will look up if there is a testcase level cluster config file. If it exists,
the config will be saved in the list
“SystemTestEnv.clusterEntityConfigDictListLastFoundInTestCase”
B.2.4. Assuming there is a cluster_config.json in
replication_testsuite/testcase_0001, this config will be effective in
testcase_0001. If there is no cluster_config.json in testcase_0002, it is going
to restore the config found in replication_testsuite. If there is no cluster
config in the testsuite level, it is going to restore the config in the system
test level (global).
B.3. Validate checksum matching across all replicas – this is validated by
comparing the checksums of the log segment by a combination of
topic-partition-offset as a key such as:
{u'broker-1': {'test_1-0/00000000000000000000.kafka': '3284084293',
'test_1-0/00000000000000010271.kafka': '853280790',
'test_1-0/00000000000000020555.kafka': '3106018734',
. . .
u'broker-2': {'test_1-0/00000000000000000000.kafka': '3284084293',
'test_1-0/00000000000000010271.kafka': '853280790',
'test_1-0/00000000000000020555.kafka': '3106018734',
. . .
u'broker-3': {'test_1-0/00000000000000000000.kafka': '3284084293',
'test_1-0/00000000000000010271.kafka': '853280790',
'test_1-0/00000000000000020555.kafka': '3106018734',
. . .
B.4. User may specify test cases to run in system_test/testcase_to_run.json
(KAFKA-503)
{
"MirrorMakerTest" : [
"testcase_0001"
],
"ReplicaBasicTest" : [
"testcase_0001",
"testcase_0002",
]
}
B.5. User may specify test cases to skip in system_test/testcase_to_skip.json
(KAFKA-503)
Similar to B.4.
B.6. System Test has a command line argument '-p' to print test case
description only (don't run any test)
To display the testcase description, run: “python -B system_test_runner.py -p”
B.7. System Test has a command line argument '-n' for not cleaning remote hosts
kafka home directories to save time or preserve the previous version binaries
C. Refactoring:
C.1. Move these log messages to DEBUG level “Found the log line” & “Unix
timestamp”
C.2. Move logging config in system_test_runner.py to an external file:
system_file/logging.conf
C.3. At the end of each test case, stop ZK at last to avoid hanging brokers
(KAFKA-507)
C.4. In the beginning of each test case, the zookeeper and broker data
directories are removed to clean up any hidden files under them (KAFKA-526)
C.5. In the test suite class (eg. ReplicaBasicTest) move the logic of setup and
initialization to base classes (ReplicationUtils & SetupUtils) (KAFKA-502)
> Simplify setup / initialization in replication_basic_test.py
> ------------------------------------------------------------
>
> Key: KAFKA-502
> URL: https://issues.apache.org/jira/browse/KAFKA-502
> Project: Kafka
> Issue Type: Sub-task
> Affects Versions: 0.8
> Reporter: John Fung
> Assignee: John Fung
> Labels: replication-testing
> Fix For: 0.8
>
> Attachments: kafka-502-v1.patch, kafka-502-v2.patch
>
>
> The setup and initialization could be simplified by moving some common code
> to a base class or some helper functions
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
