[ 
https://issues.apache.org/jira/browse/KAFKA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480275#comment-13480275
 ] 

John Fung commented on KAFKA-571:
---------------------------------

Uploaded kafka-571-v1.patch with the followings:

1. Leader Hard Failure (kill -SIGKILL) : testcase_0151 ~ 0158

2. Controller Controlled Failure (kill -SIGTERM) : testcase_0201 ~ 0208

3. Follower Controlled Failure (kill -SIGTERM) : testcase_0251 ~ 0258

4. Leader GC Pause (kill -SIGSTOP / -SIGCONT) : testcase_0301 ~ 0308 

5. Minor fix for SimpleConsumerShell (KAFKA-576)

6. Using SimpleConsumerShell to validate data loss in each replica per topic. 
If the data count doesn't match across all replicas, the test case fails.

7. Checksum matching across all replicas are now validated by merging 
individual log segment files:

    i. Sort and merge *.log per topic-partition into 1 log segment file
        # 
.../system_test/mirror_maker_testsuite/testcase_5002/logs/broker-4/kafka_server_4_logs
        #   |- test_1-0
        #        |- 00000000000000000000.index
        #        |- 00000000000000000000.log
        #        |- 00000000000000000020.index
        #        |- 00000000000000000020.log
        #        |- . . .
        #   |- test_1-1
        #        |- 00000000000000000000.index
        #        |- 00000000000000000000.log
        #        |- 00000000000000000020.index
        #        |- 00000000000000000020.log
        #        |- . . .

    ii. Get checksum of all merged log segment to its corresponding 
broker-topic-partition key:
      # {
      #   'kafka_server_1_logs:tests_1-0': 'd41d8cd98f00b204e9800998ecf8427e',
      #   'kafka_server_1_logs:tests_1-1': 'd41d8cd98f00b204e9800998ecf8427e',
      #   'kafka_server_1_logs:tests_2-0': 'd41d8cd98f00b204e9800998ecf8427e',
      #   'kafka_server_1_logs:tests_2-1': 'd41d8cd98f00b204e9800998ecf8427e',
      #   'kafka_server_2_logs:tests_1-0': 'd41d8cd98f00b204e9800998ecf8427e',
      #   'kafka_server_2_logs:tests_1-1': 'd41d8cd98f00b204e9800998ecf8427e',
      #   'kafka_server_2_logs:tests_2-0': 'd41d8cd98f00b204e9800998ecf8427e',
      #   'kafka_server_2_logs:tests_2-1': 'd41d8cd98f00b204e9800998ecf8427e'
      # }

    iii. Organize the checksum according to their topic-partition and do 
matching comparison:
      # {
      #   'test_1-0' : 
['d41d8cd98f00b204e9800998ecf8427e','d41d8cd98f00b204e9800998ecf8427e'],
      #   'test_1-1' : 
['d41d8cd98f00b204e9800998ecf8427e','d41d8cd98f00b204e9800998ecf8427e'], 
      #   'test_2-0' : 
['d41d8cd98f00b204e9800998ecf8427e','d41d8cd98f00b204e9800998ecf8427e'],
      #   'test_2-1' : 
['d41d8cd98f00b204e9800998ecf8427e','d41d8cd98f00b204e9800998ecf8427e']
      # }

8. As minor data loss is expected in leader failure cases when Acks == 1 
(KAFKA-573), now a test case will pass if the data loss is under 1% in the Acks 
== 1 cases.

                
> Add more test cases to System Test
> ----------------------------------
>
>                 Key: KAFKA-571
>                 URL: https://issues.apache.org/jira/browse/KAFKA-571
>             Project: Kafka
>          Issue Type: Sub-task
>            Reporter: John Fung
>            Assignee: John Fung
>             Fix For: 0.8
>
>         Attachments: kafka-571-v1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Reply via email to