[
https://issues.apache.org/jira/browse/KAFKA-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474428#comment-13474428
]
Jun Rao commented on KAFKA-567:
-------------------------------
Took a quick look.
Log in source broker:
system_test_1349971807_testcase_5002 jrao$ ls kafka_server_4_logs/test_1-0/*.log
kafka_server_4_logs/test_1-0/00000000000000000000.log
kafka_server_4_logs/test_1-0/00000000000000000140.log
kafka_server_4_logs/test_1-0/00000000000000000280.log
kafka_server_4_logs/test_1-0/00000000000000000020.log
kafka_server_4_logs/test_1-0/00000000000000000160.log
kafka_server_4_logs/test_1-0/00000000000000000300.log
kafka_server_4_logs/test_1-0/00000000000000000040.log
kafka_server_4_logs/test_1-0/00000000000000000180.log
kafka_server_4_logs/test_1-0/00000000000000000320.log
kafka_server_4_logs/test_1-0/00000000000000000060.log
kafka_server_4_logs/test_1-0/00000000000000000200.log
kafka_server_4_logs/test_1-0/00000000000000000340.log
kafka_server_4_logs/test_1-0/00000000000000000080.log
kafka_server_4_logs/test_1-0/00000000000000000220.log
kafka_server_4_logs/test_1-0/00000000000000000360.log
kafka_server_4_logs/test_1-0/00000000000000000100.log
kafka_server_4_logs/test_1-0/00000000000000000240.log
kafka_server_4_logs/test_1-0/00000000000000000380.log
kafka_server_4_logs/test_1-0/00000000000000000120.log
kafka_server_4_logs/test_1-0/00000000000000000260.log
Log in target broker
system_test_1349971807_testcase_5002 jrao$ ls kafka_server_7_logs/test_1-0/*.log
kafka_server_7_logs/test_1-0/00000000000000000000.log
kafka_server_7_logs/test_1-0/00000000000000000185.log
kafka_server_7_logs/test_1-0/00000000000000000312.log
kafka_server_7_logs/test_1-0/00000000000000000100.log
kafka_server_7_logs/test_1-0/00000000000000000235.log
kafka_server_7_logs/test_1-0/00000000000000000155.log
kafka_server_7_logs/test_1-0/00000000000000000262.log
It seems that the test didn't give mirror maker enough time to catch up data
after bouncing. Could you change the test to wait a bit longer after bounce?
> Replication Data Loss in Mirror Maker Bouncing testcase
> -------------------------------------------------------
>
> Key: KAFKA-567
> URL: https://issues.apache.org/jira/browse/KAFKA-567
> Project: Kafka
> Issue Type: Bug
> Reporter: John Fung
> Attachments: kafka-mirror-maker-bouncing-data-loss.patch,
> system_test_1349971807_testcase_5002.tar
>
>
> * Test Description:
> 1. Start a 3-broker cluster as source
> 2. Start a 3-broker cluster as target
> 3. Start 1 instance of Mirror Maker to replicate data from source to target
> 4. While producer is sending data into source cluster, stop Mirror Maker with
> "kill -15". Start Mirror Maker again after 1 second.
> 5. Start a consumer to consume data from target cluster.
> 6. Compare the MessageID in the data between producer log and consumer log.
> * To reproduce this issue, please do the followings:
> 1. Download the latest 0.8 branch
> 2. Apply the patch attached to this JIRA
> 3. Build kafka by running "./sbt update package"
> 4. Execute the test in directory "system_test" : "python -B
> system_test_runner.py"
> * The test result may look like the following:
> _test_case_name : testcase_5002
> _test_class_name : MirrorMakerTest
> arg : bounce_leader : false
> arg : bounce_mirror_maker : true
> arg : message_producing_free_time_sec : 15
> arg : num_iteration : 1
> arg : num_messages_to_produce_per_producer_call : 50
> arg : num_partition : 1
> arg : replica_factor : 3
> arg : sleep_seconds_between_producer_calls : 1
> validation_status :
> Log segment checksum matching across all replicas : FAILED
> Unique messages from consumer on [test_1] : 355
> Unique messages from producer on [test_1] : 400
> Validate for data matched on topic [test_1] : FAILED
> * Attached a tar file for the system test output log, the brokers' log4j
> files and data log segment files.
> * There are no unusual Exception / Error found in the logs. However, there
> are consistently data loss in this Mirror Maker bouncing test case. Not sure
> if this is related to KAFKA-552.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
