[
https://issues.apache.org/jira/browse/KAFKA-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475126#comment-13475126
]
Joel Koshy commented on KAFKA-567:
----------------------------------
+1 for kafka-567.patch
Also, right now mirror maker does not always shutdown cleanly. E.g., if you
want to use consumer.timeout.ms (and avoid having to do an external wait for
mirroring to finish). John, I'll attach a patch for that as well if it helps
the testing framework.
> Replication Data Loss in Mirror Maker Bouncing testcase
> -------------------------------------------------------
>
> Key: KAFKA-567
> URL: https://issues.apache.org/jira/browse/KAFKA-567
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: John Fung
> Assignee: Jun Rao
> Priority: Blocker
> Labels: bugs
> Attachments: kafka-567.patch,
> kafka-mirror-maker-bouncing-data-loss.patch, mirror_maker_12.log,
> system_test_1349971807_testcase_5002.tar
>
>
> * Test Description:
> 1. Start a 3-broker cluster as source
> 2. Start a 3-broker cluster as target
> 3. Start 1 instance of Mirror Maker to replicate data from source to target
> 4. While producer is sending data into source cluster, stop Mirror Maker with
> "kill -15". Start Mirror Maker again after 1 second.
> 5. Start a consumer to consume data from target cluster.
> 6. Compare the MessageID in the data between producer log and consumer log.
> * To reproduce this issue, please do the followings:
> 1. Download the latest 0.8 branch
> 2. Apply the patch attached to this JIRA
> 3. Build kafka by running "./sbt update package"
> 4. Execute the test in directory "system_test" : "python -B
> system_test_runner.py"
> * The test result may look like the following:
> _test_case_name : testcase_5002
> _test_class_name : MirrorMakerTest
> arg : bounce_leader : false
> arg : bounce_mirror_maker : true
> arg : message_producing_free_time_sec : 15
> arg : num_iteration : 1
> arg : num_messages_to_produce_per_producer_call : 50
> arg : num_partition : 1
> arg : replica_factor : 3
> arg : sleep_seconds_between_producer_calls : 1
> validation_status :
> Log segment checksum matching across all replicas : FAILED
> Unique messages from consumer on [test_1] : 355
> Unique messages from producer on [test_1] : 400
> Validate for data matched on topic [test_1] : FAILED
> * Attached a tar file for the system test output log, the brokers' log4j
> files and data log segment files.
> * There are no unusual Exception / Error found in the logs. However, there
> are consistently data loss in this Mirror Maker bouncing test case. Not sure
> if this is related to KAFKA-552.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
