Thanks for the detailed document, Jiangjie. Super helpful. Few questions:
1. You mention that "A ConsumerRebalanceListener class is created and could be wired into ZookeeperConsumerConnector to avoid duplicate messages when consumer rebalance occurs in mirror maker." Is this something the user needs to do or configure? or is the wiring of rebalance listener into the zookeeper consumer will be part of the enhancement? In other words, will we need to do anything extra to avoid duplicates during rebalance in MirrorMaker? 2. "The only source of truth for offsets in consume-then-send pattern is end user." - I assume you don't mean an actual person, right? So what does "end user" refer to? Can you clarify when will the offset commit thread commit offsets? And which JIRA implements this? 3. Maintaining message order - In which JIRA do we implement this part? Again, thanks a lot for documenting this and even more for the implementation - it is super important for many use cases. Gwen Gwen On Tue, Jan 20, 2015 at 4:31 PM, Jiangjie Qin <j...@linkedin.com.invalid> wrote: > Hi Kafka Devs, > > We are working on Kafka Mirror Maker enhancement. A KIP is posted to document > and discuss on the followings: > 1. KAFKA-1650: No Data loss mirror maker change > 2. KAFKA-1839: To allow partition aware mirror. > 3. KAFKA-1840: To allow message filtering/format conversion > Feedbacks are welcome. Please let us know if you have any questions or > concerns. > > Thanks. > > Jiangjie (Becket) Qin
