[ https://issues.apache.org/jira/browse/KAFKA-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Koshy updated KAFKA-249: ----------------------------- Attachment: KAFKA-249.v3-v4.incremental.patch Incremental patch v3 -> v4. > Separate out Kafka mirroring into a stand-alone app > --------------------------------------------------- > > Key: KAFKA-249 > URL: https://issues.apache.org/jira/browse/KAFKA-249 > Project: Kafka > Issue Type: Improvement > Components: core > Reporter: Joel Koshy > Assignee: Joel Koshy > Fix For: 0.7.1 > > Attachments: KAFKA-249.v1.patch, KAFKA-249.v2.patch, > KAFKA-249.v3-v4.incremental.patch, KAFKA-249.v3.patch, KAFKA-249.v4.patch > > > I would like to discuss on this jira, the feasibility/benefits of separating > out Kafka's mirroring feature from the broker into a stand-alone app, as it > currently has a couple of limitations and issues. > For example, we recently had to deal with Kafka mirrors that were in fact > idle due to the fact that mirror threads were not created at start-up due to > a rebalancing exception, but the Kafka broker itself did not shutdown. This > has since been fixed, but is indicative of (avoidable) problems in embedding > non-broker specific features in the broker. > Logically, it seems to make sense to separate it out to achieve better > division of labor. Furthermore, enhancements to mirroring may be less > clunky to implement and use with a stand-alone app. For example to support > custom partitioning on the target cluster, or to mirror from multiple > clusters we would probably need to be able to pass in multiple embedded > consumer/embedded producer configs, which would be less ugly if the > mirroring process were a stand-alone app. Also, if we break it out, it > would be convenient to use as a "consumption engine" for the console > consumer which will make it easier to add on features such as wildcards in > topic consumption, since it contains a ZooKeeper topic discovery component. > Any suggestions and/or objections to this? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira