Re: Mirrormaker issue with Kafka 0.9 (confluent platform 2.0)

2015-12-11 Thread Ismael Juma
On Fri, Dec 11, 2015 at 3:31 AM, Guozhang Wang wrote: > As Ewen said in KAFKA-1650 we removed the support of multiple consumer > configs in MM, so that you cannot get one MM instance to pull from multiple > origin clusters, but instead you need to use one MM instance for each > origin. > Seems l

Re: Mirrormaker issue with Kafka 0.9 (confluent platform 2.0)

2015-12-11 Thread Meghana Narasimhan
Thanks Ewen and Guozhang for the details ! On Thu, Dec 10, 2015 at 10:31 PM, Guozhang Wang wrote: > Meghana, > > As Ewen said in KAFKA-1650 we removed the support of multiple consumer > configs in MM, so that you cannot get one MM instance to pull from multiple > origin clusters, but instead you

Re: Mirrormaker issue with Kafka 0.9 (confluent platform 2.0)

2015-12-10 Thread Guozhang Wang
Meghana, As Ewen said in KAFKA-1650 we removed the support of multiple consumer configs in MM, so that you cannot get one MM instance to pull from multiple origin clusters, but instead you need to use one MM instance for each origin. The motivation for dropping this support is mainly operational:

Re: Mirrormaker issue with Kafka 0.9 (confluent platform 2.0)

2015-12-10 Thread Ewen Cheslack-Postava
Meghana, It looks like this functionality was removed in https://issues.apache.org/jira/browse/KAFKA-1650, although I don't see explicit discussion of the issue in the JIRA so I'm not sure of the exact motivation. Maybe Becket or Guozhang can offer some insight about if it is necessary (that JIRA

Mirrormaker issue with Kafka 0.9 (confluent platform 2.0)

2015-12-10 Thread Meghana Narasimhan
Hi, We upgraded to the Confluent 2.0 platform today (we were earlier using the Kafka 0.8 without the confluent platform). With the latest Kafka 0.9 that is packaged with the platform we are facing issues starting the Mirror maker with multiple consumer config files. The original command that we w