Re: kafka replication blog

2013-02-07 Thread Felix GV
Thanks Jun! I hadn't been following the discussions regarding 0.8 and replication for a little while and this was a great post to refresh my memory and get up to speed on the current replication architecture's design. -- Felix On Tue, Feb 5, 2013 at 2:21 PM, Jun Rao wrote: > I just posted the

Re: kafka replication blog

2013-02-07 Thread Vaibhav Puranik
Same here. Thanks a lot Jun. Regards, Vaibhav On Thu, Feb 7, 2013 at 10:38 AM, Felix GV wrote: > Thanks Jun! > > I hadn't been following the discussions regarding 0.8 and replication for a > little while and this was a great post to refresh my memory and get up to > speed on the current replica

Re: kafka replication blog

2013-02-07 Thread Michal Haris
Same here, summary was need as we have a fairly large ecosystem of multiple 0.7.2 clusters and I am planning to test upgrade to 0.8. However, one thing creeping at the back of my mind regarding 0.8 is something i have spotted in one thread few weeks ago namely that the rebalance behaviour of produ

when adding a new broker, does kafka automatically create partitions of existing topics on it? (0.7.1)

2013-02-07 Thread Xiaoyu Wang
Hi all, As I remembered 0.7.0 requires manually create partitions for existing topics when we add new brokers. Does 0.7.1 automatically create partitions on newly added brokers? It seems to be doing that, just want to confirm. Also, if that true, is there a way to prevent existing topic being add

Re: kafka replication blog

2013-02-07 Thread Jun Rao
We have fixed this issue in 0.8. Withreplication factor 1, if the producer doesn't care about partitioning by key, messages will be sent to partitions that are currently available. Thanks, Jun On Thu, Feb 7, 2013 at 3:11 PM, Michal Haris wrote: > Same here, summary was need as we have a fairly

Re: MirrorMaker tool hangs up when producer throw an Exception

2013-02-07 Thread Jun Rao
Currently, this is not possible. The only way to stop mirror making is to kill -15. Thanks Jun On Thu, Feb 7, 2013 at 12:01 AM, 王国栋 wrote: > Hi , > > When the producer in the MirrorMaker tool throws exceptions, I can find one > of the streaming threads stops, but the mirror make process hangs

Re: when adding a new broker, does kafka automatically create partitions of existing topics on it? (0.7.1)

2013-02-07 Thread Jun Rao
This is changed in 0.8. The # of partitions is defined at topic creation time and it doesn't change with more brokers. Thanks, Jun On Thu, Feb 7, 2013 at 1:26 PM, Xiaoyu Wang wrote: > Hi all, > > As I remembered 0.7.0 requires manually create partitions for existing > topics when we add new br