Thanks, Jun. Previously I created the topic with the specified number of partitions on the destination (target) cluster before I launched mirrormaker to do mirroring. But the mirrormaker only downloaded data for some of partitions, and in some cases, it stopped randomly. I will test this scenario again with beta1.
Regards, Libo -----Original Message----- From: Jun Rao [mailto:[email protected]] Sent: Wednesday, July 03, 2013 11:49 AM To: [email protected] Subject: Re: partition numbers and mirrormaker Yes, but you can change num.partitions in the target brokers. Thanks, Jun On Wed, Jul 3, 2013 at 6:21 AM, Yu, Libo <[email protected]> wrote: > For a topic with N partitions on the source cluster, if I let > mirrormaker to create topic automatically, the same topic on the > destination cluster will have only one partition. > Is that expected behavior? Could you just give a yes/no answer? Thanks. > > Regards, > > Libo > > > -----Original Message----- > From: Joel Koshy [mailto:[email protected]] > Sent: Tuesday, July 02, 2013 5:23 PM > To: [email protected] > Subject: Re: partition numbers and mirrormaker > > You can override the default number of partitions on the source and > target cluster to N. (num.partitions in the broker config). If you > need a higher (or lower) number of partitions for specific topics you > can override that on a per-topic basis (again, the config can be kept > identical between the source and target cluster). > > Thanks, > > Joel > > On Tue, Jul 2, 2013 at 1:26 PM, Yu, Libo <[email protected]> wrote: > > Hi > > > > I asked this question previously and didn't get an answer. > > If a topic has N partitions on source cluster, after being mirrored, > > on the destination cluster it has only one partition. > > > > To avoid this issue, before mirroring, I create the same topic with > > N partitions on the Destination cluster. But then the mirrormaker > > cannot fetch all partitions and stop randomly. > > > > So what should we expect from mirrormake and what is the right way > > to > use it? Thanks. > > > > > > Regards, > > > > Libo > > >
