Yes, Jiangjie, I do see lots of these errors "Starting preferred replica leader election for partitions” in logs. I also see lot of Produce request failure warnings in with the NotLeader Exception.
I tried switching off the auto.leader.relabalance to false. I am still noticing the rebalance happening. My understanding was the rebalance will not happen when this is set to false. Thanks Zakee > On Feb 25, 2015, at 5:17 PM, Jiangjie Qin <j...@linkedin.com.INVALID> wrote: > > I don’t think num.replica.fetchers will help in this case. Increasing > number of fetcher threads will only help in cases where you have a large > amount of data coming into a broker and more replica fetcher threads will > help keep up. We usually only use 1-2 for each broker. But in your case, > it looks that leader migration cause issue. > Do you see anything else in the log? Like preferred leader election? > > Jiangjie (Becket) Qin > > On 2/25/15, 5:02 PM, "Zakee" <kzak...@netzero.net > <mailto:kzak...@netzero.net>> wrote: > >> Thanks, Jiangjie. >> >> Yes, I do see under partitions usually shooting every hour. Anythings that >> I could try to reduce it? >> >> How does "num.replica.fetchers" affect the replica sync? Currently have >> configured 7 each of 5 brokers. >> >> -Zakee >> >> On Wed, Feb 25, 2015 at 4:17 PM, Jiangjie Qin <j...@linkedin.com.invalid> >> wrote: >> >>> These messages are usually caused by leader migration. I think as long >>> as >>> you don¹t see this lasting for ever and got a bunch of under replicated >>> partitions, it should be fine. >>> >>> Jiangjie (Becket) Qin >>> >>> On 2/25/15, 4:07 PM, "Zakee" <kzak...@netzero.net> wrote: >>> >>>> Need to know if I should I be worried about this or ignore them. >>>> >>>> I see tons of these exceptions/warnings in the broker logs, not sure >>> what >>>> causes them and what could be done to fix them. >>>> >>>> ERROR [ReplicaFetcherThread-3-5], Error for partition [TestTopic] to >>>> broker >>>> 5:class kafka.common.NotLeaderForPartitionException >>>> (kafka.server.ReplicaFetcherThread) >>>> [2015-02-25 11:01:41,785] ERROR [ReplicaFetcherThread-3-5], Error for >>>> partition [TestTopic] to broker 5:class >>>> kafka.common.NotLeaderForPartitionException >>>> (kafka.server.ReplicaFetcherThread) >>>> [2015-02-25 11:01:41,785] WARN [Replica Manager on Broker 2]: Fetch >>>> request >>>> with correlation id 950084 from client ReplicaFetcherThread-1-2 on >>>> partition [TestTopic,2] failed due to Leader not local for partition >>>> [TestTopic,2] on broker 2 (kafka.server.ReplicaManager) >>>> >>>> >>>> Any ideas? >>>> >>>> -Zakee >>>> ____________________________________________________________ >>>> Next Apple Sensation >>>> 1 little-known path to big profits >>> >>>> http://thirdpartyoffers.netzero.net/TGL3231/54ee63b9e704b63b94061st03vuc >>> >>> ____________________________________________________________ >>> Extended Stay America >>> Get Fantastic Amenities, low rates! Kitchen, Ample Workspace, Free WIFI >>> http://thirdpartyoffers.netzero.net/TGL3255/54ee66f26da6f66f10ad4mp02duc >>> > > > ____________________________________________________________ > Extended Stay America > Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed. > http://thirdpartyoffers.netzero.net/TGL3255/54ee80744cfa7747461mp13duc > <http://thirdpartyoffers.netzero.net/TGL3255/54ee80744cfa7747461mp13duc>
