I don’t think num.replica.fetchers will help in this case. Increasing number of fetcher threads will only help in cases where you have a large amount of data coming into a broker and more replica fetcher threads will help keep up. We usually only use 1-2 for each broker. But in your case, it looks that leader migration cause issue. Do you see anything else in the log? Like preferred leader election?
Jiangjie (Becket) Qin On 2/25/15, 5:02 PM, "Zakee" <kzak...@netzero.net> wrote: >Thanks, Jiangjie. > >Yes, I do see under partitions usually shooting every hour. Anythings that >I could try to reduce it? > >How does "num.replica.fetchers" affect the replica sync? Currently have >configured 7 each of 5 brokers. > >-Zakee > >On Wed, Feb 25, 2015 at 4:17 PM, Jiangjie Qin <j...@linkedin.com.invalid> >wrote: > >> These messages are usually caused by leader migration. I think as long >>as >> you don¹t see this lasting for ever and got a bunch of under replicated >> partitions, it should be fine. >> >> Jiangjie (Becket) Qin >> >> On 2/25/15, 4:07 PM, "Zakee" <kzak...@netzero.net> wrote: >> >> >Need to know if I should I be worried about this or ignore them. >> > >> >I see tons of these exceptions/warnings in the broker logs, not sure >>what >> >causes them and what could be done to fix them. >> > >> >ERROR [ReplicaFetcherThread-3-5], Error for partition [TestTopic] to >> >broker >> >5:class kafka.common.NotLeaderForPartitionException >> >(kafka.server.ReplicaFetcherThread) >> >[2015-02-25 11:01:41,785] ERROR [ReplicaFetcherThread-3-5], Error for >> >partition [TestTopic] to broker 5:class >> >kafka.common.NotLeaderForPartitionException >> >(kafka.server.ReplicaFetcherThread) >> >[2015-02-25 11:01:41,785] WARN [Replica Manager on Broker 2]: Fetch >> >request >> >with correlation id 950084 from client ReplicaFetcherThread-1-2 on >> >partition [TestTopic,2] failed due to Leader not local for partition >> >[TestTopic,2] on broker 2 (kafka.server.ReplicaManager) >> > >> > >> >Any ideas? >> > >> >-Zakee >> >____________________________________________________________ >> >Next Apple Sensation >> >1 little-known path to big profits >> >>>http://thirdpartyoffers.netzero.net/TGL3231/54ee63b9e704b63b94061st03vuc >> >> ____________________________________________________________ >> Extended Stay America >> Get Fantastic Amenities, low rates! Kitchen, Ample Workspace, Free WIFI >> http://thirdpartyoffers.netzero.net/TGL3255/54ee66f26da6f66f10ad4mp02duc >>
