Which version of Kafka are you using? Any error in the controller and
state-change log?
Thanks,
Jun
On Thu, Apr 24, 2014 at 7:37 PM, Yashika Gupta
<yashika.gu...@impetus.co.in>wrote:
> I am running a single broker and the leader column has 0 as the value.
>
> pushkar priyadarshi <priyadarshi.push...@gmail.com> wrote:
>
>
> you can use the kafka-list-topic.sh to find out if leader for particual
> topic is available.-1 in leader column might indicate trouble.
>
>
> On Fri, Apr 25, 2014 at 6:34 AM, Guozhang Wang <wangg...@gmail.com> wrote:
>
> > Could you double check if the topic LOGFILE04 is already created on the
> > servers?
> >
> > Guozhang
> >
> >
> > On Thu, Apr 24, 2014 at 10:46 AM, Yashika Gupta <
> > yashika.gu...@impetus.co.in
> > > wrote:
> >
> > > Jun,
> > >
> > > The detailed logs are as follows:
> > >
> > > 24.04.2014 13:37:31812 INFO main kafka.producer.SyncProducer -
> > > Disconnecting from localhost:9092
> > > 24.04.2014 13:37:38612 WARN main kafka.producer.BrokerPartitionInfo -
> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > > No partition metadata for topic LOGFILE04 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > > kafka.common.LeaderNotAvailableException
> > > 24.04.2014 13:37:40712 INFO main kafka.client.ClientUtils$ - Fetching
> > > metadata from broker id:0,host:localhost,port:9092 with correlation id
> 1
> > > for 1 topic(s) Set(LOGFILE04)
> > > 24.04.2014 13:37:41212 INFO main kafka.producer.SyncProducer -
> Connected
> > > to localhost:9092 for producing
> > > 24.04.2014 13:37:48812 INFO main kafka.producer.SyncProducer -
> > > Disconnecting from localhost:9092
> > > 24.04.2014 13:37:48912 WARN main kafka.producer.BrokerPartitionInfo -
> > > Error while fetching metadata [{TopicMetadata for topic LOGFILE04 ->
> > > No partition metadata for topic LOGFILE04 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [LOGFILE04]: class
> > > kafka.common.LeaderNotAvailableException
> > > 24.04.2014 13:37:49012 ERROR main
> > kafka.producer.async.DefaultEventHandler
> > > - Failed to collate messages by topic, partition due to: Failed to
> fetch
> > > topic metadata for topic: LOGFILE04
> > >
> > >
> > > 24.04.2014 13:39:96513 WARN
> > >
> >
> ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0
> > > kafka.consumer.ConsumerFetcherThread -
> > >
> >
> [ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0],
> > > Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 4;
> > ClientId:
> > >
> >
> produceLogLine2-ConsumerFetcherThread-produceLogLine2_vcmd-devanshu-1398361030812-8a0c706e-0-0;
> > > ReplicaId: -1; MaxWait: 60000 ms; MinBytes: 1 bytes; RequestInfo:
> > > [LOGFILE04,0] -> PartitionFetchInfo(2,1048576)
> > > java.net.SocketTimeoutException
> > > at
> > > sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
> > > at
> > sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> > > at
> > >
> >
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> > > at kafka.utils.Utils$.read(Unknown Source)
> > > at kafka.network.BoundedByteBufferReceive.readFrom(Unknown
> > Source)
> > > at kafka.network.Receive$class.readCompletely(Unknown Source)
> > > at
> kafka.network.BoundedByteBufferReceive.readCompletely(Unknown
> > > Source)
> > > at kafka.network.BlockingChannel.receive(Unknown Source)
> > > at kafka.consumer.SimpleConsumer.liftedTree1$1(Unknown Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > > Source)
> > > at
> > >
> >
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(Unknown
> > > Source)
> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > > at
> > > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(Unknown
> > Source)
> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > > Source)
> > > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(Unknown
> > > Source)
> > > at kafka.metrics.KafkaTimer.time(Unknown Source)
> > > at kafka.consumer.SimpleConsumer.fetch(Unknown Source)
> > > at
> kafka.server.AbstractFetcherThread.processFetchRequest(Unknown
> > > Source)
> > > at kafka.server.AbstractFetcherThread.doWork(Unknown Source)
> > > at kafka.utils.ShutdownableThread.run(Unknown Source)
> > >
> > >
> > > Regards,
> > > Yashika
> > > ________________________________________
> > > From: Jun Rao <jun...@gmail.com>
> > > Sent: Thursday, April 24, 2014 10:49 PM
> > > To: users@kafka.apache.org
> > > Subject: Re: Kafka Performance Tuning
> > >
> > > Before that error messge, the log should tell you the cause of the
> error.
> > > Could you dig that out?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Thu, Apr 24, 2014 at 10:12 AM, Yashika Gupta <
> > > yashika.gu...@impetus.co.in
> > > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I am working on a POC where I have 1 Zookeeper and 2 Kafka Brokers on
> > my
> > > > local machine. I am running 8 sets of Kafka consumers and producers
> > > running
> > > > in parallel.
> > > >
> > > > Below are my configurations:
> > > > Consumer Configs:
> > > > zookeeper.session.timeout.ms=120000
> > > > zookeeper.sync.time.ms=2000
> > > > zookeeper.connection.timeout.ms=120000
> > > > auto.commit.interval.ms=60000
> > > > rebalance.backoff.ms=2000
> > > > fetch.wait.max.ms=60000
> > > > auto.offset.reset=smallest
> > > >
> > > > Producer configs:
> > > > key.serializer.class=kafka.serializer.StringEncoder
> > > > request.required.acks=-1
> > > > message.send.max.retries=3
> > > > request.timeout.ms=60000
> > > >
> > > > I have tried various more configuration changes but I am running into
> > the
> > > > same exception again and again.
> > > >
> > > > 17.04.2014 06:31:83216 ERROR pool-5-thread-1
> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> > by
> > > > topic, partition due to: Failed to fetch topic metadata for topic:
> > > TOPIC99
> > > > 17.04.2014 06:31:85616 ERROR pool-4-thread-1
> > > > kafka.producer.async.DefaultEventHandler - Failed to collate messages
> > by
> > > > topic, partition due to: Failed to fetch topic metadata for topic:
> > > TOPIC99
> > > >
> > > > I am not able to get to the root cause of the issue.
> > > > Appreciate your help.
> > > >
> > > > Regards,
> > > > Yashika
> > > >
> > > > ________________________________
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > NOTE: This message may contain information that is confidential,
> > > > proprietary, privileged or otherwise protected by law. The message is
> > > > intended solely for the named addressee. If received in error, please
> > > > destroy and notify the sender. Any use of this email is prohibited
> when
> > > > received in error. Impetus does not represent, warrant and/or
> > guarantee,
> > > > that the integrity of this communication has been maintained nor that
> > the
> > > > communication is free of errors, virus, interception or interference.
> > > >
> > >
> > > ________________________________
> > >
> > >
> > >
> > >
> > >
> > >
> > > NOTE: This message may contain information that is confidential,
> > > proprietary, privileged or otherwise protected by law. The message is
> > > intended solely for the named addressee. If received in error, please
> > > destroy and notify the sender. Any use of this email is prohibited when
> > > received in error. Impetus does not represent, warrant and/or
> guarantee,
> > > that the integrity of this communication has been maintained nor that
> the
> > > communication is free of errors, virus, interception or interference.
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
