Ryan,
Apache mailing list does not allow attachments exceeding a certain size
limit, so the server logs is blocked.
>From the controller log it seems this only broker has failed and hence no
partitions will be available. This could be a soft failure (e.g. long GC),
or the ZK server side issues. You may want to take a look at your
controller log to see if there is any entries like "broker failure" before
the offline leader selection process.
Guozhang
On Mon, Aug 11, 2014 at 5:08 PM, Ryan Williams <rwilli...@gmail.com> wrote:
> The broker appears to be running
>
> $ telnet kafka-server 9092
> Trying...
> Connected to kafka-server
> Escape character is '^]'.
>
> I've attached today's server.log. There was a manual restart of kafka,
> which you'll notice, but that didn't fix the issue.
>
> Thanks for looking!
>
>
>
>
>
>
>
>
> On Mon, Aug 11, 2014 at 4:30 PM, Guozhang Wang <wangg...@gmail.com> wrote:
>
>> Hi Ryan,
>>
>> Could you check if all of your brokers are still live and running? Also
>> could you check the server log in addition to the producer / state-change
>> /
>> controller logs?
>>
>> Guozhang
>>
>>
>> On Mon, Aug 11, 2014 at 12:45 PM, Ryan Williams <rwilli...@gmail.com>
>> wrote:
>>
>> > I have a single broker test Kafka instance that was running fine on
>> Friday
>> > (basically out of the box configuration with 2 partitions), now I come
>> back
>> > on Monday and producers are unable to send messages.
>> >
>> > What else can i look at to debug, and prevent?
>> >
>> > I know how to recover by removing data directories for kafka and
>> zookeeper
>> > to start fresh. But, this isn't the first time this has happened, so I
>> > would like to understand it better to feel more comfortable with kafka.
>> >
>> >
>> > ===================
>> > Producer error (from console produce)
>> > ===================
>> > [2014-08-11 19:32:49,781] WARN Error while fetching metadata
>> > [{TopicMetadata for topic mytopic ->
>> > No partition metadata for topic mytopic due to
>> > kafka.common.LeaderNotAvailableException}] for topic [mytopic]: class
>> > kafka.common.LeaderNotAvailableException
>> > (kafka.producer.BrokerPartitionInfo)
>> > [2014-08-11 19:32:49,782] ERROR Failed to collate messages by topic,
>> > partition due to: Failed to fetch topic metadata for topic: mytopic
>> > (kafka.producer.async.DefaultEventHandler)
>> >
>> > ===============
>> > state-change.log
>> > ===============
>> > [2014-08-11 19:12:45,312] TRACE Controller 0 epoch 3 started leader
>> > election for partition [mytopic,0] (state.change.logger)
>> > [2014-08-11 19:12:45,321] ERROR Controller 0 epoch 3 initiated state
>> change
>> > for partition [mytopic,0] from OfflinePartition to OnlinePartition
>> failed
>> > (state.change.logger)
>> > kafka.common.NoReplicaOnlineException: No replica for partition
>> [mytopic,0]
>> > is alive. Live brokers are: [Set()], Assigned replicas are: [List(0)]
>> > at
>> >
>> >
>> kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:61)
>> > [2014-08-11 19:12:45,312] TRACE Controller 0 epoch 3 started leader
>> > election for partition [mytopic,1] (state.change.logger)
>> > [2014-08-11 19:12:45,321] ERROR Controller 0 epoch 3 initiated state
>> change
>> > for partition [mytopic,1] from OfflinePartition to OnlinePartition
>> failed
>> > (state.change.logger)
>> > kafka.common.NoReplicaOnlineException: No replica for partition
>> [mytopic,1]
>> > is alive. Live brokers are: [Set()], Assigned replicas are: [List(0)]
>> > at
>> >
>> >
>> kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:61)
>> >
>> > ===============
>> > controller.log
>> > ===============
>> > [2014-08-11 19:12:45,308] DEBUG [OfflinePartitionLeaderSelector]: No
>> broker
>> > in ISR is alive for [mytopic,1]. Pick the leader from the alive assigned
>> > replicas: (kafka.controller.OfflinePartitionLeaderSelector)
>> > [2014-08-11 19:12:45,321] DEBUG [OfflinePartitionLeaderSelector]: No
>> broker
>> > in ISR is alive for [mytopic,0]. Pick the leader from the alive assigned
>> > replicas: (kafka.controller.OfflinePartitionLeaderSelector)
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>
>
>
--
-- Guozhang
