[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198648#comment-14198648
]
Pradeep edited comment on KAFKA-1738 at 11/5/14 5:03 PM:
---------------------------------------------------------
I suggest you to run the script once on a working setup of kafka. The only
error that we see on kafka controller.log is
[2014-11-05 11:42:53,088] DEBUG [Partition state machine on Controller 0]: Live
assigned replicas for partition [topic_13,0] are: [List(0)]
(kafka.controller.PartitionStateMachine)
[2014-11-05 11:42:53,088] DEBUG [Partition state machine on Controller 0]:
Initializing leader and isr for partition [topic_13,0] to
(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:5)
(kafka.controller.PartitionStateMachine)
[2014-11-05 11:42:53,097] WARN [Controller-0-to-broker-0-send-thread],
Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
(kafka.controller.RequestSendThread)
java.io.EOFException: Received -1 when reading from channel, socket has likely
been closed.
at kafka.utils.Utils$.read(Utils.scala:381)
at
kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
at
kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
[2014-11-05 11:42:53,097] INFO [Replica state machine on controller 0]:
Invoking state change to OnlineReplica for replicas
[Topic=topic_13,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
[2014-11-05 11:42:53,097] ERROR [Controller-0-to-broker-0-send-thread],
Controller 0 epoch 5 failed to send request
Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:5;CorrelationId:16;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[topic_13,0]
->
(LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:5),ReplicationFactor:1),AllReplicas:0)
to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
(kafka.controller.RequestSendThread)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
at
kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
[2014-11-05 11:42:53,099] INFO [Controller-0-to-broker-0-send-thread],
Controller 0 connected to id:0,host:DMIPVM,port:9092 for sending state change
requests (kafka.controller.RequestSendThread)
[2014-11-05 11:45:34,605] TRACE [Controller 0]: checking need to trigger
partition rebalance (kafka.controller.KafkaController)
[2014-11-05 11:45:34,607] DEBUG [Controller 0]: preferred replicas by broker
Map(0 -> Map([topic_8,0] -> List(0), [topic_13,0] -> List(0), [topic_6,0] ->
List(0), [topic_1,0] -> List(0), [topic_9,0] -> List(0), [topic_2,0] ->
List(0), [topic_11,0] -> List(0), [topic_4,0] -> List(0), [topic_12,0] ->
List(0), [topic_7,0] -> List(0), [topic_3,0] -> List(0), [topic_10,0] ->
List(0))) (kafka.controller.KafkaController)
[2014-11-05 11:45:34,608] DEBUG [Controller 0]: topics not in preferred replica
Map() (kafka.controller.KafkaController)
[2014-11
was (Author: pradeepbadiger):
I suggest you to run the script once on a working setup of kafka.
> Partitions for topic not created after restart from forced shutdown
> -------------------------------------------------------------------
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
> Reporter: Pradeep
> Attachments: 1738.zip
>
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9 <PID of Kafka>
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic <topic_name>”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
> We see issues when we are creating new topics. This happens randomly and we
> dont know the exact reasons. We see the below logs in controller during the
> time of creation of topics which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
> [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation
> callback for [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
> Invoking state change to NewPartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
> Invoking state change to NewReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
> Invoking state change to OnlinePartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Live assigned replicas for partition [JobJTopic,0] are: [List(0)]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Initializing leader and isr for partition [JobJTopic,0] to
> (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
> Invoking state change to OnlineReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread],
> Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
> (kafka.controller.RequestSendThread)
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
> at kafka.utils.Utils$.read(Utils.scala:381)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread],
> Controller 0 epoch 2 failed to send request
> Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
> ->
> (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0)
> to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
> (kafka.controller.RequestSendThread)
> java.nio.channels.ClosedChannelException
> at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
> at
> kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
