[ https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200748#comment-14200748 ]
schandr commented on KAFKA-1738: -------------------------------- Will apply the patch against the 0.8.2-beta and post the update. > Partitions for topic not created after restart from forced shutdown > ------------------------------------------------------------------- > > Key: KAFKA-1738 > URL: https://issues.apache.org/jira/browse/KAFKA-1738 > Project: Kafka > Issue Type: Bug > Components: core > Affects Versions: 0.8.2 > Environment: Linux, 2GB RAM, 2 Core CPU > Reporter: Pradeep > Assignee: Jun Rao > Priority: Blocker > Fix For: 0.8.2 > > Attachments: 1738.zip, ServerLogForFailedTopicCreation.txt, > ServerLogForFailedTopicCreation.txt, ServerLogForSuccessfulTopicCreation.txt, > kafka-1738.patch > > > We are using Kafka Topic APIs to create the topic. But in some cases, the > topic gets created but we don't see the partition specific files and when > producer/consumer tries to get the topic metadata and it fails with > exception. Same happens if one tries to create using the command line. > k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for > topic tloader1 -> No partition metadata for topic tloader1 due to > kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class > kafka.common.UnknownTopicOrPartitionException > Steps to reproduce - > 1. Stop kafka using kill -9 <PID of Kafka> > 2. Start Kafka > 3. Create Topic with partition and replication factor of 1. > 4. Check the response “Created topic <topic_name>” > 5. Run the list command to verify if its created. > 6. Now check the data directory of kakfa. There would not be any for the > newly created topic. > We see issues when we are creating new topics. This happens randomly and we > dont know the exact reasons. We see the below logs in controller during the > time of creation of topics which doesnt have the partition files. > 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for > [JobJTopic,0] (kafka.controller.KafkaController) > [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation > callback for [JobJTopic,0] (kafka.controller.KafkaController) > [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]: > Invoking state change to NewPartition for partitions [JobJTopic,0] > (kafka.controller.PartitionStateMachine) > [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]: > Invoking state change to NewReplica for replicas > [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine) > [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]: > Invoking state change to OnlinePartition for partitions [JobJTopic,0] > (kafka.controller.PartitionStateMachine) > [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]: > Live assigned replicas for partition [JobJTopic,0] are: [List(0)] > (kafka.controller.PartitionStateMachine) > [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]: > Initializing leader and isr for partition [JobJTopic,0] to > (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2) > (kafka.controller.PartitionStateMachine) > [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]: > Invoking state change to OnlineReplica for replicas > [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine) > [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread], > Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092 > (kafka.controller.RequestSendThread) > java.io.EOFException: Received -1 when reading from channel, socket has > likely been closed. > at kafka.utils.Utils$.read(Utils.scala:381) > at > kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54) > at kafka.network.Receive$class.readCompletely(Transmission.scala:56) > at > kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29) > at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108) > at > kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread], > Controller 0 epoch 2 failed to send request > Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0] > -> > (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0) > to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker. > (kafka.controller.RequestSendThread) > java.nio.channels.ClosedChannelException > at kafka.network.BlockingChannel.send(BlockingChannel.scala:97) > at > kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132) > at > kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) -- This message was sent by Atlassian JIRA (v6.3.4#6332)