[jira] [Created] (KAFKA-9829) Kafka brokers are un-registered on Zookeeper node replacement
Pradeep created KAFKA-9829:
--
Summary: Kafka brokers are un-registered on Zookeeper node
replacement
Key: KAFKA-9829
URL: https://issues.apache.org/jira/browse/KAFKA-9829
Project: Kafka
Issue Type: Bug
Affects Versions: 0.10.2.1
Reporter: Pradeep
We have a Kafka cluster with 3 nodes connected to a Zookeeper (3.4.14) cluster
of 3 nodes in AWS. We make use of the auto-scaling group to provision nodes
upon failures. We are seeing an issue where the Kafka brokers are getting
un-registered when all the Zookeeper nodes are replaced one after the other.
Every Zookeeper node is terminated from AWS console and we wait for a
replacement node to be provisioned with Zookeeper initialized before
terminating the other node.
On every Zookeeper node replacement, the /broker/ids path show all the Kafka
broker ids in the cluster. But only on the final Zookeeper node replacement,
the content in /broker/ids become empty.
We are seeing below logs in one of the Zookeeper nodes when all of the original
nodes are replaced.
{{2020-03-26 20:29:20,303 [myid:3] - INFO
[[SessionTracker:ZooKeeperServer@355|sessiontracker:ZooKeeperServer@355]] -
Expiring session 0x10003b973b50016, timeout of 6000ms exceeded}}
{{2020-03-26 20:29:20,303 [myid:3] - INFO
[[SessionTracker:ZooKeeperServer@355|sessiontracker:ZooKeeperServer@355]] -
Expiring session 0x10003b973b5000e, timeout of 6000ms exceeded}}
{{2020-03-26 20:29:20,303 [myid:3] - INFO
[[SessionTracker:ZooKeeperServer@355|sessiontracker:ZooKeeperServer@355]] -
Expiring session 0x30003a126690002, timeout of 6000ms exceeded}}
{{2020-03-26 20:29:20,307 [myid:3] - DEBUG [CommitProcessor:3:DataTree@893] -
Deleting ephemeral node /brokers/ids/1002 for session 0x10003b973b50016}}
{{2020-03-26 20:29:20,307 [myid:3] - DEBUG [CommitProcessor:3:DataTree@893] -
Deleting ephemeral node /brokers/ids/1003 for session 0x10003b973b5000e}}
{{2020-03-26 20:29:20,307 [myid:3] - DEBUG [CommitProcessor:3:DataTree@893] -
Deleting ephemeral node /controller for session 0x30003a126690002}}
{{2020-03-26 20:29:20,307 [myid:3] - DEBUG [CommitProcessor:3:DataTree@893] -
Deleting ephemeral node /brokers/ids/1001 for session 0x30003a126690002}}
I am not sure if the issue is related to KAFKA-5473.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198688#comment-14198688
]
Pradeep edited comment on KAFKA-1738 at 11/5/14 5:26 PM:
-
Can you provide the configuration files? We are using the default
configurations and the script tries to create a topic every 10 mins. Also, i
tried telnet and the broker is up.
was (Author: pradeepbadiger):
Can you provide the configuration files? We are using the default
configurations. Also, i tried telnet and the broker is up.
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
> Attachments: 1738.zip
>
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
> We see issues when we are creating new topics. This happens randomly and we
> dont know the exact reasons. We see the below logs in controller during the
> time of creation of topics which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
> [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation
> callback for [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
> Invoking state change to NewPartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
> Invoking state change to NewReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
> Invoking state change to OnlinePartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Live assigned replicas for partition [JobJTopic,0] are: [List(0)]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Initializing leader and isr for partition [JobJTopic,0] to
> (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
> Invoking state change to OnlineReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread],
> Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
> (kafka.controller.RequestSendThread)
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
> at kafka.utils.Utils$.read(Utils.scala:381)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread],
> Controller 0 epoch 2 failed to send request
> Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
> ->
> (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,Co
[jira] [Commented] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198688#comment-14198688
]
Pradeep commented on KAFKA-1738:
Can you provide the configuration files? We are using the default
configurations. Also, i tried telnet and the broker is up.
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
> Attachments: 1738.zip
>
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
> We see issues when we are creating new topics. This happens randomly and we
> dont know the exact reasons. We see the below logs in controller during the
> time of creation of topics which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
> [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation
> callback for [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
> Invoking state change to NewPartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
> Invoking state change to NewReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
> Invoking state change to OnlinePartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Live assigned replicas for partition [JobJTopic,0] are: [List(0)]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Initializing leader and isr for partition [JobJTopic,0] to
> (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
> Invoking state change to OnlineReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread],
> Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
> (kafka.controller.RequestSendThread)
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
> at kafka.utils.Utils$.read(Utils.scala:381)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread],
> Controller 0 epoch 2 failed to send request
> Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
> ->
> (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0)
> to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
> (kafka.controller.RequestSendThread)
> java.nio.channels.ClosedChannelException
> at kafka.network.BlockingChannel.send(BlockingChanne
[jira] [Comment Edited] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198648#comment-14198648
]
Pradeep edited comment on KAFKA-1738 at 11/5/14 5:03 PM:
-
I suggest you to run the script once on a working setup of kafka. The only
error that we see on kafka controller.log is
[2014-11-05 11:42:53,088] DEBUG [Partition state machine on Controller 0]: Live
assigned replicas for partition [topic_13,0] are: [List(0)]
(kafka.controller.PartitionStateMachine)
[2014-11-05 11:42:53,088] DEBUG [Partition state machine on Controller 0]:
Initializing leader and isr for partition [topic_13,0] to
(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:5)
(kafka.controller.PartitionStateMachine)
[2014-11-05 11:42:53,097] WARN [Controller-0-to-broker-0-send-thread],
Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
(kafka.controller.RequestSendThread)
java.io.EOFException: Received -1 when reading from channel, socket has likely
been closed.
at kafka.utils.Utils$.read(Utils.scala:381)
at
kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
at
kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
[2014-11-05 11:42:53,097] INFO [Replica state machine on controller 0]:
Invoking state change to OnlineReplica for replicas
[Topic=topic_13,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
[2014-11-05 11:42:53,097] ERROR [Controller-0-to-broker-0-send-thread],
Controller 0 epoch 5 failed to send request
Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:5;CorrelationId:16;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[topic_13,0]
->
(LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:5),ReplicationFactor:1),AllReplicas:0)
to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
(kafka.controller.RequestSendThread)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
at
kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
[2014-11-05 11:42:53,099] INFO [Controller-0-to-broker-0-send-thread],
Controller 0 connected to id:0,host:DMIPVM,port:9092 for sending state change
requests (kafka.controller.RequestSendThread)
[2014-11-05 11:45:34,605] TRACE [Controller 0]: checking need to trigger
partition rebalance (kafka.controller.KafkaController)
[2014-11-05 11:45:34,607] DEBUG [Controller 0]: preferred replicas by broker
Map(0 -> Map([topic_8,0] -> List(0), [topic_13,0] -> List(0), [topic_6,0] ->
List(0), [topic_1,0] -> List(0), [topic_9,0] -> List(0), [topic_2,0] ->
List(0), [topic_11,0] -> List(0), [topic_4,0] -> List(0), [topic_12,0] ->
List(0), [topic_7,0] -> List(0), [topic_3,0] -> List(0), [topic_10,0] ->
List(0))) (kafka.controller.KafkaController)
[2014-11-05 11:45:34,608] DEBUG [Controller 0]: topics not in preferred replica
Map() (kafka.controller.KafkaController)
[2014-11
was (Author: pradeepbadiger):
I suggest you to run the script once on a working setup of kafka.
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
> Attachments: 1738.zip
>
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5.
[jira] [Commented] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198648#comment-14198648
]
Pradeep commented on KAFKA-1738:
I suggest you to run the script once on a working setup of kafka.
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
> Attachments: 1738.zip
>
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
> We see issues when we are creating new topics. This happens randomly and we
> dont know the exact reasons. We see the below logs in controller during the
> time of creation of topics which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
> [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation
> callback for [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
> Invoking state change to NewPartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
> Invoking state change to NewReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
> Invoking state change to OnlinePartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Live assigned replicas for partition [JobJTopic,0] are: [List(0)]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Initializing leader and isr for partition [JobJTopic,0] to
> (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
> Invoking state change to OnlineReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread],
> Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
> (kafka.controller.RequestSendThread)
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
> at kafka.utils.Utils$.read(Utils.scala:381)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread],
> Controller 0 epoch 2 failed to send request
> Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
> ->
> (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0)
> to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
> (kafka.controller.RequestSendThread)
> java.nio.channels.ClosedChannelException
> at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
> at
> kafka.controller.RequestSendThrea
[jira] [Comment Edited] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197400#comment-14197400
]
Pradeep edited comment on KAFKA-1738 at 11/5/14 3:51 AM:
-
Attachment 1738.zip contains logs and a script which creates a topic every 10
mins.
was (Author: pradeepbadiger):
Contains logs and a script which creates a topic every 10 mins.
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
> Attachments: 1738.zip
>
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
> We see issues when we are creating new topics. This happens randomly and we
> dont know the exact reasons. We see the below logs in controller during the
> time of creation of topics which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
> [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation
> callback for [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
> Invoking state change to NewPartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
> Invoking state change to NewReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
> Invoking state change to OnlinePartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Live assigned replicas for partition [JobJTopic,0] are: [List(0)]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Initializing leader and isr for partition [JobJTopic,0] to
> (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
> Invoking state change to OnlineReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread],
> Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
> (kafka.controller.RequestSendThread)
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
> at kafka.utils.Utils$.read(Utils.scala:381)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread],
> Controller 0 epoch 2 failed to send request
> Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
> ->
> (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0)
> to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
> (kafka.controller.RequestSendThread)
[jira] [Updated] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep updated KAFKA-1738:
---
Attachment: 1738.zip
Contains logs and a script which creates a topic every 10 mins.
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
> Attachments: 1738.zip
>
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
> We see issues when we are creating new topics. This happens randomly and we
> dont know the exact reasons. We see the below logs in controller during the
> time of creation of topics which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
> [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation
> callback for [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
> Invoking state change to NewPartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
> Invoking state change to NewReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
> Invoking state change to OnlinePartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Live assigned replicas for partition [JobJTopic,0] are: [List(0)]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Initializing leader and isr for partition [JobJTopic,0] to
> (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
> Invoking state change to OnlineReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread],
> Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
> (kafka.controller.RequestSendThread)
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
> at kafka.utils.Utils$.read(Utils.scala:381)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread],
> Controller 0 epoch 2 failed to send request
> Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
> ->
> (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0)
> to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
> (kafka.controller.RequestSendThread)
> java.nio.channels.ClosedChannelException
> at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
> at
> kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelMa
[jira] [Commented] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196328#comment-14196328
]
Pradeep commented on KAFKA-1738:
yes.. This is happening in 8.2.0-beta version which we are currently on. I had
two topic in kakfa before killing the service and i added three more topics
(ATopic, BTopic and CTopic) and all of them got created successfully as you can
see below. When i list it i can see the topics in kakfa. But if i see the data
folder in kafka, i dont see the partition folders/files for them. Let me know
if you need more details.
[root@dmipvm temp]# service kafka status
Kafka is running as 19396.
LISTEN on tcp port=
LISTEN on tcp port=53536
LISTEN on tcp port=9092
LISTEN on tcp port=48330
[root@dmipvm temp]# kill -9 19396
[root@dmipvm temp]# service kafka start
Starting kafka ... STARTED.
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic ATopic
--partitions 1 --replication-factor 1 --zookeeper localhost:2181
Created topic "ATopic".
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic BTopic
--partitions 1 --replication-factor 1 --zookeeper localhost:2181
Created topic "BTopic".
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic CTopic
--partitions 1 --replication-factor 1 --zookeeper localhost:2181
Created topic "CTopic".
[root@dmipvm temp]#
[root@DMIPVM kafka]# ls -lrt
total 16
drwxr-xr-x 2 root root 4096 Nov 4 11:32 topic_1-0
drwxr-xr-x 2 root root 4096 Nov 4 11:34 topic_2-0
-rw-r--r-- 1 root root 28 Nov 4 11:35 replication-offset-checkpoint
-rw-r--r-- 1 root root 28 Nov 4 11:36 recovery-point-offset-checkpoint
[root@DMIPVM kafka]# /apps/kafka/bin/kafka-topics.sh --list --zookeeper
localhost:2181
ATopic
BTopic
CTopic
topic_1
topic_2
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
> We see issues when we are creating new topics. This happens randomly and we
> dont know the exact reasons. We see the below logs in controller during the
> time of creation of topics which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
> [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation
> callback for [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
> Invoking state change to NewPartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
> Invoking state change to NewReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
> Invoking state change to OnlinePartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Live assigned replicas for partition [JobJTopic,0] are: [List(0)]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Initializing leader and isr for partition [JobJTopic,0] to
> (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
> Invoking state change to OnlineReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11
[jira] [Updated] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep updated KAFKA-1738:
---
Description:
We are using Kafka Topic APIs to create the topic. But in some cases, the topic
gets created but we don't see the partition specific files and when
producer/consumer tries to get the topic metadata and it fails with exception.
Same happens if one tries to create using the command line.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic tloader1 -> No partition metadata for topic tloader1 due to
kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
kafka.common.UnknownTopicOrPartitionException
Steps to reproduce -
1. Stop kafka using kill -9
2. Start Kafka
3. Create Topic with partition and replication factor of 1.
4. Check the response “Created topic ”
5. Run the list command to verify if its created.
6. Now check the data directory of kakfa. There would not be any for the
newly created topic.
We see issues when we are creating new topics. This happens randomly and we
dont know the exact reasons. We see the below logs in controller during the
time of creation of topics which doesnt have the partition files.
2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
[JobJTopic,0] (kafka.controller.KafkaController)
[2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation callback
for [JobJTopic,0] (kafka.controller.KafkaController)
[2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
Invoking state change to NewPartition for partitions [JobJTopic,0]
(kafka.controller.PartitionStateMachine)
[2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
Invoking state change to NewReplica for replicas
[Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
[2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
Invoking state change to OnlinePartition for partitions [JobJTopic,0]
(kafka.controller.PartitionStateMachine)
[2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]: Live
assigned replicas for partition [JobJTopic,0] are: [List(0)]
(kafka.controller.PartitionStateMachine)
[2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
Initializing leader and isr for partition [JobJTopic,0] to
(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
(kafka.controller.PartitionStateMachine)
[2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
Invoking state change to OnlineReplica for replicas
[Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
[2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread],
Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
(kafka.controller.RequestSendThread)
java.io.EOFException: Received -1 when reading from channel, socket has likely
been closed.
at kafka.utils.Utils$.read(Utils.scala:381)
at
kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
at
kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
[2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread],
Controller 0 epoch 2 failed to send request
Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
->
(LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0)
to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
(kafka.controller.RequestSendThread)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
at
kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
was:
We are using Kafka Topic APIs to create the topic. But in some cases, the topic
gets created but we don't see the partition specific files and when
producer/consumer tries to get the topic metadata and it fails with exception.
Same happens if one tries to create using the command line.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic tloader1 -> No partition metadata for topic tloader1 due to
kafka.common.UnknownTopicOrPartitionExceptio
[jira] [Commented] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191229#comment-14191229
]
Pradeep commented on KAFKA-1738:
Yes. We tried sending data but it gave the exception.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic JobCTopic -> No partition metadata for topic JobCTopic due to
kafka.common.UnknownTopicOrPartitionException}] for topic [JobCTopic]: class
kafka.common.UnknownTopicOrPartitionException
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep updated KAFKA-1738:
---
Description:
We are using Kafka Topic APIs to create the topic. But in some cases, the topic
gets created but we don't see the partition specific files and when
producer/consumer tries to get the topic metadata and it fails with exception.
Same happens if one tries to create using the command line.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic tloader1 -> No partition metadata for topic tloader1 due to
kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
kafka.common.UnknownTopicOrPartitionException
Steps to reproduce -
1. Stop kafka using kill -9
2. Start Kafka
3. Create Topic with partition and replication factor of 1.
4. Check the response “Created topic ”
5. Run the list command to verify if its created.
6. Now check the data directory of kakfa. There would not be any for the
newly created topic.
was:
We are using Kafka Topic APIs to create the topic. But in some cases, the topic
gets created but we don't see the partition specific files and when
producer/consumer tries to get the topic metadata and it fails with exception.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic tloader1 -> No partition metadata for topic tloader1 due to
kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
kafka.common.UnknownTopicOrPartitionException
Steps to reproduce -
1. Stop kafka using kill -9
2. Start Kafka
3. Create Topic with partition and replication factor of 1.
4. Check the response “Created topic ”
5. Run the list command to verify if its created.
6. Now check the data directory of kakfa. There would not be any for the
newly created topic.
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1738) Partitions for topic not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep updated KAFKA-1738:
---
Summary: Partitions for topic not created after restart from forced
shutdown (was: Partition files not created after restart from forced shutdown)
> Partitions for topic not created after restart from forced shutdown
> ---
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with exception.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1738) Partition files not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep updated KAFKA-1738:
---
Affects Version/s: 0.8.2
> Partition files not created after restart from forced shutdown
> --
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with exception.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1738) Partition files not created after restart from forced shutdown
[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep updated KAFKA-1738:
---
Description:
We are using Kafka Topic APIs to create the topic. But in some cases, the topic
gets created but we don't see the partition specific files and when
producer/consumer tries to get the topic metadata and it fails with exception.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic tloader1 -> No partition metadata for topic tloader1 due to
kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
kafka.common.UnknownTopicOrPartitionException
Steps to reproduce -
1. Stop kafka using kill -9
2. Start Kafka
3. Create Topic with partition and replication factor of 1.
4. Check the response “Created topic ”
5. Run the list command to verify if its created.
6. Now check the data directory of kakfa. There would not be any for the
newly created topic.
was:
We are using Kafka Topic APIs to create the topic. But in some cases, the topic
gets created but we don't see the partition specific files and when
producer/consumer tries to get the topic metadata and it fails with exception.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic tloader1 -> No partition metadata for topic tloader1 due to
kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
kafka.common.UnknownTopicOrPartitionException
> Partition files not created after restart from forced shutdown
> --
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
>Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
>Reporter: Pradeep
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with exception.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic ”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KAFKA-1738) Partition files not created after restart from forced shutdown
Pradeep created KAFKA-1738:
--
Summary: Partition files not created after restart from forced
shutdown
Key: KAFKA-1738
URL: https://issues.apache.org/jira/browse/KAFKA-1738
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.8.1.1
Environment: Linux, 2GB RAM, 2 Core CPU
Reporter: Pradeep
We are using Kafka Topic APIs to create the topic. But in some cases, the topic
gets created but we don't see the partition specific files and when
producer/consumer tries to get the topic metadata and it fails with exception.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic tloader1 -> No partition metadata for topic tloader1 due to
kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
kafka.common.UnknownTopicOrPartitionException
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KAFKA-1713) Partition files not created
Pradeep created KAFKA-1713:
--
Summary: Partition files not created
Key: KAFKA-1713
URL: https://issues.apache.org/jira/browse/KAFKA-1713
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.8.1.1
Environment: Linux, 2GB RAM, 2 Core CPU
Reporter: Pradeep
We are using Kafka Topic APIs to create the topic. But in some cases, the topic
gets created but we don't see the partition specific files and when
producer/consumer tries to get the topic metadata and it fails with exception.
k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
topic tloader1 -> No partition metadata for topic tloader1 due to
kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
kafka.common.UnknownTopicOrPartitionException
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
