[
https://issues.apache.org/jira/browse/KAFKA-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009314#comment-16009314
]
jaikiran pai commented on KAFKA-5232:
-------------------------------------
One of the crude (but easy) way to reproduce this issue is as follows:
1. Bring up Zookeeper and Kafka broker and create a topic (which has dot in its
name).
{code}
./kafka-topics.sh --create --topic hello.world-testtopic --replication-factor 1
--partitions 1 --zookeeper localhost:2181
{code}
2. Let's assume {{/tmp/kafka-logs}} is where Kafka is configured for its topic
logs. Now let's delete the newly created topic
{code}
./kafka-topics.sh --delete --topic hello.world-testtopic --zookeeper
localhost:2181
{code}
3. Immediately as soon as this topic deletion command succeeds {{kill -9
<kafka-broker-pid>}} to replicate the case where for whatever reason, Kafka
broker didn't delete the topics that have been marked for deletion.
4. Make sure the topic that we deleted is still lying around marked for
deletion, in the topic logs directory, by running:
{code}
ls /tmp/kafka-logs
{code}
You should see a directory like this, in there
{{hello.world-testtopic-0.<some-unique-id>-delete}}. If you don't find such a
directory in there, repeat steps 1 to 4 above till you end up in a state where
Kafka broker hasn't been able to trigger the deletion of the topic. Only then
move to step 5.
5. Now that the topic, with the dot character, that is marked for deletion is
lying around, start the Kafka broker. You will see exceptions like this:
{code}
[2017-05-12 21:36:27,876] ERROR There was an error in one of the threads during
logs loading: java.lang.StringIndexOutOfBoundsException: String index out of
range: -1 (kafka.log.LogManager)
[2017-05-12 21:36:27,900] FATAL [Kafka Server 0], Fatal error during
KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1967)
at kafka.log.Log$.parseTopicPartitionName(Log.scala:1146)
at kafka.log.LogManager.$anonfun$loadLogs$10(LogManager.scala:153)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:57)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[2017-05-12 21:36:27,950] INFO [Kafka Server 0], shutting down
(kafka.server.KafkaServer)
{code}
and Kafka broker never starts and keeps shutting down.
> Kafka broker fails to start if a topic containing dot in its name is marked
> for delete but hasn't been deleted during previous uptime
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-5232
> URL: https://issues.apache.org/jira/browse/KAFKA-5232
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.10.2.0, 0.10.2.1
> Reporter: jaikiran pai
>
> We are using 0.10.2.0 (but this is reproducible even with 0.10.2.1 and latest
> upstream) in our environments. Our topic names contain (one or more) dot
> characters in their name. So we have topics like {{foo.bar-testtopic}}. Topic
> deletion is enabled on the broker(s) and our application does delete the
> topics as and when necessary.
> We just ran into a case today where for some reason the Kafka broker had
> either to be taken down (or went down on its own). Some of the topics which
> were deleted (i.e. a deletion marker folder was created for them) were left
> around after Kafka broker had gone down. So the Kafka logs dir had
> directories like
> {{foo.bar-testtopic-0.bb7981c216b845648edfe6e2b0a5c050-delete}}.
> When we restarted the Kafka broker, it refused to start and kept shutting
> down and running into this exception:
> {code}
> [2017-05-12 21:36:27,876] ERROR There was an error in one of the threads
> during logs loading: java.lang.StringIndexOutOfBoundsException: String index
> out of range: -1 (kafka.log.LogManager)
> [2017-05-12 21:36:27,900] FATAL [Kafka Server 0], Fatal error during
> KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> at java.lang.String.substring(String.java:1967)
> at kafka.log.Log$.parseTopicPartitionName(Log.scala:1146)
> at kafka.log.LogManager.$anonfun$loadLogs$10(LogManager.scala:153)
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:57)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> [2017-05-12 21:36:27,950] INFO [Kafka Server 0], shutting down
> (kafka.server.KafkaServer)
> {code}
> The only way we could get past this is pointing Kafka broker to a different
> Kafka logs directory which effectively meant a lot our topics were no longer
> accessible to the application.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
