Hi,
we had the problem, that the start of kafka brokes down, because the written
Log directories are empty, have strange names and doesn't exist in Zookeeper as
a topic.
Here are a ls from the kafka logs folder:
/var/lib/kafka/logs/
drwxr-xr-x 2 kafka users 4096 24. Aug 13:22 r
drwxr-xr-x 2 kafka users 4096 24. Aug 00:54 re
drwxr-xr-x 2 kafka users 4096 29. Aug 01:34 rec
drwxr-xr-x 2 kafka users 4096 12. Sep 05:30 reci
drwxr-xr-x 2 kafka users 4096 22. Aug 01:08 recip
drwxr-xr-x 2 kafka users 4096 11. Aug 06:22 recipi
drwxr-xr-x 2 kafka users 4096 23. Aug 05:12 recipie
drwxr-xr-x 2 kafka users 4096 5. Sep 01:08 recipien
drwxr-xr-x 2 kafka users 4096 22. Aug 00:50 recipient
drwxr-xr-x 2 kafka users 4096 27. Aug 01:13 recipientE
drwxr-xr-x 2 kafka users 4096 4. Sep 01:02 recipientEv
drwxr-xr-x 2 kafka users 4096 13. Aug 05:53 recipientEve
drwxr-xr-x 2 kafka users 4096 23. Aug 01:06 recipientEven
drwxr-xr-x 2 kafka users 4096 4. Sep 01:07 recipientEvent
drwxr-xr-x 2 kafka users 4096 20. Sep 10:32 recipientEvents-0
drwxr-xr-x 2 kafka users 4096 20. Sep 10:38 recipientEvents-1
drwxr-xr-x 2 kafka users 4096 20. Sep 09:41 recipientEvents-10
drwxr-xr-x 2 kafka users 4096 20. Sep 09:26 recipientEvents-11
drwxr-xr-x 2 kafka users 4096 20. Sep 10:57 recipientEvents-12
drwxr-xr-x 2 kafka users 4096 20. Sep 11:24 recipientEvents-13
drwxr-xr-x 2 kafka users 4096 20. Sep 09:41 recipientEvents-14
drwxr-xr-x 2 kafka users 4096 20. Sep 10:51 recipientEvents-15
drwxr-xr-x 2 kafka users 4096 20. Sep 11:41 recipientEvents-16
drwxr-xr-x 2 kafka users 4096 20. Sep 10:06 recipientEvents-17
drwxr-xr-x 2 kafka users 4096 20. Sep 09:26 recipientEvents-18
drwxr-xr-x 2 kafka users 4096 20. Sep 12:02 recipientEvents-19
drwxr-xr-x 2 kafka users 4096 20. Sep 10:28 recipientEvents-2
drwxr-xr-x 2 kafka users 4096 20. Sep 10:45 recipientEvents-3
drwxr-xr-x 2 kafka users 4096 20. Sep 09:31 recipientEvents-4
drwxr-xr-x 2 kafka users 4096 20. Sep 09:03 recipientEvents-5
The error Meesage at kafka startup:
Picked up _JAVA_OPTIONS: -XX:DefaultInitialRAMFraction=128
-XX:DefaultMaxRAMFraction=16
[2012-09-20 12:03:32,509] INFO Starting Kafka server...
(kafka.server.KafkaServer)
[2012-09-20 12:03:32,588] INFO Loading log 'systemEvents-1'
(kafka.log.LogManager)
[2012-09-20 12:03:32,646] INFO Loading the last segment
/var/kafka_1_02/logs/systemEvents-1/00000000000000000000.kafka in mutable mode,
recovery false (kafka.log.Log)
[2012-09-20 12:03:32,675] INFO Loading log 'recipientEvents-3'
(kafka.log.LogManager)
[2012-09-20 12:03:32,680] INFO Loading the last segment
/var/kafka_1_02/logs/recipientEvents-3/00000000011691660453.kafka in mutable
mode, recovery false (kafka.log.Log)
[2012-09-20 12:03:32,681] INFO Loading log 'recipi' (kafka.log.LogManager)
[2012-09-20 12:03:32,697] FATAL Fatal error during KafkaServerStable startup.
Prepare to shutdown (kafka.server.KafkaServerStartable)
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1937)
at kafka.utils.Utils$.getTopicPartition(Utils.scala:558)
at kafka.log.LogManager$$anonfun$4.apply(LogManager.scala:71)
at kafka.log.LogManager$$anonfun$4.apply(LogManager.scala:65)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
at kafka.log.LogManager.<init>(LogManager.scala:65)
at kafka.server.KafkaServer.startup(KafkaServer.scala:58)
at
kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
at kafka.Kafka$.main(Kafka.scala:50)
at kafka.Kafka.main(Kafka.scala)
[2012-09-20 12:03:32,700] INFO Shutting down Kafka server
(kafka.server.KafkaServer)
[2012-09-20 12:03:32,700] INFO shutdown scheduler kafka-logcleaner-
(kafka.utils.KafkaScheduler)
[2012-09-20 12:03:32,701] INFO Kafka server shut down completed
(kafka.server.KafkaServer)
r
re
rec
reci
recip
recipi
recipie
recipien
recipient
recipientE
recipientEv
recipientEve
recipientEven
recipientEvent
Who create this empty folders? Possible broken Messages?
Best regards
Holger Buss
