Not sure if this is an os specific issue. Is it possible to reproduce this?
Thanks, Jun On Mon, Oct 8, 2012 at 3:24 AM, Holger Buss <[email protected]> wrote: > Hi Jun, > > under these "invalid" directories are nothing, there are empty. > > btw. our Kafka Version is kafka-0.7.1-1 and we are using CentOS 6. > > We're using kafka with Java. > > Best regards > Holger Buss > > > > Interesting. A valid directory name is of the form topic-partitionId. > > What's under those invalid directories (e.g, r, re, etc)? We recently > > fixed > > a topic name issue (https://issues.apache.org/jira/browse/KAFKA-495). > Not > > sure if it's related. Also, are you using a non-java producer? > > > > Thanks, > > > > Jun > > > > On Mon, Oct 1, 2012 at 4:12 AM, Holger Buss <[email protected]> wrote: > > > > Hi, > > > > we had the problem, that the start of kafka brokes down, because the > > written Log directories are empty, have strange names and doesn't exist > in > > Zookeeper as a topic. > > > > Here are a ls from the kafka logs folder: > > > > /var/lib/kafka/logs/ > > drwxr-xr-x 2 kafka users 4096 24. Aug 13:22 r > > drwxr-xr-x 2 kafka users 4096 24. Aug 00:54 re > > drwxr-xr-x 2 kafka users 4096 29. Aug 01:34 rec > > drwxr-xr-x 2 kafka users 4096 12. Sep 05:30 reci > > drwxr-xr-x 2 kafka users 4096 22. Aug 01:08 recip > > drwxr-xr-x 2 kafka users 4096 11. Aug 06:22 recipi > > drwxr-xr-x 2 kafka users 4096 23. Aug 05:12 recipie > > drwxr-xr-x 2 kafka users 4096 5. Sep 01:08 recipien > > drwxr-xr-x 2 kafka users 4096 22. Aug 00:50 recipient > > drwxr-xr-x 2 kafka users 4096 27. Aug 01:13 recipientE > > drwxr-xr-x 2 kafka users 4096 4. Sep 01:02 recipientEv > > drwxr-xr-x 2 kafka users 4096 13. Aug 05:53 recipientEve > > drwxr-xr-x 2 kafka users 4096 23. Aug 01:06 recipientEven > > drwxr-xr-x 2 kafka users 4096 4. Sep 01:07 recipientEvent > > drwxr-xr-x 2 kafka users 4096 20. Sep 10:32 recipientEvents-0 > > drwxr-xr-x 2 kafka users 4096 20. Sep 10:38 recipientEvents-1 > > drwxr-xr-x 2 kafka users 4096 20. Sep 09:41 recipientEvents-10 > > drwxr-xr-x 2 kafka users 4096 20. Sep 09:26 recipientEvents-11 > > drwxr-xr-x 2 kafka users 4096 20. Sep 10:57 recipientEvents-12 > > drwxr-xr-x 2 kafka users 4096 20. Sep 11:24 recipientEvents-13 > > drwxr-xr-x 2 kafka users 4096 20. Sep 09:41 recipientEvents-14 > > drwxr-xr-x 2 kafka users 4096 20. Sep 10:51 recipientEvents-15 > > drwxr-xr-x 2 kafka users 4096 20. Sep 11:41 recipientEvents-16 > > drwxr-xr-x 2 kafka users 4096 20. Sep 10:06 recipientEvents-17 > > drwxr-xr-x 2 kafka users 4096 20. Sep 09:26 recipientEvents-18 > > drwxr-xr-x 2 kafka users 4096 20. Sep 12:02 recipientEvents-19 > > drwxr-xr-x 2 kafka users 4096 20. Sep 10:28 recipientEvents-2 > > drwxr-xr-x 2 kafka users 4096 20. Sep 10:45 recipientEvents-3 > > drwxr-xr-x 2 kafka users 4096 20. Sep 09:31 recipientEvents-4 > > drwxr-xr-x 2 kafka users 4096 20. Sep 09:03 recipientEvents-5 > > > > > > The error Meesage at kafka startup: > > > > Picked up _JAVA_OPTIONS: -XX:DefaultInitialRAMFraction=128 > > -XX:DefaultMaxRAMFraction=16 > > [2012-09-20 12:03:32,509] INFO Starting Kafka server... > > (kafka.server.KafkaServer) > > [2012-09-20 12:03:32,588] INFO Loading log 'systemEvents-1' > > (kafka.log.LogManager) > > [2012-09-20 12:03:32,646] INFO Loading the last segment > > /var/kafka_1_02/logs/systemEvents-1/00000000000000000000.kafka in mutable > > mode, recovery false (kafka.log.Log) > > [2012-09-20 12:03:32,675] INFO Loading log 'recipientEvents-3' > > (kafka.log.LogManager) > > [2012-09-20 12:03:32,680] INFO Loading the last segment > > /var/kafka_1_02/logs/recipientEvents-3/00000000011691660453.kafka in > > mutable mode, recovery false (kafka.log.Log) > > [2012-09-20 12:03:32,681] INFO Loading log 'recipi' > (kafka.log.LogManager) > > [2012-09-20 12:03:32,697] FATAL Fatal error during KafkaServerStable > > startup. Prepare to shutdown (kafka.server.KafkaServerStartable) > > java.lang.StringIndexOutOfBoundsException: String index out of range: -1 > > at java.lang.String.substring(String.java:1937) > > at kafka.utils.Utils$.getTopicPartition(Utils.scala:558) > > at kafka.log.LogManager$$anonfun$4.apply(LogManager.scala:71) > > at kafka.log.LogManager$$anonfun$4.apply(LogManager.scala:65) > > at > > > scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34) > > at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34) > > at kafka.log.LogManager.<init>(LogManager.scala:65) > > at kafka.server.KafkaServer.startup(KafkaServer.scala:58) > > at > > kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34) > > at kafka.Kafka$.main(Kafka.scala:50) > > at kafka.Kafka.main(Kafka.scala) > > [2012-09-20 12:03:32,700] INFO Shutting down Kafka server > > (kafka.server.KafkaServer) > > [2012-09-20 12:03:32,700] INFO shutdown scheduler kafka-logcleaner- > > (kafka.utils.KafkaScheduler) > > [2012-09-20 12:03:32,701] INFO Kafka server shut down completed > > (kafka.server.KafkaServer) > > > > r > > re > > rec > > reci > > recip > > recipi > > recipie > > recipien > > recipient > > recipientE > > recipientEv > > recipientEve > > recipientEven > > recipientEvent > > > > Who create this empty folders? Possible broken Messages? > > > > Best regards > > Holger Buss > > >
