Hi all,
I am seeing some corrupt data on my 0.7.2 Kafka cluster. Every once in awhile
I'll get the following message:
Exception in thread "kafka-consumer" java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Unknown Source)
at java.nio.HeapByteBuffer.get(Unknown Source)
at kafka.message.Message.magic(Message.scala:133)
at kafka.message.Message.checksum(Message.scala:146)
at kafka.message.Message.isValid(Message.scala:158)
at
kafka.message.ByteBufferMessageSet$$anon$1.makeNextOuter(ByteBufferMessageSet.scala:129)
at
kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:166)
at
kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scala:100)
at
kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:59)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:51)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
at kafka.message.MessageSet.foreach(MessageSet.scala:87)
at
kafka.tools.SimpleConsumerShell$$anon$1$$anonfun$run$1.apply(SimpleConsumerShell.scala:97)
at
kafka.tools.SimpleConsumerShell$$anon$1$$anonfun$run$1.apply(SimpleConsumerShell.scala:94)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
at kafka.api.MultiFetchResponse.foreach(MultiFetchResponse.scala:25)
at
kafka.tools.SimpleConsumerShell$$anon$1.run(SimpleConsumerShell.scala:94)
The only way I can get past this error is to move the offset past the current
message to skip the bad data. I use the simpleConsumer.getOffsetsBefore()
method to get a valid lists of offsets and I skip to the next valid offset.
Has anyone encountered this issue before? I found a message where someone
experienced similar issues and they were running JDK 1.7 and was able to get
around it by running JDK 1.6. I am running OpenJDK 1.7 but I tried running Sun
JDK 1.6 but I still have the same problem.
My guess is that the data is corrupt somehow but I don't know how to look at
the raw data to confirm. Can anyone suggest ideas for me to debug this issue?
Thanks,
Xuyen
