[
https://issues.apache.org/jira/browse/KAFKA-397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486297#comment-13486297
]
David Arthur commented on KAFKA-397:
------------------------------------
Try cleaning out ZooKeeper as well as the Kafka logs (as Jun suggested). Based
on your exception, I'd guess that your consumer is using the last offset stored
in ZooKeeper which exceeds the current size of your logs. I have seen
exceptions like this when screwing around with the log files.
> kafka.common.InvalidMessageSizeException: null
> ----------------------------------------------
>
> Key: KAFKA-397
> URL: https://issues.apache.org/jira/browse/KAFKA-397
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: David Siegel
> Priority: Blocker
> Attachments: kafka-logs-invalid-message-size-null.tar
>
>
> I've just gotten the following error while running the zookeeper consumer.
> I made a backup of the kafka log directory and wiped the logs. I restarting
> kafka and the consumer. After processing a few hundred messages successfully
> I got the same error again. I restarted the consumer again and got the same
> error immediately. I wiped the logs yet again and reproduced the error
> again. I will attach the logs from this final run.
> I'm running Kafka 0.7.1
> I have the following message size configurations:
> producer config:
> max.message.size: 1000000
> consumer config:
> fetch.size: 2072000
> Does it matter that I'm only wiping the logs and not wiping the zookeeper
> offsets?
> 2012-07-10 02:31:21,998 ERROR [Consumer1]
> c.k.h.c.k.KafkaConsumerServiceWorker: Failed to get next student event
> kafka.common.InvalidMessageSizeException: null
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method) ~[na:1.6.0_30]
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> ~[na:1.6.0_30]
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> ~[na:1.6.0_30]
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> ~[na:1.6.0_30]
> at java.lang.Class.newInstance0(Class.java:355) ~[na:1.6.0_30]
> at java.lang.Class.newInstance(Class.java:308) ~[na:1.6.0_30]
> at
> kafka.common.ErrorMapping$.maybeThrowException(ErrorMapping.scala:53)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.message.ByteBufferMessageSet.kafka$message$ByteBufferMessageSet$$internalIterator(ByteBufferMessageSet.scala:99)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.message.ByteBufferMessageSet.iterator(ByteBufferMessageSet.scala:82)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:81)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:32)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:59)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:51)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:36)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:43)
> ~[KPIP-0.4.birdy.jar:na]
> at java.lang.Thread.run(Thread.java:662) [na:1.6.0_30]
> 2012-07-10 02:31:21,998 ERROR [Consumer1]
> c.k.h.c.k.KafkaConsumerServiceWorker: Iterator got into bad state. Thread
> exiting
> java.lang.IllegalStateException: Iterator is in failed state
> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:47)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:36)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:43)
> ~[KPIP-0.4.birdy.jar:na]
> at java.lang.Thread.run(Thread.java:662) [na:1.6.0_30]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
