[
https://issues.apache.org/jira/browse/KAFKA-397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410660#comment-13410660
]
David Siegel commented on KAFKA-397:
------------------------------------
When we set the number of partitions per topic to 100 and set the consumer to
use 100 threads, we don't see this error.
When we set the number of partitions per topic to 100 and set the consumers to
use 2 threads, we see the error consistently.
> kafka.common.InvalidMessageSizeException: null
> ----------------------------------------------
>
> Key: KAFKA-397
> URL: https://issues.apache.org/jira/browse/KAFKA-397
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: David Siegel
> Priority: Blocker
> Attachments: kafka-logs-invalid-message-size-null.tar
>
>
> I've just gotten the following error while running the zookeeper consumer.
> I made a backup of the kafka log directory and wiped the logs. I restarting
> kafka and the consumer. After processing a few hundred messages successfully
> I got the same error again. I restarted the consumer again and got the same
> error immediately. I wiped the logs yet again and reproduced the error
> again. I will attach the logs from this final run.
> I'm running Kafka 0.7.1
> I have the following message size configurations:
> producer config:
> max.message.size: 1000000
> consumer config:
> fetch.size: 2072000
> Does it matter that I'm only wiping the logs and not wiping the zookeeper
> offsets?
> 2012-07-10 02:31:21,998 ERROR [Consumer1]
> c.k.h.c.k.KafkaConsumerServiceWorker: Failed to get next student event
> kafka.common.InvalidMessageSizeException: null
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method) ~[na:1.6.0_30]
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> ~[na:1.6.0_30]
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> ~[na:1.6.0_30]
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> ~[na:1.6.0_30]
> at java.lang.Class.newInstance0(Class.java:355) ~[na:1.6.0_30]
> at java.lang.Class.newInstance(Class.java:308) ~[na:1.6.0_30]
> at
> kafka.common.ErrorMapping$.maybeThrowException(ErrorMapping.scala:53)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.message.ByteBufferMessageSet.kafka$message$ByteBufferMessageSet$$internalIterator(ByteBufferMessageSet.scala:99)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.message.ByteBufferMessageSet.iterator(ByteBufferMessageSet.scala:82)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:81)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:32)
> ~[KPIP-0.4.birdy.jar:na]
> at
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:59)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:51)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:36)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:43)
> ~[KPIP-0.4.birdy.jar:na]
> at java.lang.Thread.run(Thread.java:662) [na:1.6.0_30]
> 2012-07-10 02:31:21,998 ERROR [Consumer1]
> c.k.h.c.k.KafkaConsumerServiceWorker: Iterator got into bad state. Thread
> exiting
> java.lang.IllegalStateException: Iterator is in failed state
> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:47)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:36)
> ~[KPIP-0.4.birdy.jar:na]
> at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:43)
> ~[KPIP-0.4.birdy.jar:na]
> at java.lang.Thread.run(Thread.java:662) [na:1.6.0_30]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
