[
https://issues.apache.org/jira/browse/KAFKA-2236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902803#comment-14902803
]
Robert Metzger commented on KAFKA-2236:
---------------------------------------
I've seen this error as well in the integration tests of Flink (We are starting
a Kafka Broker with the test to test our Kafka connectors):
{code}
14:43:03,328 INFO kafka.network.Processor
- Closing socket connection to /127.0.0.1.
14:43:03,334 WARN kafka.server.KafkaApis
- [KafkaApi-0] Error while responding to offset request
java.lang.ArrayIndexOutOfBoundsException: 1
at kafka.server.KafkaApis.fetchOffsetsBefore(KafkaApis.scala:469)
at kafka.server.KafkaApis.fetchOffsets(KafkaApis.scala:449)
at kafka.server.KafkaApis$$anonfun$17.apply(KafkaApis.scala:411)
at kafka.server.KafkaApis$$anonfun$17.apply(KafkaApis.scala:402)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at kafka.server.KafkaApis.handleOffsetRequest(KafkaApis.scala:402)
at kafka.server.KafkaApis.handle(KafkaApis.scala:61)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:59)
at java.lang.Thread.run(Thread.java:745)
{code}
on the client side, we are seeing a {{kafka.common.UnknownException}}:
{code}
Caused by: java.lang.RuntimeException: Unable to get last offset for topic
customPartitioningTestTopic and partitions [FetchPartition {partition=2,
offset=-915623761776}].
Exception for partition 2: kafka.common.UnknownException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at java.lang.Class.newInstance(Class.java:438)
at kafka.common.ErrorMapping$.exceptionFor(ErrorMapping.scala:86)
at kafka.common.ErrorMapping.exceptionFor(ErrorMapping.scala)
at
org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher$SimpleConsumerThread.getLastOffset(LegacyFetcher.java:521)
at
org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher$SimpleConsumerThread.run(LegacyFetcher.java:370)
{code}
Here is the log of the travis build:
https://s3.amazonaws.com/archive.travis-ci.org/jobs/81584444/log.txt
this is the archive containing the full logs at INFO level:
https://flink-logs-us.s3.amazonaws.com/travis-artifacts/rmetzger/flink/1123/1123.4.tar.gz
(see 2.log)
It would be nice if you could address the issue with the next release.
> offset request reply racing with segment rolling
> ------------------------------------------------
>
> Key: KAFKA-2236
> URL: https://issues.apache.org/jira/browse/KAFKA-2236
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8.2.0
> Environment: Linux x86_64, java.1.7.0_72, discovered using librdkafka
> based client.
> Reporter: Alfred Landrum
> Assignee: Jason Gustafson
> Priority: Critical
> Labels: newbie
>
> My use case with kafka involves an aggressive retention policy that rolls
> segment files frequently. My librdkafka based client sees occasional errors
> to offset requests, showing up in the broker log like:
> [2015-06-02 02:33:38,047] INFO Rolled new log segment for
> 'receiver-93b40462-3850-47c1-bcda-8a3e221328ca-50' in 1 ms. (kafka.log.Log)
> [2015-06-02 02:33:38,049] WARN [KafkaApi-0] Error while responding to offset
> request (kafka.server.KafkaApis)
> java.lang.ArrayIndexOutOfBoundsException: 3
> at kafka.server.KafkaApis.fetchOffsetsBefore(KafkaApis.scala:469)
> at kafka.server.KafkaApis.fetchOffsets(KafkaApis.scala:449)
> at kafka.server.KafkaApis$$anonfun$17.apply(KafkaApis.scala:411)
> at kafka.server.KafkaApis$$anonfun$17.apply(KafkaApis.scala:402)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
> at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
> at scala.collection.AbstractTraversable.map(Traversable.scala:105)
> at kafka.server.KafkaApis.handleOffsetRequest(KafkaApis.scala:402)
> at kafka.server.KafkaApis.handle(KafkaApis.scala:61)
> at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:59)
> at java.lang.Thread.run(Thread.java:745)
> quoting Guozhang Wang's reply to my query on the users list:
> "I check the 0.8.2 code and may probably find a bug related to your issue.
> Basically, segsArray.last.size is called multiple times during handling
> offset requests, while segsArray.last could get concurrent appends. Hence
> it is possible that in line 461, if(segsArray.last.size > 0) returns false
> while later in line 468, if(segsArray.last.size > 0) could return true."
> http://mail-archives.apache.org/mod_mbox/kafka-users/201506.mbox/%3CCAHwHRrUK-3wdoEAaFbsD0E859Ea0gXixfxgDzF8E3%3D_8r7K%2Bpw%40mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
