[jira] [Created] (KAFKA-4305) Possible race condition in log segment truncation and Kafka Request Handler
Hari created KAFKA-4305: --- Summary: Possible race condition in log segment truncation and Kafka Request Handler Key: KAFKA-4305 URL: https://issues.apache.org/jira/browse/KAFKA-4305 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.10.0.1 Reporter: Hari Priority: Minor We started to observe this trace in our Kafka Server logs after migration to 0.10.0.1 {code} [2016-10-16 20:55:04,825] ERROR [KafkaApi-303] Error while responding to offset request (kafka.server.KafkaApis) java.lang.NullPointerException at kafka.server.KafkaApis.fetchOffsetsBefore(KafkaApis.scala:585) at kafka.server.KafkaApis.fetchOffsets(KafkaApis.scala:573) at kafka.server.KafkaApis$$anonfun$27.apply(KafkaApis.scala:530) at kafka.server.KafkaApis$$anonfun$27.apply(KafkaApis.scala:521) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at kafka.server.KafkaApis.handleOffsetRequest(KafkaApis.scala:521) at kafka.server.KafkaApis.handle(KafkaApis.scala:78) at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60) at java.lang.Thread.run(Thread.java:745) {code} >From a quick investigation, I suspect this could happen because of race >condition with log segment truncation and Kafka request handler ? Any thoughts ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1707) ConsumerOffsetChecker shows none partitions assigned
Hari created KAFKA-1707: --- Summary: ConsumerOffsetChecker shows none partitions assigned Key: KAFKA-1707 URL: https://issues.apache.org/jira/browse/KAFKA-1707 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.8.0 Environment: HP 40 x Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz/1.2e+02GB Reporter: Hari Assignee: Neha Narkhede bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker shows some partitions having none consumers after re-balance triggered due to new consumer joined/disconnected to the group. The lag gets piling up till the partitions are assigned to it after re-balance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1708) Consumers intermittently stop consuming till restart
Hari created KAFKA-1708: --- Summary: Consumers intermittently stop consuming till restart Key: KAFKA-1708 URL: https://issues.apache.org/jira/browse/KAFKA-1708 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.8.0 Environment: HP 40 x Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz/1.2e+02GB Reporter: Hari Assignee: Neha Narkhede Using a simple consumer, and reading messages using StreamIterator noticed that the consumptions suddenly stops and the lag starts building up till the consumer is restarted. Below is the code snippet final MapString, ListKafkaStreambyte[], byte[] streamsByName = consumerConnector.createMessageStreams(topicCountMap); ConsumerIteratorbyte[], byte[] streamIterator = streamsByName.get(topicName).get(IDX_FIRST_ITEM).iterator(); if (streamIterator.hasNext()) { final MessageAndMetadatabyte[], byte[] item = streamIterator.next(); ... } -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1707) ConsumerOffsetChecker shows none partitions assigned
[ https://issues.apache.org/jira/browse/KAFKA-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari updated KAFKA-1707: Description: bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker shows some partitions having none consumers after re-balance triggered due to new consumer joined/disconnected to the group. The lag gets piling up till the partitions are assigned to it usually after another re-balance trigger. (was: bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker shows some partitions having none consumers after re-balance triggered due to new consumer joined/disconnected to the group. The lag gets piling up till the partitions are assigned to it after re-balance.) ConsumerOffsetChecker shows none partitions assigned Key: KAFKA-1707 URL: https://issues.apache.org/jira/browse/KAFKA-1707 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.8.0 Environment: HP 40 x Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz/1.2e+02GB Reporter: Hari Assignee: Neha Narkhede Labels: patch bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker shows some partitions having none consumers after re-balance triggered due to new consumer joined/disconnected to the group. The lag gets piling up till the partitions are assigned to it usually after another re-balance trigger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)