[
https://issues.apache.org/jira/browse/KAFKA-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997655#comment-15997655
]
Matthias J. Sax commented on KAFKA-5154:
----------------------------------------
With regard to the exception you mentioned, there is one known bug:
https://issues.apache.org/jira/browse/KAFKA-4593 -- The exception itself
occurs, if during restore process, the changelog end-offset changes. This
indicates, that another task writes to the changelog topic, and thus, the task
got migrated during restoration. Thus, we can stop restoration process and kill
the task (as it is a zombie and does not own the partitions anymore). This
could happen if restore takes long and a thread misses a rebalance. Just
mentions this for clarification. As you say it's from a different application,
it should not be related to the NPE. Btw: you can't attach any logs (there is
just a file name but nothing more).
> Kafka Streams throws NPE during rebalance
> -----------------------------------------
>
> Key: KAFKA-5154
> URL: https://issues.apache.org/jira/browse/KAFKA-5154
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.0
> Reporter: Lukas Gemela
> Attachments: clio.txt.gz
>
>
> please see attached log, Kafka streams throws NullPointerException during
> rebalance, which is caught by our custom exception handler
> {noformat}
> 2017-04-30T17:44:17,675 INFO kafka-coordinator-heartbeat-thread | hades
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.coordinatorDead()
> @618 - Marking the coordinator 10.210.200.144:9092 (id: 2147483644 rack:
> null) dead for group hades
> 2017-04-30T17:44:27,395 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.onSuccess()
> @573 - Discovered coordinator 10.210.200.144:9092 (id: 2147483644 rack: null)
> for group hades.
> 2017-04-30T17:44:27,941 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinPrepare()
> @393 - Revoking previously assigned partitions [poseidonIncidentFeed-27,
> poseidonIncidentFeed-29, poseidonIncidentFeed-30, poseidonIncidentFeed-18]
> for group hades
> 2017-04-30T17:44:27,947 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.sendJoinGroupRequest()
> @407 - (Re-)joining group hades
> 2017-04-30T17:44:48,468 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.sendJoinGroupRequest()
> @407 - (Re-)joining group hades
> 2017-04-30T17:44:53,628 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.sendJoinGroupRequest()
> @407 - (Re-)joining group hades
> 2017-04-30T17:45:09,587 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.sendJoinGroupRequest()
> @407 - (Re-)joining group hades
> 2017-04-30T17:45:11,961 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.onSuccess()
> @375 - Successfully joined group hades with generation 99
> 2017-04-30T17:45:13,126 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete()
> @252 - Setting newly assigned partitions [poseidonIncidentFeed-11,
> poseidonIncidentFeed-27, poseidonIncidentFeed-25, poseidonIncidentFeed-29,
> poseidonIncidentFeed-19, poseidonIncidentFeed-18] for group hades
> 2017-04-30T17:46:37,254 INFO kafka-coordinator-heartbeat-thread | hades
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.coordinatorDead()
> @618 - Marking the coordinator 10.210.200.144:9092 (id: 2147483644 rack:
> null) dead for group hades
> 2017-04-30T18:04:25,993 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.onSuccess()
> @573 - Discovered coordinator 10.210.200.144:9092 (id: 2147483644 rack: null)
> for group hades.
> 2017-04-30T18:04:29,401 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinPrepare()
> @393 - Revoking previously assigned partitions [poseidonIncidentFeed-11,
> poseidonIncidentFeed-27, poseidonIncidentFeed-25, poseidonIncidentFeed-29,
> poseidonIncidentFeed-19, poseidonIncidentFeed-18] for group hades
> 2017-04-30T18:05:10,877 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.sendJoinGroupRequest()
> @407 - (Re-)joining group hades
> 2017-05-01T00:01:55,707 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.coordinatorDead()
> @618 - Marking the coordinator 10.210.200.144:9092 (id: 2147483644 rack:
> null) dead for group hades
> 2017-05-01T00:01:59,027 INFO StreamThread-1
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.onSuccess()
> @573 - Discovered coordinator 10.210.200.144:9092 (id: 2147483644 rack: null)
> for group hades.
> 2017-05-01T00:01:59,031 ERROR StreamThread-1
> org.apache.kafka.streams.processor.internals.StreamThread.run() @376 -
> stream-thread [StreamThread-1] Streams application error during processing:
> java.lang.NullPointerException
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:619)
> ~[kafka-streams-0.10.2.0.jar!/:?]
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:368)
> [kafka-streams-0.10.2.0.jar!/:?]
> 2017-05-01T00:02:00,038 INFO StreamThread-1
> org.apache.kafka.clients.producer.KafkaProducer.close() @689 - Closing the
> Kafka producer with timeoutMillis = 9223372036854775807 ms.
> 2017-05-01T00:02:00,949 WARN StreamThread-1
> org.apache.kafka.streams.processor.internals.StreamThread.setState() @160 -
> Unexpected state transition from PARTITIONS_REVOKED to NOT_RUNNING
> 2017-05-01T00:02:00,951 ERROR StreamThread-1
> com.williamhill.trading.platform.hades.kafka.KafkaStreamManager.uncaughtException()
> @104 - UncaughtException in thread StreamThread-1, stopping kafka streams
> java.lang.NullPointerException
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:619)
> ~[kafka-streams-0.10.2.0.jar!/:?]
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:368)
> ~[kafka-streams-0.10.2.0.jar!/:?]
> 2017-05-01T00:02:01,076 WARN kafka-streams-close-thread
> org.apache.kafka.streams.processor.internals.StreamThread.setState() @160 -
> Unexpected state transition from NOT_RUNNING to PENDING_SHUTDOWN
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
