Hi There,
My use case is to read a simple json message from Kafka queue using Spark
Structured Streaming. But I’m getting the following error message when I run
my Kafka consumer. I don’t get this error when using Spark direct stream. The
issue is happening only with structured streaming. Any help would be greatly
appreciated.
Exception in thread "main"
org.apache.spark.sql.streaming.StreamingQueryException: Not authorized to
access group:
spark-kafka-source-060f3ceb-09f4-4e28-8210-3ef8a845fc92--2038748645-driver-2
=== Streaming Query ===
Identifier: [id = 6ab10eab-4f71-435c-8705-820e66cee47e, runId =
48430367-9e14-450b-b8e0-27199b536403]
Current Committed Offsets: {}
Current Available Offsets: {}
Current State: ACTIVE
Thread State: RUNNABLE
Logical Plan:
KafkaSource[Subscribe[cla-claim-raw]]
at org.apache.spark.sql.execution.streaming.StreamExecution.org
<http://xecution.org/>$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:295)
at
org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:189)
Caused by: org.apache.kafka.common.errors.GroupAuthorizationException: Not
authorized to access group:
spark-kafka-source-060f3ceb-09f4-4e28-8210-3ef8a845fc92--2038748645-driver-2
Thanks,
Thomas Thomas
