Re: Exception in thread "main" org.apache.spark.sql.streaming.StreamingQueryException: Not authorized to access group: spark-kafka-source-060f3ceb-09f4-4e28-8210-3ef8a845fc92--2038748645-driver-2

2019-02-13 Thread Jungtaek Lim
Adding to Gabor's answer, in Spark 3.0 end users can even provide full of group id (Please refer SPARK-26350 [1]), but you may feel more convenient to use prefix of group id Gabor guided (Please refer SPARK-26121 [2]) to provide permission to broader ranges of groups. 1.

Re: Exception in thread "main" org.apache.spark.sql.streaming.StreamingQueryException: Not authorized to access group: spark-kafka-source-060f3ceb-09f4-4e28-8210-3ef8a845fc92--2038748645-driver-2

2019-02-13 Thread Gabor Somogyi
Hi Thomas, The issue occurs when the user does not have the READ permission on the consumer groups. In DStreams group ID is configured in application, for example:

Exception in thread "main" org.apache.spark.sql.streaming.StreamingQueryException: Not authorized to access group: spark-kafka-source-060f3ceb-09f4-4e28-8210-3ef8a845fc92--2038748645-driver-2

2019-02-12 Thread Allu Thomas
Hi There, My use case is to read a simple json message from Kafka queue using Spark Structured Streaming. But I’m getting the following error message when I run my Kafka consumer. I don’t get this error when using Spark direct stream. The issue is happening only with structured streaming. Any