[
https://issues.apache.org/jira/browse/SPARK-24189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464781#comment-16464781
]
bharath kumar avusherla commented on SPARK-24189:
-------------------------------------------------
I have tried both ways..
.option("kafka.isolation.level","read_committed") &
.option("kafka.isolation-level","read_committed"). But it is giving me the
following warning and starts executing.
h5. *_WARN ConsumerConfig:186 - The configuration isolation.level =
read_committed was supplied but isn't a known config._*
> Spark Strcutured Streaming not working with the Kafka Transactions
> ------------------------------------------------------------------
>
> Key: SPARK-24189
> URL: https://issues.apache.org/jira/browse/SPARK-24189
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.3.0
> Reporter: bharath kumar avusherla
> Priority: Major
>
> Was trying to read kafka transactional topic using Spark Structured Streaming
> 2.3.0 with the kafka option isolation-level = "read_committed", but spark
> reading the data immediately without waiting for the data in topic to be
> committed. In spark documentation it was mentioned as Structured Streaming
> supports Kafka version 0.10 or higher. I am using below command to test the
> scenario.
> val df = spark
> .readStream
> .format("kafka")
> .option("kafka.bootstrap.servers", "localhost:9092")
> .option("subscribe", "test-topic")
> .option("isolation-level","read_committed")
> .load()
> Can you please let me know if the transactional read is supported in SPark
> 2.3.0 strcutured Streaming or am i missing anything.
>
> Thank you.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
