[
https://issues.apache.org/jira/browse/SPARK-25052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bharath kumar avusherla closed SPARK-25052.
-------------------------------------------
> Is there any possibility that spark structured streaming generate duplicates
> in the output?
> -------------------------------------------------------------------------------------------
>
> Key: SPARK-25052
> URL: https://issues.apache.org/jira/browse/SPARK-25052
> Project: Spark
> Issue Type: Question
> Components: Spark Core
> Affects Versions: 2.3.0
> Reporter: bharath kumar avusherla
> Priority: Minor
>
> We recently observed that the spark structured streaming generated duplicates
> in the output when reading from Kafka topic and storing the output to the S3
> (and checkpointing in S3). We ran into this issue twice. This is not
> reproducible. Is there anyone has ever faced this kind of issue before? Is
> this because of S3 eventual consistency?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
