[ https://issues.apache.org/jira/browse/SPARK-34321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276865#comment-17276865 ]
L. C. Hsieh commented on SPARK-34321: ------------------------------------- Err...I made a mistake when reading the document and code. This is invalid. > Fix the guarantee of foreachBatch > --------------------------------- > > Key: SPARK-34321 > URL: https://issues.apache.org/jira/browse/SPARK-34321 > Project: Spark > Issue Type: Documentation > Components: Structured Streaming > Affects Versions: 3.2.0 > Reporter: L. C. Hsieh > Assignee: L. C. Hsieh > Priority: Major > > Similar to SPARK-28650, {{foreachBatch}} API document also documents the > guarantee: > The batchId can be used to deduplicate and transactionally write the output > (that is, the provided Dataset) to external systems. The output Dataset is > guaranteed to be exactly the same for the same batchId > But like the reason of fixing the document of {{ForeachWriter}} in > SPARK-28650, it is not hard to break the guarantee by changing the partition > number. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org