L. C. Hsieh created SPARK-34321: ----------------------------------- Summary: Fix the guarantee of foreachBatch Key: SPARK-34321 URL: https://issues.apache.org/jira/browse/SPARK-34321 Project: Spark Issue Type: Documentation Components: Structured Streaming Affects Versions: 3.2.0 Reporter: L. C. Hsieh Assignee: L. C. Hsieh
Similar to SPARK-28650, {{foreachBatch}} API document also documents the guarantee: The batchId can be used to deduplicate and transactionally write the output (that is, the provided Dataset) to external systems. The output Dataset is guaranteed to be exactly the same for the same batchId But like the reason of fixing the document of {{ForeachWriter}} in SPARK-28650, it is not hard to break the guarantee by changing the partition number. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org