dongjoon-hyun commented on PR #59759: URL: https://github.com/apache/spark/pull/59759#issuecomment-6388561382
Thank you for the review and the suggestion, @spark-test and @Sid4030. Since this will be backported to `branch-4.3`, I added an entry to the `Upgrading from Structured Streaming 4.2 to 4.3` section of `docs/streaming/ss-migration-guide.md`, including the recovery steps. I didn't add a compatibility path like `EXCEPT`, because these layouts write data which cannot be read back correctly or which breaks the collation semantics. The only exception is partitioning by all columns with JSON format, which batch writes have rejected since Apache Spark 2.0.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
