[GitHub] spark pull request #20454: [SPARK-23202][SQL] Add new API in DataSourceWrite...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20454 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20454: [SPARK-23202][SQL] Add new API in DataSourceWrite...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20454#discussion_r165245483
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceWriter.java
---
@@ -62,6 +62,15 @@
*/
DataWriterFactory createWriterFactory();
+ /**
+ * Handles a commit message on receiving from a successful data writer.
+ *
+ * If this method fails (by throwing an exception), this writing job is
considered to to have been
+ * failed, and {@link #abort(WriterCommitMessage[])} would be called.
The state of the destination
+ * is undefined and {@link #abort(WriterCommitMessage[])} may not be
able to deal with it.
--- End diff --
makes sense, let's remove the last sentence.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20454: [SPARK-23202][SQL] Add new API in DataSourceWrite...
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/20454#discussion_r165141464
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceWriter.java
---
@@ -62,6 +62,15 @@
*/
DataWriterFactory createWriterFactory();
+ /**
+ * Handles a commit message on receiving from a successful data writer.
+ *
+ * If this method fails (by throwing an exception), this writing job is
considered to to have been
+ * failed, and {@link #abort(WriterCommitMessage[])} would be called.
The state of the destination
+ * is undefined and {@link #abort(WriterCommitMessage[])} may not be
able to deal with it.
--- End diff --
What does it mean that "the state of the destination is undefined"? I think
it is sufficient to say that `abort` will be called and the contract for
aborting commits applies.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
