[
https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715953#comment-16715953
]
ASF GitHub Bot commented on SPARK-25530:
----------------------------------------
cloud-fan commented on a change in pull request #23208: [SPARK-25530][SQL] data
source v2 API refactor (batch write)
URL: https://github.com/apache/spark/pull/23208#discussion_r240448233
##########
File path:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/TableProvider.java
##########
@@ -25,7 +25,10 @@
* The base interface for v2 data sources which don't have a real catalog.
Implementations must
* have a public, 0-arg constructor.
* <p>
- * The major responsibility of this interface is to return a {@link Table} for
read/write.
+ * The major responsibility of this interface is to return a {@link Table} for
read/write. If you
+ * want to allow end-users to write data to non-existing tables via write APIs
in `DataFrameWriter`
+ * with `SaveMode`, you must return a {@link Table} instance even if the table
doesn't exist. The
+ * table schema can be empty in this case.
Review comment:
what about the file source behavior difference between `SaveMode.Append` and
the new append operator? Are you saying we should accept it and ask users to
change their code? file source is widely used with `df.write.save` API...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> data source v2 API refactor (batch write)
> -----------------------------------------
>
> Key: SPARK-25530
> URL: https://issues.apache.org/jira/browse/SPARK-25530
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Wenchen Fan
> Priority: Major
>
> Adjust the batch write API to match the read API after refactor
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
