[
https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629550#comment-16629550
]
Wenchen Fan commented on SPARK-25531:
-------------------------------------
I want to have a more structured view of the data source v2 project. It's a bad
idea to put everything under SPARK-22386, which is so general that only says
it's improvement. I'm starting to create tickets for some big steps of the data
source v2 project, like this one, like the API refactoring, and potentially the
catalog work, the custom metrics, etc. in the future.
For this particular case, the final goal is to design a new write api, for both
data source and end-users, to get rid of SaveMode. "Standardize SQL logical
plans" is how to achieve this goal IMO.
Note that all of them will be marks as "blocks SPARK-25186 Stabilize Data
Source V2 API".
> new write APIs for data source v2
> ---------------------------------
>
> Key: SPARK-25531
> URL: https://issues.apache.org/jira/browse/SPARK-25531
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.5.0
> Reporter: Wenchen Fan
> Priority: Major
>
> The current data source write API heavily depend on {{SaveMode}}, which
> doesn't have a clear semantic, especially when writing to tables.
> We should design a new set of write API without {{SaveMode}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
