Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/23208
@cloud-fan, I see that this adds `Table` and uses `TableProvider`, but I
was expecting this to also update the write side to mirror the read side, like
PR #22190 for [SPARK-25188](https://issues.apache.org/jira/browse/SPARK-25188)
(originally proposed in [discussion on
SPARK-24882](https://issues.apache.org/jira/browse/SPARK-24882?focusedCommentId=16581725&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16581725)).
The main parts that we discussed there were:
* Mirror the read side structure by adding WriteConfig. Now, that would be
adding a WriteBuilder.
* Mirroring the read life-cycle of ScanBuilder and Scan, to enable use
cases like acquiring and holding a write lock, for example.
* Using the WriteBuilder to expose more write configuration to support
overwrite and dynamic partition overwrite.
We don't need to add the overwrite mix-ins here, but I would expect to see
a WriteBuilder that returns a Writer. (`Table -> WriteBuilder -> Write` matches
`Table -> ScanBulder -> Scan`.)
The Write would expose BatchWrite and StreamWrite (if they are different)
or could directly expose the WriteFactory, commit, abort, etc.
WriteBuilder would be extensible so that SupportsOverwrite and
SupportsDynamicOverwrite can be added as mix-ins at some point.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
