Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22009#discussion_r224004348
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
---
@@ -169,15 +174,16 @@ object DataSourceV2Relation {
options: Map[String, String],
tableIdent: Option[TableIdentifier] = None,
userSpecifiedSchema: Option[StructType] = None):
DataSourceV2Relation = {
- val reader = source.createReader(options, userSpecifiedSchema)
+ val readSupport = source.createReadSupport(options,
userSpecifiedSchema)
--- End diff --
This is kind of a behavior change. Now when we append data to a data
source, the data source must be readable, to provide a schema, which will be
used to validate the input data schema.
I don't have a strong feeling. Data source v2 is marked as involving so
necessary behavior change is fine. cc @rdblue
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
