rdblue commented on a change in pull request #23266: [SPARK-26313][SQL] move read related methods from Table to read related mix-in traits URL: https://github.com/apache/spark/pull/23266#discussion_r240726681
########## File path: sql/core/src/main/java/org/apache/spark/sql/sources/v2/SupportsBatchRead.java ########## @@ -20,14 +20,27 @@ import org.apache.spark.annotation.Evolving; import org.apache.spark.sql.sources.v2.reader.Scan; import org.apache.spark.sql.sources.v2.reader.ScanBuilder; +import org.apache.spark.sql.types.StructType; /** - * An empty mix-in interface for {@link Table}, to indicate this table supports batch scan. - * <p> - * If a {@link Table} implements this interface, its {@link Table#newScanBuilder(DataSourceOptions)} - * must return a {@link ScanBuilder} that builds {@link Scan} with {@link Scan#toBatch()} - * implemented. - * </p> + * A mix-in interface for {@link Table} to provide data reading ability of batch processing. */ @Evolving -public interface SupportsBatchRead extends Table { } +public interface SupportsBatchRead extends Table { + + /** + * Returns the schema of this table. + */ + StructType schema(); Review comment: I think that the use case for a table with no schema is extremely narrow, if not laughably unlikely. Silly use cases should not cause us to make changes to an API when there is a reasonable alternative: a source with no schema should return an empty schema and signal that it wants to disable write validation (using a capability). There are two advantages to this approach: 1. Implementations need to implement `schema` and validation rules are enabled by default. If an implementation needs to remember to add a `HasSchema` interface to turn on validation rules, then the default is wrong. The result would be that people less familiar with the API will not have validated writes, and that's a problem. 2. The API is simpler and easier to understand. I'm -1 on moving this from `Table`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org