srielau commented on code in PR #46707: URL: https://github.com/apache/spark/pull/46707#discussion_r1628771638
########## sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala: ########## @@ -3488,6 +3488,35 @@ class DataSourceV2SQLSuiteV1Filter } } + test("SPARK-36680: Supports Dynamic Table Options for Spark SQL") { + val t1 = s"${catalogAndNamespace}table" + withTable(t1) { + sql(s"CREATE TABLE $t1 (id bigint, data string) USING $v2Format") + sql(s"INSERT INTO $t1 VALUES (1, 'a'), (2, 'b')") + + var df = sql(s"SELECT * FROM $t1") + var collected = df.queryExecution.optimizedPlan.collect { + case scan: DataSourceV2ScanRelation => + assert(scan.relation.options.isEmpty) + } + assert (collected.size == 1) + checkAnswer(df, Seq(Row(1, "a"), Row(2, "b"))) + + df = sql(s"SELECT * FROM $t1 WITH (`split-size` = 5)") Review Comment: Pure TVF tableName(named parameterlist), while elegant on one level may be rather confusing to users. There is no more visual cue what is a TVF and what is a table , and of course we would need a tie breaker for whom to resolve for first. This adds unwanted complexity. Adding a builtin Polymorphic TVF uses existing (and standard) language. I would like to know some examples. Teh PR mentions federation (?). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org