cloud-fan commented on code in PR #56211:
URL: https://github.com/apache/spark/pull/56211#discussion_r3327788107
##########
sql/connect/common/src/main/scala/org/apache/spark/sql/connect/Catalog.scala:
##########
@@ -392,7 +392,17 @@ class Catalog(sparkSession: SparkSession) extends
catalog.Catalog {
* @since 3.5.0
*/
override def createTable(tableName: String, path: String): DataFrame = {
- createTable(tableName, path, "parquet")
+ // Leave the source unset so the server resolves
spark.sql.sources.default, as documented
+ // above. Routing through createTable(tableName, path, "parquet") would
hardcode the provider
+ // and ignore that configuration.
+ sparkSession.execute { builder =>
Review Comment:
The two-argument overload inlines the `CreateTable` proto build here,
bypassing the delegation chain every other `createTable` overload uses (each
delegates down to the single 5-arg general overload) and duplicating that
method's `execute{...}` + `table()` block, minus `setSource`.
The behavior is correct and omitting `setSource` is necessary — `source` is
`optional` in the proto, so the server only resolves
`spark.sql.sources.default` when `hasSource()` is false, and delegating to the
5-arg overload would call `setSource("")` and pin the source to a literal empty
string. So this is non-blocking.
That said, the duplication can be removed cleanly by extracting a private
impl taking `source: Option[String]` that both the 2-arg (`None`) and 5-arg
(`Some(source)`) overloads route through, calling `setSource` only when
defined. That keeps a single place building the proto while preserving the
unset-source semantics. Your call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
