Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14207#discussion_r71456320
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -252,6 +252,115 @@ class DDLSuite extends QueryTest with
SharedSQLContext with BeforeAndAfterEach {
}
}
+ test("Create data source table with partitioning columns but no schema")
{
+ import testImplicits._
+
+ val tabName = "tab1"
+ withTempPath { dir =>
+ val pathToPartitionedTable = new File(dir, "partitioned")
+ val pathToNonPartitionedTable = new File(dir, "nonPartitioned")
+ val df = sparkContext.parallelize(1 to 10).map(i => (i,
i.toString)).toDF("num", "str")
+
df.write.format("parquet").save(pathToNonPartitionedTable.getCanonicalPath)
+
df.write.format("parquet").partitionBy("num").save(pathToPartitionedTable.getCanonicalPath)
+
+ Seq(pathToPartitionedTable, pathToNonPartitionedTable).foreach {
path =>
+ withTable(tabName) {
+ spark.sql(
+ s"""
+ |CREATE TABLE $tabName
+ |USING parquet
+ |OPTIONS (
+ | path '$path'
+ |)
+ |PARTITIONED BY (inexistentColumns)
+ """.stripMargin)
+ val catalog = spark.sessionState.catalog
+ val tableMetadata =
catalog.getTableMetadata(TableIdentifier(tabName))
+
+ val tableSchema =
DDLUtils.getSchemaFromTableProperties(tableMetadata)
+ assert(tableSchema.nonEmpty, "the schema of data source tables
are always recorded")
+ val partCols =
DDLUtils.getPartitionColumnsFromTableProperties(tableMetadata)
+
+ if (tableMetadata.storage.serdeProperties.get("path") ==
--- End diff --
In this test case, it verifies two scenarios. One is the path to a
partitioned table (i.e., `pathToPartitionedTable`); another is the path to a
non-partitioned table (i.e., `pathToNonPartitionedTable`) . This condition is
just to check which path is being used. If the path points to
`pathToNonPartitionedTable`, it will return false.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
