yihua commented on code in PR #19132:
URL: https://github.com/apache/hudi/pull/19132#discussion_r3670334841
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/dml/others/TestTimeTravelTable.scala:
##########
@@ -338,4 +338,40 @@ class TestTimeTravelTable extends HoodieSparkSqlTestBase {
}
})
}
+
+ test("Test time travel with SQL:2011 temporal clause spellings") {
Review Comment:
Addressed
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/dml/others/TestTimeTravelTable.scala:
##########
@@ -374,4 +374,39 @@ class TestTimeTravelTable extends HoodieSparkSqlTestBase {
}
}
}
+
+ test("Test time travel validation errors") {
+ withTempDir { tmp =>
+ val tableName = generateTableName
+ spark.sql(
+ s"""
+ |create table $tableName (
+ | id int,
+ | name string,
+ | price double,
+ | ts long
+ |) using hudi
+ | tblproperties (
+ | primaryKey = 'id',
+ | preCombineField = 'ts'
+ | )
+ | location '${tmp.getCanonicalPath}/$tableName'
+ """.stripMargin)
+ spark.sql(s"insert into $tableName values(1, 'a1', 10, 1000)")
+
+ // VERSION AS OF (a version with no timestamp) is rejected by the Hudi
analysis rule, which
+ // does not support version-based time travel.
+ checkExceptionContain(s"select * from $tableName VERSION AS OF 1")(
+ "Version expression is not supported for time travel")
+
+ // A timestamp expression containing a subquery is rejected because time
travel needs a
+ // constant instant. Which layer rejects it is version-dependent: Spark
3.3's native parser
+ // fails at parse time ("Invalid time travel spec: timestamp expression
cannot contain
+ // subqueries"), while Spark 3.4+ defer to the Hudi analysis rule
("timestamp expression
+ // cannot refer to any columns or contain subqueries"). Assert on the
shared substring.
+ checkExceptionContain(
+ s"select * from $tableName TIMESTAMP AS OF (select max(ts) from
$tableName)")(
+ "contain subqueries")
Review Comment:
Remove redundant comments. For the second validation, use conditions based
on Spark version (e.g. `if (HoodieSparkUtils.isSpark3_3)`) to check the full
exception message instead of "contain subqueries" which is vague.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
