0lai0 commented on code in PR #5614:
URL: https://github.com/apache/datafusion-comet/pull/5614#discussion_r3914681406
##########
spark/src/test/scala/org/apache/comet/CometCodegenSuite.scala:
##########
@@ -210,6 +210,63 @@ class CometCodegenSuite
}
}
+ private def withSequenceTable(f: => Unit): Unit = {
+ withTable("t") {
+ sql("CREATE TABLE t (a INT, b INT, d DATE) USING parquet")
+ sql("INSERT INTO t VALUES (1, 5, DATE'2024-01-01'), (9, 2,
DATE'2024-03-01')")
+ withSQLConf(
+ CometConf.COMET_SCALA_UDF_CODEGEN_ENABLED.key -> "true",
+ CometConf.COMET_EXPLAIN_CODEGEN_ENABLED.key -> "true",
+ CometConf.COMET_EXEC_PROJECT_ENABLED.key -> "true",
+ CometConf.COMET_EXTENDED_EXPLAIN_FORMAT.key ->
+ CometConf.COMET_EXTENDED_EXPLAIN_FORMAT_VERBOSE)(f)
+ }
+ }
+
+ test("sequence with leaf integral args runs natively") {
+ // Integral sequence with column-reference/literal args lowers to the
native spark_sequence
+ // kernel; no codegen-dispatch marker should appear.
+ withSequenceTable {
+ val df = sql("SELECT sequence(a, b), sequence(a, b, 2) FROM t")
Review Comment:
Thanks @sunchao, both P2 items addressed in the latest push.
1. Native gate is now Literal | Attribute | BoundReference only
(argsAreLiteralsOrRefs). Nested calls, CASE WHEN, and zero-arg UDFs fall back
to the dispatcher.
2. Fixture uses sign-correct stp so sequence(a, b, stp) is legal on both
rows.
Added sequence with zero-arg UDF stop routes through the dispatcher
(comet_seq_stopper()).
Null-short-circuit / CASE cases stay in sequence.sql.
Local validation:
CometCodegenSuite: 178/178 pass (4 sequence tests, incl. zero-arg UDF →
dispatcher)
CometSqlFileTestSuite: expressions/array/sequence.sql + sequence_ansi.sql
pass
cargo test -p datafusion-comet-spark-expr array_funcs::sequence: 5/5 pass
test-compile green on -Pspark-3.4, -Pspark-3.5, -Pspark-4.0 (shim
SequenceBatchTooLarge)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
