viirya commented on code in PR #263:
URL:
https://github.com/apache/arrow-datafusion-comet/pull/263#discussion_r1571822288
##########
spark/src/test/scala/org/apache/spark/sql/CometTestBase.scala:
##########
@@ -718,4 +718,47 @@ abstract class CometTestBase
Seq.empty
}
}
+
+ // tests one liner query without necessity to create external table
+ def testSingleLineQuery(
+ prepareQuery: String,
+ testQuery: String,
+ testName: String = "test",
+ tableName: String = "tbl",
+ excludedOptimizerRules: Option[String] = None): Unit = {
+
+ withTempDir { dir =>
+ val path = new Path(dir.toURI.toString, testName).toUri.toString
+ var data: java.util.List[Row] = new java.util.ArrayList()
+ var schema: StructType = null
+
+ withSQLConf(CometConf.COMET_ENABLED.key -> "false") {
+ val df = spark.sql(prepareQuery)
+ data = df.collectAsList()
+ schema = df.schema
+ }
+
+ spark.createDataFrame(data, schema).repartition(1).write.parquet(path)
+ readParquetFile(path, Some(schema)) { df =>
df.createOrReplaceTempView(tableName) }
+
+ // disable constant folding. This optimization rule precompute and
select value as literal
+ // which subsequently leads to false positives
+ //
+ // ConstantFolding is a operator optimization rule in Catalyst that
replaces expressions
+ // that can be statically evaluated with their equivalent literal values.
Review Comment:
This function doesn't always exclude `ConstantFolding` but passed in ones.
This comment should be moved to where you pass `ConstantFolding` when you call
`testSingleLineQuery` in `CometExpressionCoverageSuite`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
