jackylee-ch commented on code in PR #12486:
URL: https://github.com/apache/gluten/pull/12486#discussion_r3587653819
##########
gluten-ut/spark40/src/test/scala/org/apache/spark/sql/GlutenVariantSuite.scala:
##########
@@ -16,4 +16,19 @@
*/
package org.apache.spark.sql
-class GlutenVariantSuite extends VariantSuite with GlutenSQLTestsTrait {}
+class GlutenVariantSuite extends VariantSuite with GlutenSQLTestsTrait {
+ testGluten("non-literal variant_get") {
Review Comment:
Since we won't test it in Spark 4.0, can we remote it?
##########
backends-velox/src/main/scala/org/apache/gluten/execution/ColumnarPartialProjectExec.scala:
##########
@@ -297,6 +297,22 @@ object ColumnarPartialProjectExec {
HiveUDFTransformer.isHiveUDF(h) &&
!VeloxHiveUDFTransformer.isSupportedHiveUDF(h)
}
+ /**
+ * Returns true for generic catalyst expressions that Velox cannot offload
and should be pulled
+ * into ColumnarPartialProject, excluding ScalaUDF/HiveUDF and
non-row-evaluable wrappers.
+ */
+ private def isUnmappedGlutenExpr(expr: Expression): Boolean = expr match {
+ case _: ScalaUDF => false
+ case _ if HiveUDFTransformer.isHiveUDF(expr) => false
+ case _: Literal | _: AttributeReference | _: BoundReference => false
+ case _: NamedExpression => false
+ case _ =>
+ expr.deterministic &&
+ !expr.isInstanceOf[Unevaluable] &&
+ !expr.isInstanceOf[LambdaFunction] &&
+ !ExpressionMappings.expressionsMap.contains(expr.getClass)
Review Comment:
do we need to consider the
`ExpressionConverter.tryTransformWithoutExpressionMapping` whole check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
