felipepessoto commented on code in PR #12215:
URL: https://github.com/apache/gluten/pull/12215#discussion_r3634364925
##########
backends-velox/src-delta/main/scala/org/apache/gluten/config/VeloxDeltaConfig.scala:
##########
@@ -22,6 +22,8 @@ class VeloxDeltaConfig(conf: SQLConf) extends
GlutenCoreConfig(conf) {
import VeloxDeltaConfig._
def enableNativeWrite: Boolean = getConf(ENABLE_NATIVE_WRITE)
+
+ def enableNativeDmlRowIndexScan: Boolean =
getConf(ENABLE_NATIVE_DML_ROW_INDEX_SCAN)
Review Comment:
Is it the convention to have a function for all configs even if not used?
##########
gluten-delta/src/main/scala/org/apache/gluten/extension/DeltaPostTransformRules.scala:
##########
@@ -90,6 +99,25 @@ object DeltaPostTransformRules {
child.copy(output = p.output)
}
+ /**
+ * Native DELETE/UPDATE/MERGE DV support can deliberately keep the target
row-index scan in Spark.
+ * Keep only the contiguous scan-adjacent filter/project chain in Spark,
avoiding an isolated
+ * native island without propagating fallback through joins, exchanges, or
aggregations. The
+ * BitmapAggregator and the rest of the DML plan remain eligible for native
execution.
+ */
+ val keepDmlRowIndexFallbackSubtreeOnSpark: Rule[SparkPlan] = (plan:
SparkPlan) =>
Review Comment:
Do we need the same `if
(!plan.exists(_.isInstanceOf[DeltaScanTransformer]))` that `deltaSpecificRules`
uses?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
