andygrove commented on code in PR #2877:
URL: https://github.com/apache/datafusion-comet/pull/2877#discussion_r2611146317
##########
spark/src/main/scala/org/apache/comet/rules/CometScanRule.scala:
##########
@@ -160,6 +160,13 @@ case class CometScanRule(session: SparkSession) extends
Rule[SparkPlan] with Com
scanImpl = selectScan(scanExec, r.partitionSchema, hadoopConf)
}
+ // Native DataFusion doesn't support subqueries/dynamic pruning
+ if (scanImpl == SCAN_NATIVE_DATAFUSION &&
+ scanExec.partitionFilters.exists(isDynamicPruningFilter)) {
+ fallbackReasons += "Native DataFusion scan does not support
subqueries/dynamic pruning"
+ return withInfos(scanExec, fallbackReasons.toSet)
Review Comment:
I am going to to follow up with another PR to clean up fallback handling in
this file. I just followed the existing pattern for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
