akshaytayal commented on code in PR #13519:
URL: https://github.com/apache/gluten/pull/13519#discussion_r4505765347


##########
gluten-substrait/src/main/scala/org/apache/gluten/extension/columnar/validator/Validators.scala:
##########
@@ -165,6 +170,16 @@ object Validators {
     }
   }
 
+  private object FallbackByUnsupportedSparkFeatures extends Validator {
+    override def validate(plan: SparkPlan): Validator.OutCome = plan match {
+      // The scan reports `SinglePartition` and Spark may have elided a 
shuffle above it, while the
+      // native scan doesn't coalesce its splits into a single task.
+      case p: FileSourceScanExec if 
SparkShimLoader.getSparkShims.isSingleTaskExecutionScan(p) =>
+        fail(s"${p.nodeName} uses single-task execution, which is not 
supported")
+      case _ => pass()

Review Comment:
   Fixed in 2d1e740: a `LocalTableScanExec` that reports a single partition 
(Spark 4.3 single-task execution) now falls back in 
`FallbackByUnsupportedSparkFeatures`, the same as single-task file scans.
   
   Posted by an AI assistant. Flag anything wrong here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to