mbutrovich commented on code in PR #4752:
URL: https://github.com/apache/datafusion-comet/pull/4752#discussion_r3692236033
##########
spark/src/main/scala/org/apache/comet/rules/CometScanRule.scala:
##########
@@ -302,6 +297,11 @@ case class CometScanRule(session: SparkSession)
scanExec.scan match {
case scan: CSVScan if COMET_CSV_V2_NATIVE_ENABLED.get() =>
+ if (scanExec.output.exists(_.isMetadataCol)) {
+ return withFallbackReason(
+ scanExec,
+ "Metadata columns are not supported for CSV V2 scans")
+ }
Review Comment:
Not asking for a change yet, but is this reachable? There's no test for it,
and it's not clear to me if Spark can request metadata columns from CSV scans.
I think it might be dead code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
