sabbasani commented on code in PR #12726: URL: https://github.com/apache/gluten/pull/12726#discussion_r4359098757
########## gluten-substrait/src/main/scala/org/apache/gluten/execution/BasicScanExecTransformer.scala: ########## @@ -115,6 +116,9 @@ trait BasicScanExecTransformer extends LeafTransformSupport with BaseDataSource def getInputStats: Option[InputStats] = Option.empty + /** Names of the real columns this scan reads from the source, as opposed to injected ones. */ Review Comment: @jinchengchenghh Thanks for the suggestion! I've moved the logic out of the base class — BasicScanExecTransformer.neededInputFileRelatedMetadataKeys now only returns the input-file keys that appear in output, with no reference to readColumnNames. The filtering against real column names is done via the override in FileSourceScanExecTransformer (line 150–153), which uses requiredSchema.fieldNames.toSet to exclude any output named after an input-file function when that name actually belongs to a real data column. Let me know if you'd prefer the local val readColumnNames inlined directly into the filterNot call instead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
