fbocse commented on a change in pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2 (WIP) URL: https://github.com/apache/spark/pull/28027#discussion_r398901106
########## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala ########## @@ -48,6 +48,15 @@ case class DataSourceV2Relation( import DataSourceV2Implicits._ + override lazy val metadataOutput: Seq[AttributeReference] = table match { + case hasMeta: SupportsMetadataColumns => + val attrs = hasMeta.metadataColumns + val outputNames = outputSet.map(_.name).toSet + attrs.filterNot(col => outputNames.contains(col.name)).toAttributes Review comment: Can this possibly be buggy because of case-sensitivity playing a trick when filtering by Set.contains or column names are somehow "normalized" wrt to case? ```suggestion val outputNames = outputSet.map(_.name.toLowerCase()).toSet attrs.filterNot(col => outputNames.contains(col.name.toLowerCase())).toAttributes ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org