This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new b13dc699f7d [SPARK-41990][SQL][FOLLOWUP] Add comments to explain why `FieldReference.column` is used when `ParseException` happens b13dc699f7d is described below commit b13dc699f7df589663b1239e5e551815d9043f38 Author: huaxingao <huaxin_...@apple.com> AuthorDate: Mon Jan 16 16:52:46 2023 +0800 [SPARK-41990][SQL][FOLLOWUP] Add comments to explain why `FieldReference.column` is used when `ParseException` happens ### What changes were proposed in this pull request? Add comments to explain why `FieldReference.column` is used when `ParseException` happens ### Why are the changes needed? To address this [comment](https://github.com/apache/spark/pull/39564#discussion_r1070788648) ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Only comments are added. No need to test. Closes #39597 from huaxingao/followup. Authored-by: huaxingao <huaxin_...@apple.com> Signed-off-by: Wenchen Fan <wenc...@databricks.com> --- .../src/main/scala/org/apache/spark/sql/sources/filters.scala | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala index 080d17b47fb..af5e4f5ef5a 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala @@ -80,6 +80,12 @@ sealed abstract class Filter { try { FieldReference(attribute) } catch { + // The column name in V1 Filter is generated by PushableColumn. When nested + // predicate push down is enabled, PushableColumn quotes the column name. + // However, when nested predicate push down is disabled, PushableColumn + // does not quote the column. If the column name is not a valid SQL identifier, + // FieldReference.apply throws ParseException. We catch the ParseException here + // and use FieldReference.column instead. case _: ParseException => FieldReference.column(attribute) } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org