[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruce Robbins resolved SPARK-33098. ----------------------------------- Resolution: Duplicate > Explicit or implicit casts involving partition columns can sometimes result > in a MetaException. > ----------------------------------------------------------------------------------------------- > > Key: SPARK-33098 > URL: https://issues.apache.org/jira/browse/SPARK-33098 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.1.0 > Reporter: Bruce Robbins > Priority: Major > > The following cases throw > {{MetaException(message:Filtering is supported only on partition keys of type > string)}} > {noformat} > sql("create table test (a int) partitioned by (b int) stored as parquet") > sql("insert into test values (1, 1), (1, 2), (2, 2)") > // These throw MetaExceptions > sql("select * from test where b in ('2')").show(false) > sql("select * from test where cast(b as string) = '2'").show(false) > sql("select * from test where cast(b as string) in ('2')").show(false) > sql("select * from test where cast(b as string) in (2)").show(false) > sql("select cast(b as string) as b from test where b in ('2')").show(false) > sql("select cast(b as string) as b from test").filter("b = '2'").show(false) > // [1] > sql("select cast(b as string) as b from test").filter("b in (2)").show(false) > // [2] > sql("select cast(b as string) as b from test").filter("b in > ('2')").show(false) > sql("select * from test where cast(b as string) > '1'").show(false) > sql("select cast(b as string) b from test").filter("b > '1'").show(false) // > [3] > // [1] but not sql("select cast(b as string) as b from test where b = > '2'").show(false) > // [2] but not sql("select cast(b as string) as b from test where b in > (2)").show(false) > // [3] but not sql("select cast(b as string) b from test where b > > '1'").show(false) > {noformat} > The message ("Filtering is supported only on partition keys of type string") > is misleading. Filter *is* supported on integer columns, for example. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org