[ https://issues.apache.org/jira/browse/SPARK-38717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-38717: ---------------------------------- Fix Version/s: 3.3.2 (was: 3.3.1) > Handle Hive's bucket spec case preserving behaviour > --------------------------------------------------- > > Key: SPARK-38717 > URL: https://issues.apache.org/jira/browse/SPARK-38717 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.3.0 > Reporter: Peter Toth > Assignee: Peter Toth > Priority: Major > Fix For: 3.4.0, 3.3.2 > > > {code} > CREATE TABLE t( > c STRING, > B_C STRING > ) > PARTITIONED BY (p_c STRING) > CLUSTERED BY (B_C) INTO 4 BUCKETS > STORED AS PARQUET > {code} > then > {code} > SELECT * FROM t > {code} > fails with: > {code} > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Bucket columns > B_C is not part of the table columns ([FieldSchema(name:c, type:string, > comment:null), FieldSchema(name:b_c, type:string, comment:null)] > at > org.apache.hadoop.hive.ql.metadata.Table.setBucketCols(Table.java:552) > at > org.apache.spark.sql.hive.client.HiveClientImpl$.toHiveTable(HiveClientImpl.scala:1098) > at > org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$getPartitionsByFilter$1(HiveClientImpl.scala:764) > at > org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$withHiveState$1(HiveClientImpl.scala:294) > at > org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:225) > at > org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:224) > at > org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:274) > at > org.apache.spark.sql.hive.client.HiveClientImpl.getPartitionsByFilter(HiveClientImpl.scala:763) > at > org.apache.spark.sql.hive.HiveExternalCatalog.$anonfun$listPartitionsByFilter$1(HiveExternalCatalog.scala:1287) > at > org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:101) > ... 110 more > {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org