By default, the column pruner is set to true: hive.optimize.cp = true. However, when we set the hive.optimize.cp = false, then some Hive queries started to fail.
We get errors like: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row [Error getting row data with exception java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct at org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:55) at org.apache.hadoop.hive.serde2.SerDeUtils.buildJSONString(SerDeUtils.java:349) Any insight what is going on here? P.S. The reason we want to set it to false is that some Hive tables have data on S3. And seems we have to set column pruner to false to make S3 queries to work. However, by setting it to false, the HDFS queries failed. Thanks!