Michael Haeusler created HIVE-8870: -------------------------------------- Summary: errors when selecting a struct field within an array from ORC based tables Key: HIVE-8870 URL: https://issues.apache.org/jira/browse/HIVE-8870 Project: Hive Issue Type: Bug Components: File Formats, Query Processor Affects Versions: 0.13.0, 0.14.0 Environment: HDP 2.1 / HDP 2.2 (YARN, but no Tez) Reporter: Michael Haeusler
When using ORC as storage for a table, we get errors on selecting a struct field within an array. These errors do not appear with default format. {code:sql} CREATE TABLE `foobar_orc`( `uid` bigint, `elements` array<struct<elementid:bigint,foo:struct<bar:string>>>) STORED AS ORC; {code} When selecting from this _empty_ table, we get a direct NPE within the Hive CLI: {code:sql} SELECT elements.elementId FROM foobar_orc; -- FAILED: RuntimeException java.lang.NullPointerException {code} A more real-world query produces a RuntimeException / NullPointerException in the mapper: {code:sql} SELECT uid, element.elementId FROM foobar_orc LATERAL VIEW EXPLODE(elements) e AS element; Error: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) [...] Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator.initialize(ExprNodeFieldEvaluator.java:61) [...] FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask {code} Both queries run fine on a non-orc table: {code:sql} CREATE TABLE `foobar`( `uid` bigint, `elements` array<struct<elementid:bigint,foo:struct<bar:string>>>); SELECT elements.elementId FROM foobar; -- OK -- Time taken: 0.225 seconds SELECT uid, element.elementId FROM foobar LATERAL VIEW EXPLODE(elements) e AS element; -- Total MapReduce CPU Time Spent: 1 seconds 920 msec -- OK -- Time taken: 25.905 seconds {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)