[jira] [Created] (HIVE-8870) errors when selecting a struct field within an array from ORC based tables

[Michael Haeusler (JIRA)]

Michael Haeusler created HIVE-8870:
--------------------------------------

             Summary: errors when selecting a struct field within an array from 
ORC based tables
                 Key: HIVE-8870
                 URL: https://issues.apache.org/jira/browse/HIVE-8870
             Project: Hive
          Issue Type: Bug
          Components: File Formats, Query Processor
    Affects Versions: 0.13.0, 0.14.0
         Environment: HDP 2.1 / HDP 2.2 (YARN, but no Tez)
            Reporter: Michael Haeusler


When using ORC as storage for a table, we get errors on selecting a struct 
field within an array. These errors do not appear with default format.

{code:sql}
CREATE  TABLE `foobar_orc`(
  `uid` bigint,
  `elements` array<struct<elementid:bigint,foo:struct<bar:string>>>)
STORED AS ORC;
{code}

When selecting from this _empty_ table, we get a direct NPE within the Hive CLI:
{code:sql}
SELECT
  elements.elementId
FROM
  foobar_orc;
-- FAILED: RuntimeException java.lang.NullPointerException
{code}

A more real-world query produces a RuntimeException / NullPointerException in 
the mapper:
{code:sql}
SELECT
  uid,
  element.elementId
FROM
  foobar_orc
LATERAL VIEW
  EXPLODE(elements) e AS element;


Error: java.lang.RuntimeException: Error in configuring object
        at 
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
[...]
Caused by: java.lang.NullPointerException
        at 
org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator.initialize(ExprNodeFieldEvaluator.java:61)
[...]
FAILED: Execution Error, return code 2 from 
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
{code}

Both queries run fine on a non-orc table:
{code:sql}
CREATE  TABLE `foobar`(
  `uid` bigint,
  `elements` array<struct<elementid:bigint,foo:struct<bar:string>>>);  

SELECT
  elements.elementId
FROM
  foobar;
-- OK
-- Time taken: 0.225 seconds

SELECT
  uid,
  element.elementId
FROM
  foobar
LATERAL VIEW
  EXPLODE(elements) e AS element;
-- Total MapReduce CPU Time Spent: 1 seconds 920 msec
-- OK
-- Time taken: 25.905 seconds
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)