Michael Haeusler created HIVE-8870:
--------------------------------------
Summary: errors when selecting a struct field within an array from
ORC based tables
Key: HIVE-8870
URL: https://issues.apache.org/jira/browse/HIVE-8870
Project: Hive
Issue Type: Bug
Components: File Formats, Query Processor
Affects Versions: 0.13.0, 0.14.0
Environment: HDP 2.1 / HDP 2.2 (YARN, but no Tez)
Reporter: Michael Haeusler
When using ORC as storage for a table, we get errors on selecting a struct
field within an array. These errors do not appear with default format.
{code:sql}
CREATE TABLE `foobar_orc`(
`uid` bigint,
`elements` array<struct<elementid:bigint,foo:struct<bar:string>>>)
STORED AS ORC;
{code}
When selecting from this _empty_ table, we get a direct NPE within the Hive CLI:
{code:sql}
SELECT
elements.elementId
FROM
foobar_orc;
-- FAILED: RuntimeException java.lang.NullPointerException
{code}
A more real-world query produces a RuntimeException / NullPointerException in
the mapper:
{code:sql}
SELECT
uid,
element.elementId
FROM
foobar_orc
LATERAL VIEW
EXPLODE(elements) e AS element;
Error: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
[...]
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator.initialize(ExprNodeFieldEvaluator.java:61)
[...]
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
{code}
Both queries run fine on a non-orc table:
{code:sql}
CREATE TABLE `foobar`(
`uid` bigint,
`elements` array<struct<elementid:bigint,foo:struct<bar:string>>>);
SELECT
elements.elementId
FROM
foobar;
-- OK
-- Time taken: 0.225 seconds
SELECT
uid,
element.elementId
FROM
foobar
LATERAL VIEW
EXPLODE(elements) e AS element;
-- Total MapReduce CPU Time Spent: 1 seconds 920 msec
-- OK
-- Time taken: 25.905 seconds
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
