[
https://issues.apache.org/jira/browse/HIVE-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-4494:
------------------------------
Attachment: HIVE-4494.D10653.1.patch
omalley requested code review of "HIVE-4494 [jira] ORC map columns get class
cast exception in some context".
Reviewers: JIRA
HIVE-4494 fix class cast exception for some queries using complex types in ORC
Setting up the test case like:
create table map_text (
name string,
m map<string,string>
) row format delimited
fields terminated by '|'
collection items terminated by ','
map keys terminated by ':';
create table map_orc (
name string,
m map<string,string>
) stored as orc;
cat map.txt
name1|key11:value11,key12:value12,key13:value13
name2|key21:value21,key22:value22,key23:value23
name3|key31:value31,key32:value32,key33:value33
load data local inpath 'map.txt' into table map_text;
insert overwrite table map_orc select * from map_text;
Selecting the name column from orc_map will get the following exception:
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1195)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException:
org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcMapObjectInspector cannot be cast
to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
at
org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:522)
at
org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
... 22 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcMapObjectInspector cannot be cast
to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:144)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.<init>(ObjectInspectorConverters.java:307)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:270)
at
org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:482)
... 23 more
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D10653
AFFECTED FILES
data/files/orc_create.txt
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java
ql/src/test/queries/clientpositive/orc_create.q
ql/src/test/results/clientpositive/orc_create.q.out
MANAGE HERALD RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/25569/
To: JIRA, omalley
> ORC map columns get class cast exception in some context
> --------------------------------------------------------
>
> Key: HIVE-4494
> URL: https://issues.apache.org/jira/browse/HIVE-4494
> Project: Hive
> Issue Type: Bug
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: HIVE-4494.D10653.1.patch
>
>
> Setting up the test case like:
> {quote}
> create table map_text (
> name string,
> m map<string,string>
> ) row format delimited
> fields terminated by '|'
> collection items terminated by ','
> map keys terminated by ':';
> create table map_orc (
> name string,
> m map<string,string>
> ) stored as orc;
> cat map.txt
> name1|key11:value11,key12:value12,key13:value13
> name2|key21:value21,key22:value22,key23:value23
> name3|key31:value31,key32:value32,key33:value33
> load data local inpath 'map.txt' into table map_text;
> insert overwrite table map_orc select * from map_text;
> {quote}
> Selecting the name column from orc_map will get the following exception:
> {quote}
> java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1195)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 9 more
> Caused by: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ... 14 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 17 more
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 22 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcMapObjectInspector cannot be
> cast to
> org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:522)
> at
> org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
> ... 22 more
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcMapObjectInspector cannot be
> cast to
> org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:144)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.<init>(ObjectInspectorConverters.java:307)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:270)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:482)
> ... 23 more
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
