[
https://issues.apache.org/jira/browse/HIVE-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13650248#comment-13650248
]
Phabricator commented on HIVE-4494:
-----------------------------------
pamelavagata has commented on the revision "HIVE-4494 [jira] ORC map columns
get class cast exception in some context".
Owen, I had previously investigated this issue and found that the reason for
the ClassCastException was due to the following reason:
in ObjectInspectorConverters.getConverter(ObjectInspector inputOI,
ObjectInspector outputOI),
there's a referential equality check to see if the inputOI is the same as the
outputOI and returns a new Identity Converter. When running the same query with
RC, you'll find that ColumnarSerde caches the the object inspectors and reuses
them, which is why the referential equality check will pass and return an
IdentityConverter instead of trying to cast to a Settable*ObjectInspector.
If you cache the object inspectors in OrcStruct instead of having
OrcList/OrcMap object inspectors inherit from Settable*ObjectInspector, it will
fix the underlying issue. I hadn't gotten around to submitting a diff for this
yet but would be happy to push one out in the next 2 days.
REVISION DETAIL
https://reviews.facebook.net/D10653
To: JIRA, omalley
Cc: pamelavagata
> ORC map columns get class cast exception in some context
> --------------------------------------------------------
>
> Key: HIVE-4494
> URL: https://issues.apache.org/jira/browse/HIVE-4494
> Project: Hive
> Issue Type: Bug
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: HIVE-4494.D10653.1.patch
>
>
> Setting up the test case like:
> {quote}
> create table map_text (
> name string,
> m map<string,string>
> ) row format delimited
> fields terminated by '|'
> collection items terminated by ','
> map keys terminated by ':';
> create table map_orc (
> name string,
> m map<string,string>
> ) stored as orc;
> cat map.txt
> name1|key11:value11,key12:value12,key13:value13
> name2|key21:value21,key22:value22,key23:value23
> name3|key31:value31,key32:value32,key33:value33
> load data local inpath 'map.txt' into table map_text;
> insert overwrite table map_orc select * from map_text;
> {quote}
> Selecting the name column from orc_map will get the following exception:
> {quote}
> java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1195)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 9 more
> Caused by: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ... 14 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 17 more
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 22 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcMapObjectInspector cannot be
> cast to
> org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:522)
> at
> org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
> ... 22 more
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.ql.io.orc.OrcStruct$OrcMapObjectInspector cannot be
> cast to
> org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:144)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.<init>(ObjectInspectorConverters.java:307)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:270)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:482)
> ... 23 more
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
