[ https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258037#comment-13258037 ]
Hudson commented on HIVE-2958: ------------------------------ Integrated in Hive-trunk-h0.21 #1384 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1384/]) HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] (Navis Ryu via Ashutosh Chauhan) Summary: DPAL-1111 GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] This relates to https://issues.apache.org/jira/browse/HIVE-1634. The following work fine: CREATE EXTERNAL TABLE tim_hbase_occurrence ( id int, scientific_name string, data_resource_id int ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ( "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b" ) TBLPROPERTIES( "hbase.table.name" = "mini_occurrences", "hbase.table.default.storage.type" = "binary" ); SELECT * FROM tim_hbase_occurrence LIMIT 3; SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3; However, the following fails: SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY data_resource_id; The error given: 0 TS 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 7 MAP 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias tim_hbase_occurrence for file hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 forwarding 1 rows 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1 rows 2012-04-17 16:58:45,723 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"id":1444,"scientific_name":null,"data_resource_id":1081} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529) ... 9 more Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger at org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142) at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119) at org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750) at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722) ... 18 more Test Plan: EMPTY Reviewers: JIRA, ashutoshc Reviewed By: ashutoshc Differential Revision: https://reviews.facebook.net/D2871 (Revision 1328157) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328157 Files : * /hive/trunk/hbase-handler/src/test/queries/hbase_binary_external_table_queries.q * /hive/trunk/hbase-handler/src/test/results/hbase_binary_external_table_queries.q.out * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioBoolean.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioByte.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioDouble.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioFloat.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioInteger.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioLong.java * /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioShort.java > GROUP BY causing ClassCastException [LazyDioInteger cannot be cast > LazyInteger] > ------------------------------------------------------------------------------- > > Key: HIVE-2958 > URL: https://issues.apache.org/jira/browse/HIVE-2958 > Project: Hive > Issue Type: Bug > Components: HBase Handler > Affects Versions: 0.9.0 > Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today > Reporter: Tim Robertson > Assignee: Navis > Priority: Blocker > Fix For: 0.9.0 > > Attachments: HIVE-2958.D2871.1.patch > > > This relates to https://issues.apache.org/jira/browse/HIVE-1634. > The following work fine: > {code} > CREATE EXTERNAL TABLE tim_hbase_occurrence ( > id int, > scientific_name string, > data_resource_id int > ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH > SERDEPROPERTIES ( > "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b" > ) TBLPROPERTIES( > "hbase.table.name" = "mini_occurrences", > "hbase.table.default.storage.type" = "binary" > ); > SELECT * FROM tim_hbase_occurrence LIMIT 3; > SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3; > {code} > However, the following fails: > {code} > SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY > data_resource_id; > {code} > The error given: > {code} > 0 TS > 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: > Initialization Done 7 MAP > 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: > Processing alias tim_hbase_occurrence for file > hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence > 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 > forwarding 1 rows > 2012-04-17 16:58:45,714 INFO > org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows > 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 > forwarding 1 rows > 2012-04-17 16:58:45,723 FATAL ExecMapper: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row {"id":1444,"scientific_name":null,"data_resource_id":1081} > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548) > at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) > at org.apache.hadoop.mapred.Child.main(Child.java:264) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.ClassCastException: > org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to > org.apache.hadoop.hive.serde2.lazy.LazyInteger > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762) > at > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529) > ... 9 more > Caused by: java.lang.ClassCastException: > org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to > org.apache.hadoop.hive.serde2.lazy.LazyInteger > at > org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43) > at > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239) > at > org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150) > at > org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142) > at > org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119) > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750) > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722) > ... 18 more > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira