[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]

2013-01-09 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548171#comment-13548171
 ] 

Hudson commented on HIVE-2958:
--

Integrated in Hive-trunk-hadoop2 #54 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2/54/])
HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot 
be
cast LazyInteger]
(Navis Ryu via Ashutosh Chauhan)

Summary:
DPAL- GROUP BY causing ClassCastException [LazyDioInteger cannot be cast
LazyInteger]

This relates to https://issues.apache.org/jira/browse/HIVE-1634.

The following work fine:

CREATE EXTERNAL TABLE tim_hbase_occurrence (
  id int,
  scientific_name string,
  data_resource_id int
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES (
  hbase.columns.mapping = :key#b,v:scientific_name#s,v:data_resource_id#b
) TBLPROPERTIES(
  hbase.table.name = mini_occurrences,
  hbase.table.default.storage.type = binary
);
SELECT * FROM tim_hbase_occurrence LIMIT 3;
SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;

However, the following fails:

SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY
data_resource_id;

The error given:

0 TS
2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Initialization Done 7 MAP
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Processing alias tim_hbase_occurrence for file
hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7
forwarding 1 rows
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0
forwarding 1 rows
2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1
forwarding 1 rows
2012-04-17 16:58:45,723 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {id:1444,scientific_name:null,data_resource_id:1081}
at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
... 9 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722)
... 18 more

Test Plan: EMPTY

Reviewers: JIRA, ashutoshc

Reviewed By: ashutoshc

Differential Revision: https://reviews.facebook.net/D2871 (Revision 1328157)

 Result = ABORTED

[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]

2012-04-20 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258037#comment-13258037
 ] 

Hudson commented on HIVE-2958:
--

Integrated in Hive-trunk-h0.21 #1384 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1384/])
HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot 
be
cast LazyInteger]
(Navis Ryu via Ashutosh Chauhan)

Summary:
DPAL- GROUP BY causing ClassCastException [LazyDioInteger cannot be cast
LazyInteger]

This relates to https://issues.apache.org/jira/browse/HIVE-1634.

The following work fine:

CREATE EXTERNAL TABLE tim_hbase_occurrence (
  id int,
  scientific_name string,
  data_resource_id int
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES (
  hbase.columns.mapping = :key#b,v:scientific_name#s,v:data_resource_id#b
) TBLPROPERTIES(
  hbase.table.name = mini_occurrences,
  hbase.table.default.storage.type = binary
);
SELECT * FROM tim_hbase_occurrence LIMIT 3;
SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;

However, the following fails:

SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY
data_resource_id;

The error given:

0 TS
2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Initialization Done 7 MAP
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Processing alias tim_hbase_occurrence for file
hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7
forwarding 1 rows
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0
forwarding 1 rows
2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1
forwarding 1 rows
2012-04-17 16:58:45,723 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {id:1444,scientific_name:null,data_resource_id:1081}
at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at 
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
... 9 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722)
... 18 more

Test Plan: EMPTY

Reviewers: JIRA, ashutoshc

Reviewed By: ashutoshc

Differential Revision: https://reviews.facebook.net/D2871 (Revision 1328157)

 Result = FAILURE

[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]

2012-04-19 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257950#comment-13257950
 ] 

Phabricator commented on HIVE-2958:
---

ashutoshc has accepted the revision HIVE-2958 [jira] GROUP BY causing 
ClassCastException [LazyDioInteger cannot be cast LazyInteger].

  +1

REVISION DETAIL
  https://reviews.facebook.net/D2871

BRANCH
  DPAL-


 GROUP BY causing ClassCastException [LazyDioInteger cannot be cast 
 LazyInteger]
 ---

 Key: HIVE-2958
 URL: https://issues.apache.org/jira/browse/HIVE-2958
 Project: Hive
  Issue Type: Bug
  Components: HBase Handler
Affects Versions: 0.9.0
 Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today
Reporter: Tim Robertson
Assignee: Navis
Priority: Blocker
 Attachments: HIVE-2958.D2871.1.patch


 This relates to https://issues.apache.org/jira/browse/HIVE-1634.
 The following work fine:
 {code}
 CREATE EXTERNAL TABLE tim_hbase_occurrence ( 
   id int,
   scientific_name string,
   data_resource_id int
 ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH 
 SERDEPROPERTIES (
   hbase.columns.mapping = :key#b,v:scientific_name#s,v:data_resource_id#b
 ) TBLPROPERTIES(
   hbase.table.name = mini_occurrences, 
   hbase.table.default.storage.type = binary
 );
 SELECT * FROM tim_hbase_occurrence LIMIT 3;
 SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;
 {code}
 However, the following fails:
 {code}
 SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY 
 data_resource_id;
 {code}
 The error given:
 {code}
 0 TS
 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 
 Initialization Done 7 MAP
 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 
 Processing alias tim_hbase_occurrence for file 
 hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 
 forwarding 1 rows
 2012-04-17 16:58:45,714 INFO 
 org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 
 forwarding 1 rows
 2012-04-17 16:58:45,723 FATAL ExecMapper: 
 org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
 processing row {id:1444,scientific_name:null,data_resource_id:1081}
   at 
 org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
   at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
   at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:396)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
   at org.apache.hadoop.mapred.Child.main(Child.java:264)
 Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
 java.lang.ClassCastException: 
 org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to 
 org.apache.hadoop.hive.serde2.lazy.LazyInteger
   at 
 org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
   at 
 org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
   at 
 org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
   at 
 org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
   ... 9 more
 Caused by: java.lang.ClassCastException: 
 org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to 
 org.apache.hadoop.hive.serde2.lazy.LazyInteger
   at 
 org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
   at 
 org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
   at 
 org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
   at

[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]

2012-04-18 Thread Tim Robertson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13256250#comment-13256250
 ] 

Tim Robertson commented on HIVE-2958:
-

Thanks Navis for such a quick turnaround!  I have applied the patch and confirm 
it works on our cluster (with HBase 0.90.4), but can't comment on the 
implications of the changes.

 GROUP BY causing ClassCastException [LazyDioInteger cannot be cast 
 LazyInteger]
 ---

 Key: HIVE-2958
 URL: https://issues.apache.org/jira/browse/HIVE-2958
 Project: Hive
  Issue Type: Bug
  Components: HBase Handler
Affects Versions: 0.9.0
 Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today
Reporter: Tim Robertson
Assignee: Navis
Priority: Blocker
 Attachments: HIVE-2958.D2871.1.patch


 This relates to https://issues.apache.org/jira/browse/HIVE-1634.
 The following work fine:
 {code}
 CREATE EXTERNAL TABLE tim_hbase_occurrence ( 
   id int,
   scientific_name string,
   data_resource_id int
 ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH 
 SERDEPROPERTIES (
   hbase.columns.mapping = :key#b,v:scientific_name#s,v:data_resource_id#b
 ) TBLPROPERTIES(
   hbase.table.name = mini_occurrences, 
   hbase.table.default.storage.type = binary
 );
 SELECT * FROM tim_hbase_occurrence LIMIT 3;
 SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;
 {code}
 However, the following fails:
 {code}
 SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY 
 data_resource_id;
 {code}
 The error given:
 {code}
 0 TS
 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 
 Initialization Done 7 MAP
 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 
 Processing alias tim_hbase_occurrence for file 
 hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 
 forwarding 1 rows
 2012-04-17 16:58:45,714 INFO 
 org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 
 forwarding 1 rows
 2012-04-17 16:58:45,723 FATAL ExecMapper: 
 org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
 processing row {id:1444,scientific_name:null,data_resource_id:1081}
   at 
 org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
   at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
   at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:396)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
   at org.apache.hadoop.mapred.Child.main(Child.java:264)
 Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
 java.lang.ClassCastException: 
 org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to 
 org.apache.hadoop.hive.serde2.lazy.LazyInteger
   at 
 org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
   at 
 org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
   at 
 org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
   at 
 org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
   ... 9 more
 Caused by: java.lang.ClassCastException: 
 org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to 
 org.apache.hadoop.hive.serde2.lazy.LazyInteger
   at 
 org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
   at 
 org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
   at 
 org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
   at