[
https://issues.apache.org/jira/browse/HIVE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127747#comment-13127747
]
[email protected] commented on HIVE-2223:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2381/#review2595
-----------------------------------------------------------
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
<https://reviews.apache.org/r/2381/#comment5833>
Since this is a list, sequence is significant, e.g. [1,2,3] != [3,2,1]. So
perhaps we should make the hash code reflect this?
Java's AbstractList does this via:
while (i.hasNext()) {
E obj = i.next();
hashCode = 31*hashCode + (obj==null ? 0 : obj.hashCode());
}
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
<https://reviews.apache.org/r/2381/#comment5834>
For java.util.HashMap, they xor the key's hashcode with the value's
hashcode for each entry, and then sum over all entries. I suppose that's to
distinguish
{a->b, c->d} from {a->d, b->c}.
although it fails to distinguish
{a->b} from {b->a}
- John
On 2011-10-13 16:50:53, Jonathan Chang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2381/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-13 16:50:53)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Adds hash codes for List and Map object inspectors.
bq.
bq.
bq. This addresses bug HIVE-2223.
bq. https://issues.apache.org/jira/browse/HIVE-2223
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ql/src/test/queries/clientpositive/groupby_complex_types.q PRE-CREATION
bq. ql/src/test/results/clientpositive/groupby_complex_types.q.out
PRE-CREATION
bq.
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
2d45aba
bq.
bq. Diff: https://reviews.apache.org/r/2381/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added unittest.
bq.
bq.
bq. Thanks,
bq.
bq. Jonathan
bq.
bq.
> support grouping on complex types in Hive
> -----------------------------------------
>
> Key: HIVE-2223
> URL: https://issues.apache.org/jira/browse/HIVE-2223
> Project: Hive
> Issue Type: New Feature
> Reporter: Kate Ting
> Assignee: Jonathan Chang
> Priority: Minor
> Attachments: HIVE-2223.patch
>
>
> Creating a query with a GROUP BY statement when an array type column is part
> of the column list is not yet supported:
> CREATE TABLE test_group_by ( key INT, group INT, terms ARRAY<STRING>);
> SELECT key, terms, count(group) FROM test_group_by GROUP BY key, terms;
> ...
> "Hash code on complex types not supported yet."
> java.lang.RuntimeException: Error while closing operators
> at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:232)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:356)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.RuntimeException: Hash code on complex types not supported yet.
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:799)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:462)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:211)
> ... 4 more
> Caused by: java.lang.RuntimeException: Hash code on complex types not
> supported yet.
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.hashCode(ObjectInspectorUtils.java:348)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:187)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:746)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:780)
> ... 9 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
