[
https://issues.apache.org/jira/browse/HIVE-20321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582110#comment-16582110
]
Gopal V edited comment on HIVE-20321 at 8/16/18 7:52 AM:
---------------------------------------------------------
LGTM - +1
The single key ::equals() implementation has been improved by 15% across the
whole query (& dropped out of the top-5 functions).
The only test query which did not improve was {{select count(distinct
ss_item_sk), count(distinct ss_customer_sk) from store_sales;}}
was (Author: gopalv):
LGTM - +1
The single key ::equals() implementation has been improved by 15% across the
whole query I tested with (& dropped out of the top-5 functions).
The only test query which did not improve was {{select count(distinct
ss_item_sk), count(distinct ss_customer_sk) from store_sales;}}
> Vectorization: Cut down memory size of 1 col VectorHashKeyWrapper to <1
> CacheLine
> ---------------------------------------------------------------------------------
>
> Key: HIVE-20321
> URL: https://issues.apache.org/jira/browse/HIVE-20321
> Project: Hive
> Issue Type: Bug
> Reporter: Gopal V
> Assignee: Matt McCline
> Priority: Major
> Attachments: HIVE-20321.01.patch, HIVE-20321.02.patch,
> HIVE-20321.03.patch
>
>
> With a full sized LLAP instance, the memory size of the VectorHashKeyWrapper
> is bigger than the low Xmx JVMs.
> {code}
> ***** 64-bit VM: **********************************************************
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper object internals:
> OFFSET SIZE
> TYPE DESCRIPTION VALUE
> 0 16
> (object header) N/A
> 16 4
> int VectorHashKeyWrapper.hashcode N/A
> 20 4
> (alignment/padding gap)
> 24 8
> long[] VectorHashKeyWrapper.longValues N/A
> 32 8
> double[] VectorHashKeyWrapper.doubleValues N/A
> 40 8
> byte[][] VectorHashKeyWrapper.byteValues N/A
> 48 8
> int[] VectorHashKeyWrapper.byteStarts N/A
> 56 8
> int[] VectorHashKeyWrapper.byteLengths N/A
> 64 8
> org.apache.hadoop.hive.serde2.io.HiveDecimalWritable[]
> VectorHashKeyWrapper.decimalValues N/A
> 72 8
> java.sql.Timestamp[] VectorHashKeyWrapper.timestampValues N/A
> 80 8
> org.apache.hadoop.hive.common.type.HiveIntervalDayTime[]
> VectorHashKeyWrapper.intervalDayTimeValues N/A
> 88 8
> boolean[] VectorHashKeyWrapper.isNull N/A
> 96 8
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper.HashContext
> VectorHashKeyWrapper.hashCtx N/A
> Instance size: 104 bytes
> Space losses: 4 bytes internal + 0 bytes external = 4 bytes total
> {code}
> Pulling this up to a parent class allows for this to be cut down to 32 bytes
> for the single column case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
