[
https://issues.apache.org/jira/browse/HIVE-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102204#comment-14102204
]
Remus Rusanu commented on HIVE-7664:
------------------------------------
shouldn't there be a case for DECIMAL primitive category? I see a
DecimalAccessor, but no case covering it in the BatchAccessor ctor.
> VectorizedBatchUtil.addRowToBatchFrom is not optimized for Vectorized
> execution and takes 25% CPU
> -------------------------------------------------------------------------------------------------
>
> Key: HIVE-7664
> URL: https://issues.apache.org/jira/browse/HIVE-7664
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.13.1
> Reporter: Mostafa Mokhtar
> Fix For: 0.14.0
>
> Attachments: HIVE-7664.1.patch.txt
>
>
> In a Group by heavy vectorized Reducer vertex 25% of CPU is spent in
> VectorizedBatchUtil.addRowToBatchFrom().
> Looked at the code of VectorizedBatchUtil.addRowToBatchFrom and it looks like
> it wasn't optimized for Vectorized processing.
> addRowToBatchFrom is called for every row and for each row and every column
> in the batch getPrimitiveCategory is called to figure the type of each
> column, column types are stored in a HashMap, for VectorGroupByOperator
> columns types won't change between batches, so column types shouldn't be
> looked up for every row.
> I recommend storing the column type in StructObjectInspector so that other
> components can leverage this optimization.
> Also addRowToBatchFrom has a case statement for every row and every column
> used for type casting I recommend encapsulating the type logic in templatized
> methods.
> {code}
> Stack Trace Sample Count Percentage(%)
> VectorizedBatchUtil.addRowToBatchFrom 86 26.543
> AbstractPrimitiveObjectInspector.getPrimitiveCategory() 34 10.494
> LazyBinaryStructObjectInspector.getStructFieldData 25 7.716
> StandardStructObjectInspector.getStructFieldData 4 1.235
> {code}
> The query used :
> {code}
> select
> ss_sold_date_sk
> from
> store_sales
> where
> ss_sold_date between '1998-01-01' and '1998-06-01'
> group by ss_item_sk , ss_customer_sk , ss_sold_date_sk
> having sum(ss_list_price) > 50000000000000;
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
