[
https://issues.apache.org/jira/browse/HIVE-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344683#comment-16344683
]
Prasanth Jayachandran commented on HIVE-18359:
----------------------------------------------
Added fix for HIVE-18523 along with this patch. To deal with cases where tez
does not spin up any mappers, hive split generator will create a file inside
scratch directory which will be cleanup by GBY mapper where there are > 0 rows
emitted by mappers (if no mappers are launched the file will exist in reducer).
If the file does not exist, then summary row will be handled by some reducer
(not necessarily reducer with index 0).
[~jcamachorodriguez] can you please take a look at the new patch?
> Extend grouping set limits from int to long
> -------------------------------------------
>
> Key: HIVE-18359
> URL: https://issues.apache.org/jira/browse/HIVE-18359
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Priority: Major
> Attachments: HIVE-18359.1.patch, HIVE-18359.2.patch,
> HIVE-18359.3.patch, HIVE-18359.4.patch, HIVE-18359.5.patch,
> HIVE-18359.6.patch, HIVE-18359.7.patch
>
>
> Grouping sets is broken for >32 columns because of usage of Int for bitmap
> (also GROUPING__ID virtual column). This assumption breaks grouping
> sets/rollups/cube when number of participating aggregation columns is >32.
> The easier fix would be extend it to Long for now. The correct fix would be
> to use BitSets everywhere but that would require GROUPING__ID column type to
> binary which will make predicates on GROUPING__ID difficult to deal with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
