[
https://issues.apache.org/jira/browse/HIVE-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840659#comment-13840659
]
Ashutosh Chauhan commented on HIVE-5936:
----------------------------------------
I tested HIVE-5916 on a table having 2K partitions. It used to fail earlier
because we used to run out of counters earlier. Now analyze command succeeds
since we lowered the number of unique counters required. However, as you
pointed out in HIVE-5916 review we still potentially can run into counter name
limit for which we need to generate unique smaller string which is implemented
in this patch. So, it will be good to shorten counter names when needed.
I also noted that client update the partition object one by one from client to
metastore, which is both slow as well as error-prone. We should batch update
partitions from client at end of job. Thats should be another jira.
> analyze command failing to collect stats with counter mechanism
> ---------------------------------------------------------------
>
> Key: HIVE-5936
> URL: https://issues.apache.org/jira/browse/HIVE-5936
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.13.0
> Reporter: Ashutosh Chauhan
> Assignee: Navis
> Attachments: HIVE-5936.1.patch.txt, HIVE-5936.2.patch.txt,
> HIVE-5936.3.patch.txt, HIVE-5936.4.patch.txt
>
>
> With counter mechanism, MR job is successful, but StatsTask on client fails
> with NPE.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
