[
https://issues.apache.org/jira/browse/HIVE-24001?focusedWorklogId=469116&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-469116
]
ASF GitHub Bot logged work on HIVE-24001:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Aug/20 08:38
Start Date: 11/Aug/20 08:38
Worklog Time Spent: 10m
Work Description: klcopp merged pull request #1368:
URL: https://github.com/apache/hive/pull/1368
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 469116)
Time Spent: 40m (was: 0.5h)
> Don't cache MapWork in tez/ObjectCache during query-based compaction
> --------------------------------------------------------------------
>
> Key: HIVE-24001
> URL: https://issues.apache.org/jira/browse/HIVE-24001
> Project: Hive
> Issue Type: Bug
> Reporter: Karen Coppage
> Assignee: Karen Coppage
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Query-based major compaction can fail intermittently with the following issue:
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: One writer is
> supposed to handle only one bucket. We saw these 2 different buckets: 1 and 6
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFValidateAcidSortOrder.evaluate(GenericUDFValidateAcidSortOrder.java:77)
> {code}
> This is consistently preceded in the application log with:
> {code:java}
> [INFO] [TezChild] |tez.ObjectCache|: Found
> hive_20200804185133_f04cca69-fa30-4f1b-a5fe-80fc2d749f48_Map 1__MAP_PLAN__ in
> cache with value: org.apache.hadoop.hive.ql.plan.MapWork@74652101
> {code}
> Alternatively, when MapRecordProcessor doesn't find mapWork in
> tez/ObjectCache (but instead caches mapWork), major compaction succeeds.
> The failure happens because, if MapWork is reused,
> GenericUDFValidateAcidSortOrder (which is called during compaction) is also
> reused on splits belonging to two different buckets, which produces an error.
> Solution is to avoid storing MapWork in the ObjectCache during query-based
> compaction.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
