[
https://issues.apache.org/jira/browse/HIVE-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153690#comment-14153690
]
Eugene Koifman commented on HIVE-8258:
--------------------------------------
Review Comments:
1. TestCleaner has 3 unused imports
2. Cleaner: comment at lines 74-78: I think it would be good to elaborate on
why this works. Something like "any readers that acquired new locks on the
same partition will not read files we are trying to delete since they will have
been merged into other deltas/base by compaction and AcidUtils.getAcidState()
has the logic to do that"
3. Cleaner line 86: {noformat}if (!compactId2LockMap.containsKey(ci.id))
{{noformat} - I don't think this is the right map to use here
4. The cleaner may be in removeFiles() doing fs.delete() while some reader may
be calling AcidUtils.getAcidState() at exactly the same. Is this a race
condition that can cause problems? You get a list of files in getAcidState()
but by the time you query the metadata about one such file it's deleted by the
cleaner. Is the FS flexible enough for this?
> Compactor cleaners can be starved on a busy table or partition.
> ---------------------------------------------------------------
>
> Key: HIVE-8258
> URL: https://issues.apache.org/jira/browse/HIVE-8258
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 0.13.1
> Reporter: Alan Gates
> Assignee: Alan Gates
> Priority: Critical
> Attachments: HIVE-8258.patch
>
>
> Currently the cleaning thread in the compactor does not run on a table or
> partition while any locks are held on this partition. This leaves it open to
> starvation in the case of a busy table or partition. It only needs to wait
> until all locks on the table/partition at the time of the compaction have
> expired. Any jobs initiated after that (and thus any locks obtained) will be
> for the new versions of the files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
