[
https://issues.apache.org/jira/browse/HIVE-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398366#comment-15398366
]
Eugene Koifman commented on HIVE-14350:
---------------------------------------
The previous logic looked like
{noformat}
for (TxnInfo txn : txns.getOpen_txns()) {
if (txn.getState() == TxnState.OPEN) minOpenTxn = Math.min(minOpenTxn,
txn.getId());
exceptions[i++] = txn.getId();//todo: only add Aborted
}
{noformat}
so the exceptions[] should be full. I don't think there was reason to trim the
array previously.
In current implementation:
The part that is done in c'tor of ValidCompactorTxnList has to be done after
the list is sorted.
The part that is done in TxnUtils.createValidCompactTxnList is done to
eliminate empty array slots which are initialized to 0 and will affect sort.
(They could instead be filled with Long.MAX_VALUE...)
Am I answering the question?
> Aborted txns cause false positive "Not enough history available..." msgs
> ------------------------------------------------------------------------
>
> Key: HIVE-14350
> URL: https://issues.apache.org/jira/browse/HIVE-14350
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 1.3.0, 2.1.1
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Critical
> Attachments: HIVE-14350.2.patch, HIVE-14350.3.patch,
> HIVE-14350.5.patch, HIVE-14350.6.patch, HIVE-14350.7.patch,
> HIVE-14350.8.patch, HIVE-14350.9.patch
>
>
> this is a followup to HIVE-13369. Only open txns should prevent use of a
> base file. But ValidTxnList does not make a distinction between open and
> aborted txns. The presence of aborted txns causes false positives which can
> happen too often since the flow is
> 1. Worker generates a new base file,
> 2. then asynchronously Cleaner removes now-compacted aborted txns. (strictly
> speaking it's Initiator that does the actual clean up)
> So we may have base_5 and base_10 and txnid 7 aborted. Then current impl
> will disallow use of base_10 though there is no need for that. Worse, if
> txnid_4 is aborted and hasn't been purged yet, base_5 will be rejected as
> well and then an error will be raised since there is no suitable base file
> left.
> ErrorMsg.ACID_NOT_ENOUGH_HISTORY is msg produced
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
