[
https://issues.apache.org/jira/browse/HIVE-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-18589:
----------------------------------
Description:
txnid:17 starts reading T2/P1
txnid:20 does insert overwrite T1/P1, creates base_20, commits. txnid:17 still
running
txnid:21 stars reading T1/P1. It's ValidTxnList will txnid:17 as open.
before Insert overwrite was supported, only the compactor could produce base_20
by running major compaction. Major compaction erases history and so a reader
with txnid:17 open, can't use base_20.
Normally, the Cleaner is smart enough to not clean pre-compaction files if it's
possible that there is a reader that requires them. There is a safety check
that creates "Not enough history.." error if it finds that the current reader
can't properly execute based on the files available.
with the introduction of IOW on acid tables, there is another way to produce a
base. The difference is that here, the base has no history by definition and
so the same check is not needed but is triggered in the scenario above.
> java.io.IOException: Not enough history available
> -------------------------------------------------
>
> Key: HIVE-18589
> URL: https://issues.apache.org/jira/browse/HIVE-18589
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 3.0.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Critical
> Attachments: HIVE-18589.01.patch
>
>
> txnid:17 starts reading T2/P1
> txnid:20 does insert overwrite T1/P1, creates base_20, commits. txnid:17
> still running
> txnid:21 stars reading T1/P1. It's ValidTxnList will txnid:17 as open.
> before Insert overwrite was supported, only the compactor could produce
> base_20 by running major compaction. Major compaction erases history and so
> a reader with txnid:17 open, can't use base_20.
> Normally, the Cleaner is smart enough to not clean pre-compaction files if
> it's possible that there is a reader that requires them. There is a safety
> check that creates "Not enough history.." error if it finds that the current
> reader can't properly execute based on the files available.
>
> with the introduction of IOW on acid tables, there is another way to produce
> a base. The difference is that here, the base has no history by definition
> and so the same check is not needed but is triggered in the scenario above.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
