[
https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338208#comment-16338208
]
Eugene Koifman commented on HIVE-18192:
---------------------------------------
Suppose A is the set of all ValidTxnList across all active readers. Each
ValidTxnList has minOpenTxnId.
MIN_HISTORY_LEVEL allows us to determine min(minOpenTxnId) across all currently
active readers. It's not the same as COMPLETED_TXN_COMPONENTS.
Entries from COMPLETED_TXN_COMPONENTS get removed once compaction has processed
the relevant partition which can happen before all readers that think txn X is
open have gone away.
Suppose txn 17 starts at t1 and sees txnid 13 with writeID 13 open.
13 commits (via it's parent txn) at t2 > t1. (17 is still running).
Compaction runs at t3 >t2 to produce base_14 (or delta_10_14 for example) on
Table1/Part1 (17 is still running)
COMPLETED_TXN_COMPONENTS may be cleaned at this point.
at t4 > t3 17 may (multi stmt txn) needs to read Table1/Part1. It now needs to
construct a ValidWriteIDList as it would have looked like at the time 17
started to maintain SI.
But if TXNS2WRITE_ID no longer has txnid13 -> writeID13 mapping, there is no
way to know that ValidWriteIDList should have writeID13 Open.
> Introduce WriteID per table rather than using global transaction ID
> -------------------------------------------------------------------
>
> Key: HIVE-18192
> URL: https://issues.apache.org/jira/browse/HIVE-18192
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
> Affects Versions: 3.0.0
> Reporter: anishek
> Assignee: Sankar Hariappan
> Priority: Major
> Labels: ACID, DR, pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-18192.01.patch, HIVE-18192.02.patch,
> HIVE-18192.03.patch, HIVE-18192.04.patch, HIVE-18192.05.patch
>
>
> To support ACID replication, we will be introducing a per table write Id
> which will replace the transaction id in the primary key for each row in a
> ACID table.
> The current primary key is determined via
> <original transaction id, bucketid , rowid>
> which will move to
> <original write id, bucketid, rowid>
> For each table modified by the given transaction will have a table level
> write ID allocated and a persisted map of global txn id -> to table -> write
> id for that table has to be maintained to allow Snapshot isolation.
> Readers should use the combination of ValidTxnList and
> ValidWriteIdList(Table) for snapshot isolation.
>
> [Hive Replication - ACID
> Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf]
> has a section "Per Table Sequences (Write-Id)" with more detials
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
