[
https://issues.apache.org/jira/browse/HIVE-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420985#comment-16420985
]
Hive QA commented on HIVE-18747:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
57s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9928/dev-support/hive-personality.sh
|
| git revision | master / 52290e7 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9928/yetus/patch-mvninstall-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9928/yetus/patch-asflicense-problems.txt
|
| modules | C: metastore ql standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9928/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> Cleaner for TXN_TO_WRITE_ID table entries using MIN_HISTORY_LEVEL.
> ------------------------------------------------------------------
>
> Key: HIVE-18747
> URL: https://issues.apache.org/jira/browse/HIVE-18747
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
> Affects Versions: 3.0.0
> Reporter: Sankar Hariappan
> Assignee: Sankar Hariappan
> Priority: Minor
> Labels: ACID, pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-18747.01.patch, HIVE-18747.02.patch,
> HIVE-18747.03.patch, HIVE-18747.04.patch
>
>
> Per table write ID implementation (HIVE-18192) maintains a map between txn ID
> and table write ID in TXN_TO_WRITE_ID meta table.
> The entries in this table is used to generate ValidWriteIdList for the given
> ValidTxnList to ensure snapshot isolation.
> When table or database is dropped, then these entries are cleaned-up. But, it
> is necessary to clean-up for active tables too for better performance.
> TXN_TO_WRITE_ID table keeps a mapping of Transaction ID to Write ID. The
> state of each Write ID (open, committed, aborted) is determined by the state
> of the parent transaction. In order to be able to get a WriteIdList that is
> accurate wrt ValidTxnList that is locked in at the start of the transaction,
> we have to retain txnid<->writeid mapping even after the transaction ends.
> This is because a reader at Snapshot Isolation that started when transaction
> X was open, should continue to ignore the data written by X even after X
> commits.
> So we need a mechanism to know when it is safe to remove TXN_TO_WRITE_ID.
> There are 2 parts to it. When txn X is opened, it records Y=select
> min(txn_id) from TXNS where txn_state=’o’ in MIN_HISTORY(txnid,opentxnid)
> table, i.e. it adds (X, Y) to MIN_HISTORY. On commit (and abort) of X, it
> removes its own entry from MIN_HISTORY. In the absence of Aborted
> transactions, MIN_HISTORY gives us the smallest open txnid across all active
> reader snapshots. Let Z=select min(opentxnid) from MIN_HISTORY. We can
> delete entries from TXN_TO_WRITE_ID once TXN_TO_WRITE_ID.T2W_TXNID < Z since
> every active reader sees txns < Z as committed.
> If S is aborted txns, we retain the metadata about it in TXNS as long as any
> data written S may be visible to some reader in the system so that the reader
> knows to skip this data. The rules for when that is are complex but wrt to
> TXN_TO_WRITE_ID, if A=select min(TXN_ID) from TXNS where TXN_STATE=’a’, then
> it’s safe to delete from TXN_TO_WRITE_ID when TXN_TO_WRITE_ID.T2W_TXNID <
> min(Z,A).
> If no open or aborted txns exist in the system, then we need to enable
> cleanup using latest allocated value of NEXT_TXN_ID table. Delete condition
> would be TXN_TO_WRITE_ID.T2W_TXNID < min(Z,A,NEXT_TXN_ID.ntxn_next).
> Also, it is proposed to trigger cleanup on TXN_TO_WRITE_ID from initiator
> immediately after cleaning up aborted txns metadata from TXNS table.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
