Eugene Koifman created HIVE-12352: ------------------------------------- Summary: CompactionTxnHandler.markCleaned() may delete too much Key: HIVE-12352 URL: https://issues.apache.org/jira/browse/HIVE-12352 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 1.0.0 Reporter: Eugene Koifman Assignee: Eugene Koifman
Worker will start with DB in state X (wrt this partition). while it's working more txns will happen, against partition it's compacting. then this will delete state up to X and since then. There may be new delta files created between compaction starting and cleaning. These will not be compacted until more transactions happen. So this ideally should only delete up to TXN_ID that was compacted (i.e. HWM in Worker?) Then this can also run at READ_COMMITTED. So this means we'd want to store HWM in COMPACTION_QUEUE when Worker picks up the job. -- This message was sent by Atlassian JIRA (v6.3.4#6332)