[
https://issues.apache.org/jira/browse/HIVE-21757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844325#comment-16844325
]
Gopal V edited comment on HIVE-21757 at 5/20/19 9:50 PM:
---------------------------------------------------------
Internally generating write-ids also broke the query-cache, the materialized
view staleness check, but those were really performance features, not related
to data-loss due to this idea.
Compaction of files has to be strictly local to warehouse and ideally
idempotent on the table's versioning.
was (Author: gopalv):
Internally generating write-ids also broke the query-cache, the materialized
view staleness check, but those were really performance features, not related
to data-loss due to this idea.
> ACID: use a new write id for compaction's output instead of the visibility id
> -----------------------------------------------------------------------------
>
> Key: HIVE-21757
> URL: https://issues.apache.org/jira/browse/HIVE-21757
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 4.0.0
> Reporter: Vaibhav Gumashta
> Priority: Major
>
> HIVE-20823 added support for running compaction within a transaction. To
> control the visibility of the output directory, it uses
> base_writeId_visibilityId, where visibilityId is the transaction id of the
> transaction that the compactor ran in. Perhaps we can keep using the
> base_writeId format, by allocating a new writeId for the compactor and
> creating the new base/delta with that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
