[
https://issues.apache.org/jira/browse/HIVE-16964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211889#comment-16211889
]
Steve Yeom commented on HIVE-16964:
-----------------------------------
Talked with Eugene.
Also checked with the current Hive master code with the unit test,
"TestTxnCommands2#testNonAcidToAcidConversion1".
1. Currently Hive.moveAcidFiles() does not move a "_orc_acid_version" file.
This static method is called by the MoveTask for the Hive
session of th.
I.e., FileSinkOperator at map reduce task creates such a file but the
MoveTask does not move the file to the final destination dir.
2. The intention for creating a "_orc_acid_version" file is to handle the case
where we have multiple versions of ACID file formats.
I.e., in that case, we need format version info somewhere either in the
Metastore or in the directory.
As Eugene indicated, currently for ACID tables, inserter/deleters create
delta directories independently and readers read relevant dirs without conflicts
with writers via Snapshot isolation. So there can be cases to have multiple
versions of delta directories per partition or table directory since
compactors are not sync with writers. So in this case, one
"_orc_acid_version" file may be needed per delta dir.
3. Possibly like the case of micromanaged tables, we can remove the steps to
create directories in a staging are and to perform MoveTask to
move the delta and base directories along with orc_acid_version file(s) to a
final destination.
Thus based on 3 and 4, I think we can lower the priority of this jira since the
fix of this jira (moving such a file to final destination)
may not be used at all for HDP 3.0.
> _orc_acid_version file is missing
> ---------------------------------
>
> Key: HIVE-16964
> URL: https://issues.apache.org/jira/browse/HIVE-16964
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 1.0.0
> Reporter: Eugene Koifman
> Assignee: Steve Yeom
>
> OrcRecordUpdater creates OrcRecordUpdater.ACID_FORMAT in the dir that it
> creates - but there is nothing Hive.moveAcidFiles() that copies it final
> location.
> It doesn't look like CompactorMR even attempts to create it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
