ashutosh-bapat commented on a change in pull request #579: HIVE-21109 : Support
stats replication for ACID tables.
URL: https://github.com/apache/hive/pull/579#discussion_r269872622
##########
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnCommonUtils.java
##########
@@ -84,6 +86,73 @@ public static ValidTxnList
createValidReadTxnList(GetOpenTxnsResponse txns, long
return new ValidReadTxnList(exceptions, outAbortedBits, highWaterMark,
minOpenTxnId);
}
+ /**
+ * Transform a {@link
org.apache.hadoop.hive.metastore.api.GetOpenTxnsResponse} to a
+ * {@link org.apache.hadoop.hive.common.ValidTxnList}. This assumes that
the caller intends to
+ * read the files, and thus treats both open and aborted transactions as
invalid.
+ *
+ * This API is used by Hive replication which may have multiple transactions
open at a time.
+ *
+ * @param txns open txn list from the metastore
+ * @param currentTxns Current transactions that the replication has opened.
If any of the
+ * transactions is greater than 0 it will be removed from
the exceptions
+ * list so that the replication sees its own transaction
as valid.
+ * @return a valid txn list.
+ */
+ public static ValidTxnList createValidReadTxnList(GetOpenTxnsResponse txns,
Review comment:
If there were multiple transactions on the source running concurrently at a
time, there will be those many open transaction events in the dump which when
replicated will have those many open transactions at a time on the target while
replaying those events. So, there could be multiple open transactions on target
even during repl load.
The only link between CreateTableOperation#createTableReplaceMode and
Hive#alterTable is EnvironmentContext, so would could use this to pass a flag
to indicate the valid writeId list should be created using the given writeId.
But we are using Environment context to pass information only to the metastore
and not use it in-between. We could construct the valid writeId list in the
metastore directly like what we are doing for create table and partition using
that kind of flag. Does that look good?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
