sankarh commented on a change in pull request #579: HIVE-21109 : Support stats
replication for ACID tables.
URL: https://github.com/apache/hive/pull/579#discussion_r269220469
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -2950,21 +2956,33 @@ public Partition createPartition(Table tbl,
Map<String, String> partSpec) throws
int size = addPartitionDesc.getPartitionCount();
List<org.apache.hadoop.hive.metastore.api.Partition> in =
new ArrayList<org.apache.hadoop.hive.metastore.api.Partition>(size);
- AcidUtils.TableSnapshot tableSnapshot = AcidUtils.getTableSnapshot(conf,
tbl, true);
long writeId;
String validWriteIdList;
- if (tableSnapshot != null && tableSnapshot.getWriteId() > 0) {
- writeId = tableSnapshot.getWriteId();
- validWriteIdList = tableSnapshot.getValidWriteIdList();
+
+ // In case of replication, get the writeId from the source and use valid
write Id list
+ // for replication.
+ if (addPartitionDesc.getReplicationSpec() != null &&
+ addPartitionDesc.getReplicationSpec().isInReplicationScope() &&
+ addPartitionDesc.getPartition(0).getWriteId() > 0) {
+ writeId = addPartitionDesc.getPartition(0).getWriteId();
+ validWriteIdList =
Review comment:
In replication flow, it is fine to use hardcoded ValidWriteIdList as we want
to forcefully set this writeId into table or partition objects. Getting it from
current state might be wrong as we don't update ValidTxnList in conf for repl
created txns.
ValidWriteIdList is just used to check if writeId in metastore objects are
updated by any concurrent inserts. In repl load flow, it is not possible as we
replicate one event at a time or in bootstrap, no 2 threads writes into same
table.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
