github-actions[bot] commented on code in PR #66424:
URL: https://github.com/apache/doris/pull/66424#discussion_r3710591637
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -3590,17 +3590,18 @@ public void truncateTable(String dbName, String
tableName, PartitionNamesInfo pa
// which is the right behavior.
long oldPartitionId = entry.getValue();
long newPartitionId = oldToNewPartitionId.get(oldPartitionId);
+ DataProperty dataProperty =
copiedTbl.getPartitionInfo().getDataProperty(oldPartitionId);
Review Comment:
[P1] Roll back the Cloud partition committed by the losing truncate
Letting the stale truncate continue also lets the Cloud override reach
`afterCreatePartitions`, which calls `commitPartition` before the final
old-partition-ID check. By then `sendCreateTabletsRpc` has already persisted
the new tablet metadata/keys; `commitPartition` removes the PREPARED recycle
key and publishes the partition keys. If another truncate already replaced the
old partition, the later `DdlException` runs only `failedCleanCallback`, which
deletes FE inverted-index entries; it never calls `dropCloudPartition`. The
normal recycler no longer has a marker to scan, and the orphan-version pass
skips a live table with indexes, so every such collision leaves committed,
unjournaled Cloud metadata/tablets behind. Please explicitly drop/mark all
generated Cloud partition IDs on any post-commit validation failure (including
partial batches), or fence commit against the FE partition generation, and
cover this losing Cloud path in a test.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -3590,17 +3590,18 @@ public void truncateTable(String dbName, String
tableName, PartitionNamesInfo pa
// which is the right behavior.
long oldPartitionId = entry.getValue();
long newPartitionId = oldToNewPartitionId.get(oldPartitionId);
+ DataProperty dataProperty =
copiedTbl.getPartitionInfo().getDataProperty(oldPartitionId);
Partition newPartition =
createPartitionWithIndices(db.getId(), copiedTbl,
newPartitionId, entry.getKey(),
copiedTbl.getIndexIdToMeta(),
partitionsDistributionInfo.get(oldPartitionId),
-
copiedTbl.getPartitionInfo().getDataProperty(oldPartitionId),
+ dataProperty,
copiedTbl.getPartitionInfo().getReplicaAllocation(oldPartitionId), null /*
version info */,
copiedTbl.getCopiedBfColumns(), tabletIdSet,
copiedTbl.isInMemory(),
copiedTbl.getPartitionInfo().getTabletType(oldPartitionId),
-
olapTable.getPartitionInfo().getDataProperty(oldPartitionId).getStoragePolicy(),
+ dataProperty.getStoragePolicy(),
Review Comment:
[P1] Revalidate the storage policy before publishing snapshot tablets
This snapshot value can become stale during the unlocked create phase. For
example, T1 snapshots policy A; T2 completes `ALTER ... storage_policy=B`,
which updates only the currently published old tablets and installs B in the
live `PartitionInfo`; then T1 creates the unpublished replacements with A here.
The final check compares partition IDs and schema/index metadata only, and
`replacePartition` transfers the current live `DataProperty` B to the new ID.
The result is FE metadata/replay at B while the new BE tablets were created
with A and never received T2's update. Please fence creation against
creation-affecting property changes (for example, compare the snapshot
property/generation under the final write lock and retry) instead of publishing
current metadata over tablets created from an older policy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
