924060929 commented on code in PR #66348:
URL: https://github.com/apache/doris/pull/66348#discussion_r3780281791
##########
fe/fe-connector/fe-connector-hive/src/main/java/org/apache/doris/connector/hive/HiveConnectorTransaction.java:
##########
@@ -288,13 +288,23 @@ private ScheduledFuture<?> startCommitLockHeartbeat(long
lockId) {
}
private void commitWhileTableLocked() {
- validateWriteMetadataBeforePublication();
- // The classification (finishInsertTable) ran from the executor in the
legacy class; the unified SPI
- // exposes only commit(), so it runs here (before the committer) to
populate the action maps. If it
- // throws, the committer was never created and the engine's subsequent
rollback() cleans up.
- finishInsertTable(nameMapping);
- // Classification can perform metastore reads, so close that interval
before any file or HMS mutation.
- validateWriteMetadataBeforePublication();
+ try {
Review Comment:
Rechecked on current head `64eba05b2d`: this P1 is still reachable.
`PluginDrivenTransactionManager.commit()` removes the transaction before
delegating, while `HiveConnectorTransaction.commit()` still acquires the HMS
lock before entering the self-rollback boundary in `commitWhileTableLocked()`.
Therefore an `acquireExclusiveTableLock()` timeout/RPC failure closes and
deregisters the connector without aborting the valid deferred S3/Azure uploads
already accepted from BE; the executor cannot recover them with a later manager
rollback, and no Hive metadata is published. Please move lock acquisition under
an outer failure boundary that invokes connector rollback when publication has
not begun, and add a failure-injection test asserting all reported uploads are
aborted when lock acquisition fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
