sollhui commented on code in PR #64878:
URL: https://github.com/apache/doris/pull/64878#discussion_r3600083763
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java:
##########
@@ -2040,7 +2140,57 @@ public void gsonPostProcess() throws IOException {
}
}
- public abstract void modifyProperties(AlterRoutineLoadCommand command)
throws UserException;
+ public void modifyProperties(AlterRoutineLoadCommand command) throws
UserException {
+ writeLock();
+ try {
+ if (getState() != JobState.PAUSED) {
+ throw new DdlException("Only supports modification of PAUSED
jobs");
+ }
+ if (!command.hasTargetTable()) {
+ unprotectApplyAlter(command);
+ return;
+ }
+
+ Database db = Env.getCurrentInternalCatalog().getDbNullable(dbId);
+ if (db == null) {
+ throw new DdlException("Database not found: " + dbId);
+ }
+ db.readLock();
+ try {
+ Table table = db.getTableNullable(command.getTargetTableId());
+ if (table == null) {
+ throw new DdlException("Table not found: " +
command.getTargetTableId());
+ }
+ if (!(table instanceof OlapTable)) {
+ throw new DdlException("Routine load target table must be
an OLAP table");
+ }
+ table.readLock();
+ try {
+ unprotectApplyAlter(command);
Review Comment:
Please avoid doing external I/O while holding the database and target-table
metadata locks. This call runs under the job write lock, db read lock, and
target table read lock, but unprotectApplyAlter ->
KafkaRoutineLoadJob.unprotectModifyProperties -> modifyPropertiesInternal may
call KafkaUtil.getOffsetsForTimes/getRealOffsets and, in cloud mode,
MetaServiceProxy.resetRLProgress. If Kafka or MetaService is slow or
unavailable, ALTER can therefore hold these metadata locks until the network
call times out and block DDL that needs the corresponding write locks. Before
this PR, Kafka offset conversion happened before the job write lock and no
database/table lock wrapped data-source mutation, so the database/table lock
expansion is new here. Please stage the Kafka offset resolution and Cloud
progress operation outside the database/table locks, then acquire metadata
locks only for the final target revalidation and in-memory/journal commit; the
job write lock can still serialize concurr
ent ALTER operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
