Ryan19929 opened a new pull request, #68049:
URL: https://github.com/apache/doris/pull/68049
[fix](fe) Fix table stuck in WAITING_STABLE after cancelling schema change
job
### What problem does this PR solve?
Issue Number: close #66177
Problem Summary:
Reproduction:
1. Make a tablet of the table stay in the tablet scheduler for a long time.
For example, an orphan CLONE replica whose pathHash is -1 makes the scheduler
always fail with "dest replica has no slot" and remove the tablet ctx after too
many failures, while the colocate checker keeps re-adding the tablet to the
pending queue on every check round.
2. Submit an ALTER TABLE schema change on the table. The job's
checkTableStable() finds the tablet in the tablet scheduler, sets the table
state to WAITING_STABLE and the job keeps waiting in PENDING.
3. Execute CANCEL ALTER TABLE COLUMN. FE logs "set table's state to NORMAL
when cancel", but the table state is actually still WAITING_STABLE. The job has
already been marked CANCELLED, so cancelling again fails with "could not find
related job", and any subsequent DDL on the table is rejected because the table
state is not NORMAL. The table is stuck until someone manually runs ADMIN SET
TABLE ... STATUS PROPERTIES("state" = "NORMAL").
Root cause: SchemaChangeJobV2.changeTableState() only resets the table state
when the current state is SCHEMA_CHANGE, and misses the WAITING_STABLE case.
When the job is cancelled (or finished/replayed) while the table is still
WAITING_STABLE, the table state is never reset to NORMAL, while the job itself
is already in a final state and will never touch the table again.
Fix: also allow resetting the table state from WAITING_STABLE in
SchemaChangeJobV2.changeTableState(). WAITING_STABLE is only set by this job's
own checkTableStable(), so it is always safe to reset it here.
### Release note
Fix the issue that cancelling a schema change job which is still waiting for
the table to become stable leaves the table in WAITING_STABLE state forever,
blocking all subsequent DDL on the table.
### Check List (For Author)
- Test: Unit Test
- Added
SchemaChangeJobV2Test#testCancelSchemaChangeWhileTableWaitingStable. It fails
without the fix (expected: <NORMAL> but was: <WAITING_STABLE>) and passes with
the fix. The whole SchemaChangeJobV2Test (14 tests) passes.
- Behavior changed: Yes - after cancelling a schema change job that is
waiting in PENDING state, the table state is now correctly reset to NORMAL
instead of staying WAITING_STABLE
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
