guruguha opened a new pull request, #19253:
URL: https://github.com/apache/pinot/pull/19253
Fixed critical bug where DELETE /tables/{tableName} was not atomic, allowing
concurrent deletions to run in parallel with no mutual exclusion. This could
lead to race conditions, metadata corruption, and inconsistent cluster state.
## Key Changes
- **Deletion marker znode**: Added distributed locking mechanism to prevent
concurrent table deletions across controllers
- **Enhanced addTable()**: Checks for deletion markers and prevents table
recreation during deletion operations
- **24-hour expiry**: Implemented automatic expiry on deletion markers to
prevent permanent blocking
- **Performance optimization**: Batched segment move operations in
retention>0 path, reducing deletion time from hours to minutes
- **Configuration honor**: Added logic to respect table config's
deletedSegmentsRetentionPeriod setting
- **Post-deletion validation**: Ensures all table-related znodes are
successfully removed before marking deletion complete
- **Comprehensive testing**: Added unit tests for deletion marker
functionality
## Bug Claims Addressed
✅ No mutual exclusion between concurrent deletions
✅ Proceeds regardless of table existence
✅ Long race window during deletion
✅ Slow performance in retention>0 path
✅ Unconditional metadata removals
✅ Insufficient guards in table creation
## Testing
- Added comprehensive unit tests for deletion marker functionality
- All pre-commit checks pass (spotless, checkstyle, license)
- Manual verification of all bug claims from original report
## Files Modified
-
`pinot-common/src/main/java/org/apache/pinot/common/metadata/ZKMetadataProvider.java`
-
`pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java`
-
`pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java`
-
`pinot-common/src/test/java/org/apache/pinot/common/metadata/TableDeletionMarkerTest.java`
(new)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
