guruguha opened a new pull request, #19254:
URL: https://github.com/apache/pinot/pull/19254
Description: Fixed critical bug where DELETE /tables/{tableName} was not
atomic, allowing concurrent deletions to run in parallel with no mutual
exclusion. This could lead to race conditions, metadata corruption, and
inconsistent cluster state.
Key Changes
• Deletion marker znode: Added distributed locking mechanism to prevent
concurrent table deletions across controllers
• Enhanced addTable(): Checks for deletion markers and prevents table
recreation during deletion operations
• 24-hour expiry: Implemented automatic expiry on deletion markers to
prevent permanent blocking
• Performance optimization: Batched segment move operations in retention>0
path, reducing deletion time from hours to minutes
• Configuration honor: Added logic to respect table config's
deletedSegmentsRetentionPeriod setting
• Post-deletion validation: Ensures all table-related znodes are
successfully removed before marking deletion complete
• Comprehensive testing: Added unit tests for deletion marker functionality
Bug Claims Addressed
✅ No mutual exclusion between concurrent deletions
✅ Proceeds regardless of table existence
✅ Long race window during deletion
✅ Slow performance in retention>0 path
✅ Unconditional metadata removals
✅ Insufficient guards in table creation
Testing
• Added comprehensive unit tests for deletion marker functionality
• All pre-commit checks pass (spotless, checkstyle, license)
• Manual verification of all bug claims from original report
Files Modified
•
pinot-common/src/main/java/org/apache/pinot/common/metadata/ZKMetadataProvider.java
•
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
•
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java
•
pinot-common/src/test/java/org/apache/pinot/common/metadata/TableDeletionMarkerTest.java
(new)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
