9aman opened a new pull request, #15016:
URL: https://github.com/apache/pinot/pull/15016
## Issue
Fetching COMMITTING segments for the pauseless table is an expensive
operation as it requires fetching ZK metadata for all the segments and
filtering out those that have `status == COMMITTING`
## Scope of the PR.
The PR adds a new entry `/PAUSELESS_DEBUG_METADATA/{tableName} ` under the
property store to maintain the list of COMMITTING segments. This will help in
fetching the COMMITTING segments in the debug API (will be added to this PR).
### Addition of segment into the list
The segment is added at the start of the commit protocol
### Removal of segment from the list
The segment is removed from the list after the segment is marked DONE in the
`COMMIT_END_METADATA` call.
### Handling Failures in update
#### Failure in adding segment to the list
The servers, during the commit protocol, keep on retrying the commit till
the segment is marked COMMITTING. The list is updated before the segment is
marked COMMITTING (in ZK) and hence any failures will be retried.
#### Failure in removing segment from the list.
RealtimeSegmentValidationManager will remove segments from the list if there
status is DONE or the metadata is not present (possibly the segment has been
deleted).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
