KKcorps commented on PR #20111: URL: https://github.com/apache/pinot/pull/20111#issuecomment-6818166133
This comment comes from a model of upsert thread orders that StarTree runs on upsert PRs. It is a recheck of the new commit, 72558f6. **The check that now reruns under the lineage lock still closes the race this PR targets, and the model found nothing new.** A 67 minute search, 1,398 runs from 72 starting states, ended with two COMPLETED entries over one segment only on today's code, and with this commit each of those starts failed under the lock. The new commit also fails a start when retention takes a `segmentsFrom` segment out of the IdealState between the first check and the lock. Today that start writes an IN_PROGRESS entry that `endReplaceSegments` can never complete, and its output stays hidden until lineage cleanup deletes it. The model saw no wrong rows from that order, so this part saves a wasted upload and does not change what queries read. The order in the earlier comment still applies, because forceCleanup still reverts an overlapping IN_PROGRESS entry and deletes its output, as today's code does. The model covers one replica of one partition, and it runs the lineage replace only on metadataTTL tables. <!-- upsert-model-check head=72558f672b26c6d12d003fc142bcc2731953bf9d --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
