KKcorps commented on PR #20106: URL: https://github.com/apache/pinot/pull/20106#issuecomment-6540965644
This comes from a model of upsert thread orders that StarTree runs on upsert PRs. **This clears the known orders where a partial upsert consumer starts before older segments of its partition finish loading, and the model found nothing new.** That covers a served table that gets a partition in a rebalance, also when the base row sits in an uploaded segment or one a minion task built. It also covers a replica that dropped its consuming segment before the committed copy loaded, the second case in the TODO in `PartitionConsumer.run`. With `enforceConsumptionInOrder` off, the default, a lagging replica's next consumer now waits for the previous segment too. I replayed the model's known bad orders with and without the change, and none breaks only with it. A 61-minute search of 360 runs found nothing new. Nine bad orders it reached without the change could not happen with it. The model covers one replica of one partition and leaves out dedup, so a clean search only means this search found nothing. <!-- upsert-model-check head=7c25de5cf6923e49bf05fe3a46ea3169f786e19b --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
