klsince commented on issue #12396: URL: https://github.com/apache/pinot/issues/12396#issuecomment-1949460955
Thanks for reporting the issues. > For the first 2 issues, we need to figure out how to handle partial-upsert with same comparison value. For partial upsert, the insert order is critical, and we need to design a way to tell which is the final valid record. +1 to this idea. I think a key for consistency is to make Pinot logic deterministic while deciding the final valid record for a PK (between the mutable segment and the immutable segment supposed to replace the mutable one), particularly when the comparison values are same, or the sorted column might change the processing order of docs in a segment. I didn't fully get that `Handling Table Rebalance` issue. But as it has mentioned the `allSegmentsLoaded` flag, I'd assume the issue is about during table rebalance, the consuming segment might get started even before the other immutable segments in the same table partition get fully loaded on the server (?). If so, this issue and the 2nd issue of `Bug in Replace Segment` might share a potential fix, which is to start a new consuming segment only after the immutable segments in the same table partition are fully loaded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
