kishoreg commented on issue #10147: URL: https://github.com/apache/pinot/issues/10147#issuecomment-2182011009
Reviving this. I think we should go with optimistic approach as Jackie suggested. One simplification would be to enable pauseless consumption feature only in case of commit happening because of reaching row threshold. This will always ensure that the ending offset is the same on all replicas. There is no harm in going ahead with next segment immediately in this case. The edge case happens when the commit is decided based on time because each replica can commit at different offset. To avoid all edge cases, we can simply fallback to current behavior in this case. Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
