Harnoor7 opened a new issue, #14659: URL: https://github.com/apache/pinot/issues/14659
1. Currently for RTO tasks, watermark is updated in the executor after segments are uploaded. Hence, there can be possible scenarios where segments were uploaded to an offline table but RTO metadata watermark was not updated. AND RTO task generator does not validate if the segment has already been processed in the previous minion run. 2. If the time window (bucket size) of an RTO task is smaller than the segment time window, the same segment can be picked multiple times. Due to above points, Offline table can have inconsistent data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
