jadami10 commented on issue #9014: URL: https://github.com/apache/pinot/issues/9014#issuecomment-1175179709
The problem is that there is data published, and it's quite high volume. We are just filtering it all out in our decoder. So in this case, the server never actually commits the segment to advance the offsets. So a restart will cause it to go all the way back to the last time it saw a message it didn't filter out which can be hours or days. Our table config is has a bunch of custom keys for our internal stream ingestion plugin, so i'm not sure it would be helpful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
