maytasm edited a comment on pull request #11025: URL: https://github.com/apache/druid/pull/11025#issuecomment-812259294
> I meant, that might not be what users expect. I think they will want to know when they can query the new data without seeing old one. Maybe this can be addressed using #10676. > > > One problem might be auto compaction as it would try to compact the same interval again. > > Can you elaborate more on this problem? I'm not sure why it would compact the same interval again. Ah I see. For compaction task, the lag in dropping old segments would not be a problem for querying. For new data without seeing old data, I agree with you that it can be addressed using mechanism in https://github.com/apache/druid/pull/10676. I was mistaken. We will not have any problem in auto compaction with the fake empty segment. As the fake empty segment + the new real segments would fully overshadow the old segment. Marking old segments as unused is enough for auto compaction to not run on the same interval again even if old segments are drop later -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
