[GitHub] [kafka] hachikuji commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2021-03-04 Thread GitBox
hachikuji commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-790932209 @jolshan The active segment would not be replaced during cleaning at a minimum. I think the recovery logic we want is to scan backwards through the `.swap` files and collect the

[GitHub] [kafka] hachikuji commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2021-03-03 Thread GitBox
hachikuji commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-790233818 @jolshan The idea is to mimic during recovery the same operation that the log cleaner was trying to do. So if we were trying to replace segments [0, 10, 20] with [15] when the o

[GitHub] [kafka] hachikuji commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2021-03-03 Thread GitBox
hachikuji commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-790152137 @jolshan In `Log.completeSwapOperations`, we have the following comment: ``` // We create swap files for two cases: // (1) Log cleaning where multiple segment