@ijuma Thanks for pointing this out. Here is some information about the `mmap` and the error handling: - `mmap` will be done whenever the index is accessed rather than the index object gets created. To be more specific, `mmap` will be triggered in the following sceanrios: 1. Sanity check. After this patch, we only sanity check indexes of segments above the recovery points. 2. Index lookup. This can happen when consumer consumes messages, GroupMetadataManager loads offsets and the segment is truncated/deleted. 3. Access last entry of the index. The retention thread will acess the last entry of time index in order to determine the max timestamp of a segment for time-based retention.
- Since we don't provide guarantee for the flushed segments, errors (IOException) happened during loading the index file will be catured and the corresponding log dir will be marked as offline. This patch will not change this behavior because the code paths that can trigger index lazy load mentioned above are already wrapped by the `maybeHandleIOException`, which basically catches the IOException, mark the log dir to be offline and throw a KafkaStorageException. [ Full content available at: https://github.com/apache/kafka/pull/5498 ] This message was relayed via gitbox.apache.org for [email protected]
