[
https://issues.apache.org/jira/browse/KAFKA-6172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388714#comment-16388714
]
Ivan Babrou commented on KAFKA-6172:
------------------------------------
Github doesn't report this as included in 1.0.1:
*
https://github.com/apache/kafka/commit/0c895706e8ab511efe352a824a0c9e2dab62499e
> Cache lastEntry in TimeIndex to avoid unnecessary disk access
> -------------------------------------------------------------
>
> Key: KAFKA-6172
> URL: https://issues.apache.org/jira/browse/KAFKA-6172
> Project: Kafka
> Issue Type: Improvement
> Reporter: Dong Lin
> Assignee: Dong Lin
> Priority: Major
> Fix For: 1.0.1
>
>
> LogSegment.close() calls timeIndex.maybeAppend(...), which in turns make a
> number of calls to timeIndex.lastEntry(). Currently timeIndex.lastEntry()
> involves disk seek operation because it tries to read the content of the last
> few bytes of the index files on the disk. This slows down the broker shutdown
> process.
> Here is the time of LogManager.shutdown() in various settings. In all these
> tests, broker has roughly 6k partitions and 20k segments.
> - If broker does not have this patch and `log.dirs` is configured with 1 JBOD
> log directory, LogManager.shutdown() takes 15 minutes (roughly 900 seconds).
> - If broker does not have this patch and `log.dirs` is configured with 10
> JBOD log directories, LogManager.shutdown() takes 84 seconds.
> - If broker have this patch and `log.dirs` is configured with 10 JBOD log
> directories, LogManager.shutdown() takes 24 seconds.
> Thus we expect to save 71% time in LogManager.shutdown() by having this
> optimization. This patch intends to reduce the broker shutdown time by
> caching the lastEntry in memory so that broker does not have to always read
> disk to get the lastEntry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
