[ https://issues.apache.org/jira/browse/KAFKA-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao resolved KAFKA-6175. ---------------------------- Resolution: Fixed > AbstractIndex should cache index file to avoid unnecessary disk access during > resize() > -------------------------------------------------------------------------------------- > > Key: KAFKA-6175 > URL: https://issues.apache.org/jira/browse/KAFKA-6175 > Project: Kafka > Issue Type: Improvement > Reporter: Dong Lin > Assignee: Dong Lin > Fix For: 1.0.1 > > > Currently when we shutdown a broker, we will call AbstractIndex.resize() for > all segments on the broker, regardless of whether the log segment is active > or not. AbstractIndex.resize() incurs raf.setLength(), which is expensive > because it accesses disks. If we do a threaddump during either > LogManger.shutdown() or LogManager.loadLogs(), most threads are in RUNNABLE > state at java.io.RandomAccessFile.setLength(). > This patch intends to speed up broker startup and shutdown time by skipping > AbstractIndex.resize() for inactive log segments. > Here is the time of LogManager.shutdown() in various settings. In all these > tests, broker has roughly 6k partitions and 19k segments. > - If broker does not have this patch and KAFKA-6172, LogManager.shutdown() > takes 69 seconds > - If broker has KAFKA-6172 but not this patch, LogManager.shutdown() takes 21 > seconds. > - If broker has KAFKA-6172 and this patch, LogManager.shutdown() takes 1.6 > seconds. -- This message was sent by Atlassian JIRA (v6.4.14#64029)