Guozhang Wang created KAFKA-6346: ------------------------------------ Summary: Consolidate multiple background async log operations Key: KAFKA-6346 URL: https://issues.apache.org/jira/browse/KAFKA-6346 Project: Kafka Issue Type: Improvement Components: core Reporter: Guozhang Wang
On Kafka broker's log managers, today we trigger a lot of repeating and one-time async background tasks, including: 1. log deletion based on retention policy (repeating). 1.2. async log segment deletion triggered from 1 above (one-time) 2. log deletion based on replica stopping / log directory swapping (repeating) 3. log compaction, i.e. so-called "log cleaner thread" (repeating) 4. dirty log flushing 5. log recovery offsets checkpointing. 6. log starting offsets checkpointing. I suspect all these background tasks are silently eating our CPU idle time unnecessarily. We should consider 1) benchmark their impacts on CPU cost and 2) if impact is high, consider consolidating some of these tasks into fewer async background threads. -- This message was sent by Atlassian JIRA (v6.4.14#64029)