Guozhang Wang created KAFKA-6346:
------------------------------------
Summary: Consolidate multiple background async log operations
Key: KAFKA-6346
URL: https://issues.apache.org/jira/browse/KAFKA-6346
Project: Kafka
Issue Type: Improvement
Components: core
Reporter: Guozhang Wang
On Kafka broker's log managers, today we trigger a lot of repeating and
one-time async background tasks, including:
1. log deletion based on retention policy (repeating).
1.2. async log segment deletion triggered from 1 above (one-time)
2. log deletion based on replica stopping / log directory swapping (repeating)
3. log compaction, i.e. so-called "log cleaner thread" (repeating)
4. dirty log flushing
5. log recovery offsets checkpointing.
6. log starting offsets checkpointing.
I suspect all these background tasks are silently eating our CPU idle time
unnecessarily. We should consider 1) benchmark their impacts on CPU cost and 2)
if impact is high, consider consolidating some of these tasks into fewer async
background threads.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
