[
https://issues.apache.org/jira/browse/KAFKA-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13956002#comment-13956002
]
Joel Koshy commented on KAFKA-1355:
-----------------------------------
BTW, here are the configs and steps I used for this, KAFKA-1342 and KAFKA-1350:
Four brokers, 100 topics, eight partitions each.
Log4j and server configs by broker:
https://gist.github.com/anonymous/9906088
https://gist.github.com/anonymous/9906092
https://gist.github.com/anonymous/9906096
https://gist.github.com/anonymous/9906102
https://gist.github.com/anonymous/9906144
https://gist.github.com/anonymous/9906148
https://gist.github.com/anonymous/9906153
https://gist.github.com/anonymous/9906157
Producer performance: https://gist.github.com/anonymous/9906163
(At the end, just grep in the controller's request log and extract local time)
grep ControlledShutdownRequest logs/kafka-request*
> Reduce/optimize update metadata requests sent during leader election
> --------------------------------------------------------------------
>
> Key: KAFKA-1355
> URL: https://issues.apache.org/jira/browse/KAFKA-1355
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8.1
> Reporter: Joel Koshy
> Priority: Critical
> Fix For: 0.8.1.1
>
>
> This is part of the investigation into slow shutdowns in 0.8.1. While
> logging contributes to bulk of the regression, this one also adds
> quite a bit of overhead:
> In addLeaderAndIsrRequest (called for every partition that is led by the
> broker being shut down) we also add an UpdateMetadataRequest - each call to
> addUpdateMetadataRequests does two traversals over _all_ (global)
> partitions. I think it should be straightforward to optimize this a bit.
> Marking as critical, since it is not as big an overhead as the logging.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
