[
https://issues.apache.org/jira/browse/KAFKA-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14369627#comment-14369627
]
Guozhang Wang commented on KAFKA-1912:
--------------------------------------
I was thinking that in the end state the admin requests would be sent to
controller either through re-routing or directly by clients after they discover
the controller, instead of letting controller listen on ZK admin path. The main
reason behind this is that: today controller use a separate thread from the ZK
listener callback to handle the admin requests, and hence have a lot of
synchronization issues with concurrent operations. There is a proposal of
refactoring controller to use a single worker thread to handle requests
sequentially and let the incoming requests to be queued-up at the controller
(see my comments in KAFKA-1305). And in that case when broker gets the admin
requests from KafkaApis it needs to be able to forward to the request to its
hosted controller, etc.
But thinking about it a bit more I believe now this refactoring does not
necessarily require us to get rid of ZK listeners, and it may be just simpler
to let controller manage incoming queued-up admin requests still through ZK.
> Create a simple request re-routing facility
> -------------------------------------------
>
> Key: KAFKA-1912
> URL: https://issues.apache.org/jira/browse/KAFKA-1912
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jay Kreps
> Fix For: 0.8.3
>
>
> We are accumulating a lot of requests that have to be directed to the correct
> server. This makes sense for high volume produce or fetch requests. But it is
> silly to put the extra burden on the client for the many miscellaneous
> requests such as fetching or committing offsets and so on.
> This adds a ton of practical complexity to the clients with little or no
> payoff in performance.
> We should add a generic request-type agnostic re-routing facility on the
> server. This would allow any server to accept a request and forward it to the
> correct destination, proxying the response back to the user. Naturally it
> needs to do this without blocking the thread.
> The result is that a client implementation can choose to be optimally
> efficient and manage a local cache of cluster state and attempt to always
> direct its requests to the proper server OR it can choose simplicity and just
> send things all to a single host and let that host figure out where to
> forward it.
> I actually think we should implement this more or less across the board, but
> some requests such as produce and fetch require more logic to proxy since
> they have to be scattered out to multiple servers and gathered back to create
> the response. So these could be done in a second phase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
