[
https://issues.apache.org/jira/browse/KAFKA-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367966#comment-14367966
]
Jay Kreps commented on KAFKA-1912:
----------------------------------
1. I think this can be a single thread. The number of such requests is bounded
by the number of connections since we process only one request at a time. In
this respect it is not different from the purgatories, which can maintain an
entry per connection and has a background thread.
2. Probably from the client's point of view a timeout is a timeout, no?
3. I think it is okay for this queue to be unbounded as the request objects
will be small and there is an implicit bound from the number of connections
(you can never have more than one request in flight for any connection). I
agree if you tried to bound it that would add all sorts of complexity.
4. Not sure if I understand that comment. [~guozhang] can you explain a bit
more?
> Create a simple request re-routing facility
> -------------------------------------------
>
> Key: KAFKA-1912
> URL: https://issues.apache.org/jira/browse/KAFKA-1912
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jay Kreps
> Fix For: 0.8.3
>
>
> We are accumulating a lot of requests that have to be directed to the correct
> server. This makes sense for high volume produce or fetch requests. But it is
> silly to put the extra burden on the client for the many miscellaneous
> requests such as fetching or committing offsets and so on.
> This adds a ton of practical complexity to the clients with little or no
> payoff in performance.
> We should add a generic request-type agnostic re-routing facility on the
> server. This would allow any server to accept a request and forward it to the
> correct destination, proxying the response back to the user. Naturally it
> needs to do this without blocking the thread.
> The result is that a client implementation can choose to be optimally
> efficient and manage a local cache of cluster state and attempt to always
> direct its requests to the proper server OR it can choose simplicity and just
> send things all to a single host and let that host figure out where to
> forward it.
> I actually think we should implement this more or less across the board, but
> some requests such as produce and fetch require more logic to proxy since
> they have to be scattered out to multiple servers and gathered back to create
> the response. So these could be done in a second phase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
