[
https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Jiang updated THRIFT-1167:
--------------------------------
Attachment: threadedselectorthrift3.diff
Sure, this is an architecture similar to that used by other high-throughput
servers. Here's a brief summary of the changes from HsHa:
THsHaServer has an async selector thread that is responsible for accepting new
connections and selecting for IO, reading until full request frames are
buffered. It then hands off processing to a worker pool (executor) to service
the requests.
The threaded selector server splits the role of the selector thread into
multiple threads:
An accept thread responsible for accepting connections on the listen socket and
handing them off to Selector threads
A collection of (rather than one) Selector threads each responsible for a
subset of connected client sockets.
The worker pool (Executor) remains, handling invocations once requests have
been read.
This solves two potential issues with HsHa:
1. In an RPC-heavy app server where the thrift processor does relatively little
work (e.g. a service for generating UUIDs), a significant portion of CPU time
is spent handling selection on sockets and doing reading/writing from the
network. The HsHa server handles selection on all sockets and IO on a single
thread, which on a multi-core platform limits the request throughput to a
single core. This leaves the worker pool that can run on the remaining cores
underutilized. Having socket selection and IO split across multiple threads
increases maximum throughput in these situations.
2. The HsHa server accepts connections on the selector thread as fast as
possible, even when the executor service is saturated and starts throwing
RejectedExecutionException. In this case connections are closed whenever
exceptions are thrown (after a request has already been read on the connection)
potentially causing a lot of flapping. The FAIR_ACCEPT accept policy in the new
server submits the accept socket handoff to the same executor service that
handles invocations, which results in a close of a new connection immediately
if the executor is saturated.
A potential downside to this design is there are additional context switches
because accept happens on a separate thread.
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for
> accept and a separate, configurable number of Selector threads to handle IO.
> I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor
> pool running on multi-core architectures, this server allows gets throughput
> as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
