Looking at the pattern of calls to ap_core_output_filter() in the
event MPM,
it occurred to me that it may be straightforward to hand off the
writing of the
request to an async completion thread in a lot of useful real-world
cases.
In function check_pipeline_flush() in http_request.c, a flush bucket
is passed
to the output filter stack if there's no pipelined request ready to
be read (or if
keep-alives aren't enabled for the connection). When this occurs,
two things
are true:
* One or more complete responses--and no partial responses--have been
sent to the output filter stack for this connection.
* If these responses have not been sent to the client, they will
now be sent
in their entirety before the httpd attempts to do any further
reads from this
connection.
Instead of sending a flush bucket to the output filter stack in this
scenario,
though, perhaps we could send a new metadata bucket that denotes "end
of output until everything currently in the pipeline has been
written." Upon
receiving this bucket, core_output_filter would register the
connection with
a pollset to watch for writability.
A write completion thread would poll in an endless loop, writing data
from
each set-aside brigade whenever its corresponding connection became
writable.
Upon sending the last bucket of a set-aside brigade to the network,
the completion
thread would put the connection in either CONN_STATE_LINGER or
CONN_STATE_CHECK_REQUEST_LINE_READABLE state (depending on
whether keepalives were enabled) and re-register the connection with the
event MPM's main pollset to wait for readability or timeout.
The pollset used to wait for writability could be the same pollset
that's currently
used to watch for readability and timeouts. Similarly, the write
completion
thread could be combined with the current listener thread.
I'm eager to hear some feedback on this idea:
* Will it work? Or am I overlooking some design flaw?
* Will it help? I.e., will it make a noticeable improvement in the
connections-to-threads ratio?
* What about the access logger? If the writing of responses were
completed asynchronously, a request could be logged before its
response was sent--or, worse yet, before the sending of its
response failed due to, say, the client closing the connection early.
One solution that comes to mind is to have the write completion
thread invoke the access logger--perhaps triggered by a metadata
bucket in the output brigade.
-Brian
