RFC: nonblocking rewrite of ap_core_output_filter

[Brian Pane]

I just committed a new version of ap_core_output_filter() to the  
async-dev branch.
The semantics of the filter have changed in a fundamental way: the  
original version
does blocking writes and usually tries to write out all of the  
available data before
writing (except in cases where it buffers data to avoid small  
writes), whereas this

new version does nonblocking writes in most cases.

The goal of the rewrite was to set the stage for a clean  
implementation of

asynchronous write completion in the Event and/or Leader/Followers MPMs.
The nonblocking behavior, however, appears to be useful in all MPMs.   
For

example, nonblocking writes can enable mod_include to parse the next
bunch of output while awaiting an ack from the client.  To avoid  
infinite
memory usage, the new filter does blocking writes if it has >= 64KB  
of data

buffered up.

There are some significant differences from the earlier nonblocking  
output
filter that I posted a few weeks ago as part of the Event MPM async  
write

completion patch:

- If a nonblocking write attempt results in EAGAIN, the new filter  
returns

  APR_SUCCESS instead of APR_EAGAIN.  The old patch broke too much
  existing code that wasn't prepared for EAGAIN.  The new filter can  
return

  APR_SUCCESS without having actually written the entire brigade, but
  that's also true of the original ap_core_output_filter(), with its  
various

  setaside cases.

- The new filter doesn't try to ignore flush buffers like the earlier  
patch did.
  Instead, when it encounters a flush bucket, it does a blocking  
write of
  everything up to that point.  The earlier patch tried to detect  
certain

  patterns of buckets involving flush, EOS, and EOC that could be
  interpreted as "hand this data off to the write completion thread  
instead
  of actually writing it out immediately."  But that logic was too  
brittle, as

  it depended on knowledge of the bucket patterns that the httpd core
  just happens to produce.  In the new design, the core output filter
  interprets a flush bucket as "write this data out before  
returning."  To

  implement async write completion on top of this, we'll likely have to
  remove some of the points in the core that generate flush buffers--
  but that's a project for another day.

There are a few things missing in the new version:

- It doesn't concatenate sequences of really small buckets together  
the way
  the original does.  If you send it a brigade containing 16 single- 
byte buckets,
  it will do a writev of 16 bytes.  My inclination is to leave this  
"broken" in order
  to keep the code simple, unless someone has a real-world use case  
that

  produces such output.

- I haven't yet put in mod_logio support.  This shouldn't be  
difficult to add, but
  I want to do some experiments with a new design: sending an End-Of- 
Request
  bucket that calls the request logger when all the buckets in front  
of it have
  been written to the network.  If this works, the "EOR" bucket's  
destroy function

  might end up being the cleanest place to call the logio hooks.

- It doesn't yet do nonblocking reads on socket buckets.  Can anyone  
recommend

  a good test case that make use of socket buckets?

Thanks,
Brian

Errors in latest beta

[Nick]

For some reason I can no longer execute a function registered with 
req.register_cleanup:


[Sat Sep 24 11:56:18 2005] [error] [client 127.0.0.1] 
exceptions.UnboundLocalError: 1124096352
[Sat Sep 24 11:56:18 2005] [error] [client 127.0.0.1] python_cleanup: 
Error calling cleanup object 


I haven't had time to dig into the mod_python code yet, and I'll be 
somewhat busy this weekend, but I want to give a heads up.


I don't rule out the possiblity that it's because of my code ;-)  It did 
worked before I upgraded, though.  Basically I'm calling like so:


req.register_cleanup(pse_cleanuphandler, req)

Nick

Re: which apr(-util) versions for 2.2.x?

[Paul Querna]

Jeff Trawick wrote:

On 9/24/05, André Malo <[EMAIL PROTECTED]> wrote:

I've probably missed that info...

Which branches of apr are to be used with 2.2.x? Or is it just trunk?


(Just how somebody can tell me how out of touch I am)

1.0 or above.  And if it doesn't work, there's a missing version check.


Actually, we are getting to the point were I would rather just have a 
hard depends on APR >= 1.2.0


-Paul

Re: which apr(-util) versions for 2.2.x?

[Jeff Trawick]

On 9/24/05, André Malo <[EMAIL PROTECTED]> wrote:
> I've probably missed that info...
>
> Which branches of apr are to be used with 2.2.x? Or is it just trunk?

(Just how somebody can tell me how out of touch I am)

1.0 or above.  And if it doesn't work, there's a missing version check.

which apr(-util) versions for 2.2.x?

[André Malo]

I've probably missed that info...

Which branches of apr are to be used with 2.2.x? Or is it just trunk?

nd
-- 
"Das Verhalten von Gates hatte mir bewiesen, dass ich auf ihn und seine
beiden Gefährten nicht zu zählen brauchte" -- Karl May, "Winnetou III"

Im Westen was neues: