2017-07-10 11:42 GMT+02:00 Luca Toscano <toscano.l...@gmail.com>:
> Hi Rüdiger,
>
> 2017-07-10 8:31 GMT+02:00 Plüm, Rüdiger, Vodafone Group <
> ruediger.pl...@vodafone.com>:
>
>>
>>
>>
>>
>> *Von:* Luca Toscano [mailto:toscano.l...@gmail.com]
>> *Gesendet:* Samstag, 8. Juli 2017 09:52
>> *An:* Apache HTTP Server Development List <dev@httpd.apache.org>
>> *Betreff:* Re: mod_proxy_fcgi and flush
>>
>>
>>
>> Hi Jacob, Helmut!
>>
>>
>>
>> 2017-07-06 20:54 GMT+02:00 Jacob Champion <champio...@gmail.com>:
>>
>> On 07/06/2017 11:13 AM, Jim Jagielski wrote:
>>
>> works 4 me...
>>
>>
>> Doesn't for me. E.g. with a script like
>>
>> <?php
>> print("hi!\n")
>> flush();
>> sleep(1);
>> print("hi!\n");
>> ?>
>>
>> it takes 1 second to receive a single chunk with both lines in it.
>>
>> From a quick skim I assume this is because we don't use nonblocking
>> sockets in the proxy implementation. (There's even a note in mod_proxy_fcgi
>> that says, "Yes it sucks to [get the actual data] in a second recv call,
>> this will eventually change when we move to real nonblocking recv calls.")
>>
>>
>>
>> Quick check from my side too, so not sure if it makes sense. IIUC the
>> comment is about getting all the headers from the FCGI backend
>> (get_data_full(..., AP_FCGI_HEADER_LEN)), then get the response body in
>> another one (the [actual data]).
>>
>>
>>
>> I checked mod_fcgi as Helmut suggested and it seems to me that the -flush
>> feature is a simple "flush every data that you receive", so I tested the
>> following patch with Jacob's php example code and it seems doing what
>> Helmut asked (please correct me if I am wrong).
>>
>>
>>
>> Caveat: I had to set output_buffering = Off in my php-fpm's php.ini
>> config file.
>>
>>
>>
>> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-
>> force_flush.patch
>>
>>
>>
>> Flushing unconditionally is a bad idea performance wise. Please have a
>> look how ajp solved this issue:
>>
>>
>>
>> https://svn.apache.org/r327185
>>
>> https://svn.apache.org/r384580
>>
>> https://svn.apache.org/r390210
>>
>> https://svn.apache.org/viewvc/httpd/httpd/trunk/modules/prox
>> y/mod_proxy_ajp.c?r1=325879&r2=390210
>>
>>
>>
>> Hint: The above diff contains further unrelated changes.
>>
>>
>>
> Definitely, I added that quick and dirty patch to show my idea, really far
> from being ready for a commit :)
> Will review the commits that you posted, thanks!
>
With a bit of delay I have something to share to the list :)
I kept the following test as the "use case":
<?php
print("hi!\n")
flush();
sleep(1);
print("hi!\n");
?>
First of all, I tested it with httpd-trunk and it seems working fine thanks
to several core non-blocking writes, removing the need of explicit FLUSH
buckets. I am very ignorant about this part of httpd so I can't say much
more, but it is really nice to see!
I then checked Rüdiger's commits and this is my understanding about mod_ajp:
- if "flushpackets" is set to "on" by mod_proxy then mod_proxy_ajp will
insert a FLUSH bucket as soon as data is ready to be passed to the output
filter chain.
- if "flushpackets" is set to "auto" then mod_proxy_ajp will be a bit
smarter and will apr_poll for "flushwait" seconds before inserting any
FLUSH bucket, since new data might come from the backend and it would be
more efficient to force a FLUSH with more data.
IIUC mod_proxy_fcgi leverages the fact that the backend sends FCGI records
with a predefined structure (FCGI_Record), containing "FCGI headers"
(content len, padding len, version, type, etc..) that can be used to know
how much data to expect from the backend. As far as I can see (with
php-fpm) our use case is handled with multiple FCGI records, one for each
"flush()" done in the php script. mod_proxy_fcgi does the following for
each of them:
1) read the FCGI headers to gather how much data the record is carrying
(clen)
2) read the data in iobuf_size batches, sending each time the bytes
collected to the output filter chain.
3) finally read the padding bytes (if any).
I tried to use the same trick as mod_ajp for flushpackets=auto, but
apr_poll returned immediately most of the times due to the padding bytes
ready to read (took me a while to realize this simple thing).
I didn't find a real use case for flushpackets=auto in mod_proxy_fcgi, but
I have two proposals for fluspackets=on:
http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush.patch
http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush-v2.patch
The second one is a bit more conservative and does not insert the FLUSH
bucket if it knows that the content len (clen) is bigger than the
iobuf_size (since there is a goto that forces another get_data() in this
case).
Let me know if what I wrote make sense, suggestions and comments are
welcome. It would be interesting to also know from Helmut a simple (PHP?)
use case to use as a test, so I'll not base my code only on Jacob's example
(that was really useful though!!).
Thanks!
Luca
