So what you are encountering is limitations in the socket buffer size enforced by the operating system, in combination with Apache httpd applying a socket timeout.
In other words what happens is that the HTTP client isn't reading data and so the operating system level socket buffer fills up. At that point the Apache httpd write of the response data blocks with it eventually timing out, causing the initial error you see. In that situation Apache httpd will close down the connection, which results in you seeing the second error when still trying to write out more data anyway. You may be able to adjust some Apache configuration settings to try and solve this, but it would affect all requests in the context which you apply the configuration (dependent on whether done in server, VirtualHost, Directory or Location contexts). So not something you could selectively do on a per client basis. The first Apache directive to look at is SendBufferSize. https://httpd.apache.org/docs/2.4/mod/mpm_common.html#sendbuffersize If this is not set it should default to 0, which means it uses the operating system default. So you might be able to fiddle with this by setting it larger than the operating system default (although there is still some upper bound set by operating system you can go to). The next Apache directive to look at is Timeout. https://httpd.apache.org/docs/2.4/mod/core.html#timeout This would usually default to 60 seconds but some Linux distributions may override this in the Apache configuration they ship. In very old Apache versions this actually defaulted to 300 seconds, but it was made lower at some point. If playing with these, do be careful since they cause increased memory usage or cause other undesirable effects depending on traffic profile your server gets. One other thing you may be able to use is mod_ratelimit. https://httpd.apache.org/docs/2.4/mod/mod_ratelimit.html I have never actually used this and not exactly sure how it works, so is a bit of a guess, but you may be able to use this to slow down how quickly your application outputs the data. I am assuming here that this module will introduce waits into your application, by blocking your writes for a bit, to keep the flow of data being written by it under the rate limit. This would have the effect of not stuffing so much data into the response pipeline such that things work better with slower clients. Obviously using it would though penalise faster clients but you might find an acceptable balance by setting a higher rate limit for the initial burst of data and then using a lower rate after that. Graham > On 11 Jan 2024, at 7:09 am, Greg Popp <pop...@gmail.com> wrote: > > embedded > > On Wednesday, January 10, 2024 at 1:32:52 PM UTC-6 Graham Dumpleton wrote: >> Are you using mod_wsgi embedded mode or daemon mode? >> >> Graham >> >> >>> On 11 Jan 2024, at 2:44 am, Greg Popp <pop...@gmail.com <>> wrote: >>> >> >>> Hello! >>> >>> My version of mod_wsgi is running on a Centos-7 system and is at version >>> 3.4, (I know - very old) with python 2.7 >>> >>> I have been using mod_wsgi for a python application that runs a >>> command-line program and marshals the output of the command line program >>> back to an http client. The data being sent is binary and can be tens of >>> gigs in size. >>> >>> This app is "unconventional", in that it calls 'write' directly, instead of >>> returning an iterable. The problem I have had recently, is that some >>> clients are slow to read the data and the TCP buffer gets filled up. When >>> this happens, the next call to write on a full buffer causes a "failed to >>> write data" exception (which I trap) but if I try again to send the data I >>> get "client connection closed". >>> >>> Is there some config setting or methodology I can use to alleviate this >>> issue? In other words, some way to back off and wait for the buffer to >>> drain sufficiently to resume sending the data? OR - is there some way to >>> get the current size (fullness) of the TCP write buffer on the connected >>> socket? (Something like what you see from the 'ss' command line utility >>> "Send-Q" column). If I could tell how full it is and what the max size is, >>> I could implement a sleep/retry cycle of some kind. >>> >>> I have looked - even in the source code - but haven't been able to figure >>> it out if there is a way to achieve this. Thanks in advance, for your >>> attention. >>> >>> >>> >> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to modwsgi+u...@googlegroups.com <>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/modwsgi/3d97c06f-38ff-4345-af2f-eb86c2ef204cn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/modwsgi/3d97c06f-38ff-4345-af2f-eb86c2ef204cn%40googlegroups.com?utm_medium=email&utm_source=footer>. >> > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to modwsgi+unsubscr...@googlegroups.com > <mailto:modwsgi+unsubscr...@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/modwsgi/9fc6ab3e-b791-4503-a3c0-20ba273b92bdn%40googlegroups.com > > <https://groups.google.com/d/msgid/modwsgi/9fc6ab3e-b791-4503-a3c0-20ba273b92bdn%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/9F210D34-82EC-4AE7-A76E-B178782F4B3A%40gmail.com.
