[we really should move this to the [EMAIL PROTECTED] list]
On Fri, Dec 12, 2003 at 11:53:53AM +, Ben Laurie wrote:
> This was exactly the conversation we were having at the hackathon. As
> always, Windows was the problem, but I thought Bill had it licked?
Well, there are two things we have t
On Thu, Dec 11, 2003 at 10:35:28PM -0600, William A. Rowe, Jr. wrote:
> I've been thinking the same thing driving around this evening...
>
> One major goal of httpd-3.0 is *finally* arriving at something that starts
> looking async. We've kicked it around some time, but perhaps it's time
> to sta
Aaron Bannert wrote:
On Thu, Dec 11, 2003 at 01:50:46PM -0600, William A. Rowe, Jr. wrote:
But the 2.0 architecture is entirely different. We need a poll but it's not entirely
obvious where to put one...
One suggestion raised in a poll bucket: when a connection level filter cannot
read anything
I've been thinking the same thing driving around this evening...
One major goal of httpd-3.0 is *finally* arriving at something that starts
looking async. We've kicked it around some time, but perhaps it's time
to start looking at the async poll implementation, to get some idea of how
we can 'pol
On Thu, Dec 11, 2003 at 01:50:46PM -0600, William A. Rowe, Jr. wrote:
> But the 2.0 architecture is entirely different. We need a poll but it's not entirely
> obvious where to put one...
>
> One suggestion raised in a poll bucket: when a connection level filter cannot
> read anything more, it pas
[snip]
(wrowe's exposition of a possible non-blocking filter chain implementation)
Your poll bucket idea would be welcome for input, although it would
only save a bit of work since we already have APR_NONBLOCK_READ
apr_bucket_read(b, &bdata, &blen, APR_NONBLOCK_READ);
For polling output, I had
At 07:01 PM 12/10/2003, Bill Stoddard wrote:
>Aaron Bannert wrote:
>>
>>[slightly off-topic]
>>Actually, I believe that mod_cgi and mod_cgid are currently broken
>>WRT the CGI spec. The spec says that a CGI may read as much of an
>>incoming request body as it wishes and may return data as soon as
>
On Thu, 2003-12-11 at 09:54, Stas Bekman wrote:
> Sander Striker wrote:
> > On Thu, 2003-12-11 at 07:44, Stas Bekman wrote:
> >
> >
> >>I now know what the problem is. It is not a problem in httpd or its filters,
> >>but mod_perl, allocated filter struct from the pool. With many bucket brigades
Sander Striker wrote:
On Thu, 2003-12-11 at 07:44, Stas Bekman wrote:
I now know what the problem is. It is not a problem in httpd or its filters,
but mod_perl, allocated filter struct from the pool. With many bucket brigades
there were many filter invocations during the same request, resulting
On Thu, 2003-12-11 at 07:44, Stas Bekman wrote:
> I now know what the problem is. It is not a problem in httpd or its filters,
> but mod_perl, allocated filter struct from the pool. With many bucket brigades
> there were many filter invocations during the same request, resulting in
> multiple m
Stas Bekman wrote:
I'm debugging the issue. I have a good test case, having a response
handler sending 1byte followed by rflush in a loop creates lots of
buckets. I can see that each iteration allocates 40k. i.e. each new
bucket brigade and its bucket demand 40k which won't be reused till the
n
I'm debugging the issue. I have a good test case, having a response handler
sending 1byte followed by rflush in a loop creates lots of buckets. I can see
that each iteration allocates 40k. i.e. each new bucket brigade and its bucket
demand 40k which won't be reused till the next request. This ha
On Wed, 10 Dec 2003, Brian Pane wrote:
> > But doesn't unsetting the C-L header cause the C-L filter to
> > automatically attempt to generate a new C-L value?
>
> Not unless the C-L filter sees the entire response in the first brigade
> passed through it. It used to buffer the entire response in
On Dec 10, 2003, at 5:15 PM, Cliff Woolley wrote:
On Wed, 10 Dec 2003, Stas Bekman wrote:
But doesn't unsetting the C-L header cause the C-L filter to
automatically
attempt to generate a new C-L value?
Not unless the C-L filter sees the entire response in the first brigade
passed through it. It
On Wed, 10 Dec 2003, Stas Bekman wrote:
> > But doesn't unsetting the C-L header cause the C-L filter to automatically
> > attempt to generate a new C-L value?
>
> I thought that bug has been fixed long time ago. Dynamic handlers used to bump
Ryan would know. :-)
On Wed, 10 Dec 2003, William A. Rowe, Jr. wrote:
> Is it Chrises' own filter or one of ours? whichever it is, it would be nice to
> get this fixed.
Can I suggest Chris insert mod_diagnostics at different points in his
chain to identify exactly where it's buffering (if indeed that's where
his mem
Cliff Woolley wrote:
On Wed, 10 Dec 2003, Stas Bekman wrote:
No, there is no C-L header. The complete filter looks like so:
sub handler {
# Get the filter object
my($f) = @_;
# Only done on the FIRST pass of the filter
unless($f->ctx)
{
$f-
On Wed, 10 Dec 2003, Stas Bekman wrote:
> No, there is no C-L header. The complete filter looks like so:
>
> sub handler {
> # Get the filter object
> my($f) = @_;
>
> # Only done on the FIRST pass of the filter
> unless($f->ctx)
> {
> $f->r->headers_out
Cliff Woolley wrote:
On Thu, 11 Dec 2003, Ian Holsman wrote:
do the server's reply to you have a content-length header?
if so .. this is probably what is holding up the request in the server.
Yah, I was going to guess it was probably the C-L filter. But I thought
we had logic in the C-L filter
Aaron Bannert wrote:
On Wed, Dec 10, 2003 at 06:29:28PM -0500, Glenn wrote:
On Wed, Dec 10, 2003 at 03:18:44PM -0800, Stas Bekman wrote:
Are you saying that if I POST N MBytes of data to the server and just have
the server send it back to me, it won't grow by that N MBytes of memory for
the du
On Thu, 11 Dec 2003, Ian Holsman wrote:
> do the server's reply to you have a content-length header?
> if so .. this is probably what is holding up the request in the server.
Yah, I was going to guess it was probably the C-L filter. But I thought
we had logic in the C-L filter to avoid buffering
Cliff Woolley wrote:
Which is exactly what is supposed to happen.
Obviously it's not how things work at the moment, as the memory is never
freed (which could probably be dealt with), but the real problem is that
no data will leave the server out before it was completely read in.
Yes, that woul
Title: RE: filtering huge request bodies (like 650MB files)
Bug 24991 that I just fixed yesterday dealt with a memory leak when reverse proxying HTTP POST requests. The fix was done in mod_proxy_http and the patch has been submitted. I'm not sure if this is the cause of the leak tha
William A. Rowe, Jr. wrote:
At 04:57 PM 12/10/2003, Cliff Woolley wrote:
On Wed, 10 Dec 2003, Stas Bekman wrote:
Obviously it's not how things work at the moment, as the memory is never
freed (which could probably be dealt with), but the real problem is that
no data will leave the server out bef
On Wed, Dec 10, 2003 at 06:29:28PM -0500, Glenn wrote:
> On Wed, Dec 10, 2003 at 03:18:44PM -0800, Stas Bekman wrote:
> > Are you saying that if I POST N MBytes of data to the server and just have
> > the server send it back to me, it won't grow by that N MBytes of memory for
> > the duration of
On Wed, Dec 10, 2003 at 05:23:14PM -0600, William A. Rowe, Jr. wrote:
>...
> It's NOT the proxy - I've been through it many times - and AFAICT we have
> a simple leak in that we don't reuse the individual pool buckets, so memory
> creeps up over time. It isn't even the end of the world, until some
At 04:57 PM 12/10/2003, Cliff Woolley wrote:
>On Wed, 10 Dec 2003, Stas Bekman wrote:
>
>> Obviously it's not how things work at the moment, as the memory is never
>> freed (which could probably be dealt with), but the real problem is that
>> no data will leave the server out before it was complete
On Wed, Dec 10, 2003 at 03:18:44PM -0800, Stas Bekman wrote:
> Are you saying that if I POST N MBytes of data to the server and just have
> the server send it back to me, it won't grow by that N MBytes of memory for
> the duration of that request? Can you pipe the data out as it comes in? I
> th
Cliff Woolley <[EMAIL PROTECTED]> writes:
[...]
> Yes, that would be the real problem. So somewhere there is a filter
> (or maybe the proxy itself) buffering the entire data stream before
> sending it. That is a bug.
>From the proxy_http.c source in HEAD, it looks to me like
mod_proxy buffers
Cliff Woolley wrote:
On Wed, 10 Dec 2003, Stas Bekman wrote:
Chris is trying to filter a 650MB file coming in through a proxy. Obviously he
sees that httpd-2.0 is allocating > 650MB of memory, since each bucket will
use the request's pool memory and won't free it untill after the request is
over.
On Wed, 10 Dec 2003, Stas Bekman wrote:
> Chris is trying to filter a 650MB file coming in through a proxy. Obviously he
> sees that httpd-2.0 is allocating > 650MB of memory, since each bucket will
> use the request's pool memory and won't free it untill after the request is
> over.
Whoa. Obvio
Chris is trying to filter a 650MB file coming in through a proxy. Obviously he
sees that httpd-2.0 is allocating > 650MB of memory, since each bucket will
use the request's pool memory and won't free it untill after the request is
over. Now even if his machine was able to deal with one such requ
32 matches
Mail list logo
