Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-11-26 Thread Christophe JAILLET
Le 04/10/2015 12:10, minf...@apache.org a écrit : Author: minfrin Date: Sun Oct 4 10:10:51 2015 New Revision: 1706669 URL: http://svn.apache.org/viewvc?rev=1706669&view=rev Log: core: Extend support for asynchronous write completion from the network filter to any connection or request filter.

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 7:29 PM, Graham Leggett wrote: > I am wondering if the EOR bucket is a suboptimal way to clean up after > ourselves. > > The MPM itself is in a position to know when the filter buffers are all empty > and it is safe to delete a request pool. I am imagining a “shutdown set”,

AW: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Plüm , Rüdiger , Vodafone Group
> -Ursprüngliche Nachricht- > Von: Yann Ylavic [mailto:ylavic@gmail.com] > Gesendet: Dienstag, 6. Oktober 2015 17:54 > An: dev@httpd.apache.org > Betreff: Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ > modules/http/ modules/ssl/ server/ server/mpm

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Jim Jagielski
> On Oct 6, 2015, at 1:29 PM, Graham Leggett wrote: > > On 06 Oct 2015, at 7:00 PM, Yann Ylavic wrote: > >> Yet another filter which does not remove itself after the EOS, and >> destroys the EOR bucket while still using *r after. > > I am wondering if the EOR bucket is a suboptimal way to cle

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 7:00 PM, Yann Ylavic wrote: > Yet another filter which does not remove itself after the EOS, and > destroys the EOR bucket while still using *r after. I am wondering if the EOR bucket is a suboptimal way to clean up after ourselves. The MPM itself is in a position to know wh

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 7:00 PM, Yann Ylavic wrote: > On Tue, Oct 6, 2015 at 6:29 PM, Yann Ylavic wrote: >> >> all tests pass now. > > I spoke too soon... > > New segfault is in mod_substitute: Fixed in r1707091. Now all tests really pass :p I wonder how we should address the possible (third-par

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 6:29 PM, Yann Ylavic wrote: > > all tests pass now. I spoke too soon... New segfault is in mod_substitute: Program terminated with signal 11, Segmentation fault. (gdb) bt #0 0x7fa02fbaec2b in allocator_free (allocator=0x1cedcb0, node=0x0) at memory/unix/apr_pools.c:

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Jim Jagielski
+1 (concept) > On Oct 6, 2015, at 11:53 AM, Yann Ylavic wrote: > > On Tue, Oct 6, 2015 at 5:34 PM, Graham Leggett wrote: >> >> apr_bucket_simple_copy() looks wrong - in theory we should have a proper >> copy function that does the right thing with the second copy, for example by >> not copyi

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 5:53 PM, Yann Ylavic wrote: > On Tue, Oct 6, 2015 at 5:34 PM, Graham Leggett wrote: >> >> apr_bucket_simple_copy() looks wrong - in theory we should have a proper >> copy function that does the right thing with the second copy, for example by >> not copying the pool. If w

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 5:34 PM, Graham Leggett wrote: > > apr_bucket_simple_copy() looks wrong - in theory we should have a proper copy > function that does the right thing with the second copy, for example by not > copying the pool. If we blindly copy the pool (or the request containing the >

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 5:22 PM, Yann Ylavic wrote: >> Just did that, same thing. >> I was using mpm_worker, but now tried mpm_event with same segfaults. > > Looks like mod_bucketeer is the culprit. > It fails to remove itself from the filter stack on EOS, and hence > makes a copy of the EOR bucket (

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 3:36 PM, Stefan Eissing wrote: > FYI: /trunk no longer works with mod_http2. 2.4.x does. I see missing > response data, it seems, so the most likely cause are the changes in filter > handling. Did not find the time to investigate further. > > Please be aware the mod_h2 uses

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 5:22 PM, Yann Ylavic wrote: > still the new filter should probably work > with the existing... s/new filter/new filter stack mechanism/

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 3:51 PM, Yann Ylavic wrote: > On Tue, Oct 6, 2015 at 3:41 PM, Graham Leggett wrote: >> On 6 Oct 2015, at 14:32, Yann Ylavic wrote: >> >>> So it seems to relate to the EOR bucket, first not being passed >>> through, and second leading to a double-free or alike. >>> >>> Did

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 3:51 PM, Yann Ylavic wrote: > > Just did that, same thing. > I was using mpm_worker, but now tried mpm_event with same segfaults. FWIW, I'm running an (old) Debian 6.0.10 (squeeze, 64bit).

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 3:41 PM, Graham Leggett wrote: > On 6 Oct 2015, at 14:32, Yann Ylavic wrote: > >> So it seems to relate to the EOR bucket, first not being passed >> through, and second leading to a double-free or alike. >> >> Did not investigate further... > > Did you rebuild you tree comp

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 6 Oct 2015, at 14:32, Yann Ylavic wrote: > So it seems to relate to the EOR bucket, first not being passed > through, and second leading to a double-free or alike. > > Did not investigate further... Did you rebuild you tree completely clean before trying it out? Whenever I got weird behavio

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Stefan Eissing
FYI: /trunk no longer works with mod_http2. 2.4.x does. I see missing response data, it seems, so the most likely cause are the changes in filter handling. Did not find the time to investigate further. Please be aware the mod_h2 uses pool/brigade hierarchies in new and unexpected ways. The requ

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 3:32 PM, Yann Ylavic wrote: > > Just wanted to run the tests framework on trunk for a local change but > it seems that this commit (bisected to) makes many tests to segfault. Forgot to mention, same thing with follow up r1706670 applied.

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Sun, Oct 4, 2015 at 12:10 PM, wrote: > Author: minfrin > Date: Sun Oct 4 10:10:51 2015 > New Revision: 1706669 > > URL: http://svn.apache.org/viewvc?rev=1706669&view=rev > Log: > core: Extend support for asynchronous write completion from the > network filter to any connection or request filt

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 12:36 PM, Graham Leggett wrote: >> How can you be sure that you don't have transient buckets in the brigade >> that point to memory that changed or is invalid, once you reinstate the >> brigade? > > Because it’s a request filter, not a connection filter. Request filters set

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 1:07 PM, Yann Ylavic wrote: > What I meant is that passing an empty brigade to a reinstate-able > filter may make it pass its data to the next filter, so we may want to > do this only on the first reinstate-able filter starting from > r->output_filters. If we did that we creat

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 12:57 PM, Graham Leggett wrote: > > On 06 Oct 2015, at 12:36 PM, Yann Ylavic wrote: >> BTW, I wonder if we can "reinstate" the filters in arbitrary order >> like in the above loop (the order seems to depend on the calls to >> ap_filter_setaside_brigade() and internal apr_ha

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 12:15 PM, Graham Leggett wrote: > On 06 Oct 2015, at 1:39 AM, Yann Ylavic wrote: > >>> + >>> +/** Buffered data associated with the current filter. */ >>> +apr_bucket_brigade *bb; >>> + >>> +/** Dedicated pool to use for deferred writes. */ >>> +apr_pool_t *

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 12:36 PM, Yann Ylavic wrote: >> Although the brigade is empty, don't we need to prevent the brigade being >> used by multiple threads in parallel? >> Using one per connection prevents this. > > Hmm, the filters are also allocated per connection/request (on > c/r->pool), and s

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
- in /httpd/httpd/trunk: ./ include/ >> modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ >> server/mpm/simple/ >> >> On Tue, Oct 6, 2015 at 9:37 AM, Plüm, Rüdiger, Vodafone Group >> wrote: >> > >> >> -Original Message--

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 12:26 PM, Plüm, Rüdiger, Vodafone Group wrote: > How can you be sure that you don't have transient buckets in the brigade that > point to memory that changed or is invalid, once you reinstate the brigade? Because it’s a request filter, not a connection filter. Request filter

RE: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Plüm , Rüdiger , Vodafone Group
> -Original Message- > From: Graham Leggett [mailto:minf...@sharp.fm] > Sent: Dienstag, 6. Oktober 2015 12:16 > To: dev@httpd.apache.org > Subject: Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ > modules/http/ modules/ssl/ server/ server/mpm/event/

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 11:13 AM, Yann Ylavic wrote: > By defining this function in util_filter > (ap_filter_reinstate_conn()?), we also possibly could get rid of > c->empty (which looks quite hacky), using/hidding an empty brigade per > filter (in opaque data). That starts multiplying the RAM really

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Graham Leggett
On 06 Oct 2015, at 1:39 AM, Yann Ylavic wrote: >> + >> +/** Buffered data associated with the current filter. */ >> +apr_bucket_brigade *bb; >> + >> +/** Dedicated pool to use for deferred writes. */ >> +apr_pool_t *deferred_pool; > > Could we make these opaque, eg: > > # util_f

RE: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Plüm , Rüdiger , Vodafone Group
> -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Dienstag, 6. Oktober 2015 11:13 > To: dev@httpd.apache.org > Subject: Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ > modules/http/ modules/ssl/ server/ server/mpm/event/

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 9:37 AM, Plüm, Rüdiger, Vodafone Group wrote: > >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> >> > Modified: httpd/httpd/trunk/server/mpm/event/event.c >> > URL: >> http://svn.apache.org/viewvc/httpd/httpd/trunk/server/mpm/event/event.c?

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Yann Ylavic
On Tue, Oct 6, 2015 at 9:37 AM, Plüm, Rüdiger, Vodafone Group wrote: > >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> >> Also it seems that we leak the hash iterator here (on c->pool). > > Why do we leak here? apr_hash_first(NULL, is used. So no new iterator is > allocated. Correct, I tho

RE: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-06 Thread Plüm , Rüdiger , Vodafone Group
> -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Dienstag, 6. Oktober 2015 01:40 > To: dev@httpd.apache.org > Subject: Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ > modules/http/ modules/ssl/ server/ server/mpm/event/

Re: svn commit: r1706669 - in /httpd/httpd/trunk: ./ include/ modules/http/ modules/ssl/ server/ server/mpm/event/ server/mpm/motorz/ server/mpm/simple/

2015-10-05 Thread Yann Ylavic
Thanks Graham for this great patch, comments inline... On Sun, Oct 4, 2015 at 12:10 PM, wrote: > Author: minfrin > Date: Sun Oct 4 10:10:51 2015 > New Revision: 1706669 > > URL: http://svn.apache.org/viewvc?rev=1706669&view=rev > Log: > core: Extend support for asynchronous write completion fro