RE: Better SO_REUSEPORT
Great!! Thank you very much for sharing this with us! Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Thursday, October 08, 2015 10:46 AM To: dev@httpd.apache.org Cc: Lu, Yingqi Subject: Better SO_REUSEPORT Looks like we can do even better/faster with it (and latest Linux kernels), soon :) https://www.mail-archive.com/netdev@vger.kernel.org/msg81804.html Promising!
Better SO_REUSEPORT
Looks like we can do even better/faster with it (and latest Linux kernels), soon :) https://www.mail-archive.com/netdev@vger.kernel.org/msg81804.html Promising!
Re: SO_REUSEPORT
On Fri, Jun 5, 2015 at 5:11 PM, Eric Covener wrote:
> I'm trying to review & understand how this affects process management for
> things like MinSpareThreads/MaxSpareThreads e.g.
>
> -else if (idle_thread_count < min_spare_threads) {
> +else if (idle_thread_count < min_spare_threads / num_buckets) {
> /* terminate the free list */
> if (free_length == 0) { /* scoreboard is f
>
> -if (idle_thread_count > max_spare_threads) {
> +if (idle_thread_count > max_spare_threads / num_buckets) {
> /* Kill off one child */
>
> If I wanted between 100 and 200 spare threads available to do some slow
> stuff like proxy or CGI, would we really scale it back by the # of buckets
> or am I misunderstanding?
perform_idle_server_maintenance() is now called for each bucket in the
main loop, so this won't change the number of threads maintained per
loop.
Re: SO_REUSEPORT
I'm trying to review & understand how this affects process management for
things like MinSpareThreads/MaxSpareThreads e.g.
-else if (idle_thread_count < min_spare_threads) {
+else if (idle_thread_count < min_spare_threads / num_buckets) {
/* terminate the free list */
if (free_length == 0) { /* scoreboard is f
-if (idle_thread_count > max_spare_threads) {
+if (idle_thread_count > max_spare_threads / num_buckets) {
/* Kill off one child */
If I wanted between 100 and 200 spare threads available to do some slow
stuff like proxy or CGI, would we really scale it back by the # of buckets
or am I misunderstanding?
On Sun, May 17, 2015 at 4:31 PM Lu, Yingqi wrote:
> Hi Yann,
>
> Thank you very much for your help!
>
> Yingqi
>
> -Original Message-
> From: Yann Ylavic [mailto:ylavic@gmail.com]
> Sent: Saturday, May 16, 2015 3:37 AM
> To: httpd
> Subject: Re: SO_REUSEPORT
>
> On Fri, May 15, 2015 at 5:12 PM, Jeff Trawick wrote:
> > On Fri, May 15, 2015 at 11:02 AM, William A Rowe Jr >
> > wrote:
> >>
> >> My chief concern was that the phrase "Common Log" has a specific meaning
> >> to us.
> >>
> >> ap_mpm_common_log_startup() or something else descriptive would be a
> >> better name, but our crew is famous for not being terrific namers of
> things
> >> :)
> >>
> >> Did this compile with no warnings? It seems statics were used without
> >> being explicitly initialized, and I don't have my copy of K&R to check
> that
> >> these are always NULL, but guessing that's so.
> >
> >
> > yes; but ISTR that NetWare is the reason for explicit initialization in
> some
> > of our multi-platform code; I dunno the rhyme
>
> s/ap_log_common/ap_log_mpm_common/ in r1679714 and added to backport
> proposal.
>
> Regarding globals/statics explicit initializations (implicit
> initialization to {0} is required by the C standard), I don't think it
> is necessary/suitable since and it may move these variables from the
> .bss to the .data section, the former being quicker to initialize (as
> a whole) at load time (though explicit initializations to {0} usually
> go to .bss too but it depends on the compiler and/or flags, and we
> don't explicitely need .data for those).
> So I did not change the code wrt this...
>
RE: SO_REUSEPORT
Hi Yann,
Thank you very much for your help!
Yingqi
-Original Message-
From: Yann Ylavic [mailto:ylavic@gmail.com]
Sent: Saturday, May 16, 2015 3:37 AM
To: httpd
Subject: Re: SO_REUSEPORT
On Fri, May 15, 2015 at 5:12 PM, Jeff Trawick wrote:
> On Fri, May 15, 2015 at 11:02 AM, William A Rowe Jr
> wrote:
>>
>> My chief concern was that the phrase "Common Log" has a specific meaning
>> to us.
>>
>> ap_mpm_common_log_startup() or something else descriptive would be a
>> better name, but our crew is famous for not being terrific namers of things
>> :)
>>
>> Did this compile with no warnings? It seems statics were used without
>> being explicitly initialized, and I don't have my copy of K&R to check that
>> these are always NULL, but guessing that's so.
>
>
> yes; but ISTR that NetWare is the reason for explicit initialization in some
> of our multi-platform code; I dunno the rhyme
s/ap_log_common/ap_log_mpm_common/ in r1679714 and added to backport proposal.
Regarding globals/statics explicit initializations (implicit
initialization to {0} is required by the C standard), I don't think it
is necessary/suitable since and it may move these variables from the
.bss to the .data section, the former being quicker to initialize (as
a whole) at load time (though explicit initializations to {0} usually
go to .bss too but it depends on the compiler and/or flags, and we
don't explicitely need .data for those).
So I did not change the code wrt this...
Re: SO_REUSEPORT
On Fri, May 15, 2015 at 5:12 PM, Jeff Trawick wrote:
> On Fri, May 15, 2015 at 11:02 AM, William A Rowe Jr
> wrote:
>>
>> My chief concern was that the phrase "Common Log" has a specific meaning
>> to us.
>>
>> ap_mpm_common_log_startup() or something else descriptive would be a
>> better name, but our crew is famous for not being terrific namers of things
>> :)
>>
>> Did this compile with no warnings? It seems statics were used without
>> being explicitly initialized, and I don't have my copy of K&R to check that
>> these are always NULL, but guessing that's so.
>
>
> yes; but ISTR that NetWare is the reason for explicit initialization in some
> of our multi-platform code; I dunno the rhyme
s/ap_log_common/ap_log_mpm_common/ in r1679714 and added to backport proposal.
Regarding globals/statics explicit initializations (implicit
initialization to {0} is required by the C standard), I don't think it
is necessary/suitable since and it may move these variables from the
.bss to the .data section, the former being quicker to initialize (as
a whole) at load time (though explicit initializations to {0} usually
go to .bss too but it depends on the compiler and/or flags, and we
don't explicitely need .data for those).
So I did not change the code wrt this...
Re: SO_REUSEPORT
On Fri, May 15, 2015 at 11:02 AM, William A Rowe Jr wrote: > My chief concern was that the phrase "Common Log" has a specific meaning > to us. > > ap_mpm_common_log_startup() or something else descriptive would be a > better name, but our crew is famous for not being terrific namers of things > :) > > Did this compile with no warnings? It seems statics were used without > being explicitly initialized, and I don't have my copy of K&R to check that > these are always NULL, but guessing that's so. > yes; but ISTR that NetWare is the reason for explicit initialization in some of our multi-platform code; I dunno the rhyme > For clarity alone, I'd prefer these were initialized like every other > example they were adjacent to. > > > > > On Fri, May 15, 2015 at 7:06 AM, Jim Jagielski wrote: > >> We are very close... I believe wrowe has some somewhat trivial >> reservations about it, but we are awaiting 1 more vote. >> >> Someone want to address wrowes concerns on trunk and patch >> the patch (stuff like naming)? I may have time next week. >> >> > On May 14, 2015, at 7:45 PM, Yann Ylavic wrote: >> > >> > Hi Yingqi, >> > >> > 2 votes already (on 3), it makes its way ;) >> > >> > Regards, >> > Yann. >> > >> > >> > On Fri, May 15, 2015 at 1:00 AM, Lu, Yingqi >> wrote: >> >> Hi All, >> >> >> >> I just want to check if anyone gets chances to check the SO_REUSEPORT >> patch? Any feedback? >> >> >> >> Thanks, >> >> Yingqi >> >> >> >> -Original Message- >> >> From: Lu, Yingqi [mailto:yingqi...@intel.com] >> >> Sent: Friday, May 08, 2015 8:58 AM >> >> To: dev@httpd.apache.org >> >> Subject: RE: SO_REUSEPORT >> >> >> >> Hi Christophe, Jim and Yann, >> >> >> >> Thank you very much for your consideration of putting SO_REUSEPORT >> patch in the 2.4 stable release. >> >> >> >> I am also very happy that you find the white paper :-) All the most >> recent testing results are included in the white paper. Also, we have >> tested the (graceful) restart on the patch (previously, there was a bug.), >> it should be fine now. Please test it to confirm. >> >> >> >> Please let me know if you need anything else. Your help is appreciated. >> >> >> >> Thanks, >> >> Yingqi >> >> >> >> -Original Message- >> >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> >> Sent: Friday, May 08, 2015 5:02 AM >> >> To: httpd >> >> Subject: Re: SO_REUSEPORT >> >> >> >> On Fri, May 8, 2015 at 9:44 AM, Christophe JAILLET < >> christophe.jail...@wanadoo.fr> wrote: >> >>> >> >>> Maybe, 2.4.14 could focus on reviewing/merging this patch and >> >>> associated performance improvement? >> >>> To help adoption, maybe an ASF server could be upgraded with a >> >>> SO_REUSEPORT patched version of Apache to have our own measurements >> >>> and see how it scales in a real world application. >> >> >> >> I did some testing with an injector at the time of the proposal (on a >> 2.2.x version of the patch, so mainly with worker), and can confirm that it >> really scales much better. >> >> Where httpd without SO_REUSEPORT stops accepting/handling connections, >> it continues to shine with the option/buckets enabled. >> >> (I don't have the numbers for now, need to search deeper, btw the ones >> from Intel are probably more of interest...) >> >> >> >> So regarding the upgrade on infra, the difference may not be obvious >> if the tested machine is not "at the limits". >> >> >> >> One thing that probably is worth testing is (graceful) restarts, >> though. >> >> >> >> Regards, >> >> Yann. >> >> > -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: SO_REUSEPORT
My chief concern was that the phrase "Common Log" has a specific meaning to us. ap_mpm_common_log_startup() or something else descriptive would be a better name, but our crew is famous for not being terrific namers of things :) Did this compile with no warnings? It seems statics were used without being explicitly initialized, and I don't have my copy of K&R to check that these are always NULL, but guessing that's so. For clarity alone, I'd prefer these were initialized like every other example they were adjacent to. On Fri, May 15, 2015 at 7:06 AM, Jim Jagielski wrote: > We are very close... I believe wrowe has some somewhat trivial > reservations about it, but we are awaiting 1 more vote. > > Someone want to address wrowes concerns on trunk and patch > the patch (stuff like naming)? I may have time next week. > > > On May 14, 2015, at 7:45 PM, Yann Ylavic wrote: > > > > Hi Yingqi, > > > > 2 votes already (on 3), it makes its way ;) > > > > Regards, > > Yann. > > > > > > On Fri, May 15, 2015 at 1:00 AM, Lu, Yingqi wrote: > >> Hi All, > >> > >> I just want to check if anyone gets chances to check the SO_REUSEPORT > patch? Any feedback? > >> > >> Thanks, > >> Yingqi > >> > >> -Original Message- > >> From: Lu, Yingqi [mailto:yingqi...@intel.com] > >> Sent: Friday, May 08, 2015 8:58 AM > >> To: dev@httpd.apache.org > >> Subject: RE: SO_REUSEPORT > >> > >> Hi Christophe, Jim and Yann, > >> > >> Thank you very much for your consideration of putting SO_REUSEPORT > patch in the 2.4 stable release. > >> > >> I am also very happy that you find the white paper :-) All the most > recent testing results are included in the white paper. Also, we have > tested the (graceful) restart on the patch (previously, there was a bug.), > it should be fine now. Please test it to confirm. > >> > >> Please let me know if you need anything else. Your help is appreciated. > >> > >> Thanks, > >> Yingqi > >> > >> -Original Message- > >> From: Yann Ylavic [mailto:ylavic@gmail.com] > >> Sent: Friday, May 08, 2015 5:02 AM > >> To: httpd > >> Subject: Re: SO_REUSEPORT > >> > >> On Fri, May 8, 2015 at 9:44 AM, Christophe JAILLET < > christophe.jail...@wanadoo.fr> wrote: > >>> > >>> Maybe, 2.4.14 could focus on reviewing/merging this patch and > >>> associated performance improvement? > >>> To help adoption, maybe an ASF server could be upgraded with a > >>> SO_REUSEPORT patched version of Apache to have our own measurements > >>> and see how it scales in a real world application. > >> > >> I did some testing with an injector at the time of the proposal (on a > 2.2.x version of the patch, so mainly with worker), and can confirm that it > really scales much better. > >> Where httpd without SO_REUSEPORT stops accepting/handling connections, > it continues to shine with the option/buckets enabled. > >> (I don't have the numbers for now, need to search deeper, btw the ones > from Intel are probably more of interest...) > >> > >> So regarding the upgrade on infra, the difference may not be obvious if > the tested machine is not "at the limits". > >> > >> One thing that probably is worth testing is (graceful) restarts, though. > >> > >> Regards, > >> Yann. > >
Re: SO_REUSEPORT
We are very close... I believe wrowe has some somewhat trivial reservations about it, but we are awaiting 1 more vote. Someone want to address wrowes concerns on trunk and patch the patch (stuff like naming)? I may have time next week. > On May 14, 2015, at 7:45 PM, Yann Ylavic wrote: > > Hi Yingqi, > > 2 votes already (on 3), it makes its way ;) > > Regards, > Yann. > > > On Fri, May 15, 2015 at 1:00 AM, Lu, Yingqi wrote: >> Hi All, >> >> I just want to check if anyone gets chances to check the SO_REUSEPORT patch? >> Any feedback? >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Lu, Yingqi [mailto:yingqi...@intel.com] >> Sent: Friday, May 08, 2015 8:58 AM >> To: dev@httpd.apache.org >> Subject: RE: SO_REUSEPORT >> >> Hi Christophe, Jim and Yann, >> >> Thank you very much for your consideration of putting SO_REUSEPORT patch in >> the 2.4 stable release. >> >> I am also very happy that you find the white paper :-) All the most recent >> testing results are included in the white paper. Also, we have tested the >> (graceful) restart on the patch (previously, there was a bug.), it should be >> fine now. Please test it to confirm. >> >> Please let me know if you need anything else. Your help is appreciated. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Friday, May 08, 2015 5:02 AM >> To: httpd >> Subject: Re: SO_REUSEPORT >> >> On Fri, May 8, 2015 at 9:44 AM, Christophe JAILLET >> wrote: >>> >>> Maybe, 2.4.14 could focus on reviewing/merging this patch and >>> associated performance improvement? >>> To help adoption, maybe an ASF server could be upgraded with a >>> SO_REUSEPORT patched version of Apache to have our own measurements >>> and see how it scales in a real world application. >> >> I did some testing with an injector at the time of the proposal (on a 2.2.x >> version of the patch, so mainly with worker), and can confirm that it really >> scales much better. >> Where httpd without SO_REUSEPORT stops accepting/handling connections, it >> continues to shine with the option/buckets enabled. >> (I don't have the numbers for now, need to search deeper, btw the ones from >> Intel are probably more of interest...) >> >> So regarding the upgrade on infra, the difference may not be obvious if the >> tested machine is not "at the limits". >> >> One thing that probably is worth testing is (graceful) restarts, though. >> >> Regards, >> Yann.
RE: SO_REUSEPORT
Thank you very much for your help, Yann! All, please test the patch and vote for us if you like it :-) Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Thursday, May 14, 2015 4:45 PM To: httpd Subject: Re: SO_REUSEPORT Hi Yingqi, 2 votes already (on 3), it makes its way ;) Regards, Yann. On Fri, May 15, 2015 at 1:00 AM, Lu, Yingqi wrote: > Hi All, > > I just want to check if anyone gets chances to check the SO_REUSEPORT patch? > Any feedback? > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Friday, May 08, 2015 8:58 AM > To: dev@httpd.apache.org > Subject: RE: SO_REUSEPORT > > Hi Christophe, Jim and Yann, > > Thank you very much for your consideration of putting SO_REUSEPORT patch in > the 2.4 stable release. > > I am also very happy that you find the white paper :-) All the most recent > testing results are included in the white paper. Also, we have tested the > (graceful) restart on the patch (previously, there was a bug.), it should be > fine now. Please test it to confirm. > > Please let me know if you need anything else. Your help is appreciated. > > Thanks, > Yingqi > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Friday, May 08, 2015 5:02 AM > To: httpd > Subject: Re: SO_REUSEPORT > > On Fri, May 8, 2015 at 9:44 AM, Christophe JAILLET > wrote: >> >> Maybe, 2.4.14 could focus on reviewing/merging this patch and >> associated performance improvement? >> To help adoption, maybe an ASF server could be upgraded with a >> SO_REUSEPORT patched version of Apache to have our own measurements >> and see how it scales in a real world application. > > I did some testing with an injector at the time of the proposal (on a 2.2.x > version of the patch, so mainly with worker), and can confirm that it really > scales much better. > Where httpd without SO_REUSEPORT stops accepting/handling connections, it > continues to shine with the option/buckets enabled. > (I don't have the numbers for now, need to search deeper, btw the ones > from Intel are probably more of interest...) > > So regarding the upgrade on infra, the difference may not be obvious if the > tested machine is not "at the limits". > > One thing that probably is worth testing is (graceful) restarts, though. > > Regards, > Yann.
Re: SO_REUSEPORT
Hi Yingqi, 2 votes already (on 3), it makes its way ;) Regards, Yann. On Fri, May 15, 2015 at 1:00 AM, Lu, Yingqi wrote: > Hi All, > > I just want to check if anyone gets chances to check the SO_REUSEPORT patch? > Any feedback? > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Friday, May 08, 2015 8:58 AM > To: dev@httpd.apache.org > Subject: RE: SO_REUSEPORT > > Hi Christophe, Jim and Yann, > > Thank you very much for your consideration of putting SO_REUSEPORT patch in > the 2.4 stable release. > > I am also very happy that you find the white paper :-) All the most recent > testing results are included in the white paper. Also, we have tested the > (graceful) restart on the patch (previously, there was a bug.), it should be > fine now. Please test it to confirm. > > Please let me know if you need anything else. Your help is appreciated. > > Thanks, > Yingqi > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Friday, May 08, 2015 5:02 AM > To: httpd > Subject: Re: SO_REUSEPORT > > On Fri, May 8, 2015 at 9:44 AM, Christophe JAILLET > wrote: >> >> Maybe, 2.4.14 could focus on reviewing/merging this patch and >> associated performance improvement? >> To help adoption, maybe an ASF server could be upgraded with a >> SO_REUSEPORT patched version of Apache to have our own measurements >> and see how it scales in a real world application. > > I did some testing with an injector at the time of the proposal (on a 2.2.x > version of the patch, so mainly with worker), and can confirm that it really > scales much better. > Where httpd without SO_REUSEPORT stops accepting/handling connections, it > continues to shine with the option/buckets enabled. > (I don't have the numbers for now, need to search deeper, btw the ones from > Intel are probably more of interest...) > > So regarding the upgrade on infra, the difference may not be obvious if the > tested machine is not "at the limits". > > One thing that probably is worth testing is (graceful) restarts, though. > > Regards, > Yann.
RE: SO_REUSEPORT
Hi All, I just want to check if anyone gets chances to check the SO_REUSEPORT patch? Any feedback? Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Friday, May 08, 2015 8:58 AM To: dev@httpd.apache.org Subject: RE: SO_REUSEPORT Hi Christophe, Jim and Yann, Thank you very much for your consideration of putting SO_REUSEPORT patch in the 2.4 stable release. I am also very happy that you find the white paper :-) All the most recent testing results are included in the white paper. Also, we have tested the (graceful) restart on the patch (previously, there was a bug.), it should be fine now. Please test it to confirm. Please let me know if you need anything else. Your help is appreciated. Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Friday, May 08, 2015 5:02 AM To: httpd Subject: Re: SO_REUSEPORT On Fri, May 8, 2015 at 9:44 AM, Christophe JAILLET wrote: > > Maybe, 2.4.14 could focus on reviewing/merging this patch and > associated performance improvement? > To help adoption, maybe an ASF server could be upgraded with a > SO_REUSEPORT patched version of Apache to have our own measurements > and see how it scales in a real world application. I did some testing with an injector at the time of the proposal (on a 2.2.x version of the patch, so mainly with worker), and can confirm that it really scales much better. Where httpd without SO_REUSEPORT stops accepting/handling connections, it continues to shine with the option/buckets enabled. (I don't have the numbers for now, need to search deeper, btw the ones from Intel are probably more of interest...) So regarding the upgrade on infra, the difference may not be obvious if the tested machine is not "at the limits". One thing that probably is worth testing is (graceful) restarts, though. Regards, Yann.
RE: SO_REUSEPORT
Hi Christophe, Jim and Yann, Thank you very much for your consideration of putting SO_REUSEPORT patch in the 2.4 stable release. I am also very happy that you find the white paper :-) All the most recent testing results are included in the white paper. Also, we have tested the (graceful) restart on the patch (previously, there was a bug.), it should be fine now. Please test it to confirm. Please let me know if you need anything else. Your help is appreciated. Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Friday, May 08, 2015 5:02 AM To: httpd Subject: Re: SO_REUSEPORT On Fri, May 8, 2015 at 9:44 AM, Christophe JAILLET wrote: > > Maybe, 2.4.14 could focus on reviewing/merging this patch and > associated performance improvement? > To help adoption, maybe an ASF server could be upgraded with a > SO_REUSEPORT patched version of Apache to have our own measurements > and see how it scales in a real world application. I did some testing with an injector at the time of the proposal (on a 2.2.x version of the patch, so mainly with worker), and can confirm that it really scales much better. Where httpd without SO_REUSEPORT stops accepting/handling connections, it continues to shine with the option/buckets enabled. (I don't have the numbers for now, need to search deeper, btw the ones from Intel are probably more of interest...) So regarding the upgrade on infra, the difference may not be obvious if the tested machine is not "at the limits". One thing that probably is worth testing is (graceful) restarts, though. Regards, Yann.
Re: SO_REUSEPORT
On Fri, May 8, 2015 at 9:44 AM, Christophe JAILLET wrote: > > Maybe, 2.4.14 could focus on reviewing/merging this patch and associated > performance improvement? > To help adoption, maybe an ASF server could be upgraded with a SO_REUSEPORT > patched version of Apache to have our own measurements and see how it scales > in a real world application. I did some testing with an injector at the time of the proposal (on a 2.2.x version of the patch, so mainly with worker), and can confirm that it really scales much better. Where httpd without SO_REUSEPORT stops accepting/handling connections, it continues to shine with the option/buckets enabled. (I don't have the numbers for now, need to search deeper, btw the ones from Intel are probably more of interest...) So regarding the upgrade on infra, the difference may not be obvious if the tested machine is not "at the limits". One thing that probably is worth testing is (graceful) restarts, though. Regards, Yann.
Re: SO_REUSEPORT
Actually, I was going to test that exact patch this weekend, in hopes of getting it into 2.4 > On May 8, 2015, at 3:44 AM, Christophe JAILLET > wrote: > > Hi, > > The SO_REUSEPORT patch has been in trunk for a few months now and has been > proposed for backport in 2.4.x. > > Here is an interesting paper which gives a clear explanation and some > benchmark results: > http://www.intel.ie/content/dam/www/public/us/en/documents/white-papers/scaling-apache-server-performance-paper.pdf > > > Would be great if it could be committed in 2.4.x. > Having it reviewed and tested for 2.4.13 may not be possible because of the > T&R window which seems to be in the coming days/weeks. > (this already what was answered for the 2.4.11/2.4.12 release) > > > Maybe, 2.4.14 could focus on reviewing/merging this patch and associated > performance improvement? > To help adoption, maybe an ASF server could be upgraded with a SO_REUSEPORT > patched version of Apache to have our own measurements and see how it scales > in a real world application. > This has already been done in the past (testing impact of the skiplist > implementation). > > Best regards, > CJ >
SO_REUSEPORT
Hi, The SO_REUSEPORT patch has been in trunk for a few months now and has been proposed for backport in 2.4.x. Here is an interesting paper which gives a clear explanation and some benchmark results: http://www.intel.ie/content/dam/www/public/us/en/documents/white-papers/scaling-apache-server-performance-paper.pdf Would be great if it could be committed in 2.4.x. Having it reviewed and tested for 2.4.13 may not be possible because of the T&R window which seems to be in the coming days/weeks. (this already what was answered for the 2.4.11/2.4.12 release) Maybe, 2.4.14 could focus on reviewing/merging this patch and associated performance improvement? To help adoption, maybe an ASF server could be upgraded with a SO_REUSEPORT patched version of Apache to have our own measurements and see how it scales in a real world application. This has already been done in the past (testing impact of the skiplist implementation). Best regards, CJ
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yann, Thanks for your quick email. Yes, with current implementation, accept mutex is not being removed, just being cut into smaller ones. My point was with smaller system, the hardware resource is less too so that the maximum traffic it can drive is not as much as the big systems. In that sense, the child process/bucket contention may not be hugely increased compared to big system. Running at peak performance level, the total number of child process should scale with the size of the systems if there is no other hardware resource limitations. Then, the child process/bucket should maintain at the similar rate no matter of the system size if we use some reasonable ListenCoresBucketsRatio. Regarding to the "timeout" issue, I think I did not write it clearly in my last email. Testing trunk version with ServerLimit=Number_buckets=StartServer, I did not see any connection timeouts or connection losses. I only saw performance regressions. The "timeout or connection losses" issues only occur when I tested the approach that create the listen socket inside child process. In this case, master process does not control any listen sockets any more, but let each child do it on its own. If I remember correctly, I think that was your quick prototype a while back after I posted the original patch. In the original discussion thread, I mentioned the connection issues and the performance degradation as well. Again, thank you very much for your help! Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Friday, November 07, 2014 7:49 AM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Hi Yingqi, thanks for sharing your results. On Thu, Nov 6, 2014 at 9:12 PM, Lu, Yingqi wrote: > I do not see any documents regarding to this new configurable flag > ListenCoresBucketsRatio (maybe I missed it) Will do it when possible, good point. > Regarding to how to make small systems take advantage of this patch, I > actually did some testing on system with less cores. The data show that when > system has less than 16 cores, more than 1 bucket does not bring any > throughput and response time benefits. The patch is used mainly for big > systems to resolve the scalability issue. That is the reason why we > previously hard coded the ratio to 8 (impact only on system has 16 cores or > more). > > The accept_mutex is not much a bottleneck anymore with the current patch > implantation. Current implementation already cut 1 big mutex into multiple > smaller mutexes in the multiple listen statements case (each bucket has its > dedicated accept_mutex). To prove this, our data show performance parity > between 1 listen statement (listen 80, no accept_mutex) and 2 listen > statements (listen 192.168.1.1 80, listen 192.168.1.2 80, with accept_mutex) > with current trunk version. Comparing against without SO_REUSEPORT patch, we > see 28% performance gain with 1 listen statement case and 69% gain with 2 > listen statements case. With the current implementation and a reasonable number of servers (children) started, this is surely true, your numbers prove it. However, the less buckets (CPU cores), the more contention on each bucket (ie. listeners waiting on the same socket(s)/mutex). So the results with less cores are quite expected IMHO. But we can't remove the accept mutex since there will always be more servers than the number of buckets. > > Regarding to the approach that enables each child has its own listen socket, > I did some testing with current trunk version to increase the number of > buckets to be equal to a reasonable serverlimit (this avoids number of child > processes changes). I also verified that MaxClient and ThreadPerChild were > set properly. I used single listen statement so that accept_mutex was > disabled. Comparing against the current approach, this has ~25% less > throughput with significantly higher response time. > > In addition to this, implementing the listen socket for each child separately > has less performance as well as connection loss/timeout issues with current > Linux kernel. Below are more information/data we collected with "each child > process has its own listen socket" approach: > 1. During the run, we noticed that there are tons of “read timed out” errors. > These errors not only happen when the system is highly utilized, it even > happens when system is only 10% utilized. The response time was high. > 2. Compared to current trunk implementation, we found "each child has its own > listen socket approach" results in significantly higher (up to 10X) response > time at different CPU utilization levels. At peak performance level, it has > 20+% less throughput with tons of “connection reset” errors in additional to >
Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yingqi, thanks for sharing your results. On Thu, Nov 6, 2014 at 9:12 PM, Lu, Yingqi wrote: > I do not see any documents regarding to this new configurable flag > ListenCoresBucketsRatio (maybe I missed it) Will do it when possible, good point. > Regarding to how to make small systems take advantage of this patch, I > actually did some testing on system with less cores. The data show that when > system has less than 16 cores, more than 1 bucket does not bring any > throughput and response time benefits. The patch is used mainly for big > systems to resolve the scalability issue. That is the reason why we > previously hard coded the ratio to 8 (impact only on system has 16 cores or > more). > > The accept_mutex is not much a bottleneck anymore with the current patch > implantation. Current implementation already cut 1 big mutex into multiple > smaller mutexes in the multiple listen statements case (each bucket has its > dedicated accept_mutex). To prove this, our data show performance parity > between 1 listen statement (listen 80, no accept_mutex) and 2 listen > statements (listen 192.168.1.1 80, listen 192.168.1.2 80, with accept_mutex) > with current trunk version. Comparing against without SO_REUSEPORT patch, we > see 28% performance gain with 1 listen statement case and 69% gain with 2 > listen statements case. With the current implementation and a reasonable number of servers (children) started, this is surely true, your numbers prove it. However, the less buckets (CPU cores), the more contention on each bucket (ie. listeners waiting on the same socket(s)/mutex). So the results with less cores are quite expected IMHO. But we can't remove the accept mutex since there will always be more servers than the number of buckets. > > Regarding to the approach that enables each child has its own listen socket, > I did some testing with current trunk version to increase the number of > buckets to be equal to a reasonable serverlimit (this avoids number of child > processes changes). I also verified that MaxClient and ThreadPerChild were > set properly. I used single listen statement so that accept_mutex was > disabled. Comparing against the current approach, this has ~25% less > throughput with significantly higher response time. > > In addition to this, implementing the listen socket for each child separately > has less performance as well as connection loss/timeout issues with current > Linux kernel. Below are more information/data we collected with "each child > process has its own listen socket" approach: > 1. During the run, we noticed that there are tons of “read timed out” errors. > These errors not only happen when the system is highly utilized, it even > happens when system is only 10% utilized. The response time was high. > 2. Compared to current trunk implementation, we found "each child has its own > listen socket approach" results in significantly higher (up to 10X) response > time at different CPU utilization levels. At peak performance level, it has > 20+% less throughput with tons of “connection reset” errors in additional to > “read timed out” errors. Current trunk implementation does not have errors. > 3. During the graceful restart, there are tons of connection losses. Did you also set StartServers = ServerLimit? One bucket per child implies that all the children are up to receive connections or the system may distribute connections to buckets waiting for a child to handle them. Linux may distribute the connections based on the listen()ing sockets, not the ones currently being accept()ed by some child. I don't know your configuration regarding ServerLimit, or more occurrately the number of children really started during the steady state of the stress test: let that number be S. I suppose that S >= num_buckets in your tests with the current implementation, so there is always at least one child to accept() connections on a bucket, so this cannot happen. I expect that with one bucket per child (listen()ed in the parent process), any number of cores, no accept mutex, and StartServers = ServerLimit = S, the system distributes evenly the connections accross all the children, without any "read timeout" or graceful restart issue. Otherwise there is a(nother) kernel bug not worked around by the current implementation, and the same thing may happen when (S / num_buckets) reaches some limit... > > Based on the above findings, I think we may want to keep the current > approach. It is a clean, working and better performing one :-) My point is not (at all) to replace the current approach, but maybe have another ListenBuckets* directive for systems with any number of cores. This would not change the current ListenCoresBucketsRatio behaviour, just looking at another way to configure/exploit listeners buckets ;) Regards, Yann.
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yann, I do not see any documents regarding to this new configurable flag ListenCoresBucketsRatio (maybe I missed it) and also users may not be familiar with it, I still think maybe it is better to keep the default to 8 at least in the trunk. Regarding to how to make small systems take advantage of this patch, I actually did some testing on system with less cores. The data show that when system has less than 16 cores, more than 1 bucket does not bring any throughput and response time benefits. The patch is used mainly for big systems to resolve the scalability issue. That is the reason why we previously hard coded the ratio to 8 (impact only on system has 16 cores or more). The accept_mutex is not much a bottleneck anymore with the current patch implantation. Current implementation already cut 1 big mutex into multiple smaller mutexes in the multiple listen statements case (each bucket has its dedicated accept_mutex). To prove this, our data show performance parity between 1 listen statement (listen 80, no accept_mutex) and 2 listen statements (listen 192.168.1.1 80, listen 192.168.1.2 80, with accept_mutex) with current trunk version. Comparing against without SO_REUSEPORT patch, we see 28% performance gain with 1 listen statement case and 69% gain with 2 listen statements case. Regarding to the approach that enables each child has its own listen socket, I did some testing with current trunk version to increase the number of buckets to be equal to a reasonable serverlimit (this avoids number of child processes changes). I also verified that MaxClient and ThreadPerChild were set properly. I used single listen statement so that accept_mutex was disabled. Comparing against the current approach, this has ~25% less throughput with significantly higher response time. In addition to this, implementing the listen socket for each child separately has less performance as well as connection loss/timeout issues with current Linux kernel. Below are more information/data we collected with "each child process has its own listen socket" approach: 1. During the run, we noticed that there are tons of “read timed out” errors. These errors not only happen when the system is highly utilized, it even happens when system is only 10% utilized. The response time was high. 2. Compared to current trunk implementation, we found "each child has its own listen socket approach" results in significantly higher (up to 10X) response time at different CPU utilization levels. At peak performance level, it has 20+% less throughput with tons of “connection reset” errors in additional to “read timed out” errors. Current trunk implementation does not have errors. 3. During the graceful restart, there are tons of connection losses. Based on the above findings, I think we may want to keep the current approach. It is a clean, working and better performing one :-) Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Thursday, November 06, 2014 4:59 AM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Rebasing discussion here since this thread seems to be referenced in PR55897, and the discussion has somehow been forked and continued in [1]. [1]. http://mail-archives.apache.org/mod_mbox/httpd-dev/201410.mbox/%3c9acd5b67aac5594cb6268234cf29cf9aa37e9...@orsmsx113.amr.corp.intel.com%3E On Sat, Oct 11, 2014 at 1:55 AM, Lu, Yingqi wrote: > Attached patch is generated based on current trunk. It covers for > prefork/worker/event/eventopt MPM. The patch (modified) has now been applied to trunk with r1635521. On Thu, Oct 30, 2014 at 5:10 PM, Lu, Yingqi wrote: > As this is getting better, I am wondering if you guys have plan to put this > SO_REUSEPORT patch into the stable version. > If yes, do you have a rough timeline? The whole feature could certainly be proposed for 2.4.x since there is no (MAJOR) API change. On Thu, Nov 6, 2014 at 6:52 AM, Lu, Yingqi wrote: > I just took some testing on the most recent trunk version. > I found out that num_buckets is default to 1 (ListenCoresBucketsRatio is > default to 0). > Adding ListenCoresBucketsRatio is great since user can have control over this. > However, I am thinking it may be better to make this default at 8. > This will make the SO_REUSEPORT support to be default enabled (8 buckets). (8 buckets with 64 CPU cores, lucky you...). Yes this change wrt to your original patch is documented in the commit message, including how change it to an opt-out. I chose the opt-in way because I almost always find it safer, especially for backports to stable. I have no strong opinion on this regarding trunk, though, could be an opt-out (easily) there. Let's see what others say on this and the backport to 2.4.x. Anyone? > In case users are not aware of this new ListenCoresBucketsRatio > configurable flag, the
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yann, I don't mind at all. I will keep discussion following your reply there. Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Thursday, November 06, 2014 5:00 AM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Hi Yingqi, let's continue discussing this on the original thread if you don't mind, I made an update there. Thanks, Yann. On Thu, Nov 6, 2014 at 6:52 AM, Lu, Yingqi wrote: > Hi Yann, > > I just took some testing on the most recent trunk version. I found out that > num_buckets is default to 1 (ListenCoresBucketsRatio is default to 0). Adding > ListenCoresBucketsRatio is great since user can have control over this. > However, I am thinking it may be better to make this default at 8. This will > make the SO_REUSEPORT support to be default enabled (8 buckets). In case > users are not aware of this new ListenCoresBucketsRatio configurable flag, > they can still enjoy the performance benefits. > > Please let me know what you think. > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Thursday, October 30, 2014 9:10 AM > To: dev@httpd.apache.org > Subject: RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT > on trunk > > Hi Yann, > > Thank you very much for your help! > > As this is getting better, I am wondering if you guys have plan to put this > SO_REUSEPORT patch into the stable version. If yes, do you have a rough > timeline? > > The performance gain is great from the patch, I just want to more people > being able to take advantage of it. > > Thanks, > Lucy > > -Original Message- > From: Yann Ylavic [mailto:ylavic....@gmail.com] > Sent: Thursday, October 30, 2014 8:29 AM > To: httpd > Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT > on trunk > > Hi Yingqi, > > commited in r1635521, with some changes (please see commit log). > These are not big changes, and your work on removing the global variables and > restoring some previous behaviour is great, thanks for the good work. > > Regards, > Yann. > > > On Wed, Oct 29, 2014 at 6:36 PM, Lu, Yingqi wrote: >> Thank you very much for your help! >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Wednesday, October 29, 2014 10:34 AM >> To: httpd >> Subject: Re: Listeners buckets and duplication w/ and w/o >> SO_REUSEPORT on trunk >> >> Hi Yingqi, >> >> I'm working on it currently, will commit soon. >> >> Regards, >> Yann. >> >> On Wed, Oct 29, 2014 at 6:20 PM, Lu, Yingqi wrote: >>> Hi All, >>> >>> I just want to check if there is any feedback/comments on this? >>> >>> For details, please refer to Yann Ylavic's notes and my responses below. >>> >>> Thanks, >>> Yingqi >>> >>> -Original Message- >>> From: Lu, Yingqi [mailto:yingqi...@intel.com] >>> Sent: Friday, October 10, 2014 4:56 PM >>> To: dev@httpd.apache.org >>> Subject: RE: Listeners buckets and duplication w/ and w/o >>> SO_REUSEPORT on trunk >>> >>> Dear All, >>> >>> Attached patch is generated based on current trunk. It covers for >>> prefork/worker/event/eventopt MPM. It supposes to address following issues >>> regarding to SO_RESUEPORT support vs. current trunk version: >>> >>> 1. Same as current trunk version implementation, when active_CPU_num <= 8 >>> or when so_reuseport is not supported by the kernel, ap_num_buckets is set >>> to 1. In any case, there is 1 dedicated listener per bucket. >>> >>> 2. Remove global variables (mpm_listen, enable_default_listeners and >>> num_buckets). mpm_listen is changed to MPM local. enabled_default_listener >>> is completely removed. num_buckets is changed to MPM local >>> (ap_num_buckets). I rename have_so_reuseport to ap_have_so_reuseport. The >>> reason for keeping that global is because this variable is being used by >>> ap_log_common(). Based on the feedback here, I think it may not be a good >>> idea to change the function interface. >>> >>> 3. Change ap_duplicated_listener to have more parameters. This function is >>> being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In >>> this function, prefork_listener (or worker_listener/even_listener/etc) >>> array will be initialized and be set value. ap_num_buc
Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yingqi, let's continue discussing this on the original thread if you don't mind, I made an update there. Thanks, Yann. On Thu, Nov 6, 2014 at 6:52 AM, Lu, Yingqi wrote: > Hi Yann, > > I just took some testing on the most recent trunk version. I found out that > num_buckets is default to 1 (ListenCoresBucketsRatio is default to 0). Adding > ListenCoresBucketsRatio is great since user can have control over this. > However, I am thinking it may be better to make this default at 8. This will > make the SO_REUSEPORT support to be default enabled (8 buckets). In case > users are not aware of this new ListenCoresBucketsRatio configurable flag, > they can still enjoy the performance benefits. > > Please let me know what you think. > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Thursday, October 30, 2014 9:10 AM > To: dev@httpd.apache.org > Subject: RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on > trunk > > Hi Yann, > > Thank you very much for your help! > > As this is getting better, I am wondering if you guys have plan to put this > SO_REUSEPORT patch into the stable version. If yes, do you have a rough > timeline? > > The performance gain is great from the patch, I just want to more people > being able to take advantage of it. > > Thanks, > Lucy > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Thursday, October 30, 2014 8:29 AM > To: httpd > Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on > trunk > > Hi Yingqi, > > commited in r1635521, with some changes (please see commit log). > These are not big changes, and your work on removing the global variables and > restoring some previous behaviour is great, thanks for the good work. > > Regards, > Yann. > > > On Wed, Oct 29, 2014 at 6:36 PM, Lu, Yingqi wrote: >> Thank you very much for your help! >> >> Thanks, >> Yingqi >> >> -----Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Wednesday, October 29, 2014 10:34 AM >> To: httpd >> Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT >> on trunk >> >> Hi Yingqi, >> >> I'm working on it currently, will commit soon. >> >> Regards, >> Yann. >> >> On Wed, Oct 29, 2014 at 6:20 PM, Lu, Yingqi wrote: >>> Hi All, >>> >>> I just want to check if there is any feedback/comments on this? >>> >>> For details, please refer to Yann Ylavic's notes and my responses below. >>> >>> Thanks, >>> Yingqi >>> >>> -Original Message- >>> From: Lu, Yingqi [mailto:yingqi...@intel.com] >>> Sent: Friday, October 10, 2014 4:56 PM >>> To: dev@httpd.apache.org >>> Subject: RE: Listeners buckets and duplication w/ and w/o >>> SO_REUSEPORT on trunk >>> >>> Dear All, >>> >>> Attached patch is generated based on current trunk. It covers for >>> prefork/worker/event/eventopt MPM. It supposes to address following issues >>> regarding to SO_RESUEPORT support vs. current trunk version: >>> >>> 1. Same as current trunk version implementation, when active_CPU_num <= 8 >>> or when so_reuseport is not supported by the kernel, ap_num_buckets is set >>> to 1. In any case, there is 1 dedicated listener per bucket. >>> >>> 2. Remove global variables (mpm_listen, enable_default_listeners and >>> num_buckets). mpm_listen is changed to MPM local. enabled_default_listener >>> is completely removed. num_buckets is changed to MPM local >>> (ap_num_buckets). I rename have_so_reuseport to ap_have_so_reuseport. The >>> reason for keeping that global is because this variable is being used by >>> ap_log_common(). Based on the feedback here, I think it may not be a good >>> idea to change the function interface. >>> >>> 3. Change ap_duplicated_listener to have more parameters. This function is >>> being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In >>> this function, prefork_listener (or worker_listener/even_listener/etc) >>> array will be initialized and be set value. ap_num_buckets is also >>> calculated inside this function. In addition, this version solves the issue >>> with "one_process" case (current trunk version has issue with one_process >>> enabled). >>> >>> 4. Change ap_close_listener() back to previous (2.4.X
Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Rebasing discussion here since this thread seems to be referenced in PR55897, and the discussion has somehow been forked and continued in [1]. [1]. http://mail-archives.apache.org/mod_mbox/httpd-dev/201410.mbox/%3c9acd5b67aac5594cb6268234cf29cf9aa37e9...@orsmsx113.amr.corp.intel.com%3E On Sat, Oct 11, 2014 at 1:55 AM, Lu, Yingqi wrote: > Attached patch is generated based on current trunk. It covers for > prefork/worker/event/eventopt MPM. The patch (modified) has now been applied to trunk with r1635521. On Thu, Oct 30, 2014 at 5:10 PM, Lu, Yingqi wrote: > As this is getting better, I am wondering if you guys have plan to put this > SO_REUSEPORT patch into the stable version. > If yes, do you have a rough timeline? The whole feature could certainly be proposed for 2.4.x since there is no (MAJOR) API change. On Thu, Nov 6, 2014 at 6:52 AM, Lu, Yingqi wrote: > I just took some testing on the most recent trunk version. > I found out that num_buckets is default to 1 (ListenCoresBucketsRatio is > default to 0). > Adding ListenCoresBucketsRatio is great since user can have control over this. > However, I am thinking it may be better to make this default at 8. This will > make the SO_REUSEPORT > support to be default enabled (8 buckets). (8 buckets with 64 CPU cores, lucky you...). Yes this change wrt to your original patch is documented in the commit message, including how change it to an opt-out. I chose the opt-in way because I almost always find it safer, especially for backports to stable. I have no strong opinion on this regarding trunk, though, could be an opt-out (easily) there. Let's see what others say on this and the backport to 2.4.x. Anyone? > In case users are not aware of this new ListenCoresBucketsRatio > configurable flag, they can still enjoy the performance benefits. Users with 64 cores available should always care about performances tuning ;) Btw, I wonder if there are other ways to take advantage of the listeners buckets (even with fewer cores). The other advantage of SO_REUSEPORT is that, provided that each child has its own listeners bucket, we can avoid the accept mutex lock (which also seemed yo be a bottleneck if I recall your original proposal's discussion correctly). Did you make any testing without the accept mutex and with a number of buckets equal to some reasonable ServerLimit (and then play with ThreadPerChild to reach the MaxClient/MaxRequestWorkers goal)? Regards, Yann.
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yann, I just took some testing on the most recent trunk version. I found out that num_buckets is default to 1 (ListenCoresBucketsRatio is default to 0). Adding ListenCoresBucketsRatio is great since user can have control over this. However, I am thinking it may be better to make this default at 8. This will make the SO_REUSEPORT support to be default enabled (8 buckets). In case users are not aware of this new ListenCoresBucketsRatio configurable flag, they can still enjoy the performance benefits. Please let me know what you think. Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Thursday, October 30, 2014 9:10 AM To: dev@httpd.apache.org Subject: RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Hi Yann, Thank you very much for your help! As this is getting better, I am wondering if you guys have plan to put this SO_REUSEPORT patch into the stable version. If yes, do you have a rough timeline? The performance gain is great from the patch, I just want to more people being able to take advantage of it. Thanks, Lucy -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Thursday, October 30, 2014 8:29 AM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Hi Yingqi, commited in r1635521, with some changes (please see commit log). These are not big changes, and your work on removing the global variables and restoring some previous behaviour is great, thanks for the good work. Regards, Yann. On Wed, Oct 29, 2014 at 6:36 PM, Lu, Yingqi wrote: > Thank you very much for your help! > > Thanks, > Yingqi > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Wednesday, October 29, 2014 10:34 AM > To: httpd > Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT > on trunk > > Hi Yingqi, > > I'm working on it currently, will commit soon. > > Regards, > Yann. > > On Wed, Oct 29, 2014 at 6:20 PM, Lu, Yingqi wrote: >> Hi All, >> >> I just want to check if there is any feedback/comments on this? >> >> For details, please refer to Yann Ylavic's notes and my responses below. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Lu, Yingqi [mailto:yingqi...@intel.com] >> Sent: Friday, October 10, 2014 4:56 PM >> To: dev@httpd.apache.org >> Subject: RE: Listeners buckets and duplication w/ and w/o >> SO_REUSEPORT on trunk >> >> Dear All, >> >> Attached patch is generated based on current trunk. It covers for >> prefork/worker/event/eventopt MPM. It supposes to address following issues >> regarding to SO_RESUEPORT support vs. current trunk version: >> >> 1. Same as current trunk version implementation, when active_CPU_num <= 8 or >> when so_reuseport is not supported by the kernel, ap_num_buckets is set to >> 1. In any case, there is 1 dedicated listener per bucket. >> >> 2. Remove global variables (mpm_listen, enable_default_listeners and >> num_buckets). mpm_listen is changed to MPM local. enabled_default_listener >> is completely removed. num_buckets is changed to MPM local (ap_num_buckets). >> I rename have_so_reuseport to ap_have_so_reuseport. The reason for keeping >> that global is because this variable is being used by ap_log_common(). Based >> on the feedback here, I think it may not be a good idea to change the >> function interface. >> >> 3. Change ap_duplicated_listener to have more parameters. This function is >> being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In this >> function, prefork_listener (or worker_listener/even_listener/etc) array will >> be initialized and be set value. ap_num_buckets is also calculated inside >> this function. In addition, this version solves the issue with "one_process" >> case (current trunk version has issue with one_process enabled). >> >> 4. Change ap_close_listener() back to previous (2.4.X version). >> >> 5. Change dummy_connection back to previous (2.4.X version). >> >> 6. Add ap_close_duplicated_listeners(). This is called from mpms when >> stopping httpd. >> >> 7. Add ap_close_child_listener(). When listener_thread (child process in >> prefork) exit, only the dedicated listener needs to be closed (the rest are >> already being closed in child_main when the child process starts). >> >> 8. Remove duplication of listener when ap_num_buckets = 1 or without >> SO_REUSEPORT support (ap_num_buckets = 1). With so_reuseport, only >> duplicated (ap_num_buckets - 1) listeners (1 less duplication less current
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yann, Thank you very much for your help! As this is getting better, I am wondering if you guys have plan to put this SO_REUSEPORT patch into the stable version. If yes, do you have a rough timeline? The performance gain is great from the patch, I just want to more people being able to take advantage of it. Thanks, Lucy -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Thursday, October 30, 2014 8:29 AM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Hi Yingqi, commited in r1635521, with some changes (please see commit log). These are not big changes, and your work on removing the global variables and restoring some previous behaviour is great, thanks for the good work. Regards, Yann. On Wed, Oct 29, 2014 at 6:36 PM, Lu, Yingqi wrote: > Thank you very much for your help! > > Thanks, > Yingqi > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Wednesday, October 29, 2014 10:34 AM > To: httpd > Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT > on trunk > > Hi Yingqi, > > I'm working on it currently, will commit soon. > > Regards, > Yann. > > On Wed, Oct 29, 2014 at 6:20 PM, Lu, Yingqi wrote: >> Hi All, >> >> I just want to check if there is any feedback/comments on this? >> >> For details, please refer to Yann Ylavic's notes and my responses below. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Lu, Yingqi [mailto:yingqi...@intel.com] >> Sent: Friday, October 10, 2014 4:56 PM >> To: dev@httpd.apache.org >> Subject: RE: Listeners buckets and duplication w/ and w/o >> SO_REUSEPORT on trunk >> >> Dear All, >> >> Attached patch is generated based on current trunk. It covers for >> prefork/worker/event/eventopt MPM. It supposes to address following issues >> regarding to SO_RESUEPORT support vs. current trunk version: >> >> 1. Same as current trunk version implementation, when active_CPU_num <= 8 or >> when so_reuseport is not supported by the kernel, ap_num_buckets is set to >> 1. In any case, there is 1 dedicated listener per bucket. >> >> 2. Remove global variables (mpm_listen, enable_default_listeners and >> num_buckets). mpm_listen is changed to MPM local. enabled_default_listener >> is completely removed. num_buckets is changed to MPM local (ap_num_buckets). >> I rename have_so_reuseport to ap_have_so_reuseport. The reason for keeping >> that global is because this variable is being used by ap_log_common(). Based >> on the feedback here, I think it may not be a good idea to change the >> function interface. >> >> 3. Change ap_duplicated_listener to have more parameters. This function is >> being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In this >> function, prefork_listener (or worker_listener/even_listener/etc) array will >> be initialized and be set value. ap_num_buckets is also calculated inside >> this function. In addition, this version solves the issue with "one_process" >> case (current trunk version has issue with one_process enabled). >> >> 4. Change ap_close_listener() back to previous (2.4.X version). >> >> 5. Change dummy_connection back to previous (2.4.X version). >> >> 6. Add ap_close_duplicated_listeners(). This is called from mpms when >> stopping httpd. >> >> 7. Add ap_close_child_listener(). When listener_thread (child process in >> prefork) exit, only the dedicated listener needs to be closed (the rest are >> already being closed in child_main when the child process starts). >> >> 8. Remove duplication of listener when ap_num_buckets = 1 or without >> SO_REUSEPORT support (ap_num_buckets = 1). With so_reuseport, only >> duplicated (ap_num_buckets - 1) listeners (1 less duplication less current >> trunk implementation). >> >> 9. Inside each mpm, move child_bucket, child_pod and child_mutex >> (worker/prefork only) to a struct. Also, add member bucket to the same >> struct. >> >> Please review and let me know your feedback. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Tuesday, October 07, 2014 5:26 PM >> To: httpd >> Subject: Re: Listeners buckets and duplication w/ and w/o >> SO_REUSEPORT on trunk >> >> On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: >>> On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >>>> 3. Yes, I did use some extern variables. I can cha
Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yingqi, commited in r1635521, with some changes (please see commit log). These are not big changes, and your work on removing the global variables and restoring some previous behaviour is great, thanks for the good work. Regards, Yann. On Wed, Oct 29, 2014 at 6:36 PM, Lu, Yingqi wrote: > Thank you very much for your help! > > Thanks, > Yingqi > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Wednesday, October 29, 2014 10:34 AM > To: httpd > Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on > trunk > > Hi Yingqi, > > I'm working on it currently, will commit soon. > > Regards, > Yann. > > On Wed, Oct 29, 2014 at 6:20 PM, Lu, Yingqi wrote: >> Hi All, >> >> I just want to check if there is any feedback/comments on this? >> >> For details, please refer to Yann Ylavic's notes and my responses below. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Lu, Yingqi [mailto:yingqi...@intel.com] >> Sent: Friday, October 10, 2014 4:56 PM >> To: dev@httpd.apache.org >> Subject: RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT >> on trunk >> >> Dear All, >> >> Attached patch is generated based on current trunk. It covers for >> prefork/worker/event/eventopt MPM. It supposes to address following issues >> regarding to SO_RESUEPORT support vs. current trunk version: >> >> 1. Same as current trunk version implementation, when active_CPU_num <= 8 or >> when so_reuseport is not supported by the kernel, ap_num_buckets is set to >> 1. In any case, there is 1 dedicated listener per bucket. >> >> 2. Remove global variables (mpm_listen, enable_default_listeners and >> num_buckets). mpm_listen is changed to MPM local. enabled_default_listener >> is completely removed. num_buckets is changed to MPM local (ap_num_buckets). >> I rename have_so_reuseport to ap_have_so_reuseport. The reason for keeping >> that global is because this variable is being used by ap_log_common(). Based >> on the feedback here, I think it may not be a good idea to change the >> function interface. >> >> 3. Change ap_duplicated_listener to have more parameters. This function is >> being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In this >> function, prefork_listener (or worker_listener/even_listener/etc) array will >> be initialized and be set value. ap_num_buckets is also calculated inside >> this function. In addition, this version solves the issue with "one_process" >> case (current trunk version has issue with one_process enabled). >> >> 4. Change ap_close_listener() back to previous (2.4.X version). >> >> 5. Change dummy_connection back to previous (2.4.X version). >> >> 6. Add ap_close_duplicated_listeners(). This is called from mpms when >> stopping httpd. >> >> 7. Add ap_close_child_listener(). When listener_thread (child process in >> prefork) exit, only the dedicated listener needs to be closed (the rest are >> already being closed in child_main when the child process starts). >> >> 8. Remove duplication of listener when ap_num_buckets = 1 or without >> SO_REUSEPORT support (ap_num_buckets = 1). With so_reuseport, only >> duplicated (ap_num_buckets - 1) listeners (1 less duplication less current >> trunk implementation). >> >> 9. Inside each mpm, move child_bucket, child_pod and child_mutex >> (worker/prefork only) to a struct. Also, add member bucket to the same >> struct. >> >> Please review and let me know your feedback. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Tuesday, October 07, 2014 5:26 PM >> To: httpd >> Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT >> on trunk >> >> On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: >>> On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >>>> 3. Yes, I did use some extern variables. I can change the name of them to >>>> better coordinate with the variable naming conversion. We should do >>>> something with ap_prefixed? Is there anything else I should consider when >>>> I rename the variable? >>> >>> Maybe defining new functions with more arguments (to be used by the >>> existing ones with NULL or default values) is a better alternative. >> >> For example, ap_duplicate_listeners could be modified to provide mpm_listen >> and even do the computa
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Thank you very much for your help! Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Wednesday, October 29, 2014 10:34 AM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Hi Yingqi, I'm working on it currently, will commit soon. Regards, Yann. On Wed, Oct 29, 2014 at 6:20 PM, Lu, Yingqi wrote: > Hi All, > > I just want to check if there is any feedback/comments on this? > > For details, please refer to Yann Ylavic's notes and my responses below. > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Friday, October 10, 2014 4:56 PM > To: dev@httpd.apache.org > Subject: RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT > on trunk > > Dear All, > > Attached patch is generated based on current trunk. It covers for > prefork/worker/event/eventopt MPM. It supposes to address following issues > regarding to SO_RESUEPORT support vs. current trunk version: > > 1. Same as current trunk version implementation, when active_CPU_num <= 8 or > when so_reuseport is not supported by the kernel, ap_num_buckets is set to 1. > In any case, there is 1 dedicated listener per bucket. > > 2. Remove global variables (mpm_listen, enable_default_listeners and > num_buckets). mpm_listen is changed to MPM local. enabled_default_listener is > completely removed. num_buckets is changed to MPM local (ap_num_buckets). I > rename have_so_reuseport to ap_have_so_reuseport. The reason for keeping that > global is because this variable is being used by ap_log_common(). Based on > the feedback here, I think it may not be a good idea to change the function > interface. > > 3. Change ap_duplicated_listener to have more parameters. This function is > being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In this > function, prefork_listener (or worker_listener/even_listener/etc) array will > be initialized and be set value. ap_num_buckets is also calculated inside > this function. In addition, this version solves the issue with "one_process" > case (current trunk version has issue with one_process enabled). > > 4. Change ap_close_listener() back to previous (2.4.X version). > > 5. Change dummy_connection back to previous (2.4.X version). > > 6. Add ap_close_duplicated_listeners(). This is called from mpms when > stopping httpd. > > 7. Add ap_close_child_listener(). When listener_thread (child process in > prefork) exit, only the dedicated listener needs to be closed (the rest are > already being closed in child_main when the child process starts). > > 8. Remove duplication of listener when ap_num_buckets = 1 or without > SO_REUSEPORT support (ap_num_buckets = 1). With so_reuseport, only duplicated > (ap_num_buckets - 1) listeners (1 less duplication less current trunk > implementation). > > 9. Inside each mpm, move child_bucket, child_pod and child_mutex > (worker/prefork only) to a struct. Also, add member bucket to the same struct. > > Please review and let me know your feedback. > > Thanks, > Yingqi > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Tuesday, October 07, 2014 5:26 PM > To: httpd > Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT > on trunk > > On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: >> On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >>> 3. Yes, I did use some extern variables. I can change the name of them to >>> better coordinate with the variable naming conversion. We should do >>> something with ap_prefixed? Is there anything else I should consider when I >>> rename the variable? >> >> Maybe defining new functions with more arguments (to be used by the >> existing ones with NULL or default values) is a better alternative. > > For example, ap_duplicate_listeners could be modified to provide mpm_listen > and even do the computation of num_buckets and provide it (this is not an API > change since it is trunk only for now). > > ap_close_listeners() could be then restored as before (work on ap_listeners > only) and ap_close_duplicated_listeners(mpm_listen) be introduced and used in > the MPMs instead. > > Hence ap_listen_rec *mpm_listeners could be MPM local, which would > then call ap_duplicate_listeners(..., &mpm_listeners, &num_buckets) > and ap_close_duplicated_listeners(mpm_listeners) > > That's just a quick thought... > >> >> Please be aware that existing AP_DECLAREd functions API must not change >> though. >> >> Regards, >> Yann. >> >>
Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi Yingqi, I'm working on it currently, will commit soon. Regards, Yann. On Wed, Oct 29, 2014 at 6:20 PM, Lu, Yingqi wrote: > Hi All, > > I just want to check if there is any feedback/comments on this? > > For details, please refer to Yann Ylavic's notes and my responses below. > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Friday, October 10, 2014 4:56 PM > To: dev@httpd.apache.org > Subject: RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on > trunk > > Dear All, > > Attached patch is generated based on current trunk. It covers for > prefork/worker/event/eventopt MPM. It supposes to address following issues > regarding to SO_RESUEPORT support vs. current trunk version: > > 1. Same as current trunk version implementation, when active_CPU_num <= 8 or > when so_reuseport is not supported by the kernel, ap_num_buckets is set to 1. > In any case, there is 1 dedicated listener per bucket. > > 2. Remove global variables (mpm_listen, enable_default_listeners and > num_buckets). mpm_listen is changed to MPM local. enabled_default_listener is > completely removed. num_buckets is changed to MPM local (ap_num_buckets). I > rename have_so_reuseport to ap_have_so_reuseport. The reason for keeping that > global is because this variable is being used by ap_log_common(). Based on > the feedback here, I think it may not be a good idea to change the function > interface. > > 3. Change ap_duplicated_listener to have more parameters. This function is > being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In this > function, prefork_listener (or worker_listener/even_listener/etc) array will > be initialized and be set value. ap_num_buckets is also calculated inside > this function. In addition, this version solves the issue with "one_process" > case (current trunk version has issue with one_process enabled). > > 4. Change ap_close_listener() back to previous (2.4.X version). > > 5. Change dummy_connection back to previous (2.4.X version). > > 6. Add ap_close_duplicated_listeners(). This is called from mpms when > stopping httpd. > > 7. Add ap_close_child_listener(). When listener_thread (child process in > prefork) exit, only the dedicated listener needs to be closed (the rest are > already being closed in child_main when the child process starts). > > 8. Remove duplication of listener when ap_num_buckets = 1 or without > SO_REUSEPORT support (ap_num_buckets = 1). With so_reuseport, only duplicated > (ap_num_buckets - 1) listeners (1 less duplication less current trunk > implementation). > > 9. Inside each mpm, move child_bucket, child_pod and child_mutex > (worker/prefork only) to a struct. Also, add member bucket to the same struct. > > Please review and let me know your feedback. > > Thanks, > Yingqi > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Tuesday, October 07, 2014 5:26 PM > To: httpd > Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on > trunk > > On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: >> On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >>> 3. Yes, I did use some extern variables. I can change the name of them to >>> better coordinate with the variable naming conversion. We should do >>> something with ap_prefixed? Is there anything else I should consider when I >>> rename the variable? >> >> Maybe defining new functions with more arguments (to be used by the >> existing ones with NULL or default values) is a better alternative. > > For example, ap_duplicate_listeners could be modified to provide mpm_listen > and even do the computation of num_buckets and provide it (this is not an API > change since it is trunk only for now). > > ap_close_listeners() could be then restored as before (work on ap_listeners > only) and ap_close_duplicated_listeners(mpm_listen) be introduced and used in > the MPMs instead. > > Hence ap_listen_rec *mpm_listeners could be MPM local, which would then call > ap_duplicate_listeners(..., &mpm_listeners, &num_buckets) and > ap_close_duplicated_listeners(mpm_listeners) > > That's just a quick thought... > >> >> Please be aware that existing AP_DECLAREd functions API must not change >> though. >> >> Regards, >> Yann. >> >>> >>> Thanks, >>> Yingqi >>> >>> >>> -Original Message- >>> From: Yann Ylavic [mailto:ylavic@gmail.com] >>> Sent: Tuesday, October 07, 2014 4:19 PM >>> To: httpd >>
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi All, I just want to check if there is any feedback/comments on this? For details, please refer to Yann Ylavic's notes and my responses below. Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Friday, October 10, 2014 4:56 PM To: dev@httpd.apache.org Subject: RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Dear All, Attached patch is generated based on current trunk. It covers for prefork/worker/event/eventopt MPM. It supposes to address following issues regarding to SO_RESUEPORT support vs. current trunk version: 1. Same as current trunk version implementation, when active_CPU_num <= 8 or when so_reuseport is not supported by the kernel, ap_num_buckets is set to 1. In any case, there is 1 dedicated listener per bucket. 2. Remove global variables (mpm_listen, enable_default_listeners and num_buckets). mpm_listen is changed to MPM local. enabled_default_listener is completely removed. num_buckets is changed to MPM local (ap_num_buckets). I rename have_so_reuseport to ap_have_so_reuseport. The reason for keeping that global is because this variable is being used by ap_log_common(). Based on the feedback here, I think it may not be a good idea to change the function interface. 3. Change ap_duplicated_listener to have more parameters. This function is being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In this function, prefork_listener (or worker_listener/even_listener/etc) array will be initialized and be set value. ap_num_buckets is also calculated inside this function. In addition, this version solves the issue with "one_process" case (current trunk version has issue with one_process enabled). 4. Change ap_close_listener() back to previous (2.4.X version). 5. Change dummy_connection back to previous (2.4.X version). 6. Add ap_close_duplicated_listeners(). This is called from mpms when stopping httpd. 7. Add ap_close_child_listener(). When listener_thread (child process in prefork) exit, only the dedicated listener needs to be closed (the rest are already being closed in child_main when the child process starts). 8. Remove duplication of listener when ap_num_buckets = 1 or without SO_REUSEPORT support (ap_num_buckets = 1). With so_reuseport, only duplicated (ap_num_buckets - 1) listeners (1 less duplication less current trunk implementation). 9. Inside each mpm, move child_bucket, child_pod and child_mutex (worker/prefork only) to a struct. Also, add member bucket to the same struct. Please review and let me know your feedback. Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Tuesday, October 07, 2014 5:26 PM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: > On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >> 3. Yes, I did use some extern variables. I can change the name of them to >> better coordinate with the variable naming conversion. We should do >> something with ap_prefixed? Is there anything else I should consider when I >> rename the variable? > > Maybe defining new functions with more arguments (to be used by the > existing ones with NULL or default values) is a better alternative. For example, ap_duplicate_listeners could be modified to provide mpm_listen and even do the computation of num_buckets and provide it (this is not an API change since it is trunk only for now). ap_close_listeners() could be then restored as before (work on ap_listeners only) and ap_close_duplicated_listeners(mpm_listen) be introduced and used in the MPMs instead. Hence ap_listen_rec *mpm_listeners could be MPM local, which would then call ap_duplicate_listeners(..., &mpm_listeners, &num_buckets) and ap_close_duplicated_listeners(mpm_listeners) That's just a quick thought... > > Please be aware that existing AP_DECLAREd functions API must not change > though. > > Regards, > Yann. > >> >> Thanks, >> Yingqi >> >> >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Tuesday, October 07, 2014 4:19 PM >> To: httpd >> Subject: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on >> trunk >> >> Hi, >> >> some notes about the current implementation of this (trunk only). >> >> First, whether or not SO_REUSEPORT is available, we do duplicate the >> listeners. >> This, I think, is not the intention of Yingqi Lu's original proposal, and >> probably my fault since I asked for the patch to be splitted in two for a >> better understanding (finally the SO_REUSEPORT patch only has been commited). >> The fact is that without SO_REUSEPORT, this serves nothing, and we'd better
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi All, I just want to check if there is any feedback on this? Generated based on trunk version, this is to remove some code duplications/global variables. This also removes listener duplication when SO_REUSEPORT is not being used. For details, please refer to Yann Ylavic's notes and my responses below. I also attached the code changes here again in case you missed it in the original email I sent last Friday. Thanks very much, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Friday, October 10, 2014 4:56 PM To: dev@httpd.apache.org Subject: RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Dear All, Attached patch is generated based on current trunk. It covers for prefork/worker/event/eventopt MPM. It supposes to address following issues regarding to SO_RESUEPORT support vs. current trunk version: 1. Same as current trunk version implementation, when active_CPU_num <= 8 or when so_reuseport is not supported by the kernel, ap_num_buckets is set to 1. In any case, there is 1 dedicated listener per bucket. 2. Remove global variables (mpm_listen, enable_default_listeners and num_buckets). mpm_listen is changed to MPM local. enabled_default_listener is completely removed. num_buckets is changed to MPM local (ap_num_buckets). I rename have_so_reuseport to ap_have_so_reuseport. The reason for keeping that global is because this variable is being used by ap_log_common(). Based on the feedback here, I think it may not be a good idea to change the function interface. 3. Change ap_duplicated_listener to have more parameters. This function is being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In this function, prefork_listener (or worker_listener/even_listener/etc) array will be initialized and be set value. ap_num_buckets is also calculated inside this function. In addition, this version solves the issue with "one_process" case (current trunk version has issue with one_process enabled). 4. Change ap_close_listener() back to previous (2.4.X version). 5. Change dummy_connection back to previous (2.4.X version). 6. Add ap_close_duplicated_listeners(). This is called from mpms when stopping httpd. 7. Add ap_close_child_listener(). When listener_thread (child process in prefork) exit, only the dedicated listener needs to be closed (the rest are already being closed in child_main when the child process starts). 8. Remove duplication of listener when ap_num_buckets = 1 or without SO_REUSEPORT support (ap_num_buckets = 1). With so_reuseport, only duplicated (ap_num_buckets - 1) listeners (1 less duplication less current trunk implementation). 9. Inside each mpm, move child_bucket, child_pod and child_mutex (worker/prefork only) to a struct. Also, add member bucket to the same struct. Please review and let me know your feedback. Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Tuesday, October 07, 2014 5:26 PM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: > On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >> 3. Yes, I did use some extern variables. I can change the name of them to >> better coordinate with the variable naming conversion. We should do >> something with ap_prefixed? Is there anything else I should consider when I >> rename the variable? > > Maybe defining new functions with more arguments (to be used by the > existing ones with NULL or default values) is a better alternative. For example, ap_duplicate_listeners could be modified to provide mpm_listen and even do the computation of num_buckets and provide it (this is not an API change since it is trunk only for now). ap_close_listeners() could be then restored as before (work on ap_listeners only) and ap_close_duplicated_listeners(mpm_listen) be introduced and used in the MPMs instead. Hence ap_listen_rec *mpm_listeners could be MPM local, which would then call ap_duplicate_listeners(..., &mpm_listeners, &num_buckets) and ap_close_duplicated_listeners(mpm_listeners) That's just a quick thought... > > Please be aware that existing AP_DECLAREd functions API must not change > though. > > Regards, > Yann. > >> >> Thanks, >> Yingqi >> >> >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Tuesday, October 07, 2014 4:19 PM >> To: httpd >> Subject: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on >> trunk >> >> Hi, >> >> some notes about the current implementation of this (trunk only). >> >> First, whether or not SO_REUSEPORT is available, we do duplicate the >> listeners. >> This, I think, is not the intention of Yingqi Lu's ori
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Dear All, Attached patch is generated based on current trunk. It covers for prefork/worker/event/eventopt MPM. It supposes to address following issues regarding to SO_RESUEPORT support vs. current trunk version: 1. Same as current trunk version implementation, when active_CPU_num <= 8 or when so_reuseport is not supported by the kernel, ap_num_buckets is set to 1. In any case, there is 1 dedicated listener per bucket. 2. Remove global variables (mpm_listen, enable_default_listeners and num_buckets). mpm_listen is changed to MPM local. enabled_default_listener is completely removed. num_buckets is changed to MPM local (ap_num_buckets). I rename have_so_reuseport to ap_have_so_reuseport. The reason for keeping that global is because this variable is being used by ap_log_common(). Based on the feedback here, I think it may not be a good idea to change the function interface. 3. Change ap_duplicated_listener to have more parameters. This function is being called from MPM local (prefork.c/worker.c/event.c/eventopt.c). In this function, prefork_listener (or worker_listener/even_listener/etc) array will be initialized and be set value. ap_num_buckets is also calculated inside this function. In addition, this version solves the issue with "one_process" case (current trunk version has issue with one_process enabled). 4. Change ap_close_listener() back to previous (2.4.X version). 5. Change dummy_connection back to previous (2.4.X version). 6. Add ap_close_duplicated_listeners(). This is called from mpms when stopping httpd. 7. Add ap_close_child_listener(). When listener_thread (child process in prefork) exit, only the dedicated listener needs to be closed (the rest are already being closed in child_main when the child process starts). 8. Remove duplication of listener when ap_num_buckets = 1 or without SO_REUSEPORT support (ap_num_buckets = 1). With so_reuseport, only duplicated (ap_num_buckets - 1) listeners (1 less duplication less current trunk implementation). 9. Inside each mpm, move child_bucket, child_pod and child_mutex (worker/prefork only) to a struct. Also, add member bucket to the same struct. Please review and let me know your feedback. Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Tuesday, October 07, 2014 5:26 PM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: > On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >> 3. Yes, I did use some extern variables. I can change the name of them to >> better coordinate with the variable naming conversion. We should do >> something with ap_prefixed? Is there anything else I should consider when I >> rename the variable? > > Maybe defining new functions with more arguments (to be used by the > existing ones with NULL or default values) is a better alternative. For example, ap_duplicate_listeners could be modified to provide mpm_listen and even do the computation of num_buckets and provide it (this is not an API change since it is trunk only for now). ap_close_listeners() could be then restored as before (work on ap_listeners only) and ap_close_duplicated_listeners(mpm_listen) be introduced and used in the MPMs instead. Hence ap_listen_rec *mpm_listeners could be MPM local, which would then call ap_duplicate_listeners(..., &mpm_listeners, &num_buckets) and ap_close_duplicated_listeners(mpm_listeners) That's just a quick thought... > > Please be aware that existing AP_DECLAREd functions API must not change > though. > > Regards, > Yann. > >> >> Thanks, >> Yingqi >> >> >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Tuesday, October 07, 2014 4:19 PM >> To: httpd >> Subject: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on >> trunk >> >> Hi, >> >> some notes about the current implementation of this (trunk only). >> >> First, whether or not SO_REUSEPORT is available, we do duplicate the >> listeners. >> This, I think, is not the intention of Yingqi Lu's original proposal, and >> probably my fault since I asked for the patch to be splitted in two for a >> better understanding (finally the SO_REUSEPORT patch only has been commited). >> The fact is that without SO_REUSEPORT, this serves nothing, and we'd better >> use one listener per bucket (as originally proposed), or a single bucket >> with no duplication (as before) if the performance gains do not worth it. >> WDYT? >> >> Also, there is no opt-in/out for this functionalities, nor a way to >> configure number of buckets ratio wrt number of CPUs (cores). >> For example, SO_REUSEPORT also exists
Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
On Wed, Oct 8, 2014 at 2:26 AM, Yann Ylavic wrote: > On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: >> On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >>> 3. Yes, I did use some extern variables. I can change the name of them to >>> better coordinate with the variable naming conversion. We should do >>> something with ap_prefixed? Is there anything else I should consider when I >>> rename the variable? >> >> Maybe defining new functions with more arguments (to be used by the >> existing ones with NULL or default values) is a better alternative. > > For example, ap_duplicate_listeners could be modified to provide > mpm_listen and even do the computation of num_buckets and provide it > (this is not an API change since it is trunk only for now). > > ap_close_listeners() could be then restored as before (work on > ap_listeners only) and ap_close_duplicated_listeners(mpm_listen) be > introduced and used in the MPMs instead. > > Hence ap_listen_rec *mpm_listeners could be MPM local, which would > then call ap_duplicate_listeners(..., &mpm_listeners, &num_buckets) > and ap_close_duplicated_listeners(mpm_listeners) All these (new) fields could also be in a struct so that future changes won't require a new function. > > That's just a quick thought... > >> >> Please be aware that existing AP_DECLAREd functions API must not change >> though. >> >> Regards, >> Yann. >> >>> >>> Thanks, >>> Yingqi >>> >>> >>> -Original Message- >>> From: Yann Ylavic [mailto:ylavic@gmail.com] >>> Sent: Tuesday, October 07, 2014 4:19 PM >>> To: httpd >>> Subject: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk >>> >>> Hi, >>> >>> some notes about the current implementation of this (trunk only). >>> >>> First, whether or not SO_REUSEPORT is available, we do duplicate the >>> listeners. >>> This, I think, is not the intention of Yingqi Lu's original proposal, and >>> probably my fault since I asked for the patch to be splitted in two for a >>> better understanding (finally the SO_REUSEPORT patch only has been >>> commited). >>> The fact is that without SO_REUSEPORT, this serves nothing, and we'd better >>> use one listener per bucket (as originally proposed), or a single bucket >>> with no duplication (as before) if the performance gains do not worth it. >>> WDYT? >>> >>> Also, there is no opt-in/out for this functionalities, nor a way to >>> configure number of buckets ratio wrt number of CPUs (cores). >>> For example, SO_REUSEPORT also exists on *BSD, but I doubt it would work as >>> expected since IFAICT this not the same thing as in Linux (DragonFly's >>> implementation seems to be closed to Linux' one though). >>> Yet, the dynamic setsockopt() check will also succeed on BSD, and the >>> functionality be enabled. >>> So opt in (my preference) or out? >>> >>> Finally, some global variables (not even ap_ prefixed) are used to >>> communicate between listen.c and the MPM. This helps not breaking the API, >>> but this is trunk... >>> I guess we can fix it, this is just a (self or anyone's) reminder :) >>> >>> Regards, >>> Yann.
Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
On Wed, Oct 8, 2014 at 2:03 AM, Yann Ylavic wrote: > On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: >> 3. Yes, I did use some extern variables. I can change the name of them to >> better coordinate with the variable naming conversion. We should do >> something with ap_prefixed? Is there anything else I should consider when I >> rename the variable? > > Maybe defining new functions with more arguments (to be used by the > existing ones with NULL or default values) is a better alternative. For example, ap_duplicate_listeners could be modified to provide mpm_listen and even do the computation of num_buckets and provide it (this is not an API change since it is trunk only for now). ap_close_listeners() could be then restored as before (work on ap_listeners only) and ap_close_duplicated_listeners(mpm_listen) be introduced and used in the MPMs instead. Hence ap_listen_rec *mpm_listeners could be MPM local, which would then call ap_duplicate_listeners(..., &mpm_listeners, &num_buckets) and ap_close_duplicated_listeners(mpm_listeners) That's just a quick thought... > > Please be aware that existing AP_DECLAREd functions API must not change > though. > > Regards, > Yann. > >> >> Thanks, >> Yingqi >> >> >> -Original Message- >> From: Yann Ylavic [mailto:ylavic@gmail.com] >> Sent: Tuesday, October 07, 2014 4:19 PM >> To: httpd >> Subject: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk >> >> Hi, >> >> some notes about the current implementation of this (trunk only). >> >> First, whether or not SO_REUSEPORT is available, we do duplicate the >> listeners. >> This, I think, is not the intention of Yingqi Lu's original proposal, and >> probably my fault since I asked for the patch to be splitted in two for a >> better understanding (finally the SO_REUSEPORT patch only has been commited). >> The fact is that without SO_REUSEPORT, this serves nothing, and we'd better >> use one listener per bucket (as originally proposed), or a single bucket >> with no duplication (as before) if the performance gains do not worth it. >> WDYT? >> >> Also, there is no opt-in/out for this functionalities, nor a way to >> configure number of buckets ratio wrt number of CPUs (cores). >> For example, SO_REUSEPORT also exists on *BSD, but I doubt it would work as >> expected since IFAICT this not the same thing as in Linux (DragonFly's >> implementation seems to be closed to Linux' one though). >> Yet, the dynamic setsockopt() check will also succeed on BSD, and the >> functionality be enabled. >> So opt in (my preference) or out? >> >> Finally, some global variables (not even ap_ prefixed) are used to >> communicate between listen.c and the MPM. This helps not breaking the API, >> but this is trunk... >> I guess we can fix it, this is just a (self or anyone's) reminder :) >> >> Regards, >> Yann.
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Regarding to your comments #2, we tested on a 16 thread system and it does not bring any performance value. That is the reason I calculate this way. Thanks for the comments below. I will try to send out a fix soon. Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Tuesday, October 07, 2014 5:04 PM To: httpd Subject: Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: > Here is what I think. Currently (trunk version as well as my original > patch), > > 1. Without SO_REUSEPORT or when available CPU number < 8, num_bucket = 1 > anyway. It duplicates 1 listener and use that for this single bucket. If > folks think we should not duplicate in this case, I can modify the code to do > that. Yes I think the duplication should be avoided. But is one listener per bucket an interesting alternative to num_buckets = 1? > > 2. num_buckets is calculated = available_CPU_num/8. When available CPU is > less than 8, num_buckets = 1. It checks if SO_REUSEPORT is enabled in the > kernel. If yes, it will enable it. I guess that is opt-in? Maybe I > misunderstood you here, Yann. Please correct me if I do. Why fixed 8, wouldn't one with less than 16 cores want the feature? > > 3. Yes, I did use some extern variables. I can change the name of them to > better coordinate with the variable naming conversion. We should do something > with ap_prefixed? Is there anything else I should consider when I rename the > variable? Maybe defining new functions with more arguments (to be used by the existing ones with NULL or default values) is a better alternative. Please be aware that existing AP_DECLAREd functions API must not change though. Regards, Yann. > > Thanks, > Yingqi > > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Tuesday, October 07, 2014 4:19 PM > To: httpd > Subject: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on > trunk > > Hi, > > some notes about the current implementation of this (trunk only). > > First, whether or not SO_REUSEPORT is available, we do duplicate the > listeners. > This, I think, is not the intention of Yingqi Lu's original proposal, and > probably my fault since I asked for the patch to be splitted in two for a > better understanding (finally the SO_REUSEPORT patch only has been commited). > The fact is that without SO_REUSEPORT, this serves nothing, and we'd better > use one listener per bucket (as originally proposed), or a single bucket with > no duplication (as before) if the performance gains do not worth it. > WDYT? > > Also, there is no opt-in/out for this functionalities, nor a way to configure > number of buckets ratio wrt number of CPUs (cores). > For example, SO_REUSEPORT also exists on *BSD, but I doubt it would work as > expected since IFAICT this not the same thing as in Linux (DragonFly's > implementation seems to be closed to Linux' one though). > Yet, the dynamic setsockopt() check will also succeed on BSD, and the > functionality be enabled. > So opt in (my preference) or out? > > Finally, some global variables (not even ap_ prefixed) are used to > communicate between listen.c and the MPM. This helps not breaking the API, > but this is trunk... > I guess we can fix it, this is just a (self or anyone's) reminder :) > > Regards, > Yann.
Re: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
On Wed, Oct 8, 2014 at 1:50 AM, Lu, Yingqi wrote: > Here is what I think. Currently (trunk version as well as my original patch), > > 1. Without SO_REUSEPORT or when available CPU number < 8, num_bucket = 1 > anyway. It duplicates 1 listener and use that for this single bucket. If > folks think we should not duplicate in this case, I can modify the code to do > that. Yes I think the duplication should be avoided. But is one listener per bucket an interesting alternative to num_buckets = 1? > > 2. num_buckets is calculated = available_CPU_num/8. When available CPU is > less than 8, num_buckets = 1. It checks if SO_REUSEPORT is enabled in the > kernel. If yes, it will enable it. I guess that is opt-in? Maybe I > misunderstood you here, Yann. Please correct me if I do. Why fixed 8, wouldn't one with less than 16 cores want the feature? > > 3. Yes, I did use some extern variables. I can change the name of them to > better coordinate with the variable naming conversion. We should do something > with ap_prefixed? Is there anything else I should consider when I rename the > variable? Maybe defining new functions with more arguments (to be used by the existing ones with NULL or default values) is a better alternative. Please be aware that existing AP_DECLAREd functions API must not change though. Regards, Yann. > > Thanks, > Yingqi > > > -Original Message- > From: Yann Ylavic [mailto:ylavic@gmail.com] > Sent: Tuesday, October 07, 2014 4:19 PM > To: httpd > Subject: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk > > Hi, > > some notes about the current implementation of this (trunk only). > > First, whether or not SO_REUSEPORT is available, we do duplicate the > listeners. > This, I think, is not the intention of Yingqi Lu's original proposal, and > probably my fault since I asked for the patch to be splitted in two for a > better understanding (finally the SO_REUSEPORT patch only has been commited). > The fact is that without SO_REUSEPORT, this serves nothing, and we'd better > use one listener per bucket (as originally proposed), or a single bucket with > no duplication (as before) if the performance gains do not worth it. > WDYT? > > Also, there is no opt-in/out for this functionalities, nor a way to configure > number of buckets ratio wrt number of CPUs (cores). > For example, SO_REUSEPORT also exists on *BSD, but I doubt it would work as > expected since IFAICT this not the same thing as in Linux (DragonFly's > implementation seems to be closed to Linux' one though). > Yet, the dynamic setsockopt() check will also succeed on BSD, and the > functionality be enabled. > So opt in (my preference) or out? > > Finally, some global variables (not even ap_ prefixed) are used to > communicate between listen.c and the MPM. This helps not breaking the API, > but this is trunk... > I guess we can fix it, this is just a (self or anyone's) reminder :) > > Regards, > Yann.
RE: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Here is what I think. Currently (trunk version as well as my original patch), 1. Without SO_REUSEPORT or when available CPU number < 8, num_bucket = 1 anyway. It duplicates 1 listener and use that for this single bucket. If folks think we should not duplicate in this case, I can modify the code to do that. 2. num_buckets is calculated = available_CPU_num/8. When available CPU is less than 8, num_buckets = 1. It checks if SO_REUSEPORT is enabled in the kernel. If yes, it will enable it. I guess that is opt-in? Maybe I misunderstood you here, Yann. Please correct me if I do. 3. Yes, I did use some extern variables. I can change the name of them to better coordinate with the variable naming conversion. We should do something with ap_prefixed? Is there anything else I should consider when I rename the variable? Thanks, Yingqi -Original Message- From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Tuesday, October 07, 2014 4:19 PM To: httpd Subject: Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk Hi, some notes about the current implementation of this (trunk only). First, whether or not SO_REUSEPORT is available, we do duplicate the listeners. This, I think, is not the intention of Yingqi Lu's original proposal, and probably my fault since I asked for the patch to be splitted in two for a better understanding (finally the SO_REUSEPORT patch only has been commited). The fact is that without SO_REUSEPORT, this serves nothing, and we'd better use one listener per bucket (as originally proposed), or a single bucket with no duplication (as before) if the performance gains do not worth it. WDYT? Also, there is no opt-in/out for this functionalities, nor a way to configure number of buckets ratio wrt number of CPUs (cores). For example, SO_REUSEPORT also exists on *BSD, but I doubt it would work as expected since IFAICT this not the same thing as in Linux (DragonFly's implementation seems to be closed to Linux' one though). Yet, the dynamic setsockopt() check will also succeed on BSD, and the functionality be enabled. So opt in (my preference) or out? Finally, some global variables (not even ap_ prefixed) are used to communicate between listen.c and the MPM. This helps not breaking the API, but this is trunk... I guess we can fix it, this is just a (self or anyone's) reminder :) Regards, Yann.
Listeners buckets and duplication w/ and w/o SO_REUSEPORT on trunk
Hi, some notes about the current implementation of this (trunk only). First, whether or not SO_REUSEPORT is available, we do duplicate the listeners. This, I think, is not the intention of Yingqi Lu's original proposal, and probably my fault since I asked for the patch to be splitted in two for a better understanding (finally the SO_REUSEPORT patch only has been commited). The fact is that without SO_REUSEPORT, this serves nothing, and we'd better use one listener per bucket (as originally proposed), or a single bucket with no duplication (as before) if the performance gains do not worth it. WDYT? Also, there is no opt-in/out for this functionalities, nor a way to configure number of buckets ratio wrt number of CPUs (cores). For example, SO_REUSEPORT also exists on *BSD, but I doubt it would work as expected since IFAICT this not the same thing as in Linux (DragonFly's implementation seems to be closed to Linux' one though). Yet, the dynamic setsockopt() check will also succeed on BSD, and the functionality be enabled. So opt in (my preference) or out? Finally, some global variables (not even ap_ prefixed) are used to communicate between listen.c and the MPM. This helps not breaking the API, but this is trunk... I guess we can fix it, this is just a (self or anyone's) reminder :) Regards, Yann.
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Jim, I found that original patch code does not guard "ap_daemons_to_start" to be equal or bigger than num_buckets. Since each bucket has a dedicated listener assigned to it, I think we need to make sure that each bucket has at least 1 child process to start with. I created this small patch to fix the issue. Can you please help to add the changes into the trunk? Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Thursday, June 05, 2014 9:12 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Thanks very much for your help! -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Thursday, June 05, 2014 6:38 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Committed r1600656 Thx On Jun 4, 2014, at 3:39 PM, Lu, Yingqi wrote: > Hi Jim, > > I just found that prefork and worker has issue with restart. Event mpm code > is good. > > I created this small patch to fix the issue for both prefork and worker. The > patch is based on rev #1600451. > > Can you please help add the changes in the trunk? > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Tuesday, June 03, 2014 8:50 AM > To: dev@httpd.apache.org > Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Thank you very much for your help! > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Tuesday, June 03, 2014 8:31 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Next on the agenda is to push into eventopt > > On Jun 3, 2014, at 10:21 AM, Jim Jagielski wrote: > >> FTR: I saw no reason to try to handle both patches... I used the >> so_reuseport patch as the primary patch to focus on. >> >> I have some minor changes coming up to follow-up post the initial >> commit >> >> On Jun 3, 2014, at 8:51 AM, Jim Jagielski wrote: >> >>> I have folded this into trunk and am currently fixing some compile >>> warnings and errors... >>> >>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote: >>> >>>> Hi Jim, >>>> >>>> Personally, I think the second approach is better, it keeps >>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original >>>> ones, only modifies dummy_connection (). Please let me know if you have >>>> different opinions. >>>> >>>> Attached is the latest version of the two patches. They were both >>>> generated against trunk rev. 1598561. Please review them and let me know >>>> if there is anything missing. >>>> >>>> I already updated the Bugzilla database for the item 55897 and item 56279. >>>> >>>> Thanks, >>>> Yingqi >>>> >>>> -Original Message- >>>> From: Lu, Yingqi [mailto:yingqi...@intel.com] >>>> Sent: Saturday, May 31, 2014 11:48 PM >>>> To: dev@httpd.apache.org >>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >>>> SO_REUSEPORT support >>>> >>>> Hi Jim, >>>> >>>> Regarding to your comment #2, yes, you are right, it should be >>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much >>>> for catching this. >>>> >>>> Regarding to your comment #1, the patch modifies the >>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int >>>> child_bucket). Inside the function, the reference listen statement is >>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls >>>> dummy_connection(). >>>> >>>> Can we just modify the return of ap_mpm_pod_signal() from >>>> dummy_connection(pod) to dummy_connection(pod, 0) and add >>>> ap_mpm_pod_signal_ex()? >>>> >>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can >>>> modify dummy_connection() to send dummy data via all the duplicated listen >>>> statements. Then, we do not need child_bucket as the input parameter for >>>> dummy_connection(). In this case, we do not need adding >>>> ap_mpm_pod_signal_ex() too. >>>> >>>> I already tested the code for the above approaches and they bot
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Thanks very much for your help! -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Thursday, June 05, 2014 6:38 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Committed r1600656 Thx On Jun 4, 2014, at 3:39 PM, Lu, Yingqi wrote: > Hi Jim, > > I just found that prefork and worker has issue with restart. Event mpm code > is good. > > I created this small patch to fix the issue for both prefork and worker. The > patch is based on rev #1600451. > > Can you please help add the changes in the trunk? > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Tuesday, June 03, 2014 8:50 AM > To: dev@httpd.apache.org > Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Thank you very much for your help! > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Tuesday, June 03, 2014 8:31 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Next on the agenda is to push into eventopt > > On Jun 3, 2014, at 10:21 AM, Jim Jagielski wrote: > >> FTR: I saw no reason to try to handle both patches... I used the >> so_reuseport patch as the primary patch to focus on. >> >> I have some minor changes coming up to follow-up post the initial >> commit >> >> On Jun 3, 2014, at 8:51 AM, Jim Jagielski wrote: >> >>> I have folded this into trunk and am currently fixing some compile >>> warnings and errors... >>> >>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote: >>> >>>> Hi Jim, >>>> >>>> Personally, I think the second approach is better, it keeps >>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original >>>> ones, only modifies dummy_connection (). Please let me know if you have >>>> different opinions. >>>> >>>> Attached is the latest version of the two patches. They were both >>>> generated against trunk rev. 1598561. Please review them and let me know >>>> if there is anything missing. >>>> >>>> I already updated the Bugzilla database for the item 55897 and item 56279. >>>> >>>> Thanks, >>>> Yingqi >>>> >>>> -Original Message- >>>> From: Lu, Yingqi [mailto:yingqi...@intel.com] >>>> Sent: Saturday, May 31, 2014 11:48 PM >>>> To: dev@httpd.apache.org >>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >>>> SO_REUSEPORT support >>>> >>>> Hi Jim, >>>> >>>> Regarding to your comment #2, yes, you are right, it should be >>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much >>>> for catching this. >>>> >>>> Regarding to your comment #1, the patch modifies the >>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int >>>> child_bucket). Inside the function, the reference listen statement is >>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls >>>> dummy_connection(). >>>> >>>> Can we just modify the return of ap_mpm_pod_signal() from >>>> dummy_connection(pod) to dummy_connection(pod, 0) and add >>>> ap_mpm_pod_signal_ex()? >>>> >>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can >>>> modify dummy_connection() to send dummy data via all the duplicated listen >>>> statements. Then, we do not need child_bucket as the input parameter for >>>> dummy_connection(). In this case, we do not need adding >>>> ap_mpm_pod_signal_ex() too. >>>> >>>> I already tested the code for the above approaches and they both work. >>>> >>>> Please let me know which way you think is better. I can quickly send you >>>> an update for review. >>>> >>>> Thanks, >>>> Yingqi >>>> >>>> -Original Message- >>>> From: Lu, Yingqi >>>> Sent: Saturday, May 31, 2014 3:28 PM >>>> To: >>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >>>> SO_REUSEPORT support >>>> >>>> Hi Jim, >>>> >>>> Thanks very much for your email
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Committed r1600656 Thx On Jun 4, 2014, at 3:39 PM, Lu, Yingqi wrote: > Hi Jim, > > I just found that prefork and worker has issue with restart. Event mpm code > is good. > > I created this small patch to fix the issue for both prefork and worker. The > patch is based on rev #1600451. > > Can you please help add the changes in the trunk? > > Thanks, > Yingqi > > -Original Message- > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Tuesday, June 03, 2014 8:50 AM > To: dev@httpd.apache.org > Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT > support > > Thank you very much for your help! > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Tuesday, June 03, 2014 8:31 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT > support > > Next on the agenda is to push into eventopt > > On Jun 3, 2014, at 10:21 AM, Jim Jagielski wrote: > >> FTR: I saw no reason to try to handle both patches... I used the >> so_reuseport patch as the primary patch to focus on. >> >> I have some minor changes coming up to follow-up post the initial >> commit >> >> On Jun 3, 2014, at 8:51 AM, Jim Jagielski wrote: >> >>> I have folded this into trunk and am currently fixing some compile >>> warnings and errors... >>> >>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote: >>> >>>> Hi Jim, >>>> >>>> Personally, I think the second approach is better, it keeps >>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original >>>> ones, only modifies dummy_connection (). Please let me know if you have >>>> different opinions. >>>> >>>> Attached is the latest version of the two patches. They were both >>>> generated against trunk rev. 1598561. Please review them and let me know >>>> if there is anything missing. >>>> >>>> I already updated the Bugzilla database for the item 55897 and item 56279. >>>> >>>> Thanks, >>>> Yingqi >>>> >>>> -Original Message- >>>> From: Lu, Yingqi [mailto:yingqi...@intel.com] >>>> Sent: Saturday, May 31, 2014 11:48 PM >>>> To: dev@httpd.apache.org >>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >>>> SO_REUSEPORT support >>>> >>>> Hi Jim, >>>> >>>> Regarding to your comment #2, yes, you are right, it should be >>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much >>>> for catching this. >>>> >>>> Regarding to your comment #1, the patch modifies the >>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int >>>> child_bucket). Inside the function, the reference listen statement is >>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls >>>> dummy_connection(). >>>> >>>> Can we just modify the return of ap_mpm_pod_signal() from >>>> dummy_connection(pod) to dummy_connection(pod, 0) and add >>>> ap_mpm_pod_signal_ex()? >>>> >>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can >>>> modify dummy_connection() to send dummy data via all the duplicated listen >>>> statements. Then, we do not need child_bucket as the input parameter for >>>> dummy_connection(). In this case, we do not need adding >>>> ap_mpm_pod_signal_ex() too. >>>> >>>> I already tested the code for the above approaches and they both work. >>>> >>>> Please let me know which way you think is better. I can quickly send you >>>> an update for review. >>>> >>>> Thanks, >>>> Yingqi >>>> >>>> -Original Message- >>>> From: Lu, Yingqi >>>> Sent: Saturday, May 31, 2014 3:28 PM >>>> To: >>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >>>> SO_REUSEPORT support >>>> >>>> Hi Jim, >>>> >>>> Thanks very much for your email! I will look into both of them and send an >>>> update tonight! >>>> >>>> Thanks, >>>> Yingqi >>>> >>>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrot
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Jim,
I just found that prefork and worker has issue with restart. Event mpm code is
good.
I created this small patch to fix the issue for both prefork and worker. The
patch is based on rev #1600451.
Can you please help add the changes in the trunk?
Thanks,
Yingqi
-Original Message-
From: Lu, Yingqi [mailto:yingqi...@intel.com]
Sent: Tuesday, June 03, 2014 8:50 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
support
Thank you very much for your help!
Thanks,
Yingqi
-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Tuesday, June 03, 2014 8:31 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
support
Next on the agenda is to push into eventopt
On Jun 3, 2014, at 10:21 AM, Jim Jagielski wrote:
> FTR: I saw no reason to try to handle both patches... I used the
> so_reuseport patch as the primary patch to focus on.
>
> I have some minor changes coming up to follow-up post the initial
> commit
>
> On Jun 3, 2014, at 8:51 AM, Jim Jagielski wrote:
>
>> I have folded this into trunk and am currently fixing some compile
>> warnings and errors...
>>
>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote:
>>
>>> Hi Jim,
>>>
>>> Personally, I think the second approach is better, it keeps
>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones,
>>> only modifies dummy_connection (). Please let me know if you have different
>>> opinions.
>>>
>>> Attached is the latest version of the two patches. They were both generated
>>> against trunk rev. 1598561. Please review them and let me know if there is
>>> anything missing.
>>>
>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>> Sent: Saturday, May 31, 2014 11:48 PM
>>> To: dev@httpd.apache.org
>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>> SO_REUSEPORT support
>>>
>>> Hi Jim,
>>>
>>> Regarding to your comment #2, yes, you are right, it should be
>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much
>>> for catching this.
>>>
>>> Regarding to your comment #1, the patch modifies the
>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int
>>> child_bucket). Inside the function, the reference listen statement is
>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls
>>> dummy_connection().
>>>
>>> Can we just modify the return of ap_mpm_pod_signal() from
>>> dummy_connection(pod) to dummy_connection(pod, 0) and add
>>> ap_mpm_pod_signal_ex()?
>>>
>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
>>> modify dummy_connection() to send dummy data via all the duplicated listen
>>> statements. Then, we do not need child_bucket as the input parameter for
>>> dummy_connection(). In this case, we do not need adding
>>> ap_mpm_pod_signal_ex() too.
>>>
>>> I already tested the code for the above approaches and they both work.
>>>
>>> Please let me know which way you think is better. I can quickly send you an
>>> update for review.
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Lu, Yingqi
>>> Sent: Saturday, May 31, 2014 3:28 PM
>>> To:
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>> SO_REUSEPORT support
>>>
>>> Hi Jim,
>>>
>>> Thanks very much for your email! I will look into both of them and send an
>>> update tonight!
>>>
>>> Thanks,
>>> Yingqi
>>>
>>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>>>>
>>>> I also see:
>>>>
>>>> /* kill off the idle ones */
>>>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>>>> +for (i = 0; i < num_buckets; i++) {
>>>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>>>> +}
>>>>
>>>>
>>>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
>>>> retained->max_daemons_limit, i); ??
>>>>
>>>> /**
&g
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Thank you very much for your help!
Thanks,
Yingqi
-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Tuesday, June 03, 2014 8:31 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
support
Next on the agenda is to push into eventopt
On Jun 3, 2014, at 10:21 AM, Jim Jagielski wrote:
> FTR: I saw no reason to try to handle both patches... I used the
> so_reuseport patch as the primary patch to focus on.
>
> I have some minor changes coming up to follow-up post the initial
> commit
>
> On Jun 3, 2014, at 8:51 AM, Jim Jagielski wrote:
>
>> I have folded this into trunk and am currently fixing some compile
>> warnings and errors...
>>
>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote:
>>
>>> Hi Jim,
>>>
>>> Personally, I think the second approach is better, it keeps
>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones,
>>> only modifies dummy_connection (). Please let me know if you have different
>>> opinions.
>>>
>>> Attached is the latest version of the two patches. They were both generated
>>> against trunk rev. 1598561. Please review them and let me know if there is
>>> anything missing.
>>>
>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>> Sent: Saturday, May 31, 2014 11:48 PM
>>> To: dev@httpd.apache.org
>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>> SO_REUSEPORT support
>>>
>>> Hi Jim,
>>>
>>> Regarding to your comment #2, yes, you are right, it should be
>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much
>>> for catching this.
>>>
>>> Regarding to your comment #1, the patch modifies the
>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int
>>> child_bucket). Inside the function, the reference listen statement is
>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls
>>> dummy_connection().
>>>
>>> Can we just modify the return of ap_mpm_pod_signal() from
>>> dummy_connection(pod) to dummy_connection(pod, 0) and add
>>> ap_mpm_pod_signal_ex()?
>>>
>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
>>> modify dummy_connection() to send dummy data via all the duplicated listen
>>> statements. Then, we do not need child_bucket as the input parameter for
>>> dummy_connection(). In this case, we do not need adding
>>> ap_mpm_pod_signal_ex() too.
>>>
>>> I already tested the code for the above approaches and they both work.
>>>
>>> Please let me know which way you think is better. I can quickly send you an
>>> update for review.
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Lu, Yingqi
>>> Sent: Saturday, May 31, 2014 3:28 PM
>>> To:
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>> SO_REUSEPORT support
>>>
>>> Hi Jim,
>>>
>>> Thanks very much for your email! I will look into both of them and send an
>>> update tonight!
>>>
>>> Thanks,
>>> Yingqi
>>>
>>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>>>>
>>>> I also see:
>>>>
>>>> /* kill off the idle ones */
>>>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>>>> +for (i = 0; i < num_buckets; i++) {
>>>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>>>> +}
>>>>
>>>>
>>>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
>>>> retained->max_daemons_limit, i); ??
>>>>
>>>> /**
>>>> * Write data to the pipe-of-death, signalling that all child
>>>> process
>>>> * should die.
>>>> * @param pod The pipe-of-death to write to.
>>>> * @param num The number of child processes to kill
>>>> + * @param my_bucket the bucket that holds the dying child process.
>>>> */
>>>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>>>> +AP_DECLARE(void) ap_mpm_pod_killpg(
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Next on the agenda is to push into eventopt
On Jun 3, 2014, at 10:21 AM, Jim Jagielski wrote:
> FTR: I saw no reason to try to handle both patches... I used
> the so_reuseport patch as the primary patch to focus on.
>
> I have some minor changes coming up to follow-up post
> the initial commit
>
> On Jun 3, 2014, at 8:51 AM, Jim Jagielski wrote:
>
>> I have folded this into trunk and am currently fixing some
>> compile warnings and errors...
>>
>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote:
>>
>>> Hi Jim,
>>>
>>> Personally, I think the second approach is better, it keeps
>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones,
>>> only modifies dummy_connection (). Please let me know if you have different
>>> opinions.
>>>
>>> Attached is the latest version of the two patches. They were both generated
>>> against trunk rev. 1598561. Please review them and let me know if there is
>>> anything missing.
>>>
>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>> Sent: Saturday, May 31, 2014 11:48 PM
>>> To: dev@httpd.apache.org
>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
>>> support
>>>
>>> Hi Jim,
>>>
>>> Regarding to your comment #2, yes, you are right, it should be
>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much
>>> for catching this.
>>>
>>> Regarding to your comment #1, the patch modifies the
>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int
>>> child_bucket). Inside the function, the reference listen statement is
>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls
>>> dummy_connection().
>>>
>>> Can we just modify the return of ap_mpm_pod_signal() from
>>> dummy_connection(pod) to dummy_connection(pod, 0) and add
>>> ap_mpm_pod_signal_ex()?
>>>
>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
>>> modify dummy_connection() to send dummy data via all the duplicated listen
>>> statements. Then, we do not need child_bucket as the input parameter for
>>> dummy_connection(). In this case, we do not need adding
>>> ap_mpm_pod_signal_ex() too.
>>>
>>> I already tested the code for the above approaches and they both work.
>>>
>>> Please let me know which way you think is better. I can quickly send you an
>>> update for review.
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Lu, Yingqi
>>> Sent: Saturday, May 31, 2014 3:28 PM
>>> To:
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
>>> support
>>>
>>> Hi Jim,
>>>
>>> Thanks very much for your email! I will look into both of them and send an
>>> update tonight!
>>>
>>> Thanks,
>>> Yingqi
>>>
>>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>>>>
>>>> I also see:
>>>>
>>>> /* kill off the idle ones */
>>>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>>>> +for (i = 0; i < num_buckets; i++) {
>>>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>>>> +}
>>>>
>>>>
>>>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
>>>> retained->max_daemons_limit, i); ??
>>>>
>>>> /**
>>>> * Write data to the pipe-of-death, signalling that all child process
>>>> * should die.
>>>> * @param pod The pipe-of-death to write to.
>>>> * @param num The number of child processes to kill
>>>> + * @param my_bucket the bucket that holds the dying child process.
>>>> */
>>>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>>>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int
>>>> +child_bucket);
>>>>
>>>> Isn't 'num' the same in both implementation??
>>>>
>>>>> On May 31, 2014, at 12:03 PM, Jim Jagielski wrote:
>>>>>
>>>>> Sor
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
FTR: I saw no reason to try to handle both patches... I used
the so_reuseport patch as the primary patch to focus on.
I have some minor changes coming up to follow-up post
the initial commit
On Jun 3, 2014, at 8:51 AM, Jim Jagielski wrote:
> I have folded this into trunk and am currently fixing some
> compile warnings and errors...
>
> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote:
>
>> Hi Jim,
>>
>> Personally, I think the second approach is better, it keeps
>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones,
>> only modifies dummy_connection (). Please let me know if you have different
>> opinions.
>>
>> Attached is the latest version of the two patches. They were both generated
>> against trunk rev. 1598561. Please review them and let me know if there is
>> anything missing.
>>
>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>
>> Thanks,
>> Yingqi
>>
>> -Original Message-
>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>> Sent: Saturday, May 31, 2014 11:48 PM
>> To: dev@httpd.apache.org
>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
>> support
>>
>> Hi Jim,
>>
>> Regarding to your comment #2, yes, you are right, it should be
>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much for
>> catching this.
>>
>> Regarding to your comment #1, the patch modifies the
>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int
>> child_bucket). Inside the function, the reference listen statement is
>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls dummy_connection().
>>
>> Can we just modify the return of ap_mpm_pod_signal() from
>> dummy_connection(pod) to dummy_connection(pod, 0) and add
>> ap_mpm_pod_signal_ex()?
>>
>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
>> modify dummy_connection() to send dummy data via all the duplicated listen
>> statements. Then, we do not need child_bucket as the input parameter for
>> dummy_connection(). In this case, we do not need adding
>> ap_mpm_pod_signal_ex() too.
>>
>> I already tested the code for the above approaches and they both work.
>>
>> Please let me know which way you think is better. I can quickly send you an
>> update for review.
>>
>> Thanks,
>> Yingqi
>>
>> -Original Message-
>> From: Lu, Yingqi
>> Sent: Saturday, May 31, 2014 3:28 PM
>> To:
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
>> support
>>
>> Hi Jim,
>>
>> Thanks very much for your email! I will look into both of them and send an
>> update tonight!
>>
>> Thanks,
>> Yingqi
>>
>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>>>
>>> I also see:
>>>
>>> /* kill off the idle ones */
>>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>>> +for (i = 0; i < num_buckets; i++) {
>>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>>> +}
>>>
>>>
>>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
>>> retained->max_daemons_limit, i); ??
>>>
>>> /**
>>> * Write data to the pipe-of-death, signalling that all child process
>>> * should die.
>>> * @param pod The pipe-of-death to write to.
>>> * @param num The number of child processes to kill
>>> + * @param my_bucket the bucket that holds the dying child process.
>>> */
>>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int
>>> +child_bucket);
>>>
>>> Isn't 'num' the same in both implementation??
>>>
>>>> On May 31, 2014, at 12:03 PM, Jim Jagielski wrote:
>>>>
>>>> Sorry I didn't catch this earlier:
>>>>
>>>> I see
>>>>
>>>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491
>>>> -0400
>>>> @@ -267,16 +267,18 @@
>>>> * Write data to the pipe-of-death, signalling that one child process
>>>> * should die.
>>>> * @param pod the pipe-of-death to write to.
>>>> + * @param my_bucket the bucket that holds the dying child process.
>>>> */
>>>> -A
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
I have folded this into trunk and am currently fixing some
compile warnings and errors...
On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote:
> Hi Jim,
>
> Personally, I think the second approach is better, it keeps ap_mpm_pod_signal
> () and ap_mpm_pod_killpg () exactly as the original ones, only modifies
> dummy_connection (). Please let me know if you have different opinions.
>
> Attached is the latest version of the two patches. They were both generated
> against trunk rev. 1598561. Please review them and let me know if there is
> anything missing.
>
> I already updated the Bugzilla database for the item 55897 and item 56279.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Saturday, May 31, 2014 11:48 PM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
> support
>
> Hi Jim,
>
> Regarding to your comment #2, yes, you are right, it should be
> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much for
> catching this.
>
> Regarding to your comment #1, the patch modifies the
> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int
> child_bucket). Inside the function, the reference listen statement is
> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls dummy_connection().
>
> Can we just modify the return of ap_mpm_pod_signal() from
> dummy_connection(pod) to dummy_connection(pod, 0) and add
> ap_mpm_pod_signal_ex()?
>
> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
> modify dummy_connection() to send dummy data via all the duplicated listen
> statements. Then, we do not need child_bucket as the input parameter for
> dummy_connection(). In this case, we do not need adding
> ap_mpm_pod_signal_ex() too.
>
> I already tested the code for the above approaches and they both work.
>
> Please let me know which way you think is better. I can quickly send you an
> update for review.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Lu, Yingqi
> Sent: Saturday, May 31, 2014 3:28 PM
> To:
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
> support
>
> Hi Jim,
>
> Thanks very much for your email! I will look into both of them and send an
> update tonight!
>
> Thanks,
> Yingqi
>
>> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>>
>> I also see:
>>
>>/* kill off the idle ones */
>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>> +for (i = 0; i < num_buckets; i++) {
>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>> +}
>>
>>
>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
>> retained->max_daemons_limit, i); ??
>>
>> /**
>> * Write data to the pipe-of-death, signalling that all child process
>> * should die.
>> * @param pod The pipe-of-death to write to.
>> * @param num The number of child processes to kill
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int
>> +child_bucket);
>>
>> Isn't 'num' the same in both implementation??
>>
>>> On May 31, 2014, at 12:03 PM, Jim Jagielski wrote:
>>>
>>> Sorry I didn't catch this earlier:
>>>
>>> I see
>>>
>>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491
>>> -0400
>>> @@ -267,16 +267,18 @@
>>> * Write data to the pipe-of-death, signalling that one child process
>>> * should die.
>>> * @param pod the pipe-of-death to write to.
>>> + * @param my_bucket the bucket that holds the dying child process.
>>> */
>>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int
>>> +child_bucket);
>>>
>>> We can change the API at this point. We could add another function,
>>> eg ap_mpm_pod_signal_ex() which takes the int param, but we can't
>>> modify ap_mpm_pod_signal() itself.
>>>
>>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi wrote:
>>>>
>>>> Thank you very much!
>>>>
>>>> Thanks,
>>>> Yingqi
>>>>
>>>> -Original Message-
>>>> From: Jim Jagie
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi, see http://httpd.apache.org/lists.html#http-dev You should just send a mail to dev-unsubscr...@httpd.apache.org. best regards, CJ > Message du 02/06/14 16:16 > De : "Mihai Iacob" > A : dev@httpd.apache.org > Copie à : > Objet : Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT > support > > > How do I unsubscribe from this list ? > > Regards, > Mihai Iacob > DB2 Security Development > DB2 pureScale Development > Phone: (905) 413-5378 > Email: mia...@ca.ibm.com >
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
How do I unsubscribe from this list ?
Regards,
Mihai Iacob
DB2 Security Development
DB2 pureScale Development
Phone: (905) 413-5378
Email: mia...@ca.ibm.com
From: Jim Jagielski
To: dev@httpd.apache.org
Date: 06/02/2014 07:36 AM
Subject:Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
SO_REUSEPORT support
Thanks!
On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote:
> Hi Jim,
>
> Personally, I think the second approach is better, it keeps
ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones,
only modifies dummy_connection (). Please let me know if you have different
opinions.
>
> Attached is the latest version of the two patches. They were both
generated against trunk rev. 1598561. Please review them and let me know if
there is anything missing.
>
> I already updated the Bugzilla database for the item 55897 and item
56279.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Saturday, May 31, 2014 11:48 PM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
SO_REUSEPORT support
>
> Hi Jim,
>
> Regarding to your comment #2, yes, you are right, it should be
ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much
for catching this.
>
> Regarding to your comment #1, the patch modifies the dummy_connection
(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int child_bucket).
Inside the function, the reference listen statement is mpm_listen
[child_bucket]. And, ap_mpm_pod_signal() calls dummy_connection().
>
> Can we just modify the return of ap_mpm_pod_signal() from
dummy_connection(pod) to dummy_connection(pod, 0) and add
ap_mpm_pod_signal_ex()?
>
> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
modify dummy_connection() to send dummy data via all the duplicated listen
statements. Then, we do not need child_bucket as the input parameter for
dummy_connection(). In this case, we do not need adding
ap_mpm_pod_signal_ex() too.
>
> I already tested the code for the above approaches and they both work.
>
> Please let me know which way you think is better. I can quickly send you
an update for review.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Lu, Yingqi
> Sent: Saturday, May 31, 2014 3:28 PM
> To:
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
SO_REUSEPORT support
>
> Hi Jim,
>
> Thanks very much for your email! I will look into both of them and send
an update tonight!
>
> Thanks,
> Yingqi
>
>> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>>
>> I also see:
>>
>>/* kill off the idle ones */
>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>> +for (i = 0; i < num_buckets; i++) {
>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>> +}
>>
>>
>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, retained->
max_daemons_limit, i); ??
>>
>> /**
>> * Write data to the pipe-of-death, signalling that all child process
>> * should die.
>> * @param pod The pipe-of-death to write to.
>> * @param num The number of child processes to kill
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int
>> +child_bucket);
>>
>> Isn't 'num' the same in both implementation??
>>
>>> On May 31, 2014, at 12:03 PM, Jim Jagielski wrote:
>>>
>>> Sorry I didn't catch this earlier:
>>>
>>> I see
>>>
>>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16
13:07:03.892987491 -0400
>>> @@ -267,16 +267,18 @@
>>> * Write data to the pipe-of-death, signalling that one child process
>>> * should die.
>>> * @param pod the pipe-of-death to write to.
>>> + * @param my_bucket the bucket that holds the dying child process.
>>> */
>>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int
>>> +child_bucket);
>>>
>>> We can change the API at this point. We could add another function,
>>> eg ap_mpm_pod_signal_ex() which takes the int param, but we can't
>>> modify ap_mpm_pod_signal() itself.
>>>
>>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi wrote:
>>>>
>>>> Thank you very much!
>>>>
>>>> Thanks,
>>>> Yingqi
>>>>
>>>
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Thanks!
On Jun 2, 2014, at 4:22 AM, Lu, Yingqi wrote:
> Hi Jim,
>
> Personally, I think the second approach is better, it keeps ap_mpm_pod_signal
> () and ap_mpm_pod_killpg () exactly as the original ones, only modifies
> dummy_connection (). Please let me know if you have different opinions.
>
> Attached is the latest version of the two patches. They were both generated
> against trunk rev. 1598561. Please review them and let me know if there is
> anything missing.
>
> I already updated the Bugzilla database for the item 55897 and item 56279.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Saturday, May 31, 2014 11:48 PM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
> support
>
> Hi Jim,
>
> Regarding to your comment #2, yes, you are right, it should be
> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much for
> catching this.
>
> Regarding to your comment #1, the patch modifies the
> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int
> child_bucket). Inside the function, the reference listen statement is
> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls dummy_connection().
>
> Can we just modify the return of ap_mpm_pod_signal() from
> dummy_connection(pod) to dummy_connection(pod, 0) and add
> ap_mpm_pod_signal_ex()?
>
> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
> modify dummy_connection() to send dummy data via all the duplicated listen
> statements. Then, we do not need child_bucket as the input parameter for
> dummy_connection(). In this case, we do not need adding
> ap_mpm_pod_signal_ex() too.
>
> I already tested the code for the above approaches and they both work.
>
> Please let me know which way you think is better. I can quickly send you an
> update for review.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Lu, Yingqi
> Sent: Saturday, May 31, 2014 3:28 PM
> To:
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
> support
>
> Hi Jim,
>
> Thanks very much for your email! I will look into both of them and send an
> update tonight!
>
> Thanks,
> Yingqi
>
>> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>>
>> I also see:
>>
>>/* kill off the idle ones */
>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>> +for (i = 0; i < num_buckets; i++) {
>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>> +}
>>
>>
>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
>> retained->max_daemons_limit, i); ??
>>
>> /**
>> * Write data to the pipe-of-death, signalling that all child process
>> * should die.
>> * @param pod The pipe-of-death to write to.
>> * @param num The number of child processes to kill
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int
>> +child_bucket);
>>
>> Isn't 'num' the same in both implementation??
>>
>>> On May 31, 2014, at 12:03 PM, Jim Jagielski wrote:
>>>
>>> Sorry I didn't catch this earlier:
>>>
>>> I see
>>>
>>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491
>>> -0400
>>> @@ -267,16 +267,18 @@
>>> * Write data to the pipe-of-death, signalling that one child process
>>> * should die.
>>> * @param pod the pipe-of-death to write to.
>>> + * @param my_bucket the bucket that holds the dying child process.
>>> */
>>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int
>>> +child_bucket);
>>>
>>> We can change the API at this point. We could add another function,
>>> eg ap_mpm_pod_signal_ex() which takes the int param, but we can't
>>> modify ap_mpm_pod_signal() itself.
>>>
>>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi wrote:
>>>>
>>>> Thank you very much!
>>>>
>>>> Thanks,
>>>> Yingqi
>>>>
>>>> -Original Message-
>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>> Sent: Friday, May 30, 2014 7:07 AM
>>>> To:
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Jim,
Regarding to your comment #2, yes, you are right, it should be
ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much for
catching this.
Regarding to your comment #1, the patch modifies the dummy_connection(ap_pod_t
*pod) to be dummy_connection(ap_pod_t *pod, int child_bucket). Inside the
function, the reference listen statement is mpm_listen[child_bucket]. And,
ap_mpm_pod_signal() calls dummy_connection().
Can we just modify the return of ap_mpm_pod_signal() from dummy_connection(pod)
to dummy_connection(pod, 0) and add ap_mpm_pod_signal_ex()?
Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
modify dummy_connection() to send dummy data via all the duplicated listen
statements. Then, we do not need child_bucket as the input parameter for
dummy_connection(). In this case, we do not need adding ap_mpm_pod_signal_ex()
too.
I already tested the code for the above approaches and they both work.
Please let me know which way you think is better. I can quickly send you an
update for review.
Thanks,
Yingqi
-Original Message-
From: Lu, Yingqi
Sent: Saturday, May 31, 2014 3:28 PM
To:
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
support
Hi Jim,
Thanks very much for your email! I will look into both of them and send an
update tonight!
Thanks,
Yingqi
> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>
> I also see:
>
> /* kill off the idle ones */
> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
> +for (i = 0; i < num_buckets; i++) {
> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
> +}
>
>
> Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
> retained->max_daemons_limit, i); ??
>
> /**
> * Write data to the pipe-of-death, signalling that all child process
> * should die.
> * @param pod The pipe-of-death to write to.
> * @param num The number of child processes to kill
> + * @param my_bucket the bucket that holds the dying child process.
> */
> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int
> +child_bucket);
>
> Isn't 'num' the same in both implementation??
>
>> On May 31, 2014, at 12:03 PM, Jim Jagielski wrote:
>>
>> Sorry I didn't catch this earlier:
>>
>> I see
>>
>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491
>> -0400
>> @@ -267,16 +267,18 @@
>> * Write data to the pipe-of-death, signalling that one child process
>> * should die.
>> * @param pod the pipe-of-death to write to.
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int
>> +child_bucket);
>>
>> We can change the API at this point. We could add another function,
>> eg ap_mpm_pod_signal_ex() which takes the int param, but we can't
>> modify ap_mpm_pod_signal() itself.
>>
>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi wrote:
>>>
>>> Thank you very much!
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>> Sent: Friday, May 30, 2014 7:07 AM
>>> To: dev@httpd.apache.org
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>> SO_REUSEPORT support
>>>
>>> Thx! Let me review. My plan is to fold into trunk this weekend.
>>>
>>>> On May 16, 2014, at 2:53 PM, Lu, Yingqi wrote:
>>>>
>>>> Hi Jim,
>>>>
>>>> Thanks very much for clarifying this with me. I added #ifdef in the code
>>>> to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does
>>>> not use this parameter so that it remains the same.
>>>>
>>>> Attached are the two most recent patches. I already updated the bugzilla
>>>> #55897 as well.
>>>>
>>>> Thanks,
>>>> Yingqi
>>>>
>>>> -Original Message-
>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>> Sent: Thursday, May 15, 2014 7:53 AM
>>>> To: dev@httpd.apache.org
>>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>>> SO_REUSEPORT support
>>>>
>>>> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
>>>> We could either check for that during config/
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Jim,
Thanks very much for your email! I will look into both of them and send an
update tonight!
Thanks,
Yingqi
> On May 31, 2014, at 9:43 AM, "Jim Jagielski" wrote:
>
> I also see:
>
> /* kill off the idle ones */
> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
> +for (i = 0; i < num_buckets; i++) {
> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
> +}
>
>
> Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
> retained->max_daemons_limit, i); ??
>
> /**
> * Write data to the pipe-of-death, signalling that all child process
> * should die.
> * @param pod The pipe-of-death to write to.
> * @param num The number of child processes to kill
> + * @param my_bucket the bucket that holds the dying child process.
> */
> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int child_bucket);
>
> Isn't 'num' the same in both implementation??
>
>> On May 31, 2014, at 12:03 PM, Jim Jagielski wrote:
>>
>> Sorry I didn't catch this earlier:
>>
>> I see
>>
>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491
>> -0400
>> @@ -267,16 +267,18 @@
>> * Write data to the pipe-of-death, signalling that one child process
>> * should die.
>> * @param pod the pipe-of-death to write to.
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int child_bucket);
>>
>> We can change the API at this point. We could
>> add another function, eg ap_mpm_pod_signal_ex() which
>> takes the int param, but we can't modify ap_mpm_pod_signal()
>> itself.
>>
>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi wrote:
>>>
>>> Thank you very much!
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>> Sent: Friday, May 30, 2014 7:07 AM
>>> To: dev@httpd.apache.org
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
>>> support
>>>
>>> Thx! Let me review. My plan is to fold into trunk this weekend.
>>>
>>>> On May 16, 2014, at 2:53 PM, Lu, Yingqi wrote:
>>>>
>>>> Hi Jim,
>>>>
>>>> Thanks very much for clarifying this with me. I added #ifdef in the code
>>>> to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does
>>>> not use this parameter so that it remains the same.
>>>>
>>>> Attached are the two most recent patches. I already updated the bugzilla
>>>> #55897 as well.
>>>>
>>>> Thanks,
>>>> Yingqi
>>>>
>>>> -Original Message-
>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>> Sent: Thursday, May 15, 2014 7:53 AM
>>>> To: dev@httpd.apache.org
>>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>>> SO_REUSEPORT support
>>>>
>>>> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
>>>> We could either check for that during config/build or protect it with a
>>>> #ifdef in the code (and create some logging so the admin nows if it was
>>>> found or not).
>>>>
>>>>> On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote:
>>>>>
>>>>> Hi Jim,
>>>>>
>>>>> Thanks very much for your email.
>>>>>
>>>>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside
>>>>> listen.c file. If the feature is not supported on the OS (for example,
>>>>> Linux kernel < 3.9), it will fall back to the original behavior.
>>>>>
>>>>> In the bucket patch, there is no need to check the params. With single
>>>>> listen statement, it is just the default behavior.
>>>>>
>>>>> Please let me know if this answers your question.
>>>>>
>>>>> Thanks,
>>>>> Yingqi
>>>>>
>>>>> -Original Message-
>>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>>> Sent: Wednesday, May 14, 2014 6:57 AM
>>>>> To: dev@httpd.apache.org
>>>>> S
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
I also see:
/* kill off the idle ones */
-ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
+for (i = 0; i < num_buckets; i++) {
+ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
+}
Is that right? Why isn't it: ap_mpm_pod_killpg(pod,
retained->max_daemons_limit, i); ??
/**
* Write data to the pipe-of-death, signalling that all child process
* should die.
* @param pod The pipe-of-death to write to.
* @param num The number of child processes to kill
+ * @param my_bucket the bucket that holds the dying child process.
*/
-AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
+AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int child_bucket);
Isn't 'num' the same in both implementation??
On May 31, 2014, at 12:03 PM, Jim Jagielski wrote:
> Sorry I didn't catch this earlier:
>
> I see
>
> +++ httpd-trunk.new/include/mpm_common.h 2014-05-16 13:07:03.892987491
> -0400
> @@ -267,16 +267,18 @@
> * Write data to the pipe-of-death, signalling that one child process
> * should die.
> * @param pod the pipe-of-death to write to.
> + * @param my_bucket the bucket that holds the dying child process.
> */
> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int child_bucket);
>
> We can change the API at this point. We could
> add another function, eg ap_mpm_pod_signal_ex() which
> takes the int param, but we can't modify ap_mpm_pod_signal()
> itself.
>
> On May 30, 2014, at 11:15 AM, Lu, Yingqi wrote:
>
>> Thank you very much!
>>
>> Thanks,
>> Yingqi
>>
>> -Original Message-
>> From: Jim Jagielski [mailto:j...@jagunet.com]
>> Sent: Friday, May 30, 2014 7:07 AM
>> To: dev@httpd.apache.org
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
>> support
>>
>> Thx! Let me review. My plan is to fold into trunk this weekend.
>>
>> On May 16, 2014, at 2:53 PM, Lu, Yingqi wrote:
>>
>>> Hi Jim,
>>>
>>> Thanks very much for clarifying this with me. I added #ifdef in the code to
>>> check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not
>>> use this parameter so that it remains the same.
>>>
>>> Attached are the two most recent patches. I already updated the bugzilla
>>> #55897 as well.
>>>
>>> Thanks,
>>> Yingqi
>>>
>>> -Original Message-
>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>> Sent: Thursday, May 15, 2014 7:53 AM
>>> To: dev@httpd.apache.org
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>> SO_REUSEPORT support
>>>
>>> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
>>> We could either check for that during config/build or protect it with a
>>> #ifdef in the code (and create some logging so the admin nows if it was
>>> found or not).
>>>
>>> On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote:
>>>
>>>> Hi Jim,
>>>>
>>>> Thanks very much for your email.
>>>>
>>>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c
>>>> file. If the feature is not supported on the OS (for example, Linux kernel
>>>> < 3.9), it will fall back to the original behavior.
>>>>
>>>> In the bucket patch, there is no need to check the params. With single
>>>> listen statement, it is just the default behavior.
>>>>
>>>> Please let me know if this answers your question.
>>>>
>>>> Thanks,
>>>> Yingqi
>>>>
>>>> -Original Message-
>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>> Sent: Wednesday, May 14, 2014 6:57 AM
>>>> To: dev@httpd.apache.org
>>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>>> SO_REUSEPORT support
>>>>
>>>> This is very cool!
>>>>
>>>> mod_status assumes that sysconf() exists, but do we need to do a config
>>>> check on the params we use in these patches?
>>>> We look OK on Linux, FreeBSD and OSX...
>>>>
>>>> I'm +1 on folding into trunk.
>>>>
>>>> On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote:
>>>>
>>>>> Dear All,
>>>>>
>>>>> During the last couple weeks, I spent some time extending
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Sorry I didn't catch this earlier: I see +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491 -0400 @@ -267,16 +267,18 @@ * Write data to the pipe-of-death, signalling that one child process * should die. * @param pod the pipe-of-death to write to. + * @param my_bucket the bucket that holds the dying child process. */ -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod); +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int child_bucket); We can change the API at this point. We could add another function, eg ap_mpm_pod_signal_ex() which takes the int param, but we can't modify ap_mpm_pod_signal() itself. On May 30, 2014, at 11:15 AM, Lu, Yingqi wrote: > Thank you very much! > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Friday, May 30, 2014 7:07 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT > support > > Thx! Let me review. My plan is to fold into trunk this weekend. > > On May 16, 2014, at 2:53 PM, Lu, Yingqi wrote: > >> Hi Jim, >> >> Thanks very much for clarifying this with me. I added #ifdef in the code to >> check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not >> use this parameter so that it remains the same. >> >> Attached are the two most recent patches. I already updated the bugzilla >> #55897 as well. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Jim Jagielski [mailto:j...@jagunet.com] >> Sent: Thursday, May 15, 2014 7:53 AM >> To: dev@httpd.apache.org >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >> SO_REUSEPORT support >> >> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff... >> We could either check for that during config/build or protect it with a >> #ifdef in the code (and create some logging so the admin nows if it was >> found or not). >> >> On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote: >> >>> Hi Jim, >>> >>> Thanks very much for your email. >>> >>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c >>> file. If the feature is not supported on the OS (for example, Linux kernel >>> < 3.9), it will fall back to the original behavior. >>> >>> In the bucket patch, there is no need to check the params. With single >>> listen statement, it is just the default behavior. >>> >>> Please let me know if this answers your question. >>> >>> Thanks, >>> Yingqi >>> >>> -Original Message- >>> From: Jim Jagielski [mailto:j...@jagunet.com] >>> Sent: Wednesday, May 14, 2014 6:57 AM >>> To: dev@httpd.apache.org >>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >>> SO_REUSEPORT support >>> >>> This is very cool! >>> >>> mod_status assumes that sysconf() exists, but do we need to do a config >>> check on the params we use in these patches? >>> We look OK on Linux, FreeBSD and OSX... >>> >>> I'm +1 on folding into trunk. >>> >>> On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: >>> >>>> Dear All, >>>> >>>> During the last couple weeks, I spent some time extending the original two >>>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and >>>> event). Attached is the latest version of the two patches. Bugzilla >>>> database has also been updated already. The ID for the two patches are >>>> #55897 and #56279. Please refer to messages below for details on both of >>>> the patches. >>>> >>>> Quick test result on modern dual socket Intel platform (Linux Kernel >>>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897) >>>> 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 >>>> listen statements: 2.33X throughput improvement >>>> 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 >>>> listen statements: 35% throughput improvement >>>> 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 >>>> listen statements: throughput parity, but 62% response time reduction >>>> (with patch, 38% response time as original SW) >>>> >>>> Bucket patch (bugzilla #56279, only impact multiple listen statement case) >>>> 1. Prefork MPM: 2 listen statem
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Thank you very much! Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Friday, May 30, 2014 7:07 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Thx! Let me review. My plan is to fold into trunk this weekend. On May 16, 2014, at 2:53 PM, Lu, Yingqi wrote: > Hi Jim, > > Thanks very much for clarifying this with me. I added #ifdef in the code to > check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not > use this parameter so that it remains the same. > > Attached are the two most recent patches. I already updated the bugzilla > #55897 as well. > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Thursday, May 15, 2014 7:53 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff... > We could either check for that during config/build or protect it with a > #ifdef in the code (and create some logging so the admin nows if it was found > or not). > > On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote: > >> Hi Jim, >> >> Thanks very much for your email. >> >> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c >> file. If the feature is not supported on the OS (for example, Linux kernel < >> 3.9), it will fall back to the original behavior. >> >> In the bucket patch, there is no need to check the params. With single >> listen statement, it is just the default behavior. >> >> Please let me know if this answers your question. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Jim Jagielski [mailto:j...@jagunet.com] >> Sent: Wednesday, May 14, 2014 6:57 AM >> To: dev@httpd.apache.org >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >> SO_REUSEPORT support >> >> This is very cool! >> >> mod_status assumes that sysconf() exists, but do we need to do a config >> check on the params we use in these patches? >> We look OK on Linux, FreeBSD and OSX... >> >> I'm +1 on folding into trunk. >> >> On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: >> >>> Dear All, >>> >>> During the last couple weeks, I spent some time extending the original two >>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and >>> event). Attached is the latest version of the two patches. Bugzilla >>> database has also been updated already. The ID for the two patches are >>> #55897 and #56279. Please refer to messages below for details on both of >>> the patches. >>> >>> Quick test result on modern dual socket Intel platform (Linux Kernel >>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897) >>> 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 >>> listen statements: 2.33X throughput improvement >>> 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 >>> listen statements: 35% throughput improvement >>> 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 >>> listen statements: throughput parity, but 62% response time reduction (with >>> patch, 38% response time as original SW) >>> >>> Bucket patch (bugzilla #56279, only impact multiple listen statement case) >>> 1. Prefork MPM: 2 listen statements: 42% throughput improvement >>> 2. Worker MPM: 2 listen statements: 7% throughput improvement >>> >>> In all the above testing cases, significant response time reductions are >>> observed, even with throughput improvements. >>> >>> Please let me know your feedback and comments. >>> >>> Thanks, >>> Yingqi >>> Software and workloads used in performance tests may have been optimized >>> for performance only on Intel microprocessors. Performance tests, such as >>> SYSmark and MobileMark, are measured using specific computer systems, >>> components, software, operations and functions. Any change to any of those >>> factors may cause the results to vary. You should consult other information >>> and performance tests to assist you in fully evaluating your contemplated >>> purchases, including the performance of that product when combined with >>> other products. >>> >>> From: Lu, Yingqi [mailto:yingqi...@intel.com] &g
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
how do I unsubsribe from this list ? Regards, Mihai Iacob DB2 Security Development DB2 pureScale Development Phone: (905) 413-5378 Email: mia...@ca.ibm.com From: Jim Jagielski To: dev@httpd.apache.org Date: 05/30/2014 10:07 AM Subject:Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Thx! Let me review. My plan is to fold into trunk this weekend. On May 16, 2014, at 2:53 PM, Lu, Yingqi wrote: > Hi Jim, > > Thanks very much for clarifying this with me. I added #ifdef in the code to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not use this parameter so that it remains the same. > > Attached are the two most recent patches. I already updated the bugzilla #55897 as well. > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Thursday, May 15, 2014 7:53 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support > > I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff... > We could either check for that during config/build or protect it with a #ifdef in the code (and create some logging so the admin nows if it was found or not). > > On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote: > >> Hi Jim, >> >> Thanks very much for your email. >> >> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c file. If the feature is not supported on the OS (for example, Linux kernel < 3.9), it will fall back to the original behavior. >> >> In the bucket patch, there is no need to check the params. With single listen statement, it is just the default behavior. >> >> Please let me know if this answers your question. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Jim Jagielski [mailto:j...@jagunet.com] >> Sent: Wednesday, May 14, 2014 6:57 AM >> To: dev@httpd.apache.org >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >> SO_REUSEPORT support >> >> This is very cool! >> >> mod_status assumes that sysconf() exists, but do we need to do a config check on the params we use in these patches? >> We look OK on Linux, FreeBSD and OSX... >> >> I'm +1 on folding into trunk. >> >> On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: >> >>> Dear All, >>> >>> During the last couple weeks, I spent some time extending the original two patches from prefork MPM only to all three Linux MPMs (prefork, worker and event). Attached is the latest version of the two patches. Bugzilla database has also been updated already. The ID for the two patches are #55897 and #56279. Please refer to messages below for details on both of the patches. >>> >>> Quick test result on modern dual socket Intel platform (Linux Kernel >>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897) >>> 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 listen statements: 2.33X throughput improvement >>> 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen statements: 35% throughput improvement >>> 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen statements: throughput parity, but 62% response time reduction (with patch, 38% response time as original SW) >>> >>> Bucket patch (bugzilla #56279, only impact multiple listen statement case) >>> 1. Prefork MPM: 2 listen statements: 42% throughput improvement >>> 2. Worker MPM: 2 listen statements: 7% throughput improvement >>> >>> In all the above testing cases, significant response time reductions are observed, even with throughput improvements. >>> >>> Please let me know your feedback and comments. >>> >>> Thanks, >>> Yingqi >>> Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. >>> >>> From: Lu, Yingqi [mailto:yingqi...@intel.com] >>> Sent: Monday, March 17, 2014 1:41 PM >>> To: dev@httpd.apache.org >>> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >>> SO_REUSEPORT support >>> >>> Dear all, >>> >>
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Thx! Let me review. My plan is to fold into trunk this weekend. On May 16, 2014, at 2:53 PM, Lu, Yingqi wrote: > Hi Jim, > > Thanks very much for clarifying this with me. I added #ifdef in the code to > check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not > use this parameter so that it remains the same. > > Attached are the two most recent patches. I already updated the bugzilla > #55897 as well. > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Thursday, May 15, 2014 7:53 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT > support > > I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff... > We could either check for that during config/build or protect it with a > #ifdef in the code (and create some logging so the admin nows if it was found > or not). > > On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote: > >> Hi Jim, >> >> Thanks very much for your email. >> >> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c >> file. If the feature is not supported on the OS (for example, Linux kernel < >> 3.9), it will fall back to the original behavior. >> >> In the bucket patch, there is no need to check the params. With single >> listen statement, it is just the default behavior. >> >> Please let me know if this answers your question. >> >> Thanks, >> Yingqi >> >> -Original Message- >> From: Jim Jagielski [mailto:j...@jagunet.com] >> Sent: Wednesday, May 14, 2014 6:57 AM >> To: dev@httpd.apache.org >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >> SO_REUSEPORT support >> >> This is very cool! >> >> mod_status assumes that sysconf() exists, but do we need to do a config >> check on the params we use in these patches? >> We look OK on Linux, FreeBSD and OSX... >> >> I'm +1 on folding into trunk. >> >> On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: >> >>> Dear All, >>> >>> During the last couple weeks, I spent some time extending the original two >>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and >>> event). Attached is the latest version of the two patches. Bugzilla >>> database has also been updated already. The ID for the two patches are >>> #55897 and #56279. Please refer to messages below for details on both of >>> the patches. >>> >>> Quick test result on modern dual socket Intel platform (Linux Kernel >>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897) >>> 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 >>> listen statements: 2.33X throughput improvement >>> 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 >>> listen statements: 35% throughput improvement >>> 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 >>> listen statements: throughput parity, but 62% response time reduction (with >>> patch, 38% response time as original SW) >>> >>> Bucket patch (bugzilla #56279, only impact multiple listen statement case) >>> 1. Prefork MPM: 2 listen statements: 42% throughput improvement >>> 2. Worker MPM: 2 listen statements: 7% throughput improvement >>> >>> In all the above testing cases, significant response time reductions are >>> observed, even with throughput improvements. >>> >>> Please let me know your feedback and comments. >>> >>> Thanks, >>> Yingqi >>> Software and workloads used in performance tests may have been optimized >>> for performance only on Intel microprocessors. Performance tests, such as >>> SYSmark and MobileMark, are measured using specific computer systems, >>> components, software, operations and functions. Any change to any of those >>> factors may cause the results to vary. You should consult other information >>> and performance tests to assist you in fully evaluating your contemplated >>> purchases, including the performance of that product when combined with >>> other products. >>> >>> From: Lu, Yingqi [mailto:yingqi...@intel.com] >>> Sent: Monday, March 17, 2014 1:41 PM >>> To: dev@httpd.apache.org >>> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >>> SO_REUSEPORT support >>> >>> Dear all, >>> >>>
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi All, I just want to ping again on these two patches. Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Friday, May 23, 2014 9:03 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Dear All, These two patches are the modified version based on the original single patch I sent out early January. Since I sent out the original patch, I have gotten lots of positive feedbacks and suggestions from this community. Thanks very much for your support and help! The two most recent patches have addressed all the suggestions I got from you so far and they include changes for all three Linux MPMs. They have also been fully tested. Please refer to the change details and test results in the email below. If there are no other comments/suggestions, should we fold them into the trunk? Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Friday, May 16, 2014 11:53 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jim, Thanks very much for clarifying this with me. I added #ifdef in the code to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not use this parameter so that it remains the same. Attached are the two most recent patches. I already updated the bugzilla #55897 as well. Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Thursday, May 15, 2014 7:53 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff... We could either check for that during config/build or protect it with a #ifdef in the code (and create some logging so the admin nows if it was found or not). On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote: > Hi Jim, > > Thanks very much for your email. > > In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c > file. If the feature is not supported on the OS (for example, Linux kernel < > 3.9), it will fall back to the original behavior. > > In the bucket patch, there is no need to check the params. With single listen > statement, it is just the default behavior. > > Please let me know if this answers your question. > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Wednesday, May 14, 2014 6:57 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > This is very cool! > > mod_status assumes that sysconf() exists, but do we need to do a config check > on the params we use in these patches? > We look OK on Linux, FreeBSD and OSX... > > I'm +1 on folding into trunk. > > On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > >> Dear All, >> >> During the last couple weeks, I spent some time extending the original two >> patches from prefork MPM only to all three Linux MPMs (prefork, worker and >> event). Attached is the latest version of the two patches. Bugzilla database >> has also been updated already. The ID for the two patches are #55897 and >> #56279. Please refer to messages below for details on both of the patches. >> >> Quick test result on modern dual socket Intel platform (Linux Kernel >> 3.13.9) SO_REUSEPORT patch (bugzilla #55897) >> 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 >> listen statements: 2.33X throughput improvement >> 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 >> listen statements: 35% throughput improvement >> 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen >> statements: throughput parity, but 62% response time reduction (with patch, >> 38% response time as original SW) >> >> Bucket patch (bugzilla #56279, only impact multiple listen statement case) >> 1. Prefork MPM: 2 listen statements: 42% throughput improvement >> 2. Worker MPM: 2 listen statements: 7% throughput improvement >> >> In all the above testing cases, significant response time reductions are >> observed, even with throughput improvements. >> >> Please let me know your feedback and comments. >> >> Thanks, >> Yingqi >> Software and workloads used in performance tests may have been optimized for >> performance only on Intel microprocessors. Performance tests, such as >> SYSmark and MobileMark, are measured using specific computer systems, >> components, software, operations and functions. Any change to any of those >> factors
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Dear All, These two patches are the modified version based on the original single patch I sent out early January. Since I sent out the original patch, I have gotten lots of positive feedbacks and suggestions from this community. Thanks very much for your support and help! The two most recent patches have addressed all the suggestions I got from you so far and they include changes for all three Linux MPMs. They have also been fully tested. Please refer to the change details and test results in the email below. If there are no other comments/suggestions, should we fold them into the trunk? Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Friday, May 16, 2014 11:53 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jim, Thanks very much for clarifying this with me. I added #ifdef in the code to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not use this parameter so that it remains the same. Attached are the two most recent patches. I already updated the bugzilla #55897 as well. Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Thursday, May 15, 2014 7:53 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff... We could either check for that during config/build or protect it with a #ifdef in the code (and create some logging so the admin nows if it was found or not). On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote: > Hi Jim, > > Thanks very much for your email. > > In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c > file. If the feature is not supported on the OS (for example, Linux kernel < > 3.9), it will fall back to the original behavior. > > In the bucket patch, there is no need to check the params. With single listen > statement, it is just the default behavior. > > Please let me know if this answers your question. > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Wednesday, May 14, 2014 6:57 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > This is very cool! > > mod_status assumes that sysconf() exists, but do we need to do a config check > on the params we use in these patches? > We look OK on Linux, FreeBSD and OSX... > > I'm +1 on folding into trunk. > > On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > >> Dear All, >> >> During the last couple weeks, I spent some time extending the original two >> patches from prefork MPM only to all three Linux MPMs (prefork, worker and >> event). Attached is the latest version of the two patches. Bugzilla database >> has also been updated already. The ID for the two patches are #55897 and >> #56279. Please refer to messages below for details on both of the patches. >> >> Quick test result on modern dual socket Intel platform (Linux Kernel >> 3.13.9) SO_REUSEPORT patch (bugzilla #55897) >> 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 >> listen statements: 2.33X throughput improvement >> 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 >> listen statements: 35% throughput improvement >> 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen >> statements: throughput parity, but 62% response time reduction (with patch, >> 38% response time as original SW) >> >> Bucket patch (bugzilla #56279, only impact multiple listen statement case) >> 1. Prefork MPM: 2 listen statements: 42% throughput improvement >> 2. Worker MPM: 2 listen statements: 7% throughput improvement >> >> In all the above testing cases, significant response time reductions are >> observed, even with throughput improvements. >> >> Please let me know your feedback and comments. >> >> Thanks, >> Yingqi >> Software and workloads used in performance tests may have been optimized for >> performance only on Intel microprocessors. Performance tests, such as >> SYSmark and MobileMark, are measured using specific computer systems, >> components, software, operations and functions. Any change to any of those >> factors may cause the results to vary. You should consult other information >> and performance tests to assist you in fully evaluating your contemplated >> purchases, including the performance of that product when combined with >> other products. >> >> From: Lu, Yingqi
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Dear All, I am checking if there are any questions/comments on both of the patches? Also, I am wondering what the process of patch acceptance is. Please let me know if there is anything I can do to help. Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Friday, May 16, 2014 11:53 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jim, Thanks very much for clarifying this with me. I added #ifdef in the code to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not use this parameter so that it remains the same. Attached are the two most recent patches. I already updated the bugzilla #55897 as well. Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Thursday, May 15, 2014 7:53 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff... We could either check for that during config/build or protect it with a #ifdef in the code (and create some logging so the admin nows if it was found or not). On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote: > Hi Jim, > > Thanks very much for your email. > > In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c > file. If the feature is not supported on the OS (for example, Linux kernel < > 3.9), it will fall back to the original behavior. > > In the bucket patch, there is no need to check the params. With single listen > statement, it is just the default behavior. > > Please let me know if this answers your question. > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Wednesday, May 14, 2014 6:57 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > This is very cool! > > mod_status assumes that sysconf() exists, but do we need to do a config check > on the params we use in these patches? > We look OK on Linux, FreeBSD and OSX... > > I'm +1 on folding into trunk. > > On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > >> Dear All, >> >> During the last couple weeks, I spent some time extending the original two >> patches from prefork MPM only to all three Linux MPMs (prefork, worker and >> event). Attached is the latest version of the two patches. Bugzilla database >> has also been updated already. The ID for the two patches are #55897 and >> #56279. Please refer to messages below for details on both of the patches. >> >> Quick test result on modern dual socket Intel platform (Linux Kernel >> 3.13.9) SO_REUSEPORT patch (bugzilla #55897) >> 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 >> listen statements: 2.33X throughput improvement >> 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 >> listen statements: 35% throughput improvement >> 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen >> statements: throughput parity, but 62% response time reduction (with patch, >> 38% response time as original SW) >> >> Bucket patch (bugzilla #56279, only impact multiple listen statement case) >> 1. Prefork MPM: 2 listen statements: 42% throughput improvement >> 2. Worker MPM: 2 listen statements: 7% throughput improvement >> >> In all the above testing cases, significant response time reductions are >> observed, even with throughput improvements. >> >> Please let me know your feedback and comments. >> >> Thanks, >> Yingqi >> Software and workloads used in performance tests may have been optimized for >> performance only on Intel microprocessors. Performance tests, such as >> SYSmark and MobileMark, are measured using specific computer systems, >> components, software, operations and functions. Any change to any of those >> factors may cause the results to vary. You should consult other information >> and performance tests to assist you in fully evaluating your contemplated >> purchases, including the performance of that product when combined with >> other products. >> >> From: Lu, Yingqi [mailto:yingqi...@intel.com] >> Sent: Monday, March 17, 2014 1:41 PM >> To: dev@httpd.apache.org >> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >> SO_REUSEPORT support >> >> Dear all, >> >> Based on the feedback we received, we modified this patch. Here is the most >> recent version. We also modified the Bugzill
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Dear all, Any other feedback/comments/questions? Thanks, Yingqi -Original Message- From: Lu, Yingqi Sent: Wednesday, May 14, 2014 9:00 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jim, Thanks very much for your email. In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c file. If the feature is not supported on the OS (for example, Linux kernel < 3.9), it will fall back to the original behavior. In the bucket patch, there is no need to check the params. With single listen statement, it is just the default behavior. Please let me know if this answers your question. Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Wednesday, May 14, 2014 6:57 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support This is very cool! mod_status assumes that sysconf() exists, but do we need to do a config check on the params we use in these patches? We look OK on Linux, FreeBSD and OSX... I'm +1 on folding into trunk. On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > Dear All, > > During the last couple weeks, I spent some time extending the original two > patches from prefork MPM only to all three Linux MPMs (prefork, worker and > event). Attached is the latest version of the two patches. Bugzilla database > has also been updated already. The ID for the two patches are #55897 and > #56279. Please refer to messages below for details on both of the patches. > > Quick test result on modern dual socket Intel platform (Linux Kernel > 3.13.9) SO_REUSEPORT patch (bugzilla #55897) > 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 > listen statements: 2.33X throughput improvement > 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen > statements: 35% throughput improvement > 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen > statements: throughput parity, but 62% response time reduction (with patch, > 38% response time as original SW) > > Bucket patch (bugzilla #56279, only impact multiple listen statement case) > 1. Prefork MPM: 2 listen statements: 42% throughput improvement > 2. Worker MPM: 2 listen statements: 7% throughput improvement > > In all the above testing cases, significant response time reductions are > observed, even with throughput improvements. > > Please let me know your feedback and comments. > > Thanks, > Yingqi > Software and workloads used in performance tests may have been optimized for > performance only on Intel microprocessors. Performance tests, such as SYSmark > and MobileMark, are measured using specific computer systems, components, > software, operations and functions. Any change to any of those factors may > cause the results to vary. You should consult other information and > performance tests to assist you in fully evaluating your contemplated > purchases, including the performance of that product when combined with other > products. > > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Monday, March 17, 2014 1:41 PM > To: dev@httpd.apache.org > Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Dear all, > > Based on the feedback we received, we modified this patch. Here is the most > recent version. We also modified the Bugzilla database(Bugzilla# 55897 for > SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). > > Below are the changes we made into this new version: > > According to Yann Ylavic and other people's comments, we separate the > original patch between with and without SO_REUSEPORT into two separated > patches. The SO_REUSEPORT patch does not change the original listen sockets, > it just duplicate the original one into multiple ones. Since the listen > sockets are identical, there is no need to change the idle_server_maintenance > function. The bucket patch (without SO_REUSEPORT), on the other hand, it > breaks down the original listen record (if there are multiple listen socks) > to multiple listen record linked lists. In this case, idle_server_maintenance > is implemented at bucket level to address the situation that imbalanced > traffic occurs among different listen sockets/children buckets. In the bucket > patch, the polling in the child process is removed since each child only > listens to 1 sock. > > According to Arkadiusz Miskiewicz's comment, we make the "detection of > SO_REUSEPORT" at run time. > > According to Jeff Trawick's comments, > 1. We generate the patches against the httpd trunk. > 2. We tested the current pa
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Dear all, Any other feedback/comments/questions? Thanks, Yingqi -Original Message- From: Lu, Yingqi Sent: Wednesday, May 14, 2014 9:00 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jim, Thanks very much for your email. In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c file. If the feature is not supported on the OS (for example, Linux kernel < 3.9), it will fall back to the original behavior. In the bucket patch, there is no need to check the params. With single listen statement, it is just the default behavior. Please let me know if this answers your question. Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Wednesday, May 14, 2014 6:57 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support This is very cool! mod_status assumes that sysconf() exists, but do we need to do a config check on the params we use in these patches? We look OK on Linux, FreeBSD and OSX... I'm +1 on folding into trunk. On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > Dear All, > > During the last couple weeks, I spent some time extending the original two > patches from prefork MPM only to all three Linux MPMs (prefork, worker and > event). Attached is the latest version of the two patches. Bugzilla database > has also been updated already. The ID for the two patches are #55897 and > #56279. Please refer to messages below for details on both of the patches. > > Quick test result on modern dual socket Intel platform (Linux Kernel > 3.13.9) SO_REUSEPORT patch (bugzilla #55897) > 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 > listen statements: 2.33X throughput improvement > 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen > statements: 35% throughput improvement > 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen > statements: throughput parity, but 62% response time reduction (with patch, > 38% response time as original SW) > > Bucket patch (bugzilla #56279, only impact multiple listen statement case) > 1. Prefork MPM: 2 listen statements: 42% throughput improvement > 2. Worker MPM: 2 listen statements: 7% throughput improvement > > In all the above testing cases, significant response time reductions are > observed, even with throughput improvements. > > Please let me know your feedback and comments. > > Thanks, > Yingqi > Software and workloads used in performance tests may have been optimized for > performance only on Intel microprocessors. Performance tests, such as SYSmark > and MobileMark, are measured using specific computer systems, components, > software, operations and functions. Any change to any of those factors may > cause the results to vary. You should consult other information and > performance tests to assist you in fully evaluating your contemplated > purchases, including the performance of that product when combined with other > products. > > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Monday, March 17, 2014 1:41 PM > To: dev@httpd.apache.org > Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Dear all, > > Based on the feedback we received, we modified this patch. Here is the most > recent version. We also modified the Bugzilla database(Bugzilla# 55897 for > SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). > > Below are the changes we made into this new version: > > According to Yann Ylavic and other people's comments, we separate the > original patch between with and without SO_REUSEPORT into two separated > patches. The SO_REUSEPORT patch does not change the original listen sockets, > it just duplicate the original one into multiple ones. Since the listen > sockets are identical, there is no need to change the idle_server_maintenance > function. The bucket patch (without SO_REUSEPORT), on the other hand, it > breaks down the original listen record (if there are multiple listen socks) > to multiple listen record linked lists. In this case, idle_server_maintenance > is implemented at bucket level to address the situation that imbalanced > traffic occurs among different listen sockets/children buckets. In the bucket > patch, the polling in the child process is removed since each child only > listens to 1 sock. > > According to Arkadiusz Miskiewicz's comment, we make the "detection of > SO_REUSEPORT" at run time. > > According to Jeff Trawick's comments, > 1. We generate the patches against the httpd trunk. > 2. We tested the current pa
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Dear all, I sent this message twice in the morning, but I did not receive anything back. I am resending this one more time, just trying to gather your feedback/comments/questions on both of the patches. Sorry for the duplications. Thanks, Yingqi -Original Message- From: Lu, Yingqi Sent: Wednesday, May 14, 2014 9:00 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jim, Thanks very much for your email. In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c file. If the feature is not supported on the OS (for example, Linux kernel < 3.9), it will fall back to the original behavior. In the bucket patch, there is no need to check the params. With single listen statement, it is just the default behavior. Please let me know if this answers your question. Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Wednesday, May 14, 2014 6:57 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support This is very cool! mod_status assumes that sysconf() exists, but do we need to do a config check on the params we use in these patches? We look OK on Linux, FreeBSD and OSX... I'm +1 on folding into trunk. On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > Dear All, > > During the last couple weeks, I spent some time extending the original two > patches from prefork MPM only to all three Linux MPMs (prefork, worker and > event). Attached is the latest version of the two patches. Bugzilla database > has also been updated already. The ID for the two patches are #55897 and > #56279. Please refer to messages below for details on both of the patches. > > Quick test result on modern dual socket Intel platform (Linux Kernel > 3.13.9) SO_REUSEPORT patch (bugzilla #55897) > 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 > listen statements: 2.33X throughput improvement > 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen > statements: 35% throughput improvement > 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen > statements: throughput parity, but 62% response time reduction (with patch, > 38% response time as original SW) > > Bucket patch (bugzilla #56279, only impact multiple listen statement case) > 1. Prefork MPM: 2 listen statements: 42% throughput improvement > 2. Worker MPM: 2 listen statements: 7% throughput improvement > > In all the above testing cases, significant response time reductions are > observed, even with throughput improvements. > > Please let me know your feedback and comments. > > Thanks, > Yingqi > Software and workloads used in performance tests may have been optimized for > performance only on Intel microprocessors. Performance tests, such as SYSmark > and MobileMark, are measured using specific computer systems, components, > software, operations and functions. Any change to any of those factors may > cause the results to vary. You should consult other information and > performance tests to assist you in fully evaluating your contemplated > purchases, including the performance of that product when combined with other > products. > > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Monday, March 17, 2014 1:41 PM > To: dev@httpd.apache.org > Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Dear all, > > Based on the feedback we received, we modified this patch. Here is the most > recent version. We also modified the Bugzilla database(Bugzilla# 55897 for > SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). > > Below are the changes we made into this new version: > > According to Yann Ylavic and other people's comments, we separate the > original patch between with and without SO_REUSEPORT into two separated > patches. The SO_REUSEPORT patch does not change the original listen sockets, > it just duplicate the original one into multiple ones. Since the listen > sockets are identical, there is no need to change the idle_server_maintenance > function. The bucket patch (without SO_REUSEPORT), on the other hand, it > breaks down the original listen record (if there are multiple listen socks) > to multiple listen record linked lists. In this case, idle_server_maintenance > is implemented at bucket level to address the situation that imbalanced > traffic occurs among different listen sockets/children buckets. In the bucket > patch, the polling in the child process is removed since each child only > listens to 1 sock. > > According to Arkadiusz Miskiewicz's comment, we make the "detection of
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff... We could either check for that during config/build or protect it with a #ifdef in the code (and create some logging so the admin nows if it was found or not). On May 14, 2014, at 11:59 AM, Lu, Yingqi wrote: > Hi Jim, > > Thanks very much for your email. > > In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c > file. If the feature is not supported on the OS (for example, Linux kernel < > 3.9), it will fall back to the original behavior. > > In the bucket patch, there is no need to check the params. With single listen > statement, it is just the default behavior. > > Please let me know if this answers your question. > > Thanks, > Yingqi > > -Original Message- > From: Jim Jagielski [mailto:j...@jagunet.com] > Sent: Wednesday, May 14, 2014 6:57 AM > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT > support > > This is very cool! > > mod_status assumes that sysconf() exists, but do we need to do a config check > on the params we use in these patches? > We look OK on Linux, FreeBSD and OSX... > > I'm +1 on folding into trunk. > > On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > >> Dear All, >> >> During the last couple weeks, I spent some time extending the original two >> patches from prefork MPM only to all three Linux MPMs (prefork, worker and >> event). Attached is the latest version of the two patches. Bugzilla database >> has also been updated already. The ID for the two patches are #55897 and >> #56279. Please refer to messages below for details on both of the patches. >> >> Quick test result on modern dual socket Intel platform (Linux Kernel >> 3.13.9) SO_REUSEPORT patch (bugzilla #55897) >> 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 >> listen statements: 2.33X throughput improvement >> 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 >> listen statements: 35% throughput improvement >> 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen >> statements: throughput parity, but 62% response time reduction (with patch, >> 38% response time as original SW) >> >> Bucket patch (bugzilla #56279, only impact multiple listen statement case) >> 1. Prefork MPM: 2 listen statements: 42% throughput improvement >> 2. Worker MPM: 2 listen statements: 7% throughput improvement >> >> In all the above testing cases, significant response time reductions are >> observed, even with throughput improvements. >> >> Please let me know your feedback and comments. >> >> Thanks, >> Yingqi >> Software and workloads used in performance tests may have been optimized for >> performance only on Intel microprocessors. Performance tests, such as >> SYSmark and MobileMark, are measured using specific computer systems, >> components, software, operations and functions. Any change to any of those >> factors may cause the results to vary. You should consult other information >> and performance tests to assist you in fully evaluating your contemplated >> purchases, including the performance of that product when combined with >> other products. >> >> From: Lu, Yingqi [mailto:yingqi...@intel.com] >> Sent: Monday, March 17, 2014 1:41 PM >> To: dev@httpd.apache.org >> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >> SO_REUSEPORT support >> >> Dear all, >> >> Based on the feedback we received, we modified this patch. Here is the most >> recent version. We also modified the Bugzilla database(Bugzilla# 55897 for >> SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). >> >> Below are the changes we made into this new version: >> >> According to Yann Ylavic and other people's comments, we separate the >> original patch between with and without SO_REUSEPORT into two separated >> patches. The SO_REUSEPORT patch does not change the original listen sockets, >> it just duplicate the original one into multiple ones. Since the listen >> sockets are identical, there is no need to change the >> idle_server_maintenance function. The bucket patch (without SO_REUSEPORT), >> on the other hand, it breaks down the original listen record (if there are >> multiple listen socks) to multiple listen record linked lists. In this case, >> idle_server_maintenance is implemented at bucket level to address the >> situation that imbalanced traffic occurs among different liste
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Now, I just got the message I sent out yesterday, what a long delay! Actually, this morning I sent out several messages checking for other feedback/comments/questions. I guess those messages are still in the air since I have not received any copy of those mails. All, please let me know your feedback/comments/questions. Thanks very much for your time reviewing the patches. Thanks, Yingqi -Original Message- From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Wednesday, May 14, 2014 9:00 AM To: dev@httpd.apache.org Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jim, Thanks very much for your email. In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c file. If the feature is not supported on the OS (for example, Linux kernel < 3.9), it will fall back to the original behavior. In the bucket patch, there is no need to check the params. With single listen statement, it is just the default behavior. Please let me know if this answers your question. Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Wednesday, May 14, 2014 6:57 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support This is very cool! mod_status assumes that sysconf() exists, but do we need to do a config check on the params we use in these patches? We look OK on Linux, FreeBSD and OSX... I'm +1 on folding into trunk. On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > Dear All, > > During the last couple weeks, I spent some time extending the original two > patches from prefork MPM only to all three Linux MPMs (prefork, worker and > event). Attached is the latest version of the two patches. Bugzilla database > has also been updated already. The ID for the two patches are #55897 and > #56279. Please refer to messages below for details on both of the patches. > > Quick test result on modern dual socket Intel platform (Linux Kernel > 3.13.9) SO_REUSEPORT patch (bugzilla #55897) > 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 > listen statements: 2.33X throughput improvement > 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen > statements: 35% throughput improvement > 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen > statements: throughput parity, but 62% response time reduction (with patch, > 38% response time as original SW) > > Bucket patch (bugzilla #56279, only impact multiple listen statement case) > 1. Prefork MPM: 2 listen statements: 42% throughput improvement > 2. Worker MPM: 2 listen statements: 7% throughput improvement > > In all the above testing cases, significant response time reductions are > observed, even with throughput improvements. > > Please let me know your feedback and comments. > > Thanks, > Yingqi > Software and workloads used in performance tests may have been optimized for > performance only on Intel microprocessors. Performance tests, such as SYSmark > and MobileMark, are measured using specific computer systems, components, > software, operations and functions. Any change to any of those factors may > cause the results to vary. You should consult other information and > performance tests to assist you in fully evaluating your contemplated > purchases, including the performance of that product when combined with other > products. > > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Monday, March 17, 2014 1:41 PM > To: dev@httpd.apache.org > Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Dear all, > > Based on the feedback we received, we modified this patch. Here is the most > recent version. We also modified the Bugzilla database(Bugzilla# 55897 for > SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). > > Below are the changes we made into this new version: > > According to Yann Ylavic and other people's comments, we separate the > original patch between with and without SO_REUSEPORT into two separated > patches. The SO_REUSEPORT patch does not change the original listen sockets, > it just duplicate the original one into multiple ones. Since the listen > sockets are identical, there is no need to change the idle_server_maintenance > function. The bucket patch (without SO_REUSEPORT), on the other hand, it > breaks down the original listen record (if there are multiple listen socks) > to multiple listen record linked lists. In this case, idle_server_maintenance > is implemented at bucket level to address the situation that imbalanced > traffic occurs among different listen sockets/children buckets. In the bucket > patch, the poll
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Jim, Thanks very much for your email. In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c file. If the feature is not supported on the OS (for example, Linux kernel < 3.9), it will fall back to the original behavior. In the bucket patch, there is no need to check the params. With single listen statement, it is just the default behavior. Please let me know if this answers your question. Thanks, Yingqi -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Wednesday, May 14, 2014 6:57 AM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support This is very cool! mod_status assumes that sysconf() exists, but do we need to do a config check on the params we use in these patches? We look OK on Linux, FreeBSD and OSX... I'm +1 on folding into trunk. On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > Dear All, > > During the last couple weeks, I spent some time extending the original two > patches from prefork MPM only to all three Linux MPMs (prefork, worker and > event). Attached is the latest version of the two patches. Bugzilla database > has also been updated already. The ID for the two patches are #55897 and > #56279. Please refer to messages below for details on both of the patches. > > Quick test result on modern dual socket Intel platform (Linux Kernel > 3.13.9) SO_REUSEPORT patch (bugzilla #55897) > 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 > listen statements: 2.33X throughput improvement > 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen > statements: 35% throughput improvement > 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen > statements: throughput parity, but 62% response time reduction (with patch, > 38% response time as original SW) > > Bucket patch (bugzilla #56279, only impact multiple listen statement case) > 1. Prefork MPM: 2 listen statements: 42% throughput improvement > 2. Worker MPM: 2 listen statements: 7% throughput improvement > > In all the above testing cases, significant response time reductions are > observed, even with throughput improvements. > > Please let me know your feedback and comments. > > Thanks, > Yingqi > Software and workloads used in performance tests may have been optimized for > performance only on Intel microprocessors. Performance tests, such as SYSmark > and MobileMark, are measured using specific computer systems, components, > software, operations and functions. Any change to any of those factors may > cause the results to vary. You should consult other information and > performance tests to assist you in fully evaluating your contemplated > purchases, including the performance of that product when combined with other > products. > > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Monday, March 17, 2014 1:41 PM > To: dev@httpd.apache.org > Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Dear all, > > Based on the feedback we received, we modified this patch. Here is the most > recent version. We also modified the Bugzilla database(Bugzilla# 55897 for > SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). > > Below are the changes we made into this new version: > > According to Yann Ylavic and other people's comments, we separate the > original patch between with and without SO_REUSEPORT into two separated > patches. The SO_REUSEPORT patch does not change the original listen sockets, > it just duplicate the original one into multiple ones. Since the listen > sockets are identical, there is no need to change the idle_server_maintenance > function. The bucket patch (without SO_REUSEPORT), on the other hand, it > breaks down the original listen record (if there are multiple listen socks) > to multiple listen record linked lists. In this case, idle_server_maintenance > is implemented at bucket level to address the situation that imbalanced > traffic occurs among different listen sockets/children buckets. In the bucket > patch, the polling in the child process is removed since each child only > listens to 1 sock. > > According to Arkadiusz Miskiewicz's comment, we make the "detection of > SO_REUSEPORT" at run time. > > According to Jeff Trawick's comments, > 1. We generate the patches against the httpd trunk. > 2. We tested the current patches and they do not impact event and worker > mpms. If current patches can be accepted, we would be happy to extend them to > other Linux based mpms. There are not much code changes, but require some > time to setup the workload to test. > 3.
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
This is very cool! mod_status assumes that sysconf() exists, but do we need to do a config check on the params we use in these patches? We look OK on Linux, FreeBSD and OSX... I'm +1 on folding into trunk. On May 13, 2014, at 7:55 PM, Lu, Yingqi wrote: > Dear All, > > During the last couple weeks, I spent some time extending the original two > patches from prefork MPM only to all three Linux MPMs (prefork, worker and > event). Attached is the latest version of the two patches. Bugzilla database > has also been updated already. The ID for the two patches are #55897 and > #56279. Please refer to messages below for details on both of the patches. > > Quick test result on modern dual socket Intel platform (Linux Kernel 3.13.9) > SO_REUSEPORT patch (bugzilla #55897) > 1. Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 > listen statements: 2.33X throughput improvement > 2. Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen > statements: 35% throughput improvement > 3. Event MPM: 1 listen statement: 13% throughput improvement; 2 listen > statements: throughput parity, but 62% response time reduction (with patch, > 38% response time as original SW) > > Bucket patch (bugzilla #56279, only impact multiple listen statement case) > 1. Prefork MPM: 2 listen statements: 42% throughput improvement > 2. Worker MPM: 2 listen statements: 7% throughput improvement > > In all the above testing cases, significant response time reductions are > observed, even with throughput improvements. > > Please let me know your feedback and comments. > > Thanks, > Yingqi > Software and workloads used in performance tests may have been optimized for > performance only on Intel microprocessors. Performance tests, such as SYSmark > and MobileMark, are measured using specific computer systems, components, > software, operations and functions. Any change to any of those factors may > cause the results to vary. You should consult other information and > performance tests to assist you in fully evaluating your contemplated > purchases, including the performance of that product when combined with other > products. > > From: Lu, Yingqi [mailto:yingqi...@intel.com] > Sent: Monday, March 17, 2014 1:41 PM > To: dev@httpd.apache.org > Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > Dear all, > > Based on the feedback we received, we modified this patch. Here is the most > recent version. We also modified the Bugzilla database(Bugzilla# 55897 for > SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). > > Below are the changes we made into this new version: > > According to Yann Ylavic and other people’s comments, we separate the > original patch between with and without SO_REUSEPORT into two separated > patches. The SO_REUSEPORT patch does not change the original listen sockets, > it just duplicate the original one into multiple ones. Since the listen > sockets are identical, there is no need to change the idle_server_maintenance > function. The bucket patch (without SO_REUSEPORT), on the other hand, it > breaks down the original listen record (if there are multiple listen socks) > to multiple listen record linked lists. In this case, idle_server_maintenance > is implemented at bucket level to address the situation that imbalanced > traffic occurs among different listen sockets/children buckets. In the bucket > patch, the polling in the child process is removed since each child only > listens to 1 sock. > > According to Arkadiusz Miskiewicz’s comment, we make the “detection of > SO_REUSEPORT” at run time. > > According to Jeff Trawick’s comments, > 1. We generate the patches against the httpd trunk. > 2. We tested the current patches and they do not impact event and worker > mpms. If current patches can be accepted, we would be happy to extend them to > other Linux based mpms. There are not much code changes, but require some > time to setup the workload to test. > 3. We removed unnecessary comments and changed APLOGNO(). We also changed > some of the parameter/variable/function names to better represent their > meanings. > 4. There should be no build-in limitations for SO_REUSEPORT patch. For bucket > patch, the only thing is the number of children bucket only scales to > MAX_SPAWN_RATE. If there are more than 32 (current default MAX_SPQN_RATE) > listen statements specified in the httpd.conf, the number of buckets will be > fixed to 32. The reason for this is because that we implement the > idle_server_maintenance at bucket level, each bucket’s own max_spawn_rate is > set to MAX_SPAWN_RATE/num_buckets. >
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Thanks, Graham! I am looking forward to hearing your feedback. Thanks, Yingqi From: Graham Leggett [mailto:minf...@sharp.fm] Sent: Monday, April 07, 2014 12:08 PM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support On 07 Apr 2014, at 6:21 PM, "Lu, Yingqi" mailto:yingqi...@intel.com>> wrote: I just want to ping again on the modifications we made on both of the patches [bugzilla #55897 and bugzilla #56279]. Please let us know your comments and feedback. I am reattaching the patch files here in case you missed original email. I am very keen to review this, but have no time right now - sorry about that. From my side I am keen to review it soon. Regards, Graham --
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
On 07 Apr 2014, at 6:21 PM, "Lu, Yingqi" wrote: > I just want to ping again on the modifications we made on both of the patches > [bugzilla #55897 and bugzilla #56279]. Please let us know your comments and > feedback. > > I am reattaching the patch files here in case you missed original email. I am very keen to review this, but have no time right now - sorry about that. From my side I am keen to review it soon. Regards, Graham --
RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Dear all, I just want to ping again on the modifications we made on both of the patches [bugzilla #55897 and bugzilla #56279]. Please let us know your comments and feedback. Thanks, Yingqi From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Monday, March 24, 2014 1:56 PM To: dev@httpd.apache.org Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Dear all, I just want to ping on both of these two patches to see if there is anything we can do to help them get accepted. Your feedbacks and comments are very much appreciated. Thanks, Yingqi Lu From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Monday, March 17, 2014 1:41 PM To: dev@httpd.apache.org<mailto:dev@httpd.apache.org> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Dear all, Based on the feedback we received, we modified this patch. Here is the most recent version. We also modified the Bugzilla database(Bugzilla# 55897 for SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). Below are the changes we made into this new version: According to Yann Ylavic and other people's comments, we separate the original patch between with and without SO_REUSEPORT into two separated patches. The SO_REUSEPORT patch does not change the original listen sockets, it just duplicate the original one into multiple ones. Since the listen sockets are identical, there is no need to change the idle_server_maintenance function. The bucket patch (without SO_REUSEPORT), on the other hand, it breaks down the original listen record (if there are multiple listen socks) to multiple listen record linked lists. In this case, idle_server_maintenance is implemented at bucket level to address the situation that imbalanced traffic occurs among different listen sockets/children buckets. In the bucket patch, the polling in the child process is removed since each child only listens to 1 sock. According to Arkadiusz Miskiewicz's comment, we make the "detection of SO_REUSEPORT" at run time. According to Jeff Trawick's comments, 1. We generate the patches against the httpd trunk. 2. We tested the current patches and they do not impact event and worker mpms. If current patches can be accepted, we would be happy to extend them to other Linux based mpms. There are not much code changes, but require some time to setup the workload to test. 3. We removed unnecessary comments and changed APLOGNO(). We also changed some of the parameter/variable/function names to better represent their meanings. 4. There should be no build-in limitations for SO_REUSEPORT patch. For bucket patch, the only thing is the number of children bucket only scales to MAX_SPAWN_RATE. If there are more than 32 (current default MAX_SPQN_RATE) listen statements specified in the httpd.conf, the number of buckets will be fixed to 32. The reason for this is because that we implement the idle_server_maintenance at bucket level, each bucket's own max_spawn_rate is set to MAX_SPAWN_RATE/num_buckets. Again, thanks very much for all the comments and feedback. Please let us know if there are more changes we need to complete to make them accepted. Thanks, Yingqi Lu From: Lu, Yingqi Sent: Tuesday, March 04, 2014 10:43 AM To: dev@httpd.apache.org<mailto:dev@httpd.apache.org> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jeff, Thanks very much for your time reviewing the patch! We will modify the patch according to your comments and repost it here. Thanks, Yingqi From: Jeff Trawick [mailto:traw...@gmail.com] Sent: Tuesday, March 04, 2014 10:08 AM To: Apache HTTP Server Development List Subject: Re: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support On Tue, Mar 4, 2014 at 10:35 AM, Lu, Yingqi mailto:yingqi...@intel.com>> wrote: Hi All, I just want to ping again on this patch to gather your feedback and comments. Please refer to the messages below for patch details. If you need any additional information/supporting data, please let us know as well. Yeah, it has been on my todo list, but I don't have time to give an in depth review at the moment. Here are a few questions/comments. (And you'll have to deal with the fact that it is unnecessarily tedious for me to evaluate higher-level considerations if there are a lot of distractions, such as the code comments below ;) But others are of course free to chime in.) The patch should be against httpd trunk. It probably won't take much time for you to create that patch and confirm basic operation. What is the impact to other MPMs, even if they shouldn't use or don't have the necessary code to use SO_REUSEPORT at this time? Have you tried the event MPM? Is there a way for the admin to choose this behavior? Most won't care, but everyone's behavior is changed AFAICT. Are there built-in limitations in this
RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Dear all, I just want to ping on both of these two patches to see if there is anything we can do to help them get accepted. Your feedbacks and comments are very much appreciated. Thanks, Yingqi Lu From: Lu, Yingqi [mailto:yingqi...@intel.com] Sent: Monday, March 17, 2014 1:41 PM To: dev@httpd.apache.org Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Dear all, Based on the feedback we received, we modified this patch. Here is the most recent version. We also modified the Bugzilla database(Bugzilla# 55897 for SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch). Below are the changes we made into this new version: According to Yann Ylavic and other people's comments, we separate the original patch between with and without SO_REUSEPORT into two separated patches. The SO_REUSEPORT patch does not change the original listen sockets, it just duplicate the original one into multiple ones. Since the listen sockets are identical, there is no need to change the idle_server_maintenance function. The bucket patch (without SO_REUSEPORT), on the other hand, it breaks down the original listen record (if there are multiple listen socks) to multiple listen record linked lists. In this case, idle_server_maintenance is implemented at bucket level to address the situation that imbalanced traffic occurs among different listen sockets/children buckets. In the bucket patch, the polling in the child process is removed since each child only listens to 1 sock. According to Arkadiusz Miskiewicz's comment, we make the "detection of SO_REUSEPORT" at run time. According to Jeff Trawick's comments, 1. We generate the patches against the httpd trunk. 2. We tested the current patches and they do not impact event and worker mpms. If current patches can be accepted, we would be happy to extend them to other Linux based mpms. There are not much code changes, but require some time to setup the workload to test. 3. We removed unnecessary comments and changed APLOGNO(). We also changed some of the parameter/variable/function names to better represent their meanings. 4. There should be no build-in limitations for SO_REUSEPORT patch. For bucket patch, the only thing is the number of children bucket only scales to MAX_SPAWN_RATE. If there are more than 32 (current default MAX_SPQN_RATE) listen statements specified in the httpd.conf, the number of buckets will be fixed to 32. The reason for this is because that we implement the idle_server_maintenance at bucket level, each bucket's own max_spawn_rate is set to MAX_SPAWN_RATE/num_buckets. Again, thanks very much for all the comments and feedback. Please let us know if there are more changes we need to complete to make them accepted. Thanks, Yingqi Lu From: Lu, Yingqi Sent: Tuesday, March 04, 2014 10:43 AM To: dev@httpd.apache.org<mailto:dev@httpd.apache.org> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Jeff, Thanks very much for your time reviewing the patch! We will modify the patch according to your comments and repost it here. Thanks, Yingqi From: Jeff Trawick [mailto:traw...@gmail.com] Sent: Tuesday, March 04, 2014 10:08 AM To: Apache HTTP Server Development List Subject: Re: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support On Tue, Mar 4, 2014 at 10:35 AM, Lu, Yingqi mailto:yingqi...@intel.com>> wrote: Hi All, I just want to ping again on this patch to gather your feedback and comments. Please refer to the messages below for patch details. If you need any additional information/supporting data, please let us know as well. Yeah, it has been on my todo list, but I don't have time to give an in depth review at the moment. Here are a few questions/comments. (And you'll have to deal with the fact that it is unnecessarily tedious for me to evaluate higher-level considerations if there are a lot of distractions, such as the code comments below ;) But others are of course free to chime in.) The patch should be against httpd trunk. It probably won't take much time for you to create that patch and confirm basic operation. What is the impact to other MPMs, even if they shouldn't use or don't have the necessary code to use SO_REUSEPORT at this time? Have you tried the event MPM? Is there a way for the admin to choose this behavior? Most won't care, but everyone's behavior is changed AFAICT. Are there built-in limitations in this patch that we should be aware of? E.g., the free slot/spawn rate changes suggest to me that there can't be more than 1025 children??? We should assume for now that there's no reason this couldn't be committed to trunk after review/rework, so make sure it is as close as you can get it to what you think is the final form. For the configure-time check for 3.9 kernel: I think we'd also use AC_TRY_CO
RE: [PATCH ASF bugzilla# 55897] prefork_mpm patch with SO_REUSEPORT support
Hi Tim, Thanks for your email. SO_REUSEPORT feature is enabled on Linux kernel 3.9 and newer. The feature is defined at /usr/include/asm-generic/socket.h. With the old kernel, the definition is there, but is commented out. /*#define SO_REUSEPORT 15*/ The section of code below is just to define SO_REUSEPORT if it is not already being defined. The code after this is to detect if SO_REUSEPORT is supported or not. I am using x86_64 systems with Linux. If anyone finds something different on your system, please let me know. Thanks, Yingqi -Original Message- From: Tim Bannister [mailto:is...@jellybaby.net] Sent: Monday, March 17, 2014 2:31 PM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897] prefork_mpm patch with SO_REUSEPORT support I'm afraid I don't understand this particular part from httpd_trunk_so_reuseport.patch: #ifndef SO_REUSEPORT #define SO_REUSEPORT 15 #endif Why 15? Is this going to be portable across different platforms? -- Tim Bannister - is...@jellybaby.net
Re: [PATCH ASF bugzilla# 55897] prefork_mpm patch with SO_REUSEPORT support
I'm afraid I don't understand this particular part from httpd_trunk_so_reuseport.patch: #ifndef SO_REUSEPORT #define SO_REUSEPORT 15 #endif Why 15? Is this going to be portable across different platforms? -- Tim Bannister – is...@jellybaby.net
RE: SO_REUSEPORT in the children processes
Hi Yann, As we pointed out in our original discussion thread, we dropped the child process implementation due to the kernel defect with changing the number of open sockets. Now, we quickly tested this child process implementation (prefork) with our workload on a modern Xeon dual sockets server and most recent 3.13.6 kernel again. 1. We do not see "connection reset" errors during the run (ramp up and steady stay) any more. However, we noticed that our workload cannot ramp down and terminate on its own with this child process implementation. This never happened before with either "out of box" httpd or the parent process implementation. After manually force shutdown the workload, we saw these "connection reset" errors again. 2. During the run, we noticed that there are tons of "read timed out" errors. These errors not only happen when the system is highly utilized, it even happens when system is only 10% utilized. The response time was high. 3. Compared to parent process implementation, we found child process implementation results in significantly higher (up to 10X) response time (The read timed out errors are not counted in the result) at different CPU utilization levels. At peak performance level, it has ~22% less throughput with tons of "connection reset" errors in additional to "read timed out" errors. Parent process implementation does not have errors. We think the reason of above findings may be caused by: 1. Too many open sockets created by the children processes; and/or 2. Parent process does not have control, or maybe 3. Kernel defect is not fully addressed. On the other hand, the parent implementation keeps minimal number of open sockets that takes advantage of SO_REUSEPORT and keeps the environment more controllable. We are currently modifying the code based on all the feedbacks from the community with the original parent process implementation which also includes separating the original patch between with and without SO_REUSEPORT support. This would make SO_REUSEPORT patch cleaner and simpler. Thanks, Yingqi (people at work also call me Lucy:)) From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Friday, March 07, 2014 9:07 AM To: httpd Subject: SO_REUSEPORT in the children processes Hi all, the patch about SO_REUSEPORT proposed by Yingqi Lu in [1] and discussed in [2] uses listeners buckets to address a defect [3] in the current linux implementation (his patch goes beyond SO_REUSEPORT though, and suggests a new MPM even when the option is not available). Should this defect be addressed by linux folks, the event/worker/prefork MPMs could take full advantage of the option (linux-3.9+) with quite simple modifications of the current code. I'm proposing here the corresponding patch. The idea is to re-create and re-bind/listen the parent's listeners sockets for each child process, when starting, before dropping priviledges. For this, the patch introduces a new ap_reopen_listeners() function (meant to be called by each child) to do the job on the inherited listeners. It does nothing unless HAVE_SO_REUSEPORT is defined. The advantage of this approach is that no accept mutex is needed anymore (each child has its own sockets), hence the SAFE_ACCEPT macros can do nothing when HAVE_SO_REUSEPORT is defined. The new (incoming) connections are evenly distributed accross the children for all the listners (as assured by Linux when SO_REUSEPORT is used). I'm proposing the patch here so that everyone can figure out whether SO_REUSEPORT per se needs its own MPM or not (once/if the defect is fixed). The option seems to be quite easily pluggable to existing MPMs (no ABI change), and I don't see an advantage to not using it when available (and working). Also, FreeBSD has an implementation of SO_REUSEPORT, however I couldn't find whether it has the same scheduling garantee or not (at least I guess the accept mutex can be avoided too). Regarding the linux kernel defect, is someone aware of a fix/work on that in the latest versions? Finally, about the accept mutex, mpm event seems to work well without any, why prefork and worker would need one (both poll() all the listeners in a single thread, while other children can do the same)? The patch follows and is attached. It can already be tested with a workaround against the defect: don't let perform_idle_server_maintenance() create/stop children after startup (eg. StartServers, ServerLimit, Min/MaxSpareServers using the same value). Thoughts, feedbacks welcome. Regards, Yann. [1] https://issues.apache.org/bugzilla/show_bug.cgi?id=55897#c7 [2] http://mail-archives.apache.org/mod_mbox/httpd-bugs/201312.mbox/%3cbug-55897-7...@https.issues.apache.org/bugzilla/%3E [3] http://lwn.net/Articles/542629/ and http://lwn.net/Articles/542738/ Index: server/mpm/event/event.c =
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
the next one with his vendetta i just *asked* consider post plain text nothing more Am 10.03.2014 00:32, schrieb Nick Edwards: > Truer words were never spoken about Harald Reindl, this person brings > trouble to every mailing list he joins > > postfix - banned read the history > fedora - moderation no longer true for good reasons > centos - moderation/banned because *one* posting years ago > roundcube - moderation caused by Noel Butler and nobody else > dovecot - final warnings in a thread flamed by Noel Butler and youself interesing that i am still there, Noel not and he had his vendetta against Timo long ago > I do not find his hostname in bl.alt-backspace.org because it was removed months ago after even Noel Butler realized that it is the wrong way to blacklist whole networks because personal problems with a single person and if people like Noel and you just shut up many problems you list abvoe simply would not exist __ frankly who do you and Noel think that you are to rule the world? did i do anyhting bad on this list? if yes show me if no just shut up with your persnal justice trying to rule the world there is not much left to say than that reject message below : host mail.ausics.net[27.33.160.23] said: 554 5.7.1 Sender address rejected: faggots not allowed signature.asc Description: OpenPGP digital signature
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
Truer words were never spoken about Harald Reindl, this person brings trouble to every mailing list he joins postfix - banned fedora - moderation centos - moderation/banned roundcube - moderation dovecot - final warnings and they are just the lists I know of, and when moderated he is known to send direct abuse and threats to people (this is why on roundcube he will never be post status again) removing the content of such deserved vulgar (about time somebody gave him a dose of his own medicine for once) I do not find his hostname in bl.alt-backspace.org so, if thats a fabrication, well, I'll leave it to your imaginations. I suggest this list moderates him as well else this is just the start of things to come going by his very poor track record. Most people (myself included) have him filtered out anyway- now he's found his way onto this list, I no longer whitelist this list.
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
On Sun, Mar 9, 2014 at 2:01 PM, Reindl Harald wrote: > > Am 08.03.2014 01:38, schrieb Noel Butler: >> This will be dealt with off list > > with the words below which are only a part of the off-list reply It should be kept off-list. Just stop.
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
Am 08.03.2014 01:38, schrieb Noel Butler: > This will be dealt with off list with the words below which are only a part of the off-list reply and this person is allowed to maintain a RBL in australia and so anybody using "bl.alt-backspace.org" should re-consider that Noel Butler: >> listen you cock sucking little fucking faggot cunt, fuclk off with your >> fucking >> hollier than though fucking shit ok cuntface you are a nobody on any list >> depsite >> the fact you think you are, you have no fucknig right to ask anyone to type >> the >> way you want ok cunt eyes, just fuck off and spew your fucking shit is a psychopathic reaction to the the simple question post in plaintext and was the direct repsone to that off-list reply to Noel > On 08/03/2014 10:49, Reindl Harald wrote: >>> if you fail to heed that warning your the only one who will regret it >> stop your personal vendetta - the only one playing internet cop is you >> >> i have asked in a nice way to not post HTML and explained why >> the other person had no problem with my question / hin > Original-Nachricht > Betreff: Re: SO_REUSEPORT in the children processes > Datum: Fri, 07 Mar 2014 18:24:26 +0100 > Von: Reindl Harald > Antwort an: dev@httpd.apache.org > An: dev@httpd.apache.org > > Am 07.03.2014 18:07, schrieb Yann Ylavic > > can you please post plaintext instead HTML to lists > > for me such messages are unreadable after medical operations > on both eyes because you override my MUA font settings > On 08/03/2014 09:51, Reindl Harald wrote: > >> what exactly is your personal problem? >>>> can you please post plaintext instead HTML to lists >> you did see the word "please"? >>>> for me such messages are unreadable after medical operations on both eyes >>>> because you override my MUA font settings >> you understood that reason? >> >> i follow that thread longer honestly interested >> there where a few HTML respones and *all* oft hem left unread >> because they contained a lot of content >>>> he has been warned about playing net cop >> who do you think you are calling others net cop because they ask for >> easier readable posts while you are the one proven to playing net cop >> due maintaining a RBL? >>>> Is it written anywhere in the bylaws of this mailing list >> is the response you should have been attacking >> >> Am 08.03.2014 00:39, schrieb Noel Butler: >>> Harry, you have been warned before, dont bring your antics onto this list, >>> this is about the only list you have >>> been most well behaved on, unlike others, please remember our previous >>> conversations. If you think a posters >>> post violates some RFC, ignore it, or take it up with him in private, do >>> not pollute this list with your >>> bullshit. Thank you :) On Fri, 2014-03-07 at 19:13 +0100, Reindl Harald >>> wrote: >>>> Am 07.03.2014 18:58, schrieb Mikhail T.: >>>>> On 07.03.2014 12:28, Yann Ylavic wrote: >>>>>> Sorry, this was posted from gmail... >>>>> Is it written anywhere in the bylaws of this mailing list that use of >>>>> HTML is something to apologize for? >>>> nearly any mailing-list has it written clear, some even reject HTML and on >>>> some others you get warned by the >>>> owner (postfix as example) >>>> https://www.google.com/search?q=mailing+list+etiquette+plaintext+only >>>> https://www.google.at/search?q=mailing+list+etiquette+no+html >>>> _ >>>> http://www.apache.org/foundation/mailinglists.htmlNote: Spam filters don't >>>> like messages with empty Subjects; >>>> just use e.g. "subscribe" or "unsubscribe". Spam filters are also more >>>> likely to reject HTML-formatted >>>> messages; please use plain text >>>>> With all due sympathies to Reindl's medical condition, why must we -- in >>>>> the second decade of the 21st century >>>>> -- deny ourselves the means of expression afforded by HTML on this list? >>>> because the message had no single HTML formatting and was written in plain >>>> but as HTML may be a godd reason it >>>> hardly makes sense to write email in HTML, they just explode after some >>>> replies/quotes and forwardings with >>>> different clients while a conversation in plaintext survises thousands of >>>> replies without lose a sane quoting signature.asc Description: OpenPGP digital signature
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
This will be dealt with off list On 08/03/2014 09:51, Reindl Harald wrote: > what exactly is your personal problem? > can you please post plaintext instead HTML to lists you did see the word "please"? >> for me such messages are unreadable after medical operations on both eyes >> because you override my MUA font settings you understood that reason? i follow that thread longer honestly interested there where a few HTML respones and *all* oft hem left unread because they contained a lot of content >> he has been warned about playing net cop who do you think you are calling others net cop because they ask for easier readable posts while you are the one proven to playing net cop due maintaining a RBL? >> Is it written anywhere in the bylaws of this mailing list is the response you should have been attacking Am 08.03.2014 00:39, schrieb Noel Butler: > Harry, you have been warned before, dont bring your antics onto this list, > this is about the only list you have been most well behaved on, unlike > others, please remember our previous conversations. If you think a posters > post violates some RFC, ignore it, or take it up with him in private, do not > pollute this list with your bullshit. Thank you :) On Fri, 2014-03-07 at > 19:13 +0100, Reindl Harald wrote: Am 07.03.2014 18:58, schrieb Mikhail T.: On > 07.03.2014 12:28, Yann Ylavic wrote: Sorry, this was posted from gmail... Is > it written anywhere in the bylaws of this mailing list that use of HTML is > something to apologize for? nearly any mailing-list has it written clear, some even reject HTML and on some others you get warned by the owner (postfix as example) https://www.google.com/search?q=mailing+list+etiquette+plaintext+only [1] https://www.google.at/search?q=mailing+list+etiquette+no+html [2] _ http://www.apache.org/foundation/mailinglists.html [3]Note: Spam filters don't like messages with empty Subjects; just use e.g. "subscribe" or "unsubscribe". Spam filters are also more likely to reject HTML-formatted messages; please use plain text > With all due sympathies to Reindl's medical condition, why must we -- in the > second decade of the 21st century -- deny ourselves the means of expression > afforded by HTML on this list? because the message had no single HTML formatting and was written in plain but as HTML may be a godd reason it hardly makes sense to write email in HTML, they just explode after some replies/quotes and forwardings with different clients while a conversation in plaintext survises thousands of replies without lose a sane quoting Links: -- [1] https://www.google.com/search?q=mailing+list+etiquette+plaintext+only [2] https://www.google.at/search?q=mailing+list+etiquette+no+html [3] http://www.apache.org/foundation/mailinglists.html
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
what exactly is your personal problem? >> can you please post plaintext instead HTML to lists you did see the word "please"? >> for me such messages are unreadable after medical operations >> on both eyes because you override my MUA font settings you understood that reason? i follow that thread longer honestly interested there where a few HTML respones and *all* oft hem left unread because they contained a lot of content >> he has been warned about playing net cop who do you think you are calling others net cop because they ask for easier readable posts while you are the one proven to playing net cop due maintaining a RBL? >> Is it written anywhere in the bylaws of this mailing list is the response you should have been attacking Am 08.03.2014 00:39, schrieb Noel Butler: > Harry, you have been warned before, dont bring your antics onto this list, > this is about the only list you have > been most well behaved on, unlike others, please remember our previous > conversations. If you think a posters post > violates some RFC, ignore it, or take it up with him in private, do not > pollute this list with your bullshit. > > Thank you :) > > On Fri, 2014-03-07 at 19:13 +0100, Reindl Harald wrote: >> Am 07.03.2014 18:58, schrieb Mikhail T.: >> > On 07.03.2014 12:28, Yann Ylavic wrote: >> >> Sorry, this was posted from gmail... >> > Is it written anywhere in the bylaws of this mailing list >> > that use of HTML is something to apologize for? >> >> nearly any mailing-list has it written clear, some even reject HTML >> and on some others you get warned by the owner (postfix as example) >> >> https://www.google.com/search?q=mailing+list+etiquette+plaintext+only >> https://www.google.at/search?q=mailing+list+etiquette+no+html >> _ >> >> http://www.apache.org/foundation/mailinglists.html >> >> Note: Spam filters don't like messages with empty Subjects; just use e.g. >> "subscribe" or "unsubscribe". Spam filters are also more likely to reject >> HTML-formatted messages; please use plain text >> >> > With all due sympathies to Reindl's medical condition, why must we -- >> > in the second decade of the 21st century -- deny ourselves the means >> > of expression afforded by HTML on this list? >> >> because the message had no single HTML formatting and >> was written in plain but as HTML may be a godd reason >> >> it hardly makes sense to write email in HTML, they just >> explode after some replies/quotes and forwardings with >> different clients while a conversation in plaintext >> survises thousands of replies without lose a sane quoting signature.asc Description: OpenPGP digital signature
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
Harry, you have been warned before, dont bring your antics onto this list, this is about the only list you have been most well behaved on, unlike others, please remember our previous conversations. If you think a posters post violates some RFC, ignore it, or take it up with him in private, do not pollute this list with your bullshit. Thank you :) On Fri, 2014-03-07 at 19:13 +0100, Reindl Harald wrote: > Am 07.03.2014 18:58, schrieb Mikhail T.: > > On 07.03.2014 12:28, Yann Ylavic wrote: > >> Sorry, this was posted from gmail... > > Is it written anywhere in the bylaws of this mailing list > > that use of HTML is something to apologize for? > > nearly any mailing-list has it written clear, some even reject HTML > and on some others you get warned by the owner (postfix as example) > > https://www.google.com/search?q=mailing+list+etiquette+plaintext+only > https://www.google.at/search?q=mailing+list+etiquette+no+html > _ > > http://www.apache.org/foundation/mailinglists.html > > Note: Spam filters don't like messages with empty Subjects; just use e.g. > "subscribe" or "unsubscribe". Spam filters are also more likely to reject > HTML-formatted messages; please use plain text > > > With all due sympathies to Reindl's medical condition, why must we -- > > in the second decade of the 21st century -- deny ourselves the means > > of expression afforded by HTML on this list? > > because the message had no single HTML formatting and > was written in plain but as HTML may be a godd reason > > it hardly makes sense to write email in HTML, they just > explode after some replies/quotes and forwardings with > different clients while a conversation in plaintext > survises thousands of replies without lose a sane quoting > <> signature.asc Description: This is a digitally signed message part
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
On Fri, 2014-03-07 at 12:58 -0500, Mikhail T. wrote: > On 07.03.2014 12:28, Yann Ylavic wrote: > > > > > Sorry, this was posted from gmail... > > Is it written anywhere in the bylaws of this mailing list, that use of > HTML is something to apologize for? With all due sympathies to > Reindl's medical Absolutely no apology required Harald is just a system admin subscribed to this http developer list, he has no association with Apache Software Foundation, he has been warned about playing net cop where he has no authority to do so before, ignore him. signature.asc Description: This is a digitally signed message part
Re: Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
Am 07.03.2014 18:58, schrieb Mikhail T.: > On 07.03.2014 12:28, Yann Ylavic wrote: >> Sorry, this was posted from gmail... > Is it written anywhere in the bylaws of this mailing list > that use of HTML is something to apologize for? nearly any mailing-list has it written clear, some even reject HTML and on some others you get warned by the owner (postfix as example) https://www.google.com/search?q=mailing+list+etiquette+plaintext+only https://www.google.at/search?q=mailing+list+etiquette+no+html _ http://www.apache.org/foundation/mailinglists.html Note: Spam filters don't like messages with empty Subjects; just use e.g. "subscribe" or "unsubscribe". Spam filters are also more likely to reject HTML-formatted messages; please use plain text > With all due sympathies to Reindl's medical condition, why must we -- > in the second decade of the 21st century -- deny ourselves the means > of expression afforded by HTML on this list? because the message had no single HTML formatting and was written in plain but as HTML may be a godd reason it hardly makes sense to write email in HTML, they just explode after some replies/quotes and forwardings with different clients while a conversation in plaintext survises thousands of replies without lose a sane quoting signature.asc Description: OpenPGP digital signature
Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)
On 07.03.2014 12:28, Yann Ylavic wrote: > Sorry, this was posted from gmail... Is it written anywhere in the bylaws of this mailing list, that use of HTML is something to apologize for? With all due sympathies to Reindl's medical condition, why must we -- in the second decade of the 21st century -- deny ourselves the means of expression afforded by HTML on this list? He can -- ought to be able to -- adjust his MUA to use his font-preferences over whatever may be specified in the incoming email. I'd ask this question on /any/ mailing list, but it seems especially ironic among developers of Apache httpd -- software meant to distribute content, that is most often written in HTML... Granted, some HTML can be offensive even to the healthy eyes -- and ought to be viewed as impolite (not that Yann's was anything of the kind). But to request, as Reindl did, that /all/ postings -- from all participants -- be in plain text, seems rather overbearing... > Here is the plain text. Wouldn't it be better to send such a duplication to Reindl /personally/? To avoid this becoming a list-wide flame war, I ask for the powers running this list to convene and hand us the law: either HTML is acceptable (my own preference), or it is not (in which case it should be bounced back by the mailing list software with an appropriate message). -mi
Re: SO_REUSEPORT in the children processes
On Fri, Mar 7, 2014 at 6:35 PM, Eric Covener wrote: >> Sorry, this was posted from gmail... > > FWIW I did not really see the distinctive HTML look and feel reading > it on gmail. I have none... and won't uncheck the "Plain text mode" anymore. Otherwise it's almost impossible to copy/paste without erratic result. It looks and feels a bit like it wants :p
Re: SO_REUSEPORT in the children processes
> Sorry, this was posted from gmail... FWIW I did not really see the distinctive HTML look and feel reading it on gmail.
Re: SO_REUSEPORT in the children processes
On Fri, Mar 7, 2014 at 6:24 PM, Reindl Harald wrote:
>
>
> Am 07.03.2014 18:07, schrieb Yann Ylavic
>
> can you please post plaintext instead HTML to lists
>
> for me such messages are unreadable after medical operations
> on both eyes because you override my MUA font settings
>
Sorry, this was posted from gmail...
Here is the plain text.
***
Hi all,
the patch about SO_REUSEPORT proposed by Yingqi Lu in [1] and
discussed in [2] uses listeners buckets to address a defect [3] in the
current linux implementation (his patch goes beyond SO_REUSEPORT
though, and suggests a new MPM even when the option is not available).
Should this defect be addressed by linux folks, the
event/worker/prefork MPMs could take full advantage of the option
(linux-3.9+) with quite simple modifications of the current code.
I'm proposing here the corresponding patch.
The idea is to re-create and re-bind/listen the parent's listeners
sockets for each child process, when starting, before dropping
priviledges.
For this, the patch introduces a new ap_reopen_listeners() function
(meant to be called by each child) to do the job on the inherited
listeners. It does nothing unless HAVE_SO_REUSEPORT is defined.
The advantage of this approach is that no accept mutex is needed
anymore (each child has its own sockets), hence the SAFE_ACCEPT macros
can do nothing when HAVE_SO_REUSEPORT is defined.
The new (incoming) connections are evenly distributed accross the
children for all the listners (as assured by Linux when SO_REUSEPORT
is used).
I'm proposing the patch here so that everyone can figure out whether
SO_REUSEPORT per se needs its own MPM or not (once/if the defect is
fixed).
The option seems to be quite easily pluggable to existing MPMs (no ABI
change), and I don't see an advantage to not using it when available
(and working).
Also, FreeBSD has an implementation of SO_REUSEPORT,
however
I couldn't find whether it has the same scheduling garantee
or not
(at least I guess the accept mutex can be avoided too).
Regarding the linux kernel defect, is someone aware of a fix/work on
that in the latest versions?
Finally, about the accept mutex, mpm event seems to work well without
any, why prefork and worker would need one (both poll() all the
listeners in a single thread, while other children can do the same)?
The patch follows and is attached.
It can already be tested with a workaround against the defect: don't
let perform_idle_server_maintenance() create/stop children after
startup (eg. StartServers, ServerLimit, Min/MaxSpareServers using the
same value).
Thoughts, feedbacks welcome.
Regards,
Yann.
[1] https://issues.apache.org/bugzilla/show_bug.cgi?id=55897#c7
[2]
http://mail-archives.apache.org/mod_mbox/httpd-bugs/201312.mbox/%3cbug-55897-7...@https.issues.apache.org/bugzilla/%3E
[3] http://lwn.net/Articles/542629/ and http://lwn.net/Articles/542738/
Index: server/mpm/event/event.c
===
--- server/mpm/event/event.c(revision 1575322)
+++ server/mpm/event/event.c(working copy)
@@ -2356,6 +2356,13 @@ static void child_main(int child_num_arg)
/*stuff to do before we switch id's, so we have permissions. */
ap_reopen_scoreboard(pchild, NULL, 0);
+rv = ap_reopen_listeners(pchild, num_listensocks);
+if (rv != APR_SUCCESS) {
+ap_log_error(APLOG_MARK, APLOG_EMERG, rv, ap_server_conf, APLOGNO()
+ "Couldn't re-open listeners");
+clean_child_exit(APEXIT_CHILDFATAL);
+}
+
if (ap_run_drop_privileges(pchild, ap_server_conf)) {
clean_child_exit(APEXIT_CHILDFATAL);
}
Index: server/mpm/prefork/prefork.c
===
--- server/mpm/prefork/prefork.c(revision 1575322)
+++ server/mpm/prefork/prefork.c(working copy)
@@ -271,7 +271,9 @@ static void accept_mutex_off(void)
* multiple Listen statements. Define SINGLE_LISTEN_UNSERIALIZED_ACCEPT
* when it's safe in the single Listen case.
*/
-#ifdef SINGLE_LISTEN_UNSERIALIZED_ACCEPT
+#if HAVE_SO_REUSEPORT
+#define SAFE_ACCEPT(stmt)
+#elif defined(SINGLE_LISTEN_UNSERIALIZED_ACCEPT)
#define SAFE_ACCEPT(stmt) do {if (ap_listeners->next) {stmt;}} while(0)
#else
#define SAFE_ACCEPT(stmt) do {stmt;} while(0)
@@ -536,6 +538,13 @@ static void child_main(int child_num_arg)
clean_child_exit(APEXIT_CHILDFATAL);
}
+status = ap_reopen_listeners(pchild, num_listensocks);
+if (status != APR_SUCCESS) {
+ap_log_error(APLOG_MARK, APLOG_EMERG, status, ap_server_conf, APLOGNO()
+ "Couldn't re-open listeners");
+clean_child_exit(APEXIT_CHILDFATAL);
+}
+
if (ap_run_drop_privileges(pchild, ap_server_conf)) {
clean_child_exit(APEXIT_CHILDFATAL);
}
Index: server/mpm/worker/worker.c
===
Re: SO_REUSEPORT in the children processes
Am 07.03.2014 18:07, schrieb Yann Ylavic can you please post plaintext instead HTML to lists for me such messages are unreadable after medical operations on both eyes because you override my MUA font settings signature.asc Description: OpenPGP digital signature
SO_REUSEPORT in the children processes
Hi all,
the patch about SO_REUSEPORT proposed by Yingqi Lu in [1] and discussed in
[2] uses listeners buckets to address a defect [3] in the current linux
implementation (his patch goes beyond SO_REUSEPORT though, and suggests a
new MPM even when the option is not available).
Should this defect be addressed by linux folks, the event/worker/prefork
MPMs could take full advantage of the option (linux-3.9+) with quite simple
modifications of the current code.
I'm proposing here the corresponding patch.
The idea is to re-create and re-bind/listen the parent's listeners sockets
for each child process, when starting, before dropping priviledges.
For this, the patch introduces a new ap_reopen_listeners() function (meant
to be called by each child) to do the job on the inherited listeners. It
does nothing unless HAVE_SO_REUSEPORT is defined.
The advantage of this approach is that no accept mutex is needed anymore
(each child has its own sockets), hence the SAFE_ACCEPT macros can do
nothing when HAVE_SO_REUSEPORT is defined.
The new (incoming) connections are evenly distributed accross the children
for all the listners (as assured by Linux when SO_REUSEPORT is used).
I'm proposing the patch here so that everyone can figure out whether
SO_REUSEPORT per se needs its own MPM or not (once/if the defect is fixed).
The option seems to be quite easily pluggable to existing MPMs (no ABI
change), and I don't see an advantage to not using it when available (and
working).
Also, FreeBSD has an implementation of SO_REUSEPORT,
however
I couldn't find whether it has the same scheduling garantee
or not
(at least I guess the accept mutex can be avoided too).
Regarding the linux kernel defect, is someone aware of a fix/work on that
in the latest versions?
Finally, about the accept mutex, mpm event seems to work well without any,
why prefork and worker would need one (both poll() all the listeners in a
single thread, while other children can do the same)?
The patch follows and is attached.
It can already be tested with a workaround against the defect: don't let
perform_idle_server_maintenance() create/stop children after startup (eg.
StartServers, ServerLimit, Min/MaxSpareServers using the same value).
Thoughts, feedbacks welcome.
Regards,
Yann.
[1] https://issues.apache.org/bugzilla/show_bug.cgi?id=55897#c7
[2]
http://mail-archives.apache.org/mod_mbox/httpd-bugs/201312.mbox/%3cbug-55897-7...@https.issues.apache.org/bugzilla/%3E
[3] http://lwn.net/Articles/542629/ and http://lwn.net/Articles/542738/
Index: server/mpm/event/event.c
===
--- server/mpm/event/event.c(revision 1575322)
+++ server/mpm/event/event.c(working copy)
@@ -2356,6 +2356,13 @@ static void child_main(int child_num_arg)
/*stuff to do before we switch id's, so we have permissions. */
ap_reopen_scoreboard(pchild, NULL, 0);
+rv = ap_reopen_listeners(pchild, num_listensocks);
+if (rv != APR_SUCCESS) {
+ap_log_error(APLOG_MARK, APLOG_EMERG, rv, ap_server_conf, APLOGNO()
+ "Couldn't re-open listeners");
+clean_child_exit(APEXIT_CHILDFATAL);
+}
+
if (ap_run_drop_privileges(pchild, ap_server_conf)) {
clean_child_exit(APEXIT_CHILDFATAL);
}
Index: server/mpm/prefork/prefork.c
===
--- server/mpm/prefork/prefork.c(revision 1575322)
+++ server/mpm/prefork/prefork.c(working copy)
@@ -271,7 +271,9 @@ static void accept_mutex_off(void)
* multiple Listen statements. Define SINGLE_LISTEN_UNSERIALIZED_ACCEPT
* when it's safe in the single Listen case.
*/
-#ifdef SINGLE_LISTEN_UNSERIALIZED_ACCEPT
+#if HAVE_SO_REUSEPORT
+#define SAFE_ACCEPT(stmt)
+#elif defined(SINGLE_LISTEN_UNSERIALIZED_ACCEPT)
#define SAFE_ACCEPT(stmt) do {if (ap_listeners->next) {stmt;}} while(0)
#else
#define SAFE_ACCEPT(stmt) do {stmt;} while(0)
@@ -536,6 +538,13 @@ static void child_main(int child_num_arg)
clean_child_exit(APEXIT_CHILDFATAL);
}
+status = ap_reopen_listeners(pchild, num_listensocks);
+if (status != APR_SUCCESS) {
+ap_log_error(APLOG_MARK, APLOG_EMERG, status, ap_server_conf,
APLOGNO()
+ "Couldn't re-open listeners");
+clean_child_exit(APEXIT_CHILDFATAL);
+}
+
if (ap_run_drop_privileges(pchild, ap_server_conf)) {
clean_child_exit(APEXIT_CHILDFATAL);
}
Index: server/mpm/worker/worker.c
===
--- server/mpm/worker/worker.c(revision 1575322)
+++ server/mpm/worker/worker.c(working copy)
@@ -220,7 +220,9 @@ static apr_os_thread_t *listener_os_thread;
/* Locks for accept serialization */
static apr_proc_mutex_t *accept_mutex;
-#ifdef SINGLE_LISTEN_UNSERIALIZED_ACCEPT
+#if HAVE_SO_REUSEPORT
+#define SAFE_ACCEPT(stmt)
+#elif defined(SI
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Yann,
Yes, without SO_REUSEPORT, child only accepts connections from a single
listening socket only. In order to address the situation of in-balanced traffic
among different sockets/listen statements, the patch makes each bucket does its
own idler server maintenance. For example, if we have two listen statements
defined, one is very busy and the other is almost idle. The patch creates two
buckets, each listens to 1 IP:port. The busy bucket would end up with lots of
children and idle bucket would only maintain minimum number of children which
is equal to 1/2 of the min idle servers defined in the httpd.conf file.
Thanks,
Yingqi
From: Yann Ylavic [mailto:ylavic@gmail.com]
Sent: Thursday, March 06, 2014 5:49 AM
To: httpd
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT
support
On Wed, Mar 5, 2014 at 6:38 PM, Lu, Yingqi
mailto:yingqi...@intel.com>> wrote:
1. If I understand correctly (please correct me if not), do you suggest
duplicating the listen socks inside the child process with SO_REUSEPROT
enabled? Yes, I agree this would be a cleaner implementation and I actually
tried that before. However, I encountered the "connection reset" error since
the number of the child process is changing. I googled online and found it
actually being discussed here at http://lwn.net/Articles/542629/.
Actually I found that article, but expected the "defect" was solved since
then...
This looks like a thorn in the side of MPMs in general,
but couldn't find any pointer to a fix, do you know if there is some progress
on this in the latest linux kernel?
For testing purpose (until then?), you could also configure MPM prefork to not
create/terminate children processes once started (using the same value for
StartServers and ServerLimit, still MaxRequetsPerChild 0).
It could be interesting to see how SO_REUSEPORT scales in these optimal
conditions (no lock, full OS round-robin on all listeners).
For this you would have to use your former patch (duplicating listeners in each
child process), and do nothing in SAFE_ACCEPT when HAVE_SO_REUSEPORT.
Also, SO_REUSEPORT exists on (and even comes from) FreeBSD if I am not
mistaken, but it seems that there is no round-robin garantee for it, is there?
Could this patch also take advantage of BSD's SO_REUSEPORT implementation?
2. Then, I decided to do the socket duplication in the parent process. The goal
of this change is to extend the CPU thread scalability with the big thread
count system. Therefore, I just very simply defined
number_of_listen_buckets=total_number_active_thread/8, and each listen bucket
has a dedicated listener. I do not want to over duplicate the socket;
otherwise, it would create too many child processes at the beginning. One
listen bucket should have at least one child process to start with. However,
this is only my understanding and it may not be correct and complete. If you
have other ideas, please share with us. Feedbacks and comments are very welcome
here :)
The listeners buckets make sense with SO_REUSEPORT given the defect, I hope
this is temporary.
3. I am struggling with myself as well on if we should put with and without
SO_REUSEPORT into two different patches. The only reason I put them together is
because they both use the concept of listen buckets. If you think it would make
more sense to separate them into two patches, I can certainly do that. Also, I
am a little bit confused about your comments "On the other hand, each child is
dedicated, won't one have to multiply the configured ServerLimit by the number
of Listen to achieve the same (maximum theoretical) scalability with regard to
all the listeners?". Can you please explain a little bit more on this? Really
appreciate.
Sorry to have not been clear enough (nay at all).
I'm referring to the following code.
In prefork.c::make_child(), each child is assigned a listener like this (before
fork()ing) :
child_listen = mpm_listen[bucket[slot]];
and then each child will use child_listen as listeners list.
The duplicated listeners array (mpm_listen) is built by the following (new)
function :
/* This function is added for the patch. This function duplicates
* open_listeners, alloc_listener() and re-call make_sock() for the
* duplicated listeners. In this function, the newly created sockets
* will bind and listen*/
AP_DECLARE(apr_status_t) ap_post_config_listeners(server_rec *s, apr_pool_t *p,
int num_buckets) {
mpm_listen = apr_palloc(p, sizeof(ap_listen_rec*) * num_buckets);
int i;
ap_listen_rec *lr;
/* duplicate from alloc_listener() for the additional listen record*/
lr = ap_listeners;
for (i = 0; i < num_buckets; i++) {
#ifdef HAVE_SO_REUSEPORT
ap_listen_rec *templr;
ap_listen_rec *last = NULL;
while (lr) {
templr = ap_duplicate_listener(p, lr);
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Bill, I was just worried about forking mpm_prefork into mpm_prefork_buckets, and so on with worker/event/..., most of the code would have been the same. But I can't disagree with you, factorizing the existing MPMs shared codes (it seems there are quite some) and future ones into a common interface is indeed the way to go. +1 for as much MPM as needed provided minimal code is duplicated. Best regards, Yann. On Thu, Mar 6, 2014 at 5:49 PM, William A. Rowe Jr. wrote: > Yann, > > what you might wish to consider is that each individual MPM may be > compiled alongside the others. If you do a feature select, you are > left with one of the other. > > If it is designed to cohabitate, then it may share sources under the > os/ branch, but can still exist as a separate loadable MPM. Any new > MPM idea with a proper ICLA/CCLA/Code Grant, such that our users can > make the decision for themselves, I expect will be swiftly tested, > benchmarked and perhaps adopted. > > If it is to exist as a replacement of an MPM in the stable 2.4 tree, I > expect it will meet with stiff resistance. Let's work out an > appropriate way to adopt the code? > > Respectfully, > > Bill > > On Thu, Mar 6, 2014 at 8:26 AM, Yann Ylavic wrote: > > I'm not sure (yet) a new MPM is needed, or rather, multiple new MPMs are > > needed. > > > > The "bucketized" listeners is applyable to all (*nix only?) MPMs, that > would > > lead to as much forks... > > Couldn't new directives be created instead (ServerBucketsNum, Listen > > , ...), defaulting to the current behaviour? > > > > > > On Thu, Mar 6, 2014 at 2:49 PM, Jim Jagielski wrote: > >> > >> ++1. > >> > >> > >> On Mar 6, 2014, at 3:15 AM, Plüm, Rüdiger, Vodafone Group > >> wrote: > >> > >> > > >> > > >> >> -Original Message- > >> >> From: William A. Rowe Jr. [mailto:wmr...@gmail.com] > >> >> Sent: Donnerstag, 6. März 2014 06:58 > >> >> To: dev@httpd.apache.org > >> >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > >> >> SO_REUSEPORT support > >> >> > >> >> > >> >> If you want to truly re-architect the MPM, by all means, propose it > as > >> >> another MPM module. If it isn't adopted here, please don't hesitate > >> >> to offer it to interested users as separate source (although I hope > we > >> >> find a way to adopt it.) > >> >> > >> >> The idea of different MPM's was that they were swappable. MPM foo > >> >> isn't MPM bar. E.g., worker, prefork, event each have their own > tree. > >> >> Likewise, there is nothing stopping us from having 2, or 3 MPM's on > >> >> Windows, and there is nothing stopping us from stating that there is > a > >> >> prerequisite on a particular MPM of Linux 3.1 kernels or Windows > >> >> 2008+. > >> > > >> > +1 to a new MPM on trunk. This gives it more time to settle and to > >> > stabilize > >> > without disrupting current stuff. And if it is fast and stable it will > >> > certainly > >> > cause the 'older' MPM to drop in userbase :-). > >> > IMHO this would even open a path to 2.4.x provided that we do not need > >> > any other > >> > non backportable changes to do this. > >> > > >> > Regards > >> > > >> > Rüdiger > >> > > >> > > >
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Yann, what you might wish to consider is that each individual MPM may be compiled alongside the others. If you do a feature select, you are left with one of the other. If it is designed to cohabitate, then it may share sources under the os/ branch, but can still exist as a separate loadable MPM. Any new MPM idea with a proper ICLA/CCLA/Code Grant, such that our users can make the decision for themselves, I expect will be swiftly tested, benchmarked and perhaps adopted. If it is to exist as a replacement of an MPM in the stable 2.4 tree, I expect it will meet with stiff resistance. Let's work out an appropriate way to adopt the code? Respectfully, Bill On Thu, Mar 6, 2014 at 8:26 AM, Yann Ylavic wrote: > I'm not sure (yet) a new MPM is needed, or rather, multiple new MPMs are > needed. > > The "bucketized" listeners is applyable to all (*nix only?) MPMs, that would > lead to as much forks... > Couldn't new directives be created instead (ServerBucketsNum, Listen > , ...), defaulting to the current behaviour? > > > On Thu, Mar 6, 2014 at 2:49 PM, Jim Jagielski wrote: >> >> ++1. >> >> >> On Mar 6, 2014, at 3:15 AM, Plüm, Rüdiger, Vodafone Group >> wrote: >> >> > >> > >> >> -Original Message- >> >> From: William A. Rowe Jr. [mailto:wmr...@gmail.com] >> >> Sent: Donnerstag, 6. März 2014 06:58 >> >> To: dev@httpd.apache.org >> >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >> >> SO_REUSEPORT support >> >> >> >> >> >> If you want to truly re-architect the MPM, by all means, propose it as >> >> another MPM module. If it isn't adopted here, please don't hesitate >> >> to offer it to interested users as separate source (although I hope we >> >> find a way to adopt it.) >> >> >> >> The idea of different MPM's was that they were swappable. MPM foo >> >> isn't MPM bar. E.g., worker, prefork, event each have their own tree. >> >> Likewise, there is nothing stopping us from having 2, or 3 MPM's on >> >> Windows, and there is nothing stopping us from stating that there is a >> >> prerequisite on a particular MPM of Linux 3.1 kernels or Windows >> >> 2008+. >> > >> > +1 to a new MPM on trunk. This gives it more time to settle and to >> > stabilize >> > without disrupting current stuff. And if it is fast and stable it will >> > certainly >> > cause the 'older' MPM to drop in userbase :-). >> > IMHO this would even open a path to 2.4.x provided that we do not need >> > any other >> > non backportable changes to do this. >> > >> > Regards >> > >> > Rüdiger >> > >> >
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
I'm not sure (yet) a new MPM is needed, or rather, multiple new MPMs are needed. The "bucketized" listeners is applyable to all (*nix only?) MPMs, that would lead to as much forks... Couldn't new directives be created instead (ServerBucketsNum, Listen , ...), defaulting to the current behaviour? On Thu, Mar 6, 2014 at 2:49 PM, Jim Jagielski wrote: > ++1. > > > On Mar 6, 2014, at 3:15 AM, Plüm, Rüdiger, Vodafone Group < > ruediger.pl...@vodafone.com> wrote: > > > > > > >> -Original Message- > >> From: William A. Rowe Jr. [mailto:wmr...@gmail.com] > >> Sent: Donnerstag, 6. März 2014 06:58 > >> To: dev@httpd.apache.org > >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > >> SO_REUSEPORT support > >> > >> > >> If you want to truly re-architect the MPM, by all means, propose it as > >> another MPM module. If it isn't adopted here, please don't hesitate > >> to offer it to interested users as separate source (although I hope we > >> find a way to adopt it.) > >> > >> The idea of different MPM's was that they were swappable. MPM foo > >> isn't MPM bar. E.g., worker, prefork, event each have their own tree. > >> Likewise, there is nothing stopping us from having 2, or 3 MPM's on > >> Windows, and there is nothing stopping us from stating that there is a > >> prerequisite on a particular MPM of Linux 3.1 kernels or Windows > >> 2008+. > > > > +1 to a new MPM on trunk. This gives it more time to settle and to > stabilize > > without disrupting current stuff. And if it is fast and stable it will > certainly > > cause the 'older' MPM to drop in userbase :-). > > IMHO this would even open a path to 2.4.x provided that we do not need > any other > > non backportable changes to do this. > > > > Regards > > > > Rüdiger > > > >
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
On Wed, Mar 5, 2014 at 11:38 AM, Lu, Yingqi wrote: > 1. If I understand correctly (please correct me if not), do you suggest > duplicating the listen socks inside the child process with SO_REUSEPROT > enabled? Yes, I agree this would be a cleaner implementation and I actually > tried that before. However, I encountered the "connection reset" error since > the number of the child process is changing. I googled online and found it > actually being discussed here at http://lwn.net/Articles/542629/. You actually need to refer to Unix Network Programming, Vol 1 by W Richard Stevens to have a context to ask the question. Apologies if that comes across as harsh, but either Stevens is wrong, or the kernel developers are wrong. It is unlikely that both are wrong and very likely that both are not right. Sometimes we just have to send kernel bugs upstream.
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
++1. On Mar 6, 2014, at 3:15 AM, Plüm, Rüdiger, Vodafone Group wrote: > > >> -Original Message- >> From: William A. Rowe Jr. [mailto:wmr...@gmail.com] >> Sent: Donnerstag, 6. März 2014 06:58 >> To: dev@httpd.apache.org >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with >> SO_REUSEPORT support >> >> >> If you want to truly re-architect the MPM, by all means, propose it as >> another MPM module. If it isn't adopted here, please don't hesitate >> to offer it to interested users as separate source (although I hope we >> find a way to adopt it.) >> >> The idea of different MPM's was that they were swappable. MPM foo >> isn't MPM bar. E.g., worker, prefork, event each have their own tree. >> Likewise, there is nothing stopping us from having 2, or 3 MPM's on >> Windows, and there is nothing stopping us from stating that there is a >> prerequisite on a particular MPM of Linux 3.1 kernels or Windows >> 2008+. > > +1 to a new MPM on trunk. This gives it more time to settle and to stabilize > without disrupting current stuff. And if it is fast and stable it will > certainly > cause the 'older' MPM to drop in userbase :-). > IMHO this would even open a path to 2.4.x provided that we do not need any > other > non backportable changes to do this. > > Regards > > Rüdiger >
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
On Wed, Mar 5, 2014 at 6:38 PM, Lu, Yingqi wrote:
>
>
> 1. If I understand correctly (please correct me if not), do you suggest
> duplicating the listen socks inside the child process with SO_REUSEPROT
> enabled? Yes, I agree this would be a cleaner implementation and I actually
> tried that before. However, I encountered the "connection reset" error
> since the number of the child process is changing. I googled online and
> found it actually being discussed here at http://lwn.net/Articles/542629/.
>
>
Actually I found that article, but expected the "defect" was solved since
then...
This looks like a thorn in the side of MPMs in general,
but couldn't find any pointer to a fix, do you know if there is some
progress on this in the latest linux kernel?
For testing purpose (until then?), you could also configure MPM prefork to
not create/terminate children processes once started (using the same value
for StartServers and ServerLimit, still MaxRequetsPerChild 0).
It could be interesting to see how SO_REUSEPORT scales in these optimal
conditions (no lock, full OS round-robin on all listeners).
For this you would have to use your former patch (duplicating listeners in
each child process), and do nothing in SAFE_ACCEPT when HAVE_SO_REUSEPORT.
Also, SO_REUSEPORT exists on (and even comes from) FreeBSD if I am not
mistaken, but it seems that there is no round-robin garantee for it, is
there? Could this patch also take advantage of BSD's SO_REUSEPORT
implementation?
> 2. Then, I decided to do the socket duplication in the parent process.
> The goal of this change is to extend the CPU thread scalability with the
> big thread count system. Therefore, I just very simply defined
> number_of_listen_buckets=total_number_active_thread/8, and each listen
> bucket has a dedicated listener. I do not want to over duplicate the
> socket; otherwise, it would create too many child processes at the
> beginning. One listen bucket should have at least one child process to
> start with. However, this is only my understanding and it may not be
> correct and complete. If you have other ideas, please share with us.
> Feedbacks and comments are very welcome here J
>
The listeners buckets make sense with SO_REUSEPORT given the defect, I
hope this is temporary.
>
> 3. I am struggling with myself as well on if we should put with and
> without SO_REUSEPORT into two different patches. The only reason I put them
> together is because they both use the concept of listen buckets. If you
> think it would make more sense to separate them into two patches, I can
> certainly do that. Also, I am a little bit confused about your comments "On
> the other hand, each child is dedicated, won't one have to multiply the
> configured ServerLimit by the number of Listen to achieve the same (maximum
> theoretical) scalability with regard to all the listeners?". Can you please
> explain a little bit more on this? Really appreciate.
>
Sorry to have not been clear enough (nay at all).
I'm referring to the following code.
In prefork.c::make_child(), each child is assigned a listener like this
(before fork()ing) :
child_listen = mpm_listen[bucket[slot]];
and then each child will use child_listen as listeners list.
The duplicated listeners array (mpm_listen) is built by the following (new)
function :
/* This function is added for the patch. This function duplicates
* open_listeners, alloc_listener() and re-call make_sock() for the
* duplicated listeners. In this function, the newly created sockets
* will bind and listen*/
AP_DECLARE(apr_status_t) ap_post_config_listeners(server_rec *s, apr_pool_t
*p,
int num_buckets) {
mpm_listen = apr_palloc(p, sizeof(ap_listen_rec*) * num_buckets);
int i;
ap_listen_rec *lr;
/* duplicate from alloc_listener() for the additional listen record*/
lr = ap_listeners;
for (i = 0; i < num_buckets; i++) {
#ifdef HAVE_SO_REUSEPORT
ap_listen_rec *templr;
ap_listen_rec *last = NULL;
while (lr) {
templr = ap_duplicate_listener(p, lr);
ap_apply_accept_filter(p, templr, s);
if (last == NULL) {
mpm_listen[i] = last = templr;
}
else {
last->next = templr;
last = templr;
}
lr = lr->next;
}
lr = ap_listeners;
#else
mpm_listen[i] = ap_duplicate_listener(p, lr);
lr = (lr->next) ? lr->next : ap_listeners;
#endif
}
return APR_SUCCESS;
}
Since ap_duplicate_listener() will duplicate a single (unlinked) listener,
my understanding is that :
- with SO_REUSEPORT: each child will use all the listeners (the whole list
is duplicated per bucket),
- without SO_REUSEPORT: each child will use a single listener
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
On 06 Mar 2014, at 10:15 AM, "Plüm, Rüdiger, Vodafone Group" wrote: > +1 to a new MPM on trunk. This gives it more time to settle and to stabilize > without disrupting current stuff. And if it is fast and stable it will > certainly > cause the 'older' MPM to drop in userbase :-). > IMHO this would even open a path to 2.4.x provided that we do not need any > other > non backportable changes to do this. +1. Regards, Graham --
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
> -Original Message- > From: William A. Rowe Jr. [mailto:wmr...@gmail.com] > Sent: Donnerstag, 6. März 2014 06:58 > To: dev@httpd.apache.org > Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with > SO_REUSEPORT support > > > If you want to truly re-architect the MPM, by all means, propose it as > another MPM module. If it isn't adopted here, please don't hesitate > to offer it to interested users as separate source (although I hope we > find a way to adopt it.) > > The idea of different MPM's was that they were swappable. MPM foo > isn't MPM bar. E.g., worker, prefork, event each have their own tree. > Likewise, there is nothing stopping us from having 2, or 3 MPM's on > Windows, and there is nothing stopping us from stating that there is a > prerequisite on a particular MPM of Linux 3.1 kernels or Windows > 2008+. +1 to a new MPM on trunk. This gives it more time to settle and to stabilize without disrupting current stuff. And if it is fast and stable it will certainly cause the 'older' MPM to drop in userbase :-). IMHO this would even open a path to 2.4.x provided that we do not need any other non backportable changes to do this. Regards Rüdiger
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Bill, Thanks very much for your email and I am really happy that I got lots of very good feedbacks on the email list. The patch was created only for Linux Prefork mpm so that it should not impact winnt_mpm. I may misunderstand you here, but do you mean in order to adopt the patch, we need to extend it for winnt_mpm? Regarding to the testing result, what we provided was based on RHEL 6.2 (server version) with kernel 3.10.4. We measured the throughput as operations/sec as well as the response time defined by the time that a request sending from the client till it gets the response back. It is a three tier webserver workload. We measured the throughput on the frontend webserver tier (Apache httpd with Prefork + PHP as libphp5.so under httpd/modules). Thanks, Yingqi -Original Message- From: William A. Rowe Jr. [mailto:wmr...@gmail.com] Sent: Wednesday, March 05, 2014 9:58 PM To: dev@httpd.apache.org Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Yingqi, as one of the 'Windows folks' here, your idea is very intriguing, and I'm sorry that other issues have distracted me from giving it the attention it deserves. If you want to truly re-architect the MPM, by all means, propose it as another MPM module. If it isn't adopted here, please don't hesitate to offer it to interested users as separate source (although I hope we find a way to adopt it.) The idea of different MPM's was that they were swappable. MPM foo isn't MPM bar. E.g., worker, prefork, event each have their own tree. Likewise, there is nothing stopping us from having 2, or 3 MPM's on Windows, and there is nothing stopping us from stating that there is a prerequisite on a particular MPM of Linux 3.1 kernels or Windows 2008+. The Windows build system hasn't been so flexible, but this can be remediated with cmake, as folks have spent many hours to accomplish. I understand you are probably relying on functions authored entirely for the winnt_mpm, and we can re-factor those on trunk out to the os/win32/ directory so that MPM's may share them. The definition of the word "prefork" is a single thread process which handles a request. Please don't misuse the phrase, and without reviewing your code, I'll presume that is what you meant. I don't doubt your results of benchmarking, but please make note that only Windows Server OS's can actually be used to perform any benchmarks. Any 'desktop' enterprise, professional or home editions are deliberately hobbled, and IMHO the project should make no accommodation for vendor stupidity. In terms of benchmarking, I don't know how you measured, but if you can peg a machine at 95% total utilization yet httpd shows itself consuming only 70% or 60%, that means it is kernel-bound. That is usually a good thing, that the app is operating optimally and is only constrained by the architecture. I think I understand where you are going with reuseport. That doesn't equate to the Unix OS's... they can distribute the already opened listener to an unlimited number of forks. On windows, we also distribute the listener through a write/stdin channel to the child process. What doesn't work well is for parallel windows children to share certain resources such as the error log, access log etc. But we can contend with that issue. What we can't contend with is what 3rd party modules have chosen to do, and almost any patch you offer is not going to be suitable for binary compatibility with 3rd party httpd 2.4 modules compiled for windows, so your patch presented for the 2.4 branch is rejected. That said, we should endeavor to solve this for 2.6 (or 3.0 or whatever we call the 'next httpd'). We are all out of fresh ideas, so proposals such as yours are a welcome sight!!! Finally, please do have patience, large patches require time for us to digest, and we have limited amounts of that resource. As I mention, adding a whole new MPM directory to trunk, alone, should meet very little resistance for any architectures. Thank you for your posts, and please do not feel ignored. There are a handful of people active and we all have many details to attend to. Yours, Bill On Fri, Jan 24, 2014 at 5:25 PM, Lu, Yingqi wrote: > Dear All, > > > > Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread > Intel Xeon 2600 series systems, using an open source three tier social > networking web server workload, revealed performance scaling issues. > In current software single listen statement (listen 80) provides > better scalability due to un-serialized accept. However, when system > is under very high load, this can lead to big number of child > processes stuck in D state. On the other hand, the serialized accept approach > cannot scale with the high load either.
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
On Thursday 06 of March 2014, William A. Rowe Jr. wrote: > If you want to truly re-architect the MPM, by all means, propose it as > another MPM module. If it isn't adopted here, please don't hesitate > to offer it to interested users as separate source (although I hope we > find a way to adopt it.) > > The idea of different MPM's was that they were swappable. MPM foo > isn't MPM bar. E.g., worker, prefork, event each have their own tree. > Likewise, there is nothing stopping us from having 2, or 3 MPM's on > Windows, and there is nothing stopping us from stating that there is a > prerequisite on a particular MPM of Linux 3.1 kernels or Windows > 2008+. I dislike idea of "yet another mpm". More mpm means that each mpm gets lower developer resources and lower testing (and external mpm, distributed outside apache get almost no devs and no testing). Less MPMs is better IMO. So better to improve existing ones than invent new one. > (although I hope we find a way to adopt it.) +1 -- Arkadiusz Miśkiewicz, arekm / maven.pl
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Yingqi, as one of the 'Windows folks' here, your idea is very intriguing, and I'm sorry that other issues have distracted me from giving it the attention it deserves. If you want to truly re-architect the MPM, by all means, propose it as another MPM module. If it isn't adopted here, please don't hesitate to offer it to interested users as separate source (although I hope we find a way to adopt it.) The idea of different MPM's was that they were swappable. MPM foo isn't MPM bar. E.g., worker, prefork, event each have their own tree. Likewise, there is nothing stopping us from having 2, or 3 MPM's on Windows, and there is nothing stopping us from stating that there is a prerequisite on a particular MPM of Linux 3.1 kernels or Windows 2008+. The Windows build system hasn't been so flexible, but this can be remediated with cmake, as folks have spent many hours to accomplish. I understand you are probably relying on functions authored entirely for the winnt_mpm, and we can re-factor those on trunk out to the os/win32/ directory so that MPM's may share them. The definition of the word "prefork" is a single thread process which handles a request. Please don't misuse the phrase, and without reviewing your code, I'll presume that is what you meant. I don't doubt your results of benchmarking, but please make note that only Windows Server OS's can actually be used to perform any benchmarks. Any 'desktop' enterprise, professional or home editions are deliberately hobbled, and IMHO the project should make no accommodation for vendor stupidity. In terms of benchmarking, I don't know how you measured, but if you can peg a machine at 95% total utilization yet httpd shows itself consuming only 70% or 60%, that means it is kernel-bound. That is usually a good thing, that the app is operating optimally and is only constrained by the architecture. I think I understand where you are going with reuseport. That doesn't equate to the Unix OS's... they can distribute the already opened listener to an unlimited number of forks. On windows, we also distribute the listener through a write/stdin channel to the child process. What doesn't work well is for parallel windows children to share certain resources such as the error log, access log etc. But we can contend with that issue. What we can't contend with is what 3rd party modules have chosen to do, and almost any patch you offer is not going to be suitable for binary compatibility with 3rd party httpd 2.4 modules compiled for windows, so your patch presented for the 2.4 branch is rejected. That said, we should endeavor to solve this for 2.6 (or 3.0 or whatever we call the 'next httpd'). We are all out of fresh ideas, so proposals such as yours are a welcome sight!!! Finally, please do have patience, large patches require time for us to digest, and we have limited amounts of that resource. As I mention, adding a whole new MPM directory to trunk, alone, should meet very little resistance for any architectures. Thank you for your posts, and please do not feel ignored. There are a handful of people active and we all have many details to attend to. Yours, Bill On Fri, Jan 24, 2014 at 5:25 PM, Lu, Yingqi wrote: > Dear All, > > > > Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel > Xeon 2600 series systems, using an open source three tier social networking > web server workload, revealed performance scaling issues. In current > software single listen statement (listen 80) provides better scalability due > to un-serialized accept. However, when system is under very high load, this > can lead to big number of child processes stuck in D state. On the other > hand, the serialized accept approach cannot scale with the high load either. > In our analysis, a 32-thread system, with 2 listen statements specified, > could scale to just 70% utilization, and a 64-thread system, with signal > listen statement specified (listen 80, 4 network interfaces), could scale to > only 60% utilization. > > > > Based on those findings, we created a prototype patch for prefork mpm which > extends performance and thread utilization. In Linux kernel newer than 3.9, > SO_REUSEPORT is enabled. This feature allows multiple sockets listen to the > same IP:port and automatically round robins connections. We use this feature > to create multiple duplicated listener records of the original one and > partition the child processes into buckets. Each bucket listens to 1 > IP:port. In case of old kernel which does not have the SO_REUSEPORT enabled, > we modified the "multiple listen statement case" by creating 1 listen record > for each listen statement and partitioning the child processes into > different buckets. Each bucket listens to 1 IP:port. >
RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
Hi Yann, Thanks very much for your email. 1. If I understand correctly (please correct me if not), do you suggest duplicating the listen socks inside the child process with SO_REUSEPROT enabled? Yes, I agree this would be a cleaner implementation and I actually tried that before. However, I encountered the "connection reset" error since the number of the child process is changing. I googled online and found it actually being discussed here at http://lwn.net/Articles/542629/. 2. Then, I decided to do the socket duplication in the parent process. The goal of this change is to extend the CPU thread scalability with the big thread count system. Therefore, I just very simply defined number_of_listen_buckets=total_number_active_thread/8, and each listen bucket has a dedicated listener. I do not want to over duplicate the socket; otherwise, it would create too many child processes at the beginning. One listen bucket should have at least one child process to start with. However, this is only my understanding and it may not be correct and complete. If you have other ideas, please share with us. Feedbacks and comments are very welcome here :) 3. I am struggling with myself as well on if we should put with and without SO_REUSEPORT into two different patches. The only reason I put them together is because they both use the concept of listen buckets. If you think it would make more sense to separate them into two patches, I can certainly do that. Also, I am a little bit confused about your comments "On the other hand, each child is dedicated, won't one have to multiply the configured ServerLimit by the number of Listen to achieve the same (maximum theoretical) scalability with regard to all the listeners?". Can you please explain a little bit more on this? Really appreciate. This is our first patch to the open source and Apache community. We are still on the learning curve about a lot of things. Your feedback and comments really help us! Please let me know if you have any further questions. Thanks, Yingqi From: Yann Ylavic [mailto:ylavic@gmail.com] Sent: Wednesday, March 05, 2014 5:04 AM To: httpd Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support Hi Yingqi, I'm a bit confused about the patch, mainly because it seems to handle the same way both with and without SO_REUSEPORT available, while SO_REUSEPORT could (IMHO) be handled in children only (a less intrusive way). With SO_REUSEPORT, I would have expected the accept mutex to be useless since, if I understand correcly the option, multiple processes/threads can accept() simultaneously provided they use their own socket (each one bound/listening on the same addr:port). Couldn't then each child duplicate the listeners (ie. new socket+bind(SO_REUSEPORT)+listen), before switching UIDs, and then poll() all of them without synchronisation (accept() is probably not an option for timeout reasons), and then get fair scheduling from the OS (for all the listeners)? Is the lock still needed because the duplicated listeners are inherited from the parent process? Without SO_REUSEPORT, if I understand correctly still, each child will poll() a single listener to avoid the serialized accept. On the other hand, each child is dedicated, won't one have to multiply the configured ServerLimit by the number of Listen to achieve the same (maximum theoretical) scalability with regard to all the listeners? I don't pretend it is a good or bad thing, just figuring out what could then be a "rule" to size the configuration (eg. MaxClients/ServerLimit/#cores/#Listen). It seems to me that the patches with and without SO_REUSEPORT should be separate ones, but I may be missing something. Also, but this is not related to this patch particularly (addressed to who knows), it's unclear to me why an accept mutex is needed at all. Multiple processes poll()ing the same inherited socket is safe but not multiple ones? Is that an OS issue? Process wide only? Still (in)valid in latest OSes? Thanks for the patch anyway, it looks promising. Regards, Yann. On Sat, Jan 25, 2014 at 12:25 AM, Lu, Yingqi mailto:yingqi...@intel.com>> wrote: Dear All, Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel Xeon 2600 series systems, using an open source three tier social networking web server workload, revealed performance scaling issues. In current software single listen statement (listen 80) provides better scalability due to un-serialized accept. However, when system is under very high load, this can lead to big number of child processes stuck in D state. On the other hand, the serialized accept approach cannot scale with the high load either. In our analysis, a 32-thread system, with 2 listen statements specified, could scale to just 70% utilization, and a 64-thread system, with signal listen statement specified (listen 80, 4 ne
Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support
On Wed, Mar 5, 2014 at 2:04 PM, Yann Ylavic wrote: > Also, but this is not related to this patch particularly (addressed to > who knows), it's unclear to me why an accept mutex is needed at all. > Multiple processes poll()ing the same inherited socket is safe but not > multiple ones? Is that an OS issue? Process wide only? Still (in)valid in > latest OSes? > I mean when SINGLE_LISTEN_UNSERIALIZED_ACCEPT is set only, the OS has the capability to do unserialized accept with one socket but not any more.
