Re: Allow SSLProxy* config in context?
On 4/14/2016 3:08 AM, Rainer Jung wrote: > Your idea to allow selecting a client cert based on CN or DN sounds > attractive to me as well. But since it wouldn't help with other per > backend settings (like different Verify settings) we might even think > about combining both. As long as your only problem would be client > certs, you can select via CN or DN and keep the config vhost global, > but once you need more adjustments per backend, you would start to > configure SSLProxy* inside . > > WDYT? Yep! Agreed - I was focusing on the use case presented, but didn't think about other cases such as different SSLProxyVerify settings and friends such as this. I'd cast a vote for the more comprehensive solution you and Graham shared that makes each option available in a directory context. Adding yet another directive would satisfy this one use case, but the bigger picture would serve users better and be less confusing. Food for thought - I haven't dug into these... * Not sure if it would be a handy target of opportunity or PITA... Since ProxyPass is currently server/vhost/dir configurable, should we think to include this capability as arguments to ProxyPass/BalancerMember, too? * Maybe I missed it, but I don't think the proposed ideas truly provide per-backend configuration. Rather, they provide configuration for all backends in a dir context if you land on a balancer containing many targets. Is that OK, or should we be thinking about ways to make it even more granular? (the use cases that come to mind are rather contrived, so I may be overthinking it) * Would each of these per-dir configs post-merge result in a distinct SSL_CTX that needs to be created in mod_ssl? If so, does that require significant refactoring in mod_ssl or does the proxy module manage these? -- Daniel Ruggeri
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On Thu, Apr 14, 2016 at 10:58 PM, olli hauer wrote: > > OK, I've attached the output from a 2.4.18 scoreboard some sec. after > ab -k -n 10 -c 100 -f TLS1.2 $host/$url > > the 2.4.18 scoreboard looks similar to 2.4.20 with your patch Thank you very much olli.
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On 2016-04-14 22:43, Yann Ylavic wrote: > On Thu, Apr 14, 2016 at 10:28 PM, olli hauer wrote: >> >> I've done some tests with 2.4.19 there is maybe an interesting detail. >> With 2.4.19 the last request is empty for an idle worker, with 2.4.20 not >> (shows the client, proto, Vhost and request) > > Sorry for the confusion, 2.4.19 (not released) contained the change > already, the relevant version to compare with is 2.4.18... > OK, I've attached the output from a 2.4.18 scoreboard some sec. after ab -k -n 10 -c 100 -f TLS1.2 $host/$url the 2.4.18 scoreboard looks similar to 2.4.20 with your patch -- olli Server Version: Apache/2.4.18 (FreeBSD) OpenSSL/1.0.2g SVN/1.8.15 mod_wsgi/4.4.22 Python/2.7.11 Server MPM: event Server Built: Apr 14 2016 22:44:09 Current Time: Thursday, 14-Apr-2016 22:45:55 CEST Restart Time: Thursday, 14-Apr-2016 22:45:33 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 21 seconds Server load: 0.69 0.70 0.77 Total accesses: 0 - Total Traffic: 0 kB CPU Usage: u0 s0 cu0 cs0 0 requests/sec - 0 B/second - 1 requests currently being processed, 74 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 97140 0 yes 0 25 0 0 0 97141 0 yes 1 24 0 0 0 97142 0 yes 0 25 0 0 0 Sum 0 1 74 0 0 0 Current Time: Thursday, 14-Apr-2016 22:47:16 CEST Restart Time: Thursday, 14-Apr-2016 22:45:33 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 1 minute 43 seconds Server load: 0.70 0.71 0.77 Total accesses: 99957 - Total Traffic: 21.7 MB CPU Usage: u22.9766 s8.60156 cu0 cs0 - 30.7% CPU load 970 requests/sec - 216.0 kB/second - 227 B/request 1 requests currently being processed, 124 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 97140 0 yes 0 25 0 0 0 97141 1 yes 1 24 0 0 0 97142 0 yes 0 25 0 0 0 97192 0 yes 0 25 0 0 0 97212 0 yes 0 25 0 0 0 Sum 1 1 124 0 0 0 _R__ _... Scoreboard Key: "_" Waiting for Connection, "S" Starting up, "R" Reading Request, "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup, "C" Closing connection, "L" Logging, "G" Gracefully finishing, "I" Idle cleanup of worker, "." Open slot with no current process Srv PID Acc M CPU SS Req ConnChild Slot Client VHost Request 0-0 97140 0/1377/1377 _ 10.77 39 0 0.0 0.29 0.29$client 0-0 97140 0/1556/1556 _ 10.73 39 0 0.0 0.33 0.33$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1355/1355 _ 10.73 39 0 0.0 0.29 0.29$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1385/1385 _ 10.77 39 0 0.0 0.29 0.29$client 0-0 97140 0/1326/1326 _ 10.74 39 0 0.0 0.28 0.28$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1402/1402 _ 10.76 39 0 0.0 0.30 0.30$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1381/1381 _ 10.77 39 0 0.0 0.29 0.29$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1467/1467 _ 10.77 39 0 0.0 0.31 0.31$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1486/1486 _ 10.73 39 0 0.0 0.32 0.32$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1420/1420 _ 10.76 39 0 0.0 0.30 0.30$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1377/1377 _ 10.77 39 0 0.0 0.29 0.29$client 0-0 97140 0/1289/1289 _ 10.74 39 0 0.0 0.27 0.27$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1436/1436 _ 10.73 39 0 0.0 0.31 0.31$client $host:443 GET /server-status/ HTTP/1.0 0-0 97140 0/1352/1352 _ 10.72 39 0
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On Thu, Apr 14, 2016 at 10:28 PM, olli hauer wrote: > > I've done some tests with 2.4.19 there is maybe an interesting detail. > With 2.4.19 the last request is empty for an idle worker, with 2.4.20 not > (shows the client, proto, Vhost and request) Sorry for the confusion, 2.4.19 (not released) contained the change already, the relevant version to compare with is 2.4.18...
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On 2016-04-14 22:14, Yann Ylavic wrote: > On Thu, Apr 14, 2016 at 10:05 PM, olli hauer wrote: >> On 2016-04-14 21:48, Yann Ylavic wrote: >>> On Thu, Apr 14, 2016 at 9:40 PM, olli hauer wrote: I've done a quick test with $ ab -n 1 -c 100 $host/$url During the test the count of idle worker are incrementing and decrementing but it ab has finished the requests the count of idle workers stays on the last highest count ... >>> >>> This looks normal to me, did the bahaviour changed w.r.t. 2.4.19? >>> >>> When the stress is running, the number of idlers (workers that do >>> nothing) fluctuates, according to the number of connections/requests >>> and children created to sustain the load. >>> >>> When the load stops, the number of idlers increases, and won't >>> decrease unless MaxSpareThreads is reached (i.e. children are stopped, >>> each releasing ThreadsPerChild idlers). >>> >> >> Ah, OK this will explain the numbers. >> Unluckily I haven't looked to scoreboard with 2.4.19, but give me some >> minutes ... > > Could you also confirm with 2.4.20 + my patch that the Client, VHost > and Request colomns are always filled with latest values (not empty) > for idle workers (used at least once)? > > This requires "ExtendedStatus on" (or default) for the corresponding > table to be displayed by mod_status. > > Thanks for testing! > Yes, I can confirm this, see the log in the answer to Rainer. -- olli
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On 2016-04-14 22:05, Rainer Jung wrote: > Am 14.04.2016 um 22:05 schrieb olli hauer: >> On 2016-04-14 21:48, Yann Ylavic wrote: >>> On Thu, Apr 14, 2016 at 9:40 PM, olli hauer wrote: I've done a quick test with $ ab -n 1 -c 100 $host/$url During the test the count of idle worker are incrementing and decrementing but it ab has finished the requests the count of idle workers stays on the last highest count ... >>> >>> This looks normal to me, did the bahaviour changed w.r.t. 2.4.19? >>> >>> When the stress is running, the number of idlers (workers that do >>> nothing) fluctuates, according to the number of connections/requests >>> and children created to sustain the load. >>> >>> When the load stops, the number of idlers increases, and won't >>> decrease unless MaxSpareThreads is reached (i.e. children are stopped, >>> each releasing ThreadsPerChild idlers). >>> >> >> Ah, OK this will explain the numbers. >> Unluckily I haven't looked to scoreboard with 2.4.19, but give me some >> minutes ... > > Not that important for the current discussion, but please note that "ab" does > not use Keep-Alive connections by default. If you want to use HTTP > Keep-Alive, add "-k". > > Without it you create many new connections per second (depending on > throughput this can easily be more than 1000 connections per second) which > are only used very shortly (one request) and after shutting down they might > end up in TIME_WAIT and stay there e.g. for a minute depending on OS details > (you can check via "netstat"). Once you get more than a few thousand > connections in TIME_WAIT, TCP/IP could get slower. Thus "ab -k" will often > result in more stable behavior. > Hi Rainer, Thanks for the hint! (indeed without -k I can simulate the svnstat tool that pulled my svn servers down) I've done some tests with 2.4.19 there is maybe an interesting detail. With 2.4.19 the last request is empty for an idle worker, with 2.4.20 not (shows the client, proto, Vhost and request) I've attached the results as .txt file. -- olli httpd some seconds after start: - Server MPM: event Server Built: Apr 14 2016 22:02:14 Current Time: Thursday, 14-Apr-2016 22:03:24 CEST Restart Time: Thursday, 14-Apr-2016 22:03:08 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 16 seconds Server load: 1.21 0.69 0.66 Total accesses: 0 - Total Traffic: 0 kB CPU Usage: u0 s0 cu0 cs0 0 requests/sec - 0 B/second - 1 requests currently being processed, 74 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 56486 0 yes 0 25 0 0 0 56487 0 yes 0 25 0 0 0 56488 0 yes 1 24 0 0 0 Sum 0 1 74 0 0 0 during ab test - Current Time: Thursday, 14-Apr-2016 22:04:14 CEST Restart Time: Thursday, 14-Apr-2016 22:03:08 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 1 minute 6 seconds Server load: 12.23 3.14 1.53 Total accesses: 7868 - Total Traffic: 1.8 MB CPU Usage: u86.8047 s1.15625 cu0 cs0 - 133% CPU load 119 requests/sec - 28.2 kB/second - 242 B/request 78 requests currently being processed, 97 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 56486 4 yes 1 24 0 0 0 56487 7 yes 7 18 0 0 0 56488 10 yes 8 17 0 0 1 56506 10 yes 3 22 0 0 7 56560 25 yes 24 1 0 0 1 56561 5 yes 14 11 0 0 0 56562 8 yes 21 4 0 0 0 Sum 69 78 97 0 0 9 some time after ab finished - Current Time: Thursday, 14-Apr-2016 22:06:59 CEST Restart Time: Thursday, 14-Apr-2016 22:03:08 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 3 minutes 51 seconds Server load: 1.17 2.09 1.39 Total accesses: 10011 - Total Traffic: 2.5 MB CPU Usage: u109.742 s1.46094 cu0 cs0 - 48.1% CPU load 43.3 requests/sec - 10.9 kB/second - 257 B/request 1 requests currently being processed, 174 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 56486 0 yes 0 25 0 0 0 56487 0 yes 1 24 0 0 0 56488 0 yes 0 25 0 0 0 56506 0 yes 0 25 0 0 0 56560 0 yes 0 25 0 0 0 56561 0 yes 0 25 0
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On Thu, Apr 14, 2016 at 10:05 PM, olli hauer wrote: > On 2016-04-14 21:48, Yann Ylavic wrote: >> On Thu, Apr 14, 2016 at 9:40 PM, olli hauer wrote: >>> I've done a quick test with >>> $ ab -n 1 -c 100 $host/$url >>> >>> During the test the count of idle worker are incrementing and decrementing >>> but it ab has finished the requests the count of idle workers stays on the >>> last highest count ... >> >> This looks normal to me, did the bahaviour changed w.r.t. 2.4.19? >> >> When the stress is running, the number of idlers (workers that do >> nothing) fluctuates, according to the number of connections/requests >> and children created to sustain the load. >> >> When the load stops, the number of idlers increases, and won't >> decrease unless MaxSpareThreads is reached (i.e. children are stopped, >> each releasing ThreadsPerChild idlers). >> > > Ah, OK this will explain the numbers. > Unluckily I haven't looked to scoreboard with 2.4.19, but give me some > minutes ... Could you also confirm with 2.4.20 + my patch that the Client, VHost and Request colomns are always filled with latest values (not empty) for idle workers (used at least once)? This requires "ExtendedStatus on" (or default) for the corresponding table to be displayed by mod_status. Thanks for testing!
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
Am 14.04.2016 um 22:05 schrieb olli hauer: On 2016-04-14 21:48, Yann Ylavic wrote: On Thu, Apr 14, 2016 at 9:40 PM, olli hauer wrote: I've done a quick test with $ ab -n 1 -c 100 $host/$url During the test the count of idle worker are incrementing and decrementing but it ab has finished the requests the count of idle workers stays on the last highest count ... This looks normal to me, did the bahaviour changed w.r.t. 2.4.19? When the stress is running, the number of idlers (workers that do nothing) fluctuates, according to the number of connections/requests and children created to sustain the load. When the load stops, the number of idlers increases, and won't decrease unless MaxSpareThreads is reached (i.e. children are stopped, each releasing ThreadsPerChild idlers). Ah, OK this will explain the numbers. Unluckily I haven't looked to scoreboard with 2.4.19, but give me some minutes ... Not that important for the current discussion, but please note that "ab" does not use Keep-Alive connections by default. If you want to use HTTP Keep-Alive, add "-k". Without it you create many new connections per second (depending on throughput this can easily be more than 1000 connections per second) which are only used very shortly (one request) and after shutting down they might end up in TIME_WAIT and stay there e.g. for a minute depending on OS details (you can check via "netstat"). Once you get more than a few thousand connections in TIME_WAIT, TCP/IP could get slower. Thus "ab -k" will often result in more stable behavior. Regards, Rainer
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On 2016-04-14 21:48, Yann Ylavic wrote: > On Thu, Apr 14, 2016 at 9:40 PM, olli hauer wrote: >> I've done a quick test with >> $ ab -n 1 -c 100 $host/$url >> >> During the test the count of idle worker are incrementing and decrementing >> but it ab has finished the requests the count of idle workers stays on the >> last highest count ... > > This looks normal to me, did the bahaviour changed w.r.t. 2.4.19? > > When the stress is running, the number of idlers (workers that do > nothing) fluctuates, according to the number of connections/requests > and children created to sustain the load. > > When the load stops, the number of idlers increases, and won't > decrease unless MaxSpareThreads is reached (i.e. children are stopped, > each releasing ThreadsPerChild idlers). > Ah, OK this will explain the numbers. Unluckily I haven't looked to scoreboard with 2.4.19, but give me some minutes ... -- olli
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On 2016-04-14 21:47, Eric Covener wrote: > On Thu, Apr 14, 2016 at 3:40 PM, olli hauer wrote: >> During the test the count of idle worker are incrementing and decrementing >> but it ab has finished the requests the count of idle workers stays on the >> last highest count ... > > What was your MaxSpareThreads during the test? > The system is running with the default settings: StartServers 3 MinSpareThreads 75 MaxSpareThreads250 ThreadsPerChild 25 MaxRequestWorkers 400 MaxConnectionsPerChild 0
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On Thu, Apr 14, 2016 at 9:40 PM, olli hauer wrote: > I've done a quick test with > $ ab -n 1 -c 100 $host/$url > > During the test the count of idle worker are incrementing and decrementing > but it ab has finished the requests the count of idle workers stays on the > last highest count ... This looks normal to me, did the bahaviour changed w.r.t. 2.4.19? When the stress is running, the number of idlers (workers that do nothing) fluctuates, according to the number of connections/requests and children created to sustain the load. When the load stops, the number of idlers increases, and won't decrease unless MaxSpareThreads is reached (i.e. children are stopped, each releasing ThreadsPerChild idlers). Regards, Yann.
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On Thu, Apr 14, 2016 at 3:40 PM, olli hauer wrote: > During the test the count of idle worker are incrementing and decrementing > but it ab has finished the requests the count of idle workers stays on the > last highest count ... What was your MaxSpareThreads during the test?
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On 2016-04-14 20:36, Yann Ylavic wrote: > On Thu, Apr 14, 2016 at 7:10 PM, olli hauer wrote: >> I got this morning a request from a FreeBSD user to back-port r1739008, and >> also already the feedback that scoreboard is usable again with the patch. >> >> Since I cannot find a reference in branches/2.4.x/STATUS for r1739008 I told >> the user I will ask additional upstream devs if they think it is OK to >> include the patch until a new httpd release will be shipped. > > r1739008+r1739146+r1739151 proposed for backport (full patch is [1]). Let's > wait for votes... > > Regards, Yann. > > [1] > http://home.apache.org/~ylavic/patches/httpd-2.4.x-scoreboard_preserve.patch > Hi Yann, thanks for the link! I've done a quick test with $ ab -n 1 -c 100 $host/$url During the test the count of idle worker are incrementing and decrementing but it ab has finished the requests the count of idle workers stays on the last highest count ... httpd some seconds after start: - Server Version: Apache/2.4.20 (FreeBSD) OpenSSL/1.0.2g SVN/1.8.15 mod_wsgi/4.4.22 Python/2.7.11 Server MPM: event Server Built: Apr 14 2016 21:09:43 Current Time: Thursday, 14-Apr-2016 21:22:13 CEST Restart Time: Thursday, 14-Apr-2016 21:22:09 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 4 seconds Server load: 0.29 1.36 1.11 Total accesses: 0 - Total Traffic: 0 kB CPU Usage: u0 s0 cu0 cs0 0 requests/sec - 0 B/second - 1 requests currently being processed, 74 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 31501 0 yes 0 25 0 0 0 31502 0 yes 1 24 0 0 0 31503 0 yes 0 25 0 0 0 Sum 0 1 74 0 0 0 after some manual requests: - Current Time: Thursday, 14-Apr-2016 21:22:48 CEST Restart Time: Thursday, 14-Apr-2016 21:22:09 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 39 seconds Server load: 0.28 1.24 1.08 Total accesses: 5 - Total Traffic: 18 kB CPU Usage: u.0234375 s0 cu0 cs0 - .0601% CPU load .128 requests/sec - 472 B/second - 3686 B/request 1 requests currently being processed, 99 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 31501 0 yes 0 25 0 0 0 31502 1 yes 1 24 0 0 0 31503 0 yes 0 25 0 0 0 31505 0 yes 0 25 0 0 0 Sum 1 1 99 0 0 0 give some load with ab -n 1 -c 100 - Current Time: Thursday, 14-Apr-2016 21:23:29 CEST Restart Time: Thursday, 14-Apr-2016 21:22:09 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 1 minute 20 seconds Server load: 1.49 1.38 1.13 Total accesses: 3315 - Total Traffic: 923 kB CPU Usage: u36.7969 s.570313 cu0 cs0 - 46.7% CPU load 41.4 requests/sec - 11.5 kB/second - 285 B/request 86 requests currently being processed, 89 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 31501 3 yes 1 24 0 0 0 31502 2 yes 0 25 0 0 1 31503 25 yes 25 0 0 0 1 31505 25 yes 21 4 0 0 1 31579 2 yes 23 2 0 0 0 31580 3 yes 14 11 0 0 0 31581 3 yes 2 23 0 0 1 Sum 63 86 89 0 0 4 During the test the idle workers are going up and down but stay on the highest value after the test is finished. ab requests finished, wait some time ... -- Current Time: Thursday, 14-Apr-2016 21:29:12 CEST Restart Time: Thursday, 14-Apr-2016 21:22:09 CEST Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 7 minutes 3 seconds Server load: 0.20 0.71 0.92 Total accesses: 10024 - Total Traffic: 2.9 MB CPU Usage: u109.984 s1.4375 cu0 cs0 - 26.3% CPU load 23.7 requests/sec - 7.0 kB/second - 302 B/request 1 requests currently being processed, 174 idle workers PID Connections Threads Async connections total accepting busyidlewriting keep-alive closing 31501 0 yes 0 25 0 0 0 31502 0 yes 0 25 0 0 0 31503 0 yes 0 25 0 0 0 31505 0 yes 0 25 0 0 0 31579 0 yes 0 25 0 0 0 31580
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On Thu, Apr 14, 2016 at 7:10 PM, olli hauer wrote: > I got this morning a request from a FreeBSD user to back-port r1739008, and > also already the feedback that scoreboard is usable again with the patch. > > Since I cannot find a reference in branches/2.4.x/STATUS for r1739008 I told > the > user I will ask additional upstream devs if they think it is OK to include > the patch until a new httpd release will be shipped. r1739008+r1739146+r1739151 proposed for backport (full patch is [1]). Let's wait for votes... Regards, Yann. [1] http://home.apache.org/~ylavic/patches/httpd-2.4.x-scoreboard_preserve.patch
Re: New segfault with 2.4.20 with mod_perl
The defect appears to be in t/protocol/TestProtocol/pseudo_http.pm...
First, the handler is registered using
PerlProcessConnectionHandler TestProtocol::pseudo_http
so its activities are outside of the request handling phase.
Note that this logic has been broken, for a long time;
2.4.1>
Order Deny,Allow
Allow from @servername@
Where @servername@ is a hostname, this module defect was
identified in version 2.4.20 when we began using the per-req
hostname in comparison (based on r->useragent_addr, which
is obviously is null during part of the read_request phase).
But this module using mod_access_compat during the connection
phase has been broken for much longer, since Allow from {ip-addr}
would already have failed since 2.4.1 was released, due to the
same null r->useragent_addr.
Effectively, mod_access_compat.c never supported per-connection
IP addresses since it was added. The fact that it supported
per-connection hostname comparison was a quirk, and that the
pseudo_http tests only looked at hostname and not ip comparisons
was an oversight.
But the module will fail in other manners if attempting to use
http request_rec processing since that record is never fleshed
out with the proper read/post_read request hook phases.
My thought is to simply decouple access_compat from this
module test... opinions?
See also; https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820824;msg=5
On Thu, Apr 14, 2016 at 11:55 AM, William A Rowe Jr
wrote:
> We can be more vigilant about unexpectedly null values, however...
>
> how are you running request processing in the connection callback
> of mod_perl? That makes no sense, and probably signals a deeper
> logic error.
>
> The access checker is configured per-dir, so until the request rec
> is completely initialized during read_request, this doesn't make
> much sense to me (full backtrace .. including frames #6-#10, for
> those who are curious...)
>
> Either the callback list registered for modperl_callback_connection,
> or the Perl_runops_standard, or the Perl_pp_entersub invoking the
> run_access_checker hook seem the most suspect here.
>
> #0 apr_getnameinfo (hostname=hostname@entry=0x7fd4461ee368, sockaddr=0x0,
> flags=flags@entry=0)
> at /tmp/buildd/apr-1.5.2/network_io/unix/sockaddr.c:663
> #1 0x55feaf0f513a in ap_get_useragent_host (r=r@entry=0x7fd4461ee0a0,
> type=type@entry=3,
> str_is_ip=str_is_ip@entry=0x7fd44740c9c4) at core.c:990
> #2 0x7fd4519d7212 in find_allowdeny (r=r@entry=0x7fd4461ee0a0,
> method=method@entry=0, a=,
> a=) at mod_access_compat.c:279
> #3 0x7fd4519d74b2 in check_dir_access (r=0x7fd4461ee0a0) at
> mod_access_compat.c:332
> #4 0x55feaf0f8f30 in ap_run_access_checker (r=r@entry=0x7fd4461ee0a0) at
> request.c:87
> #5 0x7fd448a6f7dd in XS_Apache2__RequestRec_run_access_checker
> (my_perl=0x55feb2964a20, cv=)
> at HookRun.c:235
> #6 0x7fd44f5f7e6a in Perl_pp_entersub () from
> /usr/lib/x86_64-linux-gnu/libperl.so.5.22
> #7 0x7fd44f5f0ca6 in Perl_runops_standard () from
> /usr/lib/x86_64-linux-gnu/libperl.so.5.22
> #8 0x7fd44f575f06 in Perl_call_sv () from
> /usr/lib/x86_64-linux-gnu/libperl.so.5.22
> #9 0x7fd44f91ec28 in modperl_callback
> (my_perl=my_perl@entry=0x55feb2964a20, handler=0x7fd4461f2750,
> p=p@entry=0x7fd4461f2028, r=r@entry=0x0, s=s@entry=0x7fd453ddc628,
> args=0x55feb3beebd0)
> at modperl_callback.c:100
> #10 0x7fd44f91f576 in modperl_callback_run_handlers (idx=0,
> type=type@entry=1, r=r@entry=0x0,
> c=, s=0x7fd453ddc628, pconf=pconf@entry=0x0, plog=0x0,
> ptemp=0x0, run_mode=MP_HOOK_RUN_FIRST)
> at modperl_callback.c:236
> #11 0x7fd44f91fd4f in modperl_callback_connection (idx=,
> c=,
> run_mode=) at modperl_callback.c:359
> #12 0x55feaf10cdf0 in ap_run_process_connection
> (c=c@entry=0x7fd4461f22b8) at connection.c:42
> #13 0x55feaf10d340 in ap_process_connection (c=c@entry=0x7fd4461f22b8,
> csd=csd@entry=0x7fd4461f20a0)
> at connection.c:226
> #14 0x7fd4523f3e6b in process_socket (bucket_alloc=0x7fd4461f0028,
> my_thread_num=1, my_child_num=0,
> sock=0x7fd4461f20a0, p=0x7fd4461f2028, thd=0x7fd453eb27a0) at worker.c:631
> #15 worker_thread (thd=0x7fd453eb27a0, dummy=) at worker.c:990
> #16 0x7fd453418454 in start_thread (arg=0x7fd44740d700) at
> pthread_create.c:334
> #17 0x7fd453155ecd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
>
>
> Before we chase down a potential non-defect in httpd, any thoughts
> on the underlying modperl or script logic?
>
>
> On Thu, Apr 14, 2016 at 1:44 AM, Takashi Sato wrote:
>
>> r->useragent_addr is assigned on ap_read_request (http_core.c),
>> called from ap_process_http_(async_)connection
>> called from process_connection hook (APR_HOOK_REALLY_LAST).
>>
>> The SEGV occured on process_connection hook, maybe before
>> ap_process_http_(async_)connecti
Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/ssl_engine_kernel.c server/con
On 2016-04-13 22:04, Yann Ylavic wrote: > On Thu, Feb 25, 2016 at 11:27 AM, wrote: >> Author: icing >> Date: Thu Feb 25 10:27:27 2016 >> New Revision: 1732275 >> >> URL: http://svn.apache.org/viewvc?rev=1732275&view=rev >> Log: >> merging pre_close_connection hook, prep_lingering_close and >> ap_update_child() additions from trunk >> >> Modified: > [] >> httpd/httpd/branches/2.4.x/server/scoreboard.c > [] ... > > This seems to have changed mod_status' output. > Prior to this change the previous values were displayed until the > worker was reused for another connection/request, now it is reset as > soon as the worker is idle (i.e. each connection/request end). > > I prefer the new behaviour, but there are some concerns on users@... > I got this morning a request from a FreeBSD user to back-port r1739008, and also already the feedback that scoreboard is usable again with the patch. Since I cannot find a reference in branches/2.4.x/STATUS for r1739008 I told the user I will ask additional upstream devs if they think it is OK to include the patch until a new httpd release will be shipped. Do you see any potential issues if r1739008 will be back-ported, or is there additional work for a better patch in work? -- olli
Re: New segfault with 2.4.20 with mod_perl
We can be more vigilant about unexpectedly null values, however... how are you running request processing in the connection callback of mod_perl? That makes no sense, and probably signals a deeper logic error. The access checker is configured per-dir, so until the request rec is completely initialized during read_request, this doesn't make much sense to me (full backtrace .. including frames #6-#10, for those who are curious...) Either the callback list registered for modperl_callback_connection, or the Perl_runops_standard, or the Perl_pp_entersub invoking the run_access_checker hook seem the most suspect here. #0 apr_getnameinfo (hostname=hostname@entry=0x7fd4461ee368, sockaddr=0x0, flags=flags@entry=0) at /tmp/buildd/apr-1.5.2/network_io/unix/sockaddr.c:663 #1 0x55feaf0f513a in ap_get_useragent_host (r=r@entry=0x7fd4461ee0a0, type=type@entry=3, str_is_ip=str_is_ip@entry=0x7fd44740c9c4) at core.c:990 #2 0x7fd4519d7212 in find_allowdeny (r=r@entry=0x7fd4461ee0a0, method=method@entry=0, a=, a=) at mod_access_compat.c:279 #3 0x7fd4519d74b2 in check_dir_access (r=0x7fd4461ee0a0) at mod_access_compat.c:332 #4 0x55feaf0f8f30 in ap_run_access_checker (r=r@entry=0x7fd4461ee0a0) at request.c:87 #5 0x7fd448a6f7dd in XS_Apache2__RequestRec_run_access_checker (my_perl=0x55feb2964a20, cv=) at HookRun.c:235 #6 0x7fd44f5f7e6a in Perl_pp_entersub () from /usr/lib/x86_64-linux-gnu/libperl.so.5.22 #7 0x7fd44f5f0ca6 in Perl_runops_standard () from /usr/lib/x86_64-linux-gnu/libperl.so.5.22 #8 0x7fd44f575f06 in Perl_call_sv () from /usr/lib/x86_64-linux-gnu/libperl.so.5.22 #9 0x7fd44f91ec28 in modperl_callback (my_perl=my_perl@entry=0x55feb2964a20, handler=0x7fd4461f2750, p=p@entry=0x7fd4461f2028, r=r@entry=0x0, s=s@entry=0x7fd453ddc628, args=0x55feb3beebd0) at modperl_callback.c:100 #10 0x7fd44f91f576 in modperl_callback_run_handlers (idx=0, type=type@entry=1, r=r@entry=0x0, c=, s=0x7fd453ddc628, pconf=pconf@entry=0x0, plog=0x0, ptemp=0x0, run_mode=MP_HOOK_RUN_FIRST) at modperl_callback.c:236 #11 0x7fd44f91fd4f in modperl_callback_connection (idx=, c=, run_mode=) at modperl_callback.c:359 #12 0x55feaf10cdf0 in ap_run_process_connection (c=c@entry=0x7fd4461f22b8) at connection.c:42 #13 0x55feaf10d340 in ap_process_connection (c=c@entry=0x7fd4461f22b8, csd=csd@entry=0x7fd4461f20a0) at connection.c:226 #14 0x7fd4523f3e6b in process_socket (bucket_alloc=0x7fd4461f0028, my_thread_num=1, my_child_num=0, sock=0x7fd4461f20a0, p=0x7fd4461f2028, thd=0x7fd453eb27a0) at worker.c:631 #15 worker_thread (thd=0x7fd453eb27a0, dummy=) at worker.c:990 #16 0x7fd453418454 in start_thread (arg=0x7fd44740d700) at pthread_create.c:334 #17 0x7fd453155ecd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 Before we chase down a potential non-defect in httpd, any thoughts on the underlying modperl or script logic? On Thu, Apr 14, 2016 at 1:44 AM, Takashi Sato wrote: > r->useragent_addr is assigned on ap_read_request (http_core.c), > called from ap_process_http_(async_)connection > called from process_connection hook (APR_HOOK_REALLY_LAST). > > The SEGV occured on process_connection hook, maybe before > ap_process_http_(async_)connection, > > #11 0x7fd44f91fd4f in modperl_callback_connection (idx= out>, c=, > run_mode=) at modperl_callback.c:359 > #12 0x55feaf10cdf0 in ap_run_process_connection > (c=c@entry=0x7fd4461f22b8) at connection.c:42 > #13 0x55feaf10d340 in ap_process_connection > (c=c@entry=0x7fd4461f22b8, csd=csd@entry=0x7fd4461f20a0) > at connection.c:226 > > so r->useragent_addr had not been assigned any value. >
Re: Unintended Side Effects (Was: Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/
> Am 14.04.2016 um 15:09 schrieb Jim Jagielski : > > We should always ensure that when we fix something, or > when we add something, we don't change the current behavior > of httpd in an unintended way. This rev is a great example of > that. The clearing of those scoreboard slots is completely > unrelated to the actual change itself. Yes, I meant well, but should have known better or at least asked for confirmation on the list. I have no excuse. Scoreboard is one of the areas where unwanted changes can slip through more easily than in other places, as there are no test cases covering this text output. Since its functionality is vital for a lot of users, maybe we should invest into having a formatted, parseable variant of its output. That we could use in tests. > Of course, the "blame" is shared because even if the patch > does change something, it is up to all of us to review it, > not just by running and testing the code, but actually > *inspecting* the patch for what it does. > > The very fact that this change made it all the way > thru the backport process to end up in 2.4.x is a reminder > that we all need to stay vigilant. Anything that is not covered by test code will break sooner or later unless the original, sole author becomes immortal. That is not meant as an excuse, just an observation. -Stefan
Unintended Side Effects (Was: Re: svn commit: r1732275 - in /httpd/httpd/branches/2.4.x: ./ include/ap_mmn.h include/http_connection.h include/scoreboard.h modules/generators/mod_status.c modules/ssl/
We should always ensure that when we fix something, or when we add something, we don't change the current behavior of httpd in an unintended way. This rev is a great example of that. The clearing of those scoreboard slots is completely unrelated to the actual change itself. Of course, the "blame" is shared because even if the patch does change something, it is up to all of us to review it, not just by running and testing the code, but actually *inspecting* the patch for what it does. The very fact that this change made it all the way thru the backport process to end up in 2.4.x is a reminder that we all need to stay vigilant.
Re: Allow SSLProxy* config in context?
Am 14.04.2016 um 02:57 schrieb Daniel Ruggeri:
On 4/13/2016 2:22 PM, Rainer Jung wrote:
We could pass the worker name from mod_proxy to mod_ssl via a
connection note, similar to currently already passing the SNI name via
the connection note proxy-request-hostname.
+1 on the connection note idea, but see below about having to inform the
callback function about too much information
On 4/13/2016 10:04 AM, Graham Leggett wrote:
What I had in mind was a syntax where the certs were named, for example:
SSLProxyCertificate foo /path/to/foo.cert
Followed somewhere else by:
SSLProxyEnable foo
On one hand, this is an attractive idea because we have already
conditioned users/administrators to think about naming things via config
as you can do with authn providers and aliases.
On another hand, there are two gotchas I see. Giving something a name in
the configuration does not necessarily mean anything to the file or
certificate store because they aren't married in the same way that authn
providers and their aliases are. That is, if a different team/individual
manages the certificate store than the server configuration, the two
would have to keep in sync with additions or removals. This can be a
PITA if you have a short certificate lifecycle and have to renew often.
PLUS, there are cases where there may be many certificates in the file.
Combine that with the previous point and you could have a nightmare.
Perhaps in addition to or instead of aliases via config, something more
"dynamic" could be used where a DN, CN or some other attribute about the
cert that was parsed during startup becomes the representation?
...
#As today
SSLProxyMachineCertificateFile /path/to/file
#Plus new stuff
SSLProxyMachineCertificateCN myIdentity
#or
SSLProxyMachineCertificateDN /C=US/ST=Missouri/L=St.
Louis/O=BitNebula/CN=DanielRuggeri.bitnebula.com
This could be a really clean solution because the proxy can just pass
this name or DN to the ssl module in a note and the ssl module could
make the selection of certs from that data.
Stashing this (completely untested and thrown together) code around line
1778 in modules/ssl/ssl_engine_kernel.c could be all that's needed in
mod_ssl
char *preferred_name;
..
if (preferred_name) {
for (i = 0; i < sk_X509_INFO_num(certs); i++) {
X509_NAME *name;
info = sk_X509_INFO_value(certs, i);
name = X509_get_subject_name(inf->x509);
for(j = 0; j < X509_NAME_entry_count(name); j++) {
const char *this_name = //something
if (strcmp(preferred_name, this_name) == 0) {
modssl_proxy_info_log(c, info, APLOGNO(02279)
"found acceptable cert");
modssl_set_cert_info(info, x509, pkey);
return TRUE;
}
}
}
ap_log_error(APLOG_MARK, APLOG_WARNING, 0, s, APLOGNO(00)
SSLPROXY_CERT_CB_LOG_FMT
"server configuration requested certificate name %s"
"but it was not found in the certificate store",
preferred_name);
}
I haven't cracked open the proxy code to see what it would take to
collapse this down to a per proxy/worker/etc, but it doesn't seem like
terrible endeavor.
Yeah that's looks very feasible and would allow the actual use case. I
was wondering whether more SSLProxy* directives would benefit from a per
backend config possibility. And as a variation of Graham's suggestion, I
was thinking about (example)
http://mycluster>
SSLProxyMachineCertificateFile /path/to/file
... proxy settings ...
In the vhost config we would not then have an array or hash of ssl proxy
ocnfig setting. The key to the hash would be "http://mycluster"; in the
example above. And mod_proxy would forward "http://mycluster"; to mod_ssl
via connection notes to allow mod_ssl to select the right proxy ssl config.
So there would be no additional name "foo", instead we would reuse the
name of the worker from the surrounding proxy block. The selection of
the certificate would still happen based on the CA, but it would be
selected from the right store (the one associated with "http://mycluster";).
I see your point about having to administer more ssl files outside of
the config, ie. client cert files or directories per backend. I would
hope that those would be manageable by choosing some naming convention
for the file names.
Your idea to allow selecting a client cert based on CN or DN sounds
attractive to me as well. But since it wouldn't help with other per
backend settings (like different Verify settings) we might even think
about combining both. As long as your only problem would be client
certs, you can select via CN or DN and keep the config vhost global, but
once you need more adjustments per backend, you would start to configure
SSLProxy* i
Re: Allow SSLProxy* config in context?
On Wed, Apr 13, 2016 at 7:49 PM, Rainer Jung wrote: > Am 13.04.2016 um 17:04 schrieb Graham Leggett: >> >> The catch is that mod_ssl forces us to declare SSL certs and keys server >> wide, not per directory, loaded and parsed at startup. We however want to >> specify certs per directory. > > Per directory or better in some new way per proxy backend (or proxy worker, > proxy balancer). IIUC, the block is a per_dir context already, which can/could accept any directive provided their ap_check_cmd_context() allows it (we may need to declare a new PROXY_CONF). So how about making per_dir SSLProxy* directives, restricted to server and context? This would let the loading (and validation) work like currently, mod_ssl could still do its standalone post_config stuff (server AND per_dir wise). At runtime, proxy_walk() would still do the merging (there may be same SSLProxy* in both and which need merging, but that would be a single one since we restrict those directives to server and context). Finally, if ssl_proxy_enable[_ex]() is given r->per_dir_config, it could initialize the backend connection with the right context. Wouldn't that work without so many changes?
