Re: timeout queues in event mpm

[Jim Jagielski]

Yeah, I saw your tweet… This is really cool and I never bothered
to check raw req/per/sec stuff for my acna11 talk because,
as it was said in the trafficserver talk, rps is ridiculous
anyway ;)

Would love to see this folded into trunk and even 2.4.0
if we have the time and inclination. When I get back to the
office (still in SF), I hope to commit some Simple patches
that I've been playing around with.

On Nov 13, 2011, at 4:24 PM, Paul Querna wrote:

> I've created a branch event-performance with this change and several
> other changes:
> 
> 
> I did some basic benchmarking to validate it, though if anyone has a
> real test lab setup that can throw huge traffic numbers at it that
> would be very helpful.
> 
> For the "It works" default index page, I got the following:
> 
> event mpm trunk: 15210.07 req/second
> event mpm performance branch: 15775.42 req/second (~4%)
> nginx 0.7.65-1ubuntu2: 12070.35
> 
> Event MPM was using a 100% default install configuration, nginx was
> using Ubuntu 10.04.3 LTS default configuration, baring replacement of
> the index.html to match Apache's.  I included nginx just to make sure
> the we were in the same ballparks, I imagine a well tuned nginx config
> can go much higher, but the same is true of Apache too :)
> 
> As an aside, In either case, using apachebench for this is kinda..
> pointless, it's maxing itself out on a single core, and the machine
> is... idle.  I'd be very happy if someone wrote a new benchmarking
> tool and/or revamped flood. sigh.
> 
> Will try to do some more profiling and then get this into trunk in the
> next day or two.
> 
> On Fri, Nov 11, 2011 at 8:07 PM, Paul Querna  wrote:
>> hi,
>> 
>> After r1201149, we now lock for lots of things, where in an ideal
>> case, we shouldn't need it.
>> 
>> I'm toying around with ideas on how to eliminate the need for a mutex at all.
>> 
>> My current 'best' idea I think:
>> 
>> 1) Create a new struct, ap_pollset_operation_and_timeout_info_t, which
>> contains a what pollset operation to do (Add, Remove, Flags),
>> timeout_type, the timeout value, and a pointer to the conn_rec.
>> 
>> 2) Implement a single-reader single writer circular ring buffer of
>> these structures. (Using 2x uint16_t head/end offsets stuffed into a
>> single uint32_t so we can make it portability atomic using
>> apr_atomic_cas32)
>> 
>> 3) Allocate this ring buffer per-worker.
>> 
>> 4) Have the single Event thread de-queue operations from all the worker 
>> threads.
>> 
>> This would remove the 4 or 5 separate timeout queues we have
>> developed, and their associated mutex, and basically move all of the
>> apr_pollset operations to the single main thread.
>> 
>> Without modification to the event mpm, it would potentially cause some
>> issues as the event thread isn't always waking up that often, but I
>> think we can figure out a way to do this without too much pain (either
>> a pipe trigger to wake up when in 'slow mode', or just lower the
>> default timeout on the pollset form 500ms to like 5ms).
>> 
>> Thoughts?
>> 
>> Thanks,
>> 
>> Paul
>> 
> 

Re: timeout queues in event mpm

[Paul Querna]

I've created a branch event-performance with this change and several
other changes:
 

I did some basic benchmarking to validate it, though if anyone has a
real test lab setup that can throw huge traffic numbers at it that
would be very helpful.

For the "It works" default index page, I got the following:

event mpm trunk: 15210.07 req/second
event mpm performance branch: 15775.42 req/second (~4%)
nginx 0.7.65-1ubuntu2: 12070.35

Event MPM was using a 100% default install configuration, nginx was
using Ubuntu 10.04.3 LTS default configuration, baring replacement of
the index.html to match Apache's.  I included nginx just to make sure
the we were in the same ballparks, I imagine a well tuned nginx config
can go much higher, but the same is true of Apache too :)

As an aside, In either case, using apachebench for this is kinda..
pointless, it's maxing itself out on a single core, and the machine
is... idle.  I'd be very happy if someone wrote a new benchmarking
tool and/or revamped flood. sigh.

Will try to do some more profiling and then get this into trunk in the
next day or two.

On Fri, Nov 11, 2011 at 8:07 PM, Paul Querna  wrote:
> hi,
>
> After r1201149, we now lock for lots of things, where in an ideal
> case, we shouldn't need it.
>
> I'm toying around with ideas on how to eliminate the need for a mutex at all.
>
> My current 'best' idea I think:
>
> 1) Create a new struct, ap_pollset_operation_and_timeout_info_t, which
> contains a what pollset operation to do (Add, Remove, Flags),
> timeout_type, the timeout value, and a pointer to the conn_rec.
>
> 2) Implement a single-reader single writer circular ring buffer of
> these structures. (Using 2x uint16_t head/end offsets stuffed into a
> single uint32_t so we can make it portability atomic using
> apr_atomic_cas32)
>
> 3) Allocate this ring buffer per-worker.
>
> 4) Have the single Event thread de-queue operations from all the worker 
> threads.
>
> This would remove the 4 or 5 separate timeout queues we have
> developed, and their associated mutex, and basically move all of the
> apr_pollset operations to the single main thread.
>
> Without modification to the event mpm, it would potentially cause some
> issues as the event thread isn't always waking up that often, but I
> think we can figure out a way to do this without too much pain (either
> a pipe trigger to wake up when in 'slow mode', or just lower the
> default timeout on the pollset form 500ms to like 5ms).
>
> Thoughts?
>
> Thanks,
>
> Paul
>

Re: [Discuss] [VOTE] Formal deprecation of 2.0.x branch

[Jeff Trawick]

On Sat, Nov 12, 2011 at 3:38 PM, Noel Butler  wrote:
> On Sat, 2011-11-12 at 07:00 -0600, William A. Rowe Jr. wrote:
>
> On 11/11/2011 3:12 PM, Paul Querna wrote:
>>
>> I don't see why we would continue to support 2.0.x for longer than
>> Redhat's already long support cycles.
>
> I don't see why we would tie this to RedHat's schedule or any other
> vendor, including my own.
>
>
> +1
> Apache (like most daemons), move forward, it's tuff luck if certain distros
> (which RedHat is not the only popular one), that insists on continuing to
> live in the dark ages.

Businesses are generally happy to pay to keep certain environments in
the (mostly) dark ages.  Whether for proprietary software or
vendor-supported open source, there's a lot of value in using a stable
stack which is changed only for critical fixes, in order to avoid
dealing with behavior changes, the inevitable regressions that come
from wide open releases/branches, etc..  Businesses want to put most
of their efforts into new projects (which will use current stacks) and
minimize excitement (and resulting efforts) on old projects that
aren't yet slated for replacement or at least bringing up to
¤tYear.

When we put a branch out to pasture the obvious positive is the strong
message to those with leeway to invest in an upgrade and retest
("recertification") their applications but a drawback is the
disappearance of a place to share efforts among those who still must
maintain the branch, whether for their [employer's] own projects or
for their customers.

[RESULT] Re: [VOTE] Release 2.3.15-beta as beta

[Jim Jagielski]

The vote is CLOSED and passes with many more than 3 +1
(binding) votes…

Thx to all…

I will start the move to release during the day with
hopes that we can announce tomorrow (Monday)!!

w00t!

On Nov 9, 2011, at 6:24 AM, Jim Jagielski wrote:

> The 2.3.15-beta (prerelease) tarballs are available for download at test:
> 
>   http://httpd.apache.org/dev/dist/
> 
> I'm calling a VOTE on releasing these as 2.3.15-beta BETA and,
> with luck, this will be our last beta and the next release in ~2weeks
> or less will be 2.4.0 GA!!
> 
> Vote will last the normal 72 hours...
> 

Re: [VOTE] Formal deprecation of 2.0.x branch

[Eric Covener]

On Fri, Nov 11, 2011 at 11:49 AM, William A. Rowe Jr.
 wrote:
> Stealing a plan executed by Colm for 1.3, I'd like to propose that
> we set a two week window following committers' return-from-ApacheCon
> to execute any backports "of general interest" and apply important
> fixes/backports to pregsub allocation and non-absolute uri parsing.
>

+1

Re: [VOTE] Release 2.3.15-beta as beta

[Steffen]

Forgot:
Build with apr-1.4.5 apr-util-1.3.12 apr-iconv-1.2.1 pcre 8.20 lua 5.1 
libxml2 2.7.8 openssl-1.0.0e zlib-1.2.5


Stefan, you wrote:
- convince some more people to install it on their production servers
(Steffen?)

I like to do, but the SSL/Rewrite issue below is blocking here to give it a 
try.



Steffen


-Original Message- 
From: Steffen
Sent: Saturday, November 12, 2011 4:26 PM Newsgroups: 
gmane.comp.apache.devel

To: dev@httpd.apache.org
Subject: Re: [VOTE] Release 2.3.15-beta as beta

Building fine on Windows, except mod_lua is complaining that it cannot fine
mod_ssl.h, just copied it and all fine.

Still the issue:
When run in DOS box, not shutting down when closing window, as service no
problem.

A real problematic one is:

When running still issues with SSL, pages and/or image not displayed, is
random. Some errors from the browser:

Unable to make a secure connection to the server. This may be a problem with
the server, or it may be requiring a client authentication certificate that
you don't have.
Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error.

The webpage at https://www.land10mail.com/ might be temporarily down or it
may have moved permanently to a new web address.
Error 15 (net::ERR_SOCKET_NOT_CONNECTED): Unknown error

With 2.2.21 and the exact same config, no problems.

The config is:

For SSL, running a Apache 443 only in front of a Apache 80. Using signed
certificate.

It is a minimal config with a commonly used rewrite:

Listen 443
SSLEngine on
DocumentRoot f:/web/unknown
RewriteEngine on
RewriteRule /(.*) http://%{HTTP_HOST}/$1 [P,L]


In the log no clue, only
[ssl:info] [pid 6836:tid 2588] (70014)End of file found: [client
85.223.52.177:38857] SSL input filter read failed.
But that I see also with 2.2.21

Looks like more errors when I have "AcceptFilter https none" instead of
leaving this out.




Steffen


-Original Message- 
From: Jim Jagielski

Sent: Wednesday, November 09, 2011 3:24 PM Newsgroups:
gmane.comp.apache.devel
To: dev@httpd.apache.org
Subject: [VOTE] Release 2.3.15-beta as beta

The 2.3.15-beta (prerelease) tarballs are available for download at test:

http://httpd.apache.org/dev/dist/

I'm calling a VOTE on releasing these as 2.3.15-beta BETA and,
with luck, this will be our last beta and the next release in ~2weeks
or less will be 2.4.0 GA!!

Vote will last the normal 72 hours...

Re: [VOTE] Release 2.3.15-beta as beta

[Issac Goldstand]

On 11/11/2011 13:28, Stefan Fritsch wrote:
>
> Are the build problems on Windows a blocker?

I think we verbally agreed that they weren't, so unless someone vetoes
that statement on-list, we're good to go.

Having said that, it's still on my personal to-do list (eventually)

  Issac

Re: Improving SSL config

[Kaspar Brand]

On 07.10.2011 07:10, William A. Rowe Jr. wrote:
> Exactly... we should default to a server with a preference for cryptographic
> strength, but I have no objection to offering a commented-out, clearly
> documented 'alternative' configuration favoring performance, provided that
> is clearly labeled as 'not for sensitive data'.

Now that the dust after the "BEAST" bang has settled somewhat (and
it's clear that it needs to / will be fixed on the client side [1][2][3]),
I think it's a good time to revisit the default setting for
SSLCipherSuite - at least for trunk and 2.4.

My proposal is something like the attached patch - thoughts, objections?

Kaspar


[1] https://bugzilla.mozilla.org/show_bug.cgi?id=665814
[2] http://codereview.chromium.org/7621002/
[3] http://my.opera.com/securitygroup/blog/2011/09/28/the-beast-ssl-tls-issue
Index: docs/conf/extra/httpd-ssl.conf.in
===
--- docs/conf/extra/httpd-ssl.conf.in   (revision 1201408)
+++ docs/conf/extra/httpd-ssl.conf.in   (working copy)
@@ -48,12 +48,19 @@
 #   SSL Cipher Suite:
 #   List the ciphers that the client is permitted to negotiate.
 #   See the mod_ssl documentation for a complete list.
-SSLCipherSuite RC4-SHA:AES128-SHA:ALL:!aNULL:!EXP:!LOW:!MD5:!SSLV2:!NULL
+SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5
 
-#   SSL Cipher Honor Order:
-#   On a busy HTTPS server you may want to enable this directive
-#   to force clients to use one of the faster ciphers like RC4-SHA
-#   or AES128-SHA in the order defined by SSLCipherSuite.
+#   Speed-optimized SSL Cipher configuration:
+#   If speed is your main concern (on busy HTTPS servers e.g.),
+#   you might want to force clients to specific, performance-
+#   optimized ciphers. In this case, prepend those ciphers
+#   to the SSLCipherSuite list, and enable SSLHonorCipherOrder.
+#   Caveat: by giving precedence to RC4-SHA and AES128-SHA
+#   (as in the example below), most connections will no longer
+#   have perfect forward secrecy - if the server's key is
+#   compromised, captures of past or future traffic must be
+#   considered compromised, too.
+#SSLCipherSuite RC4-SHA:AES128-SHA:HIGH:MEDIUM:!aNULL:!MD5
 #SSLHonorCipherOrder on 
 
 #   Pass Phrase Dialog:

Re: svn commit: r1200040 - in /httpd/httpd/trunk: CHANGES modules/ssl/mod_ssl.c modules/ssl/ssl_engine_config.c modules/ssl/ssl_engine_init.c modules/ssl/ssl_engine_kernel.c modules/ssl/ssl_private.h

[Kaspar Brand]

On 10.11.2011 00:37, pque...@apache.org wrote:
> Author: pquerna
> Date: Wed Nov  9 23:37:37 2011
> New Revision: 1200040
> 
> URL: http://svn.apache.org/viewvc?rev=1200040&view=rev
> Log:
> Add support for RFC 5077 TLS Session tickets.  This adds two new directives:
> 
> * SSLTicketKeyFile: To store the private information for the encryption of 
> the ticket.
> * SSLTicketKeyDefault To set the default, otherwise the first listed token is 
> used.  This enables key rotation across servers.

It's not completely clear to me how these two directives interact - what
does "first listed token" relate to? Can multiple SSLTicketKeyFile
directives appear within a VirtualHost?

> --- httpd/httpd/trunk/CHANGES [utf-8] (original)
> +++ httpd/httpd/trunk/CHANGES [utf-8] Wed Nov  9 23:37:37 2011
> @@ -1,6 +1,9 @@
>   -*- coding: utf-8 
> -*-
>  Changes with Apache 2.3.16
>  
> +  *) mod_ssl: Add support for RFC 5077 TLS Session tickets.
> + [Paul Querna]

This is somewhat misleading, I think. Session tickets are supported in
mod_ssl as soon as you compile it against OpenSSL 0.9.8f or later (they
default to on in OpenSSL, SSL_OP_NO_TICKET would have to be set
otherwise). What your patch adds, OTOH, is allowing explicit control of
the ticket encryption/decryption keys.

> Modified: httpd/httpd/trunk/modules/ssl/mod_ssl.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/ssl/mod_ssl.c?rev=1200040&r1=1200039&r2=1200040&view=diff
> ==
> --- httpd/httpd/trunk/modules/ssl/mod_ssl.c (original)
> +++ httpd/httpd/trunk/modules/ssl/mod_ssl.c Wed Nov  9 23:37:37 2011
> @@ -79,6 +79,14 @@ static const command_rec ssl_config_cmds
>  SSL_CMD_SRV(FIPS, FLAG,
>  "Enable FIPS-140 mode "
>  "(`on', `off')")
> +#ifdef HAVE_TLSEXT_TICKETS
> +SSL_CMD_SRV(TicketKeyFile, TAKE2,
> +"Key file to use for encrypting and decrypting the client 
> ticket (RFC 5077) "
> +"(keyname '/path/to/file')")

I suggest to add some info about the contents of these files (like "48
random bytes in binary format"). Also, the documentation of this
directive should encourage users to regularly change these keys.

> Modified: httpd/httpd/trunk/modules/ssl/ssl_engine_config.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/ssl/ssl_engine_config.c?rev=1200040&r1=1200039&r2=1200040&view=diff
> ==
> --- httpd/httpd/trunk/modules/ssl/ssl_engine_config.c (original)
> +++ httpd/httpd/trunk/modules/ssl/ssl_engine_config.c Wed Nov  9 23:37:37 2011
> @@ -200,6 +200,12 @@ static SSLSrvConfigRec *ssl_config_serve
>  sc->fips   = UNSET;
>  #endif
>  
> +#ifdef HAVE_TLSEXT_TICKETS
> +sc->default_ticket_name = NULL;
> +sc->default_ticket = NULL;
> +sc->tickets = apr_array_make(p, 4, sizeof(modssl_ticket_t*));

Maybe a stupid question, but I don't (yet) see the reason for using an
array with four elements... could you perhaps shed some more light on this?

> Modified: httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c?rev=1200040&r1=1200039&r2=1200040&view=diff
> ==
> --- httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c (original)
> +++ httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c Wed Nov  9 23:37:37 2011
> @@ -2067,3 +2067,94 @@ static int ssl_find_vhost(void *serverna
>  return 0;
>  }
>  #endif
> +
> +#ifdef HAVE_TLSEXT_TICKETS
> +
> +#ifndef tlsext_tick_md
> +#ifdef OPENSSL_NO_SHA256
> +#define tlsext_tick_md   EVP_sha1
> +#else
> +#define tlsext_tick_md   EVP_sha256
> +#endif
> +#endif

That's something which belongs into ssl_private.h, I think.


As a general comment, I would like to see some guidelines in the
documentation as to when an explicit configuration of TLS session ticket
keys really makes sense - and how to create/maintain the key files, in
this case. For a default standalone setup, people are still better off
with using OpenSSL's default of randomly generated session keys, IMO
(except that the lifetime of these keys might be too long, if httpd is
run for longer periods without any reload/restart).

Kaspar