Re: mod_remoteip and mod_http2 combined

[William A Rowe Jr]

On Sat, Feb 18, 2017 at 4:25 PM, Daniel Ruggeri <drugg...@apache.org> wrote:
> On 2017-02-15 09:07 (-0600), William A Rowe Jr <wr...@rowe-clan.net> wrote:
>> On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen <san...@hoentjen.eu> wrote:
>> >
>> > mod_remote ip has:
>> > /* mod_proxy creates outgoing connections - we don't want those */
>> > if (!remoteip_is_server_port(c->local_addr->port)) {
>> > return DECLINED;
>> > }
>> > I am guessing something similar is needed for h2 connections?
>>
>> I suspect that the mod_remoteip logic is wrong, that it should be guarding
>> against any subordinate connections and examining only explicitly configured
>> ports / origin IPs. the PROXY protocol is not part of the HTTP protocol and
>> incompatible with it, so the trust list logic isn't directly compatible 
>> (this is
>> clearly explained in the PROXY pseudo-RFC.)
>>
>
> Hi, Bill. That is what the module is doing. The original authors wrote it to 
> have a list of virtual hosts it is explicitly enabled for and explicitly 
> disabled for. I added a third list for optional vhosts. In the pre_connection 
> hook, it checks to see if the connection's local_addr (which should normally 
> be the server's IP) is explicitly configured to enable PROXY handling. It 
> then checks to see if the local port is a server port.
>
> Looking at the logs shared, 192.168.122.249:84 is the server IP:Port combo 
> and is also the local IP:Port from mod_h2. If h2 sets the master of this 
> connection, then we could skip the whole ordeal with this patch:
>
> Index: modules/metadata/mod_remoteip.c
> ===
> --- modules/metadata/mod_remoteip.c (revision 1781701)
> +++ modules/metadata/mod_remoteip.c (working copy)
> @@ -862,6 +862,10 @@
>  remoteip_conn_config_t *conn_conf;
>  int optional;
>
> +if (c->master != NULL) {
> +return DECLINED;
> +}
> +
>  conf = ap_get_module_config(ap_server_conf->module_config,
>  _module);
>
> .. but I don't know if that potentially means we are looking at the wrong 
> connection.

That should be close, but need to ensure c->master is initialized for
http as well
where there is no master/subordinate.

If the 'optional' (unwise) feature were removed, the decision to
inject the filter before
the http or h2 filter would be trivial, it would step out of the way
after the first-pass
(and perhaps not need to live on the filter stack at all - if we do a
fixed read against
the core filter - we hopefully know the number of bytes affected early
and can then
do a second read to complete the v1 vs v2 read?) --- all before we are
in a multiple
pipeline state.

If we move to a conn_rec oriented one-shot nothing happens during the request
phase at all.

By looking at the protocol filter stack, we should be able to glean
whether we are
talking to the core filter, or an 'unexpected' non-network filter, right?

Re: [2.2 PATCH] fix HttpProtocolOptions (etc) merging

[William A Rowe Jr]

On Mon, Feb 20, 2017 at 10:16 AM, Joe Orton  wrote:
>
> FYI, since I've seen people thinking seriously about test suites, this
> kind of issue is impossible to test comprehensively with the current
> test framework. We really need to spin up httpd multiple times with
> different configurations, to test different combinations of global to
> vhost merges with default vs non-default options.
>
> (If there is some way to do that with the current test suite, I'd be
> interested to hear anyway.)

The headache is that testing anything more complex than loopback
implies you kick off a remote with -server and then pound on that
remote it from the client/framework.

Perhaps if we spun up parallel instances in the same port-ordered
manner, but those instances were truly distinct configs? (These need
distinct var/ directories/pid files etc, of course.)

Re: svn commit: r1783256 - /httpd/httpd/branches/2.4.x/STATUS

[William A Rowe Jr]

On Sat, Feb 18, 2017 at 4:44 PM, Daniel Ruggeri  wrote:
>
> Hi, Bill;
>I've replied about the pre_connnection situation - hoping someone can
> give the proposed patch a test as I don't have a handy H2 testbed.

Yup! Will review that thread - it's the -1 half (as opposed to a general -0 half
for a 'pause' request while I was trying to get to reviewing the
original commit.)

> On the other comment, can you help me understand what redundant code is
> happening per-request? When manipulating the request, there are only
> four things happening differently:
> 1. A check that we have data stored away from the connection filter
> 2. A check that the connection data has a client IP
> 3. The assignment of the data to the request_rec's structure and logging
> at TRACE1
> 4. If no data was found, a check to see if it was optional and a logging
> statement/return according to that result

AIUI; the directives are all configured per-Server, the PROXY protocol data
is fixed for the lifespan of the Connection.  The PROXY protocol is
significantly
more binding that either x-f-f or even x-remoteip. I'm not even sure where the
'optional' scheme originated; if present when not allowed, that's a probable
abuse pattern, and when not present when honored, that too indicates some
malfunction and traffic shouldn't proceed IMO. I don't know that the optional
list should be shipped, it's far too simple to create a completely insecure
setup that won't raise eyebrows. The PROXY protocol reference spec states
the connection (by origin or destination IP) follows the PROXY protocol, or
it does not.

Beyond that concern, I'm wondering if we shouldn't be using the *original*
design of mod_remoteip, changing the conn_rec client_addr/client_ip (and
null out remote_host/logname) and never alter it between requests.

We can leave a conn pool note behind for the per-req processing, to retrieve
the proxy IP into a req variable if desired doing the rest of the
remoteip request
phase, but the remaining per-req code and processing is near insignificant.

Thoughts?

> This should all be quite straight forward per request... In fact, it's a
> much shorter logical path and less work than having to parse the
> X-Forwarded-For header.

So I was unspooling how we would handle stacked variables.

Any PROXY protocol is the nearest hop; if multiple PROXY protocol header
lines occurred, the closest would be transmitted first, etc.

All local x-remoteip style values would be the next most distant hop; very
similar to the haproxy protocol, it indicates some absolutely trusted edge
router/balancer.

Any x-f-f that occurs would reflect all the next most distant hops. Finally,
any 'Forwarded' header (rfc7239) are the most distant hops. I'm basing
that conclusion on the fact that all 'Forwarded'-aware intermediaries which
construct a 'Forwarded' header would not carry the x-f-f, but concatenate
these as closer than the nearest 'Forwarded'-aware hop. So the presence
of an x-f-f header indicates the presence of a 'Forwarded'-unaware agent
between this incoming connection and the closest 'Forwarded'-aware agent.

I'm not suggesting these two enhancements, PROXY and RFC7239 are
intertwined, we can certainly ship them in different releases, but I was
having problems working out X-F-F vs Forwarded until I was working
through the PROXY logic and came to the conclusion above, and am
looking for others to sanity-check my logic on this.

Re: httpd 2.4.25, mpm_event, ssl: segfaults

[William A Rowe Jr]

On Feb 17, 2017 2:52 PM, "William A Rowe Jr" <wr...@rowe-clan.net> wrote:

On Feb 17, 2017 1:02 PM, "Jacob Champion" <champio...@gmail.com> wrote:

`EnableMMAP on` appears to boost performance for static files, yes, but is
that because of mmap() itself, or because our bucket brigades configure
themselves more optimally in the mmap() code path? Yann's research is
starting to point towards the latter IMO.


This may be as simple as the page manager caching and reusing the
un-cleared page mapping on subsequent hits. You would need to
overwhelmingly vary the page content served to test this theory. But the
same caching wins for libld[l] ... Which doesn't segv during os updates.
Probably due to copy-on-write mechanics.


(With traditional and sendfile, you still have copy once on read - even if
that file is sitting in FS cache.)

Re: httpd 2.4.25, mpm_event, ssl: segfaults

[William A Rowe Jr]

On Feb 17, 2017 1:02 PM, "Jacob Champion"  wrote:

`EnableMMAP on` appears to boost performance for static files, yes, but is
that because of mmap() itself, or because our bucket brigades configure
themselves more optimally in the mmap() code path? Yann's research is
starting to point towards the latter IMO.


This may be as simple as the page manager caching and reusing the
un-cleared page mapping on subsequent hits. You would need to
overwhelmingly vary the page content served to test this theory. But the
same caching wins for libld[l] ... Which doesn't segv during os updates.
Probably due to copy-on-write mechanics.

Re: [2.2 PATCH] fix HttpProtocolOptions (etc) merging

[William A Rowe Jr]

Great catch; +1 to commit to 2.2.x and
http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x-merge-http-strict/
branches.

And thanks for adding the breadcrumb for the next sucker to miss this :-O

On Fri, Feb 17, 2017 at 3:30 AM, Joe Orton  wrote:
> Found during QA of the CVE-2016-8743 patch here.  The merging logic in
> merge_core_server_configs is (confusingly) inverted from 2.2 to 2.4, so
> e.g. HttpProtocolOptions doesn't inherit from global to vhost configs in
> 2.2.32. :(
>
> Index: server/core.c
> ===
> --- server/core.c   (revision 1783354)
> +++ server/core.c   (working copy)
> @@ -546,15 +546,19 @@
> ? virt->merge_trailers
> : base->merge_trailers;
>
> -if (virt->http09_enable != AP_HTTP09_UNSET)
> -conf->http09_enable = virt->http09_enable;
> +if (conf->http09_enable == AP_HTTP09_UNSET)
> +conf->http09_enable = base->http09_enable;
>
> -if (virt->http_conformance != AP_HTTP_CONFORMANCE_UNSET)
> -conf->http_conformance = virt->http_conformance;
> +if (conf->http_conformance == AP_HTTP_CONFORMANCE_UNSET)
> +conf->http_conformance = base->http_conformance;
>
> -if (virt->http_methods != AP_HTTP_METHODS_UNSET)
> -conf->http_methods = virt->http_methods;
> +if (conf->http_methods == AP_HTTP_METHODS_UNSET)
> +conf->http_methods = base->http_methods;
>
> +/* N.B. If you backport things here from 2.4, note that the
> + * merging logic needs to be inverted, since conf is initially a
> + * copy of vertv not basev. */
> +
>  return conf;
>  }
>
>

Re: svn commit: r1783317 - /httpd/httpd/trunk/modules/ssl/ssl_engine_init.c

[William A Rowe Jr]

On Feb 16, 2017 17:33, "Jacob Champion" <champio...@gmail.com> wrote:

On 02/16/2017 03:16 PM, William A Rowe Jr wrote:

> With no docs to that effect, and trying to predict what 1.2.0 might do
> to us, the explicit avoidance seems safer, no?
>

There are docs to that effect for 1.1.0.

https://www.openssl.org/docs/man1.1.0/crypto/EC_GROUP_free.html
https://www.openssl.org/docs/man1.1.0/crypto/EC_KEY_free.html

"If [group, key] is NULL nothing is done."


Is there somewhere it is documented what openssl's NULL-safe policy is?
>

Closest I can find quickly is a recent discussion:
https://github.com/openssl/openssl/issues/2271

>From Rich Salz: "All free functions, like C free will be a no-op for NULL.
No other changes are planned at this time. Having said that, I am only one
member of the team, and I may leave or get out-voted by others. What I can
promise is that we are going to try to avoid semantic changes before 1.2,
which is unlikely to happen this year."

And from Kurt Roeckx: "Calling free() with NULL is expected and documented
behaviour, we actually changed it in all functions to work. It seems very
unlikely that we will change that."


Excellent, I'll back out that change and the needless _internal() case
tomorrow if nobody beats me to it.

Thanks for the citations!

Re: Topic for discussion... 2.4.26

[William A Rowe Jr]

With the passing of OpenSSL 1.0.1, is OpenSSL 1.1.0 on our radar for the
next release?

I'm not clear how that merge branch is intended to be used, I'm don't
understand whether we propose to adopt every feature and API change commit
to modules/ssl/* - and why it has been rebased, unless we intend to svn cp
the resulting tree on top of modules/ssl/.

I'm set up to review it against 1.1.0, if I understood how that branch
would be applied.


On Feb 16, 2017 11:25 AM, "Jim Jagielski"  wrote:

> Would be nice, I think, to start discussion on a T of 2.4.26 and
> to open the doors to who wants to RM. Note, that if *nobody*
> offers to RM, I will... and no matter what, I offer to help
> whoever wishes to RM.
>

Re: SSL_CTX_set_ecdh_auto noop OpenSSL 1.1.0?

[William A Rowe Jr]

On Thu, Feb 16, 2017 at 4:39 PM, Yann Ylavic <ylavic@gmail.com> wrote:
> On Thu, Feb 16, 2017 at 11:33 PM, Yann Ylavic <ylavic@gmail.com> wrote:
>> On Thu, Feb 16, 2017 at 10:52 PM, William A Rowe Jr <wr...@rowe-clan.net> 
>> wrote:
>>> I'm not clear that this was a good usage of the current API...
>>>
>>> In file included from httpd-2.x/modules/ssl/ssl_private.h:90:0,
>>>  from httpd-2.x/modules/ssl/ssl_engine_init.c:29:
>>> httpd-2.x/modules/ssl/ssl_engine_init.c: In function 
>>> ‘ssl_init_server_certs’:
>>> include/openssl/ssl.h:1287:51: warning: statement with no effect
>>> [-Wunused-value]
>>>  # define SSL_CTX_set_ecdh_auto(dummy, onoff)  ((onoff) != 0)
>>>^
>>> httpd-2.x/modules/ssl/ssl_engine_init.c:1328:9: note: in expansion of
>>> macro ‘SSL_CTX_set_ecdh_auto’
>>>  SSL_CTX_set_ecdh_auto(mctx->ssl_ctx, 1);
>>>  ^
>>
>> Looks like OpenSSL missed a cast to void in its macro implementation.
>> I think goal was to still evaluate "onoff", but in this case
>> "((void)((onoff) != 0))" or the usual "do (void)((onoff) != 0); while
>> (0)" would have been more clean/compatible...
>
> Wait, SSL_CTX_set_ecdh_auto() used to return an int so the macro is
> right actually.
> Hmm, picky compiler :)
>
>>
>> I guess we'll have to work around this with our own (void) casting.
>
> That still holds...

It was complaining about the unused 'dummy'. IMO, and perhaps the fact
that (1 != 0) evaluates to a constant expression.

Easier to drop it from the compilation path for 1.1.0+

Re: svn commit: r1783305 - /httpd/httpd/trunk/modules/ssl/ssl_util_ocsp.c

[William A Rowe Jr]

On Thu, Feb 16, 2017 at 4:45 PM, Yann Ylavic  wrote:
> On Thu, Feb 16, 2017 at 10:26 PM,   wrote:
>> Author: wrowe
>> Date: Thu Feb 16 21:26:34 2017
>> New Revision: 1783305
>>
>> URL: http://svn.apache.org/viewvc?rev=1783305=rev
>> Log:
>> Fix OpenSSL 1.1.0 breakage in r1781575; BIO_s_file_internal() is gone.
>
> Shouldn't this commit (and follow ups) be merged in
> branches/2.4.x-openssl-1.1.0-compat ?

Yes; however it isn't clear to me what that branch is; a patch management
branch? or something else?

Also wondering if the follow-up patch is actually required... wouldn't we be
just as safe using BIO_s_file() with older openssl flavors? I found the patch
in the early work on compatibility, but it seems we've dropped the conditional
in the current trunk/ sources.

@@ -357,7 +357,11 @@ static STACK_OF(X509) *modssl_read_ocsp_
 unsigned long err;
 STACK_OF(X509) *other_certs = NULL;

+#if OPENSSL_VERSION_NUMBER < 0x1010L
+if ((bio = BIO_new(BIO_s_file_internal())) == NULL)
+#else
 if ((bio = BIO_new(BIO_s_file())) == NULL)
+#endif
 return NULL;
 if (BIO_read_filename(bio, file) <= 0) {
 BIO_free(bio);

Re: svn commit: r1783317 - /httpd/httpd/trunk/modules/ssl/ssl_engine_init.c

[William A Rowe Jr]

On Thu, Feb 16, 2017 at 4:48 PM, Yann Ylavic  wrote:
> On Thu, Feb 16, 2017 at 11:27 PM,   wrote:
>> Author: wrowe
>> Date: Thu Feb 16 22:27:24 2017
>> New Revision: 1783317
>>
>> URL: http://svn.apache.org/viewvc?rev=1783317=rev
>> Log:
>> Avoid unnecessary code (the deprecation macro wrapper itself emits unused 
>> args
>> warnings) in OpenSSL 1.1.0 and avoid _free()ing NULL references.
>>
>>
>> Modified:
>> httpd/httpd/trunk/modules/ssl/ssl_engine_init.c
> []
>>  #endif
>>  }
>> -EC_KEY_free(eckey);
>> -EC_GROUP_free(ecparams);
>> +#endif
>> +if (eckey)
>> +EC_KEY_free(eckey);
>> +if (ecparams)
>> +EC_GROUP_free(ecparams);
>
> ISTM that these are NULL safe already.

With no docs to that effect, and trying to predict what 1.2.0 might do
to us, the
explicit avoidance seems safer, no?

Is there somewhere it is documented what openssl's NULL-safe policy is?

Re: FYI brotli

[William A Rowe Jr]

Funny you mention it. Nginx had it first anyways, and was (perhaps still
is) using the deprecated API that dies with libbrotli rev 1.0.0 - part of
that delay might have been affording ngnix a chance to adapt. Versioning
their installed library should allow both to be installed at once.

So... providing our users enough info to actually use the module is worth
hardly a nickle? 95c to win an undocumented feature on a bullet list?
Providing no documentation to enable it isn't a failure by httpd? Whatever
indeed.

gzip and brotli, head to head at compression level 5, % cpu load increase
is generally larger than the % bytes saved. brotli becomes much more
interesting at much higher pre-compression values, with a corresponding
savings in cpu due to fewer bytes read/transmitted.




On Feb 16, 2017 13:56, "Jim Jagielski" <j...@jagunet.com> wrote:

Whatever... nginx will have it 1st anyway. And once
again we fail our users by having a nickel holding up
a dollar.

> On Feb 16, 2017, at 2:48 PM, William A Rowe Jr <wr...@rowe-clan.net>
wrote:
>
> On Thu, Feb 16, 2017 at 12:47 PM, Jim Jagielski <j...@jagunet.com> wrote:
>>
>>> On Feb 16, 2017, at 1:15 PM, William A Rowe Jr <wr...@rowe-clan.net>
wrote:
>>>
>>>
>>> I concur with Evgeny Kotkov that an ABI stable dependency is appropriate
>>> before adding this to httpd 2.4.x - so far as I've read none have
suggested
>>> this as an experimental addition to 2.4.
>>
>> I do. We release it w/ the caveat that it is dependent on a
>> library that may change. Until that happens, we have users able
>> to use Brotli. That is a Good Thing IMO.
>
> Let's examine your claim. There is no documentation at all of where to
> find brotli. http://brotli.org/ is now one day old (congrats!!!) - but
not yet
> exactly discoverable, http://lmgtfy.com/?q=brotli+package
>
> The package lives now in Debian stretch and sid. Archlinux packages
> seem out-of-sync. This only exists on Fedora as copr package so far.
> So without telling users where to find it, "users able to use Brotli" is a
> false statement.
>
> OS distributors are waiting for 1.0.0 to package this. What's our rush?
>
>> Re: the docs, seem a minor nit to hold back, esp when you
>> don't even quantify how the docs are a complete mess nor the
>> requirement for an example in docs/conf. What other requirements
>> will you be spinning up?
>
> Missing 1. where to find brotli? 2. working config examples? These
> points might seem counter-intuitive, but without such simple things,
> the code is just dead weight.
>
> I'm investigating the interaction between brotli and deflate so we can
> support both simultaniously before feeding it to our stable users. Halting
> compression for half of the browsers in order to win a small percentage
> gain for the other half of the browsers seems sub-optimal, no?
>
> I'll be glad to discover that the brotli and deflate filters already
coexist
> peacefully without double-compressing.

Re: FYI brotli

[William A Rowe Jr]

On Mon, Jan 16, 2017 at 2:28 PM, Evgeny Kotkov
 wrote:
>
> There is, however, a potential problem with backporting mod_brotli, since
> it relies on the Brotli library 1.0.0, which has not yet been released.
> In other words, if the upstream changes the API or the library layout
> or their pkg-config files after mod_brotli is shipped with httpd 2.4.x, it's
> either going to stop building or working.

... or stop being updated, if the 0.5.x branch is abandoned and the new
library versioned for binary compatibility.

That said, isn't it more effective for brotli to reversion the lib*.so.n.n.n
files to facilitate users of the already compiled ngnix module? The entire
point of versioning is to allow multiple consumers to use one of several
binary builds.

> My impression on this is that mod_brotli is only safe to backport after the
> Brotli authors publish the 1.0.0 version of their library.  But perhaps I am
> missing something?
>
> (Apart from this, I think that Brotli did change the layout of their 
> pkg-config
> files in [https://github.com/google/brotli/commit/fe9f9a91], and it requires
> an update in the filters/config.m4 file; I'll do that.)

They may change the package composition again, IIUC, renaming /usr/bin/bro
to /usr/bin/brotli or similar, due to an established /usr/bin/bro in many linux
distributors' packages. But a build from 1.0.0 trunk today indicates this hasn't
happened yet.

Thanks again for your earlier clarifications.

Re: FYI brotli

[William A Rowe Jr]

On Thu, Feb 16, 2017 at 2:27 PM, Evgeny Kotkov
<evgeny.kot...@visualsvn.com> wrote:
> William A Rowe Jr <wr...@rowe-clan.net> writes:
>
>> My open questions; has this been entirely reviewed in conjunction with h2?
>> Will A-E: br,gzip,deflate axe all others from that list when deciding to
>> enable brotli? (I presume not-yet.) Will gzip filter work where A-E: gzip was
>> given without br, or are we ceasing to encode half of the web if the user
>> elects to serve brotli compression?
>
> Hi William,
>
> Answering some of the questions:
>
> 1) I did test mod_brotli in conjunction with mod_http2 around the time it
>was committed.  As far as I remember, I didn't spot anything unusual
>or any issues.

Cool; would be nice to do something in the perl-framework.

> 2) The brotli and deflate output filters can coexist.
>
>Moreover, mod_brotli was written with a particular use case in mind
>where this module is added to an existing mod_deflate installation,
>and results in sending brotli-encoded data only to the clients that
>advertise they know how to deal with it via "Accept-Encoding: br".
>
>"Accept-Encoding: br, gzip, deflate" is not going to double-encode
>the data, as both mod_brotli and mod_deflate are smart enough to only
>kick in for identity Content-Encoding.  "Accept-Encoding: gzip" is going
>to use only the mod_deflate's filter, and mod_brotli will remove itself
>from the chain, after not finding "br" in the Accept-Encoding.
>
>There are a couple of tests from https://svn.apache.org/r1761716 that
>verify this behavior.  (However, I think that right now there is no
>explicit test for the case of sending "Accept-Encoding: gzip, deflate"
>to a location with both mod_brotli and mod_deflate.)

Indeed - I just expanded it to test this combination and the test passes.
Very reassuring.

I don't think we need to consider the priority choice of the user-agent,
it's on our cpu-intensive server to elect it's idea of best compression.

We could add a test for inverted filter order but I don't consider that
particularly critical.

SSL_CTX_set_ecdh_auto noop OpenSSL 1.1.0?

[William A Rowe Jr]

I'm not clear that this was a good usage of the current API...

In file included from httpd-2.x/modules/ssl/ssl_private.h:90:0,
 from httpd-2.x/modules/ssl/ssl_engine_init.c:29:
httpd-2.x/modules/ssl/ssl_engine_init.c: In function ‘ssl_init_server_certs’:
include/openssl/ssl.h:1287:51: warning: statement with no effect
[-Wunused-value]
 # define SSL_CTX_set_ecdh_auto(dummy, onoff)  ((onoff) != 0)
   ^
httpd-2.x/modules/ssl/ssl_engine_init.c:1328:9: note: in expansion of
macro ‘SSL_CTX_set_ecdh_auto’
 SSL_CTX_set_ecdh_auto(mctx->ssl_ctx, 1);
 ^

This looks like a no-op now in OpenSSL 1.1.0.

/*
 * ...otherwise, enable auto curve selection (OpenSSL 1.0.2 and later)
 * or configure NIST P-256 (required to enable ECDHE for earlier versions)
 */
else {
#if defined(SSL_CTX_set_ecdh_auto)
SSL_CTX_set_ecdh_auto(mctx->ssl_ctx, 1);
#else
eckey = EC_KEY_new_by_curve_name(NID_X9_62_prime256v1);
SSL_CTX_set_tmp_ecdh(mctx->ssl_ctx, eckey);
#endif

Re: FYI brotli

[William A Rowe Jr]

On Thu, Feb 16, 2017 at 12:47 PM, Jim Jagielski <j...@jagunet.com> wrote:
>
>> On Feb 16, 2017, at 1:15 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>>
>>
>> I concur with Evgeny Kotkov that an ABI stable dependency is appropriate
>> before adding this to httpd 2.4.x - so far as I've read none have suggested
>> this as an experimental addition to 2.4.
>
> I do. We release it w/ the caveat that it is dependent on a
> library that may change. Until that happens, we have users able
> to use Brotli. That is a Good Thing IMO.

Let's examine your claim. There is no documentation at all of where to
find brotli. http://brotli.org/ is now one day old (congrats!!!) - but not yet
exactly discoverable, http://lmgtfy.com/?q=brotli+package

The package lives now in Debian stretch and sid. Archlinux packages
seem out-of-sync. This only exists on Fedora as copr package so far.
So without telling users where to find it, "users able to use Brotli" is a
false statement.

OS distributors are waiting for 1.0.0 to package this. What's our rush?

> Re: the docs, seem a minor nit to hold back, esp when you
> don't even quantify how the docs are a complete mess nor the
> requirement for an example in docs/conf. What other requirements
> will you be spinning up?

Missing 1. where to find brotli? 2. working config examples? These
points might seem counter-intuitive, but without such simple things,
the code is just dead weight.

I'm investigating the interaction between brotli and deflate so we can
support both simultaniously before feeding it to our stable users. Halting
compression for half of the browsers in order to win a small percentage
gain for the other half of the browsers seems sub-optimal, no?

I'll be glad to discover that the brotli and deflate filters already coexist
peacefully without double-compressing.

Re: FYI brotli

[William A Rowe Jr]

To close up some loose ends/confusion;

On Mon, Jan 16, 2017 at 6:42 PM, Jacob Champion <champio...@gmail.com> wrote:
> On 01/16/2017 04:06 PM, William A Rowe Jr wrote:
>>
>> Before we push this at users.. is there a concern that brotoli
>> compression has similar dictionary or simply size based vulnerabilities
>> as deflate?
>
> If you mean HTTP compression oracles (BREACH et al), then I would expect
> *any* compression technique to be vulnerable, brotli included.

Looking deeper into this class of vulnerability, one of two things must be true;
the request headers are encoded (al la TLS compression, which has been
dropped from the TLS 1.3 spec), *or* the response body is dynamic and can
be abused to decipher the response contents.

So there is no issue with serving pre-compressed content, ever (since that
is - by definition - static :)  There is no issue compressing on the fly any
static content.  There will be significant risk to some dynamic responses
on some sites.

>> If so, maybe we teach both to step out of the way when SSL encryption
>> filters are in place?
>
> Current guidance to avoid BREACH is still, AFAIK, to avoid situations where
> third-party data is being sent in the same response as first-party secrets.
> I don't think we have a way to know when this is happening; it's up to the
> server admin to disable compression in dangerous situations. (If anyone
> knows of an update to this, please jump in.)
>
> Disabling compression whenever TLS is enabled (assuming I've understood your
> suggestion correctly) is IMO too broad a stroke, and will de-optimize sites
> that have been correctly designed to avoid compression oracle attacks.

Agreed. Note that Google decided against using T-E: br for any http:// traffic
due to frequent proxy misconfiguration. /shrug - Firefox is willing and we
have no reason to explicitly disable it.

I concur with Evgeny Kotkov that an ABI stable dependency is appropriate
before adding this to httpd 2.4.x - so far as I've read none have suggested
this as an experimental addition to 2.4.

Between this and the lack of any example in docs/conf/ and a complete
mess of an example here it's obvious that this proposal is premature;
http://httpd.apache.org/docs/trunk/mod/mod_brotli.html#precompressed
(Looks like copy-paste errors).

My open questions; has this been entirely reviewed in conjunction with h2?
Will A-E: br,gzip,deflate axe all others from that list when deciding to
enable brotli? (I presume not-yet.) Will gzip filter work where A-E: gzip was
given without br, or are we ceasing to encode half of the web if the user
elects to serve brotli compression?

Looking at that precompression example, it seems like mod_brotli can
be patched to avoid double-brotli if the user elected to treat static .br
files as either a content-type or a transfer-encoding. We need some way
for mod_deflate and mod_brotli to avoid stepping on each other's toes,
the toes of any third-party encoding module, and on the administrator's
explicit configuration, IMO, before this would be ready for a stable branch.

Re: mod_remoteip and mod_http2 combined

[William A Rowe Jr]

On Wed, Feb 15, 2017 at 9:02 AM, Sander Hoentjen  wrote:
>
> mod_remote ip has:
> /* mod_proxy creates outgoing connections - we don't want those */
> if (!remoteip_is_server_port(c->local_addr->port)) {
> return DECLINED;
> }
> I am guessing something similar is needed for h2 connections?

I suspect that the mod_remoteip logic is wrong, that it should be guarding
against any subordinate connections and examining only explicitly configured
ports / origin IPs. the PROXY protocol is not part of the HTTP protocol and
incompatible with it, so the trust list logic isn't directly compatible (this is
clearly explained in the PROXY pseudo-RFC.)

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Sun, Jan 22, 2017 at 7:52 PM, Noel Butler  wrote:

> Perhaps the only person who wont bend over and take it up the arse like
> some people here expect, if I have an opinion, i'll voice it
>

Noel, your immediately prior post was an interesting example, although I
fail
to see how that particular example applies. When a project is dysfunctional
in multiple aspects, it is easy to pin the tail on the donkey of your
personally
most aggravating aspect.

This specif post was not welcome, helpful, nor productive.

You may want to press pause on sharing your voice for the time being, until
you find a way to express it in a more useful manner? Shouting doesn't help,
nor do expletives. If you need to be reminded again, our chair will take
care
of executing the unsubscribe function.

Re: rfc7231 - content-md5

[William A Rowe Jr]

On Fri, Jan 20, 2017 at 1:49 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Fri, Jan 20, 2017 at 1:21 PM, Dirk-Willem van Gulik
> <di...@webweaving.org> wrote:
>> RFC 7231 has retired Content-MD5.
>>
>> Fair game to remove it from -trunk - or make it squeek 'debrecated' at WARN 
>> or INFO and retire it at the next minor release ?

+1

> Removing what, precisely? Content-MD5 headers aren't implemented in trunk.

Sorry, I missed a -i arg to grep :)

Yes, the default_handler behavior in trunk/server/core.c can simply be removed,
along with the core.c ContentDigest directive handling. I think it
should also be
removed from modules/cache/mod_cache.c as it is not a valid entity header.

The many unset Content-MD5 actions must remain IMO to guard against our
sharing this upstream or downstream.

The http_core.h core flag content_md5 and values AP_CONTENT_MD5_*
should probably remain until 3.0 to avoid unnecessary API changes. a doxygen
/** @deprecated Unused flag */ against that struct member will help us mop these
up during any 3.0 review.

Re: rfc7231 - content-md5

[William A Rowe Jr]

On Fri, Jan 20, 2017 at 1:21 PM, Dirk-Willem van Gulik
 wrote:
> RFC 7231 has retired Content-MD5.
>
> Fair game to remove it from -trunk - or make it squeek 'debrecated' at WARN 
> or INFO and retire it at the next minor release ?

Removing what, precisely? Content-MD5 headers aren't implemented in trunk.

Note that https://www.rfc-editor.org/rfc/rfc7616.txt still provides
for MD5 hashed
digest auth keys. So removing this altogether will break mod_auth_digest in a
manner that breaks existing user auth.

Re: Alternate versioning proposal: patch line releases

[William A Rowe Jr]

On Fri, Jan 20, 2017 at 9:43 AM, Eric Covener  wrote:
>
> Maybe a [POLL] thread is in order, specifically for the topic of
> enhancements/stability in 2.4 and ignoring aspirations about a new
> versioning system or 3.0.
>
> e.g.
>
> 2.4.x is:
>  [ ] evolving just fine
>  [ ] too unstable due to new features and we need to do something about it.
>  [ ] too unstable due to code/review/test escapes and we need to do
> something about it.

Perhaps too many choices?

One possible poll;

New features/enhancements;
[ ] belong on 2.4.x
[ ] should be released as 2.5.0

An entirely different possible poll;

[Multiple Choice - pick two(?) top issues] Apache HTTP Server is lacking in
[ ] new features and enhancements
[ ] code quality
[ ] code review
[ ] test scripts
[ ] contributors
[ ] documentation
[ ] cooperation

Cross pollinating a poll with multiple a/b decision points (is it good/bad,
should we solve x/y) dilutes the chance to work out root causes and
to work out potential solutions. "Do something about it" is a particularly
silly question, since committers will focus on what they want to focus
on, there is no ASF script for "tell people what to work on."

And of course, a new thread Subject: to whatever poll someone wants
to run. I don't plan to submit a poll, I'm just working from all individuals'
responses to these several threads to determine where we sit on the
whole spectrum of rapid updates vs version stability, to base any formal
proposal and [vote] later on. I particularly don't care where 2.4.x goes,
I'm more interested in agreeing to a process and moving on.

Re: Alternate versioning proposal: patch line releases

[William A Rowe Jr]

On Thu, Jan 19, 2017 at 5:49 PM, Jacob Champion  wrote:
> This is somewhat orthogonal to Bill's current suggestion. It solves a
> different set of problems, more related to the short-term
> features-versus-regressions argument and less related to the long-term ABI
> arguments. Both are important to me, and I agree with many pieces of what
> both Jim and Bill are saying.
>
> (This is a combination of a bunch of different ideas from different people,
> on this list and off, so thank you all for discussing.)
>
> == Proposal ==
>
> 2.4.25.x is on a cadence: it's T'd like clockwork every month. Releases
> from that line still follow the necessary voting procedure. Meanwhile, once
> we decide we have enough new features for 2.4.26, we T that. If 2.4.26
> releases, and we decide we need some fixes, we spin up a 2.4.26.x branch and
> move to a cadence on it.

This is the OpenSSL model which httpd effectively mirrors today, although
their .x is an alpha sequence, and your proposal could be read with either
semantic.

However, OpenSSL changes very little in terms of it's scope, and isn't the
best model for an evolutionary, development driven project like httpd.

1.0.0 and 1.1.0 both refactored much of the API in terms of opacity of internal
data members.

1.0.0, 1.0.1, 1.0.2 were incremental TLS features, but as these were defined
by spec, the scope of those upgrades was clear.

So I'd anticipate 1.1.1 if the API is extended further for new TLS features
without other changes, or I'd anticipate 1.2.0 if the API will be further
refactored (or both.)

But given the amount of recoding required to migrate from 1.0.x -> 1.1.0,
as illustrated by a bunch of patches here for mod_ssl, most modern
developers would have considered that rework a 2.0 scope change.

At least the OpenSSL project accomplishes what you outlined above,
irrespective of their number-alpha'ing schema.

Re: Alternate versioning proposal: patch line releases

[William A Rowe Jr]

On Fri, Jan 20, 2017 at 8:07 AM, Graham Leggett  wrote:
> On 20 Jan 2017, at 2:15 AM, Jacob Champion  wrote:
>
>> Ignore the versioning number then; that's not really the core of my 
>> proposal. The key points I'm making are
>>
>> - introduce the concept of a low-risk release line
>
> We have always had this, the current low-risk release line is called v2.4.x.
>
>> - introduce a stable cadence with as little risk to end users as possible
>
> We have always had this.

Many if not most developers disagree with you, most developers agree that
adding features and enhancements is disruptive. 2.4.x adds features and
enhancements to every release, and is therefore not low-risk, and absolutely
not "as little risk as possible".

C.f. para 4; 
https://en.wikipedia.org/wiki/Software_versioning#Change_significance

Re: Reset out x.minor.z definition of 'minor' at httpd?

[William A Rowe Jr]

On Thu, Jan 19, 2017 at 6:05 PM, Jim Jagielski  wrote:
> Bill wrote:
>
>>I think one of our disconnects with 2.4 -> 2.6 is that in any other 
>>framework, there would be
>> no ABI breakage in 2.6. That breakage would be deferred to and shipped as 
>> 3.0.
>
> Huh? For just one single, simple example, what about APR??

Those are the rules of APR today.

httpd today compiles against either apr-1 or apr-2.

I don't understand your question?

> Are we going to now redefine the standards of semantic versioning??

Maybe this will help illustrate our conflicting perspectives on versioning;

  w.m.n.x

I understand your interpretation as
w == wow factor (major breakage)
m = minor (also major breakage)
n = subversion (new features + enhancements, no breakage)

While I understand versioning as
m = major (major breakage)
n = minor (avoid breakage, new features + enhancements)
x = subversion (feature stable, bug + regression fixes only)

I understand you (and perhaps Graham) are arguing there is no need for x?

I argue there is no need for w.

Are we going to move to a model where we have four part designations?
Or can we move to a model where version-minor represents *frequent*
and less disruptive releases to incorporate enhancements?

Re: Reset out x.minor.z definition of 'minor' at httpd?

[William A Rowe Jr]

On Thu, Jan 19, 2017 at 6:12 PM, David Zuelke  wrote:
> I don't know any framework/language/library out there that handles it that 
> strictly. Nginx, or Ruby, or PHP, or whatever...
>
> From x.y.z to x.y.z+1, retain full compatibility.
>
> From x.y.z to x.y+1.0, keep external API compatibility, break ABI if needed, 
> break internal API if absolutely needed
>
> From x.y.z to x+1.0.0, break anything you want.
>
> One issue IMO is that there are a lot of things in flight for 2.6 and 3.0, 
> and some of these are features, while other are big architectural overhauls. 
> The former are for 2.6, and the latter very clearly belong into 3.0. There's 
> no reason why both can't be worked on concurrently.

That's what I'm proposing... a model where x.y.[0-#] releases remain consistent,
like all the packages you mention above, and unlike httpd.

I liked your highlight of "if [absolutely] needed". That's where our
trunk (2.next)
has spent four years diverging from 2.current, mostly unnecessarily.

The fact that the code *could* be backported to 2.4.x (in your PHP and Ruby
examples, such backports aren't even acceptable) means that it *could* have
stayed consistent on our 2.next trunk.

Re: Reset out x.minor.z definition of 'minor' at httpd?

[William A Rowe Jr]

On Fri, Jan 20, 2017 at 4:04 AM, Graham Leggett <minf...@sharp.fm> wrote:
> On 19 Jan 2017, at 11:43 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
>> I think one of our disconnects with 2.4 -> 2.6 is that in any other
>> framework, there would be no ABI breakage in 2.6. That breakage
>> would be deferred to and shipped as 3.0.
>>
>> The httpd project choose to call 2.minor releases as breaking
>> changes. Due to poor design choices, or frequent refactorings,
>> this essentially shifted our release numbering schema down one
>> digit. So 2.6.0 is not an enhancement release, but a major release
>> in the general understanding of these things. This might be the root
>> of Jim's and my failure to come to any consensus.
>
> I don’t see any relationship between poor design choices / frequent 
> refactorings and a decision made in the late 1990s on a version numbering 
> system.

I do. Remember that the version numbering elected for 2.0 was based on
the direct and recent experience of 1.2.x and 1.3.x. There was no API
or ABI assurance throughout 1.2.x until 1.3.12-1.3.14 time frame
(which became the ABI final 1.3.x representation).  Third party
modules would have to be rebuilt, and sometimes patched, between
subversion releases.

Understanding that context is necessary to understand why 2.0
numbering was adopted. Floating ABIs during 2.{odd} releases, fixed
ABIs during 2.{even} releases balanced the desires for a messy
development phase and a stable maintenance phase. When you look at
early (2.0.0 - 2.0.36) development as an initial {odd} Floating
ABI/API period, it makes sense.


>> Right now, there are a number of gratuitous breaking changes on trunk
>> that change binary ABI. I'm considering a trunk fork to preserve these
>> changes (branches/3.x-future/?) and reverting every change to trunk/
>> which was otherwise a no-op. Namespace, decoration, anything that
>> prevents a user from loading a 2.4.x module in 2.next. And then we
>> can look at what is a valid reason to take us to 3.0.
>
> -1 (veto).

To be clear, procedural decisions can't be vetoed. But specific code
changes can be vetoed.

I can veto and revert each individual commit on trunk which breaks the
API or ABI in an unnecessary manner, pointing at that specific
breakage, and invite the committer to propose the change using the
existing interfaces. Even if that commit dates back soon after the
2011 fork. Where the code is accepted in 2.4.x, I can offer the
author's own 2.4.x backport commit to align their work in an API
stable manner to what is shipping and finally trusted. If it was good
enough to ship in 2.4, there better be an awfully good reason for a
code divergence. In almost every case, it was a sloppy trunk/ commit
and some careful thought applied to how to introduce it into 2.4.x.

And no, you can't then veto such specific vetos. The window to veto
code is before the ASF releases the code.

For forward-ports, you would look awfully silly vetoing a patch
accepted in 2.4.x.

In light of your objection, I won't proceed with a preservation
branch, but take the veto knife directly to trunk.


> As you are well aware, the jump from v2.0 to v2.2, from v2.2 to v2.4, and the 
> jump from v2.4 to v2.6 involves breaking changes, and this is a well 
> established and stable pattern that is well understood by our end users. The 
> “problem” you’re trying to solve does not exist.

There is nothing in httpd which is stable, and 2.4.x certainly hasn't
been well established. Not even 50% of Apache httpd deployments use
2.4.x almost four years later, and 2.4.25 looks so considerably
different than 2.4.1 that it cannot be called a maintenance branch. It
is impossible to identify from "2.4" what point in its evolution is
causing a reported issue without knowing the subversion. A handful of
2.4.x releases walked out the door without some significant regression
- not a proud track record. So this problem which I'm trying to solve
is clearly present.

The second inherent problem you sum up below also certainly exists;


> Arbitrarily reverting the work of others displays a profound lack of respect 
> for those members, committers and contributors to the ASF who have 
> contributed to our codebase. This behaviour discourages collaboration between 
> the community and project, and puts this project at risk.

Not releasing their contributions puts the project at risk of having
contributors walk away from offering further contributions. That was
well established in earlier threads this past month, and originates
from a pattern where trunk/ is not released. My goal is simply a
better user experience trusting they can make subversion upgrades with
no disruption (which has not been possible throughout 2.4.x), also
trusting frequent minor version upgrades to deliver new features, and
treating all s

Re: Reset out x.minor.z definition of 'minor' at httpd?

[William A Rowe Jr]

On Thu, Jan 19, 2017 at 6:46 PM, Eric Covener <cove...@gmail.com> wrote:
> On Thu, Jan 19, 2017 at 4:43 PM, William A Rowe Jr <wr...@rowe-clan.net> 
> wrote:
>> I think one of our disconnects with 2.4 -> 2.6 is that in any other
>> framework, there would be no ABI breakage in 2.6. That breakage
>> would be deferred to and shipped as 3.0.
>>
>> The httpd project choose to call 2.minor releases as breaking
>> changes. Due to poor design choices, or frequent refactorings,
>> this essentially shifted our release numbering schema down one
>> digit. So 2.6.0 is not an enhancement release, but a major release
>> in the general understanding of these things. This might be the root
>> of Jim's and my failure to come to any consensus.
>>
>> Right now, there are a number of gratuitous breaking changes on trunk
>> that change binary ABI. I'm considering a trunk fork to preserve these
>> changes (branches/3.x-future/?) and reverting every change to trunk/
>> which was otherwise a no-op. Namespace, decoration, anything that
>> prevents a user from loading a 2.4.x module in 2.next. And then we
>> can look at what is a valid reason to take us to 3.0.
>
> The  "why" missing here is presumably to allow a 2.6 to be cut from
> trunk. But in that case, why not just fork it from 2.4 w/o a major nor
> magic cookie bump and let 2.4 become the more conservative stream?

Just so we are all in agreement, 2.4 has been neither conservative nor
maintenance in any conventional sense, for the past four years. Instead
we have users relying on the stable 2.2 release which we will discard
midyear, 4 1/2 years after 2.4 had shipped.


> New major/minor downstream releases can jump to 2.6 without much fuss,
> and current ones can try to track closer to current 2.4.x as they age.

What I'm proposing is that 2.6.0, 2.8.0, 2.10.0 (or we get out of odds/evens
at this point) are rapid feature releases, no less often than once a year.
For some arbitrary period, perhaps a year, fixes are applied to 2.prior.n+1
and yes, those can be tracked. Users can pick the .zero release to grab
the new features now, or wait for .1 or .2 to pick up any regression fixes.
But within two years, not four, 2.prior would be abandoned as 2.current
has been stabilizing (no features after .0) and 2.next (new features) is
about to be released.

This is closer to the model at most other ASF projects.


> We can find some new religion about how trunk work if some concise
> policy doc appears that we can all agree on.

That's the next step after concluding these dialogs, to weave together the
desires of the divergent committers' interests.


On Fri, Jan 20, 2017 at 3:09 AM, Stefan Eissing
<stefan.eiss...@greenbytes.de> wrote:
>
> Just some brainstorming:
>
> LTS/Stable/Feature branches
>
> 2.2.x/2.4.x/2.6.xfor now

Note 2.2.x is not LTS. It is EOL July.

> 2.2.x/2.6.x/2.8.xif we think new features in 2.6 are stable and want to 
> add more features
> 2.4.x/2.4.x/2.6.xif we end LTS for 2.2, the new LTS can be a stable branch

So 2.4.x again is implicitly unstable.

> 2.2.x/2.4.x/2.8.xin extreme cases, we could ditch a feature branch and 
> move on.

Note that I'm suggesting *much* more frequent version minor releases.

I'd like to see us get *all* new features in users hands within 12
months of first commit.
Not the current pattern not those few which are backported and
destablizing to the
current release - vs those which are not backported, not tested, and
destablizing to
current vs. trunk repository contents.

I think it obviates the need for {even} {odd} if we move rapidly from
2.5.0-beta to
2.5.1-beta to 2.5.2-GA - branch trunk at 2.5.2-GA or 2.5.1-beta for
all new feature
work toward 2.6.0 and ensure 2.6.0-beta gets shipped in months, not a
half decade.

> - we continue working in trunk
> - backports to LTS/stable branches as now, review then commit
> - backports to feature branches: commit, then review
> - LTS is the support promise, stable/feature can move more freely
> - Feature is completely "experimental", we make no promises
> - Distros can support stable longer than we do, which is basically the model 
> they are doing now for their LTS.
> - people will argue for more than 1 LTS release, but I think that is too much 
> for the project, more something for a commercial undertaking

Note I'm suggesting that 2.current and 2.prior get fixes. We can leave the
hassle of backporting further than that to commercial or other endeavors, there
should be no point in time where the choice between those two is worse than
staying back at 2.ancient.

Note I'm suggesting that feature is 2.5.0-beta, 2.5.1-beta, right up until we
declare 2.5.x-GA. That release no longer gets enhancement/features, but
at that point we already have 2.6.0-beta 

Reset out x.minor.z definition of 'minor' at httpd?

[William A Rowe Jr]

I think one of our disconnects with 2.4 -> 2.6 is that in any other
framework, there would be no ABI breakage in 2.6. That breakage
would be deferred to and shipped as 3.0.

The httpd project choose to call 2.minor releases as breaking
changes. Due to poor design choices, or frequent refactorings,
this essentially shifted our release numbering schema down one
digit. So 2.6.0 is not an enhancement release, but a major release
in the general understanding of these things. This might be the root
of Jim's and my failure to come to any consensus.

Right now, there are a number of gratuitous breaking changes on trunk
that change binary ABI. I'm considering a trunk fork to preserve these
changes (branches/3.x-future/?) and reverting every change to trunk/
which was otherwise a no-op. Namespace, decoration, anything that
prevents a user from loading a 2.4.x module in 2.next. And then we
can look at what is a valid reason to take us to 3.0.

There might be some minor breakage that requires a recompilation,
such as a user module allocating a req_rec. That is solved by setting
a showstopper to the 2.next release to introduce _create() accessors
to every structure belonging to the httpd API.  There would be side
determinations... would we institute such a  change with the release
of 2.5, or 2.6, or 3.0?

Once this is complete, binary breaking ABI changes would live in
their own feature development forks. They would never be folded
into trunk until there was a group consensus to push forward to
version next.0.0 in the immediate future. trunk/ would cease to be
a sandbox of ideas unready for release.

It's likely a couple-week process that I can't start till the end of next
week. But I think the frustration about not being able to ship new
features has a lot to do with our history of breaking API's every time
we ticked up the version minor number.

It might be that we won't end up shipping 2.6 because other good
and necessary binary-breaking-changes occur between now and
in the near future, or that these exist on trunk. But if we defer the
gruntwork of renaming things and whitespace mop-up etc until
a short several-week period prior to the actual major.0.0 release,
then we can keep trunk/ in a much more releasable state for all
the new features.

We may have the impetus for a 3.0 release in the near future, we
may not, but I'm sure the idea that 2.5 is a 'major' ABI-breaking
release is not serving us well.

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Thu, Jan 19, 2017 at 6:29 AM, Jim Jagielski  wrote:
>
> Here's the real issue, as I see it. If there have been "recent
> breakages" it is not due to the release process, but rather
> the *testing* process. That is, not enough people testing
> 2.4-HEAD until we actually get close to a release. The idea
> should be that 2.4-HEAD is ALWAYS in a releasable and "runnable"
> state, it being RTC after all.

The idea should be that  *** TRUNK *** is always in a "run-able" and
releasable state, not withstanding legitimate showstoppers and some
multi-developer coordination in sorting out platform or other quirks
which are interdependent. RTC/CTR is a red herring. It simply flags
whether a couple of reviews occur before commit, not whether that
code deserves further review, or is subject to future veto, or may be
subject to further revision.

trunk/ should not be treated with any less caution than branches/2.4.x/,
that's what feature development branches are for.


From; http://httpd.apache.org/dev/guidelines.html

When to Commit a Change

Ideas must be review-then-commit; patches can be commit-then-review.
With a commit-then-review process, we trust that the developer doing
the commit has a high degree of confidence in the change. Doubtful
changes, new features, and large-scale overhauls need to be discussed
before being committed to a repository. Any change that affects the
semantics of arguments to configurable directives, significantly adds
to the runtime size of the program, or changes the semantics of an
existing API function must receive consensus approval on the mailing
list before being committed.

Each developer is responsible for notifying the mailing list and
adding an action item to STATUS when they have an idea for a new
feature or major change to propose for the product. The distributed
nature of the Apache project requires an advance notice of 48 hours in
order to properly review a major change -- consensus approval of
either the concept or a specific patch is required before the change
can be committed. Note that a member might veto the concept (with an
adequate explanation), but later rescind that veto if a specific patch
satisfies their objections. No advance notice is required to commit
singular bug fixes.

Related changes should be committed as a group, or very closely
together. Half-completed projects should not be committed unless doing
so is necessary to pass the baton to another developer who has agreed
to complete the project in short order. All code changes must be
successfully compiled on the developer's platform before being
committed.

The current source code tree should be capable of complete compilation
at all times. However, it is sometimes impossible for a developer on
one platform to avoid breaking some other platform when a change is
committed, particularly when completing the change requires access to
a special development tool on that other platform. If it is
anticipated that a given change will break some other platform, the
committer must indicate that in the commit log.

The committer is responsible for the quality of any third-party code
or documentation they commit to the repository. All software committed
to the repository must be covered by the Apache LICENSE or contain a
copyright and license that allows redistribution under the same
conditions as the Apache LICENSE.

A committed change must be reversed if it is vetoed by one of the
voting members and the veto conditions cannot be immediately satisfied
by the equivalent of a "bug fix" commit. The veto must be rescinded
before the change can be included in any public release.


From; http://httpd.apache.org/dev/release.html

When do I know if it is a good time to release?

Generally speaking, when some useful changes have been applied since
the last release and there are no showstoppers left to be resolved. It
is our convention to indicate showstoppers in the STATUS file in the
repository. A showstopper entry does not imply that a release cannot
be made -- it is more of an indication of current project consensus
and a reminder of what fixes are on the critical path. Each RM gets to
choose when to cut a release candidate based on the current content of
subversion. The entire PMC gets to decide whether or not that
candidate deserves to be released.

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Thu, Jan 19, 2017 at 2:52 PM, Noel Butler <noel.but...@ausics.net> wrote:
>
> On 20/01/2017 05:54, William A Rowe Jr wrote:
>
>> posts, I don't think you will find a single post where I suggested
>> that there is an issue with the frequency of releases, but please
>> feel free.
>
> I believe that was me :)
>
>> You've put restated the argument again this month that if we
>> don't enhance and add features, we will lose user share to
>> another web server.
>
> Frequent releases set off alarms in system admins minds, frequent releases 
> give the view of unstable/unreliable software, and that is the largest cause 
> of movement to an alternative.

I don't believe anyone has that opinion of nghttp2.

I think Tatsuhiro's release model for nghttp2 has been brilliant.

Monthly, there are new features (1.next.0). If there are regressions
between 1.next.0 releases, these are 1.current.1, 1.current.2 and
so on.

If the code works, he ships it. I'm impressed, and don't have any
negative perception that getting bug fixes out very quickly says
anything negative about nghttp2.

Re: clang-analyzer?

[William A Rowe Jr]

On Thu, Jan 19, 2017 at 1:30 PM, Ruediger Pluem  wrote:
>
> If they are no-ops as you state in 3. how could they introduce regressions?

They are still a text and code change. Cleaning up a cast, for example may
change the alignment differently between various 32 and 64 bit architectures.

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Thu, Jan 19, 2017 at 6:29 AM, Jim Jagielski <j...@jagunet.com> wrote:
>
>> On Jan 18, 2017, at 8:35 PM, Eric Covener <cove...@gmail.com> wrote:
>>
>> On Wed, Jan 18, 2017 at 6:12 PM, William A Rowe Jr <wr...@rowe-clan.net> 
>> wrote:
>>> I'm wondering if there is anyone interested in a regression-fix-only 2.4.26 
>>> that
>>> finally proves to be a workable upgrade for all httpd users?
>>
>> It sounds reasonable to me, but I think it's a bit of an oversell --
>> It's just going to be a little bit of stabilization.
>
> Agreed, but what did you expect? If anyone other than Bill
> would have proposed this, he would have complained that
> having such frequent releases is bad, etc...

Jim, are you really going to start ad hominem attacks again on dev?
I thought Eric asked nicely enough for this to desist.

Don't put words into my mouth. If you want to quote one of my
posts, I don't think you will find a single post where I suggested
that there is an issue with the frequency of releases, but please
feel free.

You'll find posts stating that the next release should contain 'X'
where 'X' was generally a security or regression fix that had not
received sufficient attention to backport.

I used to be of the mind that changing from 2.4 to 2.6 was some
radical change that our users would resist. I've come to the
conclusion that the pace of adoption of 2.0 from 1.3, or 2.2
from 2.0, or now from 2.2 to 2.4... little of this had to do with
the major or minor reversion. Users simply don't update,
except as provided by their environmental provider (some
OS distro or cpanel or other deployment schema.)

If anything, Bill and others have recently argued that version
numbers, even minor, and major, are cheap. We should use
them IMO.

> Here's the real issue, as I see it. If there have been "recent
> breakages" it is not due to the release process, but rather
> the *testing* process. That is, not enough people testing
> 2.4-HEAD until we actually get close to a release. The idea
> should be that 2.4-HEAD is ALWAYS in a releasable and "runnable"
> state, it being RTC after all.

My perception is that many enhancements get rammed into
the maintenance branch in the final days of a candidate
because new features are neat, instead of applying them
weeks ahead of a tag and ferreting out breakage across
the breadth of architectures and deployments that httpd
enjoys.

There are three reviewers. If things are going into a release
they either aren't getting sufficient review by those three
people, or three people are inadequate, or more likely...

Our testing process can never and will never capture 10%
of the various ways different developers and users have
deployed and customized their httpd environment, as is
illustrated in our bug tracker.

But having some approachable httpd.a.o/test/ pages that
get mentioned in our 'reporting bugs' pages might help
encourage users with more novel schemas to propose
some test case patches to our test framework? That
would increase coverage, it will always remain a fraction
of the anticipated and unexpected use cases.

> This is only, clearly, an attempt by Bill to commandeer the
> 2.4 release process, nothing more. Again, anyone is free
> to RM... there is no need to fabricate reasons to do so.
> If Bill wants to do a quick 2.4.26 with the current fixes
> then sure, why not. Especially if it will be "one we think is
> good", and we can pat Bill on the back for a job well done.

But the onus should not be on Bill's Releases to win some
feature-frozen and reduced-breakage/regression-fixing
packages for our users, while users expect Jim's Releases
to offer the newest (and potentially/eventually regression
inducing) features and enhancements.

You've put restated the argument again this month that if we
don't enhance and add features, we will lose user share to
another web server. I've restated the argument again this
month that unnecessary regressions cost us user share.
You restate above that a lack of testing causes regressions,
I am restating that adding features to maintenance releases
causes regressions.

This isn't my effort to commandeer the release process, it
is actually my third distinct proposal in a month to find some
major.minor.subversion schema which would be respected
and honored by the entire PMC. We can have a war of the
releases, that seems to be what you want. I would rather
see new committers and pmc members jump into the role
of RM - one that doesn't require years of dev list history
to anticipate whether the community will accept a release
tag at a particular point in time.

It isn't about Bill and Jim, Jim. It is about a community of
coders, who aren't listening well to the needs of one segment
or another of our users and consumers, and our use the
designated 'maintenance' release as a substitute for our
shipping any new releases.

Re: clang-analyzer?

[William A Rowe Jr]

On Mon, Jan 9, 2017 at 3:48 AM, Graham Leggett  wrote:
> On 08 Jan 2017, at 4:45 AM, Leif Hedstrom  wrote:
>
>> I ran clang-analyzer against the HTTPD master branch, and it found 126 
>> issues. Many of these are benign, but I was curious if the community has any 
>> thoughts on this? With another project, I’ve found that keep static code 
>> analysis to zero issues can really help finding new, serious issues 
>> (basically, we put the tree in failed state if there’s a new static code 
>> analysis issue).
>>
>> The issues are all over the source code, in core and mod_’s alike. It’d be 
>> pretty tedious to file individual tickets for each of them, so curious if 
>> there’s any interest in cleaning this up to start with a clean state? It’d 
>> then be easy to add clang-analyzer to the release process for example.
>
> Adding clang-analyzer to a make target (not a default part of the build) 
> would be a good step, it would make it easy for anyone to run it if they had 
> it available.
>
> The most effective contributions would be patches to fix each one. From 
> experience it is difficult to fix these sort of things without the ability to 
> rerun the analyser to ensure the issue is gone, and every now and again 
> issues uncover things that may take some time to fix. Agreed that getting 
> these things to zero would be a good thing to have.

Fixing those that are bugs is a no-brainer.

Getting these to zero, let's say the 80% that don't represent bugs...
is a challenge.

1. We fix on trunk. Backports of later bug fixes are harder to apply.

2. We fix on trunk and backport to 2.4. Simplifies any later backports.
   But this also introduces unnecessary regressions.

My personal preference...

3. We create a patch to trunk for these issues (in a working branch
   would seem most straightforward, but a git clone and fork could
   also hold this work in progress.) Apply only once the beta to our
   next major.minor appears to be acceptable for GA, and in one final
   beta cycle, introduce these no-op changes, including whatever
   formatting and whitespace cleanup is desired by the group.

Third option also makes backports to the legacy branch harder to
apply, but at this point, fewer backports would be expected, so the
net merge time and hassle should be less than the first option of
committing no-op fixes to trunk as they are encountered.

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Wed, Jan 18, 2017 at 7:35 PM, Eric Covener <cove...@gmail.com> wrote:
> On Wed, Jan 18, 2017 at 6:12 PM, William A Rowe Jr <wr...@rowe-clan.net> 
> wrote:
>> I'm wondering if there is anyone interested in a regression-fix-only 2.4.26 
>> that
>> finally proves to be a workable upgrade for all httpd users?
>
> It sounds reasonable to me, but I think it's a bit of an oversell --
> It's just going to be a little bit of stabilization.
>
> AFAICT so far there is:
>
> One 2.4.25 regression committed:
>
>   *) mod_proxy_{ajp,fcgi}: Fix a possible crash when reusing an established
> backend connection, happening with LogLevel trace2 or higher configured,
> or at any log level with compilers not detected as C99 compliant (e.g.
> MSVC on Windows).  [Yann Ylavic]
>
> One older regression listed as a showstopper:
>
>  *) PR 60576: 2.4.21 broke PHP-FPM with the patch to strip the bogus 
> "proxy://"
> prefix from SCRIPT_FILENAME. We need to revert to the previous behavior
> ASAP to avoid any further hurdles with FCGI implementations while we 
> figure
> this out.
>
> And a fix for an old bug that missed being backported until a bugzilla review:
>
>   *) mod_proxy_fcgi, mod_fcgid: Fix crashes in ap_fcgi_encoded_env_len() when
> modules add empty environment variables to the request. PR60275.
> []
>
> Is there anything else we should list as a showstopper?

That is my underlying question; what else qualifies?

Win32 build fix of mod_status is already in 2.4.x branch two hours following
the 2.4.25 re-tag, so that also is resolved.

Yann's proposal to accept the newly-prohibited obs-fold that we approved into
2.2.32 would also seem to qualify.

So far, I haven't heard from an httpd committer who is actually interested in
our shipping such a release. I'd be happy to tag and roll, but this is more so
a poll whether there is interest in having non-enhancement 2.4.x releases
by the PMC/committer core, or whether there is a desire for a  fork for users
who are not PMC members but are actively interested in pursuing a "stable"
build.

Cheers,

Bill

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Tue, Jan 3, 2017 at 2:18 AM, Graham Leggett <minf...@sharp.fm> wrote:
> On 03 Jan 2017, at 2:11 AM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
>> So I'd like to know, in light of a perpetual chain of (often build and/or 
>> run-time breaking regression) enhancements, if there is support for a 
>> 2.4.24.x release chain during the 3.0 transition? And support for 
>> potentially 3x backports to 2.4.x, 2.4.24.x and 2.2.x, of really serious bug 
>> fixes?
>>
>> It's clear this project doesn't agree, so the question twists to how we 
>> agree to disagree.
>
> Can you clarify the problem you’re trying to solve?

There are multiple recent breakages in httpd. The backport proposal to 2.4
extending mod_dav would have introduced yet another. Other features on
the table such as proxy protocol enhancement, or mod_status extensions
all introduce more and more opportunities to introduce yet another release
with regressions.

I'm wondering if there is anyone interested in a regression-fix-only 2.4.26 that
finally proves to be a workable upgrade for all httpd users? Only bug fixes
which correct defects introduced during 2.4.x would be addressed. Then
skip on to 2.4.27 with new regressions, and repeat the mop-up process in
2.4.28, etc.

Most projects would call these 2.5.0 new features, 2.5.x regression and
bug fixes, 2.6.0 new features, 2.6.x regression and bug fixes etc.

But httpd seems to want to keep a unique numbering schema, because
other numbering conventions weren't invented here.

Is it really unreasonable to expect subsequent releases to not introduce
new errors and become more error-free between enchancement releases?

Re: JSON for mod_status

[William A Rowe Jr]

Really, this is now in the PMC's court. Doug and Aaron designed the BMX
bean structure and module implementation. I'm aware that jfc's crew has
also been a consumer of the module, so it already falls into that multi-vendor,
multi-use case scenario.

I'll leave this to them to advocate for httpd adoption of the work, I was just
the messenger and sometimes-maintainer.

As far as well-defined, the existing 'additonal HTML' structure for mod_status
essentially sucks; there is no way for mod_status to comprehend the data
coming back to it. It simply consists of an HTML text stream. It knows not
one thing about the extended html presented by an arbitrary third party
module.

Moving forwards with your suggestion, it is simple to pass a new 'format'
enum to the modules with the callback, so they know it should be plain
html, modern html, xml, json, or any other representation we want to add.
Modules who don't implement a given value will simply not append their
results in a particular format. So if we want to add a particularly well-
formed xml for xslt representation of status data sometime in the future,
mod_status will not know how to interpret this for other third party mods,
and those that aren't patched for the new 'format' value will simply not
add results.

The bean concept defined to mod_bmx what the data format was. I'm
not sure it was comprehensive enough in terms of arrays of arrays or
other prospective use cases. E.g. could a mod_status_bmx actually
interrogate the bmx providers and offer some legible html status
output today or xml for xslt presentation some day in the future? That's
a good question and I'd like to hear Aaron's or Doug's thoughts.

Bill




On Tue, Jan 17, 2017 at 12:33 PM, Jim Jagielski  wrote:
> It all depends on what Bill decides regarding mod_bmx and if
> it is something we intent to backport to 2.4.x
>
> Still not sure on how to *use* BMX, or how other modules
> "hook in" (right now we have several modules hook into
> mod_status so the "how" is well known and documented), so
> I would require some sort of docs in addition to the actual
> code, of course.
>
>> On Jan 17, 2017, at 12:35 PM, Luca Toscano  wrote:
>>
>>
>>
>> 2016-11-30 18:54 GMT+01:00 Jim Jagielski :
>> I'm thinking about adding JSON support to mod_status...
>> the "plain" version output really stinks and lacks parity
>> w/ the info we provide via HTML, and it would be nice
>> to produce a really easily parseable format.
>>
>> Thoughts...?
>>
>> I know it was extensively discussed, but do we have an agreement about a 
>> plan to add this feature?  :)
>>
>> Luca
>>
>>
>>
>

Re: FYI brotli

[William A Rowe Jr]

Before we push this at users.. is there a concern that brotoli compression
has similar dictionary or simply size based vulnerabilities as deflate?

If so, maybe we teach both to step out of the way when SSL encryption
filters are in place?

On Jan 16, 2017 10:14, "Jim Jagielski"  wrote:

> Just a head's up that I am working on the backport proposal/patch
> for brotli for 2.4.x...
>

Re: mod_lets-encrypt

[William A Rowe Jr]

On Sat, Jan 14, 2017 at 1:05 PM, Stefan Sperling  wrote:
> On Sat, Jan 14, 2017 at 07:15:29PM +0100, Dirk-Willem van Gulik wrote:
>> In fact - that may be a nice feature - an, essential, empheral port.
>
> Would that work for web servers behind firewalls?

Most configured in that scenario need pinholes. I'm not sure that a well
known but non-root port would be considered sufficiently secure by the
letsencrypt architects. Seems worth raising the original question, with
your firewall scenario in mind, and let their architects ponder the idea.

At the end of the day, only the one in control of those firewalls should
be respected by letsencrypt as the owner of the specific domain. That's
why the firewall is in front of their resources in the first place.

Obviously the actual port number isn't a valid security restriction in the
real world in the first place; there are plenty of ways to abuse a front
end firewall/load balancer behind the lines, and many appliances and
OS's don't require root or can be configured to permissively let other
accounts listen on particular low ports, and as long as you are physically
able to access the cables... but letsencrypt is not concerned with that.
It's simply that they are able to validate that you are in some way in control
of the target of the corresponding DNS A or  records.

Re: mod_lets-encrypt

[William A Rowe Jr]

On Sat, Jan 14, 2017 at 12:15 PM, Dirk-Willem van Gulik
<di...@webweaving.org> wrote:
>
> On 14 Jan 2017, at 19:05, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
> Any mod_letsencrypt can provision the certs but needs to do so
> while still root, before servicing requests (although there could be
> some bounce-step where the MPM begins satisfying requests,
> including the verification request necessary for letsencrypt.) We
> certainly don't want to parse any web response whatsoever while
> running as root.
>
> Some of this will be needed - we need to be root to bind to port 80 — as the
> protocol (in my reading) seems to demand it (now would be a good time to
> petition the draft to change this for a random higher port).
>
> In fact - that may be a nice feature - an, essential, empheral port.

My thinking was more along the lines that we fork off a process, much like
mod_cgid or mod_ftp does to handle low numbered ports. What is actually
handed across the domain socket or equivalent is structured simply enough
to not be harmed by an errant child process, or if needed by the server
earlier, then forked with the lesser run-as-httpd user permissions to speak
some predictable and strictly constructed message back up to the root parent
process.

It might be nice to our users if the cert/key isn't in the keystore,
that the server
starts anyways with a dummy cert, until it finally resolves to the
letsencrypt CA
server and completes that handshake. The server can respond to that child
process modifying the keystore and exiting by initiating a graceful restart to
begin serving traffic with the newly obtained cert/key material.

Somewhere in this equation lies accessor functions to mod_ssl to allow us
access to keystores. Which would be a very useful facility, well beyond just
letsencrypt. Whether those cert/key pairs live in ldap or some other place.
Imagine a small cluster, say 4 instances of httpd, where only one performs
the letsencrypt dance and the others search a memcache or other remote
resource for the resulting cert/key materials (locked down well, we would
hope.)

Re: mod_lets-encrypt

[William A Rowe Jr]

On Sat, Jan 14, 2017 at 10:22 AM, Eric Covener  wrote:
> On Sat, Jan 14, 2017 at 11:19 AM, Eric Covener  wrote:
>>
>> I think if a feature/directive will turn on something that will write
>> to configured keystores, it really shouldn't do or dictate much else.
>
> Poorly phrased, but I think obtaining a cert should be separate from
> things like further SSL configuration.

I think Dirk is suggesting that the core mod_ssl continues to exist, with
sane defaults that require next to no specific directives other than to
perhaps set the https protocol on port 443, and (I vote optionally) have
a one line toggle for rewriting all port 80 requests to 443.

Note that h2 requests will continue to be honored on either port 80
or 443, so this has to be crafted somewhat carefully.

I'm 100% in support of ensuring that mod_ssl runs with the most
sensible choices in the most minimal config.

Any mod_letsencrypt can provision the certs but needs to do so
while still root, before servicing requests (although there could be
some bounce-step where the MPM begins satisfying requests,
including the verification request necessary for letsencrypt.) We
certainly don't want to parse any web response whatsoever while
running as root.

I do believe the proposal should require a one line directive to
enable this, particularly for the compiled-in static many modules
build of httpd. It's shouldn't be simply a matter of loading some
mod_letsencrypt without also some 'LetsEncrypt on" directive
in the ssl vhost config.

[VOTE] [RESULTS] Release httpd-2.2.32

[William A Rowe Jr]

On Mon, Jan 9, 2017 at 12:21 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
> +/-1
> [ ] Release 2.2.32 as legacy GA

With more than sufficient numbers of PMC votes and a significant
number of non-binding votes, all +1 and no -1 objections,

This vote passes.

Shifting artifacts to the releases tree for mirror staging and an
announcement tomorrow. Again thanks to everyone who helped
throughout the past 18 mos to get us to 2.2.32.

Re: [VOTE] Release httpd-2.2.32

[William A Rowe Jr]

On Mon, Jan 9, 2017 at 12:21 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
> Your votes, please?
>
> +/-1
> [+1] Release 2.2.32 as legacy GA

Looking good on some of the ancient OS's, no regressions uncovered on
RHEL5 on s390[x], ppc[64] and intel, Solaris 10.x on sparc and intel,
HPUX 11.23 on ia64, or on Windows 2008. Still crosschecking across
more modern OS's, but to round out a great list of votes, my own +1.
Thanks again to all who tested, both binding votes and otherwise.

Re: svn commit: r1778004 - /httpd/httpd/branches/2.4.x/STATUS

[William A Rowe Jr]

On Thu, Jan 12, 2017 at 10:54 AM, Yann Ylavic <ylavic@gmail.com> wrote:
> On Thu, Jan 12, 2017 at 5:32 PM, William A Rowe Jr <wr...@rowe-clan.net> 
> wrote:
>>>
>>> + So the only fix allowing us to use PCRE 10 in httpd 2.4 would be to 
>>> write
>>> + this as a thread safe storage buffer for the majority of cases (<10 
>>> $args)
>>> + and we don't have a portable tls mechanism to do so.
>
> Don't we, apr_threadkey_private_*()?

Which in turn... requires the apr_pool_t * all over again, so not really;

https://apr.apache.org/docs/apr/2.0/group__apr__thread__proc.html#gacb9553b2b3165388bfc0a54b1148804a

>> Does anyone have a good suggestion on this that would be back-portable
>> and threadsafe for 2.4, before I surrender and make the trunk-only change
>> to include the worker's/thread's pool for this reusable buffer tagged with a
>> pooldata(m) in a new apr_pool_t * passed to the ap_regexec[_len] functions?
>
> Maybe having the match_data in a TLS created with a reasonable size
> (i.e. 50) on the first use?

For our purposes, I think the generic buffer of 10 ($0-$9) for most cases
is probably sufficient. But if folks like the idea (and consider that TLS may
be supporting almost 2k worker threads on an MPM like win32) perhaps
we use a quadratic reallocator, to leap from 10 to 40 to 160 or something
like that? But I would expect some 160 args to be a defective config or
malicious .htaccess exploit.

Our API isn't suited to named args, etc. At some point, a richer API might
want a bigger default. But allocating and freeing (or using a pool based
alternate allocator, with no requirement to 'free()') may be the most
efficient way to handle excessively long ovectors.

Re: svn commit: r1778004 - /httpd/httpd/branches/2.4.x/STATUS

[William A Rowe Jr]

On Mon, Jan 9, 2017 at 10:12 AM,   wrote:
> Author: wrowe
> Date: Mon Jan  9 16:12:53 2017
> New Revision: 1778004
>
>  PATCHES/ISSUES THAT ARE BEING WORKED
>[ New entried should be added at the START of the list ]
> @@ -275,6 +273,27 @@ PATCHES/ISSUES THAT ARE BEING WORKED
>   (& also, making the structure change with apr-util version
>   means it breaks binary compat across an apr-util upgrade?)
>
> +   * Support PCRE2 (10.x) in place of PCRE (8.x).
> + Submitted by: wrowe, Petr Pisar [ppisar redhat.com]
> + trunk patches:
> + http://svn.apache.org/r1773454
> + http://svn.apache.org/r1773741
> + http://svn.apache.org/r1773742
> + http://svn.apache.org/r1773839
> + http://svn.apache.org/r1773870
> + http://svn.apache.org/r1773882
> + wrowe notes that the current code is too inefficient, owing to the fact
> + that the ovector is a required allocation and is no longer allocated on
> + the stack, by design. The correct fix is an apr userdata allocation on
> + the appropriate pool, which would be thread-safe, but the actual API of
> + ap_regexec[_len]() offers us no pool. We cannot associate that pool with
> + the ap_regex_t, because a single regex may be used by many threads in
> + parallel and is not thread-safe beyond initialization.
> + So the only fix allowing us to use PCRE 10 in httpd 2.4 would be to 
> write
> + this as a thread safe storage buffer for the majority of cases (<10 
> $args)
> + and we don't have a portable tls mechanism to do so.

Does anyone have a good suggestion on this that would be back-portable
and threadsafe for 2.4, before I surrender and make the trunk-only change
to include the worker's/thread's pool for this reusable buffer tagged with a
pooldata(m) in a new apr_pool_t * passed to the ap_regexec[_len] functions?

Re: svn commit: r17757 - in /dev/httpd: Announcement2.2.html Announcement2.2.txt

[William A Rowe Jr]

On Tue, Jan 10, 2017 at 6:53 PM,   wrote:
> Author: wrowe
> Date: Wed Jan 11 00:53:47 2017
> New Revision: 17757
>
> Log:
> Not really sure if 'httpoxy' falls in this category

We call out "security defects", httpoxy is not a (web server) defect,
so my inclination
is to call out "security defects and enhancements" or omit httpoxy
from the primary
announcement text


> Modified:
> dev/httpd/Announcement2.2.html
> dev/httpd/Announcement2.2.txt

The draft is at
http://httpd.apache.org/dev/dist/Announcement2.2.html
http://httpd.apache.org/dev/dist/Announcement2.2.txt

Any remaining edits to suggest? The files live in
https://dist.apache.org/repos/dist/dev/httpd/

Re: [VOTE] Release httpd-2.2.32

[William A Rowe Jr]

On Wed, Jan 11, 2017 at 10:16 AM, Dale Ghent  wrote:
>
> Not a voting member, but dropping by to say that this is compiling and 
> working fine on OmniOS.

Just a reminder, whether it's a binding vote or just commentary, votes from
those beyond the usual suspects/PMC are always warmly welcomed!

Thanks for the feedback,

Bill

Re: svn commit: r1777998 - /httpd/httpd/branches/2.2.x/STATUS

[William A Rowe Jr]

As this seems (once applied to 2.4) to be an accepted part of the overall patch,
Yann you might want to add this to the merge/backport patch branches as part
of our overall, recommended patches against 2.2/2.4.



On Mon, Jan 9, 2017 at 9:53 AM,   wrote:
> Author: wrowe
> Date: Mon Jan  9 15:53:52 2017
> New Revision: 1777998
>
> URL: http://svn.apache.org/viewvc?rev=1777998=rev
> Log:
> Reviewed and approved, because 2.2.x is not likely to have another bugfix
> release beyond security fixes.
>
> For the same reason, demote months-old proposal to stalled, as this is not
> likely to gain another reviewer.
>
>
> Modified:
> httpd/httpd/branches/2.2.x/STATUS
>
> Modified: httpd/httpd/branches/2.2.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/STATUS?rev=1777998=1777997=1777998=diff
> ==
> --- httpd/httpd/branches/2.2.x/STATUS (original)
> +++ httpd/httpd/branches/2.2.x/STATUS Mon Jan  9 15:53:52 2017
> @@ -102,9 +102,20 @@ RELEASE SHOWSTOPPERS:
>  PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
>[ start all new proposals below, under PATCHES PROPOSED. ]
>
> +  *) http: allow folding in check_headers(), still compliant with RFC 7230 
> (3.2.4).
> + trunk patch: http://svn.apache.org/r1777460
> +  http://svn.apache.org/r1777672
> + 2.4.x patch: N/A
> + 2.2.x patch: 
> http://home.apache.org/~ylavic/patches/httpd-2.2.x-r1777460-v3.patch
> +  (needed because of s/APLOGNO//)
> + +1: ylavic, covener, wrowe
> +
>  PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>[ New proposals should be added at the end of the list ]
>
> +
> +PATCHES/ISSUES THAT ARE STALLED:
> +
>*) mod_proxy_connect: The connect method doesn't work if the client is
>   connecting to the apache proxy through an ssl socket. Fixed.
>   [Brad Boyer, Mark Cave-Ayland, Julian Gilbey, Fabrice Durand,
> @@ -126,16 +137,6 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>   the new code would be now very closed to 2.4.x's, might be
>   worth it for (one of) the latest 2.2.x release, and RIP...
>
> -  *) http: allow folding in check_headers(), still compliant with RFC 7230 
> (3.2.4).
> - trunk patch: http://svn.apache.org/r1777460
> -  http://svn.apache.org/r1777672
> - 2.4.x patch: N/A
> - 2.2.x patch: 
> http://home.apache.org/~ylavic/patches/httpd-2.2.x-r1777460-v3.patch
> -  (needed because of s/APLOGNO//)
> - +1: ylavic, covener
> -
> -PATCHES/ISSUES THAT ARE STALLED
> -
> * mod_proxy_balancer: Always initialize the shared parameters of a load
>   balancer member, even if it is already used as standalone.  PR 57067.
>   2.2.x patch: 
> http://people.apache.org/~ylavic/httpd-2.2.x-mod_proxy_balancer-lbfactor.patch
>
>

[VOTE] Release httpd-2.2.32

[William A Rowe Jr]

The pre-release candidate tarballs of Apache legacy httpd 2.2.32
can be found in;

http://httpd.apache.org/dev/dist/

Thanks to all for patches and reviews to get us to this point.
STATUS file is updated to reflect end of maintenance Jul 1 '17.

Your votes, please?

+/-1
[ ] Release 2.2.32 as legacy GA



Note .tar contains apr 1.5.2, apr-util 1.5.4 while -win32-src.zip
also contains apr-iconv 1.2.1, in keeping with 2.2 conventions.
New language in INSTALLING and additional language in the
Announcement will warn users away from the also bundled
PCRE and Expat sources, in favor of current releases.

Re: clang-analyzer?

[William A Rowe Jr]

Several times a year, we get offers or full dumps of programmatic static
code analysis.

We have, for decades, rejected it all, and invited reporters to bring
specific analysis of actually problematic cases back to the list (or
security@, as applicable.)

If anyone is interested, we consistently invite reports of actual defects
or security issues to be resolved.

Cheers,

Bill

On Jan 7, 2017 8:45 PM, "Leif Hedstrom"  wrote:

> Howdy,
>
> I ran clang-analyzer against the HTTPD master branch, and it found 126
> issues. Many of these are benign, but I was curious if the community has
> any thoughts on this? With another project, I’ve found that keep static
> code analysis to zero issues can really help finding new, serious issues
> (basically, we put the tree in failed state if there’s a new static code
> analysis issue).
>
> The issues are all over the source code, in core and mod_’s alike. It’d be
> pretty tedious to file individual tickets for each of them, so curious if
> there’s any interest in cleaning this up to start with a clean state? It’d
> then be easy to add clang-analyzer to the release process for example.
>
> Thoughts?
>
> — leif
>
>

Re: how make backend applications aware about tls-offloading

[William A Rowe Jr]

On Sat, Jan 7, 2017 at 2:30 AM, Reindl Harald  wrote:
> * Apache Trafficserver in front
> * ATS configured for TLS-offloading
> * connection to backend-httpd on the LAN unencrypted
> * mod_remoteip correctly configured on backend httpd
>
> is there any way to make the backend php application aware that in fact
> $_SERVER['HTTPS'] and $_SERVER['REQUEST_SCHEME'] should be 'on' / https://
> in case of generate absolute URLs like for emails
>
> in a perfect world this would be handeled like the transparent translation
> of the client IP with
> https://httpd.apache.org/docs/current/mod/mod_remoteip.html and it's
> RemoteIPInternalProxy and a header like "X-Forwarded-TLS"
>
> something like below where "X-TLS-Offloading" is only evaluated from
> "RemoteIPInternalProxy" pyhsical addressess
>
> RemoteIPHeader X-Forwarded-For
> RemoteTLSHeaderX-TLS-Offloading
> RemoteIPInternalProxy  192.168.196.1


The presence of TLS is a rather strict contract, in this example it
doesn't exist
end-to-end, so there is a wire segment that is sniffable (and yes you can also
argue that a physical box, given access, is sniffable). This particular example
would require all RemoteIPInternalProxy gateways to correctly de-taint any
previous X-TLS-Offloading value. In other words, many ways to get this wrong
in a day where end-to-end encryption is required by many policies.

I'm hesitant to introduce this into mod_remoteip... the configuration
of absolute
URL's in text/html output would probably follow the same setup as any other
situation where the app server is not physically on the wire, but hiding behind
a gateway. mod_proxy_html or mod_sed in httpd easily handle this for apps
that can't be reconfigured, I'd presume ATS offers a similar feature.

You can float this as a patch to mod_remoteip in bugzilla. It might not be
accepted, but that would be a place for interested users to find such an
example hack and perhaps adopt it.

Re: Tagging update for 2.2.32

[William A Rowe Jr]

This was the patch Victor was asking if you would verify...

http://home.apache.org/~ylavic/patches/httpd-2.2.x-r1753592.patch

That should be resolving the late declaration bug.

On Fri, Jan 6, 2017 at 8:18 PM, NormW  wrote:
> G/A
> I'm building the 2.2.x svn source tree, not a tag...
> Norm
>
>
> On 7/01/2017 1:09 PM, NormW wrote:
>>
>> G/A
>> As of 2 mins ago...
>>
>>> Building D:/Projects/svn/httpd-2.2.x/modules/proxy
>>> Calling NWGNUproxy
>>> CC   mod_proxy.c
>>> ### mwccnlm Compiler:
>>> #File: mod_proxy.c
>>> # 
>>> #1093:  int post_status = proxy_run_post_request(worker,
>>> balancer, r, conf);
>>> #   Error:  ^^^
>>> #   expression syntax error
>>> #   Too many errors printed, aborting program
>>
>>
>> Norm
>> On 7/01/2017 12:36 PM, Victor J. Orlikowski wrote:
>>>
>>> On Fri, Jan 06, 2017, at 08:13 PM, Victor J. Orlikowski wrote:

 Sigh. Missed the SHOWSTOPPER from the C89 thread.

>>>
>>> Wait a sec; re-reviewing that patch...Norm's issue should be
>>> addressed with the backport of r1753592.
>>>
>>> Norm - could you re-try that compile with Yann's backport?
>>>
>>> Best,
>>> Victor
>>>
>>
>

Re: svn commit: r1777453 - /httpd/httpd/branches/2.2.x/STATUS

[William A Rowe Jr]

On Fri, Jan 6, 2017 at 6:51 PM, Yann Ylavic  wrote:
> On Thu, Jan 5, 2017 at 12:39 PM,   wrote:
>> Author: ylavic
>> Date: Thu Jan  5 11:39:58 2017
>> New Revision: 1777453
>>
>> URL: http://svn.apache.org/viewvc?rev=1777453=rev
>> Log:
>> Promote r1753592 as showstopper, since it also fixes build with MSVC.
>> (Not worth fixing the build if we can backport this change, two fixes for 
>> the price of one :)
>>
>> Modified:
>> httpd/httpd/branches/2.2.x/STATUS
>>
>> Modified: httpd/httpd/branches/2.2.x/STATUS
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/STATUS?rev=1777453=1777452=1777453=diff
>> ==
>> --- httpd/httpd/branches/2.2.x/STATUS (original)
>> +++ httpd/httpd/branches/2.2.x/STATUS Thu Jan  5 11:39:58 2017
>> @@ -102,6 +102,16 @@ RELEASE SHOWSTOPPERS:
>>  PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
>>[ start all new proposals below, under PATCHES PROPOSED. ]
>>
>> +  *) mod_proxy: Correctly consider error response codes by the backend when
>> + processing failonstatus. PR 59869
>> +  Trunk version of patch:
>> + http://svn.apache.org/r1753592
>> +  Backport version for 2.4.x of patch:
>> + http://svn.apache.org/r1756562
>> +  Backport version for 2.2.x of patch:
>> + http://home.apache.org/~ylavic/patches/httpd-2.2.x-r1753592.patch
>> +  +1: ylavic
>
> Ouch, really meant SHOWSTOPPER here, and moved it to the wrong section.
> Re-moved in r140, c89 issue which would be unfortunate to fix
> without addressing PR 59869 at the same time...

Thought that's what you might have meant, but the showstopper is the smallest
patch that solves the gating issue. (I know, three non-reviewers of the recent
security refactoring are laughing their a$$e$ off at such a comment from me :)

Seems it has the votes, so we can start moving forwards, too bad so many
compiler solutions are so happy to accept c++ syntax for c sources without
a whimper.

Re: httpd-2.2.x and C89... ;-(

[William A Rowe Jr]

Great catch, thanks Norm. That too is part of the r1753592 backport
proposal, hoping someone is willing to look at these proposals.



On Fri, Jan 6, 2017 at 6:31 PM, NormW  wrote:
> G/M
> Did a test build of the 2.2.x tree and all builds nicely with exception of
> the following; if release is 'in progress' I leave to others to decide what
> to do about it.
>
>> D:\Projects\svn\httpd-2.2.x>svn diff
>> Index: modules/proxy/mod_proxy.c
>> ===
>> --- modules/proxy/mod_proxy.c   (revision 1777591)
>> +++ modules/proxy/mod_proxy.c   (working copy)
>> @@ -1088,9 +1088,9 @@
>>   * backend itself but by the proxy e.g. a bad gateway) in order
>> to give
>>   * ap_proxy_post_request a chance to act correctly on the status
>> code.
>>   */
>> +int post_status = proxy_run_post_request(worker, balancer, r,
>> conf);
>>  saved_status = r->status;
>>  r->status = access_status;
>> -int post_status = proxy_run_post_request(worker, balancer, r,
>> conf);
>>  /*
>>   * Only restore r->status if it has not been changed by
>>   * ap_proxy_post_request as we assume that this change was
>> intentional.
>
>
> Norm

Tagging update for 2.2.32

[William A Rowe Jr]

Folks,

Since there are many fewer pairs of eyeballs on this branch, I've completed
builds across Linux, Windows, Solaris and HPUX (and trying to salvage my
AIX environment that went sideways).  One of my $dayjob teammates offered
to jump in, so he's been able to review 2.2.32-dev plus the following STATUS
proposals across all of those architectures with me today;

PATCHES PROPOSED TO BACKPORT FROM TRUNK:
  [ New proposals should be added at the end of the list ]

  *) mod_proxy: Correctly consider error response codes by the backend when
 processing failonstatus. PR 59869
  Trunk version of patch:
 http://svn.apache.org/r1753592
  Backport version for 2.4.x of patch:
 http://svn.apache.org/r1756562
  Backport version for 2.2.x of patch:
 http://home.apache.org/~ylavic/patches/httpd-2.2.x-r1753592.patch
  +1: ylavic, wrowe

  *) mod_proxy_connect: The connect method doesn't work if the client is
 connecting to the apache proxy through an ssl socket. Fixed.
 [Brad Boyer, Mark Cave-Ayland, Julian Gilbey, Fabrice Durand,
 David Gence, Tim Dodge, Per Gunnar Hans, Emmanuel Elango,
 Kevin Croft, Rudolf Cardinal]
 PR: 29744
 Trunk Patch: https://svn.apache.org/r813178
  https://svn.apache.org/r1642857
  https://svn.apache.org/r1670324
  [plus patch and changes noted above]
 This code has evolved in trunk/2.4.x (including fixes) since the original
 commit (r813178 in 2009), specifically r1642857 and r1670324 from 2.4.x.
 2.2.x old r813178 patch:
https://bz.apache.org/bugzilla/attachment.cgi?id=30144
 Delta to proposed patch:
http://home.apache.org/~ylavic/patches/mod_proxy_connect-id30144_vs_r1670324.diff
 2.2.x Proposed Patch:
http://home.apache.org/~ylavic/patches/httpd-2.2.x-mod_proxy_connect-transfer.patch
 +1: wrowe, ylavic

  *) mod_proxy: Play/restore the TLS-SNI on new backend connections which
 had to be issued because the remote closed the previous/reusable one
 during idle (keep-alive) time.
 trunk patch: http://svn.apache.org/r1729826
  http://svn.apache.org/r1729847
  http://svn.apache.org/r1732986
  http://svn.apache.org/r1733056
 2.4.x patch: http://svn.apache.org/r1733474
 2.2.x patch:
http://home.apache.org/~ylavic/patches/httpd-2.2.x-mod_proxy-SNI_reuse.patch
 +1: ylavic, wrowe

  *) mod_proxy: Use the correct server name for SNI in case the backend
 SSL connection itself is established via a proxy server.  PR 57139
 trunk patch: http://svn.apache.org/r1634120
 2.4.x patch: http://svn.apache.org/r1673941
 2.2.x patch: trunk works (modulo CHANGES)
 +1: ylavic, wrowe

The other proves too much of a moving target to chase it today.

No regressions noted, everything looking good to him. I'd like to put
all these lingering proxy fixes to bed, since they are in 2.4.x, but we
need one more voulenteer/reviewer to make that happen.

In hopes that someone might review this weekend, I'll wait until first
thing Monday to tag and roll, but won't be delaying this any further.
This also gives me a chance to ensure it is building on AIX, once my
build environment is restored.

Bill

Re: svn commit: r1777460 - /httpd/httpd/trunk/modules/http/http_filters.c

[William A Rowe Jr]

On Fri, Jan 6, 2017 at 11:44 AM, Eric Covener  wrote:
> On Fri, Jan 6, 2017 at 12:06 PM, Jacob Champion  wrote:
>>> Modified:
>>> httpd/httpd/trunk/modules/http/http_filters.c
>>>
>>> Modified: httpd/httpd/trunk/modules/http/http_filters.c
>>> URL:
>>> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/http_filters.c?rev=1777460=1777459=1777460=diff
>>>
>>> ==
>>> [...]
>>> @@ -683,8 +726,10 @@ static APR_INLINE int check_headers(requ
>>>
>>>  ctx.r = r;
>>>  ctx.strict = (conf->http_conformance != AP_HTTP_CONFORMANCE_UNSAFE);
>>> -return apr_table_do(check_header, , r->headers_out, NULL) &&
>>> -   apr_table_do(check_header, , r->err_headers_out, NULL);
>>> +ctx.unfold = (!r->content_type || strncmp(r->content_type,
>>> +  "message/http", 12));
>>
>>
>> I don't think this unfolding exception should exist, at least not in this
>> part of the code. My reading of 7230 is not that folded headers are allowed
>> when the Content-Type is message/http, but rather that folded headers are
>> allowed *inside* the message/http payload body:
>>
>>This specification deprecates such
>>line folding except within the message/http media type
>>(Section 8.3.1). A sender MUST NOT generate a message that includes
>>line folding (i.e., that has any field-value that contains a match to
>>the obs-fold rule) unless the message is intended for packaging
>>within the message/http media type.
>>
>> IOW, the message/http payload body is allowed to be line-length limited, I
>> assume because it's a message/* media type. But that doesn't apply to the
>> HTTP-level headers.
>
> I think you're right.

This is correct. As a deprecation, it is an absolute prohibition against sending
obs-fold as HTTP protocol bytes. It is -not- a prohibition against receiving an
obs-fold from an earlier generation HTTP server. So this change ensure that
a legacy app server generating obs-folds will not be passed on to a server.

If you wanted to generate message/http media, you would filter the content
on line length to fold it appropriately. The origin server or app's preferences
are irrelevant.

I believe Yann's approach is correct.

Re: Fixing module-specific, public include/*.h file inclusion on trunk

[William A Rowe Jr]

On Fri, Dec 16, 2016 at 1:22 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Fri, Dec 16, 2016 at 12:57 PM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
>>
>> So today's primary bogus result is courtesy of is due to leaving
>> public headers hiding in modules/class/*.h paths for our builds.
>>
>> I'd suggest one of two approaches, pick a favorite solution?
>>
>>  [  ] Have the top level build copy all modules/*/mod_*.h
>>   definitions to the build tree include/ path (conditionally,
>>   based on datestamp)?
>>
>>  [  ] Make individual modules/*/Makefile[*] responsible for
>>   moving their headers into the build tree include/ path
>>   (conditionally, based on datestamp)?
>
>
> The challenge with option 2 is that we run the makefiles in
> sequence by directory name. It introduces the need to reorder
> the subdirectories by function, so that each makefile runs based
> on its dependencies (and reciprocal dependencies require the
> current /I "../foomodules/" hack.)

Decided that POLS applies here... we add all of these ../modules/foo/
paths to the unix compile command, we should do the same throughout
each and every make component in Windows (and Netware) so this
is less likely to bite us again before we entirely transition to CMake.

Re: svn commit: r1777460 - /httpd/httpd/trunk/modules/http/http_filters.c

[William A Rowe Jr]

On Thu, Jan 5, 2017 at 5:14 PM, Yann Ylavic  wrote:
> On Thu, Jan 5, 2017 at 11:49 PM, Yann Ylavic  wrote:
>>
>> But if any of you fears a possible regression for older 2.2.x apps (I
>> see now that Eric included a test, I personnaly tested it this
>> afternoon with a custom integration suite too), I'm happy to propose
>> and vote it.
>
> FWIW, proposed in STATUS for both to 2.2 and 2.4.

Excellent.

Since all other changes had lived on trunk and been accepted by several
more committers towards 2.4 backport, I'm more comfortable working in
the 2.4-stable changes onto 2.2 at this moment than adding new work,
but that's just my 2c. I'll vet and approve the patch after we wrap up 2.2.32
but if others jump on board to review it, I'm not about to object.

Re: svn commit: r1777460 - /httpd/httpd/trunk/modules/http/http_filters.c

[William A Rowe Jr]

On Thu, Jan 5, 2017 at 4:05 PM, Eric Covener  wrote:
> Do we want this for the 2.2 release?

I don't feel strongly about this.

It is such an unusual edge case (I believe Yann pointed out it was a custom
module he was working around) that it should rarely be seen in the wild.

I'd be fine if we want to accelerate this into 2.2.32, or get 2.2.32
out and then
backport to have the patch available for the very few who are impacted.

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Thu, Jan 5, 2017 at 12:50 PM, Jacob Champion  wrote:
> On 01/04/2017 11:55 AM, Graham Leggett wrote:
>>
>> On 04 Jan 2017, at 8:37 PM, Jacob Champion  wrote:
>>>
>>> So, there's 3k of the 20k. And remember, my point was that we can
>>> fix what I call "dead code" with good old fashioned legwork. I
>>> don't advocate trashing trunk, and I don't think having "dead code"
>>> is a disaster or a stain on anyone here. I just don't think it's
>>> appropriate to spin up an RC from trunk as-is.
>>
>>
>> Look for the discussion that occurred around November 2011 when v2.4
>> was released:
>>
>> http://smtp.grokbase.com/t/apache/dev/11bb6wswq2/branched-httpd-2-4-x
>>
>>  We came to the same conclusion then.
>
>
> I started a longer reply to what you wrote earlier in the email, but I think
> I need to clear up any misunderstandings I have with this part first.
>
> From offlist discussions I've had with other committers (and Bill's recent
> reply to this thread), my understanding was that an alpha/beta branch would
> be forked from the current tip of trunk, followed by testing and additional
> feature work, until a .0 release is voted on.
>
> The conversation you linked to appears to modify that somewhat: we started
> tagging trunk directly with alpha/beta, and at some point decided to fork a
> 2.4.x from a mostly current trunk. It also adds the information that 2.4.x
> was still CTR, up to the .0 release. But in both cases, the statement "we
> plan to fork 2.6.x from a current-ish trunk commit" seems to hold up pretty
> well.
>
> Is that correct?

Following past practices, and a desire to reduce the amount of effort by
committers, I'd strongly object to a fork before 3.0 (or 2.6) is finalized.
It's a procedural-process sort of question, so it's a straightforward PMC
majority vote.

I'm also against forking at 3.0 or 2.6 until someone wants to introduce
an API breaking change, unless we solidify some new process for having
more predictable and frequent bug fix releases and split these from our
constant churn of feature changes. In that case, feature changes do map
well to trunk until there is a breaking ABI change, while we would need to
collect such bug fixes on a new branch.

Re: 2.2 needs a reviewer for http strict backport ...

[William A Rowe Jr]

On Thu, Jan 5, 2017 at 3:03 AM, Yann Ylavic  wrote:
> On Thu, Jan 5, 2017 at 3:02 AM, Eric Covener  wrote:
>>
>> 2.2 running clean under test suite for me on Linux.
>
> Same here, thanks Eric for backporting.
>
> PS: I had to apply the OPENSSL_NO_SSL3 patch for my debian
> (libssl-1.0.2) to compile.
> One more vote please ;)
> PPS: The other (annoying) gcc warnings fix patches are welcome too :)

So I'm currently running regressions against all of the proposed backports,
including proxy / ssl bug fixes, given that SNI is very widely deployed now,
and SSL3 is now super-discouraged. Good set of proposals, Yann - thanks.

A few proxy bug fixes still need one more reviewer to be accepted.
I'll backport all which are accepted, and unless anyone raises an issue,
will move ahead with a tag tomorrow (Friday) to kick off a 2.2.32 release
vote.

Re: A new release process?

[William A Rowe Jr]

On Tue, Jan 3, 2017 at 1:32 PM, Jacob Champion  wrote:
> On 12/29/2016 08:16 PM, David Zuelke wrote:
>>
>> The tl;dr of this approach is that
>>
>> - any x.y.z release only introduces bugfixes. These releases are done
>> every four weeks, like clockwork. If a fix doesn't make the cut for a
>> release, it'll end up in the next one; - x.y.0 releases, on the other
>> hand, may introduce new features, fixes, and deprecations, but no
>> breaking changes; - x.0.0 releases are the big ones (think PHP 7.0.0
>> in late 2015). where backward compatibility may be changed, etc.
>
> My favorite pieces here are
> - the introduction of a release cadence
> - the separation of bugfix releases from feature releases
>
> Both make life easier -- for us, for users, and for intermediate maintainers
> -- for the reasons you've mentioned.

Consider also our consumers, module authors who rely on us to provide an
API they can design to. All of our subsidiary components, be it apr, or expat,
or pcre, or openssl, or libxml2, or lua, or nghttp2... and most of the
expansions
to httpd, be it python or perl or whatnot, all follow essentially this model.

That isn't how httpd evolved; if you look far far back into the pre-1.0 history
and even through the evolution up to 1.3.12, there wasn't a particular rhyme
or reason other than what the major patch submitters wanted to have happen.
So when pools are introduced, minor version bump - a subversion bump was
obviously not going to cut it. But there was not necessarily any compatibility
at all with previously compiled modules.

This all changed radically with 1.3.14 and 2.0 later on. But the aversion from
back in the 1.2 and 1.3 days to declaring the version minors are precious
resources has not changed.

In the meantime, nghttp2 just published the 18th version minor by year end
last year. And remains solidly compatible due to good design from the get go,
it is still major version 1. Will be interesting to watch and see when enough
API issues have accumulated to ditch major v1 for v2.

We are certainly the enigma, and if there is one reason for developers to
disabused themselves from your API, that is likely making shit up as you
go along. I know a lot of folks frustrated back in the PHP 3-4 days, and it
seems like this solution has worked out very well for the PHP crew. And
there was an equal amount of chaos around Perl's versioning and lifecycle
evolution, but that also seems to be working out pretty well.


>> [...]
>>
>> There are a bunch of technicalities that would need adjusting to fit
>> HTTPD, such as release intervals, release management (for PHP, every
>> x.y.* series has two managers who jointly coordinate releases), etc,
>> but overall the idea is, IMO, worth considering.
>
> Our current process is also fairly labor-intensive, so moving to a quick
> release cadence without simultaneously fixing that is not likely to end up
> well, IMO. There are other discussions on the list for improving our
> automated QA, which I think is probably step 1.

Releases themselves are not terribly challenging. The investment, outside
of the reviews and testing we hope many committers are engaged in,
boils down to less than an hour of actual mechanics, and some time to
prepare specific announcement details or merge back in some sequestered
security patches if those have to be applied.


>> As a, more or less, "outside observer", I happen to think that the
>> current method of voting on finals, instead of a practice of rolling
>> out RCs (that are then left up for testing for at least a week), is
>> fundamentally broken. The 2.4 changelog in particular is littered
>> with releases that were never officially published. For users, that's
>> really confusing.
>
> To a certain extent I think this is something that people get used to
> easily... but for security patches in *particular*, I agree. It's
> disconcerting to hear that CVE 2099-BLAH was fixed in 2.4.Z and then find
> out that 2.4.Z was never released.

We say this, that subversion numbers are cheap, and the lack of 2.4.24
really isn't confusing. But I have to answer questions from at least 15
individuals over the course of the following month when that happens :)
So apparently it might not confuse us, but it does confuse the rest of
the world more than we might expect.


>> For maintainers, it's painful to start over the
>> process each time, and it sometimes leads to months and months
>> without a release that contains certain fixes.
>
> The pain of discarding and restarting your manual tests because someone else
> found a regression on another platform two hours into the release vote comes
> to mind, yes.

On that, adding features to a security release 4 days before tag and roll is
a sure fire path to a bad tag. But features-features-features, gotta collect
them all :)

Looking at a long term solution to the specific quirk that occurred in 2.4.24
and 2.4.25, but the general truth of throwing in one more late last feature

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

To your questions of history;

On Wed, Jan 4, 2017 at 12:37 PM, Jacob Champion  wrote:
>
> 3) mod_apreq2
>
> 1000 lines, added in 2011, no meaningful code changes since addition, no
> tests, no documented public release of libapreq2 since 2010. (It does have
> public documentation. And it seems like there's history here I don't
> understand. Maybe it lives somewhere else and was being folded into httpd?)

Yes, this comes from the Apache Perl (mod_perl) project (also
Apache::Test which they still maintain and release.) As does the Win32
flavor of apxs.

> So, there's 3k of the 20k. And remember, my point was that we can fix what I
> call "dead code" with good old fashioned legwork. I don't advocate trashing
> trunk, and I don't think having "dead code" is a disaster or a stain on
> anyone here. I just don't think it's appropriate to spin up an RC from trunk
> as-is.

Historically, e.g. between 2.0 and 2.2, we have had a series of public
alphas followed by betas labeled 2.1.x-alpha (and -beta), until it was
clean enough to go GA as 2.2.0.

I doubt anyone is thinking of going straight to any 2.6 or 3.0 release
without a series of 2.5-alpha/beta releases. We especially considered
third party module authors when we moved to that schema, so that they
would have the opportunity to adjust for any API or behavior changes
long before a GA announcement, including a feedback loop to ask us to
improve that API for their consumption. Few seem to have taken us up
on that opportunity, but we can always hope for more participation the
next cycle.

Re: Could/Shouldn't check_header() allow folding?

[William A Rowe Jr]

On Wed, Jan 4, 2017 at 11:12 AM, Yann Ylavic  wrote:
>
> This would work for me (on the proxy side), too.
> The patch (attached) is a bit longer, but still reasonable IMHO.
> WDYT?

Not understanding if (!header->key) { continue; } - why success if there is
a dead ': UnnamedValue' entry in the output headers table? Unclear why
apr_table_do needed to be re-implemented.

Does it make sense to smash surrounding whitespace here? A single SP
is going to have the same semantic value as multiple SP/TAB characters.

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Wed, Jan 4, 2017 at 9:47 AM, Graham Leggett <minf...@sharp.fm> wrote:
> On 04 Jan 2017, at 3:16 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
>>> Can you give us an example of this dead code?
>>
>> svn diff --ignore-properties --no-diff-deleted -x --ignore-all-space
>> https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/server
>> https://svn.apache.org/repos/asf/httpd/httpd/trunk/server
>> svn diff --ignore-properties --no-diff-deleted -x --ignore-all-space
>> https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules
>> https://svn.apache.org/repos/asf/httpd/httpd/trunk/modules
>
> That’s not dead code, that’s just the difference between v2.4 and trunk.

So long as the project chooses not to release it, it sits in a repository DoA.

One of the interesting assertions is that C-T-R -> Trunk, R-T-C Trunk
-> 2.4 is the correct and customary state of contributing to httpd.

At one time this project operated in C-T-R -> Trunk -> Release mode.
It would be productive to return to that model.

The goal of R-T-C on 2.4.x is simply to eliminate (unsuccessfully) ABI
breakage and regressions between subversion releases.

Re: Could/Shouldn't check_header() allow folding?

[William A Rowe Jr]

On Wed, Jan 4, 2017 at 7:21 AM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Wed, Jan 4, 2017 at 6:57 AM, Yann Ylavic <ylavic@gmail.com> wrote:
>> I'm using a (third-party/closed) module which replaces newlines in
>> header values (like base64 encoded PEMs) with obs-fold.
>
> If we accept obs-fold from CGI, or internally within the
> headers_out, we must
> replace them with a single SP and conform to the spec on the wire.

What about either an optional filter, or simply plug in a module that
only implements a fixup hook, which can be configured on potentially
offending requests to scan and correct the headers_out members?

Such a module could also do something to work around other CTLs,
and invalid header token names, to avoid 500 error conditions and
attempt to still forward some response.

Re: Could/Shouldn't check_header() allow folding?

[William A Rowe Jr]

On Wed, Jan 4, 2017 at 6:57 AM, Yann Ylavic  wrote:
> I'm using a (third-party/closed) module which replaces newlines in
> header values (like base64 encoded PEMs) with obs-fold.
> That's probably obsolete, but not forbidden per se...

Actually, it is, c.f. 3.2.4 of RFC 7230

   [...] This specification deprecates such
   line folding except within the message/http media type
   (Section 8.3.1).  A sender MUST NOT generate a message that includes
   line folding (i.e., that has any field-value that contains a match to
   the obs-fold rule) unless the message is intended for packaging
   within the message/http media type.

> How about something like:
>
> Index: modules/http/http_filters.c
> ===
> --- modules/http/http_filters.c(revision 1776920)
> +++ modules/http/http_filters.c(working copy)
> @@ -701,19 +701,26 @@ static int check_header(void *arg, const char *nam
>  return 0;
>  }
>
> -if (ctx->strict) {
> -test = ap_scan_http_token(name);
> -}
> -else {
> -test = ap_scan_vchar_obstext(name);
> -}
> -if (*test) {
> -ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, ctx->r, APLOGNO(02429)
> -  "Response header name '%s' contains invalid "
> -  "characters, aborting request",
> -  name);
> -return 0;
> -}
> +test = name;
> +do {
> +if (ctx->strict) {
> +test = ap_scan_http_token(test);
> +}
> +else {
> +test = ap_scan_vchar_obstext(test);
> +}
> +if (*test) {
> +if (test[0] != CR || test[1] != LF || (test[2] != ' ' &&
> +   test[2] != '\t')) {
> +ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, ctx->r, 
> APLOGNO(02429)
> +  "Response header name '%s' contains invalid "
> +  "characters, aborting request",
> +  name);
> +return 0;
> +}
> +test += 3;
> +}
> +} while (*test);
>
>  test = ap_scan_http_field_content(val);
>  if (*test) {
> ?

-1. If we accept obs-fold from CGI, or internally within the
headers_out, we must
replace them with a single SP and conform to the spec on the wire.

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Wed, Jan 4, 2017 at 2:13 AM, Graham Leggett  wrote:
> On 03 Jan 2017, at 10:47 PM, Jacob Champion  wrote:
>
>> I don't feel that trunk is a dead branch, but I do think there is dead code 
>> in trunk.
>
> Can you give us an example of this dead code?

svn diff --ignore-properties --no-diff-deleted -x --ignore-all-space
https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/server
https://svn.apache.org/repos/asf/httpd/httpd/trunk/server
svn diff --ignore-properties --no-diff-deleted -x --ignore-all-space
https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules
https://svn.apache.org/repos/asf/httpd/httpd/trunk/modules

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Tue, Jan 3, 2017 at 7:04 PM, Noel Butler  wrote:
>
> On 03/01/2017 23:11, Jim Jagielski wrote:
>
> Back in the "old days" we used to provide complimentary builds
> for some OSs... I'm not saying we go back and do that necessarily,
> but maybe also providing easily consumable other formats when we
> do a release, as a "service" to the community might make a lot
> of sense.
>
>
> 2 years ago it was decided to stop the official -deps (despite they are 
> included in dev still)... now you want to bring it back? (you'd have to if 
> you're going to roll usable binary packages or your "community service" 
> re-built packages are going to be broken)

I don't think he said that. For years httpd shipped the compiled
current openssl, expat, pcre sources as a binary. There was no sources
package of these, although we did provide the .diff to get the
packages to build correctly.

Because HTTP/2 requires OpenSSL 1.0.2, that will have to be part of
most packages, including semi-modern Linux flavors.

PCRE[2] is unavoidable, and while libxml2 can sub in for libexpat, the
SVN project would rather we bound to libexpat for specific features
they rely on.


> Although I as many others here prefer to roll our own due to our configs, and 
> not having to deal with bloat, I can see this having a positive effect for 
> users of a couple of distros who when they release brand new releases, come 
> with antiquated junk thats outdated and stays outdated, to give those users a 
> choice of using a modern code set would be good, but requires long term 
> dedication.

Agreed - it simply has to land somewhere like /opt/apache/httpd/ or
whatnot, to disambiguate whatever the user builds for themself in
/usr/local/ and what the OS might provision in /usr/

Re: svn commit: r1775789 - /httpd/httpd/branches/2.2.x/STATUS

[William A Rowe Jr]

On Tue, Jan 3, 2017 at 11:04 AM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Tue, Jan 3, 2017 at 9:55 AM, Eric Covener <cove...@gmail.com> wrote:
>> I am not completely following how the branch or patch were assembled,
>> but I am seeing a failure that is missing content from the initial
>> trunk work (1426877)
>> that was also in the initial 2.4.x backport (1772678).
>>
>> It is causing frequent crashes on EOF of a keepalive conn for me
>>
>> The missing bit that sticks out/causes the crash is server/protocol.c
>> ap_read_request():
>>
>> 1276 default:
>> 1277 apr_brigade_destroy(tmp_bb);
>> 1278 r = NULL;
>>  ^ missing
>> 1279 return r;
>> 1280 }
>> 1281 }
>> 1282
>>
>> Bill do you recall if there was perhaps a hand-resolved merge conflict
>> in this area?
>
> That makes perfect sense due to the s/goto traceout/return r/ conflicts,
> none of the patch applied in that section, and I simply missed this line.
> The traceout change wasn't merged since 2.2 has no support for that
> loglevel.
>
> Feel free to patch on the merge branch, or I will fix this myself this
> afternoon.

Looking more closely at this code, the four essential errors all return
NULL, and default: should have returned NULL, which leaves me
wondering why HTTP_REQUEST_TIMEOUT returns the request_rec
when no other error cases do so?

Re: svn commit: r1775789 - /httpd/httpd/branches/2.2.x/STATUS

[William A Rowe Jr]

On Tue, Jan 3, 2017 at 9:55 AM, Eric Covener  wrote:
> I am not completely following how the branch or patch were assembled,
> but I am seeing a failure that is missing content from the initial
> trunk work (1426877)
> that was also in the initial 2.4.x backport (1772678).
>
> It is causing frequent crashes on EOF of a keepalive conn for me
>
> The missing bit that sticks out/causes the crash is server/protocol.c
> ap_read_request():
>
> 1276 default:
> 1277 apr_brigade_destroy(tmp_bb);
> 1278 r = NULL;
>  ^ missing
> 1279 return r;
> 1280 }
> 1281 }
> 1282
>
> Bill do you recall if there was perhaps a hand-resolved merge conflict
> in this area?

That makes perfect sense due to the s/goto traceout/return r/ conflicts,
none of the patch applied in that section, and I simply missed this line.
The traceout change wasn't merged since 2.2 has no support for that
loglevel.

Feel free to patch on the merge branch, or I will fix this myself this
afternoon.

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Jan 3, 2017 07:11, "Jim Jagielski"  wrote:

Back in the "old days" we used to provide complimentary builds
for some OSs... I'm not saying we go back and do that necessarily,
but maybe also providing easily consumable other formats when we
do a release, as a "service" to the community might make a lot
of sense.


It could be really helpful. Or we can follow svn's lead and hand it
entirely off to the broader community, which proved really effective on
Windows, given the number of distros to now choose between. I haven't seen
similar for RHEL users, for example.

That said, only one major Linux distro (April Ubuntu LTS) is at OpenSSL
1.0.2, which is a necessary part of http/2's special sauce.

Re: [proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

On Jan 3, 2017 02:19, "Graham Leggett"  wrote:


Can you clarify the problem you’re trying to solve?

v3.0 and v2.6 are just numbers. For modest changes, we move to v2.6. For a
very large architecture change (for example, the addition of filters in
v1.x to v2.x), we move to 3.0.

Is there a very large architecture change planned by anybody?

In my experience, things that felt initially like big changes have actually
turned out to be rather modest changes that are still possible to backport
to v2.4 without an issue. For this reason I haven’t seen a reason to push
very hard for v2.6, never mind v3.0.


I do, the very specific problem is that trunk/, and therefore all
more-than-modest (or just neglected) contributions are now four years stale
and abandoned.

A certain way to push off contributors is to ignore their patch
submissions. The second method is to commit them to a dead fork.

If trunk/ is a dead fork, it may be time for httpd to admit this, trash it
and re-fork trunk from 2.4.x branch.

Beyond this, see the recent appeal for major.minor breaking change wish
list on trunk/STATUS, that is a different thread for you to chime in on.

Back to topic, who is interested in a stable release chain, since 2.4.x has
not been that?

Re: Httpd security reveals

[William A Rowe Jr]

On Mon, Jan 2, 2017 at 11:49 PM, Eric Covener <cove...@gmail.com> wrote:
> On Mon, Jan 2, 2017 at 11:48 PM, William A Rowe Jr <wr...@rowe-clan.net> 
> wrote:
>> So, Jacob and I... He did most of the grunt work, I only pushed off the
>> underlying premise... Have a very very long list of real and potential
>> security patches.
>>
>> I am asking publicly of (often obstanant) httpd pmc folks, do we proceed
>> without a 2.2 mitigation? Those in the know, already know.
>>
>> Happy to RM Wed a.m. if we have the votes.
>
> Sorry, I totally missed that you had completed/proposed the backport
> of the showstopper to 2.2.x. I should have reviewed it earlier but I
> will work on it Tuesday.

Clarification, I completed 'a' proposal, and reordered a number of other
accepted backports to incorporate all of work in sequence. As Yann had
pointed out 'patch doesn't apply'... when it was all disordered.

There are a couple additions in the past week that should be considered
from the 2.4.x-http-strict branch, but the patch has been complete for
some time.

Cheers,

Bill

Httpd security reveals

[William A Rowe Jr]

So, Jacob and I... He did most of the grunt work, I only pushed off the
underlying premise... Have a very very long list of real and potential
security patches.

I am asking publicly of (often obstanant) httpd pmc folks, do we proceed
without a 2.2 mitigation? Those in the know, already know.

Happy to RM Wed a.m. if we have the votes.

[proposed] 2.4 Maintenance SIG

[William A Rowe Jr]

So far, discussions are polarized on a single axis...

East: Let's work on 3.0; whatever is going on in 2.4 won't distract me, I
won't spend time reviewing enhancements, because 3.0 is the goal.

West: Let's keep the energy going on 2.4 enhancements, I won't spend time
on 3.0 usability because it isn't ready or necessary.

There is a disconnect, because 'East' folks can't actually put up with the
breakage introduced by enhancements they can't review on 2.4, but all feel
responsible to keeping 2.4 usable to EOL.

So I'd like to know, in light of a perpetual chain of (often build and/or
run-time breaking regression) enhancements, if there is support for a
2.4.24.x release chain during the 3.0 transition? And support for
potentially 3x backports to 2.4.x, 2.4.24.x and 2.2.x, of really serious
bug fixes?

It's clear this project doesn't agree, so the question twists to how we
agree to disagree.

Re: Automated tests

[William A Rowe Jr]

On Dec 30, 2016 14:55, "Stefan Fritsch"  wrote:

Hi,

it's quite rare that I have a bit of time for httpd nowadays. But  I want to
comment on a mail that Jacob Champion wrote on -security that contains some
valid points about the lack of our test framework. I am posting this to -dev
with his permission.

On Wednesday, 21 December 2016 08:55:30 CET Jacob Champion wrote:
> - Our APIs are really complex, and we don't really have unit tests for
> them. Nor are the internal APIs documented as well as the external APIs
> are. We had a few false starts for security fixes this release that were
> later shown to break something else, and I think that's related.

Yes, httpd lacks unit tests. One problem is that many APIs depend on very
complex structs like request_rec, conn_rec, server_conf, etc. In order to
write unit tests for such APIs, one would need to write quite a bit of
infrastructure to set these things up. I think it would be worth the effort,
but it's not a small task. As there does not seem to be anybody with enough
spare time to do it, one could possibly ask someone (CII?) for funding.

A possible approach would be to compile the unit tests in the server and
execute them on startup if a special define is given (like the various
DUMP_*
defines). Not sure how to get access to all the static helper function for
unit
tests, though. Unless one would somehow include the tests in the same .c
file.


Thinking two things would help.

Splitting our functional utilities into a libaputil would make it much
easier to write the tests that exercise these elements of our code.

And what I found easiest is a dedicated module to provide diagnostics or
tests. When not loaded, they are skipped.

Re: svn commit: r1776575 - in /httpd/httpd/trunk: docs/log-message-tags/next-number docs/manual/mod/mod_remoteip.xml modules/metadata/mod_remoteip.c

[William A Rowe Jr]

Simply address the issue that caused the -1... 'Code mostly copyright...'
needs to be 'Portions copyright'... A statement which is unlikely to become
entirely false.


On Dec 30, 2016 18:28, "Daniel Ruggeri" <drugg...@primary.net> wrote:

> Aye - I suspected this would raise eyebrows so I did bring it up a few
> times [1][2]. I'm sure we're in agreement that attribution is important
> in the Open Source world so I'd like to be sure it's done appropriately.
> I'm happy to fix.
>
> Currently, though, I'm not sure how best to handle this veto... what
> would be the preferred path forward? As a first step, I've remove the
> three lines mentioned here and added to CHANGES in r1776674. The 2.4
> backport has also been modified (simply by removing the lines since
> CHANGES already has attribution to the original authors which seems to
> be preferred per your message).
>
> Does that work or did you have another approach in mind?
>
> [1]
> https://lists.apache.org/thread.html/95173df808992097f565bc7bcd06a0
> 1b2129415bff0aae6c608b82fe@%3Cdev.httpd.apache.org%3E
> [2]
> https://lists.apache.org/thread.html/28e660f38d945216d9d0bb4cba3e1b
> 4336a4c5051a46f17c8f99a0f0@%3Cdev.httpd.apache.org%3E
>
>
> --
> Daniel Ruggeri
>
> On 12/30/2016 8:00 PM, William A Rowe Jr wrote:
> > -1 (yes, veto.)
> >
> > In general, as the original author of this particular module, you
> > might notice I don't claim attribution. We rely on svn provenance and
> > Changes to describe code legacy, so these make me very uncomfortable,
> > especially when injected into our sources. That isn't the key issue.
> >
> > The statement itself may be presently true. The statement a number of
> > years from now might be radically false. The presence of this
> > statement makes it impossible for the future svn hacker to know when
> > to modify the statement or determine when the sources have changed
> > such that it becomes untrue.
> >
> > It is a relativistic value judgement, and these aren't useful as
> > legalistic elements, so the conventional 'portions copyright...' sort
> > of language is used instead.
>
>

Re: svn commit: r1776575 - in /httpd/httpd/trunk: docs/log-message-tags/next-number docs/manual/mod/mod_remoteip.xml modules/metadata/mod_remoteip.c

[William A Rowe Jr]

On Dec 30, 2016 06:20,  wrote:

Author: druggeri
Date: Fri Dec 30 14:20:48 2016
New Revision: 1776575

URL: http://svn.apache.org/viewvc?rev=1776575=rev
Log:
Merge new PROXY protocol code into mod_remoteip

Modified:
httpd/httpd/trunk/docs/log-message-tags/next-number
httpd/httpd/trunk/docs/manual/mod/mod_remoteip.xml
httpd/httpd/trunk/modules/metadata/mod_remoteip.c



==
--- httpd/httpd/trunk/modules/metadata/mod_remoteip.c (original)
+++ httpd/httpd/trunk/modules/metadata/mod_remoteip.c Fri Dec 30 14:20:48
2016
@@ -12,15 +12,20 @@
  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  * See the License for the specific language governing permissions and
  * limitations under the License.
+ *
+ * The majority of the input filter code for PROXY protocol support is
+ * Copyright 2014 Cloudzilla Inc.
  */


-1 (yes, veto.)

In general, as the original author of this particular module, you might
notice I don't claim attribution. We rely on svn provenance and Changes to
describe code legacy, so these make me very uncomfortable, especially when
injected into our sources. That isn't the key issue.

The statement itself may be presently true. The statement a number of years
from now might be radically false. The presence of this statement makes it
impossible for the future svn hacker to know when to modify the statement
or determine when the sources have changed such that it becomes untrue.

It is a relativistic value judgement, and these aren't useful as legalistic
elements, so the conventional 'portions copyright...' sort of language is
used instead.

Re: Post 2.4.25

[William A Rowe Jr]

On Thu, Dec 29, 2016 at 8:23 AM, Jim Jagielski <j...@jagunet.com> wrote:
>
>> On Dec 28, 2016, at 6:28 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>>
>> Because fixing r->uri is such a priority, trust that I'll be voting every 
>> 2.6 candidate a -1 until it exists. I don't know why the original httpd 
>> founders are so hung up on version number conservation, they are cheap, but 
>> we are breaking a key field of a core request structure and no other OSS 
>> project would be stupid enough to call that n.m+1.
>
> Who is digging in their heels and blocking new development
> now?
>
> So you are admitting that you will "veto" (although you
> can't veto a release) any 2.5.* "releases" unless and
> until r->uri is "fixed".

Wow, Jim, how did you misread my assertion that I'd oppose 2.6 GA
or 3.0 GA release until feature "X", where "X" represents the heavy-lift
of not using filesystem syntax as the uri path except for files, honoring
the URI and HTTP RFC, and therefore requiring some module authors to
re-think how they consumed or changed/assigned r->uri. Modules such
as proxy would actually pass on the *presented* uri (if valid) to the back
end http server - just imagine that. That change I'm expecting we all
want to call out as 3.0 for our developers, even though there are no
directives changed for our users so administration doesn't change.

How did you jump to the conclusion that I'd block an -alpha or -beta
release on the 2.5.x trunk? Usually takes some number of incremental
-alpha/-beta tags to get to GA.

And how did you translate 'vote -1' to veto?

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Thu, Dec 29, 2016 at 8:25 AM, Jim Jagielski <j...@jagunet.com> wrote:
> It wasn't the paste that was the problem, but the inability
> of other email clients to determine from your email what
> parts/sections are quoted from *previous* emails.

Yann pointed me in the right direction, I believe it is fixed now.

Thanks for the heads-up!

>> On Dec 28, 2016, at 5:49 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>>
>> Hi Jim,
>>
>> Talk to Google and the OpenOffice Team, that was a paste from OpenOffice 
>> Calc.
>>
>> I'll be happy to start summarizing as a shared Google sheet.

Google sheet might still be useful, so I'll maintain that as a general purpose
collection of shared-with-httpd-dev tabs.

On the subject of r->uri [was: Post 2.4.25]

[William A Rowe Jr]

On Wed, Dec 28, 2016 at 6:42 PM, Yann Ylavic <ylavic@gmail.com> wrote:

> [Bill, you definitely should do something with your email client, e.g.
> using plain text only, replying to your messages breaks indentation
> level (like the number of '>' preceding/according to the initial
> message)].
>

(Again, it's gmail, /shrug. I can attempt to undecorate but doubt I'm
moving to a local client/mail store again. If anyone has good gmail
formatting tips for their default settings, I'd love a pointer.)


> On Thu, Dec 29, 2016 at 12:28 AM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
> >
> > On Dec 24, 2016 07:57, "Jim Jagielski" <j...@jagunet.com> wrote:
> >
> > Well as breaking changes go, changing URI to remain an encoded value and
> to
> > mandate module authors accept that req_rec is free threaded are breaking
> > changes.
>
> Not sure what the second point means, but preserving r->uri while also
> allowing to (and internally work with) an escaped form of the URI does
> not necessarily require an API change.
> (We could possibly have an opaque struct to manipulate the URI in its
> different forms and some helper(s) be compatible with the API changes'
> requirements, e.g. 2.4.x).
>

To be clear, this isn't possible.

There are multiple meanings of every path segment character which is
in the reserved set. There is no way to preserve these multiple meanings
in a decoded context. The parallel entities may exist in any undecoded
string. So r->uri, if it still exists, will be subsumed by some variable
like
r->uri_path_unencoded and be retrievable into a decoded form.

Functions such as ap_hook_map_to_storage, in the filesystem backend,
will only be interested in the decoded form. Functions such as the http
proxy module will only be interested in passing a never-mangled version
of the encoded uri.

Even if r->uri is available as a read-only input, there is no simple way for
httpd to resolve r->uri manipulations if changed in place (it isn't const)
and whether an r->uri_path_unencoded mismatch which is canonical,
and what mishmash the legacy abuser of r->uri did with these parallel
reserved characters in their encoded and unnencoded forms. We are
stuck with the current mess of various %-escape workarounds until
we replace the core assumption.

This deserves a long discussion which already exists in the security@
list, but needs to be pushed outward on dev@, preferably by the original
authors of these thoughts. That includes the r->uri preserving flavor
that you mention above, as well as the various discussions about the
% entity encoding, and my concerns about canonicalization. With some
first-level triage already complete, there is no reason for uri discussion
to remain 'behind the curtain.'

Re: Post 2.4.25

[William A Rowe Jr]

On Dec 24, 2016 08:32, "Eric Covener"  wrote:

> I'm not saying we don't do one so we can do the other; I'm
> saying we do both, at the same time, in parallel. I still
> don't understand why that concept is such an anathema to some
> people.

I also worry about our ability to deliver a 3.0 with enough
re-architecture for us and and function for users, vs a more
continuous delivery (apologies for bringing buzzaords to dev@httpd)
cadence on 2.4 as we've been in.


Here is the confusion (see the versioning thread.)

2.6 is a break in ABI compatibility.

3.0 is a break in API compatibility.

Size in this case doesn't matter. Any break at all merits these changes.

We are not a commercial product. We are httpd. Nobody cares what the
version no is other than us, they very largely install and forget, OS
vendors grab new at one point in their distribution gathering phase and
don't revisit.

Adoption outside of OS distros is largely irrelevant. Talk about
do-nothing, PCRE2 has been out a very long time with all the activity and
no adoption, PCRE 8.x is on life support with little pulse and is the
defacto standard.

Your assumptions don't reflect the actual adoption behaviors.

Re: Post 2.4.25

[William A Rowe Jr]

On Dec 24, 2016 07:57, "Jim Jagielski"  wrote:


> On Dec 24, 2016, at 8:29 AM, Rich Bowen  wrote:
>
> On 12/23/2016 03:52 PM, Jim Jagielski wrote:
>> Personally, I don't think that backporting stuff to
>> 2.4 prevents or disallows development on 2.6/3.0. In
>> fact, I think it helps. We can easily do both...
>> after all, we are still "working" on 2.2.
>>
>> As I have also stated, my personal belief is that
>> 2.4 is finally reaching some traction, and if we
>> "turn off" development/enhancement of 2.4, we will
>> stop the uptake of 2.4 in its track. We need to keep
>> 2.4 viable and worthwhile we, at the same time, work
>> on 2.6/3.0. I think we all understand that getting
>> 2.6/3.0 out will not be a quick and/or painless
>> action.
>
> From my perspective, watching Nginx gain traction through superior
> marketing, and channeling Dilbert's Pointy Haired Boss in assuming that
> everything which I have never done must be simple, I, for one, would
> like to see us release a 2.6, and more generally, to release a 2.x every
> 2 years, or less, rather than every 4 years, or more.
>
> My opinion on this, I would emphasize, is 100% marketing, and 0%
> technical. I realize we "don't do" marketing, but if we want to still ve
> having the fun of doing this in another 20 years, it may be necessary to
> get our name out there a little more frequently in terms of doing new
> things. We are frankly not great at telling the world about the cool new
> things we're doing.
>

Yeah, right now the way we are "doing marketing" is by
continually adding features and enhancements to 2.4... It
is what keeps 2.4 relevant and is what either keeps current
httpd users using httpd or maybe help those on the fence decide
on httpd instead of nginx/whatever.


And again to play devil's advocate, how has that worked out in the four
years of httpd 2.4?

My point is that even having a 6 month minimal (and that
is, IMO, widely optimistic and unrealistic) of "no new
features for 2.4" means that we are basically giving people
reasons to drop httpd. It would be a widely different story
if (1) trunk was ready to release and (2) we "committed" to
releasing trunk quickly by focusing on low-hanging fruit
which would make lives happier and better for our end-users.
As I said, my fear is that we will not be able to "control"
ourselves in limiting what is in 2.6, which will push the
actual release far past the point where it is even relevant.


Well as breaking changes go, changing URI to remain an encoded value and to
mandate module authors accept that req_rec is free threaded are breaking
changes. We did that on conn_rec post-2.2. But I suspect that we have now
done this to req_rec with the event mpm and sf's http2 enhancements already
on 2.4?

To be clear, if our goal was "Fork trunk as 2.5 NOW, polish
and tune 2.5 'as-is' with minimal major refactoring with
the goal of getting 2.6 out ASAP" then yeah, sure, the idea
of "no new features in 2.4" would have some merit. But based
on current conversation, it's obvious that, at least to me,
that won't happen and we will be continually refactoring 2.6
to make it a 3.0.


Two mistakes. If we commit to a new contract on these two things, there is
never 2.6.

Because fixing r->uri is such a priority, trust that I'll be voting every
2.6 candidate a -1 until it exists. I don't know why the original httpd
founders are so hung up on version number conservation, they are cheap, but
we are breaking a key field of a core request structure and no other OSS
project would be stupid enough to call that n.m+1.

So you can presume there is no such thing as 2.6.

Error 2 and you ignored the first reply, there is no branch until 3.0
occurs.

I'll save that detail for the next post.

Again, you don't "stop" development on a
current release until the next release is ready or, at least,
"this close" to being ready. You don't sacrifice the present,
nor do you leave your users in that limbo.


Users had been in limbo 10 months for security fixes and far longer for bug
fixes PatchAvailable in bugzilla, and you are worried about feature
development on an maintenance branch. Sigh...

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Dec 28, 2016 10:34, "William A Rowe Jr" <wr...@rowe-clan.net> wrote:


Specific
Revision
Of all Most
Recent
Of m.m Of all
Apache/1.3.x 391898 3.33% 1.3.42 42392 10.82% 0.36%
Apache/2.0.x 551117 4.68% 2.0.64 36944 6.70% 0.31%
Apache/2.2.x 7129391 60.49% 2.2.31 1332448 18.78% 11.31%
Apache/2.4.x 3713364 31.51% 2.4.17+ 1502061 42.90% 12.74%

11785770
2.4.23 754385 21.54% 6.40%


Since this table is illegible to some, please see the second tab of

https://docs.google.com/spreadsheets/d/1aOxBRZ2IHsUJJcQNXu-oe6la4wMRIHN2mOlJCQGRy0k/edit?usp=drivesdk

The first tab is a crossref of many OS distribution components used by
httpd, as well as httpd itself.

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

Hi Jim,

Talk to Google and the OpenOffice Team, that was a paste from OpenOffice
Calc.

I'll be happy to start summarizing as a shared Google sheet.

Cheers,

Bill


On Dec 28, 2016 14:22, "Jim Jagielski" <j...@jagunet.com> wrote:

> Bill, I don't know if it's just my Email client or not (doesn't
> look like it) but could you fix your Email client? It's impossible to
> reply and have the quoted parts parsed out correctly. I think
> it's to do w/ your messages being RTF or something.
>
> Thx!
>
> Included is an example of how a Reply misses quote levels...
>
> > On Dec 28, 2016, at 1:34 PM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
> >
> > On Wed, Dec 28, 2016 at 9:13 AM, Jim Jagielski <j...@jagunet.com> wrote:
> > cPanel too... They are moving to EA4 which is Apache 2.4.
> >
> > If not moved yet, that example wouldn't be helpful, it reinforces my
> point
> > four years later. But EA itself seems to track pretty closely to the most
> > contemperanious versions, looks like within a month.
> >
> >
> > So the idea that supplemental (ie: 2.4.x->2.4.y) patches don't
> > have the reach or range of larger ones (2.4.x->2.6/3.0) isn't
> > quite accurate.
> >
> > It's entirely accurate. It isn't all-encompassing. We have that data too,
> > let's tear down SecuritySpace's Nov '16 dataset;
> > http://www.securityspace.com/s_survey/data/201611/servers.html
> >
>
>

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Wed, Dec 28, 2016 at 9:13 AM, Jim Jagielski  wrote:

> cPanel too... They are moving to EA4 which is Apache 2.4.
>

If not moved yet, that example wouldn't be helpful, it reinforces my point
four years later. But EA itself seems to track pretty closely to the most
contemperanious versions, looks like within a month.


So the idea that supplemental (ie: 2.4.x->2.4.y) patches don't
> have the reach or range of larger ones (2.4.x->2.6/3.0) isn't
> quite accurate.
>

It's entirely accurate. It isn't all-encompassing. We have that data too,
let's tear down SecuritySpace's Nov '16 dataset;
http://www.securityspace.com/s_survey/data/201611/servers.html

First off, if you follow that link, you'll find much larger numbers
associated
to those specific revisions shipped with the likes of RHEL or CentOS, Ubuntu
(particularly -LTS flavors), etc etc etc. That was my contention in the top
post. But let's quantify 'accuracy' as you defined it in the reply...

Specific
Revision
Of all Most
Recent
Of m.m Of all
Apache/1.3.x 391898 3.33% 1.3.42 42392 10.82% 0.36%
Apache/2.0.x 551117 4.68% 2.0.64 36944 6.70% 0.31%
Apache/2.2.x 7129391 60.49% 2.2.31 1332448 18.78% 11.31%
Apache/2.4.x 3713364 31.51% 2.4.17+ 1502061 42.90% 12.74%

11785770
2.4.23 754385 21.54% 6.40%

The applicable data are 37.47% of all 'Apache[/n[.n[.n]]]' items, meaning
that some 2/3rds of users drop the ServerTokens down to product only
or major version only, and we can't derive anything useful from them, so
we will ignore the Apache and Apache/2 references for our % evaluations,
'Of all' refers to those with at least Apache/2.x designations.

I included 2.4.17-2.4.23 as an item, because that group are the versions
that released within the past year of this particular survey data (that does
include the then-current 2.4.23.)

The 'Of m.m' - same major.minor - backs out that Apache/2.x (without a
known subversion) from the calculation because we can't tell whether they
are the corresponding or a different subversion.

Of httpd users we can quantify, 6.4% updated within months of the 2.4.23
release (your 'power users' classification.) That minority doesn't move the
needle much on total adoption of httpd vs. others.

Only 11.3% bothered to pick up the final 2.2.31 that has been out
over a year, and combined with 12.74% running some 2.4.17...2.4.23,
*** only 24% *** run a version that had been a current release within
the preceding year.  E.g. of those running a somewhat-current version,
more than 1/4 are running the July 2.4.23 release by the end of November.
Note that Fedora 25 didn't move the needle much on this, it shipped GA
in December.

aren't the ones we are talking about in the 1st place. We are
> talking about real, "power" users, who want/need the latest
> and greatest.
>

Not if you are talking overall adoption rate. As illustrated, those
users adopting 2.4.23 already are an nearly accidental minority,
after 5 mos half of the 'current' 2.4 users are running 2.4.23, the
other half are running a flavor between 12 and 6 mos old. That
looks like overall random distribution by deployment date, with
no particular effort expended on 'staying current'.

Re: The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Wed, Dec 28, 2016 at 9:05 AM, Jan Ehrhardt <php...@ehrhardt.nl> wrote:

> William A Rowe Jr in gmane.comp.apache.devel (Tue, 27 Dec 2016 23:35:50
> -0600):
> >But the vast majority of httpd, nginx, and yes - even IIS
> >users are all running what they were handed from their
> >OS distribution.
>
> Do not underestimate the influence of control panels. On all my Centos
> servers I am running Directadmin. DA always offers to upgrade to the
> latest release within a day after the release. Hence, I am running
> Apache 2.4.25 everywhere at the moment.
>
> Excellent pointer. Thanks Jan.

The Version Bump fallacy [Was Re: Post 2.4.25]

[William A Rowe Jr]

On Fri, Dec 23, 2016 at 2:52 PM, Jim Jagielski  wrote:

>
> As I have also stated, my personal belief is that
> 2.4 is finally reaching some traction, and if we
> "turn off" development/enhancement of 2.4, we will
> stop the uptake of 2.4 in its track.


This is where I think we have a disconnect.

Our adoption is *broadly* based on the OS distributions
from vendors, not from people picking up our sources.
Yes - some integrate directly from source, and others
use a non-OS distribution.

But the vast majority of httpd, nginx, and yes - even IIS
users are all running what they were handed from their
OS distribution. This is why an amazing number of people
run 2.4.3-2.4.10 and soon, 2.4.18, even though these are
all already out of date. Once RHEL, Ubuntu LTS, SUSE
or others pick up a specific rev, that's where the typical
user is going to land for the next several years.

The raw stats show a couple of interesting things, IMO;
https://w3techs.com/technologies/overview/web_server/all
While we have slipped somewhat, the old adage that
httpd or another "Web Server" must sit in front of the
cobbled-together app servers doesn't apply anymore.
Code like Tomcat, etc, is now far more robust and
capable of sitting on the outward facing edge of the DMZ.

The two runners up in web server space have essentially
switched places, nginx now has the market penetration
that IIS once enjoyed. IIS now amounts to a fraction of
what it once did, essentially the 'everything else' share
that used to be held by webservers we don't think about
any more, such as Sun's, lighttpd, etc. And of course
custom server agents of the top 10 data providers skew
the results significantly.

Other surveys paint the data a little differently;
https://news.netcraft.com/archives/2016/12/21/december-
2016-web-server-survey.html
http://www.securityspace.com/s_survey/data/201611/index.html

Next up, we will see broad distribution of 2.4.23 - why?
Because that shipped in Fedora 25 which will very likely
become RHEL 8. E.g. we missed the boat, Generally
the Fedora release a year out from RHEL GA become
the shipping packages, and the pattern suggests this
early winter release becomes an early winter '17 RHEL.

If we don't ship improvements, we wither and fall into
oblivion. It does not matter that these are called 2.4.26
because *no vendor will ship it*. Not until they start
gathering the components of their next major release.
Which means, they are shipping are least interesting
sources over and over because we aren't shipping new
major releases.

So I'd respectively suggest that adding a feature to
2.4 vs releasing the feature in 3.0 makes not one
iota of difference in goodwill/adoption. The next major
releases who code freeze after 3.0 has shipped will
be in position to pick up and distribute 3.0. All the
rest will be stuck in the past.

But as a bottom line, all those users stuck in the past
until their OS catches up will have little opinion about
a feature in a 2.4.x release they will never see, since
their vendor keeps shipping the same 2.4.n that their
OS revision had initially shipped.
.

Re: Post 2.4.25

[William A Rowe Jr]

On Dec 23, 2016 9:58 PM, "Jim Jagielski"  wrote:

Well, since I am actively working on trunk, I am obviously interested in
seeing continued work being done on it and the work being usable to our
users in a timely fashion. Since backports to 2.2 have not affected work on
2.4 or trunk, it is obvious as well that any backport efforts for 2.4 won't
be any issue at all, so work on trunk will be unrestricted.


Restrictions, no, never. But if I had to ask how did that work out for me
merging antique commits back at 2.4 and from 2.4 to 2.2, you don't want my
opinion on your therom.

I hope your enthusiasm regarding timeframes is warranted and accurate.
Obviously my efforts are directed towards what is best for our community
and am looking forward to how we continue with next gen.


As do I.

Different refutation of your underlying therom on versioning will happen on
or after the weekend. In the meantime...

A joyous belated Solstice, happy Hanukkah, merry Christmas, or just wishing
everyone a good weekend. You have all been some of.my most favorite people
now for 15+ years.  See you all next year :)

Re: Post 2.4.25

[William A Rowe Jr]

On Fri, Dec 23, 2016 at 2:20 PM, Jim Jagielski  wrote:

> For me, it would be moving as much as we can from
> trunk to 2.4


-1. To echo your frequent use of media to emphasize
the point, with a song nearly as old as us;
https://www.youtube.com/watch?v=EsCyC1dZiN8

Next step is to actually end enhancements alltogether
against 2.4 (we've done that some time ago, security
issues notwithstanding, on 2.2), and push all of the
enhancement effort towards 3.0 (2.5-dev). Of course,
we should continue to pick up bug fixes and help those
still on 2.4 have a good day.

Let those users looking for cool new things pick up
the 3.0 release.

Or else you are kicking 'everything we can't' further
down the road, again dismissing all of the activity
of so many of our fellow committers. Stale stuff on
trunk/ now dates to over 4 years ago.

That state of things simply sucks.

Re: T of 2.4.24

[William A Rowe Jr]

On Fri, Dec 9, 2016 at 8:03 AM, Jim Jagielski <j...@jagunet.com> wrote:

>
> > On Dec 9, 2016, at 12:20 AM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
> >
> > On Thu, Dec 8, 2016 at 12:16 PM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
> >
> > @VP Legal, is this worth an escalation? You didn't see fit to respond
> today,
> > but I think this falls under the purview of your committee, w.r.t.
> unapproved
> > release artifacts living at www.apache.org/dist/. Did you have any
> thoughts
> > or opinions one way or another?
>
> How is this different from, say, the win32 src zips or the
> complimentary binary builds?


That's an interesting question, or questions...

For starters, source aren't binaries, but of course you knew that, as our
esteemed VP, Legal.

When ASF projects convey binaries, they convey them (purportedly) based
on the current jars/wars of the dependencies (are there other dependent
projects? SVN doesn't ship binaries, and I have no clue what OpenOffice
does. Other non-java examples are few and far between.)

These are fetched up fresh from maven or whatnot, and don't have a lot of
bearing on how non-java projects do things. AIUI, those jars don't even
supplant what is already provisioned, if those are more current, unless
the manifest demands an old rev.

The prior win32 src (before I committed that to branch, not trunk, and
didn't worry our silly heads about crlf after I wrote the apr fix script)
didn't include extra artifacts, unless you count apr-iconv. And I have
deep reservations about that call, if you've seen my comments about
what citrus might bring us and lack of maintaining that BSD iconv fork.

Thanks for the redaction on the 2.4.25-deps artifact. Frankly, I would
not have helped you push that out the door without that one concession.
And mad props to JChapmion for pushing the announce, since I didn't
have ASF smtp at the ready. So as always, it was an effort of many.

Fundamental issue with pushing -deps of, say, APR 1.5.2, is that the
following week is that 1.5.3 with bug fixes is released. Is the httpd
project responsible for updating -deps? Or f' ya all, download this
package... it won't hurt you... hopefully? Believe me, I went through
all that as an httpd win32 binary distributor who bundled openssl,
so I know this specific pain-point, and sense of responsibility, and
did have to ship new interim binaries when bad things were disclosed.

I hope you sort this out in your ombudsman role, because this is the
test of whether you understand ASF responsibilities, both legally,
and in the sense of our entire ecosystem, and the will of your specific
project who had a very firm position, before you undermined it.

Cheers, and a Merry Christmas!

Bill

Re: svn commit: r1775705 - in /httpd/httpd/branches/2.2.x-merge-http-strict: ./ server/protocol.c

[William A Rowe Jr]

On Thu, Dec 22, 2016 at 6:00 PM, Yann Ylavic  wrote:

> On Thu, Dec 22, 2016 at 8:20 PM,   wrote:
> > Author: wrowe
> > Date: Thu Dec 22 19:20:25 2016
> > New Revision: 1775705
> >
> > URL: http://svn.apache.org/viewvc?rev=1775705=rev
> > Log:
> > Backports: r1185385
> > Submitted by: sf
> > Downgrade some more log messages indicating client errors from level
> error to
> > info. Add log messages for various reasons to return HTTP_BAD_REQUEST.
>
> Hmm, in 2.2, users are used to get an error message when httpd refuses
> the request (moreover before any module had seen the request).
>
> Is it really 2.2 material?
>
> >
> > Modified: httpd/httpd/branches/2.2.x-merge-http-strict/server/protocol.c
> > URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x-
> merge-http-strict/server/protocol.c?rev=1775705=
> 1775704=1775705=diff
> > 
> ==
> > --- httpd/httpd/branches/2.2.x-merge-http-strict/server/protocol.c
> (original)
> > +++ httpd/httpd/branches/2.2.x-merge-http-strict/server/protocol.c Thu
> Dec 22 19:20:25 2016
> > @@ -702,9 +702,21 @@ static int table_do_fn_check_lengths(voi
> > "\n\n",
> > ap_escape_html(r->pool, key),
> > "\n", NULL));
> > +ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, "Request header exceeds
> "
> > +  "LimitRequestFieldSize after merging: %s", key);
> >  return 0;
> >  }
> >
> > +/* get the length of the field name for logging, but no more than 80
> bytes */
> > +#define LOG_NAME_MAX_LEN 80
> > +static int field_name_len(const char *field)
> > +{
> > +const char *end = ap_strchr_c(field, ':');
> > +if (end == NULL || end - field > LOG_NAME_MAX_LEN)
> > +return LOG_NAME_MAX_LEN;
> > +return end - field;
> > +}
> > +
> >  AP_DECLARE(void) ap_get_mime_headers_core(request_rec *r,
> apr_bucket_brigade *bb)
> >  {
> >  char *last_field = NULL;
> > @@ -755,6 +767,9 @@ AP_DECLARE(void) ap_get_mime_headers_cor
> >  ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r,
> >"Request header exceeds
> LimitRequestFieldSize: "
> >"%.*s", field_name_len(field), field);
> > +ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r,
> > +  "Request header exceeds
> LimitRequestFieldSize: "
> > +  "%.*s", field_name_len(field), field);
>
> Double logging?
>
> >  }
> >  return;
> >  }
> > @@ -786,6 +801,10 @@ AP_DECLARE(void) ap_get_mime_headers_cor
> >"Request header exceeds
> LimitRequestFieldSize "
> >"after folding: %.*s",
> >field_name_len(last_field),
> last_field);
> > +ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r,
> > +  "Request header exceeds
> LimitRequestFieldSize "
> > +  "after folding: %.*s",
> > +  field_name_len(last_field),
> last_field);
>
> Likewise?
>
> >  return;
> >  }
> >
> > @@ -811,6 +830,9 @@ AP_DECLARE(void) ap_get_mime_headers_cor
> >  apr_table_setn(r->notes, "error-notes",
> > "The number of request header fields
> "
> > "exceeds this server's limit.");
> > +ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r,
> > +  "Number of request headers exceeds "
> > +  "LimitRequestFields");
> >  return;
> >  }
> >
> > @@ -829,6 +851,10 @@ AP_DECLARE(void) ap_get_mime_headers_cor
> >"separator: %.*s",
> (int)LOG_NAME_MAX_LEN,
> >last_field);
> >
> > +ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r,
> > +  "Request header field is missing ':' "
> > +  "separator: %.*s",
> (int)LOG_NAME_MAX_LEN,
> > +  last_field);
>
> Likewise?
>
> >  return;
> >  }
>

Yes, doubled merge. Believe I have these cleaned up in r1775774.

Re: svn commit: r1775696 - /httpd/httpd/branches/2.2.x-merge-http-strict/

[William A Rowe Jr]

On Thu, Dec 22, 2016 at 5:50 PM, Yann Ylavic  wrote:

> On Thu, Dec 22, 2016 at 7:45 PM,   wrote:
> > Author: wrowe
> > Date: Thu Dec 22 18:45:59 2016
> > New Revision: 1775696
> >
> > URL: http://svn.apache.org/viewvc?rev=1775696=rev
> > Log:
> > Backports: r1100511
> > Submitted by: covener
> > use APR_STATUS_IS_TIMEUP() instead of direct comparison with APR_TIMEUP.
> >
> > Modified:
> > httpd/httpd/branches/2.2.x-merge-http-strict/   (props changed)
>
> No (non-props) change?
>

Red herring, it came into 2.2.x branch through the original commit, from
the svn blame on the 2.2.x-merge file.

Re: svn commit: r1769669 [2/2] - in /httpd/httpd/branches/2.4.x-merge-http-strict: ./ docs/manual/ docs/manual/mod/ include/ server/

[William A Rowe Jr]

On Thu, Dec 22, 2016 at 11:37 AM, Rainer Jung <rainer.j...@kippdata.de>
wrote:

> Am 22.12.2016 um 18:25 schrieb William A Rowe Jr:
>
>> On Thu, Dec 22, 2016 at 9:29 AM, Eric Covener <cove...@gmail.com
>> <mailto:cove...@gmail.com>> wrote:
>>
>> I think the log severity changes below could use some eyes, especially
>> in context of 2.2.  Are these lowered because they're redundant?  I
>> haven't yet looked.
>>
>> I am tempted to leave the old severities for 2.2 and wait and see if
>> it's confusing in 2.4 (should not have to enable DEBUG to see the
>> cause of a 400 error)
>>
>>
>> Log severity is lowered because successfully throwing off a (generally
>> deliberately) bad request is of no interest to an operator at loglevel
>> error/warn, any more than any successful request is of interest.
>>
>> This is just log pollution. There is no action for the operator to take.
>>
>
> Hmmm, but if it isn't about a deliberately bad request but a bad client,
> the admin could consider setting "HttpProtocolOptions Unsafe" (and now get
> what he asked for. I must admit, I didn't check whether the log lines in
> question are in code areas which are controlled by HttpProtocolOptions or
> not.
>

That's true. However, every admin/operator knows that if the app dev team
or users are complaining that the request doesn't work, the first thing any
admin does is enable loglevel debug to see what's going on.

The CTL's aren't override-able (see the security reports 2.4 detailed
description) by our group concensus, and my UnsafeWhitespace proposal was
rejected. But there is still information at debug level to know what sort
of problem occurred without breaking out wireshark first.

Once that is sorted, it's still noise and needs to be disabled again. The
broken clients will be the very odd exception, not a rule.

Re: svn commit: r1769669 [2/2] - in /httpd/httpd/branches/2.4.x-merge-http-strict: ./ docs/manual/ docs/manual/mod/ include/ server/

[William A Rowe Jr]

On Thu, Dec 22, 2016 at 9:29 AM, Eric Covener  wrote:

> I think the log severity changes below could use some eyes, especially
> in context of 2.2.  Are these lowered because they're redundant?  I
> haven't yet looked.
>
> I am tempted to leave the old severities for 2.2 and wait and see if
> it's confusing in 2.4 (should not have to enable DEBUG to see the
> cause of a 400 error)


Log severity is lowered because successfully throwing off a (generally
deliberately) bad request is of no interest to an operator at loglevel
error/warn, any more than any successful request is of interest.

This is just log pollution. There is no action for the operator to take.

Re: APLOGNO in 2.2 backports

[William A Rowe Jr]

On Thu, Dec 22, 2016 at 9:33 AM, Eric Covener  wrote:

> Should we have something like this in 2.2:
>
> #define APLOGNO(s) ""
>
> So subsequent backports don't need manual merging when the context has
> a message ID in it?
>

IMO, no.

Because if you are applying it as a patch to, say, httpd 2.2.17, the patch
will
still be a broken patch. It would only work from 2.2.32 and onwards (and
after
2.2.32 I'm crossing fingers there isn't any need for onwards.)

Re: svn commit: r17503 - in /dev/httpd: Announcement2.4.html Announcement2.4.txt

[William A Rowe Jr]

On Tue, Dec 20, 2016 at 12:34 PM, William A Rowe Jr <wr...@rowe-clan.net>
wrote:

> On Tue, Dec 20, 2016 at 12:13 PM, <wr...@apache.org> wrote:
>
>> Author: wrowe
>> Date: Tue Dec 20 18:13:12 2016
>> New Revision: 17503
>>
>> Log:
>> Record security errata, edits in the next 45 minutes are most welcomed
>>
>> Modified:
>> dev/httpd/Announcement2.4.html
>> dev/httpd/Announcement2.4.txt
>>
>
> I'll take the Announcement live on the hour (30 min from now), anyone
> who wants to edit, the path is a little unusual;
>
> https://dist.apache.org/repos/dist/dev/httpd
>
> I'll pick up any final edits before replicating to release/dev/httpd
>
> Trying to get as much of httpd_vulnerabilities.xml caught up ahead
> of the announce as is reasonable. Doesn't help that the latest
> dnf update vim completely smashed my vim behavior ;-/
>

Accomplished with some help from jchampion as I didn't have any
smtp set up for @apache personality (the joys of being gmail-bound.)

Review of the website etc welcomed. Thanks to all who helped get
this ready, thanks again to Jim for RM'ing!

Re: Old Announcement Logo

[William A Rowe Jr]

Sure thing. I only needed ...2.4.txt in its final form. Go for it :)

Bill

On Tue, Dec 20, 2016 at 1:02 PM, Jacob Champion 
wrote:

> Rich pointed out that our announcements are still using the old logo:
>
> https://www.apache.org/dist/httpd/Announcement2.4.html
>
> Should we add our new banner (https://httpd.apache.org/imag
> es/httpd_logo_wide_new.png) to the dist repo and use it instead?
>
> --Jacob
>

Re: svn commit: r17503 - in /dev/httpd: Announcement2.4.html Announcement2.4.txt

[William A Rowe Jr]

On Tue, Dec 20, 2016 at 12:13 PM,  wrote:

> Author: wrowe
> Date: Tue Dec 20 18:13:12 2016
> New Revision: 17503
>
> Log:
> Record security errata, edits in the next 45 minutes are most welcomed
>
> Modified:
> dev/httpd/Announcement2.4.html
> dev/httpd/Announcement2.4.txt
>

I'll take the Announcement live on the hour (30 min from now), anyone
who wants to edit, the path is a little unusual;

https://dist.apache.org/repos/dist/dev/httpd

I'll pick up any final edits before replicating to release/dev/httpd

Trying to get as much of httpd_vulnerabilities.xml caught up ahead
of the announce as is reasonable. Doesn't help that the latest
dnf update vim completely smashed my vim behavior ;-/