Re: svn commit: r1779578 - /httpd/httpd/trunk/docs/manual/mod/mod_proxy_http2.xml

[ste...@eissing.org]

> Am 20.01.2017 um 09:45 schrieb elu...@apache.org:
> 
> Author: elukey
> Date: Fri Jan 20 08:45:40 2017
> New Revision: 1779578
> 
> URL: http://svn.apache.org/viewvc?rev=1779578&view=rev
> Log:
> Added more details to mod-proxy-http2's doc
> 
> Modified:
>httpd/httpd/trunk/docs/manual/mod/mod_proxy_http2.xml
> 
> Modified: httpd/httpd/trunk/docs/manual/mod/mod_proxy_http2.xml
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/mod/mod_proxy_http2.xml?rev=1779578&r1=1779577&r2=1779578&view=diff
> ==
> --- httpd/httpd/trunk/docs/manual/mod/mod_proxy_http2.xml (original)
> +++ httpd/httpd/trunk/docs/manual/mod/mod_proxy_http2.xml Fri Jan 20 08:45:40 
> 2017
> @@ -41,9 +41,14 @@
> have to be present in the server.
> 
> mod_proxy_http2 works with incoming requests
> -over HTTP/1.1 and HTTP/2 requests. If mod_http2
> -handles the frontend connection, requests against the same HTTP/2
> -backend are sent over a single connection, whenever possible.
> +over HTTP/1.1 and HTTP/2 requests. In both cases, requests proxied
> +to the same backend are sent over a single connection
> +whenever possible (namely when the connection can be re-used).
> +
> +mod_proxy_http2 will not use the HTTP/2 protocol
> +when the frontend requests use HTTP/1.1.
> +This means that HTTP/2 will be used to proxy requests to a capable 
> backend
> +only when the frontend requests use the same protocol.
> 

Not correct. Maybe my explanation was not good. mod_proxy_http2 will always use 
HTTP/2 in the backend connection. That connection will however only do one 
request at a time if the frontend is HTTP/1.1.

> This module relies on http://nghttp2.org/";>libnghttp2
> to provide the core http/2 engine.
> 
> 

Re: svn commit: r1783759 - in /httpd/httpd/trunk: docs/log-message-tags/next-number include/mod_core.h modules/filters/mod_include.c modules/http2/h2_conn.c server/main.c server/mpm/event/event.c serv

[ste...@eissing.org]

Uhm, don't know what it is, but trunk runs out of open files in my load tests 
now after a few seconds...

> Am 20.02.2017 um 14:50 schrieb yla...@apache.org:
> 
> Author: ylavic
> Date: Mon Feb 20 13:50:49 2017
> New Revision: 1783759
> 
> URL: http://svn.apache.org/viewvc?rev=1783759&view=rev
> Log:
> Follow up to r1783755: update APLOGNO.
> 
> Modified:
>httpd/httpd/trunk/docs/log-message-tags/next-number
>httpd/httpd/trunk/include/mod_core.h
>httpd/httpd/trunk/modules/filters/mod_include.c
>httpd/httpd/trunk/modules/http2/h2_conn.c
>httpd/httpd/trunk/server/main.c
>httpd/httpd/trunk/server/mpm/event/event.c
>httpd/httpd/trunk/server/mpm_unix.c
>httpd/httpd/trunk/server/request.c
> 
> Modified: httpd/httpd/trunk/docs/log-message-tags/next-number
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/log-message-tags/next-number?rev=1783759&r1=1783758&r2=1783759&view=diff
> ==
> --- httpd/httpd/trunk/docs/log-message-tags/next-number (original)
> +++ httpd/httpd/trunk/docs/log-message-tags/next-number Mon Feb 20 13:50:49 
> 2017
> @@ -1 +1 @@
> -10003
> +10004
> 
> Modified: httpd/httpd/trunk/include/mod_core.h
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/include/mod_core.h?rev=1783759&r1=1783758&r2=1783759&view=diff
> ==
> --- httpd/httpd/trunk/include/mod_core.h (original)
> +++ httpd/httpd/trunk/include/mod_core.h Mon Feb 20 13:50:49 2017
> @@ -45,6 +45,10 @@ AP_DECLARE_DATA extern ap_filter_rec_t *
> AP_DECLARE_DATA extern ap_filter_rec_t *ap_http_outerror_filter_handle;
> AP_DECLARE_DATA extern ap_filter_rec_t *ap_byterange_filter_handle;
> 
> +typedef struct {
> +apr_bucket_brigade *tmp_bb;
> +} http_conn_config;
> +
> /*
>  * These (input) filters are internal to the mod_core operation.
>  */
> 
> Modified: httpd/httpd/trunk/modules/filters/mod_include.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/mod_include.c?rev=1783759&r1=1783758&r2=1783759&view=diff
> ==
> --- httpd/httpd/trunk/modules/filters/mod_include.c (original)
> +++ httpd/httpd/trunk/modules/filters/mod_include.c Mon Feb 20 13:50:49 2017
> @@ -3783,6 +3783,7 @@ static apr_status_t send_parsed_content(
> /* if something's left over, pass it along */
> if (!APR_BRIGADE_EMPTY(pass_bb)) {
> rv = ap_pass_brigade(f->next, pass_bb);
> +apr_brigade_cleanup(pass_bb);
> }
> else {
> rv = APR_SUCCESS;
> 
> Modified: httpd/httpd/trunk/modules/http2/h2_conn.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_conn.c?rev=1783759&r1=1783758&r2=1783759&view=diff
> ==
> --- httpd/httpd/trunk/modules/http2/h2_conn.c (original)
> +++ httpd/httpd/trunk/modules/http2/h2_conn.c Mon Feb 20 13:50:49 2017
> @@ -280,7 +280,7 @@ conn_rec *h2_slave_create(conn_rec *mast
> status = apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT, pool);
> if (status != APR_SUCCESS) {
> ap_log_cerror(APLOG_MARK, APLOG_ERR, status, master, 
> -  APLOGNO() "h2_session(%ld-%d): create slave mutex",
> +  APLOGNO(10004) "h2_session(%ld-%d): create slave 
> mutex",
>   master->id, slave_id);
> apr_pool_destroy(pool);
> return NULL;
> 
> Modified: httpd/httpd/trunk/server/main.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/server/main.c?rev=1783759&r1=1783758&r2=1783759&view=diff
> ==
> --- httpd/httpd/trunk/server/main.c (original)
> +++ httpd/httpd/trunk/server/main.c Mon Feb 20 13:50:49 2017
> @@ -735,6 +735,8 @@ int main(int argc, const char * const ar
> apr_pool_clear(pconf);
> ap_clear_auth_internal();
> 
> +apr_sleep(apr_time_from_sec(10));
> +
> ap_main_state = AP_SQ_MS_CREATE_CONFIG;
> ap_config_generation++;
> for (mod = ap_prelinked_modules; *mod != NULL; mod++) {
> 
> Modified: httpd/httpd/trunk/server/mpm/event/event.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/server/mpm/event/event.c?rev=1783759&r1=1783758&r2=1783759&view=diff
> ==
> --- httpd/httpd/trunk/server/mpm/event/event.c (original)
> +++ httpd/httpd/trunk/server/mpm/event/event.c Mon Feb 20 13:50:49 2017
> @@ -2135,7 +2135,7 @@ static void * APR_THREAD_FUNC listener_t
>  ptrans);
> if (rc != APR_SUCCESS) {
> ap_log_error(APLOG_MARK, APLOG_CRIT, rc,
> - ap_server_conf, APLOGNO()
> + ap_serve

Re: svn commit: r1790620 - /httpd/httpd/trunk/modules/http2/mod_http2.dsp

[ste...@eissing.org]

Thanks!

> Am 07.04.2017 um 22:05 schrieb jaillet...@apache.org:
> 
> Author: jailletc36
> Date: Fri Apr  7 20:05:29 2017
> New Revision: 1790620
> 
> URL: http://svn.apache.org/viewvc?rev=1790620&view=rev
> Log:
> h2_worker.c is no more, so remove reference to this file.
> 
> Modified:
>httpd/httpd/trunk/modules/http2/mod_http2.dsp
> 
> Modified: httpd/httpd/trunk/modules/http2/mod_http2.dsp
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/mod_http2.dsp?rev=1790620&r1=1790619&r2=1790620&view=diff
> ==
> --- httpd/httpd/trunk/modules/http2/mod_http2.dsp (original)
> +++ httpd/httpd/trunk/modules/http2/mod_http2.dsp Fri Apr  7 20:05:29 2017
> @@ -181,10 +181,6 @@ SOURCE=./h2_util.c
> # End Source File
> # Begin Source File
> 
> -SOURCE=./h2_worker.c
> -# End Source File
> -# Begin Source File
> -
> SOURCE=./h2_workers.c
> # End Source File
> # Begin Source File
> 
> 

ssl infrastructure

[ste...@eissing.org]

As you might know, I have started writing an Apache module to bring SSL using 
the Rust based rustls library to the server. Currently, I have base connection 
filter working and can serve requests. Nothing fancy, but the Rust lib is 
loaded and does the job.

My goal for this project is to have a module that will co-exist with mod_ssl, 
because:
- rustls is not openssl in terms of functions and tweaks it supports. There 
will be many people that rely on those or have backends with older TLS versions 
which rustls cannot serve.
- Some people might want to use this selectively on some (maybe the publicly 
exposed) ports of their installations.

So, it will _not_ pretend to be mod_ssl. It will have its own configuration 
directives. Which also means it will not provide sneaky optional functions, 
such as:
  ssl_var_lookup
  ssl_is_https
  ssl_proxy_enable
  ssl_engine_disable
  ssl_engine_set

or have hooks like;
  add_cert_files
  add_fallback_cert_files

The obvious thing how we could provide for this is to offer these functions and 
hooks in the core server. There would be
  ap_ssl_var_lookup
  ap_is_https
  ap_ssl_proxy_enable
  etc.

implemented as hooks where mod_ssl and mod_tls would register and give 
information on their respective connections or on the proxy settings they are 
configured with. mod_ssl would continue to offer its optional functions as it 
does now. But the uses of these in our own modules we would adapt.

This would be nothing specific for the module I write, but available to anyone 
who wants to provide a SSL implementation. It might be  a way ahead for a 
mod_ssl2 that gets rid of old openssl APIs and SSL3xxx things. Or it might be 
that someone writes a mod_ressl without needing to listen to its faked openssl 
version defines etc.

Does this sound like a good development for httpd? Are there alternatives or 
risks that I have missed? Would very much like to hear from you.

Cheers and have a nice weekend,

Stefan

Re: svn commit: r1887390 - /httpd/httpd/branches/2.4.x/modules/http/http_etag.c

[ste...@eissing.org]

Ha! Misery loves company, it seems... ;-)

More constructive: with modern computers that fast, should we add a dependency 
of log-tags to the default build targets? or at least "install" or somesuch?

The machines are made to serve!

> Am 09.03.2021 um 19:15 schrieb yla...@apache.org:
> 
> Author: ylavic
> Date: Tue Mar  9 18:15:22 2021
> New Revision: 1887390
> 
> URL: http://svn.apache.org/viewvc?rev=1887390&view=rev
> Log:
> Fix APLOGNO(10258) missing from r1887384.
> 
> Modified:
>httpd/httpd/branches/2.4.x/modules/http/http_etag.c
> 
> Modified: httpd/httpd/branches/2.4.x/modules/http/http_etag.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/modules/http/http_etag.c?rev=1887390&r1=1887389&r2=1887390&view=diff
> ==
> --- httpd/httpd/branches/2.4.x/modules/http/http_etag.c (original)
> +++ httpd/httpd/branches/2.4.x/modules/http/http_etag.c Tue Mar  9 18:15:22 
> 2021
> @@ -133,7 +133,7 @@ static char *make_digest_etag(request_re
> }
> 
> if ((status = apr_file_seek(fd, APR_END, &len)) != APR_SUCCESS) {
> -ap_log_rerror(APLOG_MARK, APLOG_ERR, status, r, APLOGNO()
> +ap_log_rerror(APLOG_MARK, APLOG_ERR, status, r, APLOGNO(10258)
>   "Make etag: could not seek");
> if (er->pathname) {
> apr_file_close(fd);
> 
> 

Re: svn commit: r1889049 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

Thanks, jailletc, as of the changes from yesterday, the PR should be "cleaner".

Btw. can we add the .gitignore to our branches/2.4.x so it gets synched?

> Am 21.04.2021 um 11:22 schrieb jaillet...@apache.org:
> 
> Author: jailletc36
> Date: Wed Apr 21 09:22:11 2021
> New Revision: 1889049
> 
> URL: http://svn.apache.org/viewvc?rev=1889049&view=rev
> Log:
> Add a comment on a backport proposal
> 
> [skip ci]
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1889049&r1=1889048&r2=1889049&view=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Wed Apr 21 09:22:11 2021
> @@ -171,6 +171,7 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>  +1: icing, ylavic
>  ylavic: some "<<< HEAD" hunk in ap_mmn.h should be remove on
>  backport..
> +jailletc36: pull/179 has been updated with r1889009
> 
>   *) mod_socache_shmcb: be safe from socache_shmcb_destroy() late call.  PR 
> 59798
>  trunk patch: http://svn.apache.org/r1888266
> 
> 

Re: svn commit: r1891730 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

Will throw this into my testsuite shredder today. Thank Yann, for bringing this 
in!

> Am 22.07.2021 um 19:35 schrieb yla...@apache.org:
> 
> Author: ylavic
> Date: Thu Jul 22 17:35:21 2021
> New Revision: 1891730
> 
> URL: http://svn.apache.org/viewvc?rev=1891730&view=rev
> Log:
> Propose [skip ci].
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1891730&r1=1891729&r2=1891730&view=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Thu Jul 22 17:35:21 2021
> @@ -247,6 +247,31 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>   
> https://patch-diff.githubusercontent.com/raw/apache/httpd/pull/194.patch
>  +1: jfclere:
> 
> +  *) mod_proxy: Fix icomplete initialization of BalancerMember(s) from the
> +balancer-manager, which can lead to a crash.
> + trunk patch: http://svn.apache.org/r1891477
> +  http://svn.apache.org/r1891591
> + 2.4.x patch: svn merge -c 1891477,1891591 ^/httpd/httpd/trunk .
> + +1: ylavic,
> +
> +  *) mpm_event: Fix graceful stop/restart of children processes if 
> connections
> +are in lingering close for too long.
> + trunk patch: http://svn.apache.org/r1891716
> +  http://svn.apache.org/r1891717
> +  http://svn.apache.org/r1891718
> +  http://svn.apache.org/r1891719
> +  http://svn.apache.org/r1891721
> +  http://svn.apache.org/r1891724
> +  http://svn.apache.org/r1891726
> +  http://svn.apache.org/r1891727
> +  http://svn.apache.org/r1891728
> + backport: wget 
> http://people.apache.org/~ylavic/patches/2.4.x-mpm_event_graceful_linger.patch
> + + patch -p0 < 2.4.x-mpm_event_graceful_linger.patch
> + + svn merge --record-only -c 
> 1891716-1891719,1891721,1891724,1891726-1891727 ^/httpd/httpd/trunk .
> + + svn merge -c 1891728 ^/httpd/httpd/trunk .
> + + make update-changes
> + +1: ylavic,
> +
> 
> PATCHES/ISSUES THAT ARE BEING WORKED
>   [ New entries should be added at the START of the list ]
> 
> 

Re: http2 test suite

[ste...@eissing.org]

> Am 20.08.2021 um 13:46 schrieb Joe Orton :
> 
> On Fri, Aug 20, 2021 at 11:35:45AM +0200, Stefan Eissing wrote:
>> https://github.com/apache/httpd/pull/260 
>> a PR with the http2 test suite in trunk/test/modules/http2
>> 
>> How to use:
>> 
>> • run configure again after you checked this out
>> • the following components need to be installed on your system:
>>   • python3, pytest
>>   • curl, nghttp, h2load
>> 
>> run the tests:
>>> make install
>>> cd test
>>> pytest
> 
> Awesome!

Thanks!

>> This starts the installed httpd on local ports 
>> 40001 + 40002, runs the test suite and tears it down again. 
>> To run individual test cases, use
>> 
>>> cd test/modules/http2
>>> pytest -k test_004   # run all tests in test_004_post.py
>>> pytest -k test_004_07# run test 07 in test_004_post.py
>> 
>> Next would be the definition to run this in a Docker file via 
>> Travis. If someone familiar with that setup could help me to 
>> start this?
> 
> Does this need to run in Docker, or can it run directly in a Linux vm?  

For me, it runs directly on MacOS and I believe it should also on any linux
with the prerequisites installed.

> It would make sense to me to add separate job in Travis for this which 
> has the right Debian packages installed, and adjust test/travis*.sh to 
> make run the tests in a similar way to how TEST_SSL etc work.

That is probably a good start. When this works reliably, we may add
a flag to the common linux script to run it. The additional packages
should not be really a burden for docker, I believe.

>> PS. I made a PR to not disturb our existing travis setup, but if
>> trunk is the better place to refine this, just say so.
> 
> FWIW I'd say this is mostly personal preference, unless you expect to 
> break trunk and hold up others working there, it's always fine to work 
> on trunk. You get the cost/benefit of Travis for your changes either 
> way.

Ok, will merge it to trunk later today or quite soon.

> 
> Regards, Joe

Re: http2 test suite

[ste...@eissing.org]

Done in r1892476. Looking forward on how this works on your machines.

- Stefan

> Am 20.08.2021 um 13:50 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 20.08.2021 um 13:46 schrieb Joe Orton :
>> 
>> On Fri, Aug 20, 2021 at 11:35:45AM +0200, Stefan Eissing wrote:
>>> https://github.com/apache/httpd/pull/260 
>>> a PR with the http2 test suite in trunk/test/modules/http2
>>> 
>>> How to use:
>>> 
>>> • run configure again after you checked this out
>>> • the following components need to be installed on your system:
>>>  • python3, pytest
>>>  • curl, nghttp, h2load
>>> 
>>> run the tests:
>>>> make install
>>>> cd test
>>>> pytest
>> 
>> Awesome!
> 
> Thanks!
> 
>>> This starts the installed httpd on local ports 
>>> 40001 + 40002, runs the test suite and tears it down again. 
>>> To run individual test cases, use
>>> 
>>>> cd test/modules/http2
>>>> pytest -k test_004   # run all tests in test_004_post.py
>>>> pytest -k test_004_07# run test 07 in test_004_post.py
>>> 
>>> Next would be the definition to run this in a Docker file via 
>>> Travis. If someone familiar with that setup could help me to 
>>> start this?
>> 
>> Does this need to run in Docker, or can it run directly in a Linux vm?  
> 
> For me, it runs directly on MacOS and I believe it should also on any linux
> with the prerequisites installed.
> 
>> It would make sense to me to add separate job in Travis for this which 
>> has the right Debian packages installed, and adjust test/travis*.sh to 
>> make run the tests in a similar way to how TEST_SSL etc work.
> 
> That is probably a good start. When this works reliably, we may add
> a flag to the common linux script to run it. The additional packages
> should not be really a burden for docker, I believe.
> 
>>> PS. I made a PR to not disturb our existing travis setup, but if
>>> trunk is the better place to refine this, just say so.
>> 
>> FWIW I'd say this is mostly personal preference, unless you expect to 
>> break trunk and hold up others working there, it's always fine to work 
>> on trunk. You get the cost/benefit of Travis for your changes either 
>> way.
> 
> Ok, will merge it to trunk later today or quite soon.
> 
>> 
>> Regards, Joe

Re: http2 test suite

[ste...@eissing.org]

The http2 test suites in https://github.com/icing/mod_h2 and svn trunk are 
now nearly identical. I made some small adjustment and also have them
running in debian sid and archlinux docker images (make docker-test in
the github repro).

Cheers, Stefan

> Am 20.08.2021 um 18:08 schrieb ste...@eissing.org:
> 
> Done in r1892476. Looking forward on how this works on your machines.
> 
> - Stefan
> 
>> Am 20.08.2021 um 13:50 schrieb ste...@eissing.org:
>> 
>> 
>> 
>>> Am 20.08.2021 um 13:46 schrieb Joe Orton :
>>> 
>>> On Fri, Aug 20, 2021 at 11:35:45AM +0200, Stefan Eissing wrote:
>>>> https://github.com/apache/httpd/pull/260 
>>>> a PR with the http2 test suite in trunk/test/modules/http2
>>>> 
>>>> How to use:
>>>> 
>>>> • run configure again after you checked this out
>>>> • the following components need to be installed on your system:
>>>> • python3, pytest
>>>> • curl, nghttp, h2load
>>>> 
>>>> run the tests:
>>>>> make install
>>>>> cd test
>>>>> pytest
>>> 
>>> Awesome!
>> 
>> Thanks!
>> 
>>>> This starts the installed httpd on local ports 
>>>> 40001 + 40002, runs the test suite and tears it down again. 
>>>> To run individual test cases, use
>>>> 
>>>>> cd test/modules/http2
>>>>> pytest -k test_004   # run all tests in test_004_post.py
>>>>> pytest -k test_004_07# run test 07 in test_004_post.py
>>>> 
>>>> Next would be the definition to run this in a Docker file via 
>>>> Travis. If someone familiar with that setup could help me to 
>>>> start this?
>>> 
>>> Does this need to run in Docker, or can it run directly in a Linux vm?  
>> 
>> For me, it runs directly on MacOS and I believe it should also on any linux
>> with the prerequisites installed.
>> 
>>> It would make sense to me to add separate job in Travis for this which 
>>> has the right Debian packages installed, and adjust test/travis*.sh to 
>>> make run the tests in a similar way to how TEST_SSL etc work.
>> 
>> That is probably a good start. When this works reliably, we may add
>> a flag to the common linux script to run it. The additional packages
>> should not be really a burden for docker, I believe.
>> 
>>>> PS. I made a PR to not disturb our existing travis setup, but if
>>>> trunk is the better place to refine this, just say so.
>>> 
>>> FWIW I'd say this is mostly personal preference, unless you expect to 
>>> break trunk and hold up others working there, it's always fine to work 
>>> on trunk. You get the cost/benefit of Travis for your changes either 
>>> way.
>> 
>> Ok, will merge it to trunk later today or quite soon.
>> 
>>> 
>>> Regards, Joe
> 

Re: http2 test suite

[ste...@eissing.org]

> Am 23.08.2021 um 14:40 schrieb Joe Orton :
> 
> On Mon, Aug 23, 2021 at 12:59:19PM +0200, ste...@eissing.org wrote:
>> The http2 test suites in https://github.com/icing/mod_h2 and svn trunk are 
>> now nearly identical. I made some small adjustment and also have them
>> running in debian sid and archlinux docker images (make docker-test in
>> the github repro).
> 
> It works on my machine with r1892545, which I hope does not break things 
> on your machine(s)?

Nice! Don't see a problem with that change.

> Started an attempt at getting Travis working here: 
> https://github.com/apache/httpd/pull/262
> 
> ... but it's quite likely to need some tweaks to the installed package 
> set to get working at minimum.


In my Dockerfiles on https://github.com/icing/mod_h2/tree/master/docker I
install everything (plus the apach2 package). Maybe that helps.

- Stefan

> 
> Regards, Joe
> 
> 
>> 
>> Cheers, Stefan
>> 
>>> Am 20.08.2021 um 18:08 schrieb ste...@eissing.org:
>>> 
>>> Done in r1892476. Looking forward on how this works on your machines.
>>> 
>>> - Stefan
>>> 
>>>> Am 20.08.2021 um 13:50 schrieb ste...@eissing.org:
>>>> 
>>>> 
>>>> 
>>>>> Am 20.08.2021 um 13:46 schrieb Joe Orton :
>>>>> 
>>>>> On Fri, Aug 20, 2021 at 11:35:45AM +0200, Stefan Eissing wrote:
>>>>>> https://github.com/apache/httpd/pull/260 
>>>>>> a PR with the http2 test suite in trunk/test/modules/http2
>>>>>> 
>>>>>> How to use:
>>>>>> 
>>>>>> • run configure again after you checked this out
>>>>>> • the following components need to be installed on your system:
>>>>>> • python3, pytest
>>>>>> • curl, nghttp, h2load
>>>>>> 
>>>>>> run the tests:
>>>>>>> make install
>>>>>>> cd test
>>>>>>> pytest
>>>>> 
>>>>> Awesome!
>>>> 
>>>> Thanks!
>>>> 
>>>>>> This starts the installed httpd on local ports 
>>>>>> 40001 + 40002, runs the test suite and tears it down again. 
>>>>>> To run individual test cases, use
>>>>>> 
>>>>>>> cd test/modules/http2
>>>>>>> pytest -k test_004   # run all tests in test_004_post.py
>>>>>>> pytest -k test_004_07# run test 07 in test_004_post.py
>>>>>> 
>>>>>> Next would be the definition to run this in a Docker file via 
>>>>>> Travis. If someone familiar with that setup could help me to 
>>>>>> start this?
>>>>> 
>>>>> Does this need to run in Docker, or can it run directly in a Linux vm?  
>>>> 
>>>> For me, it runs directly on MacOS and I believe it should also on any linux
>>>> with the prerequisites installed.
>>>> 
>>>>> It would make sense to me to add separate job in Travis for this which 
>>>>> has the right Debian packages installed, and adjust test/travis*.sh to 
>>>>> make run the tests in a similar way to how TEST_SSL etc work.
>>>> 
>>>> That is probably a good start. When this works reliably, we may add
>>>> a flag to the common linux script to run it. The additional packages
>>>> should not be really a burden for docker, I believe.
>>>> 
>>>>>> PS. I made a PR to not disturb our existing travis setup, but if
>>>>>> trunk is the better place to refine this, just say so.
>>>>> 
>>>>> FWIW I'd say this is mostly personal preference, unless you expect to 
>>>>> break trunk and hold up others working there, it's always fine to work 
>>>>> on trunk. You get the cost/benefit of Travis for your changes either 
>>>>> way.
>>>> 
>>>> Ok, will merge it to trunk later today or quite soon.
>>>> 
>>>>> 
>>>>> Regards, Joe
>>> 
>> 
> 

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 13:53 schrieb Yann Ylavic :
> 
> On Wed, Aug 25, 2021 at 1:14 AM Travis CI  wrote:
>> 
>> apache / httpd
>> 
>> trunk
>> 
>> Build #1831 was broken
>> 21 mins and 33 secs
>> Yann Ylavic
>> 243c5fa CHANGESET →
>> 
>> mpm_{event,worker,prefork}: late stop of children processes on restart.
> 
> (unrelated to this change, Build #1819 failed the same earlier).
> 
> Here: https://app.travis-ci.com/github/apache/httpd/jobs/533578536#L4103
> 
> It seems that when exiting a stream can be destroyed while in
>  h2_mplx_s_task_done::s_task_done()::mst_check_data_for()
> when the mplx lock is released (thus stream->id faults).
> 
> Should the caller(s) of mst_check_data_for() pass stream->id (under
> the lock) instead?
> This would avoid the fault but that's still a potentially destroyed
> stream being fifo'ed, though we are exiting so it might not matter..
> 
> A better fix maybe, Stefan?

I think the "stream->id" needs to be save before giving up the lock. 
I'll make the change, but currently my test suite throws several 
other errors in trunk. Something changed...

Btw. I am working on a real fix of this messy code in http2, using 
a apr_pollset for monitoring connections and ongoing requests. Also
cleaning up other parts of the code, reducing complexity.



I'll see what version of trunk still runs my tests and report back.

- Stefan

> 
> Cheers;
> Yann.

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 15:06 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 25.08.2021 um 13:53 schrieb Yann Ylavic :
>> 
>> On Wed, Aug 25, 2021 at 1:14 AM Travis CI  wrote:
>>> 
>>> apache / httpd
>>> 
>>> trunk
>>> 
>>> Build #1831 was broken
>>> 21 mins and 33 secs
>>> Yann Ylavic
>>> 243c5fa CHANGESET →
>>> 
>>> mpm_{event,worker,prefork}: late stop of children processes on restart.
>> 
>> (unrelated to this change, Build #1819 failed the same earlier).
>> 
>> Here: https://app.travis-ci.com/github/apache/httpd/jobs/533578536#L4103
>> 
>> It seems that when exiting a stream can be destroyed while in
>> h2_mplx_s_task_done::s_task_done()::mst_check_data_for()
>> when the mplx lock is released (thus stream->id faults).
>> 
>> Should the caller(s) of mst_check_data_for() pass stream->id (under
>> the lock) instead?
>> This would avoid the fault but that's still a potentially destroyed
>> stream being fifo'ed, though we are exiting so it might not matter..
>> 
>> A better fix maybe, Stefan?
> 
> I think the "stream->id" needs to be save before giving up the lock. 
> I'll make the change, but currently my test suite throws several 
> other errors in trunk. Something changed...
> 
> Btw. I am working on a real fix of this messy code in http2, using 
> a apr_pollset for monitoring connections and ongoing requests. Also
> cleaning up other parts of the code, reducing complexity.
> 
> <https://github.com/icing/mod_h2/tree/icing/pipes>
> 
> I'll see what version of trunk still runs my tests and report back.


Ok, r1892586 still runs the http2 tests nicely. Seems that r1892587 
broke things. Note that the http2 test suite does a lot of server
reloads.

Stefan

> 
> - Stefan
> 
>> 
>> Cheers;
>> Yann.

Re: http2 test suite

[ste...@eissing.org]

> Am 25.08.2021 um 15:30 schrieb Joe Orton :
> 
> On Mon, Aug 23, 2021 at 03:03:34PM +0200, ste...@eissing.org wrote:
>> 
>> 
>>> Am 23.08.2021 um 14:40 schrieb Joe Orton :
>>> 
>>> On Mon, Aug 23, 2021 at 12:59:19PM +0200, ste...@eissing.org wrote:
>>>> The http2 test suites in https://github.com/icing/mod_h2 and svn trunk are 
>>>> now nearly identical. I made some small adjustment and also have them
>>>> running in debian sid and archlinux docker images (make docker-test in
>>>> the github repro).
>>> 
>>> It works on my machine with r1892545, which I hope does not break things 
>>> on your machine(s)?
>> 
>> Nice! Don't see a problem with that change.
>> 
>>> Started an attempt at getting Travis working here: 
>>> https://github.com/apache/httpd/pull/262
>>> 
>>> ... but it's quite likely to need some tweaks to the installed package 
>>> set to get working at minimum.
>> 
>> 
>> In my Dockerfiles on https://github.com/icing/mod_h2/tree/master/docker I
>> install everything (plus the apach2 package). Maybe that helps.
> 
> I'm not sure whether all the packages in that are really needed, e.g. 
> tqdm and pycurl?
> 

No, I use tqdm in my load scenarios, not needed to the test suite itself.

> This is as far as I got so far in Travis: 
> 
> https://app.travis-ci.com/github/apache/httpd/builds/236173863#L2985

This looks like the recent "reload" optimization, as I see similar failures now 
on my machine.

> Any idea what causes those failures?
> 
> There is just one change to test/modules/http2 which I haven't pushed to 
> trunk, is this OK?
> 
> https://github.com/notroj/httpd/commit/72a037003cae0c8c634313d0707311222fe237e0

Looks fine.

> 
> Regards, Joe
> 
> 
>> 
>> - Stefan
>> 
>>> 
>>> Regards, Joe
>>> 
>>> 
>>>> 
>>>> Cheers, Stefan
>>>> 
>>>>> Am 20.08.2021 um 18:08 schrieb ste...@eissing.org:
>>>>> 
>>>>> Done in r1892476. Looking forward on how this works on your machines.
>>>>> 
>>>>> - Stefan
>>>>> 
>>>>>> Am 20.08.2021 um 13:50 schrieb ste...@eissing.org:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> Am 20.08.2021 um 13:46 schrieb Joe Orton :
>>>>>>> 
>>>>>>> On Fri, Aug 20, 2021 at 11:35:45AM +0200, Stefan Eissing wrote:
>>>>>>>> https://github.com/apache/httpd/pull/260 
>>>>>>>> a PR with the http2 test suite in trunk/test/modules/http2
>>>>>>>> 
>>>>>>>> How to use:
>>>>>>>> 
>>>>>>>> • run configure again after you checked this out
>>>>>>>> • the following components need to be installed on your system:
>>>>>>>> • python3, pytest
>>>>>>>> • curl, nghttp, h2load
>>>>>>>> 
>>>>>>>> run the tests:
>>>>>>>>> make install
>>>>>>>>> cd test
>>>>>>>>> pytest
>>>>>>> 
>>>>>>> Awesome!
>>>>>> 
>>>>>> Thanks!
>>>>>> 
>>>>>>>> This starts the installed httpd on local ports 
>>>>>>>> 40001 + 40002, runs the test suite and tears it down again. 
>>>>>>>> To run individual test cases, use
>>>>>>>> 
>>>>>>>>> cd test/modules/http2
>>>>>>>>> pytest -k test_004   # run all tests in test_004_post.py
>>>>>>>>> pytest -k test_004_07# run test 07 in test_004_post.py
>>>>>>>> 
>>>>>>>> Next would be the definition to run this in a Docker file via 
>>>>>>>> Travis. If someone familiar with that setup could help me to 
>>>>>>>> start this?
>>>>>>> 
>>>>>>> Does this need to run in Docker, or can it run directly in a Linux vm?  
>>>>>> 
>>>>>> For me, it runs directly on MacOS and I believe it should also on any 
>>>>>> linux
>>>>>> with the prerequisites installed.
>>>>>> 
>>>>>>> It would make sense to me to add separate job in Travis for this which 
>>>>>>> has the right Debian packages installed, and adjust test/travis*.sh to 
>>>>>>> make run the tests in a similar way to how TEST_SSL etc work.
>>>>>> 
>>>>>> That is probably a good start. When this works reliably, we may add
>>>>>> a flag to the common linux script to run it. The additional packages
>>>>>> should not be really a burden for docker, I believe.
>>>>>> 
>>>>>>>> PS. I made a PR to not disturb our existing travis setup, but if
>>>>>>>> trunk is the better place to refine this, just say so.
>>>>>>> 
>>>>>>> FWIW I'd say this is mostly personal preference, unless you expect to 
>>>>>>> break trunk and hold up others working there, it's always fine to work 
>>>>>>> on trunk. You get the cost/benefit of Travis for your changes either 
>>>>>>> way.
>>>>>> 
>>>>>> Ok, will merge it to trunk later today or quite soon.
>>>>>> 
>>>>>>> 
>>>>>>> Regards, Joe

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 15:15 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 25.08.2021 um 15:06 schrieb ste...@eissing.org:
>> 
>> 
>> 
>>> Am 25.08.2021 um 13:53 schrieb Yann Ylavic :
>>> 
>>> On Wed, Aug 25, 2021 at 1:14 AM Travis CI  wrote:
>>>> 
>>>> apache / httpd
>>>> 
>>>> trunk
>>>> 
>>>> Build #1831 was broken
>>>> 21 mins and 33 secs
>>>> Yann Ylavic
>>>> 243c5fa CHANGESET →
>>>> 
>>>> mpm_{event,worker,prefork}: late stop of children processes on restart.
>>> 
>>> (unrelated to this change, Build #1819 failed the same earlier).
>>> 
>>> Here: https://app.travis-ci.com/github/apache/httpd/jobs/533578536#L4103
>>> 
>>> It seems that when exiting a stream can be destroyed while in
>>> h2_mplx_s_task_done::s_task_done()::mst_check_data_for()
>>> when the mplx lock is released (thus stream->id faults).
>>> 
>>> Should the caller(s) of mst_check_data_for() pass stream->id (under
>>> the lock) instead?
>>> This would avoid the fault but that's still a potentially destroyed
>>> stream being fifo'ed, though we are exiting so it might not matter..
>>> 
>>> A better fix maybe, Stefan?
>> 
>> I think the "stream->id" needs to be save before giving up the lock. 
>> I'll make the change, but currently my test suite throws several 
>> other errors in trunk. Something changed...
>> 
>> Btw. I am working on a real fix of this messy code in http2, using 
>> a apr_pollset for monitoring connections and ongoing requests. Also
>> cleaning up other parts of the code, reducing complexity.
>> 
>> <https://github.com/icing/mod_h2/tree/icing/pipes>
>> 
>> I'll see what version of trunk still runs my tests and report back.
> 
> 
> Ok, r1892586 still runs the http2 tests nicely. Seems that r1892587 
> broke things. Note that the http2 test suite does a lot of server
> reloads.

Follow up: if I do a "time.sleep(2)" after each "apachectl" in my 
test suite, it works fine.

Speculating: my test suite does commonly a GET after a reload to 
see if the reloaded server is responding. And then fires requests
against the new configuration.

Now, if the server is always responding, the test sill get - sometimes -
send request to an old gen child which then answers incorrectly. So
I see 404 responses where a 200 should be as the config does not match.

In this new world of always responding servers, when is a test suite
supposed to know that now the server will respond with the newly laoded
config?

- Stefan

> 
> Stefan
> 
>> 
>> - Stefan
>> 
>>> 
>>> Cheers;
>>> Yann.

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 15:54 schrieb Yann Ylavic :
> 
> On Wed, Aug 25, 2021 at 3:37 PM ste...@eissing.org  wrote:
>> 
>> Follow up: if I do a "time.sleep(2)" after each "apachectl" in my
>> test suite, it works fine.
>> 
>> Speculating: my test suite does commonly a GET after a reload to
>> see if the reloaded server is responding. And then fires requests
>> against the new configuration.
>> 
>> Now, if the server is always responding, the test sill get - sometimes -
>> send request to an old gen child which then answers incorrectly. So
>> I see 404 responses where a 200 should be as the config does not match.
>> 
>> In this new world of always responding servers, when is a test suite
>> supposed to know that now the server will respond with the newly laoded
>> config?
> 
> Maybe we could have access to the generation number
> (AP_MPMQ_GENERATION) in a variable like "MPM_GENERATION" which
> mod_headers could use to set in a response header?
> Or a test only module which does that specifically could be compiled
> by the framework and LoadModule'd?

I could add that in the test config as header to a certain location.

However, will all new connections get to new gen children once the
new number has been spotted?

> 
> Cheers;
> Yann.

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 16:04 schrieb Yann Ylavic :
> 
> On Wed, Aug 25, 2021 at 3:56 PM ste...@eissing.org  wrote:
>> 
>> 
>> 
>>> Am 25.08.2021 um 15:54 schrieb Yann Ylavic :
>>> 
>>> On Wed, Aug 25, 2021 at 3:37 PM ste...@eissing.org  
>>> wrote:
>>>> 
>>>> Follow up: if I do a "time.sleep(2)" after each "apachectl" in my
>>>> test suite, it works fine.
>>>> 
>>>> Speculating: my test suite does commonly a GET after a reload to
>>>> see if the reloaded server is responding. And then fires requests
>>>> against the new configuration.
>>>> 
>>>> Now, if the server is always responding, the test sill get - sometimes -
>>>> send request to an old gen child which then answers incorrectly. So
>>>> I see 404 responses where a 200 should be as the config does not match.
>>>> 
>>>> In this new world of always responding servers, when is a test suite
>>>> supposed to know that now the server will respond with the newly laoded
>>>> config?
>>> 
>>> Maybe we could have access to the generation number
>>> (AP_MPMQ_GENERATION) in a variable like "MPM_GENERATION" which
>>> mod_headers could use to set in a response header?
>>> Or a test only module which does that specifically could be compiled
>>> by the framework and LoadModule'd?
>> 
>> I could add that in the test config as header to a certain location.
>> 
>> However, will all new connections get to new gen children once the
>> new number has been spotted?
> 
> Yes, that's guaranteed by the MPM which increases the gen number just
> before creating new children (after having asked the old ones to
> stop), no old child will ever be created with a new generation number.
> ap_mpm_query(AP_MPMQ_GENERATION, &generation) within a child will
> always return the one that was forked.

Thanks, I'll give this a shot. 

Joe: as long as that is not solved, the tests will not be reliable, I'm afraid.

- Stefan

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 16:16 schrieb Yann Ylavic :
> 
> On Wed, Aug 25, 2021 at 4:04 PM Yann Ylavic  wrote:
>> 
>> On Wed, Aug 25, 2021 at 3:56 PM ste...@eissing.org  
>> wrote:
>>> 
>>> I could add that in the test config as header to a certain location.
>>> 
>>> However, will all new connections get to new gen children once the
>>> new number has been spotted?
>> 
>> Yes, that's guaranteed by the MPM which increases the gen number just
>> before creating new children (after having asked the old ones to
>> stop), no old child will ever be created with a new generation number.
>> ap_mpm_query(AP_MPMQ_GENERATION, &generation) within a child will
>> always return the one that was forked.
> 
> Hmm, you asked for a different question actually, the new
> *connections* done after the gen bump are not guaranteed to be handled
> by a new gen child, because it may take some (short) time before the
> old gen listeners stop listening.
> There is no way to guarantee where connections will go in the first
> times of a restart I'm afraid.
> At least your test client can know whether it's handled by an old or
> new gen and drop the connection in the former case..

Thanks for confirming what I suspected. Concurrency is a bitch.

Hmm, hate to add timed waits here, as who knows how Travis will
work with those. Will experiment a bit.

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

Ok, brute-forced it a bit. The test suite now always does
a stop/start cycle on config changes as reload now longer
works reliably. Takes a bit more time, but not overly so
on my machine.

Cheers,
Stefan

> Am 25.08.2021 um 16:22 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 25.08.2021 um 16:16 schrieb Yann Ylavic :
>> 
>> On Wed, Aug 25, 2021 at 4:04 PM Yann Ylavic  wrote:
>>> 
>>> On Wed, Aug 25, 2021 at 3:56 PM ste...@eissing.org  
>>> wrote:
>>>> 
>>>> I could add that in the test config as header to a certain location.
>>>> 
>>>> However, will all new connections get to new gen children once the
>>>> new number has been spotted?
>>> 
>>> Yes, that's guaranteed by the MPM which increases the gen number just
>>> before creating new children (after having asked the old ones to
>>> stop), no old child will ever be created with a new generation number.
>>> ap_mpm_query(AP_MPMQ_GENERATION, &generation) within a child will
>>> always return the one that was forked.
>> 
>> Hmm, you asked for a different question actually, the new
>> *connections* done after the gen bump are not guaranteed to be handled
>> by a new gen child, because it may take some (short) time before the
>> old gen listeners stop listening.
>> There is no way to guarantee where connections will go in the first
>> times of a restart I'm afraid.
>> At least your test client can know whether it's handled by an old or
>> new gen and drop the connection in the former case..
> 
> Thanks for confirming what I suspected. Concurrency is a bitch.
> 
> Hmm, hate to add timed waits here, as who knows how Travis will
> work with those. Will experiment a bit.

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

It should be fixed now with r1892599, I believe.

> Am 25.08.2021 um 13:53 schrieb Yann Ylavic :
> 
> On Wed, Aug 25, 2021 at 1:14 AM Travis CI  wrote:
>> 
>> apache / httpd
>> 
>> trunk
>> 
>> Build #1831 was broken
>> 21 mins and 33 secs
>> Yann Ylavic
>> 243c5fa CHANGESET →
>> 
>> mpm_{event,worker,prefork}: late stop of children processes on restart.
> 
> (unrelated to this change, Build #1819 failed the same earlier).
> 
> Here: https://app.travis-ci.com/github/apache/httpd/jobs/533578536#L4103
> 
> It seems that when exiting a stream can be destroyed while in
>  h2_mplx_s_task_done::s_task_done()::mst_check_data_for()
> when the mplx lock is released (thus stream->id faults).
> 
> Should the caller(s) of mst_check_data_for() pass stream->id (under
> the lock) instead?
> This would avoid the fault but that's still a potentially destroyed
> stream being fifo'ed, though we are exiting so it might not matter..
> 
> A better fix maybe, Stefan?
> 
> Cheers;
> Yann.

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 17:06 schrieb Yann Ylavic :
> 
> Thanks, looks good, it probably needs a backport to 2.4.x since the
> first travis failure was there (Build #1819).
> Let's see how it works after several runs before maybe.

+1

Let's see what Joe and Travis can do to wreck it.

> On Wed, Aug 25, 2021 at 5:02 PM ste...@eissing.org  wrote:
>> 
>> It should be fixed now with r1892599, I believe.
>> 
>>> Am 25.08.2021 um 13:53 schrieb Yann Ylavic :
>>> 
>>> On Wed, Aug 25, 2021 at 1:14 AM Travis CI  wrote:
>>>> 
>>>> apache / httpd
>>>> 
>>>> trunk
>>>> 
>>>> Build #1831 was broken
>>>> 21 mins and 33 secs
>>>> Yann Ylavic
>>>> 243c5fa CHANGESET →
>>>> 
>>>> mpm_{event,worker,prefork}: late stop of children processes on restart.
>>> 
>>> (unrelated to this change, Build #1819 failed the same earlier).
>>> 
>>> Here: https://app.travis-ci.com/github/apache/httpd/jobs/533578536#L4103
>>> 
>>> It seems that when exiting a stream can be destroyed while in
>>> h2_mplx_s_task_done::s_task_done()::mst_check_data_for()
>>> when the mplx lock is released (thus stream->id faults).
>>> 
>>> Should the caller(s) of mst_check_data_for() pass stream->id (under
>>> the lock) instead?
>>> This would avoid the fault but that's still a potentially destroyed
>>> stream being fifo'ed, though we are exiting so it might not matter..
>>> 
>>> A better fix maybe, Stefan?
>>> 
>>> Cheers;
>>> Yann.
>> 

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 17:34 schrieb Joe Orton :
> 
> On Wed, Aug 25, 2021 at 05:08:06PM +0200, ste...@eissing.org wrote:
>> 
>> 
>>> Am 25.08.2021 um 17:06 schrieb Yann Ylavic :
>>> 
>>> Thanks, looks good, it probably needs a backport to 2.4.x since the
>>> first travis failure was there (Build #1819).
>>> Let's see how it works after several runs before maybe.
>> 
>> +1
>> 
>> Let's see what Joe and Travis can do to wreck it.
> 
> Well I could accidentally cancel your builds, how's that for starters?
> 
> (I restarted the most recent one, was trying to cancel the older PR 
> build, but hit the wrong button on the wrong page, sorry!)
> 
> Nice job getting this one fixed.
> 
> Regards, Joe
> 
> p.s. did you forget the APLOGNO()?
> p.p.s just kidding

I see that we are getting into the right spirit here! ;-)

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 25.08.2021 um 17:34 schrieb Joe Orton :
> 
> On Wed, Aug 25, 2021 at 05:08:06PM +0200, ste...@eissing.org wrote:
>> 
>> 
>>> Am 25.08.2021 um 17:06 schrieb Yann Ylavic :
>>> 
>>> Thanks, looks good, it probably needs a backport to 2.4.x since the
>>> first travis failure was there (Build #1819).
>>> Let's see how it works after several runs before maybe.
>> 
>> +1
>> 
>> Let's see what Joe and Travis can do to wreck it.
> 
> Well I could accidentally cancel your builds, how's that for starters?
> 
> (I restarted the most recent one, was trying to cancel the older PR 
> build, but hit the wrong button on the wrong page, sorry!)
> 
> Nice job getting this one fixed.
> 
> Regards, Joe
> 
> p.s. did you forget the APLOGNO()?
> p.p.s just kidding
> 

https://app.travis-ci.com/github/apache/httpd

>   assert env.apache_restart() == 0
3026E   assert 1 == 0
3027E+  where 1 = >()
3028E+where > = .apache_restart
3029
3030test/modules/http2/test_002_curl_basics.py:11: AssertionError
3031 Captured stderr setup 
-
3032WARNING: exit 1, stdout: , stderr: (98)Address already in use: AH00072: 
make_sock: could not bind to address [::]:40001
3033(98)Address already in use: AH00072: make_sock: could not bind to address 
0.0.0.0:40001
3034AH03272: no listening sockets available, shutting down
3035

Looks like the stop/start cycle instead of reload is not timing safe. If httpd 
is no longer responding, it does not mean that the ports are available again 
for the new process. Reloads are just better here, as wait loops can be 
avoided. Hmmthinking...

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 26.08.2021 um 10:16 schrieb Joe Orton :
> 
> On Thu, Aug 26, 2021 at 09:51:07AM +0200, ste...@eissing.org wrote:
>> https://app.travis-ci.com/github/apache/httpd
>> 
>>>  assert env.apache_restart() == 0
>> 3026E   assert 1 == 0
>> 3027E+  where 1 = > >()
>> 3028E+where > > = > 0x7ffb03366a60>.apache_restart
>> 3029
>> 3030test/modules/http2/test_002_curl_basics.py:11: AssertionError
>> 3031 Captured stderr setup 
>> -
>> 3032WARNING: exit 1, stdout: , stderr: (98)Address already in use: AH00072: 
>> make_sock: could not bind to address [::]:40001
>> 3033(98)Address already in use: AH00072: make_sock: could not bind to 
>> address 0.0.0.0:40001
>> 3034AH03272: no listening sockets available, shutting down
>> 3035
>> 
>> Looks like the stop/start cycle instead of reload is not timing safe. 
>> If httpd is no longer responding, it does not mean that the ports are 
>> available again for the new process. Reloads are just better here, as 
>> wait loops can be avoided. Hmmthinking...
> 
> Yeah :( Next build I've pushed has r1892598 reverted and a sleep(2) 
> after running apachectl, that was sufficient for me when I was testing 
> locally before.

Can we add a configuration directive for this server behaviour? Like

AcceptOnGraceful on|off

So that people who do not want the server to accept new connections
during graceful reloads can influence this?

- Stefan

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 26.08.2021 um 10:27 schrieb Ruediger Pluem :
> 
> 
> 
> On 8/26/21 10:21 AM, ste...@eissing.org wrote:
>> 
>> 
>>> Am 26.08.2021 um 10:16 schrieb Joe Orton :
>>> 
>>> On Thu, Aug 26, 2021 at 09:51:07AM +0200, ste...@eissing.org wrote:
>>>> https://app.travis-ci.com/github/apache/httpd
>>>> 
>>>>> assert env.apache_restart() == 0
>>>> 3026E   assert 1 == 0
>>>> 3027E+  where 1 = >>> >()
>>>> 3028E+where >>> > = >>> 0x7ffb03366a60>.apache_restart
>>>> 3029
>>>> 3030test/modules/http2/test_002_curl_basics.py:11: AssertionError
>>>> 3031 Captured stderr setup 
>>>> -
>>>> 3032WARNING: exit 1, stdout: , stderr: (98)Address already in use: 
>>>> AH00072: make_sock: could not bind to address [::]:40001
>>>> 3033(98)Address already in use: AH00072: make_sock: could not bind to 
>>>> address 0.0.0.0:40001
>>>> 3034AH03272: no listening sockets available, shutting down
>>>> 3035
>>>> 
>>>> Looks like the stop/start cycle instead of reload is not timing safe. 
>>>> If httpd is no longer responding, it does not mean that the ports are 
>>>> available again for the new process. Reloads are just better here, as 
>>>> wait loops can be avoided. Hmmthinking...
>>> 
>>> Yeah :( Next build I've pushed has r1892598 reverted and a sleep(2) 
>>> after running apachectl, that was sufficient for me when I was testing 
>>> locally before.
>> 
>> Can we add a configuration directive for this server behaviour? Like
>> 
>> AcceptOnGraceful on|off
>> 
>> So that people who do not want the server to accept new connections
>> during graceful reloads can influence this?
> 
> What about a graceful stop first and a start afterwards?

Just tested: "graceful-stop" seems to make it worse. Connections are 
being served using the old configuration.

> 
> Regards
> 
> Rüdiger
> 

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

Joe: with r1892615 I fixed a bug in waiting for the server to become
unresponsive after a stop. This *should* make timed waits unnecessary
in the http2 test suite.

- Stefan

> Am 26.08.2021 um 10:43 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 26.08.2021 um 10:27 schrieb Ruediger Pluem :
>> 
>> 
>> 
>> On 8/26/21 10:21 AM, ste...@eissing.org wrote:
>>> 
>>> 
>>>> Am 26.08.2021 um 10:16 schrieb Joe Orton :
>>>> 
>>>> On Thu, Aug 26, 2021 at 09:51:07AM +0200, ste...@eissing.org wrote:
>>>>> https://app.travis-ci.com/github/apache/httpd
>>>>> 
>>>>>>assert env.apache_restart() == 0
>>>>> 3026E   assert 1 == 0
>>>>> 3027E+  where 1 = >>>> >()
>>>>> 3028E+where >>>> > = >>>> at 0x7ffb03366a60>.apache_restart
>>>>> 3029
>>>>> 3030test/modules/http2/test_002_curl_basics.py:11: AssertionError
>>>>> 3031 Captured stderr setup 
>>>>> -
>>>>> 3032WARNING: exit 1, stdout: , stderr: (98)Address already in use: 
>>>>> AH00072: make_sock: could not bind to address [::]:40001
>>>>> 3033(98)Address already in use: AH00072: make_sock: could not bind to 
>>>>> address 0.0.0.0:40001
>>>>> 3034AH03272: no listening sockets available, shutting down
>>>>> 3035
>>>>> 
>>>>> Looks like the stop/start cycle instead of reload is not timing safe. 
>>>>> If httpd is no longer responding, it does not mean that the ports are 
>>>>> available again for the new process. Reloads are just better here, as 
>>>>> wait loops can be avoided. Hmmthinking...
>>>> 
>>>> Yeah :( Next build I've pushed has r1892598 reverted and a sleep(2) 
>>>> after running apachectl, that was sufficient for me when I was testing 
>>>> locally before.
>>> 
>>> Can we add a configuration directive for this server behaviour? Like
>>> 
>>> AcceptOnGraceful on|off
>>> 
>>> So that people who do not want the server to accept new connections
>>> during graceful reloads can influence this?
>> 
>> What about a graceful stop first and a start afterwards?
> 
> Just tested: "graceful-stop" seems to make it worse. Connections are 
> being served using the old configuration.
> 
>> 
>> Regards
>> 
>> Rüdiger

Re: Broken: apache/httpd#1831 (trunk - 243c5fa)

[ste...@eissing.org]

> Am 26.08.2021 um 16:44 schrieb Joe Orton :
> 
> On Thu, Aug 26, 2021 at 01:11:16PM +0200, ste...@eissing.org wrote:
>> Joe: with r1892615 I fixed a bug in waiting for the server to become
>> unresponsive after a stop. This *should* make timed waits unnecessary
>> in the http2 test suite.
> 
> Great stuff, thanks for all your work Stefan.  
> 
> It worked locally and once in Travis, merged to trunk in r1892620.

\o/ Now, when I break things, everyone will see it...wait!

server reload responsiveness

[ste...@eissing.org]

I really like the amazing improvement by Yann to keep 
the server responding during reloads! Well done!

What I wonder, hoping that we are able to backport it, 
how this could affect productions. I know from feedback
on h2 that some systems have memory pressure during
reloads and whished for h2 workers to close down asap
on a graceful.

Won't such system not also be affected by the new
behaviour? Do we need to provide a way to opt out
of this?

Cheers,
Stefan

http2 test suite

[ste...@eissing.org]

With the http2 test suite now running in CI (thanks Joe for
all the CI work!), we should also put that into 2.4.x. It
should run there without any changes, as I use it myself
that way.

Related to that, do we exempt "./test" from RTC in 2.4.x/STATUS?

- Stefan

release?

[ste...@eissing.org]

In what state is our release handling? Given someone holding my hand, could I 
do it? Or is it better to look someone over the shoulder while he does it?

Cheers,
Stefan

Re: release?

[ste...@eissing.org]

> Am 30.08.2021 um 22:53 schrieb Christophe JAILLET 
> :
> 
> 
> Le 30/08/2021 à 13:53, Eric Covener a écrit :
>> On Mon, Aug 30, 2021 at 7:36 AM ste...@eissing.org  
>> wrote:
>>> In what state is our release handling? Given someone holding my hand, could 
>>> I do it? Or is it better to look someone over the shoulder while he does it?
>> If there is an over-the-shoulder session I would like to tag along.  I
>> am flexible on time of day but I am GMT+4 (EDT).  I can host on webex.
>> Otherwise if you just want to struggle through it I can tag along but
>> I have no experience.
> 
> I can give another try with my limited experience.
> 
> I definitively would like to add a --dry-run option for all scripts so that 
> they can be run for learning purpose without the fear of un-expected impact 
> on svn.
> 
> Existing scripts are not that easy to read at first, but are understanbdable 
> and following http://httpd.apache.org/dev/release.html#how-to-do-a-release 
> helps a lot. The scripts should also be tweaked because of the latest changes 
> in several places (at least web site update (now on github) and CVE 
> announcement (if any) now that part of the process is handled elsewhere).
> 
> The CVE announcement should be much easier, now that we have a "Send these 
> Emails" on cveprocess.a.o. This should simplify part of the process where we 
> were preparing some scripts to send the announcement emails.
> 
> I've been lacking time for httpd since many weeks, but I should be able to RM 
> next week if needed.

I would like to look over your shoulder/help where I can. Maybe Eric can make a 
WebEx for us - that would make following along much easier, I guess.

Looking at the description link (thanks) I see that there are still a lot of 
"manual" things involved. And a "--dry-run" is definitely a thing we want. Will 
have a look at those scripts in the next days, to see what I can add here.

- Stefan
> 
> CJ
> 
>> Also: Anyone who has a showstopper to delay a release (even if not yet
>> proposed) please add it to 2.4.x STATUS so we can get things in order.
>> 

release scripts

[ste...@eissing.org]

Having a look at the release scripts and how they work. 

I think we can use SVN and its branches/tags in a bit smarter way
and make things easier. As Christophe mentioned, he would like a
"--dry-run" option, because when things do not run smoothly to the
end, you have not only a local messed-up state but the SVN repro
itself is also changed without easy remedies.

I propose to do all work up to release testing in separate,
temporary branches that can simply be removed and restarted.
This would not require a rewrite of the release scripts, but
some re-ordering and branch name changes.

- checkout branches/2.4.x
  - make sure change-entries are merged, commit
  - test branch locally for health
  - svn cp . branches/2.4.49-tmp
- svn switch to branches/2.4.49-tmp
  - update versions, build docs, etc on that
  - commit this into branches/2.4.49-tmp
  - svn mv branches/2.4.49-tmp branches/2.4.49-candidate
- svn switch to branches/2.4.49-candidate
  - create tar files and signature from -candidate
  - make them available for testing
  - announce release voting
#
# everything up to here can be reverted by "svn rm"
#
- switch to branches/2.4.x 
  - bump versions in branches/2.4.x to 2.4.50

- Stefan

-
Summary of what the release process involves and what each script does:
0. tag.sh
   svn checkout $HEAD of branch
   check version numbers
   check SVN tag for version does not exist
   check PGP/GPG key in httpd-dist/KEYS
   update Copyright in docs/manual/style/xsl/common.xsl
   checkout docs-build, build docs, remove docs-build checkout
   run "update-changes" using Makfile.in
   Set AP_SERVER_DEVBUILD_BOOLEAN to 0 in include/ap_release.h
   svn commit all changes
   Change local include/ap_release.h back
   bump up patch version in CHANGES, STATUS, docs/manual/style/version.ent
   before committing this, copy $HEAD of branch to tags/$version
   svn commit the bump-up changes to branch
1a. release.sh
   svn export $HEAD of branches/2.4.x
   svn export of dependencies (APR)
   apply patches (currently broken)
   prepare CHANGES_
   invoke roll.sh
1b. roll.sh
   run releasecheck.sh (checking libtool availability? for what?)
   remove files not wanted in release (fixed list)
   touch .y dependants (fixed list)
   remove doc sources (*.xml *.xsl)
   move srclib into -deps sibling directory
   build tar balls
   build checksum files (lots of alternatives if openssl is not found, do we 
need that?)
   signing the files
   substitute new version number in various files
2. voting
3. push.sh
   checkout https://dist.apache.org/repos/dist/
   remove old CHANGES_$major.minor
   generate new CHANGES_$version, CHANGES_$major.minor
   modify Announcement$major.minor
   svn move generated tar, deps and CHANGES
   svn commit
4. update bugzilla
5. announce.sh
   lots of messing with dist and pmc to get security details in
   some sanity checks
   send mails
   send more mails
   svn commit changes to "$dist_base" "$branch_base" "$private_base" 
"$site_base"

Re: [VOTE] http2 test suite

[ste...@eissing.org]

> Am 01.09.2021 um 11:56 schrieb Joe Orton :
> 
> On Mon, Aug 30, 2021 at 08:39:17AM +0200, Ruediger Pluem wrote:
>> On 8/28/21 4:00 PM, Yann Ylavic wrote:
>>> On Fri, Aug 27, 2021 at 10:07 AM ste...@eissing.org  
>>> wrote:
>>>> Related to that, do we exempt "./test" from RTC in 2.4.x/STATUS?
>>> 
>>> +1 for CTR, that's how the tests framework works de facto already (a
>>> single framework/trunk is run for both trunk and 2.4.x).
>> 
>> +1 on CTR for this.
> 
> +1 from me too to make test/ CTR for 2.4.x, let's make this a proper 
> vote thread.
> 
> (I would hold off a week or so on enabling the h2 tests in Travis for 
> 2.4.x until we are confident they are stable in trunk CI runs)

+1

Re: release scripts

[ste...@eissing.org]

> Am 01.09.2021 um 13:15 schrieb Eric Covener :
> 
> On Wed, Sep 1, 2021 at 6:08 AM ste...@eissing.org  wrote:
>> 
>> Having a look at the release scripts and how they work.
>> 
>> I think we can use SVN and its branches/tags in a bit smarter way
>> and make things easier. As Christophe mentioned, he would like a
>> "--dry-run" option, because when things do not run smoothly to the
>> end, you have not only a local messed-up state but the SVN repro
>> itself is also changed without easy remedies.
> 
> LGTM. Maybe even separate -tmp and -candidate branches aren't needed?

We can do without, yes. Seems more simple that way.

> It's all scratch space for the RM.
> Does the release branch get moved to a tag at the end?

Yes, that line was missing, I saw too late and did not
want to add another mail to my wall of text. So

> svn mv branches/2.4.49-candidate tags/2.4.49

would be the last thing at the voting.

Re: release scripts

[ste...@eissing.org]

OK, small hickup. The build shell for the manuals wants a JDK 1.8 or 11...
uhm...I need a time machine, or we can change the check there.

Since I am not experienced in our manual stuff, can someone knowledgable check 
this?

- Stefan

> Am 01.09.2021 um 13:27 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 01.09.2021 um 13:15 schrieb Eric Covener :
>> 
>> On Wed, Sep 1, 2021 at 6:08 AM ste...@eissing.org  wrote:
>>> 
>>> Having a look at the release scripts and how they work.
>>> 
>>> I think we can use SVN and its branches/tags in a bit smarter way
>>> and make things easier. As Christophe mentioned, he would like a
>>> "--dry-run" option, because when things do not run smoothly to the
>>> end, you have not only a local messed-up state but the SVN repro
>>> itself is also changed without easy remedies.
>> 
>> LGTM. Maybe even separate -tmp and -candidate branches aren't needed?
> 
> We can do without, yes. Seems more simple that way.
> 
>> It's all scratch space for the RM.
>> Does the release branch get moved to a tag at the end?
> 
> Yes, that line was missing, I saw too late and did not
> want to add another mail to my wall of text. So
> 
>> svn mv branches/2.4.49-candidate tags/2.4.49
> 
> would be the last thing at the voting.

Re: Failed: apache/httpd#1856 (candidate-2.4.49 - cd5b61d)

[ste...@eissing.org]

Not sure what to make of this. I think testing a release candidate is fine, but
I do not really see what was wrong. 

Maybe tomorrow, I see it...

- Stefan

> Am 02.09.2021 um 18:12 schrieb Travis CI :
> 
> apache / httpd
> candidate-2.4.49
> Build #1856 failed24 mins and 12 secs
> Stefan Eissingcd5b61d CHANGESET →
> release candidate 2.4.49
> 
> git-svn-id: 
> https://svn.apache.org/repos/asf/httpd/httpd/tags/candidate-2.4.49@1892807 
> 13f79535-47bb-0310-9956-ffa450edef68
> Want to know about upcoming build environment updates?
> 
> Would you like to stay up-to-date with the upcoming Travis CI build 
> environment updates? We set up a mailing list for you!
> 
> SIGN UP HERE
>  Documentation about Travis CI
> Have any questions? We're here to help.
> Unsubscribe from build emails from the apache/httpd repository.
> To unsubscribe from all build emails, please update your settings.
> 
> Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy Jacops | 
> Contact: cont...@travis-ci.com | Amtsgericht Charlottenburg, Berlin, HRB 
> 140133 B | Umsatzsteuer-ID gemäß §27 a Umsatzsteuergesetz: DE282002648

Re: svn commit: r1892841 - /httpd/httpd/trunk/.travis.yml

[ste...@eissing.org]

Thanks, Joe! \o/

> Am 03.09.2021 um 09:54 schrieb jor...@apache.org:
> 
> Author: jorton
> Date: Fri Sep  3 07:54:03 2021
> New Revision: 1892841
> 
> URL: http://svn.apache.org/viewvc?rev=1892841&view=rev
> Log:
> Adjust Travis conditions for candidate-2.4.x tags.
> 
> Modified:
>httpd/httpd/trunk/.travis.yml
> 
> Modified: httpd/httpd/trunk/.travis.yml
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/.travis.yml?rev=1892841&r1=1892840&r2=1892841&view=diff
> ==
> --- httpd/httpd/trunk/.travis.yml (original)
> +++ httpd/httpd/trunk/.travis.yml Fri Sep  3 07:54:03 2021
> @@ -38,8 +38,8 @@ env:
> # definitions to either:
> #  condition_24x_only => run the job only for 2.4.x
> #  condition_not_24x  => run the job everywhere EXCEPT 2.4.x
> -_cond1: &condition_24x_only (branch is present AND branch = 2.4.x) OR (tag 
> is present AND tag ~= /^2.4/)
> -_cond2: &condition_not_24x (branch is not present OR branch != 2.4.x) AND 
> (tag is not present OR tag !~ /^2.4/)
> +_cond1: &condition_24x_only (branch is present AND branch = 2.4.x) OR (tag 
> is present AND (tag ~= /^2.4/ OR tag ~= /^candidate-2.4/))
> +_cond2: &condition_not_24x (branch is not present OR branch != 2.4.x) AND 
> (tag is not present OR (tag !~ /^2.4/ AND tag !~ /^candidate-2.4/))
> 
> jobs:
>   include:
> 
> 

Re: Broken: apache/httpd#1862 (2.4.x - b15b9a5)

[ste...@eissing.org]

> Am 03.09.2021 um 11:57 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/3/21 11:17 AM, Travis CI wrote:
>> apache
>> 
>> /
>> 
>> httpd
>> 
>> 
>> 
>> branch icon2.4.x 
>> 
>> build has failed
>> Build #1862 was broken 
>> 
>> arrow to build time
>> clock icon26 mins and 38 secs
>> 
> 
> It is a read after free in mod_http2. Do we miss a backport?

Yes. It was a "wait and see what travis says" and then never picked up.

> ==64184==ERROR: AddressSanitizer: heap-use-after-free on address 
> 0x62500dcca9a0 at pc 0x7fe9704ad024 bp 0x7fe952100d30 sp
> 0x7fe952100d20
> 
> READ of size 4 at 0x62500dcca9a0 thread T54
> 
>#0 0x7fe9704ad023 in mst_check_data_for 
> /home/travis/build/apache/httpd/modules/http2/h2_mplx.c:618
> 
>#1 0x7fe9704b1807 in s_task_done 
> /home/travis/build/apache/httpd/modules/http2/h2_mplx.c:811
> 
>#2 0x7fe9704b1807 in h2_mplx_s_task_done 
> /home/travis/build/apache/httpd/modules/http2/h2_mplx.c:842
> 
>#3 0x7fe9704f0cd7 in slot_run 
> /home/travis/build/apache/httpd/modules/http2/h2_workers.c:245
> 
>#4 0x7fe974fba47a in dummy_worker threadproc/unix/thread.c:147
> 
>#5 0x7fe974f2d608 in start_thread 
> (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
> 
>#6 0x7fe974e54292 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)
> 
> 0x62500dcca9a0 is located 160 bytes inside of 8192-byte region 
> [0x62500dcca900,0x62500dccc900)
> 
> freed by thread T59 here:
> 
>#0 0x7fe9751ed7cf in __interceptor_free 
> (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
> 
>#1 0x7fe974f97926 in allocator_free memory/unix/apr_pools.c:492
> 
>#2 0x7fe974f97926 in apr_pool_destroy memory/unix/apr_pools.c:1026
> 
>#3 0x7fe9704ab589 in m_stream_destroy_iter 
> /home/travis/build/apache/httpd/modules/http2/h2_mplx.c:316
> 
>#4 0x7fe974f7a025 in apr_hash_do tables/apr_hash.c:542
> 
>#5 0x7fe9704ea299 in h2_ihash_iter 
> /home/travis/build/apache/httpd/modules/http2/h2_util.c:283
> 
>#6 0x7fe9704b415f in m_purge_streams 
> /home/travis/build/apache/httpd/modules/http2/h2_mplx.c:324
> 
>#7 0x7fe9704b415f in m_purge_streams 
> /home/travis/build/apache/httpd/modules/http2/h2_mplx.c:320
> 
>#8 0x7fe9704b415f in h2_mplx_m_dispatch_master_events 
> /home/travis/build/apache/httpd/modules/http2/h2_mplx.c:1098
> 
>#9 0x7fe9704cd643 in dispatch_master 
> /home/travis/build/apache/httpd/modules/http2/h2_session.c:2078
> 
>#10 0x7fe9704cd643 in h2_session_process 
> /home/travis/build/apache/httpd/modules/http2/h2_session.c:2278
> 
>#11 0x7fe97049336c in h2_conn_run 
> /home/travis/build/apache/httpd/modules/http2/h2_conn.c:214
> 
>#12 0x7fe9704a7ace in h2_h2_process_conn 
> /home/travis/build/apache/httpd/modules/http2/h2_h2.c:631
> 
>#13 0x7fe9704a7ace in h2_h2_process_conn 
> /home/travis/build/apache/httpd/modules/http2/h2_h2.c:549
> 
>#14 0x56201065b6be in ap_run_process_connection 
> /home/travis/build/apache/httpd/server/connection.c:42
> 
>#15 0x56201069b6ef in process_socket 
> /home/travis/build/apache/httpd/server/mpm/event/event.c:1053
> 
>#16 0x56201069d367 in worker_thread 
> /home/travis/build/apache/httpd/server/mpm/event/event.c:2142
> 
>#17 0x7fe974fba47a in dummy_worker threadproc/unix/thread.c:147
> 
>#18 0x7fe974f2d608 in start_thread 
> (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
> 
> 
> 
> Regards
> 
> Rüdiger

Re: svn commit: r1892852 - in /httpd/dev-tools/v2: common-lib.sh make-candidate.sh make-tars.sh push-tars.sh

[ste...@eissing.org]

> Am 03.09.2021 um 13:46 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/3/21 1:28 PM, ic...@apache.org wrote:
>> Author: icing
>> Date: Fri Sep  3 11:28:23 2021
>> New Revision: 1892852
>> 
>> URL: http://svn.apache.org/viewvc?rev=1892852&view=rev
>> Log:
>> * all scripts have a -h option
>> * make-tars.sh has option "-s user" to specify the pgp/gpg siging user
>> * push-tars.sh prepares the distribution of the tars for apache.org
>>   - creates/modifies CHANGES and Announcement
>>   - ask for confirmation before "svn commit" of changes
>>   - preps a voting email
>> 
>> 
>> Added:
>>httpd/dev-tools/v2/push-tars.sh
>> Modified:
>>httpd/dev-tools/v2/common-lib.sh
>>httpd/dev-tools/v2/make-candidate.sh
>>httpd/dev-tools/v2/make-tars.sh
>> 
> 
>> 
>> Modified: httpd/dev-tools/v2/make-tars.sh
>> URL: 
>> http://svn.apache.org/viewvc/httpd/dev-tools/v2/make-tars.sh?rev=1892852&r1=1892851&r2=1892852&view=diff
>> ==
>> --- httpd/dev-tools/v2/make-tars.sh (original)
>> +++ httpd/dev-tools/v2/make-tars.sh Fri Sep  3 11:28:23 2021
>> @@ -21,23 +21,34 @@ set -e
>> 
>> usage () {
>> cat <&2
>> -usage: $0 [version [signing-user]]
>> -  create the release tars from a SVN candidate. The candidate is exported
>> -  to the local dist directory from 'tags/candidate-$version', prepped
>> -  (buildconf, depending srclib, etc.) and then made into tarballs with
>> -  checksums and signatures.
>> -  Options:
>> +usage: $0 [options] [version]
>> +  create the release tars from a SVN candidate. The candidate is expected
>> +  to be found in SVN at 'tags/candidate-$version', as prepared by the
>> +  'make-candidate.sh' script.
>> +  If no version is given, the current version in the checkout is used.
>> +  Arguments:
>> versionas 'm.n.p(-suffix)?', the version to use.
>> -signing-user   the gpg2/gpg/pgp identity used for signing
>> +  Options:
>> +-h print usage information
>> +-s sign-user   use gpg/pgp key from 'sign-user'
>> EOF
>>   exit 1
>> }
>> 
>> source `dirname $0`/common-lib.sh
>> 
>> +signing_user=
>> +while getopts "hy" opt; do
> 
> Shouldn't this be hs instead of hy ?

Keen eyes! Yes, will fix, thanks.

>> +case $opt in
>> +h)  usage
>> +;;
>> +s)  signing_user="$OPTARG"
>> +;;
>> +esac
>> +done
>> +
>> detect_checkout
>> detect_version $1
>> -signing_user=$2
>> 
>> PROJECT=`basename $SVN_BASE`
>> SVN_DEST="tags/candidate-${VERSION}"
>> 
> 
> 
> Regards
> 
> Rüdiger

Re: release?

[ste...@eissing.org]

The v2 release scripts in ^/httpd/dev-tools do now work for me
to create the tarballs, checksums and signatures for a release
vote and push them to dist.apache.org.

The steps after a vote need some more work. I will do that in the
coming days. However, since we can do a vote now, this is not that
pressing.

The "announce.sh" part should stay usable as it is. I have no plans 
to update that one, since I have 0 experience what this really does 
and where any problems/room for improvements are.

Cheers,
Stefan


> Am 02.09.2021 um 16:53 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/2/21 3:06 PM, Eric Covener wrote:
>> Since you are going through this I wanted to mention:
>> 
>> I think the public doc we have should mention everything that's done
>> during ther release, even the security stuff that is somewhat private.
>> The ASF-wide security policy is already public
>> (https://www.apache.org/security/committers.html) and this is just the
>> mechanics of it for us.
>> 
>> Anyone object?  This way we have one linear place to point to.
> 
> +1 Looks sensible. The details of an actual security issue should not be 
> public until we make it so, but the procedure we use can be.
> 
> Regards
> 
> Rüdiger
> 

backport review encouragement

[ste...@eissing.org]

In branches/2.4.x/STATUS we have:

- 1 showstopper who needs an additional vote
- 5 backports, one a bit stale, one missing a vote and 3 new ones by Christophe

If you can find the time to look at one of those, it be great.

- Stefan

Re: *** SPAM *** svn commit: r1892963 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

> Am 06.09.2021 um 10:42 schrieb Marion et Christophe JAILLET 
> :
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>> Message du 06/09/21 09:42
>> De : ic...@apache.org
>> A : c...@httpd.apache.org
>> Copie à : 
>> Objet : *** SPAM *** svn commit: r1892963 - 
>> /httpd/httpd/branches/2.4.x/STATUS
>> 
>> Author: icing
>> Date: Mon Sep 6 07:42:34 2021
>> New Revision: 1892963
>> 
>> URL: http://svn.apache.org/viewvc?rev=1892963&view=rev
>> Log:
>> assuming Christophe is for the backport he proposed. [skip ci]
> 
> Wrong assumption. :)
> 
> I sometimes propose for backport things that I don't vote for.
> This happens when it really looks like a good candidate, but I don't have the 
> knowledge to have my own opinion on it.
> All I could do is to run the test framework, but travis already does it for 
> us.

Sorry about this! I live and learn! At least try to...

> So, you don't have my +1 for this one :(
> (but I'm sure Yann or anyone else will add his +1)
> 
> CJ
> 
>> 
>> Modified:
>> httpd/httpd/branches/2.4.x/STATUS
>> 
>> Modified: httpd/httpd/branches/2.4.x/STATUS
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1892963&r1=1892962&r2=1892963&view=diff
>> ==
>> --- httpd/httpd/branches/2.4.x/STATUS (original)
>> +++ httpd/httpd/branches/2.4.x/STATUS Mon Sep 6 07:42:34 2021
>> @@ -201,7 +201,7 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>> *) mpm_prefork: mask signals during ap_run_child_init()
>> trunk patch: http://svn.apache.org/r1890465
>> 2.4.x patch: svn merge -c 1890465 ^/httpd/httpd/trunk .
>> - +1: 
>> + +1: jailletc36, 
>> 
>> PATCHES/ISSUES THAT ARE BEING WORKED
>> [ New entries should be added at the START of the list ]
>> 
>> 
>> 

Re: svn commit: r1892987 - in /httpd/httpd/trunk: docs/log-message-tags/next-number modules/proxy/proxy_util.c

[ste...@eissing.org]

> Am 06.09.2021 um 13:21 schrieb yla...@apache.org:
> 
> Author: ylavic
> Date: Mon Sep  6 11:21:49 2021
> New Revision: 1892987
> 
> URL: http://svn.apache.org/viewvc?rev=1892987&view=rev
> Log:
> mod_proxy: Follow up to r1892986: APLOGNO()
> 
> Stefan get out of this body! :)

Was I not wearing my mask?

> Modified:
>httpd/httpd/trunk/docs/log-message-tags/next-number
>httpd/httpd/trunk/modules/proxy/proxy_util.c
> 
> Modified: httpd/httpd/trunk/docs/log-message-tags/next-number
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/log-message-tags/next-number?rev=1892987&r1=1892986&r2=1892987&view=diff
> ==
> --- httpd/httpd/trunk/docs/log-message-tags/next-number (original)
> +++ httpd/httpd/trunk/docs/log-message-tags/next-number Mon Sep  6 11:21:49 
> 2021
> @@ -1 +1 @@
> -10292
> +10293
> 
> Modified: httpd/httpd/trunk/modules/proxy/proxy_util.c
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/proxy_util.c?rev=1892987&r1=1892986&r2=1892987&view=diff
> ==
> --- httpd/httpd/trunk/modules/proxy/proxy_util.c (original)
> +++ httpd/httpd/trunk/modules/proxy/proxy_util.c Mon Sep  6 11:21:49 2021
> @@ -2288,7 +2288,7 @@ static int fix_uds_filename(request_rec
> uds_path = ap_runtime_dir_relative(r->pool, urisock.path);
> }
> if (!uds_path) {
> -ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r, APLOGNO()
> +ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r, APLOGNO(10292)
> "Invalid proxy UDS filename (%s)", r->filename);
> return 0;
> }
> 
> 

release and site changes

[ste...@eissing.org]

Does someone have an idea what needs to change regarding our website and 
release updates?

Since I have my hands dirty already, now would be a good time to have me do 
it...

- Stefan

Re: Errored: apache/httpd#1888 (trunk - 47e6ece)

[ste...@eissing.org]

> Am 06.09.2021 um 15:45 schrieb Yann Ylavic :
> 
> On Mon, Sep 6, 2021 at 3:08 PM Travis CI  wrote:
>> 
>> apache / httpd
>> 
>> trunk
>> 
>> Build #1888 has errored
>> 25 mins and 54 secs
> 
> Hmm, several errors that look like travis "internal" failures, am I
> missing something?

I think you are right.

unused function

[ste...@eissing.org]

proxy_util.c:4447:23: error: unused function 'ap_filter_input_pending' 
[-Werror,-Wunused-function]

in branches/2.4.x on a maintainer build. Can we remove this backport fragment?

Re: svn commit: r1892997 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

Updated the PR by merging current 2.4.x, updated the documentation and answered 
the comments. Sorry for missing this so long!

> Am 06.09.2021 um 21:43 schrieb rpl...@apache.org:
> 
> Author: rpluem
> Date: Mon Sep  6 19:43:39 2021
> New Revision: 1892997
> 
> URL: http://svn.apache.org/viewvc?rev=1892997&view=rev
> Log:
> * Comment [skip ci]
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1892997&r1=1892996&r2=1892997&view=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Mon Sep  6 19:43:39 2021
> @@ -171,6 +171,9 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>  2.4.x patch: https://github.com/apache/httpd/pull/203.diff
>PR: https://github.com/apache/httpd/pull/203
>  +1: icing, minfrin
> + rpluem says: I made comments on 
> https://github.com/apache/httpd/pull/203,
> + but I cannot find them commented. Further more the PR currently has 
> merge
> + conflicts.
> 
>   *) mod_ssl: Support logging private key material for use with
>  wireshark via log file given by SSLKEYLOGFILE environment
> 
> 

Re: svn commit: r1892988 - in /httpd/dev-tools/v2: README common-lib.sh make-candidate.sh update-branch.sh

[ste...@eissing.org]

> Am 06.09.2021 um 17:55 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/6/21 1:35 PM, ic...@apache.org wrote:
>> Author: icing
>> Date: Mon Sep  6 11:35:41 2021
>> New Revision: 1892988
>> 
>> URL: http://svn.apache.org/viewvc?rev=1892988&view=rev
>> Log:
>> * added 'update-branch.sh' to increase the patch level and all depending 
>> information after a release
>> 
>> 
>> Added:
>>httpd/dev-tools/v2/update-branch.sh   (with props)
>> Modified:
>>httpd/dev-tools/v2/README
>>httpd/dev-tools/v2/common-lib.sh
>>httpd/dev-tools/v2/make-candidate.sh
>> 
> 
>> Modified: httpd/dev-tools/v2/common-lib.sh
>> URL: 
>> http://svn.apache.org/viewvc/httpd/dev-tools/v2/common-lib.sh?rev=1892988&r1=1892987&r2=1892988&view=diff
>> ==
>> --- httpd/dev-tools/v2/common-lib.sh (original)
>> +++ httpd/dev-tools/v2/common-lib.sh Mon Sep  6 11:35:41 2021
> 
>> +
>> +if test -d docs/manual; then
>> +  # Execute ./build.sh all convmap to ensure that the documentation 
>> transformations are up to date.
>> +  echo "rebuilding manuals"
>> +  pushd docs/manual >/dev/null
>> +  #See http://httpd.apache.org/docs-project/docsformat.html for these 
>> instructions
>> +  if test -d build; then
>> +svn update
> 
> I guess this needs to be
> 
> svn update build
> 
> as build is a separate checkout.

Thanks, will fix. These very late scripts are tricky to test without actually 
doing a release.
> 
> Regards
> 
> Rüdiger

release scripts

[ste...@eissing.org]

After some learning experience (*cough*) I committed a new
version of the scripts, now in $DEV_TOOLS/release that go
so far as staging local changes to all repositories after
a successful vote.

I will test some more tomorrow and add the announce mailing
part. The real test will then come when actually doing a
release as testing is hard to cover all code paths otherwise.

I have silently sneaked in - as some fringe benefit to all
the pain - a '-rcN' suffix for vote candidates. Since we
will be able to cancel a release much later now without
modifying a source repository.

The -rcN tars for voting are named as such, but the directories
contained have the version without any suffix. So, a -rcN
tar can be renamed to the release tar without changes when
the vote has passed.


Cheers, Stefan 

Re: svn commit: r1893011 - /httpd/httpd/trunk/test/time-sem.c

[ste...@eissing.org]

> Am 07.09.2021 um 22:04 schrieb Christophe JAILLET 
> :
> 
> Le 07/09/2021 à 10:52, yla...@apache.org a écrit :
>> Author: ylavic
>> Date: Tue Sep  7 08:52:23 2021
>> New Revision: 1893011
>> URL: http://svn.apache.org/viewvc?rev=1893011&view=rev
>> Log:
>> test/time-sem.c: unlock the accept mutex before exiting (error conditions).
>> Modified:
>> httpd/httpd/trunk/test/time-sem.c
> 
> Hi,
> 
> just for my understanding, does anyone use these tests?
> Are they run on travis? If no, should they be?
> 
> Same question, concerning the unitest framework (httpdunit)?
> This is a great idea, but it looks mostly unused.

I had a quick look since in mod_h2 I have some unit tests that may get 
transferred too. But then other things occupied my time.

> 
> It tried to play with it a few years ago and found it quite hard to use, 
> because most of our functions need some "context" (i.e. a request, a pool, a 
> config, ...)
> I was wondering if implementing such unittest as a module would make sense?
> We could plug nearly anywhere with some hooks to have a meaningful context. 
> We could also implement a new hook to let modules implement tests for 
> functions that are not exported.
> All this done, it could be run from our perl test framework as any other 
> module.

Hmm, that would be a neat way to solve the export restrictions. Build in 
maintainer mode only or some other define.

Re: svn commit: r1893113 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

The way I do 2.4.x updates right now:

while true; do svn up; sleep .1; done

;-)

> Am 08.09.2021 um 10:29 schrieb yla...@apache.org:
> 
> Author: ylavic
> Date: Wed Sep  8 08:29:19 2021
> New Revision: 1893113
> 
> URL: http://svn.apache.org/viewvc?rev=1893113&view=rev
> Log:
> Backported in r1893111. [skip ci]
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1893113&r1=1893112&r2=1893113&view=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Wed Sep  8 08:29:19 2021
> @@ -141,12 +141,6 @@ RELEASE SHOWSTOPPERS:
> PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
>   [ start all new proposals below, under PATCHES PROPOSED. ]
> 
> -  *) mod_proxy: Don't canonicalize with both nocanon and 
> ProxyPassInterpolateEnv
> - On.  PR 65549.
> - trunk patch: http://svn.apache.org/r1893016
> - 2.4.x patch: svn merge -c 1893016 ^/httpd/httpd/trunk .
> - +1: ylavic, covener, rpluem
> -
>   *) mod_unique_id: Earlier request_time initialization in 
> read_request_line(),
>  plus use of an atomic counter to close the race condition.
>  trunk patch: http://svn.apache.org/r1893001
> 
> 

Re: svn commit: r1893113 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

> Am 08.09.2021 um 10:36 schrieb Yann Ylavic :
> 
> On Wed, Sep 8, 2021 at 10:30 AM ste...@eissing.org  wrote:
>> 
>> The way I do 2.4.x updates right now:
>> 
>> while true; do svn up; sleep .1; done
>> 
>> ;-)
> 
> Personally I whine once the checkin fails only :)

Touchè!

project website update

[ste...@eissing.org]

Need help to figure out how out website updates now after the "move" (whatever 
that means).

Formerly:

- changes committed into 
https://svn.apache.org/repos/asf/httpd/site/trunk/content
- TMP_KEY for from https://cms.apache.org/httpd/publish
- POST with same form fields done to trigger the update (I assume)

So, what is now the way?

- Stefan

Re: project website update

[ste...@eissing.org]

> Am 08.09.2021 um 12:53 schrieb Christophe JAILLET 
> :
> 
> Le 08/09/2021 à 12:13, ste...@eissing.org a écrit :
>> Need help to figure out how out website updates now after the "move" 
>> (whatever that means).
>> Formerly:
>> - changes committed into 
>> https://svn.apache.org/repos/asf/httpd/site/trunk/content
> Changes committed in https://github.com/apache/httpd-site
> (so needs to be listed as a contributor, I guess)

Thanks, Christophe. So that needs some git handling to update the site. Hmm, 
either
being submitter or make a fork and create a PR. The former sounds simpler.

>> - TMP_KEY for from https://cms.apache.org/httpd/publish
>> - POST with same form fields done to trigger the update (I assume)
> Nothing to do, it is now done automatically by some magic put in place by 
> infra.
> When pushed on github, it lands on h.a.o.

Ack.

> CJ
> 
>> So, what is now the way?
>> - Stefan
> 

release roll soon?

[ste...@eissing.org]

FYI: from the script side, I am ready to roll the first candidate.
- We have one security issue in a not quite complete state
- There are 2 possible back ports hanging in STATUS

We can see tomorrow how comfy we are and either I roll right away or
we target Monday/Tuesday, I suppose.

cheers, Stefan

Re: svn commit: r1893180 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

Thanks for the nice dev-tools, Rüdiger!

> Am 09.09.2021 um 16:57 schrieb rpl...@apache.org:
> 
> Author: rpluem
> Date: Thu Sep  9 14:57:48 2021
> New Revision: 1893180
> 
> URL: http://svn.apache.org/viewvc?rev=1893180&view=rev
> Log:
> * Vote and promote [skip ci]
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1893180&r1=1893179&r2=1893180&view=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Thu Sep  9 14:57:48 2021
> @@ -141,19 +141,6 @@ RELEASE SHOWSTOPPERS:
> PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
>   [ start all new proposals below, under PATCHES PROPOSED. ]
> 
> -PATCHES PROPOSED TO BACKPORT FROM TRUNK:
> -  [ New proposals should be added at the end of the list ]
> -
> -  *) mod_xml2enc: Correctly handle Microsoft OOXML documents.  PR 64339
> - trunk patch: http://svn.apache.org/r1884505
> - 2.4.x patch: svn merge -c 1884505 ^/httpd/httpd/trunk .
> - +1: 
> - jailletc36: there has been some discussion about how to fix the issue.
> - I'm not sure that the commit above is the best solution
> - and other alternatives have been discussed in PR 64339.  If r1884505
> - is not the right fix, then it should be removed from trunk.
> - Putting it here is a way to revive the discussion
> -
>   *) back port: new ap_ssl_bind_outgoing for multi ssl support in proxy 
> connections
>  core/mod_proxy/mod_ssl:
>Adding outgoing flag to conn_rec, indicating a connection is
> @@ -170,11 +157,20 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>  trunk patch: http://svn.apache.org/r1890605
>  2.4.x patch: https://github.com/apache/httpd/pull/203.diff
>PR: https://github.com/apache/httpd/pull/203
> - +1: icing, minfrin
> - rpluem says: I made comments on 
> https://github.com/apache/httpd/pull/203,
> - but I cannot find them commented. Further more the PR currently has 
> merge
> - conflicts.
> - icing says: conflicts resolved, comments commented.
> + +1: icing, minfrin, rpluem
> +
> +PATCHES PROPOSED TO BACKPORT FROM TRUNK:
> +  [ New proposals should be added at the end of the list ]
> +
> +  *) mod_xml2enc: Correctly handle Microsoft OOXML documents.  PR 64339
> + trunk patch: http://svn.apache.org/r1884505
> + 2.4.x patch: svn merge -c 1884505 ^/httpd/httpd/trunk .
> + +1: 
> + jailletc36: there has been some discussion about how to fix the issue.
> + I'm not sure that the commit above is the best solution
> + and other alternatives have been discussed in PR 64339.  If r1884505
> + is not the right fix, then it should be removed from trunk.
> + Putting it here is a way to revive the discussion
> 
> PATCHES/ISSUES THAT ARE BEING WORKED
>   [ New entries should be added at the START of the list ]
> 
> 

Re: release roll soon?

[ste...@eissing.org]

> Am 10.09.2021 um 09:02 schrieb Joe Orton :
> 
> On Thu, Sep 09, 2021 at 03:23:13PM -0700, Gregg Smith wrote:
>> Since OpenSSL 3.0.0 GA came out yesterday (Californuts time) I think it
>> would be nice to have r1891138 backported for those wishing to try it out.
>> What you say?
> 
> I'd say it's better to try to get a successful release out, then try to 
> get new features in the stable branch.  (In fact, I'd be quite happy if 
> we had 2.5.x/2.6 released and stopped trying new features in 2.4 :)
> 
> That revision is not sufficient, I have a hopefully-complete set of 
> OpenSSL 3.0 backports at: https://github.com/apache/httpd/pull/258

Do you want that in 2.4.49? (we can always to a 2.4.50 OpenSSL3 release shortly 
afterwards, imo)

- Stefan

Re: svn commit: r1893216 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

FYI: this proposed backport works nicely in 2.4.x, however needs revisiting 
when the "new" graceful-continues-serving feature is ever ported back.

- Stefan

> Am 10.09.2021 um 10:20 schrieb ic...@apache.org:
> 
> Author: icing
> Date: Fri Sep 10 08:20:14 2021
> New Revision: 1893216
> 
> URL: http://svn.apache.org/viewvc?rev=1893216&view=rev
> Log:
> backport proposal added. [skip ci]
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1893216&r1=1893215&r2=1893216&view=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Fri Sep 10 08:20:14 2021
> @@ -154,6 +154,13 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>  is not the right fix, then it should be removed from trunk.
>  Putting it here is a way to revive the discussion
> 
> +  *) mod_http2: when a server is restarted gracefully, any idle h2 worker
> + threads are shut down immediately.
> + trunk patch: http://svn.apache.org/r1893214
> +  http://svn.apache.org/r1893215
> + 2.4.x patch: svn merge -c 1893214, 1893215 ^/httpd/httpd/trunk .
> + +1: icing,
> +
> PATCHES/ISSUES THAT ARE BEING WORKED
>   [ New entries should be added at the START of the list ]
> 
> 
> 

Re: release roll soon?

[ste...@eissing.org]

> Am 10.09.2021 um 11:07 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/10/21 10:50 AM, Joe Orton wrote:
>> On Fri, Sep 10, 2021 at 09:42:10AM +0200, ste...@eissing.org wrote:
>>> 
>>> 
>>>> Am 10.09.2021 um 09:02 schrieb Joe Orton :
>>>> 
>>>> On Thu, Sep 09, 2021 at 03:23:13PM -0700, Gregg Smith wrote:
>>>>> Since OpenSSL 3.0.0 GA came out yesterday (Californuts time) I think it
>>>>> would be nice to have r1891138 backported for those wishing to try it out.
>>>>> What you say?
>>>> 
>>>> I'd say it's better to try to get a successful release out, then try to 
>>>> get new features in the stable branch.  (In fact, I'd be quite happy if 
>>>> we had 2.5.x/2.6 released and stopped trying new features in 2.4 :)
>>>> 
>>>> That revision is not sufficient, I have a hopefully-complete set of 
>>>> OpenSSL 3.0 backports at: https://github.com/apache/httpd/pull/258
>>> 
>>> Do you want that in 2.4.49? (we can always to a 2.4.50 OpenSSL3 
>>> release shortly afterwards, imo)
>> 
>> For me, I'd not want to delay or risk regressions in .49 for this, it's 
>> only a small niche of users who care about it at the moment.  I plan to 
>> propose the PR for backport after the next release.
> 
> +1
> 
>> 
>> (It'd be nice to get 3.0 building in Travis so we can be more confident 
>> about keeping that working, not sure if anybody is testing trunk against 
>> it regularly right now?)
> 
> +1

So far, I hear that people think we should make a 2.4.49 based 
on the current 2.4.x. 

I will do some IRL errands and things and come back to this 
in the afternoon. If this still stands then, I'll create a 
2.4.49-rc1 and put that to the vote.

cheers,
Stefan

Re: release roll soon?

[ste...@eissing.org]

APR experts: I build the -deps tar with apr 1.7.0 / apr-util 1.6.1. Those are 
looked up at the site as the latest, just like the old scripts did.
However, that will not configure on my macOS. The branches/1.7.x which I 
normally use does.

./include/apr.h:561:2: error: Can not determine the proper size for pid_t

any one can help my poor memory on how to work around that?

I have the insane plan to actually test the tars before putting them out for 
voting...

- Stefan

> Am 10.09.2021 um 12:12 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/10/21 12:07 PM, ste...@eissing.org wrote:
>> 
> 
>> So far, I hear that people think we should make a 2.4.49 based 
>> on the current 2.4.x. 
>> 
>> I will do some IRL errands and things and come back to this 
>> in the afternoon. If this still stands then, I'll create a 
>> 2.4.49-rc1 and put that to the vote.
>> 
> 
> Sounds good. Thanks for all your work on this and especially on the scripting.
> 
> Regards
> 
> Rüdiger

Re: release roll soon?

[ste...@eissing.org]

> Am 10.09.2021 um 16:31 schrieb Yann Ylavic :
> 
> On Fri, Sep 10, 2021 at 4:18 PM ste...@eissing.org  wrote:
>> 
>> APR experts: I build the -deps tar with apr 1.7.0 / apr-util 1.6.1. Those 
>> are looked up at the site as the latest, just like the old scripts did.
>> However, that will not configure on my macOS. The branches/1.7.x which I 
>> normally use does.
>> 
>> ./include/apr.h:561:2: error: Can not determine the proper size for pid_t
>> 
>> any one can help my poor memory on how to work around that?
> 
> You could test the non-deps tarball and have your usual 1.7.x checkout
> in srclib?
> The -deps are only provided for convenience, in this case it simply
> does not help macOS users.
> If you want to email me with the tarball I can smoke test it on linux at 
> least.

Found https://bz.apache.org/bugzilla/show_bug.cgi?id=64753

Switch the configure.in to the one in branches/1.7.x, buildconf again and now 
it compiles

Seems, a new APR release would be nice for the poor macOS people...
> 
>> 
>> I have the insane plan to actually test the tars before putting them out for 
>> voting...
> 
> That'll teach you..

I learn so much from my mistakes, I just make some more!

> 
> 
> Cheers;
> Yann.

[VOTE] Release httpd-2.4.49-rc1 as httpd-2.4.49

[ste...@eissing.org]

Hi, all;
   Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release
this candidate tarball httpd-2.4.49-rc1 as 2.4.49:
[ ] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: 525378680b3474ff319b83af76565891f8b98331 *httpd-2.4.49-rc1.tar.gz
sha256: 345d3b9b218b1974d1cebd5ae72f6a661d83b52d839310222ff9ec94abb62205 
*httpd-2.4.49-rc1.tar.gz
sha512: 
8efa12f239e1075c0eb8634dde5fa12e73b766a6a8f17882d6bedab8be3e02a1a15be8288413bb6da5be34e58a6e239342cdcb59ebe2d8d88ea4712028b03e5f
 *httpd-2.4.49-rc1.tar.gz

The SVN candidate source is found at tags/candidate-2.4.49-rc1.

PS. Some slight change to previous releases:
The tarballs carry a prefix '-rc1' but the directory it unpacks
to is 'httpd-2.4.49'. This is to make sure that, when you vote 
on a tarball and it is accepted, that we can release this very
thing you voted on.
All other things should be the same as in previous releases.

Re: trunk/rc usable with OpenSSL 3.0.0 ?

[ste...@eissing.org]

> Am 13.09.2021 um 07:23 schrieb Dennis Clarke :
> 
> 
> ALL :
> 
> 
> I may receive no reply to this but in general I have been able to build
> Apache httpd from any release tarball as well as from trunk. When httpd
> needed to get TLS 1.3 working it was a slam dunk to get that working and
> it did. However now we have OpenSSL 3.0.0 and it seems that neither the
> latest RC works nor does trunk.
> 
> So then ... how to proceed ?

The plan is to make a "OpenSSL 3.0" ready release soon after 2.4.49,
anticipating also a possible (likely?) OpenSSL 3.0.1, as a common
then when releases are done and the test base broadens significantly.

That's my understanding.

One could argue, that 2.4.49 should do that as well, which would mean
delaying it. And there are security relevant changes, not visible in 
the candidate, that sit on a timeline.

My personal opinion is that we need to release every other month and
take into it what is ready. The old model of waiting till all stars
align - which is nice as a developer - does not work for CVEs.

- Stefan

> 
> 
> -- 
> Dennis Clarke
> RISC-V/SPARC/PPC/ARM/CISC
> UNIX and Linux spoken
> GreyBeard and suspenders optional
> 
> 
> PS: trunk 1893292 fails even autoreconf and then more horror follows

Re: trunk/rc usable with OpenSSL 3.0.0 ?

[ste...@eissing.org]

> Am 13.09.2021 um 11:05 schrieb Dennis Clarke :
> 
> On 9/13/21 04:22, Joe Orton wrote:
>> On Mon, Sep 13, 2021 at 01:23:37AM -0400, Dennis Clarke wrote:
>>> 
>>> ALL :
>>> 
>>> 
>>> I may receive no reply to this but in general I have been able to build
>>> Apache httpd from any release tarball as well as from trunk. When httpd
>>> needed to get TLS 1.3 working it was a slam dunk to get that working and
>>> it did. However now we have OpenSSL 3.0.0 and it seems that neither the
>>> latest RC works nor does trunk.
>>> 
>>> So then ... how to proceed ?
>> 
>> What fails with trunk?  
>> 
>> It's expected that httpd 2.4 doesn't support 3.0 yet, hopefully we can 
>> get this in for a future release but OpenSSL 3.0 has been a moving 
>> target until just six days ago.
>> 
>> Regards, Joe
>> 
> 
> Why "expected" that httpd 2.4 doesn't support 3.0 ?
> 
> While I realize that 3.0.0 is very shiney new and still has a green glow
> to is we also know that the beta program has been in place for months
> and the release candidates go back a year.
> 
> You have me at a loss.
> 
> That Apache httpd, the biggest web server on planet Earth ( let me check
> mars ) has never looked at OpenSSL 3.0.0 as an event in the mail? It has
> been shipped. Delivered. Done. It works. What are you saying?

What we are saying and what you found out on testing it is that 2.4.49
is not ready for OpenSSL 3.0. No laying blame anywhere will make it so.

Rather than painting this black and white picture, you have to see
that there have been adaptations to changes in OpenSSL 3.0. They are
just not complete. 

If that could have been better, well, of course. I could say that You 
could have done the necessary also. But such discussions do not lead
us anywhere.

2.4.49 contains relevant changes for people who run OpenSSL 1.1 and
other SSL libraries. And I think it should therefore ship as a better
2.4.48. Unless we find a regression.

- Stefan


> -- 
> Dennis Clarke
> RISC-V/SPARC/PPC/ARM/CISC
> UNIX and Linux spoken
> GreyBeard and suspenders optional

Re: [VOTE] Release httpd-2.4.49-rc1 as httpd-2.4.49

[ste...@eissing.org]

Currently, we have 8 +1 votes and a -1, non binding, on OpenSSL 3.0 
compatibility. I will give this another day and plan to conclude 
the results tomorrow.

Many thanks to all the people testing this candidate!

Cheers,
Stefan

> Am 13.09.2021 um 22:31 schrieb Christophe JAILLET 
> :
> 
> Le 10/09/2021 à 17:23, ste...@eissing.org a écrit :
>> Hi, all;
>>Please find below the proposed release tarball and signatures:
>> https://dist.apache.org/repos/dist/dev/httpd/
>> I would like to call a VOTE over the next few days to release
>> this candidate tarball httpd-2.4.49-rc1 as 2.4.49:
>> [X] +1: It's not just good, it's good enough!
>> [ ] +0: Let's have a talk.
>> [ ] -1: There's trouble in paradise. Here's what's wrong.
>> The computed digests of the tarball up for vote are:
>> sha1: 525378680b3474ff319b83af76565891f8b98331 *httpd-2.4.49-rc1.tar.gz
>> sha256: 345d3b9b218b1974d1cebd5ae72f6a661d83b52d839310222ff9ec94abb62205 
>> *httpd-2.4.49-rc1.tar.gz
>> sha512: 
>> 8efa12f239e1075c0eb8634dde5fa12e73b766a6a8f17882d6bedab8be3e02a1a15be8288413bb6da5be34e58a6e239342cdcb59ebe2d8d88ea4712028b03e5f
>>  *httpd-2.4.49-rc1.tar.gz
>> The SVN candidate source is found at tags/candidate-2.4.49-rc1.
>> PS. Some slight change to previous releases:
>> The tarballs carry a prefix '-rc1' but the directory it unpacks
>> to is 'httpd-2.4.49'. This is to make sure that, when you vote
>> on a tarball and it is accepted, that we can release this very
>> thing you voted on.
>> All other things should be the same as in previous releases.
> 
> +1 ; Tested only with event.
> 
> Tested with:
> Linux pop-os 5.11.0
> gcc (Ubuntu 10.3.0-1ubuntu1) 10.3.0
> OpenSSL 1.1.1j  16 Feb 2021
>  libssl-dev 1.1.1j
>  libbrotli-dev 1.0.9
>  libjansson-dev 2.13.1
>  libnghttp2-dev 1.43.0
>  libpcre2-dev 10.36
>  liblua5.3-dev 5.3.3
>  libsystemd-dev 247.3
>  libldap2-dev 2.4.57+dfsg
>  libxml2-dev 2.9.10+dfsg
>  libcurl4-openssl-dev 7.74.0
> 
> 
> Also, a few things should be fixed, but can be done later:
>   LuaHookPreTranslateName  is not documented (neither in trunk nor in 
> 2.4.x)
>is not documented (neither in trunk nor in 
> 2.4.x)
> 
> 
> 
> Thx a lot Stefan for RM and tools.
> 
> CJ

Re: [VOTE] Release httpd-2.4.49-rc1 as httpd-2.4.49, PASSED

[ste...@eissing.org]

According to <https://httpd.apache.org/dev/release.html>, I declare
this vote as PASSED with 8 +1 and a single -1.

I will stage the release tarballs for propagating through the mirrors
and announce the release of 2.4.49 tomorrow.

Thanks for all the people who helped in making and testing this!

Cheers,
Stefan

> Am 14.09.2021 um 11:49 schrieb Steffen :
> 
> All fine on windows till now.
> 
> Only waiting for a full-cycle test mod_md.
> 
> PS.
> Thanks Stefan,
> but I hope not a release every month or two/three, cost too much of my 
> cycles/free time.
> 
> On 10-9-2021 17:23, ste...@eissing.org wrote:
>> Hi, all;
>>Please find below the proposed release tarball and signatures:
>> https://dist.apache.org/repos/dist/dev/httpd/
>> 
>> I would like to call a VOTE over the next few days to release
>> this candidate tarball httpd-2.4.49-rc1 as 2.4.49:
>> [ ] +1: It's not just good, it's good enough!
>> [ ] +0: Let's have a talk.
>> [ ] -1: There's trouble in paradise. Here's what's wrong.
>> 
>> The computed digests of the tarball up for vote are:
>> sha1: 525378680b3474ff319b83af76565891f8b98331 *httpd-2.4.49-rc1.tar.gz
>> sha256: 345d3b9b218b1974d1cebd5ae72f6a661d83b52d839310222ff9ec94abb62205 
>> *httpd-2.4.49-rc1.tar.gz
>> sha512: 
>> 8efa12f239e1075c0eb8634dde5fa12e73b766a6a8f17882d6bedab8be3e02a1a15be8288413bb6da5be34e58a6e239342cdcb59ebe2d8d88ea4712028b03e5f
>>  *httpd-2.4.49-rc1.tar.gz
>> 
>> The SVN candidate source is found at tags/candidate-2.4.49-rc1.
>> 
>> PS. Some slight change to previous releases:
>> The tarballs carry a prefix '-rc1' but the directory it unpacks
>> to is 'httpd-2.4.49'. This is to make sure that, when you vote
>> on a tarball and it is accepted, that we can release this very
>> thing you voted on.
>> All other things should be the same as in previous releases.
> 

Re: [VOTE] Release httpd-2.4.49-rc1 as httpd-2.4.49, PASSED

[ste...@eissing.org]

> Am 15.09.2021 um 10:12 schrieb Dennis Clarke :
> 
> On 9/15/21 04:04, ste...@eissing.org wrote:
>> According to <https://httpd.apache.org/dev/release.html>, I declare
>> this vote as PASSED with 8 +1 and a single -1.
>> 
>> I will stage the release tarballs for propagating through the mirrors
>> and announce the release of 2.4.49 tomorrow.
>> 
>> Thanks for all the people who helped in making and testing this!
>> 
> 
> At great risk of being "that one guy" I have yet to see 2.4.49 compile
> and work anywhere in my old old UNIX systems world. Regardless of the
> OpenSSL version. I will try again tomorrow and see if I can get past the
> essential config and compile stage.

I don't think you will be seen as "that one guy", but someone who builds on 
platforms that no other does or has access to. If there is something
broken there, it will need fixing.

Thanks for your help here.

Cheers,
Stefan


> 
> -- 
> Dennis Clarke
> RISC-V/SPARC/PPC/ARM/CISC
> UNIX and Linux spoken
> GreyBeard and suspenders optional

sending announcement mail

[ste...@eissing.org]

...is failing for me. The method from the old announce.sh script
does not give any error on curl upload, but no mail appears.

Anyone got a suggestion?

- Stefan

Re: sending announcement mail

[ste...@eissing.org]

> Am 16.09.2021 um 12:40 schrieb Greg Stein :
> 
> Fails, how? ... email to annou...@apache.org needs to come from your 
> @apache.org account and include a Reply-To. The moderators may have bounced 
> your message, if it didn't include some key aspects (eg. download links, 
> where to find KEYs, etc)

Yes, exactly. It did not fail on send, it did just not appear and Mark Thomas 
as moderator of announce@a.o did not see it. But now it appeared and I did a 
retry on the announce@httpd.a.o a minute ago.

Maybe my initial attempts triggered some spammer detected and blackholed the 
correct mails afterwards. Or there was a queue stuck somewhere. I cannot tell 
and thus I opened a ticket at infra 
https://issues.apache.org/jira/browse/INFRA-22338.

Cheers,
Stefan

> 
> Cheers,
> -g
> 
> 
> On Thu, Sep 16, 2021 at 4:26 AM ste...@eissing.org  wrote:
> ...is failing for me. The method from the old announce.sh script
> does not give any error on curl upload, but no mail appears.
> 
> Anyone got a suggestion?
> 
> - Stefan

Re: sending announcement mail

[ste...@eissing.org]

> Am 16.09.2021 um 13:46 schrieb Daniel Gruno :
> 
> it's on announce@httpd.a.o already. I modded it through.
> 
> You need to send it to annou...@apache.org as well.

I got the mail from annou...@apache.org, but not from annou...@httpd.apache.org

Alternate realities?

> 
> On 16/09/2021 06.03, ste...@eissing.org wrote:
>>> Am 16.09.2021 um 12:40 schrieb Greg Stein :
>>> 
>>> Fails, how? ... email to annou...@apache.org needs to come from your 
>>> @apache.org account and include a Reply-To. The moderators may have bounced 
>>> your message, if it didn't include some key aspects (eg. download links, 
>>> where to find KEYs, etc)
>> Yes, exactly. It did not fail on send, it did just not appear and Mark 
>> Thomas as moderator of announce@a.o did not see it. But now it appeared and 
>> I did a retry on the announce@httpd.a.o a minute ago.
>> Maybe my initial attempts triggered some spammer detected and blackholed the 
>> correct mails afterwards. Or there was a queue stuck somewhere. I cannot 
>> tell and thus I opened a ticket at infra 
>> https://issues.apache.org/jira/browse/INFRA-22338.
>> Cheers,
>> Stefan
>>> 
>>> Cheers,
>>> -g
>>> 
>>> 
>>> On Thu, Sep 16, 2021 at 4:26 AM ste...@eissing.org  
>>> wrote:
>>> ...is failing for me. The method from the old announce.sh script
>>> does not give any error on curl upload, but no mail appears.
>>> 
>>> Anyone got a suggestion?
>>> 
>>> - Stefan
> 

Re: sending announcement mail

[ste...@eissing.org]

> Am 16.09.2021 um 13:50 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 16.09.2021 um 13:46 schrieb Daniel Gruno :
>> 
>> it's on announce@httpd.a.o already. I modded it through.
>> 
>> You need to send it to annou...@apache.org as well.
> 
> I got the mail from annou...@apache.org, but not from 
> annou...@httpd.apache.org
> 
> Alternate realities?

Hmm, maybe my MacOS mail client confuses me...

> 
>> 
>> On 16/09/2021 06.03, ste...@eissing.org wrote:
>>>> Am 16.09.2021 um 12:40 schrieb Greg Stein :
>>>> 
>>>> Fails, how? ... email to annou...@apache.org needs to come from your 
>>>> @apache.org account and include a Reply-To. The moderators may have 
>>>> bounced your message, if it didn't include some key aspects (eg. download 
>>>> links, where to find KEYs, etc)
>>> Yes, exactly. It did not fail on send, it did just not appear and Mark 
>>> Thomas as moderator of announce@a.o did not see it. But now it appeared and 
>>> I did a retry on the announce@httpd.a.o a minute ago.
>>> Maybe my initial attempts triggered some spammer detected and blackholed 
>>> the correct mails afterwards. Or there was a queue stuck somewhere. I 
>>> cannot tell and thus I opened a ticket at infra 
>>> https://issues.apache.org/jira/browse/INFRA-22338.
>>> Cheers,
>>> Stefan
>>>> 
>>>> Cheers,
>>>> -g
>>>> 
>>>> 
>>>> On Thu, Sep 16, 2021 at 4:26 AM ste...@eissing.org  
>>>> wrote:
>>>> ...is failing for me. The method from the old announce.sh script
>>>> does not give any error on curl upload, but no mail appears.
>>>> 
>>>> Anyone got a suggestion?
>>>> 
>>>> - Stefan
>> 
> 

Re: sending announcement mail

[ste...@eissing.org]

> Am 16.09.2021 um 13:57 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 16.09.2021 um 13:50 schrieb ste...@eissing.org:
>> 
>> 
>> 
>>> Am 16.09.2021 um 13:46 schrieb Daniel Gruno :
>>> 
>>> it's on announce@httpd.a.o already. I modded it through.
>>> 
>>> You need to send it to annou...@apache.org as well.
>> 
>> I got the mail from annou...@apache.org, but not from 
>> annou...@httpd.apache.org
>> 
>> Alternate realities?
> 
> Hmm, maybe my MacOS mail client confuses me...

I give up. Could you mail 
<https://dist.apache.org/repos/dist/release/httpd/Announcement2.4.txt> to the 
list where it missing?

>>> 
>>> On 16/09/2021 06.03, ste...@eissing.org wrote:
>>>>> Am 16.09.2021 um 12:40 schrieb Greg Stein :
>>>>> 
>>>>> Fails, how? ... email to annou...@apache.org needs to come from your 
>>>>> @apache.org account and include a Reply-To. The moderators may have 
>>>>> bounced your message, if it didn't include some key aspects (eg. download 
>>>>> links, where to find KEYs, etc)
>>>> Yes, exactly. It did not fail on send, it did just not appear and Mark 
>>>> Thomas as moderator of announce@a.o did not see it. But now it appeared 
>>>> and I did a retry on the announce@httpd.a.o a minute ago.
>>>> Maybe my initial attempts triggered some spammer detected and blackholed 
>>>> the correct mails afterwards. Or there was a queue stuck somewhere. I 
>>>> cannot tell and thus I opened a ticket at infra 
>>>> https://issues.apache.org/jira/browse/INFRA-22338.
>>>> Cheers,
>>>> Stefan
>>>>> 
>>>>> Cheers,
>>>>> -g
>>>>> 
>>>>> 
>>>>> On Thu, Sep 16, 2021 at 4:26 AM ste...@eissing.org  
>>>>> wrote:
>>>>> ...is failing for me. The method from the old announce.sh script
>>>>> does not give any error on curl upload, but no mail appears.
>>>>> 
>>>>> Anyone got a suggestion?
>>>>> 
>>>>> - Stefan

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

> Am 16.09.2021 um 14:04 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/16/21 12:33 PM, ic...@apache.org wrote:
>> This is an automated email from the ASF dual-hosted git repository.
>> 
>> icing pushed a commit to branch main
>> in repository https://gitbox.apache.org/repos/asf/httpd-site.git
>> 
>> 
>> The following commit(s) were added to refs/heads/main by this push:
>> new 51476c4  publishing release httpd-2.4.49
>> 51476c4 is described below
>> 
>> commit 51476c4d2261ea5b68951054a50c08a249ea1306
>> Author: Stefan Eissing 
>> AuthorDate: Thu Sep 16 09:58:23 2021 +0200
>> 
>>publishing release httpd-2.4.49
>> ---
>> content/doap.rdf|  4 ++--
>> content/download.md | 24 
>> content/index.md|  6 +++---
>> 3 files changed, 17 insertions(+), 17 deletions(-)
> 
> The vulnerabilties page is not updated yet. I guess this can be fixed by 
> copying the respective json files
> to https://github.com/apache/httpd-site/tree/main/content/security/json
> Does https://cveprocess.apache.org/ provide any RESTFUL API to extract the 
> JSON from the tool such that this can be scripted?

With links such as these it looks unguessable

blob:https://cveprocess.apache.org/3c51c80b-1d31-403a-9d1d-95c928f91574

(involuntarily learning about 'blob:' url schemes...aaah!)

> 
> Regards
> 
> Rüdiger
> 
> 

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

> Am 16.09.2021 um 14:14 schrieb ste...@eissing.org:
> 
>> 
>> Am 16.09.2021 um 14:04 schrieb Ruediger Pluem :
>> 
>> 
>> 
>> On 9/16/21 12:33 PM, ic...@apache.org wrote:
>>> This is an automated email from the ASF dual-hosted git repository.
>>> 
>>> icing pushed a commit to branch main
>>> in repository https://gitbox.apache.org/repos/asf/httpd-site.git
>>> 
>>> 
>>> The following commit(s) were added to refs/heads/main by this push:
>>>new 51476c4  publishing release httpd-2.4.49
>>> 51476c4 is described below
>>> 
>>> commit 51476c4d2261ea5b68951054a50c08a249ea1306
>>> Author: Stefan Eissing 
>>> AuthorDate: Thu Sep 16 09:58:23 2021 +0200
>>> 
>>>   publishing release httpd-2.4.49
>>> ---
>>> content/doap.rdf|  4 ++--
>>> content/download.md | 24 
>>> content/index.md|  6 +++---
>>> 3 files changed, 17 insertions(+), 17 deletions(-)
>> 
>> The vulnerabilties page is not updated yet. I guess this can be fixed by 
>> copying the respective json files
>> to https://github.com/apache/httpd-site/tree/main/content/security/json
>> Does https://cveprocess.apache.org/ provide any RESTFUL API to extract the 
>> JSON from the tool such that this can be scripted?
> 
> With links such as these it looks unguessable
> 
> blob:https://cveprocess.apache.org/3c51c80b-1d31-403a-9d1d-95c928f91574
> 
> (involuntarily learning about 'blob:' url schemes...aaah!)

And analyzing the page, the JSON data is embedded in the Javascript embedded in 
the HTML. Hmm...

> 
>> 
>> Regards
>> 
>> Rüdiger

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

Rüdiger, you are also probably looking at this. Who runs he shell scripts to 
generate the mds? It seems we need to do that on changing the cves?

Also, the converter script stumbles on CVEs without "timeline".

Are you on it or should I?

> Am 16.09.2021 um 14:17 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 16.09.2021 um 14:14 schrieb ste...@eissing.org:
>> 
>>> 
>>> Am 16.09.2021 um 14:04 schrieb Ruediger Pluem :
>>> 
>>> 
>>> 
>>> On 9/16/21 12:33 PM, ic...@apache.org wrote:
>>>> This is an automated email from the ASF dual-hosted git repository.
>>>> 
>>>> icing pushed a commit to branch main
>>>> in repository https://gitbox.apache.org/repos/asf/httpd-site.git
>>>> 
>>>> 
>>>> The following commit(s) were added to refs/heads/main by this push:
>>>>   new 51476c4  publishing release httpd-2.4.49
>>>> 51476c4 is described below
>>>> 
>>>> commit 51476c4d2261ea5b68951054a50c08a249ea1306
>>>> Author: Stefan Eissing 
>>>> AuthorDate: Thu Sep 16 09:58:23 2021 +0200
>>>> 
>>>>  publishing release httpd-2.4.49
>>>> ---
>>>> content/doap.rdf|  4 ++--
>>>> content/download.md | 24 
>>>> content/index.md|  6 +++---
>>>> 3 files changed, 17 insertions(+), 17 deletions(-)
>>> 
>>> The vulnerabilties page is not updated yet. I guess this can be fixed by 
>>> copying the respective json files
>>> to https://github.com/apache/httpd-site/tree/main/content/security/json
>>> Does https://cveprocess.apache.org/ provide any RESTFUL API to extract the 
>>> JSON from the tool such that this can be scripted?
>> 
>> With links such as these it looks unguessable
>> 
>> blob:https://cveprocess.apache.org/3c51c80b-1d31-403a-9d1d-95c928f91574
>> 
>> (involuntarily learning about 'blob:' url schemes...aaah!)
> 
> And analyzing the page, the JSON data is embedded in the Javascript embedded 
> in the HTML. Hmm...
> 
>> 
>>> 
>>> Regards
>>> 
>>> Rüdiger

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

Just pushed and it updated.

Summary:
- the "optional" timeline needs to contain an entry "2.4.49 released" or the 
conversion fails
- the "version_affected" combobox field in the UI needs a selection other than 
default or the conversion fails

Something to improve in the cveprocess and our release scripts.

> Am 16.09.2021 um 14:38 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/16/21 2:25 PM, ste...@eissing.org wrote:
>> Rüdiger, you are also probably looking at this. Who runs he shell scripts to 
>> generate the mds? It seems we need to do that on changing the cves?
> 
> These are run on github side when you commit.
> 
>> 
>> Also, the converter script stumbles on CVEs without "timeline".
> 
> Yes, CVE-2021-34798 CVE-2021-39275  CVE-2021-40438 are missing the timeline
> 
> Regards
> 
> Rüdiger
> 
>> 
>> Are you on it or should I?
> 
> Are you able to update the timeline in 3 ones above?
> 
> Regards
> 
> Rüdiger
> 

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

And thanks, Rüdiger, for noticing and the quick fixes.\o/

> Am 16.09.2021 um 14:53 schrieb ste...@eissing.org:
> 
> Just pushed and it updated.
> 
> Summary:
> - the "optional" timeline needs to contain an entry "2.4.49 released" or the 
> conversion fails
> - the "version_affected" combobox field in the UI needs a selection other 
> than default or the conversion fails
> 
> Something to improve in the cveprocess and our release scripts.
> 
>> Am 16.09.2021 um 14:38 schrieb Ruediger Pluem :
>> 
>> 
>> 
>> On 9/16/21 2:25 PM, ste...@eissing.org wrote:
>>> Rüdiger, you are also probably looking at this. Who runs he shell scripts 
>>> to generate the mds? It seems we need to do that on changing the cves?
>> 
>> These are run on github side when you commit.
>> 
>>> 
>>> Also, the converter script stumbles on CVEs without "timeline".
>> 
>> Yes, CVE-2021-34798 CVE-2021-39275  CVE-2021-40438 are missing the timeline
>> 
>> Regards
>> 
>> Rüdiger
>> 
>>> 
>>> Are you on it or should I?
>> 
>> Are you able to update the timeline in 3 ones above?
>> 
>> Regards
>> 
>> Rüdiger
>> 
> 

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

> Am 16.09.2021 um 15:01 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/16/21 2:59 PM, ste...@eissing.org wrote:
>> And thanks, Rüdiger, for noticing and the quick fixes.\o/
> 
> And thanks to you for all the release and scripting work.

I think we should request some download url feature from the cveprocess, so 
that we can automate that part as well. The timeline entry should be added 
automatically. The "affected_by" we can at least check and report.


> 
> Regards
> 
> Rüdiger
> 

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

> Am 16.09.2021 um 21:44 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/16/21 7:13 PM, Eric Covener wrote:
>> On Thu, Sep 16, 2021 at 12:58 PM Mark J Cox  wrote:
>>> 
>>> Hi; at the moment the ASF customisation to the tool is tracked in my github 
>>> fork along with issues.  There's no specific place to discuss it other than 
>>> secur...@apache.org.  That's all just because there's only me having worked 
>>> on it.
>>> 
>>> There are going to be some big changes needed to the tool and running 
>>> instance in the coming months to support the new CVE Project v5.0 JSON 
>>> schema, as that is required for more of the future CVE project automation 
>>> (such as live submission to their database), so that will likely take up 
>>> all the time I can personally spend updating the tool in the near future.
>> 
>> For the sake of discussion/argument: Do we want/need to reproduce this
>> information on the website or is linking to the changelog sufficient?
>> We lose our pseudo-scoring and the range of affected versions.  We
>> could bake them into our changelog-entry authoring/review.
> 
> I like to keep our current vulnerabilities page. On the contrary. I would 
> like to see it extended with the revision numbers that
> fixed the actual issue.

+1. makes sense to me.

> I like the vulnerabilities page we and Tomcat has very much as it eases the 
> search and doesn't force me to got through changelogs
> or other information not that quickly available.

Given the answer by Mark on extensibility of the cveprocess site, we should 
make a solution based on our own pmc repository.

What makes most sense to me is to copy the CVE-JSON to the pmc repro, when a 
CVE is "ready" (from our side) for a release. Let readiness.sh check on that 
and also verify that all fields we need are in there.

Since we no longer need to have unreleased version numbers, we can make a 
directory like "2.4.50" and add them there. The release scripts can then pick 
them up, put the info in the CHANGES and add them to the site. With an added 
"timeline" entry for date and release number.

The only manual thing is then to copy the JSON from the website once into a 
local file and the rest is automated. We can skip on creating a CHANGES per CVE 
and create that from the JSON. The CVEPROCESS file is then also obsolete. Just 
the existence of the JSON file in a version directory is enough.

- Stefan

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

> Am 17.09.2021 um 08:54 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/16/21 10:12 PM, ste...@eissing.org wrote:
>> 
>> 
>>> Am 16.09.2021 um 21:44 schrieb Ruediger Pluem :
>>> 
>>> 
>>> 
>>> On 9/16/21 7:13 PM, Eric Covener wrote:
>>>> On Thu, Sep 16, 2021 at 12:58 PM Mark J Cox  wrote:
>>>>> 
>>>>> Hi; at the moment the ASF customisation to the tool is tracked in my 
>>>>> github fork along with issues.  There's no specific place to discuss it 
>>>>> other than secur...@apache.org.  That's all just because there's only me 
>>>>> having worked on it.
>>>>> 
>>>>> There are going to be some big changes needed to the tool and running 
>>>>> instance in the coming months to support the new CVE Project v5.0 JSON 
>>>>> schema, as that is required for more of the future CVE project automation 
>>>>> (such as live submission to their database), so that will likely take up 
>>>>> all the time I can personally spend updating the tool in the near future.
>>>> 
>>>> For the sake of discussion/argument: Do we want/need to reproduce this
>>>> information on the website or is linking to the changelog sufficient?
>>>> We lose our pseudo-scoring and the range of affected versions.  We
>>>> could bake them into our changelog-entry authoring/review.
>>> 
>>> I like to keep our current vulnerabilities page. On the contrary. I would 
>>> like to see it extended with the revision numbers that
>>> fixed the actual issue.
>> 
>> +1. makes sense to me.
>> 
>>> I like the vulnerabilities page we and Tomcat has very much as it eases the 
>>> search and doesn't force me to got through changelogs
>>> or other information not that quickly available.
>> 
>> Given the answer by Mark on extensibility of the cveprocess site, we should 
>> make a solution based on our own pmc repository.
>> 
>> What makes most sense to me is to copy the CVE-JSON to the pmc repro, when a 
>> CVE is "ready" (from our side) for a release. Let readiness.sh check on that 
>> and also verify that all fields we need are in there.
>> 
>> Since we no longer need to have unreleased version numbers, we can make a 
>> directory like "2.4.50" and add them there. The release scripts can then 
>> pick them up, put the info in the CHANGES and add them to the site. With an 
>> added "timeline" entry for date and release number.
> 
> I understand that there is no need to burn version numbers any longer with 
> the new release scripts, but why would a failed release
> matter to this process?


One thing that was nagging me in the release scripts: the steps run over some 
time (up to a week possibly), and assume that nothing changes in the pmc 
repository during this time. So the scripts might pick up a "ready" CVE that is 
not part of the tarballs.

I was thinking of create a version directory to fix that, but that seems 
overkill on reconsidering it. I am not thinking that the release
scripts should, when creating the candidate, create a pmc/SECURITY/version 
directory and move all ready CVEs there. Later stages of release scripts would 
them only consider those.

> 
>> 
>> The only manual thing is then to copy the JSON from the website once into a 
>> local file and the rest is automated. We can skip on creating a CHANGES per 
>> CVE and create that from the JSON. The CVEPROCESS file is then also 
>> obsolete. Just the existence of the JSON file in a version directory is 
>> enough.
> 
> Although I hate having a manual step in it, it sounds good. But I guess at 
> least short to mid term the manual step cannot be
> avoided. So lets go with it and make the best from the current. But if we no 
> longer create a CHANGES per CVE we need to fill out
> what should get into CHANGES somewhere in the CVE editor in an agreed field 
> and an agreed way (e.g. indenting). Furthermore the
> question is what happens to changes that become CVE's later and hence the 
> changes entry was already committed with the fix.

Hating manual too, but seems we cannot avoid that for some time.

We could also keep CHANGES, but I think we could also format the CVE-JSON in a 
much better way for the changelog entries, basically 
in a format similar to our vulnerabilities pages (which do take all data from 
CVE-JSON already).

Cheers,
Stefan

> 
> Regards
> 
> Rüdiger

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

> Am 17.09.2021 um 11:18 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/17/21 9:36 AM, ste...@eissing.org wrote:
>> 
>> 
>>> Am 17.09.2021 um 08:54 schrieb Ruediger Pluem :
>>> 
>>> 
>>> 
>>> On 9/16/21 10:12 PM, ste...@eissing.org wrote:
>>>> 
>>>> 
>>>>> Am 16.09.2021 um 21:44 schrieb Ruediger Pluem :
>>>>> 
>>>>> 
>>>>> 
>>>>> On 9/16/21 7:13 PM, Eric Covener wrote:
>>>>>> On Thu, Sep 16, 2021 at 12:58 PM Mark J Cox  wrote:
>>>>>>> 
>>>>>>> Hi; at the moment the ASF customisation to the tool is tracked in my 
>>>>>>> github fork along with issues.  There's no specific place to discuss it 
>>>>>>> other than secur...@apache.org.  That's all just because there's only 
>>>>>>> me having worked on it.
>>>>>>> 
>>>>>>> There are going to be some big changes needed to the tool and running 
>>>>>>> instance in the coming months to support the new CVE Project v5.0 JSON 
>>>>>>> schema, as that is required for more of the future CVE project 
>>>>>>> automation (such as live submission to their database), so that will 
>>>>>>> likely take up all the time I can personally spend updating the tool in 
>>>>>>> the near future.
>>>>>> 
>>>>>> For the sake of discussion/argument: Do we want/need to reproduce this
>>>>>> information on the website or is linking to the changelog sufficient?
>>>>>> We lose our pseudo-scoring and the range of affected versions.  We
>>>>>> could bake them into our changelog-entry authoring/review.
>>>>> 
>>>>> I like to keep our current vulnerabilities page. On the contrary. I would 
>>>>> like to see it extended with the revision numbers that
>>>>> fixed the actual issue.
>>>> 
>>>> +1. makes sense to me.
>>>> 
>>>>> I like the vulnerabilities page we and Tomcat has very much as it eases 
>>>>> the search and doesn't force me to got through changelogs
>>>>> or other information not that quickly available.
>>>> 
>>>> Given the answer by Mark on extensibility of the cveprocess site, we 
>>>> should make a solution based on our own pmc repository.
>>>> 
>>>> What makes most sense to me is to copy the CVE-JSON to the pmc repro, when 
>>>> a CVE is "ready" (from our side) for a release. Let readiness.sh check on 
>>>> that and also verify that all fields we need are in there.
>>>> 
>>>> Since we no longer need to have unreleased version numbers, we can make a 
>>>> directory like "2.4.50" and add them there. The release scripts can then 
>>>> pick them up, put the info in the CHANGES and add them to the site. With 
>>>> an added "timeline" entry for date and release number.
>>> 
>>> I understand that there is no need to burn version numbers any longer with 
>>> the new release scripts, but why would a failed release
>>> matter to this process?
>> 
>> 
>> One thing that was nagging me in the release scripts: the steps run over 
>> some time (up to a week possibly), and assume that nothing changes in the 
>> pmc repository during this time. So the scripts might pick up a "ready" CVE 
>> that is not part of the tarballs.
>> 
>> I was thinking of create a version directory to fix that, but that seems 
>> overkill on reconsidering it. I am not thinking that the release
> 
> I am a little bit confused with this and the following sentence.
> Do you want to say I am *not* thinking or I am thinking?

Sorry, not enough coffee. "I am *now* thinking..."

> 
>> scripts should, when creating the candidate, create a pmc/SECURITY/version 
>> directory and move all ready CVEs there. Later stages of release scripts 
>> would them only consider those.
>> 
> 
> Regards
> 
> Rüdiger

Re: [httpd-site] branch main updated: publishing release httpd-2.4.49

[ste...@eissing.org]

> Am 17.09.2021 um 11:20 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 17.09.2021 um 11:18 schrieb Ruediger Pluem :
>> 
>> 
>> 
>> On 9/17/21 9:36 AM, ste...@eissing.org wrote:
>>> 
>>> 
>>>> Am 17.09.2021 um 08:54 schrieb Ruediger Pluem :
>>>> 
>>>> 
>>>> 
>>>> On 9/16/21 10:12 PM, ste...@eissing.org wrote:
>>>>> 
>>>>> 
>>>>>> Am 16.09.2021 um 21:44 schrieb Ruediger Pluem :
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 9/16/21 7:13 PM, Eric Covener wrote:
>>>>>>> On Thu, Sep 16, 2021 at 12:58 PM Mark J Cox  wrote:
>>>>>>>> 
>>>>>>>> Hi; at the moment the ASF customisation to the tool is tracked in my 
>>>>>>>> github fork along with issues. There's no specific place to discuss it 
>>>>>>>> other than secur...@apache.org.  That's all just because there's only 
>>>>>>>> me having worked on it.
>>>>>>>> 
>>>>>>>> There are going to be some big changes needed to the tool and running 
>>>>>>>> instance in the coming months to support the new CVE Project v5.0 JSON 
>>>>>>>> schema, as that is required for more of the future CVE project 
>>>>>>>> automation (such as live submission to their database), so that will 
>>>>>>>> likely take up all the time I can personally spend updating the tool 
>>>>>>>> in the near future.
>>>>>>> 
>>>>>>> For the sake of discussion/argument: Do we want/need to reproduce this
>>>>>>> information on the website or is linking to the changelog sufficient?
>>>>>>> We lose our pseudo-scoring and the range of affected versions.  We
>>>>>>> could bake them into our changelog-entry authoring/review.
>>>>>> 
>>>>>> I like to keep our current vulnerabilities page. On the contrary. I 
>>>>>> would like to see it extended with the revision numbers that
>>>>>> fixed the actual issue.
>>>>> 
>>>>> +1. makes sense to me.
>>>>> 
>>>>>> I like the vulnerabilities page we and Tomcat has very much as it eases 
>>>>>> the search and doesn't force me to got through changelogs
>>>>>> or other information not that quickly available.
>>>>> 
>>>>> Given the answer by Mark on extensibility of the cveprocess site, we 
>>>>> should make a solution based on our own pmc repository.
>>>>> 
>>>>> What makes most sense to me is to copy the CVE-JSON to the pmc repro, 
>>>>> when a CVE is "ready" (from our side) for a release. Let readiness.sh 
>>>>> check on that and also verify that all fields we need are in there.
>>>>> 
>>>>> Since we no longer need to have unreleased version numbers, we can make a 
>>>>> directory like "2.4.50" and add them there. The release scripts can then 
>>>>> pick them up, put the info in the CHANGES and add them to the site. With 
>>>>> an added "timeline" entry for date and release number.
>>>> 
>>>> I understand that there is no need to burn version numbers any longer with 
>>>> the new release scripts, but why would a failed release
>>>> matter to this process?
>>> 
>>> 
>>> One thing that was nagging me in the release scripts: the steps run over 
>>> some time (up to a week possibly), and assume that nothing changes in the 
>>> pmc repository during this time. So the scripts might pick up a "ready" CVE 
>>> that is not part of the tarballs.
>>> 
>>> I was thinking of create a version directory to fix that, but that seems 
>>> overkill on reconsidering it. I am not thinking that the release
>> 
>> I am a little bit confused with this and the following sentence.
>> Do you want to say I am *not* thinking or I am thinking?
> 
> Sorry, not enough coffee. "I am *now* thinking..."

To spell it out more clearly:

- when a CVE is considered "ready" by us, but not set 
  to READY in the cveprocess (it is not released yet), 
  we manually copy the CVE-JSON from the site into 
  "pmc/SECURITY/" dir.
- cancelling a candidate would remove that dir again
- dev-tools/r4-stage-release.sh takes the CVEs from that
  dir and adds content to CHANGES, copies to site etc.

This means that when CVEs become ready after a candidate is
built, it won't slip accidentally into the announcements
or CHANGES.

There is the one manual step of copying, but the rest is
handled by scripts.

- Stefan

Re: sending announcement mail

[ste...@eissing.org]

> Am 18.09.2021 um 13:18 schrieb Rainer Jung :
> 
> Am 16.09.2021 um 13:59 schrieb ste...@eissing.org:
>>> Am 16.09.2021 um 13:57 schrieb ste...@eissing.org:
>>>> Am 16.09.2021 um 13:50 schrieb ste...@eissing.org:
>>>>> Am 16.09.2021 um 13:46 schrieb Daniel Gruno :
>>>>> 
>>>>> it's on announce@httpd.a.o already. I modded it through.
>>>>> 
>>>>> You need to send it to annou...@apache.org as well.
>>>> 
>>>> I got the mail from annou...@apache.org, but not from 
>>>> annou...@httpd.apache.org
>>>> 
>>>> Alternate realities?
>>> 
>>> Hmm, maybe my MacOS mail client confuses me...
>> I give up. Could you mail 
>> <https://dist.apache.org/repos/dist/release/httpd/Announcement2.4.txt> to 
>> the list where it missing?
> 
> I think there is still no announcement on annou...@apache.org. At least I 
> don't see it here
> 
> https://mail-archives.apache.org/mod_mbox/www-announce/202109.mbox/browser
> 
> and I think I also didn't get it. For the previous 2.4.48 there was one from 
> Christophe on June 1st:
> 
> https://mail-archives.apache.org/mod_mbox/www-announce/202106.mbox/browser


Thanks for confirming. I now did now a subscribe as 'ic...@apache.org' and send 
the announce mail there again. Let's see if it now goes through.

> Thanks and regards,
> 
> Rainer

Re: sending announcement mail

[ste...@eissing.org]

Ok, so the problem was that 'ic...@apache.org' was not subscribed there - sigh.🙈

> Am 18.09.2021 um 13:48 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 18.09.2021 um 13:18 schrieb Rainer Jung :
>> 
>> Am 16.09.2021 um 13:59 schrieb ste...@eissing.org:
>>>> Am 16.09.2021 um 13:57 schrieb ste...@eissing.org:
>>>>> Am 16.09.2021 um 13:50 schrieb ste...@eissing.org:
>>>>>> Am 16.09.2021 um 13:46 schrieb Daniel Gruno :
>>>>>> 
>>>>>> it's on announce@httpd.a.o already. I modded it through.
>>>>>> 
>>>>>> You need to send it to annou...@apache.org as well.
>>>>> 
>>>>> I got the mail from annou...@apache.org, but not from 
>>>>> annou...@httpd.apache.org
>>>>> 
>>>>> Alternate realities?
>>>> 
>>>> Hmm, maybe my MacOS mail client confuses me...
>>> I give up. Could you mail 
>>> <https://dist.apache.org/repos/dist/release/httpd/Announcement2.4.txt> to 
>>> the list where it missing?
>> 
>> I think there is still no announcement on annou...@apache.org. At least I 
>> don't see it here
>> 
>> https://mail-archives.apache.org/mod_mbox/www-announce/202109.mbox/browser
>> 
>> and I think I also didn't get it. For the previous 2.4.48 there was one from 
>> Christophe on June 1st:
>> 
>> https://mail-archives.apache.org/mod_mbox/www-announce/202106.mbox/browser
> 
> 
> Thanks for confirming. I now did now a subscribe as 'ic...@apache.org' and 
> send the announce mail there again. Let's see if it now goes through.
> 
>> Thanks and regards,
>> 
>> Rainer

2.4.49 release report

[ste...@eissing.org]

Compiling the release experience.

Apache httpd 2.4.49 was released on September 15/16 20201. 
There were changes to the release process and some resulting
hickups, but it went through.

New in the release process were:
- a switch from always incrementing version numbers to
  release candidate numberings.
- adaptations of our process to the general apache security
  CVE handling from cveprocess.apache.org

The switch away from incrementing version numbers before
a release voting led in the past to confusions to our users
and extra work on our part. Users, for example, overlooked 
CHANGES reported on unreleased versions. CVEs were reported 
on versions the users never saw.

With the new release candidate numbers, we can keep the next
release number stable (whatever source revision will be selected).
We can now communicate "this will be fixed in 2.4.50" and this
will be the version that users get.

The CVE handling via cveprocess.apache.org is seen as an
overall improvements to the process. However, lacking an
API usable for automation, it still involves manual steps
which we would like to automate more.

For example, since we cannot download CVE JSON data, release
and "readiness" scripts could not do a full status check. This
led to missing fields being unnoticed during release. As
a result, vulnerability pages became 404s on our site and
we needed manual intervention to get it right.

We will adjust our processes to have a minimum of manual
steps here and check data completeness before release. We hope
that mid-term, the cveprocess site can offer non-browser access
to features. Maybe apache infra can be of help. This should
be beneficial to all apache projects.

Then we had some things fumbled by our new release manager (myself):
- the RMs PGP key was kept in the KEYS file, but not registered
  in the directories and as its apache committers pgp key. This
  led to irritations for folks that verified our tarballs.
- The general announcement emails did not go through for
  annou...@apache.org, moderators did not see it. The issue,
  as it turned out later, was that the RM was not subscribed to
  that list with his apache email id. The list silently dropped
  the mails.
- A twitter announcement for @apache_httpd was not generated.
  We need to handshake with the holder of that handle on how to
  get this out in the future.

This should serve as a record for things to improve in the next
release - while memory of this one is still fresh. Please add to
this anything I might have missed or additional things you like
us to tackle in the next release.

Thanks,
Stefan
  

Re: svn commit: r1862475 - in /httpd/httpd/trunk: CHANGES modules/http2/h2_conn.c modules/http2/h2_version.h server/mpm/event/event.c

[ste...@eissing.org]

> Am 14.09.2021 um 13:43 schrieb Ruediger Pluem :
> 
> When looking at this again while researching something I couldn't answer 
> myself the questions below
> and as event mpm and mod_http2 are sometimes pretty complex I thought I ask 
> :-)
> 
> On 7/3/19 3:46 PM, ic...@apache.org wrote:
>> Author: icing
>> Date: Wed Jul  3 13:46:31 2019
>> New Revision: 1862475
>> 
>> URL: http://svn.apache.org/viewvc?rev=1862475&view=rev
>> Log:
>>  *) mod_http2/mpm_event: Fixes the behaviour when a HTTP/2 connection has 
>> nothing
>> more to write with streams ongoing (flow control block). The timeout 
>> waiting
>> for the client to send WINODW_UPDATE was incorrectly KeepAliveTimeout 
>> and not
>> Timeout as it should be. Fixes PR 63534. [Yann Ylavic, Stefan Eissing]
>> 
>> 
>> Modified:
>>httpd/httpd/trunk/CHANGES
>>httpd/httpd/trunk/modules/http2/h2_conn.c
>>httpd/httpd/trunk/modules/http2/h2_version.h
>>httpd/httpd/trunk/server/mpm/event/event.c
>> 
> 
>> Modified: httpd/httpd/trunk/modules/http2/h2_conn.c
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_conn.c?rev=1862475&r1=1862474&r2=1862475&view=diff
>> ==
>> --- httpd/httpd/trunk/modules/http2/h2_conn.c (original)
>> +++ httpd/httpd/trunk/modules/http2/h2_conn.c Wed Jul  3 13:46:31 2019
>> @@ -231,6 +231,13 @@ apr_status_t h2_conn_run(conn_rec *c)
>> case H2_SESSION_ST_BUSY:
>> case H2_SESSION_ST_WAIT:
>> c->cs->state = CONN_STATE_WRITE_COMPLETION;
>> +if (c->cs && (session->open_streams || 
>> !session->remote.emitted_count)) {
>> +/* let the MPM know that we are not done and want
>> + * the Timeout behaviour instead of a KeepAliveTimeout
>> + * See PR 63534. 
>> + */
>> +c->cs->sense = CONN_SENSE_WANT_READ;
> 
> Can we get here with session->open_streams > 0 and all the open streams are 
> only producing output and none of them is waiting for
> something from the client?

Yes, but only when the HTTP/2 flow control blocks any progress. New data has to 
arrive from the client before the server is able to send more. So, we are 
waiting for new data from the client in the form of window updates.

>> +}
>> break;
>> case H2_SESSION_ST_CLEANUP:
>> case H2_SESSION_ST_DONE:
>> 
> 
>> Modified: httpd/httpd/trunk/server/mpm/event/event.c
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/trunk/server/mpm/event/event.c?rev=1862475&r1=1862474&r2=1862475&view=diff
>> ==
>> --- httpd/httpd/trunk/server/mpm/event/event.c (original)
>> +++ httpd/httpd/trunk/server/mpm/event/event.c Wed Jul  3 13:46:31 2019
>> @@ -1153,10 +1153,11 @@ read_request:
>> else if (ap_filter_should_yield(c->output_filters)) {
>> pending = OK;
>> }
>> -if (pending == OK) {
>> +if (pending == OK || (pending == DECLINED &&
>> +  cs->pub.sense == CONN_SENSE_WANT_READ)) {
>> /* Still in WRITE_COMPLETION_STATE:
>> - * Set a write timeout for this connection, and let the
>> - * event thread poll for writeability.
>> + * Set a read/write timeout for this connection, and let the
>> + * event thread poll for read/writeability.
>>  */
>> cs->queue_timestamp = apr_time_now();
>> notify_suspend(cs);
> 
> Hm. Showing following code lines from trunk for my next question:
> 
> 
>update_reqevents_from_sense(cs, -1);
> 
> if cs->pub.sense == CONN_SENSE_WANT_READ we subscribe on to POLLIN only on 
> the pfd
> 
> 
>apr_thread_mutex_lock(timeout_mutex);
>TO_QUEUE_APPEND(cs->sc->wc_q, cs);
>rv = apr_pollset_add(event_pollset, &cs->pfd);
> 
> If the read event triggers we will get back to the
> 
>if (cs->pub.state == CONN_STATE_WRITE_COMPLETION) {
>int pending = DECLINED;
> 
> above. This means we try flush the output queue if it is not empty and if it 
> is empty after this we would fall through to
> set cs->pub.state = CONN_STATE_CHECK_REQUEST_LINE_READABLE; and then add the 
> pfd to the pollset again and poll again. This should
> trigger the read event again and we would process the input. But if I see 
> this correctly we would need to poll twice for getting
> the read data processed.
> OTOH if we do not manage to clear the output queue above we would add the pfd 
> to the pollset again but this time for writing and
> only once all output has been processed we would take care of the reading.
> Maybe we should take care of both?

Hmm, but can it? In my understanding CONN_STATE_WRITE_COMPLETION is intended to 
flush the out buffers before putting in new work.

Since conn_rec

Re: svn commit: r1862475 - in /httpd/httpd/trunk: CHANGES modules/http2/h2_conn.c modules/http2/h2_version.h server/mpm/event/event.c

[ste...@eissing.org]

> Am 20.09.2021 um 11:17 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 14.09.2021 um 13:43 schrieb Ruediger Pluem :
>> 
>> When looking at this again while researching something I couldn't answer 
>> myself the questions below
>> and as event mpm and mod_http2 are sometimes pretty complex I thought I ask 
>> :-)
>> 
>> On 7/3/19 3:46 PM, ic...@apache.org wrote:
>>> Author: icing
>>> Date: Wed Jul  3 13:46:31 2019
>>> New Revision: 1862475
>>> 
>>> URL: http://svn.apache.org/viewvc?rev=1862475&view=rev
>>> Log:
>>> *) mod_http2/mpm_event: Fixes the behaviour when a HTTP/2 connection has 
>>> nothing
>>>more to write with streams ongoing (flow control block). The timeout 
>>> waiting
>>>for the client to send WINODW_UPDATE was incorrectly KeepAliveTimeout 
>>> and not
>>>Timeout as it should be. Fixes PR 63534. [Yann Ylavic, Stefan Eissing]
>>> 
>>> 
>>> Modified:
>>>   httpd/httpd/trunk/CHANGES
>>>   httpd/httpd/trunk/modules/http2/h2_conn.c
>>>   httpd/httpd/trunk/modules/http2/h2_version.h
>>>   httpd/httpd/trunk/server/mpm/event/event.c
>>> 
>> 
>>> Modified: httpd/httpd/trunk/modules/http2/h2_conn.c
>>> URL: 
>>> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_conn.c?rev=1862475&r1=1862474&r2=1862475&view=diff
>>> ==
>>> --- httpd/httpd/trunk/modules/http2/h2_conn.c (original)
>>> +++ httpd/httpd/trunk/modules/http2/h2_conn.c Wed Jul  3 13:46:31 2019
>>> @@ -231,6 +231,13 @@ apr_status_t h2_conn_run(conn_rec *c)
>>>case H2_SESSION_ST_BUSY:
>>>case H2_SESSION_ST_WAIT:
>>>c->cs->state = CONN_STATE_WRITE_COMPLETION;
>>> +if (c->cs && (session->open_streams || 
>>> !session->remote.emitted_count)) {
>>> +/* let the MPM know that we are not done and want
>>> + * the Timeout behaviour instead of a KeepAliveTimeout
>>> + * See PR 63534. 
>>> + */
>>> +c->cs->sense = CONN_SENSE_WANT_READ;
>> 
>> Can we get here with session->open_streams > 0 and all the open streams are 
>> only producing output and none of them is waiting for
>> something from the client?
> 
> Yes, but only when the HTTP/2 flow control blocks any progress. New data has 
> to arrive from the client before the server is able to send more. So, we are 
> waiting for new data from the client in the form of window updates.
> 
>>> +}
>>>break;
>>>case H2_SESSION_ST_CLEANUP:
>>>case H2_SESSION_ST_DONE:
>>> 
>> 
>>> Modified: httpd/httpd/trunk/server/mpm/event/event.c
>>> URL: 
>>> http://svn.apache.org/viewvc/httpd/httpd/trunk/server/mpm/event/event.c?rev=1862475&r1=1862474&r2=1862475&view=diff
>>> ==
>>> --- httpd/httpd/trunk/server/mpm/event/event.c (original)
>>> +++ httpd/httpd/trunk/server/mpm/event/event.c Wed Jul  3 13:46:31 2019
>>> @@ -1153,10 +1153,11 @@ read_request:
>>>else if (ap_filter_should_yield(c->output_filters)) {
>>>pending = OK;
>>>}
>>> -if (pending == OK) {
>>> +if (pending == OK || (pending == DECLINED &&
>>> +  cs->pub.sense == CONN_SENSE_WANT_READ)) {
>>>/* Still in WRITE_COMPLETION_STATE:
>>> - * Set a write timeout for this connection, and let the
>>> - * event thread poll for writeability.
>>> + * Set a read/write timeout for this connection, and let the
>>> + * event thread poll for read/writeability.
>>> */
>>>cs->queue_timestamp = apr_time_now();
>>>notify_suspend(cs);
>> 
>> Hm. Showing following code lines from trunk for my next question:
>> 
>> 
>>   update_reqevents_from_sense(cs, -1);
>> 
>> if cs->pub.sense == CONN_SENSE_WANT_READ we subscribe on to POLLIN only on 
>> the pfd
>> 
>> 
>>   apr_thread_mutex_lock(timeout_mutex);
>>   TO_QUEUE_APPEND(cs->sc->wc_q, cs);
>>   rv = apr_pollset_ad

Re: svn commit: r1862475 - in /httpd/httpd/trunk: CHANGES modules/http2/h2_conn.c modules/http2/h2_version.h server/mpm/event/event.c

[ste...@eissing.org]

> Am 20.09.2021 um 12:27 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/20/21 11:17 AM, ste...@eissing.org wrote:
>> 
>> 
>>> Am 14.09.2021 um 13:43 schrieb Ruediger Pluem :
>>> 
>>> When looking at this again while researching something I couldn't answer 
>>> myself the questions below
>>> and as event mpm and mod_http2 are sometimes pretty complex I thought I ask 
>>> :-)
>>> 
>>> On 7/3/19 3:46 PM, ic...@apache.org wrote:
>>>> Author: icing
>>>> Date: Wed Jul  3 13:46:31 2019
>>>> New Revision: 1862475
>>>> 
>>>> URL: http://svn.apache.org/viewvc?rev=1862475&view=rev
>>>> Log:
>>>> *) mod_http2/mpm_event: Fixes the behaviour when a HTTP/2 connection has 
>>>> nothing
>>>>more to write with streams ongoing (flow control block). The timeout 
>>>> waiting
>>>>for the client to send WINODW_UPDATE was incorrectly KeepAliveTimeout 
>>>> and not
>>>>Timeout as it should be. Fixes PR 63534. [Yann Ylavic, Stefan Eissing]
>>>> 
>>>> 
>>>> Modified:
>>>>   httpd/httpd/trunk/CHANGES
>>>>   httpd/httpd/trunk/modules/http2/h2_conn.c
>>>>   httpd/httpd/trunk/modules/http2/h2_version.h
>>>>   httpd/httpd/trunk/server/mpm/event/event.c
>>>> 
>>> 
>>>> Modified: httpd/httpd/trunk/modules/http2/h2_conn.c
>>>> URL: 
>>>> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_conn.c?rev=1862475&r1=1862474&r2=1862475&view=diff
>>>> ==
>>>> --- httpd/httpd/trunk/modules/http2/h2_conn.c (original)
>>>> +++ httpd/httpd/trunk/modules/http2/h2_conn.c Wed Jul  3 13:46:31 2019
>>>> @@ -231,6 +231,13 @@ apr_status_t h2_conn_run(conn_rec *c)
>>>>case H2_SESSION_ST_BUSY:
>>>>case H2_SESSION_ST_WAIT:
>>>>c->cs->state = CONN_STATE_WRITE_COMPLETION;
>>>> +if (c->cs && (session->open_streams || 
>>>> !session->remote.emitted_count)) {
>>>> +/* let the MPM know that we are not done and want
>>>> + * the Timeout behaviour instead of a KeepAliveTimeout
>>>> + * See PR 63534. 
>>>> + */
>>>> +c->cs->sense = CONN_SENSE_WANT_READ;
>>> 
>>> Can we get here with session->open_streams > 0 and all the open streams are 
>>> only producing output and none of them is waiting for
>>> something from the client?
>> 
>> Yes, but only when the HTTP/2 flow control blocks any progress. New data has 
>> to arrive from the client before the server is able to send more. So, we are 
>> waiting for new data from the client in the form of window updates.
> 
> Thanks, but as you state in this case it wants to read data, not for a new 
> request for getting output data sending for existing
> requests started again. This sounds fine. I was worried that we could wait 
> for input data here without expecting any.
> 
>> 
>>>> +}
>>>>break;
>>>>case H2_SESSION_ST_CLEANUP:
>>>>case H2_SESSION_ST_DONE:
>>>> 
>>> 
>>>> Modified: httpd/httpd/trunk/server/mpm/event/event.c
>>>> URL: 
>>>> http://svn.apache.org/viewvc/httpd/httpd/trunk/server/mpm/event/event.c?rev=1862475&r1=1862474&r2=1862475&view=diff
>>>> ==
>>>> --- httpd/httpd/trunk/server/mpm/event/event.c (original)
>>>> +++ httpd/httpd/trunk/server/mpm/event/event.c Wed Jul  3 13:46:31 2019
>>>> @@ -1153,10 +1153,11 @@ read_request:
>>>>else if (ap_filter_should_yield(c->output_filters)) {
>>>>pending = OK;
>>>>}
>>>> -if (pending == OK) {
>>>> +if (pending == OK || (pending == DECLINED &&
>>>> +  cs->pub.sense == CONN_SENSE_WANT_READ)) {
>>>>/* Still in WRITE_COMPLETION_STATE:
>>>> - * Set a write timeout for this connection, and let the
>>>> - * event thread poll for writeability.
>>>> + * Set a read/write timeout for this connection, and 

Re: crash in process_lingering_close

[ste...@eissing.org]

> Am 21.09.2021 um 11:48 schrieb Ruediger Pluem :
> 
> I noticed a crash in process_lingering_close
> 
> The crash happens here:
> 
> #0  0x7f5d4af33b28 in apr_socket_timeout_set (sock=sock@entry=0x0, 
> t=t@entry=200) at network_io/unix/sockopt.c:86
>stat = 
> #1  0x004706a4 in process_lingering_close (cs=0x7f5c88000ed0) at 
> event.c:1466
>csd = 0x0
>dummybuf = '\000' ...
>nbytes = 0
>rv = 
>q = 
> #2  0x0047154b in worker_thread (thd=0x1a31480, dummy= out>) at event.c:2142
>csd = 0x7f5c88000cd0
>cs = 0x0
>te = 0x0
>ptrans = 0x7f5c88000c58
>ti = 
>process_slot = 3
>thread_slot = 1
>rv = 0
>is_idle = 0
> #3  0x7f5d4a48015a in start_thread () from /lib64/libpthread.so.0
> No symbol table info available.
> #4  0x7f5d49fadf73 in clone () from /lib64/libc.so.6
> 
> 
> The reason is that in this case a third party module that has hooked into the 
> pre_connection hook returns an error which causes
> the hook to stop running. This prevents core_pre_connection from running 
> which is APR_HOOK_REALLY_LAST.
> Hence the socket is not set into c->conn_config and hence ap_get_conn_socket 
> returns NULL.
> The quick fix that prevents that from happening is
> 
> Index: server/mpm/event/event.c
> ===
> --- server/mpm/event/event.c  (revision 1893352)
> +++ server/mpm/event/event.c  (working copy)
> @@ -1040,6 +1040,16 @@
> ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, APLOGNO(00469)
>   "process_socket: connection aborted");
> c->aborted = 1;
> +/*
> + * In case we errored, the pre_connection hook of the core
> + * module maybe did not run (it is APR_HOOK_REALLY_LAST) and
> + * hence the socket might not have been set in c->conn_config
> + * and the lingering close process cannot fetch it from there
> + * via ap_get_conn_socket. Hence set it in this case.
> + */
> +if (!ap_get_conn_socket(c)) {
> +ap_set_core_module_config(c->conn_config, sock);
> +}
> }
> 
> /**
> 
> 
> But I think it is incomplete. I think we should do all that 
> core_pre_connection does.
> We cannot call it from event.c as it is a static in core.c. If we agree that 
> we want to execute what it does I see two ways forward:
> 
> 1. Make core_pre_connection part of the public API and have the 
> pre_connection hook of core just call it.
> 2. Create a wrapper around ap_run_pre_connection as a public API and if 
> ap_run_pre_connection returns != OK && != DONE do
> things like setting c->aborted to 1 and call core_pre_connection or do a 
> subset of it.
> 
> I would be in favour of 2. as 1. seems to provide an API function for only a 
> very specific case and of limited use. 2. seems to
> deliver a more "save" version of ap_run_pre_connection that could be used in 
> several locations as a drop in replacement for
> ap_run_pre_connection.

+1 for approach 2

Re: Separate beam timeout in mod_http2?

[ste...@eissing.org]

> Am 22.09.2021 um 17:30 schrieb Ruediger Pluem :
> 
> Currently we use the timeout setting of a virtual server hardcoded as the 
> beam timeout.
> While I think that this is a good default I have situations where I think it 
> would be beneficial to set this timeout separately.
> Opinions on a directive that allows to set this explicitly with a default of 
> the timeout setting of a virtual server?

I think I have to bring over my github branch "icing/pipes" into trunk soon. 

That is a thing that is fixed there. It sets the timeout of the 
request_rec->server on  the beams. I think this is what we want. Or do you 
foresee a special need here for h2 requests to be configured?

- Stefan

Re: svn commit: r1893530 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

> Am 23.09.2021 um 09:25 schrieb rpl...@apache.org:
> 
> Author: rpluem
> Date: Thu Sep 23 07:25:24 2021
> New Revision: 1893530
> 
> URL: http://svn.apache.org/viewvc?rev=1893530&view=rev
> Log:
> * Comment [skip ci]
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1893530&r1=1893529&r2=1893530&view=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Thu Sep 23 07:25:24 2021
> @@ -163,6 +163,7 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>   http://svn.apache.org/1893220
>  2.4.x patch: svn merge -c 1893214,1893215,1893220 ^/httpd/httpd/trunk .
>  +1: icing,
> + rpluem says: This one has merge conflicts.

Arg. modules/http2 in 2.4.x and trunk have been gone out of sync
and I am wading through changesets that have never been backported
and cause the newer changes to give merge conflicts.

FTR: I hate our trunk development model.

Re: svn commit: r1893543 - /httpd/httpd/branches/2.4.x/STATUS

[ste...@eissing.org]

> Am 23.09.2021 um 11:28 schrieb Yann Ylavic :
> 
> On Thu, Sep 23, 2021 at 11:14 AM  wrote:
>> 
>> Author: icing
>> Date: Thu Sep 23 09:13:59 2021
>> New Revision: 1893543
>> 
>> URL: http://svn.apache.org/viewvc?rev=1893543&view=rev
>> Log:
>> Reload of proposed mod_http2 backport of server graceful handling,
>> including all the tiny code changes that were never proposed.
>> 
>> 
>> Modified:
>>httpd/httpd/branches/2.4.x/STATUS
>> 
>> Modified: httpd/httpd/branches/2.4.x/STATUS
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1893543&r1=1893542&r2=1893543&view=diff
>> ==
>> --- httpd/httpd/branches/2.4.x/STATUS (original)
>> +++ httpd/httpd/branches/2.4.x/STATUS Thu Sep 23 09:13:59 2021
>> @@ -166,12 +166,14 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>>   *) mod_http2: when a server is restarted gracefully, any idle h2 worker
>>  threads are shut down immediately.
>>  Also, change OpenSSL API use for deprecations in OpenSLL 3.0.
>> + Adds all other, never proposed code changes to make a clean
>> + sync of http2 sources.
>>  trunk patch: http://svn.apache.org/r1893214
>>   http://svn.apache.org/r1893215
>>   http://svn.apache.org/r1893220
>> - 2.4.x patch: svn merge -c 1893214,1893215,1893220 ^/httpd/httpd/trunk .
>> +  and other never proposed code changes
>> + PR: https://github.com/apache/httpd/pull/270
> 
> This means that the never proposed code changes are in the PR only and
> not in trunk?

The code in 2.4.x would after this be identical to trunk, including all the
tiny things over the years that have never been proposed.

> Cheers;
> Yann.

Re: svn commit: r1893542 [1/12] - in /httpd/httpd/branches/2.4.x: ./ test/modules/ test/modules/http2/ test/modules/http2/conf/ test/modules/http2/data/ test/modules/http2/htdocs/ test/modules/http2/h

[ste...@eissing.org]

> Am 23.09.2021 um 11:32 schrieb Yann Ylavic :
> 
> On Thu, Sep 23, 2021 at 10:56 AM  wrote:
>> 
>> Author: icing
>> Date: Thu Sep 23 08:56:33 2021
>> New Revision: 1893542
>> 
>> URL: http://svn.apache.org/viewvc?rev=1893542&view=rev
>> Log:
>> * test: adding the modules/http2 test suite for easy availability
>>   in 2.4.x.
>> * configure.in: adding the test/modules/http2 .in files to process
> 
> Is this a backport or a special tests for 2.4.x?
> (wondering why there is no mergeinfo)
> 
> Cheers;
> Yann.
> 
> PS: sorry for being picky, but later merging gets easier in the end if
> we track merges..

Ok, ok. 

Re: svn commit: r1893549 - in /httpd/httpd/trunk: .travis.yml test/travis_run_linux.sh

[ste...@eissing.org]

Thanks, Rüdiger. I was about to go looking for that...

> Am 23.09.2021 um 13:12 schrieb rpl...@apache.org:
> 
> Author: rpluem
> Date: Thu Sep 23 11:12:25 2021
> New Revision: 1893549
> 
> URL: http://svn.apache.org/viewvc?rev=1893549&view=rev
> Log:
> * t/modules/http2.t is gone since 1893547. So do not use it any longer in
>  HTTP/2 tests and remove the dependency on Protocol::HTTP2::Client. 
>  Update test/travis_run_linux.sh to handle NO_TEST_FRAMEWORK.
> 
> Modified:
>httpd/httpd/trunk/.travis.yml
>httpd/httpd/trunk/test/travis_run_linux.sh
> 
> Modified: httpd/httpd/trunk/.travis.yml
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/.travis.yml?rev=1893549&r1=1893548&r2=1893549&view=diff
> ==
> --- httpd/httpd/trunk/.travis.yml (original)
> +++ httpd/httpd/trunk/.travis.yml Thu Sep 23 11:12:25 2021
> @@ -271,7 +271,7 @@ jobs:
>   env: APR_VERSION=1.7.0
>APU_VERSION=1.6.1 APU_CONFIG="--with-crypto"
>CONFIG="--enable-mods-shared=reallyall --with-mpm=event 
> --enable-mpms-shared=event"
> -   TESTS="t/modules/http2.t" TEST_INSTALL=1 TEST_H2=1
> +   NO_TEST_FRAMEWORK=1 TEST_INSTALL=1 TEST_H2=1
>   addons:
> apt:
>   sources:
> @@ -382,7 +382,7 @@ before_install:
>   - if [ "$TRAVIS_OS_NAME" = "linux" -a ! -v SKIP_TESTING ]; then
> if [ -v CLEAR_CACHE ]; then rm -rf ~/perl5; fi;
> cpanm --local-lib=~/perl5 local::lib && eval $(perl -I 
> ~/perl5/lib/perl5/ -Mlocal::lib);
> -CC=gcc cpanm --notest Net::SSL LWP::Protocol::https ExtUtils::Embed 
> Test::More AnyEvent DateTime HTTP::DAV Protocol::HTTP2::Client FCGI 
> AnyEvent::WebSocket::Client Apache::Test;
> +CC=gcc cpanm --notest Net::SSL LWP::Protocol::https ExtUtils::Embed 
> Test::More AnyEvent DateTime HTTP::DAV FCGI AnyEvent::WebSocket::Client 
> Apache::Test;
> fi
> 
> before_script:
> 
> Modified: httpd/httpd/trunk/test/travis_run_linux.sh
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/test/travis_run_linux.sh?rev=1893549&r1=1893548&r2=1893549&view=diff
> ==
> --- httpd/httpd/trunk/test/travis_run_linux.sh (original)
> +++ httpd/httpd/trunk/test/travis_run_linux.sh Thu Sep 23 11:12:25 2021
> @@ -94,22 +94,24 @@ if ! test -v SKIP_TESTING; then
> # Try to keep all potential coredumps from all processes
> sudo sysctl -w kernel.core_uses_pid=1 2>/dev/null || true
> 
> -if test -v WITH_TEST_SUITE; then
> -make check TESTS="${TESTS}" TEST_CONFIG="${TEST_ARGS}"
> -RV=$?
> -else
> -test -v TEST_INSTALL || make install
> -pushd test/perl-framework
> -perl Makefile.PL -apxs $PREFIX/bin/apxs
> -make test APACHE_TEST_EXTRA_ARGS="${TEST_ARGS} ${TESTS}"
> +if ! test -v NO_TEST_FRAMEWORK; then
> +if test -v WITH_TEST_SUITE; then
> +make check TESTS="${TESTS}" TEST_CONFIG="${TEST_ARGS}"
> RV=$?
> -popd
> -fi
> +else
> +test -v TEST_INSTALL || make install
> +pushd test/perl-framework
> +perl Makefile.PL -apxs $PREFIX/bin/apxs
> +make test APACHE_TEST_EXTRA_ARGS="${TEST_ARGS} ${TESTS}"
> +RV=$?
> +popd
> +fi
> 
> -# Skip further testing if a core dump was created during the test
> -# suite run above.
> -if test $RV -eq 0 && test -n "`ls test/perl-framework/t/core{,.*} 
> 2>/dev/null`"; then
> -RV=4
> +# Skip further testing if a core dump was created during the test
> +# suite run above.
> +if test $RV -eq 0 && test -n "`ls test/perl-framework/t/core{,.*} 
> 2>/dev/null`"; then
> +RV=4
> +fi
> fi
> 
> if test -v TEST_SSL -a $RV -eq 0; then
> 
> 

Re: svn commit: r1893563 - in /httpd/httpd/trunk/modules/http2: h2_version.h h2_workers.c

[ste...@eissing.org]

> Am 23.09.2021 um 15:51 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/23/21 3:44 PM, ic...@apache.org wrote:
>> Author: icing
>> Date: Thu Sep 23 13:44:58 2021
>> New Revision: 1893563
>> 
>> URL: http://svn.apache.org/viewvc?rev=1893563&view=rev
>> Log:
>>  * mod_http2: fix version suffix after sync with github
>>Make shutdown worker wait loop robust against timed wait
>>interruptions and report the correct seconds waited.
>> 
>> 
>> Modified:
>>httpd/httpd/trunk/modules/http2/h2_version.h
>>httpd/httpd/trunk/modules/http2/h2_workers.c
>> 
> 
>> Modified: httpd/httpd/trunk/modules/http2/h2_workers.c
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http2/h2_workers.c?rev=1893563&r1=1893562&r2=1893563&view=diff
>> ==
>> --- httpd/httpd/trunk/modules/http2/h2_workers.c (original)
>> +++ httpd/httpd/trunk/modules/http2/h2_workers.c Thu Sep 23 13:44:58 2021
>> @@ -319,9 +319,9 @@ static void workers_abort_idle(h2_worker
>> static apr_status_t workers_pool_cleanup(void *data)
>> {
>> h2_workers *workers = data;
>> -apr_time_t timout = apr_time_from_sec(1);
>> +apr_time_t end, timout = apr_time_from_sec(1);
> 
> timeout instead of timout ? :-)

It's a short timeout. ;-)

> 
>> apr_status_t rv;
>> -int i, n = 5;
>> +int n, wait_sec = 5;
>> 
>> ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, workers->s,
>>  "h2_workers: cleanup %d workers idling",
> 
> 
> Regards
> 
> Rüdiger

Re: Separate beam timeout in mod_http2?

[ste...@eissing.org]

> Am 24.09.2021 um 12:05 schrieb Ruediger Pluem :
> 
> 
> 
> On 9/22/21 6:14 PM, Ruediger Pluem wrote:
>> 
>> 
>> On 9/22/21 5:41 PM, ste...@eissing.org wrote:
>>> 
>>>> Am 22.09.2021 um 17:30 schrieb Ruediger Pluem :
>>>> 
>>>> Currently we use the timeout setting of a virtual server hardcoded as the 
>>>> beam timeout.
>>>> While I think that this is a good default I have situations where I think 
>>>> it would be beneficial to set this timeout separately.
>>>> Opinions on a directive that allows to set this explicitly with a default 
>>>> of the timeout setting of a virtual server?
>>> 
>>> I think I have to bring over my github branch "icing/pipes" into trunk 
>>> soon. 
>>> 
>>> That is a thing that is fixed there. It sets the timeout of the 
>>> request_rec->server on  the beams. I think this is what we want. Or do you 
>>> foresee a special need here for h2 requests to be configured?
>> 
>> I would like to configure it separately. I see a use case where we have a 
>> blocking write on socket side, because we have a slow
>> client and multiple streams on the h2 TCP connection that I give the single 
>> streams / tasks more time to beam things to the master
>> connection. In an ideal case I could configure this timeout on location / if 
>> level, but being able to do this on virtual host
>> level would be already very helpful.
>> 
> 
> Something like https://github.com/icing/mod_h2/pull/221 ?
> 

I commented on the PR, so we can have the discussion there.

> Regards
> 
> Rüdiger

pollset WAKEABLE

[ste...@eissing.org]

Does anyone know which platforms do *not* support WAKEABLE apr pollsets?

And if http2 should support this, would be adding an own wakeup pipe be the way 
to go?

Thanks,
Stefan

Re: pollset WAKEABLE

[ste...@eissing.org]

> Am 28.09.2021 um 14:04 schrieb Yann Ylavic :
> 
> On Tue, Sep 28, 2021 at 1:08 PM ste...@eissing.org  wrote:
>> 
>> Does anyone know which platforms do *not* support WAKEABLE apr pollsets?
> 
> Last time I checked (IIRC) all platforms support it on the APR side,
> every platform handles pipes at least it seems.
> Possibly it wasn't the case some (long) time ago, we are probably
> overly cautious with APR_POLLSET_WAKEABLE availability check in
> mpm_event..
> 
>> 
>> And if http2 should support this, would be adding an own wakeup pipe be the 
>> way to go?
> 
> Looks overly cautious to me :)

Uhm, due to an error report from Windows, I am reading the APR pollset
implementation again. And Windows does not support polling on pipes?

There is a special hack inside apr to realize a wakeup pipe, 
but otherwise it returns APR_EBADF. If I read it correctly...

That would pose some major difficulties.

> 
> 
> Cheers;
> Yann.

Re: pollset WAKEABLE

[ste...@eissing.org]

> Am 28.09.2021 um 14:10 schrieb ste...@eissing.org:
> 
> 
> 
>> Am 28.09.2021 um 14:04 schrieb Yann Ylavic :
>> 
>> On Tue, Sep 28, 2021 at 1:08 PM ste...@eissing.org  
>> wrote:
>>> 
>>> Does anyone know which platforms do *not* support WAKEABLE apr pollsets?
>> 
>> Last time I checked (IIRC) all platforms support it on the APR side,
>> every platform handles pipes at least it seems.
>> Possibly it wasn't the case some (long) time ago, we are probably
>> overly cautious with APR_POLLSET_WAKEABLE availability check in
>> mpm_event..
>> 
>>> 
>>> And if http2 should support this, would be adding an own wakeup pipe be the 
>>> way to go?
>> 
>> Looks overly cautious to me :)
> 
> Uhm, due to an error report from Windows, I am reading the APR pollset
> implementation again. And Windows does not support polling on pipes?
> 
> There is a special hack inside apr to realize a wakeup pipe, 
> but otherwise it returns APR_EBADF. If I read it correctly...
> 
> That would pose some major difficulties.

Hmm, apr_file_pipe_create() under Windows makes me cry...

This means I need another strategy under Windows. *sadge*

>> 
>> 
>> Cheers;
>> Yann.

mod_http2 progress

[ste...@eissing.org]

Just a FYI: I announced that I had a rewrite of the connection/stream
handling in the works and would bring that into trunk soon. An update
on how that is going:

v2.0.0-rc4 is on github now and solves the problem with pipes on our
beloved Window platform, thanks to user @nono303 for the testing.

I have asked on twitter for some more volunteers on testing that, so
maybe next week, when nothing more negative comes up, I will bring
that into trunk.

On a "real" OS, there will then be a apr_pollset for monitoring main
connections and streams (secondary connections). And those will then
have a file handle for input polling. This gives opportunities for
certain proxy modules to use pollsets also, if we want to.

Overall, this allowed the implementation to reduce complexity. The
naming should now make an easier read. Where applicable "conn_rec *c"
has been renamed to "c1" for a main connection and "c2" for a
secondary one and filters/functions have been renamed to show
which connection type they are intended for.

Beams have been simplified and no longer use "proxy" buckets. MMAP
buckets are transferred more efficiently.

Rüdiger's "H2StreamTimeout" has been added and test cases gave some
surprises. Turns out ap_die() behaves different depending on the
presence of the H1 header filter in the out filter chain. Who knew?

Performance is improved under load. I think its basically less CPU
usage that raises the request/s ceiling by 5-10% on my macOS machine
when serving static files.

Cheers,
Stefan