date:20181018


On 10/18/2018 07:29 PM, Rainer Jung wrote:

Am 19.10.2018 um 00:46 schrieb Dennis Clarke:

On 10/18/2018 04:57 PM, Rainer Jung wrote:

Am 18.10.2018 um 21:55 schrieb Dennis Clarke:
You debugger output shows jump = 0x101e2915c. This address is not 
divisible by 8, so it seems it confirms the alignment problem.




  0x101e2915c ??



I'm referring to https://bugs.php.net/bug.php?id=76745 and the line:



ah .. I was wondering where you saw that address ... I was looking at my
 xterm  :-\



...
t@1 (l@1) program terminated by signal BUS (invalid address alignment)
Current function is set_jump
   641   jump->next = NULL;
(dbx) where
current thread: t@1
=>[1] set_jump(jump = 0x101e2915c, compiler = 0x101e280b0, flags = 0), 
line 641 in "sljitLir.c"

^^0x101e2915c^
   [2] sljit_emit_jump(compiler = 0x101e280b0, type = 24), line 1320 in 
"sljitNativeSPARC_common.c"
   [3] mainloop_entry(common = 0x7fffd180, hascrorlf = 0), line 
3372 in "pcre_jit_compile.c"

...


Right.

I will circle back around on this after I see httpd 2.4.37/38/39 built
and running.  Don't be surprised if I may have questions and more
odd things.  I see a bootstrap of gcc happening tonight.

Dennis

Re: httpd and php integration


Am 19.10.2018 um 00:46 schrieb Dennis Clarke:

On 10/18/2018 04:57 PM, Rainer Jung wrote:

Am 18.10.2018 um 21:55 schrieb Dennis Clarke:
You debugger output shows jump = 0x101e2915c. This address is not 
divisible by 8, so it seems it confirms the alignment problem.




  0x101e2915c ??

Not sure where you are seeing that but .. I am going back to start over
with some updates and possibly a new gcc bootstrap and pcre and whatever
else I can throw at the wall here. Really the Apache 2.4.37 ( today 37
and tomorrow 38? ) is what I am aiming for.


I'm referring to https://bugs.php.net/bug.php?id=76745 and the line:

...
t@1 (l@1) program terminated by signal BUS (invalid address alignment)
Current function is set_jump
  641   jump->next = NULL;
(dbx) where 


current thread: t@1
=>[1] set_jump(jump = 0x101e2915c, compiler = 0x101e280b0, flags = 0), 
line 641 in "sljitLir.c"

^^0x101e2915c^
  [2] sljit_emit_jump(compiler = 0x101e280b0, type = 24), line 1320 in 
"sljitNativeSPARC_common.c"
  [3] mainloop_entry(common = 0x7fffd180, hascrorlf = 0), line 
3372 in "pcre_jit_compile.c"

...

Regards,

Rainer

Re: httpd and php integration


On 10/18/2018 04:57 PM, Rainer Jung wrote:

Am 18.10.2018 um 21:55 schrieb Dennis Clarke:

On 10/18/2018 03:42 PM, Rainer Jung wrote:

Am 18.10.2018 um 21:18 schrieb Dennis Clarke:

On 10/18/2018 02:12 PM, Daniel Ruggeri wrote:

php: "5.6.38"




I do build PHP 7.x myself including recent library versions both on 
some Linux platforms as well as on Solaris Sparc. But the typical use 
is via PHP-FPM, not mod_php. So httpd integration is done via 
mod_proxy_fcgi which is part of httpd nowadays. I do not test PHP 
with the httpd test suite, only with the tests that come with PHP 
itself.


Yes, I see the php-fpm approach and ye old mod_php was fine for php 5.x
world but the world has moved onwards.

When running the test suite I typically get about 50-80 test failures 
on Linux and 150-200 test failures on Solaris (of about 13000 
non-skipped tests).


I get outright SIGBUS core dump :-(


Yes, I sometimes get them and try to get them analyzed (by myself) and 
fixed (by the PHP people). On Solaris Sparc the SIGBUS almost always 
comes from bad alignment, where an 8 byte type is not located at an 
address divisible by 8. X86_64 is not sensible to this and of course the 
PHP people do not typically test on Sparc.


*nod*

Concerning the 7.2.8 failure: the biggest difference between our setups 
is probably the compiler. I wanted to keep deltas to the mostly used 
platform Linux small, so I build using GCC (which I compile myself).


You and I seem to be the last people still bootstrapping gcc on Solaris
sparc but that is another discussion on another maillist ;-)

The 
alignment in question could well be sensible to the compiler used. 
Furthermore I am doing still 32 bit builds, and it looks like you are 
doing 64 bit builds.


Pure 64-bit has been my way for almost the last decade. I think I waved
farewell to Solaris 8 back in 2008 and that resulted in a near revolt
within "Blastwave" because, well, money.  That is an old story.  Also we
all thought OpenSolaris was going to change the landscape.  Another old
story.  However my 64-bit stack has been the floor plan stuff that kept
a small finance company running production software for many years and
that included recent Apache httpd and php and the usual stuff needed for
Wordpress sites and a lot of custom internal stuff.  Gets more difficult
every day to deal with updates however and here we are looking at httpd
with tls 1.3 and with httpd ver 2.4.37.  All good stuff to be sure.



Looking at your issue

https://bugs.php.net/bug.php?id=76745

I did not myself observe this problem. It looks like a pcre problem, 


I was thinking the same and built my own.

beta $ file /usr/local/bin/pcretest
/usr/local/bin/pcretest: ELF 64-bit MSB executable SPARCV9 Version 1, 
UltraSPARC3 Extensions Required, dynamically linked, not stripped

beta $
beta $ /usr/local/bin/pcretest -C
PCRE version 8.40 2017-01-11
Compiled with
  8-bit support
  UTF-8 support
  16-bit support
  UTF-16 support
  32-bit support
  UTF-32 support
  Unicode properties support
  No just-in-time compiler support
  Newline sequence is LF
  \R matches all Unicode newlines
  Internal link size = 4
  POSIX malloc threshold = 10
  Parentheses nest limit = 250
  Default match limit = 1000
  Default recursion depth limit = 1000
  Match recursion uses heap
beta $

Works very very well. So I am surprised at the alignment. In fact I have 
*had* it running neatly for a while. I often wondered what that internal

link size really means ... hrmmm .. 4 ? 32-bit? Could be a red herring.

not 
really a PHP problem. The pcre they bundle in 7.2.8 is version 8.41. 
Recent for 8.x would be 8.42, but even in 7.2.11 it is still at 8.41. On 
the other hand, I did not find an immediate information, that 8.42 fixes 
an alignment problem in the jit. But it does contain quite a few jit 
changes.


OKay .. I am going back to the source well and perhaps bootstrap GCC 8.x
again and why not? It has been a over a month since I looked.


You debugger output shows jump = 0x101e2915c. This address is not 
divisible by 8, so it seems it confirms the alignment problem.




 0x101e2915c ??

Not sure where you are seeing that but .. I am going back to start over
with some updates and possibly a new gcc bootstrap and pcre and whatever
else I can throw at the wall here. Really the Apache 2.4.37 ( today 37
and tomorrow 38? ) is what I am aiming for.


You could try to reproduce the problem by compiling pcre 8.41 standalone 
and run the pcre test suite.


Yep. That first.

Uf you can reproduce, you can check, 
whether it is goine in 8.42 and then at least point the php people to 
that finding, asking them for an upgrade. Finally you can also try to 
add pcre 8.42 into your php source tree under ext/pcre/pcrelib.


I may do both.  Actually ... now that I think on it.

It seems 
PHP does not change any of the files, but you would also need to add the 
generated files config.h, pcre_chartables.c and pcre.h. You can probably 
get those by fi

Re: svn commit: r1844286 - /httpd/test/framework/trunk/t/ssl/ocsp.t


Thanks!

Am 18.10.2018 um 23:10 schrieb yla...@apache.org:

Author: ylavic
Date: Thu Oct 18 21:10:10 2018
New Revision: 1844286

URL: http://svn.apache.org/viewvc?rev=1844286&view=rev
Log:
Add 'use Net::SSLeay' required by Net::SSLeay::OPENSSL_VERSION_NUMBER().

Modified:
 httpd/test/framework/trunk/t/ssl/ocsp.t

Modified: httpd/test/framework/trunk/t/ssl/ocsp.t
URL: 
http://svn.apache.org/viewvc/httpd/test/framework/trunk/t/ssl/ocsp.t?rev=1844286&r1=1844285&r2=1844286&view=diff
==
--- httpd/test/framework/trunk/t/ssl/ocsp.t (original)
+++ httpd/test/framework/trunk/t/ssl/ocsp.t Thu Oct 18 21:10:10 2018
@@ -6,6 +6,8 @@ use Apache::TestSSLCA;
  use Apache::TestRequest;
  use Apache::TestConfig ();
  
+use Net::SSLeay

+
  #if keepalives are on, renegotiation not happen again once
  #a client cert is presented.  so on test #3, the cert from #2
  #will be used.  this test scenerio would never

Re: httpd and php integration


Am 18.10.2018 um 21:55 schrieb Dennis Clarke:

On 10/18/2018 03:42 PM, Rainer Jung wrote:

Am 18.10.2018 um 21:18 schrieb Dennis Clarke:

On 10/18/2018 02:12 PM, Daniel Ruggeri wrote:

php: "5.6.38"




I do build PHP 7.x myself including recent library versions both on 
some Linux platforms as well as on Solaris Sparc. But the typical use 
is via PHP-FPM, not mod_php. So httpd integration is done via 
mod_proxy_fcgi which is part of httpd nowadays. I do not test PHP with 
the httpd test suite, only with the tests that come with PHP itself.


Yes, I see the php-fpm approach and ye old mod_php was fine for php 5.x
world but the world has moved onwards.

When running the test suite I typically get about 50-80 test failures 
on Linux and 150-200 test failures on Solaris (of about 13000 
non-skipped tests).


I get outright SIGBUS core dump :-(


Yes, I sometimes get them and try to get them analyzed (by myself) and 
fixed (by the PHP people). On Solaris Sparc the SIGBUS almost always 
comes from bad alignment, where an 8 byte type is not located at an 
address divisible by 8. X86_64 is not sensible to this and of course the 
PHP people do not typically test on Sparc.


Concerning the 7.2.8 failure: the biggest difference between our setups 
is probably the compiler. I wanted to keep deltas to the mostly used 
platform Linux small, so I build using GCC (which I compile myself). The 
alignment in question could well be sensible to the compiler used. 
Furthermore I am doing still 32 bit builds, and it looks like you are 
doing 64 bit builds.


Looking at your issue

https://bugs.php.net/bug.php?id=76745

I did not myself observe this problem. It looks like a pcre problem, not 
really a PHP problem. The pcre they bundle in 7.2.8 is version 8.41. 
Recent for 8.x would be 8.42, but even in 7.2.11 it is still at 8.41. On 
the other hand, I did not find an immediate information, that 8.42 fixes 
an alignment problem in the jit. But it does contain quite a few jit 
changes.


You debugger output shows jump = 0x101e2915c. This address is not 
divisible by 8, so it seems it confirms the alignment problem.


You could try to reproduce the problem by compiling pcre 8.41 standalone 
and run the pcre test suite. Uf you can reproduce, you can check, 
whether it is goine in 8.42 and then at least point the php people to 
that finding, asking them for an upgrade. Finally you can also try to 
add pcre 8.42 into your php source tree under ext/pcre/pcrelib. It seems 
PHP does not change any of the files, but you would also need to add the 
generated files config.h, pcre_chartables.c and pcre.h. You can probably 
get those by first running configure and maybe also make for a 
standalone pcre 8.42.


Concerning your foo.php script, I get the same results, but it starts 
working when using INPUT_ENV. I don't know what the definition of 
INPUT_SERVER for PHP CLI is. At least for PHP with FCGI it seems to be 
known that INPUT_SERVER doesn't work well. So probably not something 
specific to your build.


Regards,

Rainer


beta $ TEST_PHP_EXECUTABLE=sapi/cli/php sapi/cli/php run-tests.php

=
PHP : sapi/cli/php
PHP_SAPI    : cli
PHP_VERSION : 7.2.8
ZEND_VERSION: 3.2.0
PHP_OS  : SunOS - SunOS beta 5.10 Generic_150400-61 sun4u
INI actual  : /usr/local/lib/php.ini
More .INIs  :
-
PHP : 
/usr/local/build/php-7.2.8_SunOS5.10_sparc64vii+.001/sapi/phpdbg/phpdbg

PHP_SAPI    : phpdbg
PHP_VERSION : 7.2.8
ZEND_VERSION: 3.2.0
PHP_OS  : SunOS - SunOS beta 5.10 Generic_150400-61 sun4u
INI actual  : /usr/local/lib/php.ini
More .INIs  :
-
CWD : /usr/local/build/php-7.2.8_SunOS5.10_sparc64vii+.001
Extra dirs  :
VALGRIND    : Not used
=
TIME START 2018-10-18 19:18:58
=
Bus Error(coredump)
beta $
beta $ dbx sapi/cli/php 
time_1539890338-pid_2068-uid_16411-gid_20002-fid_php.core

For information about new features see `help changes'
To remove this message, put `dbxenv suppress_startup_message 8.2' in 
your .dbxrc

Reading php
core file header read successfully
Reading ld.so.1
Reading libresolv.so.2
Reading librt.so.1
Reading libm.so.2
Reading libnsl.so.1
Reading libsocket.so.1
Reading libz.so.1.2.8
Reading libxml2.so.2.9.2
Reading liblzma.so.5.2.1
Reading libpthread.so.1
Reading libiconv.so.2.6.0
Reading libc.so.1
Reading libaio.so.1
Reading libmd.so.1
Reading libc_psr.so.1
Reading en_US.UTF-8.so.3
Reading methods_unicode.so.3
t@1 (l@1) program terminated by signal BUS (invalid address alignment)
Current function is set_label
   630   label->next = NULL;
(dbx) where
current thread: t@1
=>[1] set_label(label = 0x101ffa82c, compiler = 0x101ff9780), line 630 
in "sljitLir.c"
   [2] sl

httpd and php integration


On 10/18/2018 03:42 PM, Rainer Jung wrote:

Am 18.10.2018 um 21:18 schrieb Dennis Clarke:

On 10/18/2018 02:12 PM, Daniel Ruggeri wrote:

php: "5.6.38"




I do build PHP 7.x myself including recent library versions both on some 
Linux platforms as well as on Solaris Sparc. But the typical use is via 
PHP-FPM, not mod_php. So httpd integration is done via mod_proxy_fcgi 
which is part of httpd nowadays. I do not test PHP with the httpd test 
suite, only with the tests that come with PHP itself.


Yes, I see the php-fpm approach and ye old mod_php was fine for php 5.x
world but the world has moved onwards.

When running the test suite I typically get about 50-80 test failures on 
Linux and 150-200 test failures on Solaris (of about 13000 non-skipped 
tests).


I get outright SIGBUS core dump :-(

beta $ TEST_PHP_EXECUTABLE=sapi/cli/php sapi/cli/php run-tests.php

=
PHP : sapi/cli/php
PHP_SAPI: cli
PHP_VERSION : 7.2.8
ZEND_VERSION: 3.2.0
PHP_OS  : SunOS - SunOS beta 5.10 Generic_150400-61 sun4u
INI actual  : /usr/local/lib/php.ini
More .INIs  :
-
PHP : 
/usr/local/build/php-7.2.8_SunOS5.10_sparc64vii+.001/sapi/phpdbg/phpdbg

PHP_SAPI: phpdbg
PHP_VERSION : 7.2.8
ZEND_VERSION: 3.2.0
PHP_OS  : SunOS - SunOS beta 5.10 Generic_150400-61 sun4u
INI actual  : /usr/local/lib/php.ini
More .INIs  :
-
CWD : /usr/local/build/php-7.2.8_SunOS5.10_sparc64vii+.001
Extra dirs  :
VALGRIND: Not used
=
TIME START 2018-10-18 19:18:58
=
Bus Error(coredump)
beta $
beta $ dbx sapi/cli/php 
time_1539890338-pid_2068-uid_16411-gid_20002-fid_php.core

For information about new features see `help changes'
To remove this message, put `dbxenv suppress_startup_message 8.2' in 
your .dbxrc

Reading php
core file header read successfully
Reading ld.so.1
Reading libresolv.so.2
Reading librt.so.1
Reading libm.so.2
Reading libnsl.so.1
Reading libsocket.so.1
Reading libz.so.1.2.8
Reading libxml2.so.2.9.2
Reading liblzma.so.5.2.1
Reading libpthread.so.1
Reading libiconv.so.2.6.0
Reading libc.so.1
Reading libaio.so.1
Reading libmd.so.1
Reading libc_psr.so.1
Reading en_US.UTF-8.so.3
Reading methods_unicode.so.3
t@1 (l@1) program terminated by signal BUS (invalid address alignment)
Current function is set_label
  630   label->next = NULL;
(dbx) where 


current thread: t@1
=>[1] set_label(label = 0x101ffa82c, compiler = 0x101ff9780), line 630 
in "sljitLir.c"
  [2] sljit_emit_label(compiler = 0x101ff9780), line 1251 in 
"sljitNativeSPARC_common.c"
  [3] php__pcre_jit_compile(re = 0x101ff9570, extra = 0x101ff9600, mode 
= 0), line 11081 in "pcre_jit_compile.c"
  [4] php_pcre_study(external_re = 0x101ff9570, options = 1, errorptr = 
0x7fffdb10), line 1630 in "pcre_study.c"
  [5] pcre_get_compiled_regex_cache(regex = 0x7d0b5630), line 
548 in "php_pcre.c"
  [6] php_do_pcre_match(execute_data = 0x7d023810, return_value 
= 0x7d01efa0, global = 0), line 729 in "php_pcre.c"
  [7] zif_preg_match(execute_data = 0x7d023810, return_value = 
0x7d01efa0), line 1147 in "php_pcre.c"
  [8] ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER(execute_data = 
0x7d01e510), line 617 in "zend_vm_execute.h"
  [9] execute_ex(ex = 0x7d01c030), line 59739 in 
"zend_vm_execute.h"
  [10] zend_execute(op_array = 0x7d07e540, return_value = 
(nil)), line 63776 in "zend_vm_execute.h"
  [11] zend_execute_scripts(type = 8, retval = (nil), file_count = 3, 
... = 0x17d06c060, ...), line 1496 in "zend.c"
  [12] php_execute_script(primary_file = 0x70f8), line 2590 
in "main.c"

  [13] do_cli(argc = 2, argv = 0x101c8ed20), line 1011 in "php_cli.c"
  [14] main(argc = 2, argv = 0x101c8ed20), line 1404 in "php_cli.c"
(dbx) list
  630   label->next = NULL;
  631   label->size = compiler->size;
  632   if (compiler->last_label)
  633   compiler->last_label->next = label;
  634   else
  635   compiler->labels = label;
  636   compiler->last_label = label;
  637   }
  638
  639   static SLJIT_INLINE void set_jump(struct sljit_jump *jump, 
struct sljit_compiler *compiler, sljit_s32 flags)

(dbx) print label
label = 0x101ffa82c
(dbx) print *label
*label = {
next = (nil)
addr = 0
size = 0
}
(dbx) quit

Which has me wonder how that is possible given that I tend to enforce
a 64-bit alignment with -xmemalign=8s .

Regardless php has been a battle on Solaris sparc. Would love to know 
how you managed to get any sort of reasonable results.




I did get some false positive fixed by the PHP people, but got somewhat 
tired of analyzing the big number of failing tests.

Re: [VOTE] Release httpd-2.4.37


Am 18.10.2018 um 21:18 schrieb Dennis Clarke:

On 10/18/2018 02:12 PM, Daniel Ruggeri wrote:

php: "5.6.38"


Slightly off topic but I see you have ye old php 5.6.38 there. Was this
built and installed yourself?

Just curious is there is any guidance anywhere regarding php 7.x which
builds but it is a religious experience complete with prayer and black
cats under a full moon and scotch :

beta $ ./sapi/cli/php --version
PHP 7.2.8 (cli) (built: Aug 15 2018 03:13:17) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
beta $

beta $ file ./sapi/cli/php
./sapi/cli/php: ELF 64-bit MSB executable SPARCV9 Version 1, dynamically 
linked, not stripped

beta $

The php folks are entirely and completely horrific at dealing with
anything that appears broken and their testing docs are quite blunt
about the fact that they don't want to hear from you.  In fact, you
are wrong, they are correct, go away :

beta $ more README.TESTING
[IMPORTANT NOTICE]
--
  Failed tests usually indicate a problem with your local system setup
and not within PHP itself (at least for official PHP release versions).
You may decide to automatically submit a test summary to our QA workflow
at the end of a test run.
  Please do *not* submit a failed test as a bug or ask for help on why
it failed on your system without providing substantial backup information
on *why* the test failed on your special setup. Thank you :-)


So have you tried anything with php 7.x from sources and made any 
progress with manually integrating httpd and php 7.x ??


Dennis Clarke


I do build PHP 7.x myself including recent library versions both on some 
Linux platforms as well as on Solaris Sparc. But the typical use is via 
PHP-FPM, not mod_php. So httpd integration is done via mod_proxy_fcgi 
which is part of httpd nowadays. I do not test PHP with the httpd test 
suite, only with the tests that come with PHP itself.


When running the test suite I typically get about 50-80 test failures on 
Linux and 150-200 test failures on Solaris (of about 13000 non-skipped 
tests).


I did get some false positive fixed by the PHP people, but got somewhat 
tired of analyzing the big number of failing tests.


Regards,

Rainer

Re: svn commit: r1843478 - /httpd/test/framework/trunk/t/ssl/ocsp.t

2018-10-18 Thread Jim Jagielski




> On Oct 16, 2018, at 11:36 AM, William A Rowe Jr  wrote:
> 
> To button this issue up, it's clear to me that Jim had transposed the meaning 
> of result values from posix commands, and that was the origin of 
> irrationality in this discussion.
> 

Actually, I did not. But thanks for playing. I will ignore the implied insult.

Re: [VOTE] Release httpd-2.4.37


On 10/18/2018 02:12 PM, Daniel Ruggeri wrote:

php: "5.6.38"


Slightly off topic but I see you have ye old php 5.6.38 there. Was this
built and installed yourself?

Just curious is there is any guidance anywhere regarding php 7.x which
builds but it is a religious experience complete with prayer and black
cats under a full moon and scotch :

beta $ ./sapi/cli/php --version
PHP 7.2.8 (cli) (built: Aug 15 2018 03:13:17) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
beta $

beta $ file ./sapi/cli/php
./sapi/cli/php: ELF 64-bit MSB executable SPARCV9 Version 1, dynamically 
linked, not stripped

beta $

The php folks are entirely and completely horrific at dealing with
anything that appears broken and their testing docs are quite blunt
about the fact that they don't want to hear from you.  In fact, you
are wrong, they are correct, go away :

beta $ more README.TESTING
[IMPORTANT NOTICE]
--
 Failed tests usually indicate a problem with your local system setup
and not within PHP itself (at least for official PHP release versions).
You may decide to automatically submit a test summary to our QA workflow
at the end of a test run.
 Please do *not* submit a failed test as a bug or ask for help on why
it failed on your system without providing substantial backup information
on *why* the test failed on your special setup. Thank you :-)


So have you tried anything with php 7.x from sources and made any 
progress with manually integrating httpd and php 7.x ??


Dennis Clarke

Re: [VOTE] Release httpd-2.4.37

2018-10-18 Thread Daniel Ruggeri


On 2018-10-18 09:36, Daniel Ruggeri wrote:

Hi, all;
   Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this
candidate tarball as 2.4.37:
[ ] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: b0521606d1df54bb425adcdecf6348f126aa352c *httpd-2.4.37.tar.gz
sha256:
aa97a834a32d51974be8d8a013b561e28d327387cb1da2c3c2762acd0146aabd
*httpd-2.4.37.tar.gz


+1 from me based on the following setup

system:
  kernel:
name: Linux
release: 3.16.0-4-amd64
version: #1 SMP Debian 3.16.51-3 (2017-12-13)
machine: x86_64

  libraries:
openssl: "1.1.1"
openldap: "2.4.46"
apr: "1.6.5"
apr-util: "1.6.1"
iconv: "1.2.2"
brotli: "1.0.6"
nghttp2: "1.34.0"
zlib: "1.2.11"
pcre: "8.42"
libxml2: "2.9.8"
php: "5.6.38"
lua: "5.3.5"
curl: "7.61.1"
--
Daniel Ruggeri

Re: svn commit: r1844231 - in /httpd/httpd/branches/2.4.x/docs/manual: ./ faq/ howto/ misc/ mod/ platform/ programs/ rewrite/ ssl/ style/ style/lang/ style/xsl/util/ vhosts/

2018-10-18 Thread Lucien Gentis



Le 18/10/2018 à 15:49, William A Rowe Jr a écrit :
Please never do this again on the eve of a release, it is not easily 
reviewed

and is very inconsiderate to the RM. This doesn't meet the idea of minimal
scope, or the spirit of docs@h.a.o being exempt from backport review(!)

That said...

https://svn.apache.org/viewvc?limit_changes=0&view=revision&revision=1844231

httpd/httpd/branches/2.4.x/docs/manual/style/lang/fr.xml
httpd/httpd/branches/2.4.x/docs/manual/style/lang-targets.xml
httpd/httpd/branches/2.4.x/docs/manual/style/manual.fr.xsl
httpd/httpd/branches/2.4.x/docs/manual/style/xsl/util/designations.xml

These were sources, and MUST NOT be changed on a regeneration commit!

Changed paths:458

This corresponds to 454 non-/style/ changes, which have a 2:1 
effective count

to update the map file (.html) and .html.fr.utf8 result files.


OK, I forgot committing style changes before rebuild, apologize.




However there are a couple other quirks,

httpd/httpd/branches/2.4.x/docs/manual/mod/mod_ssl.xml.meta
httpd/httpd/branches/2.4.x/docs/manual/programs/ab.xml.meta

Each of these updated the 'fr' translation from outdated to current. 
That could

have been done as a distinct pass.



I committed last changes on these 2 files (mod_ssl.xml.fr and ab.xml.fr) 
last wednesday ???





I count an effective 226 new files, so contrasting to

https://svn.apache.org/viewvc?limit_changes=0&view=revision&revision=1844229

Author: lgentis
Date: Thu Oct 18 11:20:54 2018
New Revision: 1844229

URL: http://svn.apache.org/viewvc?rev=1844229&view=rev
Log:
fr doc : deleting ISO-8859-1 html files.

Changed paths:226

It is reasonable to conclude that this patch was correct.

As I understand it, relying on the ".html" named var content map 
files, there were no further changes required to 
docs/conf/extra/httpd-manual.conf.in  - 
correct? It all simply resolves itself based on the content-language 
match, and the codepage/charset were incidental.


This looks correct for release, but wasn't how I wanted to spend the 
morning. Hoping to save some others from the same exercise.



On Thu, Oct 18, 2018 at 6:38 AM > wrote:


Author: lgentis
Date: Thu Oct 18 11:38:42 2018
New Revision: 1844231

URL: http://svn.apache.org/viewvc?rev=1844231&view=rev
Log:
fr doc : adding UTF-8 html files.


[This commit notification would consist of 79 parts,
which exceeds the limit of 50 ones, so it was shortened to the
summary.]

Re: OCSP in 2.4 with OpenSSL 0.9.8(zh)


Some answers inline and the solution at the end ...

Am 18.10.2018 um 15:01 schrieb William A Rowe Jr:
On Thu, Oct 18, 2018 at 7:27 AM Rainer Jung > wrote:


I get test suite failures for t/ssl/ocsp.t when the server is build
against OpenSSL 0.9.8zh. I can't judge on whether that is expected for
OpenSSL 0.9.8.


A very good question, and I can't either. Can you confirm your openssl
command line tool has the `openssl ocsp` mini-responder by posting the
results of an `openssl ocsp -help` invocation?


$ openssl ocsp -help
OCSP utility
Usage ocsp [options]
...

$ openssl version -a
OpenSSL 0.9.8zh 3 Dec 2015
built on: Tue Sep 11 11:20:47 CEST 2018
platform: linux-x86_64
options:  bn(64,64) md2(int) rc4(1x,char) des(idx,cisc,16,int) idea(int) 
blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT 
-DDSO_DLFCN -DHAVE_DLFCN_H -fPIC -g -Wall -fno-strict-aliasing 
-Wa,--noexecstack -m64 -DL_ENDIAN -DTERMIO -O3 -Wall -DMD32_REG_T=int 
-DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM 
-DAES_ASM

OPENSSLDIR: "..."


It might be that we never handled ocsp here.

It might also be that your $openssl resolves to a system tool which is not
in sync with the openssl tested in httpd. You may want to override that 
value.


Should not. I'm handling so many OpenSSL versions on the client and 
server side, so I'm typicaly really careful to set up the PATH etc. so 
the right tools are found. But even the platform openssl supports ocsp.



And may be httpd never supported the ocsp directives with 0.9.8, so our
tests for the micro responder and the version of httpd alone are not 
sufficient.


Found it: OpenSSL 0.9.8 doesn't allow "ocsp -reqin -" which is used by 
the ocsp.pl script in the test suite. Reading from stdin works in 1.0.2 
but throws the following error in 0.9.8:


Error Opening OCSP request file
3487:error:02001002:system library:fopen:No such file or 
directory:bss_file.c:124:fopen('-','rb')

3487:error:2006D080:BIO routines:BIO_new_file:no such file:bss_file.c:127:

It also does not work in 1.0.1, so our OCSP tests should not run for 
OpenSSL < 1.0.2 (or we must fix the perl script by using a tmp file).


Regards,

Rainer

[VOTE] Release httpd-2.4.37

2018-10-18 Thread Daniel Ruggeri


Hi, all;
   Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.37:

[ ] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: b0521606d1df54bb425adcdecf6348f126aa352c *httpd-2.4.37.tar.gz
sha256: aa97a834a32d51974be8d8a013b561e28d327387cb1da2c3c2762acd0146aabd 
*httpd-2.4.37.tar.gz


--
Daniel Ruggeri

Re: OCSP in 2.4 with OpenSSL 0.9.8(zh)

On Thu, Oct 18, 2018 at 8:01 AM William A Rowe Jr 
wrote:

> On Thu, Oct 18, 2018 at 7:27 AM Rainer Jung 
> wrote:
>
>> I get test suite failures for t/ssl/ocsp.t when the server is build
>> against OpenSSL 0.9.8zh. I can't judge on whether that is expected for
>> OpenSSL 0.9.8.
>
>
> A very good question, and I can't either. Can you confirm your openssl
> command line tool has the `openssl ocsp` mini-responder by posting the
> results of an `openssl ocsp -help` invocation?
>
> It might be that we never handled ocsp here.
>
> It might also be that your $openssl resolves to a system tool which is not
> in sync with the openssl tested in httpd. You may want to override that
> value.
>

To override, it seems you need to use an envvar;

./Apache-Test/lib/Apache/TestSSLCA.pm:

my $openssl = $ENV{APACHE_TEST_OPENSSL_CMD} || 'openssl';

We offer no t/TEST -openssl= option.

Re: [NOTICE] Intent to T&R 2.4.37 - about 12:00 GMT tomorrow

2018-10-18 Thread Daniel Ruggeri

On 2018-10-18 07:12, Rainer Jung wrote:

Am 17.10.2018 um 13:41 schrieb Daniel Ruggeri:

Hi, all;
With the fix for detected OpenSSL 1.1.1 issues now backported to
2.4.x, I would like to tag the next version of our venerable server
soon.

I have already successfully completed the test suite against my
"latest sources" docker environment and am watching for any smoke
detected in [1]. Feeling good about this one :-)

How about roughly 24 hours from now?

[1]
https://lists.apache.org/thread.html/48de97bd66ceabcf84a3719b36cd69274cb8c4b64d68c46696beb906@

In the meantime most of my tests finished. The two small mod_ssl
patches applied this morning were not part of the testing but seem
simple enough to understand and should pose no risk.

My testing showed:

- t/ssl/ocsp.t fails in test 2 and 3 (lines 43 and 49) when the server
is build using OpenSSL 0.9.8zh:
Can't connect to localhost:8535 (SSL connect attempt failed because of
handshake problems error:14094410:SSL routines:SSL3_READ_BYTES:sslv3
alert handshake failure)
SSL connect attempt failed because of handshake problems
error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake
failure at
/shared/build/dev/httpd/install/Bundle-ApacheTest/20180911-0.9.8zh-1/rhel7.x86_64/lib/perl5/LWP/Protocol/http.pm
line 50.

I don't know whether that is expected for old OpenSSL, so can not
judge on criticality.

- t/modules/http2.t fails when the server is build using OpenSSL
0.9.8zh with the "Bad plan. You planned 52 tests..." message
indicating, that h2 using TLS does not work. It happens on all
platforms, but not if the client also uses OpenSSL 0.9.8zh.

I don't know whether that is expected for old OpenSSL, so can not
judge on criticality.

- only once out of 68 runs on Solaris failure in t/modules/cgi.t test
54 in line 232. There log contents are checked and the file system is
on NFS. Might be, that this is a timing issue in the test. Not a
show-stopper for me.

- only once out of 68 runs on Solaris failure in t/ssl/proxy.t test
106 in line 131. /eat_post responds with a proxy error (502) instead
of 200 with the posted content length as the response body. Need to
investigate but would also say not a show-stopper, because only on
Solaris and only once.

- some crashes on Solaris when building the server statically linked.
Only with event MPM and looks like always at the end of a process
lifetime, typically during shutdown. Maybe a problem with duplicate
OpenSSL unloading/cleanup (apr-util plus mod_ssl). I think its a known
problem, but no fix yet available. Since it should not happen to
processes which are in use I would say it is more of an annoyance and
not a show-stopper.

Regards,

Rainer

Thank you so much for the thorough testing. I see that the H2 failure
case makes sense based on feedback. I also suspect there is a strong
lead on the ocsp case. I'm also pleased to see the backports have
already made it into 2.4.x so I think we're good to go.

--
Daniel Ruggeri

Re: svn commit: r1844231 - in /httpd/httpd/branches/2.4.x/docs/manual: ./ faq/ howto/ misc/ mod/ platform/ programs/ rewrite/ ssl/ style/ style/lang/ style/xsl/util/ vhosts/

Please never do this again on the eve of a release, it is not easily
reviewed
and is very inconsiderate to the RM. This doesn't meet the idea of minimal
scope, or the spirit of docs@h.a.o being exempt from backport review(!)

That said...

https://svn.apache.org/viewvc?limit_changes=0&view=revision&revision=1844231

httpd/httpd/branches/2.4.x/docs/manual/style/lang/fr.xml
httpd/httpd/branches/2.4.x/docs/manual/style/lang-targets.xml
httpd/httpd/branches/2.4.x/docs/manual/style/manual.fr.xsl
httpd/httpd/branches/2.4.x/docs/manual/style/xsl/util/designations.xml

These were sources, and MUST NOT be changed on a regeneration commit!

Changed paths: 458

This corresponds to 454 non-/style/ changes, which have a 2:1 effective
count
to update the map file (.html) and .html.fr.utf8 result files.

However there are a couple other quirks,

httpd/httpd/branches/2.4.x/docs/manual/mod/mod_ssl.xml.meta
httpd/httpd/branches/2.4.x/docs/manual/programs/ab.xml.meta

Each of these updated the 'fr' translation from outdated to current. That
could
have been done as a distinct pass.

I count an effective 226 new files, so contrasting to

https://svn.apache.org/viewvc?limit_changes=0&view=revision&revision=1844229

Author: lgentis
Date: Thu Oct 18 11:20:54 2018
New Revision: 1844229

URL: http://svn.apache.org/viewvc?rev=1844229&view=rev
Log:
fr doc : deleting ISO-8859-1 html files.

Changed paths: 226

It is reasonable to conclude that this patch was correct.

As I understand it, relying on the ".html" named var content map files,
there were no further changes required to docs/conf/extra/
httpd-manual.conf.in - correct? It all simply resolves itself based on the
content-language match, and the codepage/charset were incidental.

This looks correct for release, but wasn't how I wanted to spend the
morning. Hoping to save some others from the same exercise.

On Thu, Oct 18, 2018 at 6:38 AM  wrote:

> Author: lgentis
> Date: Thu Oct 18 11:38:42 2018
> New Revision: 1844231
>
> URL: http://svn.apache.org/viewvc?rev=1844231&view=rev
> Log:
> fr doc : adding UTF-8 html files.
>
>
> [This commit notification would consist of 79 parts,
> which exceeds the limit of 50 ones, so it was shortened to the summary.]
>

Re: OCSP in 2.4 with OpenSSL 0.9.8(zh)

On Thu, Oct 18, 2018 at 7:27 AM Rainer Jung  wrote:

> I get test suite failures for t/ssl/ocsp.t when the server is build
> against OpenSSL 0.9.8zh. I can't judge on whether that is expected for
> OpenSSL 0.9.8.

A very good question, and I can't either. Can you confirm your openssl
command line tool has the `openssl ocsp` mini-responder by posting the
results of an `openssl ocsp -help` invocation?

It might be that we never handled ocsp here.

It might also be that your $openssl resolves to a system tool which is not
in sync with the openssl tested in httpd. You may want to override that
value.

And may be httpd never supported the ocsp directives with 0.9.8, so our
tests for the micro responder and the version of httpd alone are not
sufficient.

Re: Keeping backported CHANGES in trunk CHANGES?

On Thu, Oct 18, 2018 at 5:21 AM Rainer Jung  wrote:

> In trunk we do now have a 2.5 CHANGES file, ie. the file contains
> entries for 2.5.0-alpha and the entries above those under the 2.5.1
> heading.
>
> I think we should add entries under 2.5.1 even if things get likely
> backported and such items should no longer be removed when being
> backported. Before 2.5.0-alpha, that was just a candidate CHANGES file
> for the things not in 2.4.x, but now it should also document the changes
> after 2.5.0-alpha. Similar to how I think it was done while switching
> from 2.2 to 2.3/2.4.

That would make sense, note 2.5.0-alpha did not receive a successful
release vote, so we are still on the ground floor, so to speak.

We really need some logic to keep the two CHANGES files in-sync. The
end result, in 2.6.0, would be to have everything applicable listed as its
public/GA change from 2.4.x revision on, with everything strictly in the
alpha/beta 2.5.x change cycles listed under their respective update.

Re: [Discussion] Limit the scope of 2.4.x patches until 2.4.next is released?

On Thu, Oct 18, 2018 at 4:56 AM Rainer Jung  wrote:

> - The other one goes back to the other big refactoring which allowed to
> use SSLProxy* directives in  containers, first released in 2.4.32
> this year. It fixes a missing config merge (very small patch). This is
> not related to the OpenSSL 1.1.1 changes but is a small patch fixing a
> regression introduced a few months ago. I think it could go into 2.4.37,
> but would understand a more restricted approach.
>

I agree that reverting any now discovered/resolved regression falls under
the idea of a maintenance release scope, even one more narrowed to
"don't break the release 2x in a row" narrower scope.

Re: [NOTICE] Intent to T&R 2.4.37 - about 12:00 GMT tomorrow


Am 18.10.2018 um 14:23 schrieb Stefan Eissing:




Am 18.10.2018 um 14:12 schrieb Rainer Jung :

- t/modules/http2.t fails when the server is build using OpenSSL 0.9.8zh with the 
"Bad plan.  You planned 52 tests..." message indicating, that h2 using TLS does 
not work. It happens on all platforms, but not if the client also uses OpenSSL 0.9.8zh.

I don't know whether that is expected for old OpenSSL, so can not judge on 
criticality.


AFAICT, correct me if I am wrong, OpenSSL 0.9.8 does not support TLSv1.2 and is 
therefore unusable with h2. The test suite seems to be unprepared for this 
scenario. I will remove it after the next release. It is not worth fixing in 
its current form.


OK thanks, so we can ignore these failures.

Regards,

Rainer

OCSP in 2.4 with OpenSSL 0.9.8(zh)

I get test suite failures for t/ssl/ocsp.t when the server is build 
against OpenSSL 0.9.8zh. I can't judge on whether that is expected for 
OpenSSL 0.9.8.


Example error log:

...
18 14:15:11.833126 [ssl:debug] ssl_util_ocsp.c(406): Configuring Trusted 
OCSP certificates

...
18 14:15:12.238943 [ssl:info] AH01876: mod_ssl/2.4.36 compiled against 
Server: Apache/2.4.36, Library: OpenSSL/0.9.8zh

...
18 14:15:14.015398 [ssl:info] AH01964: Connection to child 0 established 
(server localhost:8535)
18 14:15:14.015949 [ssl:debug] ssl_engine_kernel.c(2328): AH02043: SSL 
virtual host for servername localhost found
18 14:15:14.143610 [ssl:info] AH02008: SSL library error 1 in handshake 
(server localhost:8535)
18 14:15:14.143662 [ssl:info] SSL Library Error: error:140890C7:SSL 
routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a certificate 
-- No CAs known to server for verification?
18 14:15:14.143670 [ssl:info] AH01998: Connection closed to child 0 with 
abortive shutdown (server localhost:8535)


18 14:15:14.166594 [ssl:info] AH01964: Connection to child 1 established 
(server localhost:8535)
18 14:15:14.166901 [ssl:debug] ssl_engine_kernel.c(2328): AH02043: SSL 
virtual host for servername localhost found
18 14:15:14.208760 [ssl:debug] ssl_engine_kernel.c(1749): AH02275: 
Certificate Verification, depth 1, CRL checking mode: none (0) [subject: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: B959B377BC9B01EE / notbefore: Oct 
18 01:35:05 2018 GMT / notafter: Oct 18 01:35:05 2019 GMT]
18 14:15:14.208953 [ssl:debug] ssl_engine_kernel.c(1749): AH02275: 
Certificate Verification, depth 0, CRL checking mode: none (0) [subject: 
emailAddress=test-...@httpd.apache.org,CN=client_ok,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 09 / notbefore: Oct 18 01:35:08 
2018 GMT / notafter: Oct 18 01:35:08 2019 GMT]
18 14:15:14.209355 [ssl:debug] ssl_util_ocsp.c(99): AH01973: connecting 
to OCSP responder 'localhost:8529'
18 14:15:14.209449 [ssl:debug] ssl_util_ocsp.c(125): AH01975: sending 
request to OCSP responder
18 14:15:14.270405 [ssl:debug] ssl_util_ocsp.c(235): AH01981: OCSP 
response header: Date: Thu, 18 Oct 2018 12:15:14 GMT
18 14:15:14.270423 [ssl:debug] ssl_util_ocsp.c(235): AH01981: OCSP 
response header: Server: Apache/2.4.36 (Unix) OpenSSL/0.9.8zh
18 14:15:14.270428 [ssl:debug] ssl_util_ocsp.c(235): AH01981: OCSP 
response header: Vary: In-If1
18 14:15:14.270432 [ssl:debug] ssl_util_ocsp.c(235): AH01981: OCSP 
response header: DMMATCH1: 1
18 14:15:14.270436 [ssl:debug] ssl_util_ocsp.c(235): AH01981: OCSP 
response header: Connection: close
18 14:15:14.270440 [ssl:debug] ssl_util_ocsp.c(235): AH01981: OCSP 
response header: Content-Type: application/ocsp-response

18 14:15:14.276787 [ssl:error] AH01988: failed to decode OCSP response data

18 14:15:14.276823 [ssl:error] SSL Library Error: error:0D06B08E:asn1 
encoding routines:ASN1_D2I_READ_BIO:not enough data


18 14:15:14.276950 [ssl:info] AH02276: Certificate Verification: Error 
(50): application verification failure [subject: 
emailAddress=test-...@httpd.apache.org,CN=client_ok,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 09 / notbefore: Oct 18 01:35:08 
2018 GMT / notafter: Oct 18 01:35:08 2019 GMT]
18 14:15:14.277136 [ssl:info] AH02008: SSL library error 1 in handshake 
(server localhost:8535)
18 14:15:14.277156 [ssl:info] SSL Library Error: error:140890B2:SSL 
routines:SSL3_GET_CLIENT_CERTIFICATE:no certificate returned
18 14:15:14.277162 [ssl:info] AH01998: Connection closed to child 1 with 
abortive shutdown (server localhost:8535)


18 14:15:14.284803 [ssl:info] AH01964: Connection to child 0 established 
(server localhost:8535)
18 14:15:14.285098 [ssl:debug] ssl_engine_kernel.c(2328): AH02043: SSL 
virtual host for servername localhost found
18 14:15:14.326054 [ssl:debug] ssl_engine_kernel.c(1749): AH02275: 
Certificate Verification, depth 1, CRL checking mode: none (0) [subject: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: B959B377BC9B01EE / notbefore: Oct 
18 01:35:05 2018 GMT / notafter: Oct 18 01:35:05 2019 GMT]
18 14:15:14.326248 [ssl:debug] ssl_engine_kernel.c(1749): AH02275: 
Certificate Verification, depth 0, CRL checking mode: none (0) [subject: 
emailAddress=test-...@httpd.apache.org,CN=client_revoked,OU=ht

Re: [NOTICE] Intent to T&R 2.4.37 - about 12:00 GMT tomorrow

2018-10-18 Thread Stefan Eissing




> Am 18.10.2018 um 14:12 schrieb Rainer Jung :
> 
> - t/modules/http2.t fails when the server is build using OpenSSL 0.9.8zh with 
> the "Bad plan.  You planned 52 tests..." message indicating, that h2 using 
> TLS does not work. It happens on all platforms, but not if the client also 
> uses OpenSSL 0.9.8zh.
> 
> I don't know whether that is expected for old OpenSSL, so can not judge on 
> criticality.

AFAICT, correct me if I am wrong, OpenSSL 0.9.8 does not support TLSv1.2 and is 
therefore unusable with h2. The test suite seems to be unprepared for this 
scenario. I will remove it after the next release. It is not worth fixing in 
its current form.

-Stefan

Re: [NOTICE] Intent to T&R 2.4.37 - about 12:00 GMT tomorrow

Am 17.10.2018 um 13:41 schrieb Daniel Ruggeri:

Hi, all;
With the fix for detected OpenSSL 1.1.1 issues now backported to 2.4.x,
I would like to tag the next version of our venerable server soon.

I have already successfully completed the test suite against my "latest
sources" docker environment and am watching for any smoke detected in
[1]. Feeling good about this one :-)

How about roughly 24 hours from now?

[1]
https://lists.apache.org/thread.html/48de97bd66ceabcf84a3719b36cd69274cb8c4b64d68c46696beb906@

In the meantime most of my tests finished. The two small mod_ssl patches
applied this morning were not part of the testing but seem simple enough
to understand and should pose no risk.

My testing showed:

I don't know whether that is expected for old OpenSSL, so can not judge
on criticality.

- t/modules/http2.t fails when the server is build using OpenSSL 0.9.8zh
with the "Bad plan. You planned 52 tests..." message indicating, that
h2 using TLS does not work. It happens on all platforms, but not if the
client also uses OpenSSL 0.9.8zh.

I don't know whether that is expected for old OpenSSL, so can not judge
on criticality.

- only once out of 68 runs on Solaris failure in t/modules/cgi.t test 54
in line 232. There log contents are checked and the file system is on
NFS. Might be, that this is a timing issue in the test. Not a
show-stopper for me.

- only once out of 68 runs on Solaris failure in t/ssl/proxy.t test 106
in line 131. /eat_post responds with a proxy error (502) instead of 200
with the posted content length as the response body. Need to investigate
but would also say not a show-stopper, because only on Solaris and only
once.

Regards,

Rainer

ap_request_core_filter issues

2018-10-18 Thread Joe Orton

On Thu, Oct 18, 2018 at 12:51:06PM +0200, Ruediger Pluem wrote:
> >>>
> >>> Curiously inefficient writev use when stracing the process, though, 
> >>> dunno what's going on there (trunk/prefork):
> >>>
> >>> writev(46, [{iov_base="\r\n", iov_len=2}], 1) = 2
> >>> writev(46, [{iov_base="1f84\r\n", iov_len=6}], 1) = 6
> >>> writev(46, [{iov_base="\n >>> iov_len=7820}], 1) = 7820
> >>> writev(46, [{iov_base="\n\n"..., 
> >>> iov_len=248}], 1) = 248
> >>
> >> The reason is ap_request_core_filter. It iterates over the brigade and 
> >> hands over each bucket alone to
> >> ap_core_output_filter. IMHO a bug.
> > 
> > How about the attached patch for fixing?
> 
> Scratch that. I guess it is much more complex.

Hmm, well it works for me and looks correct, it batches up those the 
writev(,,1) calls as expected, but it seems YMMV :)

I notice the function is doing unconditional blocking reads without 
flushing first - rule 10 of the golden rules of output filters! --

 status = apr_bucket_read(bucket, &data, &size, APR_BLOCK_READ);

Re: svn commit: r1841225 - /httpd/httpd/trunk/modules/dav/main/props.c

2018-10-18 Thread Joe Orton

On Thu, Oct 18, 2018 at 11:09:13AM +0200, Ruediger Pluem wrote:
> On 10/17/2018 07:47 PM, Joe Orton wrote:
> > On Wed, Oct 17, 2018 at 03:32:34PM +0100, Joe Orton wrote:
> >> I see constant memory use for a simple PROPFIND/depth:1 for the 
> >> attached, though I'm not sure this is sufficient to repro the problem 
> >> you saw before.
> 
> Thanks for having a look. My test case was opening a large directory (about 
> 50,000 files)
> with doplhin under RedHat 6. Memory usage remains constant after the patch. 
> So patch
> seems to make sense.

OK thanks.  I've tested this works to fix the memory consumption in that 
case (both as committed in trunk and with my change to it), when the 
PROPFIND is for *only* dead properties.

There is still more work to make memory use constant for liveprops.  
mod_dav_svn has done this already I assume, since the scratchpool was 
added to fix this issue with SVN IIRC.  So it should be possible with 
some effort for mod_dav_fs too, I will hopefully return to that 
sometime.

> Any idea if we could hit a similar issue with the two other remaining callers 
> of dav_open_propdb?
> 
> dav_gen_supported_live_props
> dav_method_proppatch

The specific issue here is in the "walker" function iterations - it was 
long known this can consume all your RAM for "Depth: infinity" walks, 
but I guess nobody ever thought about it too hard for Depth: 1.  LABEL, 
PROPFIND and LOCK/UNLOCK may all be affected I think.

Regards, Joe

Re: svn commit: r1841225 - /httpd/httpd/trunk/modules/dav/main/props.c

2018-10-18 Thread Yann Ylavic

On Thu, Oct 18, 2018 at 11:29 AM Ruediger Pluem  wrote:
> >
> > The reason is ap_request_core_filter. It iterates over the brigade and 
> > hands over each bucket alone to
> > ap_core_output_filter. IMHO a bug.
>
> How about the attached patch for fixing?

+1, nice way to address it!

Thanks,
Yann.

Re: Crash with SSL renegotiations in 2.4.x branch

2018-10-18 Thread Michael Kaufmann


Backported in 1844223, will be part of 2.4.37.

Thanks again!

Rainer


Great! Thanks a lot for proposing & backporting.

Regards,
Michael

Re: svn commit: r1841225 - /httpd/httpd/trunk/modules/dav/main/props.c

2018-10-18 Thread Ruediger Pluem




On 10/18/2018 11:29 AM, Ruediger Pluem wrote:
> 
> 
> On 10/18/2018 11:09 AM, Ruediger Pluem wrote:
>>
>>
>> On 10/17/2018 07:47 PM, Joe Orton wrote:
>>> On Wed, Oct 17, 2018 at 03:32:34PM +0100, Joe Orton wrote:
 I see constant memory use for a simple PROPFIND/depth:1 for the 
 attached, though I'm not sure this is sufficient to repro the problem 
 you saw before.
> 
>>>
>>> Curiously inefficient writev use when stracing the process, though, 
>>> dunno what's going on there (trunk/prefork):
>>>
>>> writev(46, [{iov_base="\r\n", iov_len=2}], 1) = 2
>>> writev(46, [{iov_base="1f84\r\n", iov_len=6}], 1) = 6
>>> writev(46, [{iov_base="\n>> iov_len=7820}], 1) = 7820
>>> writev(46, [{iov_base="\n\n"..., 
>>> iov_len=248}], 1) = 248
>>>
>>>
>>
>> The reason is ap_request_core_filter. It iterates over the brigade and hands 
>> over each bucket alone to
>> ap_core_output_filter. IMHO a bug.
> 
> How about the attached patch for fixing?

Scratch that. I guess it is much more complex.

Regards

Rüdiger

Keeping backported CHANGES in trunk CHANGES?

In trunk we do now have a 2.5 CHANGES file, ie. the file contains 
entries for 2.5.0-alpha and the entries above those under the 2.5.1 heading.


I think we should add entries under 2.5.1 even if things get likely 
backported and such items should no longer be removed when being 
backported. Before 2.5.0-alpha, that was just a candidate CHANGES file 
for the things not in 2.4.x, but now it should also document the changes 
after 2.5.0-alpha. Similar to how I think it was done while switching 
from 2.2 to 2.3/2.4.


Regards,

Rainer

Re: Crash with SSL renegotiations in 2.4.x branch


Am 18.10.2018 um 11:09 schrieb Rainer Jung:

Hi Michael,

Am 18.10.2018 um 09:39 schrieb Michael Kaufmann:

Hi,

there's a bug in the current 2.4.x branch of httpd which leads to 
crashes for SSL renegotiations.


The variable "ctx" is always NULL in ssl_engine_kernel.c, 
ssl_hook_Access_classic(), and it's used here:


if (!(cert_store ||
 (cert_store = SSL_CTX_get_cert_store(ctx
...

In trunk, this bug has been fixed in r1828793. Please backport this 
for 2.4.37.


Thanks for letting us know. Indeed the backport is missing.

Unfortunately the test suite seems to not cover the case that triggers 
the crash. It seems to be when OptRenegotiate is set and we do have 
client certs from the original handshake, but those certs were not 
verified and we want a reneg now.


I will propose for backport now.


Backported in 1844223, will be part of 2.4.37.

Thanks again!

Rainer

Re: svn commit: r1844002 - in /httpd/httpd/trunk: CHANGES modules/ssl/ssl_engine_config.c

2018-10-18 Thread Stefan Eissing

Ok, the vote storm (category 3) was released and my proposal is moot. ;-)

> Am 18.10.2018 um 11:26 schrieb Stefan Eissing :
> 
> Can we not just make a ssl-for-2.4.37 branch, merge the mod_ssl related 
> changes there and do one row of tests and vote on it? Maybe attach the branch 
> revision to the vote that was tested...
> 
> Seems to be able to save work, or?
> 
>> Am 18.10.2018 um 11:22 schrieb Yann Ylavic :
>> 
>> On Thu, Oct 18, 2018 at 11:18 AM Rainer Jung  wrote:
>>> 
>>> This fix at least formally applies to 2.4.x as well? Shouldn't it get
>>> backported?
>> 
>> +1
>> 
>> Regards,
>> Yann.
>

Re: [Discussion] Limit the scope of 2.4.x patches until 2.4.next is released?


Am 15.10.2018 um 16:10 schrieb William A Rowe Jr:
Like my beg for getting us to the 2.4.35 release tag, I'd like to 
propose we keep patches to branches/2.4.x/ generally within the scope of 
straightening out the remaining quirks related to the OpenSSL 1.1.1 API 
and library behavior changes (and similar corrections for any alternate 
library implementations such as LibreSSL or BoringSSL.)


This isn't a vote per se... just an ask whether we collectively want to 
defer all potentially disruptive changes for a release following 
2.4.next. We can certainly resume with that next release on an expedited 
basis, within a month or few (as opposed to waiting 6 months as has been 
typical.)


It appears that dropping in OpenSSL 1.1.1 into a previously working 
httpd built against 1.1.0 is not the "plug and play" replacement that 
the OpenSSL team originally envisioned, and deliberately building any 
previous release of httpd against 1.1.1 is similarly broken.


Thoughts? Other concerns?


I think everyone is aiming for a mostly riskfree 2.4.next. Apart from 
the already committed mod_ssl change to unbreak h2 with OpenSSL 1.1.1, 
two further mod_ssl backports popped up:


- One is r1828793, prevents crashes for certain configs and is directly 
related to the refactoring done for OpenSSL 1.1.1 support (but does 
happen for TLS < 1.3). I guess this clearly goes into the "straightening 
out the remaining quirks related to the OpenSSL 1.1.1 API" category.


- The other one goes back to the other big refactoring which allowed to 
use SSLProxy* directives in  containers, first released in 2.4.32 
this year. It fixes a missing config merge (very small patch). This is 
not related to the OpenSSL 1.1.1 changes but is a small patch fixing a 
regression introduced a few months ago. I think it could go into 2.4.37, 
but would understand a more restricted approach.


Regards,

Rainer

Re: [NOTICE] Intent to T&R 2.4.37 - about 12:00 GMT tomorrow

Am 17.10.2018 um 13:41 schrieb Daniel Ruggeri:

Hi, all;
With the fix for detected OpenSSL 1.1.1 issues now backported to 2.4.x,
I would like to tag the next version of our venerable server soon.

I have already successfully completed the test suite against my "latest
sources" docker environment and am watching for any smoke detected in
[1]. Feeling good about this one :-)

How about roughly 24 hours from now?

[1]
https://lists.apache.org/thread.html/48de97bd66ceabcf84a3719b36cd69274cb8c4b64d68c46696beb906@

There's a new crash report from Michael Kaufmann about one (small)
missing mod_ssl backport that leads to crashes under certain conditions
during renegotiation. That case seems to be not covered by the test
suite. I proposed the backport a few minutes ago. This one is probably a
show-stopper.

And there's another (small) backport candidate (r1844002) that I added
to STATUS now. No crash, but missing config merging leading to
unexpected behavior for some configurations. This one is not a
regression but was introduced this year in 2.4.32.

Finally I observed during my tests for the current 2.4 head some crashes
but only on Solaris and only for statically linked httpd when the event
MPM is used (MPM still dynamically loaded). The crashes always happen
during shutdown and it looks to me as all or most of them happen during
OpenSSL deinit. I vaguely remember there was some discussion about how
to sync OpenSSL unloading between the various consumers we have, eg.
apr-util and mod_ssl. So probably there's currently no easy solution for
this. I don't know whether this is new and it does not happen always.
Not a show-stopper for me, since the crashes only happen when a child
process ends. Still an annoyance, but IMHO not critical.

Regards,

Rainer

Re: svn commit: r1841225 - /httpd/httpd/trunk/modules/dav/main/props.c

2018-10-18 Thread Ruediger Pluem



On 10/18/2018 11:09 AM, Ruediger Pluem wrote:
> 
> 
> On 10/17/2018 07:47 PM, Joe Orton wrote:
>> On Wed, Oct 17, 2018 at 03:32:34PM +0100, Joe Orton wrote:
>>> I see constant memory use for a simple PROPFIND/depth:1 for the 
>>> attached, though I'm not sure this is sufficient to repro the problem 
>>> you saw before.

>>
>> Curiously inefficient writev use when stracing the process, though, 
>> dunno what's going on there (trunk/prefork):
>>
>> writev(46, [{iov_base="\r\n", iov_len=2}], 1) = 2
>> writev(46, [{iov_base="1f84\r\n", iov_len=6}], 1) = 6
>> writev(46, [{iov_base="\n> iov_len=7820}], 1) = 7820
>> writev(46, [{iov_base="\n\n"..., 
>> iov_len=248}], 1) = 248
>>
>>
> 
> The reason is ap_request_core_filter. It iterates over the brigade and hands 
> over each bucket alone to
> ap_core_output_filter. IMHO a bug.

How about the attached patch for fixing?

Regards

Rüdiger
Index: server/request.c
===
--- server/request.c	(revision 1843972)
+++ server/request.c	(working copy)
@@ -2102,6 +2102,8 @@
 seen_eor = 1;
 }
 else {
+int morphing_bucket = 0;
+
 /* if the core has set aside data, back off and try later */
 if (!flush_upto) {
 if (ap_filter_should_yield(f->next)) {
@@ -2124,13 +2126,25 @@
 if (status != APR_SUCCESS) {
 break;
 }
+morphing_bucket = 1;
 }
 
 /* pass each bucket down the chain */
 APR_BUCKET_REMOVE(bucket);
 APR_BRIGADE_INSERT_TAIL(tmp_bb, bucket);
+
+/*
+ * If we had a morphing bucket better pass things down the chain
+ * to avoid that we consume too much memory
+ */
+if (morphing_bucket) {
+status = ap_pass_brigade(f->next, tmp_bb);
+apr_brigade_cleanup(tmp_bb);
+}
 }
+}
 
+if ((status == APR_SUCCESS) && !APR_BRIGADE_EMPTY(tmp_bb)) {
 status = ap_pass_brigade(f->next, tmp_bb);
 apr_brigade_cleanup(tmp_bb);
 }

Re: svn commit: r1844002 - in /httpd/httpd/trunk: CHANGES modules/ssl/ssl_engine_config.c

2018-10-18 Thread Stefan Eissing

Can we not just make a ssl-for-2.4.37 branch, merge the mod_ssl related changes 
there and do one row of tests and vote on it? Maybe attach the branch revision 
to the vote that was tested...

Seems to be able to save work, or?

> Am 18.10.2018 um 11:22 schrieb Yann Ylavic :
> 
> On Thu, Oct 18, 2018 at 11:18 AM Rainer Jung  wrote:
>> 
>> This fix at least formally applies to 2.4.x as well? Shouldn't it get
>> backported?
> 
> +1
> 
> Regards,
> Yann.

Re: svn commit: r1844002 - in /httpd/httpd/trunk: CHANGES modules/ssl/ssl_engine_config.c

2018-10-18 Thread Yann Ylavic

On Thu, Oct 18, 2018 at 11:18 AM Rainer Jung  wrote:
>
> This fix at least formally applies to 2.4.x as well? Shouldn't it get
> backported?

+1

Regards,
Yann.

Re: svn commit: r1844002 - in /httpd/httpd/trunk: CHANGES modules/ssl/ssl_engine_config.c

This fix at least formally applies to 2.4.x as well? Shouldn't it get 
backported?


Due to the below svn log message the bug was introduced by the feature 
that SSLProxy* can be used in  sections. That feature got 
backported to 2.4.x, so probably this fix here should be backported as well.


Regards,

Rainer

Am 16.10.2018 um 14:55 schrieb rpl...@apache.org:

Author: rpluem
Date: Tue Oct 16 12:55:01 2018
New Revision: 1844002

URL: http://svn.apache.org/viewvc?rev=1844002&view=rev
Log:
* Correctly merge configurations that have client certificates set
   by SSLProxyMachineCertificate{File|Path}.
   The certificates and keys loaded during configuration time got lost during
   runtime if e.g. SSLProxyMachineCertificate{File|Path} was set on virtual host
   level and there was an SSL directive at directory level, e.g. SSLRequire.
   This fixes a regression likely introduced in r1740928.

Modified:
 httpd/httpd/trunk/CHANGES
 httpd/httpd/trunk/modules/ssl/ssl_engine_config.c

Modified: httpd/httpd/trunk/CHANGES
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/CHANGES?rev=1844002&r1=1844001&r2=1844002&view=diff
==
--- httpd/httpd/trunk/CHANGES [utf-8] (original)
+++ httpd/httpd/trunk/CHANGES [utf-8] Tue Oct 16 12:55:01 2018
@@ -1,6 +1,9 @@
   -*- coding: utf-8 -*-
  Changes with Apache 2.5.1
  
+  *) mod_ssl: Correctly merge configurations that have client certificates set

+ by SSLProxyMachineCertificate{File|Path}. [Ruediger Pluem]
+
*) core: Ensure that aborted connections are logged as such. PR 62823
   [Arnaud Grandville ]
  


Modified: httpd/httpd/trunk/modules/ssl/ssl_engine_config.c
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/ssl/ssl_engine_config.c?rev=1844002&r1=1844001&r2=1844002&view=diff
==
--- httpd/httpd/trunk/modules/ssl/ssl_engine_config.c (original)
+++ httpd/httpd/trunk/modules/ssl/ssl_engine_config.c Tue Oct 16 12:55:01 2018
@@ -433,6 +433,8 @@ static void modssl_ctx_cfg_merge_proxy(a
  cfgMergeString(pkp->cert_file);
  cfgMergeString(pkp->cert_path);
  cfgMergeString(pkp->ca_cert_file);
+cfgMergeString(pkp->certs);
+cfgMergeString(pkp->ca_certs);
  }
  
  void *ssl_config_perdir_merge(apr_pool_t *p, void *basev, void *addv)

Re: Crash with SSL renegotiations in 2.4.x branch