Re: child exit on self-proxy

[Stefan Eissing]

Coming back to this: I vote for applying this to trunk. On a fresh setup, I do 
not see and assert failures in my test.

- Stefan

> Am 08.07.2021 um 20:17 schrieb Yann Ylavic :
> 
> On Thu, Jul 8, 2021 at 4:21 PM Stefan Eissing
>  wrote:
>> 
>> I see many of those:
>> [Thu Jul 08 14:16:55.301670 2021] [mpm_event:error] [pid 81101:tid 
>> 123145411510272] (9)Bad file descriptor: AH00468: error closing socket 
>> -1/7ff9cf0086b0 from process_socket
>> 
>> which come from event.c#1263: rc = start_lingering_close_blocking(cs);
>> calling event.c#864: kill_connection_ex(cs, from);
>> and event.c#837: close_socket_nonblocking_ex(cs->pfd.desc.s, from);
> 
> OK, so I think this could be addressed by the attached patch.
> This is the same as v0 plus some changes in ap_start_lingering_close()
> (from server/connection.c) to not close the socket on error.
> ap_start_lingering_close() did not close the socket consistently, so
> the caller had to call apr_socket_close() anyway and sometimes fail..
> The close on failure is moved to ap_lingering_close() which is the one
> that should care.
> 
> Still the AP_DEBUG_ASSERT(0) triggering with this?
> 

Re: svn commit: r1891407 - /httpd/site/trunk/content/docs-project/contribute.mdtext

[Eric Covener]

Should we remove r/w from /httpd/site/trunk/ ?

On Tue, Jul 13, 2021 at 2:18 AM Ruediger Pluem  wrote:
>
>
>
> On 7/12/21 9:23 PM, Dave Fisher wrote:
> > I went ahead and made this change to GitHub.com/apache/httpd-site 
> > 
> >
> > I also made a change to the newest pelican build method in 
> > GitHub.com/apache/infrastructure-pelican
> > 
> >
> > I’ll be monitoring dev@httpd.a.o  for a few more days 
> > before I unsubscribe.
>
> Thank you for your support with this.
>
> Regards
>
> Rüdiger
>


-- 
Eric Covener
cove...@gmail.com

Re: svn commit: r1891407 - /httpd/site/trunk/content/docs-project/contribute.mdtext

[Rich Bowen]

On 7/9/21 4:10 PM, Ruediger Pluem wrote:

I guess with the site now moved over to Pelican / github this would need to 
fixed here:
https://github.com/apache/httpd-site/blob/main/content/docs-project/contribute.md

See also:https://infra.apache.org/asf-pelican.html



I see. I misunderstood. I thought that Github was just a mirror.

--
Rich Bowen - rbo...@rcbowen.com
@rbowen

Re: svn commit: r1891407 - /httpd/site/trunk/content/docs-project/contribute.mdtext

[Ruediger Pluem]

On 7/13/21 2:12 PM, Rich Bowen wrote:
> 
> 
> On 7/9/21 4:10 PM, Ruediger Pluem wrote:
>> I guess with the site now moved over to Pelican / github this would need to 
>> fixed here:
>> https://github.com/apache/httpd-site/blob/main/content/docs-project/contribute.md
>>
>> See also:https://infra.apache.org/asf-pelican.html
> 
> 
> I see. I misunderstood. I thought that Github was just a mirror.

For all stuff but for httpd-site it is a mirror and svn is canonical.
httpd-site is now natively on Github since the site migration.

Regards

Rüdiger

Re: migration of the HTTPD project website

[Ruediger Pluem]

On 7/5/21 9:55 AM, Ruediger Pluem wrote:
> 
> 
> On 6/30/21 11:40 PM, Greg Stein wrote:
>> On Wed, Jun 30, 2021 at 1:50 AM Ruediger Pluem > > wrote:
>>> ...
>>
>> I guess he talks about
>>
>> # Get the tooling
>> svn co https://svn.apache.org/repos/asf/httpd/site/trunk/tools tools
>>
>> I think we should find a new location for this as this is not really 
>> site related.
>> How about
>>
>> https://svn.apache.org/repos/asf/httpd/dev-tools/trunk
>>
>> (svn mv https://svn.apache.org/repos/asf/httpd/site/trunk/tools 
>> https://svn.apache.org/repos/asf/httpd/dev-tools/trunk)
>>
>>
>> If we don't intend to make *releases* of these tools, then I recommend 
>> dropping "trunk", and just call it .../httpd/dev-tools/ 
>>
>> IMO, trunk/tags/branches ("TTB") should only be used in relation to 
>> artifacts that will be packaged/released by the Foundation.
> 
> I don't think that we want to release them. I am fine with omitting trunk 
> from the path. Anyone objected to
> 
> svn mv https://svn.apache.org/repos/asf/httpd/site/trunk/tools 
> https://svn.apache.org/repos/asf/httpd/dev-tools/
> 
> ?

Done as r1891502.

Regards

Rüdiger

Re: child exit on self-proxy

[Stefan Eissing]

Found one condition where the assertion failure happens. The test case I have 
does this:

sock = socket.create_connection((host, int(env.https_port)))
try:
sock.settimeout(0.5)
sock.recv(1024)
assert False
except Exception as ex:
print(f"as expected: {ex}")
sock.close()

Which is all that's needed to trigger the assertion. The server is not stopped.

The log for this thread only gives:
[Tue Jul 13 13:51:29.476747 2021] [mpm_event:trace4] [pid 31582:tid 
123145480441856] event.c(858): [client 127.0.0.1:51768] lingering close in 
state 5
[Tue Jul 13 13:51:29.476769 2021] [mpm_event:trace4] [pid 31582:tid 
123145480441856] event.c(833): [remote 127.0.0.1:51768] kill connection in 
state 5
[Tue Jul 13 13:51:29.476775 2021] [mpm_event:trace4] [pid 31582:tid 
123145480441856] event.c(563): closing socket -1/7fa24681d0b0 from 
process_socket
[Tue Jul 13 13:51:29.476779 2021] [mpm_event:error] [pid 31582:tid 
123145480441856] (9)Bad file descriptor: AH00468: error closing socket 
-1/7fa24681d0b0 from process_socket
[Tue Jul 13 13:51:29.476799 2021] [core:crit] [pid 31582:tid 123145480441856] 
AH00102: [Tue Jul 13 13:51:29 2021] file event.c, line 569, assertion "0" failed

I added a trace4 "closing socket" before the apr_socket_close(), grepping for 
that in the log shows:
[Tue Jul 13 13:51:28.467677 2021] [mpm_event:trace4] [pid 31582:tid 
123145479905280] event.c(563): closing socket 15/7fa24681d0a0 from 
process_lingering_close
[Tue Jul 13 13:51:29.476775 2021] [mpm_event:trace4] [pid 31582:tid 
123145480441856] event.c(563): closing socket -1/7fa24681d0b0 from 
process_socket
[Tue Jul 13 13:51:29.476779 2021] [mpm_event:error] [pid 31582:tid 
123145480441856] (9)Bad file descriptor: AH00468: error closing socket 
-1/7fa24681d0b0 from process_socket
[Tue Jul 13 13:51:29.984132 2021] [mpm_event:trace4] [pid 31584:tid 
123145470783488] event.c(563): closing socket -1/7fa24681f0b0 from 
process_socket
[Tue Jul 13 13:51:29.984138 2021] [mpm_event:error] [pid 31584:tid 
123145470783488] (9)Bad file descriptor: AH00468: error closing socket 
-1/7fa24681f0b0 from process_socket



Thread 21 Crashed:
0   libsystem_kernel.dylib  0x7fff202e292e __pthread_kill + 10
1   libsystem_pthread.dylib 0x7fff203115bd pthread_kill + 263
2   libsystem_c.dylib   0x7fff202664ab __abort + 139
3   libsystem_c.dylib   0x7fff20266420 abort + 135
4   httpd   0x00010dbfdb72 ap_log_assert + 130 
(log.c:1649)
5   mod_mpm_event.so0x00010e5154bf 
close_socket_nonblocking_ex + 223 (event.c:569)
6   mod_mpm_event.so0x00010e515158 kill_connection_ex + 
67 (event.c:835) [inlined]
7   mod_mpm_event.so0x00010e515158 
start_lingering_close_blocking_ex + 184 (event.c:862)
8   mod_mpm_event.so0x00010e514da9 process_socket + 
1289 (event.c:1261)
9   mod_mpm_event.so0x00010e514769 worker_thread + 809 
(event.c:2394)
10  libapr-1.0.dylib0x00010ddb9341 dummy_worker + 17 
(thread.c:147)
11  libsystem_pthread.dylib 0x7fff203118fc _pthread_start + 224
12  libsystem_pthread.dylib 0x7fff2030d443 thread_start + 15



- Stefan

> Am 13.07.2021 um 12:39 schrieb Stefan Eissing :
> 
> Coming back to this: I vote for applying this to trunk. On a fresh setup, I 
> do not see and assert failures in my test.
> 
> - Stefan
> 
>> Am 08.07.2021 um 20:17 schrieb Yann Ylavic :
>> 
>> On Thu, Jul 8, 2021 at 4:21 PM Stefan Eissing
>>  wrote:
>>> 
>>> I see many of those:
>>> [Thu Jul 08 14:16:55.301670 2021] [mpm_event:error] [pid 81101:tid 
>>> 123145411510272] (9)Bad file descriptor: AH00468: error closing socket 
>>> -1/7ff9cf0086b0 from process_socket
>>> 
>>> which come from event.c#1263: rc = start_lingering_close_blocking(cs);
>>> calling event.c#864: kill_connection_ex(cs, from);
>>> and event.c#837: close_socket_nonblocking_ex(cs->pfd.desc.s, from);
>> 
>> OK, so I think this could be addressed by the attached patch.
>> This is the same as v0 plus some changes in ap_start_lingering_close()
>> (from server/connection.c) to not close the socket on error.
>> ap_start_lingering_close() did not close the socket consistently, so
>> the caller had to call apr_socket_close() anyway and sometimes fail..
>> The close on failure is moved to ap_lingering_close() which is the one
>> that should care.
>> 
>> Still the AP_DEBUG_ASSERT(0) triggering with this?
>> 
> 

Re: child exit on self-proxy

[Stefan Eissing]

Wait a minute...need to confirm that I am not lying and used an older patch 
version...

> Am 13.07.2021 um 16:01 schrieb Stefan Eissing :
> 
> Found one condition where the assertion failure happens. The test case I have 
> does this:
> 
> sock = socket.create_connection((host, int(env.https_port)))
> try:
>sock.settimeout(0.5)
>sock.recv(1024)
>assert False
> except Exception as ex:
>print(f"as expected: {ex}")
> sock.close()
> 
> Which is all that's needed to trigger the assertion. The server is not 
> stopped.
> 
> The log for this thread only gives:
> [Tue Jul 13 13:51:29.476747 2021] [mpm_event:trace4] [pid 31582:tid 
> 123145480441856] event.c(858): [client 127.0.0.1:51768] lingering close in 
> state 5
> [Tue Jul 13 13:51:29.476769 2021] [mpm_event:trace4] [pid 31582:tid 
> 123145480441856] event.c(833): [remote 127.0.0.1:51768] kill connection in 
> state 5
> [Tue Jul 13 13:51:29.476775 2021] [mpm_event:trace4] [pid 31582:tid 
> 123145480441856] event.c(563): closing socket -1/7fa24681d0b0 from 
> process_socket
> [Tue Jul 13 13:51:29.476779 2021] [mpm_event:error] [pid 31582:tid 
> 123145480441856] (9)Bad file descriptor: AH00468: error closing socket 
> -1/7fa24681d0b0 from process_socket
> [Tue Jul 13 13:51:29.476799 2021] [core:crit] [pid 31582:tid 123145480441856] 
> AH00102: [Tue Jul 13 13:51:29 2021] file event.c, line 569, assertion "0" 
> failed
> 
> I added a trace4 "closing socket" before the apr_socket_close(), grepping for 
> that in the log shows:
> [Tue Jul 13 13:51:28.467677 2021] [mpm_event:trace4] [pid 31582:tid 
> 123145479905280] event.c(563): closing socket 15/7fa24681d0a0 from 
> process_lingering_close
> [Tue Jul 13 13:51:29.476775 2021] [mpm_event:trace4] [pid 31582:tid 
> 123145480441856] event.c(563): closing socket -1/7fa24681d0b0 from 
> process_socket
> [Tue Jul 13 13:51:29.476779 2021] [mpm_event:error] [pid 31582:tid 
> 123145480441856] (9)Bad file descriptor: AH00468: error closing socket 
> -1/7fa24681d0b0 from process_socket
> [Tue Jul 13 13:51:29.984132 2021] [mpm_event:trace4] [pid 31584:tid 
> 123145470783488] event.c(563): closing socket -1/7fa24681f0b0 from 
> process_socket
> [Tue Jul 13 13:51:29.984138 2021] [mpm_event:error] [pid 31584:tid 
> 123145470783488] (9)Bad file descriptor: AH00468: error closing socket 
> -1/7fa24681f0b0 from process_socket
> 
> 
> 
> Thread 21 Crashed:
> 0   libsystem_kernel.dylib0x7fff202e292e __pthread_kill + 10
> 1   libsystem_pthread.dylib   0x7fff203115bd pthread_kill + 263
> 2   libsystem_c.dylib 0x7fff202664ab __abort + 139
> 3   libsystem_c.dylib 0x7fff20266420 abort + 135
> 4   httpd 0x00010dbfdb72 ap_log_assert + 130 
> (log.c:1649)
> 5   mod_mpm_event.so  0x00010e5154bf 
> close_socket_nonblocking_ex + 223 (event.c:569)
> 6   mod_mpm_event.so  0x00010e515158 kill_connection_ex + 
> 67 (event.c:835) [inlined]
> 7   mod_mpm_event.so  0x00010e515158 
> start_lingering_close_blocking_ex + 184 (event.c:862)
> 8   mod_mpm_event.so  0x00010e514da9 process_socket + 
> 1289 (event.c:1261)
> 9   mod_mpm_event.so  0x00010e514769 worker_thread + 809 
> (event.c:2394)
> 10  libapr-1.0.dylib  0x00010ddb9341 dummy_worker + 17 
> (thread.c:147)
> 11  libsystem_pthread.dylib   0x7fff203118fc _pthread_start + 224
> 12  libsystem_pthread.dylib   0x7fff2030d443 thread_start + 15
> 
> 
> 
> - Stefan
> 
>> Am 13.07.2021 um 12:39 schrieb Stefan Eissing :
>> 
>> Coming back to this: I vote for applying this to trunk. On a fresh setup, I 
>> do not see and assert failures in my test.
>> 
>> - Stefan
>> 
>>> Am 08.07.2021 um 20:17 schrieb Yann Ylavic :
>>> 
>>> On Thu, Jul 8, 2021 at 4:21 PM Stefan Eissing
>>>  wrote:
 
 I see many of those:
 [Thu Jul 08 14:16:55.301670 2021] [mpm_event:error] [pid 81101:tid 
 123145411510272] (9)Bad file descriptor: AH00468: error closing socket 
 -1/7ff9cf0086b0 from process_socket
 
 which come from event.c#1263: rc = start_lingering_close_blocking(cs);
 calling event.c#864: kill_connection_ex(cs, from);
 and event.c#837: close_socket_nonblocking_ex(cs->pfd.desc.s, from);
>>> 
>>> OK, so I think this could be addressed by the attached patch.
>>> This is the same as v0 plus some changes in ap_start_lingering_close()
>>> (from server/connection.c) to not close the socket on error.
>>> ap_start_lingering_close() did not close the socket consistently, so
>>> the caller had to call apr_socket_close() anyway and sometimes fail..
>>> The close on failure is moved to ap_lingering_close() which is the one
>>> that should care.
>>> 
>>> Still the AP_DEBUG_ASSERT(0) triggering with this?
>>> 
>> 
> 

Re: child exit on self-proxy

[Stefan Eissing]

In Yann's v3 of the patch, it triggers only when I stop the server while the 
test case is ongoing,

Thread 2 Crashed:
0   libsystem_kernel.dylib  0x7fff202e292e __pthread_kill + 10
1   libsystem_pthread.dylib 0x7fff203115bd pthread_kill + 263
2   libsystem_c.dylib   0x7fff202664ab __abort + 139
3   libsystem_c.dylib   0x7fff20266420 abort + 135
4   httpd   0x00010625bb72 ap_log_assert + 130 
(log.c:1649)
5   mod_mpm_event.so0x000106b73549 
close_socket_nonblocking + 217 (event.c:569)
6   mod_mpm_event.so0x000106b731f2 kill_connection + 64 
(event.c:834) [inlined]
7   mod_mpm_event.so0x000106b731f2 
start_lingering_close_blocking + 178 (event.c:860)
8   mod_mpm_event.so0x000106b72e52 process_socket + 
1282 (event.c:1258)
9   mod_mpm_event.so0x000106b72819 worker_thread + 809 
(event.c:2390)
10  libapr-1.0.dylib0x000106412341 dummy_worker + 17 
(thread.c:147)
11  libsystem_pthread.dylib 0x7fff203118fc _pthread_start + 224
12  libsystem_pthread.dylib 0x7fff2030d443 thread_start + 15


[Tue Jul 13 14:12:18.800125 2021] [mpm_event:trace4] [pid 47786:tid 
123145484374016] event.c(563): closing socket 15/7f95ba07b4a0
[Tue Jul 13 14:12:19.551039 2021] [mpm_event:trace4] [pid 47787:tid 
12314548824] event.c(1322): closing listeners (connection_count=0)
[Tue Jul 13 14:12:19.551146 2021] [mpm_event:trace4] [pid 47786:tid 
123145497251840] event.c(1322): closing listeners (connection_count=1)
[Tue Jul 13 14:12:19.551176 2021] [mpm_event:trace4] [pid 47788:tid 
12314548824] event.c(1322): closing listeners (connection_count=0)
[Tue Jul 13 14:12:19.551209 2021] [mpm_event:trace4] [pid 47786:tid 4476272128] 
event.c(563): closing socket 15/7f95ba07b4b0
[Tue Jul 13 14:12:19.552305 2021] [mpm_event:trace4] [pid 47792:tid 
12314548824] event.c(1322): closing listeners (connection_count=0)
[Tue Jul 13 14:12:19.805283 2021] [mpm_event:trace4] [pid 47786:tid 
123145484910592] event.c(563): closing socket -1/7f95ba07b4b0
[Tue Jul 13 14:12:19.805289 2021] [mpm_event:error] [pid 47786:tid 
123145484910592] (9)Bad file descriptor: AH00468: error closing socket 
-1/7f95ba07b4b0

You can see that socket 7f95ba07b4b0 is closed twice. I attach my log and v5 
patch (which is Yann's plus some extra log around closing).


error_log
Description: Binary data


event_ka_no_lingerv5.diff
Description: Binary data

> Am 13.07.2021 um 16:03 schrieb Stefan Eissing :
> 
> Wait a minute...need to confirm that I am not lying and used an older patch 
> version...
> 
>> Am 13.07.2021 um 16:01 schrieb Stefan Eissing :
>> 
>> Found one condition where the assertion failure happens. The test case I 
>> have does this:
>> 
>> sock = socket.create_connection((host, int(env.https_port)))
>> try:
>>   sock.settimeout(0.5)
>>   sock.recv(1024)
>>   assert False
>> except Exception as ex:
>>   print(f"as expected: {ex}")
>> sock.close()
>> 
>> Which is all that's needed to trigger the assertion. The server is not 
>> stopped.
>> 
>> The log for this thread only gives:
>> [Tue Jul 13 13:51:29.476747 2021] [mpm_event:trace4] [pid 31582:tid 
>> 123145480441856] event.c(858): [client 127.0.0.1:51768] lingering close in 
>> state 5
>> [Tue Jul 13 13:51:29.476769 2021] [mpm_event:trace4] [pid 31582:tid 
>> 123145480441856] event.c(833): [remote 127.0.0.1:51768] kill connection in 
>> state 5
>> [Tue Jul 13 13:51:29.476775 2021] [mpm_event:trace4] [pid 31582:tid 
>> 123145480441856] event.c(563): closing socket -1/7fa24681d0b0 from 
>> process_socket
>> [Tue Jul 13 13:51:29.476779 2021] [mpm_event:error] [pid 31582:tid 
>> 123145480441856] (9)Bad file descriptor: AH00468: error closing socket 
>> -1/7fa24681d0b0 from process_socket
>> [Tue Jul 13 13:51:29.476799 2021] [core:crit] [pid 31582:tid 
>> 123145480441856] AH00102: [Tue Jul 13 13:51:29 2021] file event.c, line 569, 
>> assertion "0" failed
>> 
>> I added a trace4 "closing socket" before the apr_socket_close(), grepping 
>> for that in the log shows:
>> [Tue Jul 13 13:51:28.467677 2021] [mpm_event:trace4] [pid 31582:tid 
>> 123145479905280] event.c(563): closing socket 15/7fa24681d0a0 from 
>> process_lingering_close
>> [Tue Jul 13 13:51:29.476775 2021] [mpm_event:trace4] [pid 31582:tid 
>> 123145480441856] event.c(563): closing socket -1/7fa24681d0b0 from 
>> process_socket
>> [Tue Jul 13 13:51:29.476779 2021] [mpm_event:error] [pid 31582:tid 
>> 123145480441856] (9)Bad file descriptor: AH00468: error closing socket 
>> -1/7fa24681d0b0 from process_socket
>> [Tue Jul 13 13:51:29.984132 2021] [mpm_event:trace4] [pid 31584:tid 
>> 123145470783488] event.c(563): closing socket -1/7fa24681f0b0 from 
>> process_socket
>> [Tue Jul 13 13:51:29.984138 2021] [mpm_event:error] [pid 31584:tid 
>> 123145470783488] (9)Bad file descr

Re: svn commit: r1891407 - /httpd/site/trunk/content/docs-project/contribute.mdtext

[Greg Stein]

On Tue, Jul 13, 2021 at 7:22 AM Ruediger Pluem  wrote:

> On 7/13/21 2:12 PM, Rich Bowen wrote:
> > On 7/9/21 4:10 PM, Ruediger Pluem wrote:
>
>...

> >> See also:https://infra.apache.org/asf-pelican.html
> >
> > I see. I misunderstood. I thought that Github was just a mirror.
>
> For all stuff but for httpd-site it is a mirror and svn is canonical.
> httpd-site is now natively on Github since the site migration.
>

The primary benefit here is that you can visit github.com and use the
"pencil" in the upper right when viewing a .md file, to edit that specific
page. The preview displays it in "github flavored markdown", and the
Pelican process uses the exact same C library to render final html pages.
(the CSS will be different, of course) ... this provides a pretty simple
process for editing/previewing the httpd website.

Cheers,
-g

ps. note that we also synchronize the git repo back/forth to
gitbox.apache.org and allow people to commit there, should they disagree
with establishing a GitHub account.