In Yann's v3 of the patch, it triggers only when I stop the server while the test case is ongoing,
Thread 2 Crashed: 0 libsystem_kernel.dylib 0x00007fff202e292e __pthread_kill + 10 1 libsystem_pthread.dylib 0x00007fff203115bd pthread_kill + 263 2 libsystem_c.dylib 0x00007fff202664ab __abort + 139 3 libsystem_c.dylib 0x00007fff20266420 abort + 135 4 httpd 0x000000010625bb72 ap_log_assert + 130 (log.c:1649) 5 mod_mpm_event.so 0x0000000106b73549 close_socket_nonblocking + 217 (event.c:569) 6 mod_mpm_event.so 0x0000000106b731f2 kill_connection + 64 (event.c:834) [inlined] 7 mod_mpm_event.so 0x0000000106b731f2 start_lingering_close_blocking + 178 (event.c:860) 8 mod_mpm_event.so 0x0000000106b72e52 process_socket + 1282 (event.c:1258) 9 mod_mpm_event.so 0x0000000106b72819 worker_thread + 809 (event.c:2390) 10 libapr-1.0.dylib 0x0000000106412341 dummy_worker + 17 (thread.c:147) 11 libsystem_pthread.dylib 0x00007fff203118fc _pthread_start + 224 12 libsystem_pthread.dylib 0x00007fff2030d443 thread_start + 15 [Tue Jul 13 14:12:18.800125 2021] [mpm_event:trace4] [pid 47786:tid 123145484374016] event.c(563): closing socket 15/7f95ba07b4a0 [Tue Jul 13 14:12:19.551039 2021] [mpm_event:trace4] [pid 47787:tid 123145488666624] event.c(1322): closing listeners (connection_count=0) [Tue Jul 13 14:12:19.551146 2021] [mpm_event:trace4] [pid 47786:tid 123145497251840] event.c(1322): closing listeners (connection_count=1) [Tue Jul 13 14:12:19.551176 2021] [mpm_event:trace4] [pid 47788:tid 123145488666624] event.c(1322): closing listeners (connection_count=0) [Tue Jul 13 14:12:19.551209 2021] [mpm_event:trace4] [pid 47786:tid 4476272128] event.c(563): closing socket 15/7f95ba07b4b0 [Tue Jul 13 14:12:19.552305 2021] [mpm_event:trace4] [pid 47792:tid 123145488666624] event.c(1322): closing listeners (connection_count=0) [Tue Jul 13 14:12:19.805283 2021] [mpm_event:trace4] [pid 47786:tid 123145484910592] event.c(563): closing socket -1/7f95ba07b4b0 [Tue Jul 13 14:12:19.805289 2021] [mpm_event:error] [pid 47786:tid 123145484910592] (9)Bad file descriptor: AH00468: error closing socket -1/7f95ba07b4b0 You can see that socket 7f95ba07b4b0 is closed twice. I attach my log and v5 patch (which is Yann's plus some extra log around closing).
error_log
Description: Binary data
event_ka_no_lingerv5.diff
Description: Binary data
> Am 13.07.2021 um 16:03 schrieb Stefan Eissing <stefan.eiss...@greenbytes.de>:
>
> Wait a minute...need to confirm that I am not lying and used an older patch
> version...
>
>> Am 13.07.2021 um 16:01 schrieb Stefan Eissing <stefan.eiss...@greenbytes.de>:
>>
>> Found one condition where the assertion failure happens. The test case I
>> have does this:
>>
>> sock = socket.create_connection((host, int(env.https_port)))
>> try:
>> sock.settimeout(0.5)
>> sock.recv(1024)
>> assert False
>> except Exception as ex:
>> print(f"as expected: {ex}")
>> sock.close()
>>
>> Which is all that's needed to trigger the assertion. The server is not
>> stopped.
>>
>> The log for this thread only gives:
>> [Tue Jul 13 13:51:29.476747 2021] [mpm_event:trace4] [pid 31582:tid
>> 123145480441856] event.c(858): [client 127.0.0.1:51768] lingering close in
>> state 5
>> [Tue Jul 13 13:51:29.476769 2021] [mpm_event:trace4] [pid 31582:tid
>> 123145480441856] event.c(833): [remote 127.0.0.1:51768] kill connection in
>> state 5
>> [Tue Jul 13 13:51:29.476775 2021] [mpm_event:trace4] [pid 31582:tid
>> 123145480441856] event.c(563): closing socket -1/7fa24681d0b0 from
>> process_socket
>> [Tue Jul 13 13:51:29.476779 2021] [mpm_event:error] [pid 31582:tid
>> 123145480441856] (9)Bad file descriptor: AH00468: error closing socket
>> -1/7fa24681d0b0 from process_socket
>> [Tue Jul 13 13:51:29.476799 2021] [core:crit] [pid 31582:tid
>> 123145480441856] AH00102: [Tue Jul 13 13:51:29 2021] file event.c, line 569,
>> assertion "0" failed
>>
>> I added a trace4 "closing socket" before the apr_socket_close(), grepping
>> for that in the log shows:
>> [Tue Jul 13 13:51:28.467677 2021] [mpm_event:trace4] [pid 31582:tid
>> 123145479905280] event.c(563): closing socket 15/7fa24681d0a0 from
>> process_lingering_close
>> [Tue Jul 13 13:51:29.476775 2021] [mpm_event:trace4] [pid 31582:tid
>> 123145480441856] event.c(563): closing socket -1/7fa24681d0b0 from
>> process_socket
>> [Tue Jul 13 13:51:29.476779 2021] [mpm_event:error] [pid 31582:tid
>> 123145480441856] (9)Bad file descriptor: AH00468: error closing socket
>> -1/7fa24681d0b0 from process_socket
>> [Tue Jul 13 13:51:29.984132 2021] [mpm_event:trace4] [pid 31584:tid
>> 123145470783488] event.c(563): closing socket -1/7fa24681f0b0 from
>> process_socket
>> [Tue Jul 13 13:51:29.984138 2021] [mpm_event:error] [pid 31584:tid
>> 123145470783488] (9)Bad file descriptor: AH00468: error closing socket
>> -1/7fa24681f0b0 from process_socket
>>
>>
>>
>> Thread 21 Crashed:
>> 0 libsystem_kernel.dylib 0x00007fff202e292e __pthread_kill + 10
>> 1 libsystem_pthread.dylib 0x00007fff203115bd pthread_kill + 263
>> 2 libsystem_c.dylib 0x00007fff202664ab __abort + 139
>> 3 libsystem_c.dylib 0x00007fff20266420 abort + 135
>> 4 httpd 0x000000010dbfdb72 ap_log_assert + 130
>> (log.c:1649)
>> 5 mod_mpm_event.so 0x000000010e5154bf
>> close_socket_nonblocking_ex + 223 (event.c:569)
>> 6 mod_mpm_event.so 0x000000010e515158 kill_connection_ex +
>> 67 (event.c:835) [inlined]
>> 7 mod_mpm_event.so 0x000000010e515158
>> start_lingering_close_blocking_ex + 184 (event.c:862)
>> 8 mod_mpm_event.so 0x000000010e514da9 process_socket +
>> 1289 (event.c:1261)
>> 9 mod_mpm_event.so 0x000000010e514769 worker_thread + 809
>> (event.c:2394)
>> 10 libapr-1.0.dylib 0x000000010ddb9341 dummy_worker + 17
>> (thread.c:147)
>> 11 libsystem_pthread.dylib 0x00007fff203118fc _pthread_start + 224
>> 12 libsystem_pthread.dylib 0x00007fff2030d443 thread_start + 15
>>
>>
>>
>> - Stefan
>>
>>> Am 13.07.2021 um 12:39 schrieb Stefan Eissing
>>> <stefan.eiss...@greenbytes.de>:
>>>
>>> Coming back to this: I vote for applying this to trunk. On a fresh setup, I
>>> do not see and assert failures in my test.
>>>
>>> - Stefan
>>>
>>>> Am 08.07.2021 um 20:17 schrieb Yann Ylavic <ylavic....@gmail.com>:
>>>>
>>>> On Thu, Jul 8, 2021 at 4:21 PM Stefan Eissing
>>>> <stefan.eiss...@greenbytes.de> wrote:
>>>>>
>>>>> I see many of those:
>>>>> [Thu Jul 08 14:16:55.301670 2021] [mpm_event:error] [pid 81101:tid
>>>>> 123145411510272] (9)Bad file descriptor: AH00468: error closing socket
>>>>> -1/7ff9cf0086b0 from process_socket
>>>>>
>>>>> which come from event.c#1263: rc = start_lingering_close_blocking(cs);
>>>>> calling event.c#864: kill_connection_ex(cs, from);
>>>>> and event.c#837: close_socket_nonblocking_ex(cs->pfd.desc.s, from);
>>>>
>>>> OK, so I think this could be addressed by the attached patch.
>>>> This is the same as v0 plus some changes in ap_start_lingering_close()
>>>> (from server/connection.c) to not close the socket on error.
>>>> ap_start_lingering_close() did not close the socket consistently, so
>>>> the caller had to call apr_socket_close() anyway and sometimes fail..
>>>> The close on failure is moved to ap_lingering_close() which is the one
>>>> that should care.
>>>>
>>>> Still the AP_DEBUG_ASSERT(0) triggering with this?
>>>> <event_ka_no_lingerv3.diff>
>>>
>>
>
