[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

I'd be interested in hearing Ronald's opinion. (Added to nosy.)

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

But here, it is write() that returns EPROTOTYPE. See msg394798.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[STINNER Victor]

STINNER Victor  added the comment:

http://erickt.github.io/blog/2014/11/19/adventures-in-debugging-a-potential-osx-kernel-bug/
 says:

"If we trigger a send while the kernel is in the middle of tearing down the 
socket, it returns EPROTOTYPE." (instead of ECONNRESET)

Maybe send() could raise a ConnectionResetError exception on EPROTOTYPE?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

I haven't observed this issue on the CI since GH-26893 through GH-26896 was 
merged. Marking this as resolved. If anyone disagrees, feel free to reopen :)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 71ba16b21cb35923098026117b5e6d823c5f5707 by Miss Islington (bot) 
in branch '3.8':
bpo-44229: Ignore spurious EPROTOTYPE on macOS in test_ssl (GH-26893) (GH-26895)
https://github.com/python/cpython/commit/71ba16b21cb35923098026117b5e6d823c5f5707


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

FYI, here's a CI failure against 3.9:
https://github.com/python/cpython/runs/2894813367?check_suite_focus=true

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Marking bpo-44237 as a duplicate of this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Marking bpo-43855 as a duplicate of this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[miss-islington]

miss-islington  added the comment:


New changeset b3fac2926b23b4f1342099e591aa3fed7f16876d by Miss Islington (bot) 
in branch '3.10':
bpo-44229: Ignore spurious EPROTOTYPE on macOS in test_ssl (GH-26893)
https://github.com/python/cpython/commit/b3fac2926b23b4f1342099e591aa3fed7f16876d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[miss-islington]

miss-islington  added the comment:


New changeset 0796e21fea31fe7b697d84c8d03186817792c458 by Miss Islington (bot) 
in branch '3.9':
bpo-44229: Ignore spurious EPROTOTYPE on macOS in test_ssl (GH-26893)
https://github.com/python/cpython/commit/0796e21fea31fe7b697d84c8d03186817792c458


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[miss-islington]

Change by miss-islington :


--
pull_requests: +25472
pull_request: https://github.com/python/cpython/pull/26896

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[miss-islington]

Change by miss-islington :


--
pull_requests: +25471
pull_request: https://github.com/python/cpython/pull/26895

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset b5a52eef67997246b4235b5407e52a01e822ce56 by Erlend Egeberg 
Aasland in branch 'main':
bpo-44229: Ignore spurious EPROTOTYPE on macOS in test_ssl (GH-26893)
https://github.com/python/cpython/commit/b5a52eef67997246b4235b5407e52a01e822ce56


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +25470
pull_request: https://github.com/python/cpython/pull/26894

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
keywords: +patch
pull_requests: +25469
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26893

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

I can reproduce these issues easily on 3.7, 3.8, and 3.9:

$ python3.7 -m test test_ssl -u all -F -m test_get_server_certificate -v
[...]
test_get_server_certificate (test.test_ssl.SimpleBackgroundTests) ...  server:  
new connection from ('127.0.0.1', 57294)

 server:  bad connection attempt from ('127.0.0.1', 57294):
Traceback (most recent call last):
   File "/Users/erlendaasland/install/lib/python3.7/test/test_ssl.py", line 
2313, in wrap_conn
self.sock, server_side=True)
   File "/Users/erlendaasland/install/lib/python3.7/ssl.py", line 423, in 
wrap_socket
session=session
   File "/Users/erlendaasland/install/lib/python3.7/ssl.py", line 870, in 
_create
self.do_handshake()
   File "/Users/erlendaasland/install/lib/python3.7/ssl.py", line 1139, in 
do_handshake
self._sslobj.do_handshake()
 OSError: [Errno 41] Protocol wrong type for socket
ERROR

==
ERROR: test_get_server_certificate (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File "/Users/erlendaasland/install/lib/python3.7/test/test_ssl.py", line 
2108, in test_get_server_certificate
_test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA)
  File "/Users/erlendaasland/install/lib/python3.7/test/test_ssl.py", line 
2273, in _test_get_server_certificate
pem = ssl.get_server_certificate((host, port), ca_certs=cert)
  File "/Users/erlendaasland/install/lib/python3.7/ssl.py", line 1313, in 
get_server_certificate
with  create_connection(addr) as sock:
  File "/Users/erlendaasland/install/lib/python3.7/socket.py", line 728, in 
create_connection
raise err
  File "/Users/erlendaasland/install/lib/python3.7/socket.py", line 716, in 
create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused


$ python3.8 -m test test_ssl -u all -F -m test_get_server_certificate -v
test_ssl: testing with 'OpenSSL 1.1.1k  25 Mar 2021' (1, 1, 1, 11, 15)
  under Mac ('10.16', ('', '', ''), 'x86_64')
  HAS_SNI = True
  OP_ALL = 0x8054
  OP_NO_TLSv1_1 = 0x1000
test_get_server_certificate (test.test_ssl.SimpleBackgroundTests) ...  server:  
new connection from ('127.0.0.1', 59370)

 server:  bad connection attempt from ('127.0.0.1', 59370):
Traceback (most recent call last):
   File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/test/test_ssl.py",
 line 2348, in wrap_conn
self.sslconn = self.server.context.wrap_socket(
   File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 
500, in wrap_socket
return self.sslsocket_class._create(
   File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 
1040, in _create
self.do_handshake()
   File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 
1309, in do_handshake
self._sslobj.do_handshake()
 OSError: [Errno 41] Protocol wrong type for socket
ERROR

==
ERROR: test_get_server_certificate (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/test/test_ssl.py",
 line 2142, in test_get_server_certificate
_test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA)
  File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/test/test_ssl.py",
 line 2309, in _test_get_server_certificate
pem = ssl.get_server_certificate((host, port), ca_certs=cert)
  File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 
1484, in get_server_certificate
with context.wrap_socket(sock) as sslsock:
  File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 
500, in wrap_socket
return self.sslsocket_class._create(
  File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 
1040, in _create
self.do_handshake()
  File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 
1309, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 54] Connection reset by peer


$ python3.9 -m test test_ssl -u all -F -m test_get_server_certificate -v
[...]
test_ssl: testing with 'OpenSSL 1.1.1k  25 Mar 2021' (1, 1, 1, 11, 15)
  under Mac ('10.16', ('', '', ''), 'x86_64')
  HAS_SNI = True
  OP_ALL = 0x8054
  OP_NO_TLSv1_1 = 0x1000
test_get_server_certificate (test.test_ssl.SimpleBackgroundTests) ...  server:  
new connection from ('127.0.0.1', 60096)

 server:  bad connection attempt from ('127.0.0.1', 60096):
Traceback (most recent call last):
   File 

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

... should perhaps also return True in that case, but it does not seem to have 
any thing to say.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Using this patch, I'm (so far) no longer able to reproduce these test failures:

diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
index cee97a8302..c9c1546467 100644
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -2443,9 +2443,13 @@ def wrap_conn(self):
 self.server.conn_errors.append(str(e))
 if self.server.chatty:
 handle_error("\n server:  bad connection attempt from " + 
repr(self.addr) + ":\n")
-self.running = False
-self.server.stop()
-self.close()
+
+# bpo-44229, bpo-43855, bpo-44237, and bpo-33450:
+# Ignore spurious EPROTOTYPE returned by write() on macOS.
+if e.errno != errno.EPROTOTYPE and sys.platform != "darwin":
+self.running = False
+self.server.stop()
+self.close()
 return False
 else:
 
self.server.shared_ciphers.append(self.sslconn.shared_ciphers())



Running for 40 minutes now, right now passing 6k iterations for 
test_get_server_certificate, test_msg_callback_deadlock_bpo43577, and 
test_get_server_certificate_sni.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

See also bpo-33450.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Only on MacOS afaik

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Has these issues (ConnectionRefusedError/ConnectionResetError during handshake) 
been reported on other systems than macOS? AFAICS, they've only been reported 
for the macOS CI.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Although I simphatise with the frustration and I would really like to skip them 
myself, as a release manager I am quite concerned that skip them would masks an 
actual regression, as these failures are not present in previous versions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Two GHI test macOS failures yesterday, once on main, this is on 3.10 backport
https://github.com/python/cpython/pull/26850/checks?check_run_id=2885797677

ERROR: test_get_server_certificate (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File "/Users/runner/work/cpython/cpython/Lib/test/test_ssl.py", line 2180, in 
test_get_server_certificate
_test_get_server_certificate(self, *self.server_addr, cert=SIGNING_CA)
  File "/Users/runner/work/cpython/cpython/Lib/test/test_ssl.py", line 2377, in 
_test_get_server_certificate
pem = ssl.get_server_certificate((host, port), ca_certs=cert)
  File "/Users/runner/work/cpython/cpython/Lib/ssl.py", line 1527, in 
get_server_certificate
with create_connection(addr, timeout=timeout) as sock:
  File "/Users/runner/work/cpython/cpython/Lib/socket.py", line 844, in 
create_connection
raise err
  File "/Users/runner/work/cpython/cpython/Lib/socket.py", line 832, in 
create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

So far reported (including #44237) ConnectionRefusedError in
test_get_server_certificate X many
test_msg_callback_deadlock_bpo43577
test_get_server_certificate_sni

ConnectionResetError: 
test_connect_cadata

Please, please, skip the randomly failing tests, as is done for other modules.  
This has been going on a month now.  It waste other people's time, making 
merging more painful.

--
priority: normal -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[Terry J. Reedy]

Terry J. Reedy  added the comment:

See #44237 for another repeatedly failing (false positive) ssl test.  These and 
other randomly failing tests should be skipped for routine CI regression 
testing of PRs.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44229] test_ssl: test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail randomly on the macOS CI

[STINNER Victor]

Change by STINNER Victor :


--
title: Intermittent connection errors in ssl tests on macOS CI -> test_ssl: 
test_get_server_certificate() and test_msg_callback_deadlock_bpo43577() fail 
randomly on the macOS CI

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com