[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

STINNER Victor added the comment: I'm not 100% sure that all issues are fixed, but the tests seem much more reliable yet. I close the issue. If the bug reoccurs, I will reopen the issue or open a new one. -- priority: deferred blocker -> resolution: -> fixed stage: patch review ->

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: Should we close this issue now? -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Change by miss-islington : -- pull_requests: +6805 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

miss-islington added the comment: New changeset 4b828467a3fcec0c1947e8326f67b8db12a4f303 by Miss Islington (bot) in branch '3.7': bpo-32458: Further tune start_tls tests (GH-7166) https://github.com/python/cpython/commit/4b828467a3fcec0c1947e8326f67b8db12a4f303 -- nosy:

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

STINNER Victor added the comment: While Yury seems unable to reproduce the bug, it's easy for me to reproduce it on Linux. After 2 hours of debugging, I found the root issue: a race condition not in the test, but in asyncio itself! => bpo-33674 "asyncio: race condition in SSLProtocol".

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: New changeset 8267ea2e84d355f00654dec3ad782fc7b1f680f1 by Yury Selivanov in branch 'master': bpo-32458: Further tune start_tls tests (#7166) https://github.com/python/cpython/commit/8267ea2e84d355f00654dec3ad782fc7b1f680f1 --

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Change by Yury Selivanov : -- pull_requests: +6801 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: https://github.com/python/cpython/pull/7130 is in. It includes many tests for TLS tests as well as bugfixes to asyncio code. Hopefully this all will make tests more stable. -- ___ Python tracker

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Change by Giampaolo Rodola' : -- nosy: -giampaolo.rodola ___ Python tracker ___ ___

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

STINNER Victor added the comment: > OK, sounds good, will do that. Race condition! I created https://github.com/python/cpython/pull/7157 to increase the timeout from 5 seconds to 60 seconds. -- ___ Python tracker

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Change by STINNER Victor : -- pull_requests: +6793 ___ Python tracker ___ ___

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: > If the purpose is to detect when the test hangs: use a timeout of 5 minutes, > or at least 1 minute (60 seconds). OK, sounds good, will do that. -- ___ Python tracker

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

STINNER Victor added the comment: > Yeah, I agree. The current timeout for the test is 5 seconds and it tries to > transfer 2mb of data, which isn't that much actually. I'll bump the timeout > and reduce the amount of data transferred in >

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: > It's a race condition which doesn't depend on the OS, but on the system load. Yeah, I agree. The current timeout for the test is 5 seconds and it tries to transfer 2mb of data, which isn't that much actually. I'll bump the timeout

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

STINNER Victor added the comment: I just reproduced the issue on Linux. Open 3 terminals and run the commands in parallel: (1) ./python -m test test_asyncio -m test_start_tls_server_1 -F (2) ./python -m test -j16 -r (3) ./python -m test -j16 -r It's a race condition

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: Christian, I believe https://github.com/python/cpython/pull/7130/commits/bd17a5593f03e3f2d39a64f5afd3ffb77c97e914 fixes the issue with OpenSSL 1.1.1 (socket.shutdown was missing and asyncio's protocol didn't receive EOF, instead the

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Terry J. Reedy added the comment: 'deleted somehow': Christian added himself while I had this page open, so when I submitted without refreshing, the nosy list did not include him. When I saw the red warning, I did not notice the nosy list change; I should have. --

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Christian Heimes added the comment: Linux -- ___ Python tracker ___ ___ Python-bugs-list

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: Terry, you somehow deleted Christian from the nosy list. Christian, > It's failing reproducible with OpenSSL 1.1.1 and TLS 1.3 enabled. I haven't > seen it failing with TLS 1.2 yet. On Linux or Windows? -- nosy:

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Terry J. Reedy added the comment: I presume the above is using the month-old openssl-bin-1.1.0h. -- ___ Python tracker ___

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Terry J. Reedy added the comment: I have not seen test_asyncio fail on AppVeyor since about Sunday. https://ci.appveyor.com/project/python/cpython/history shows only a couple of failures, which could very well be real. I have seen test_asyncio fail on Travis since Sunday.

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Christian Heimes added the comment: It's failing reproducible with OpenSSL 1.1.1 and TLS 1.3 enabled. I haven't seen it failing with TLS 1.2 yet. -- nosy: +christian.heimes ___ Python tracker

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: > But in msg317468 Victor asserts that it sometimes fails on Linux, too? Hm, I missed that. I'll definitely take a look. -- ___ Python tracker

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Ned Deily added the comment: > Last time I looked into this I couldn't reproduce the failures on my Windows > 10 VM, so it seems like an AppVeyor-specific problem. I'll take another look > on Monday. But in msg317468 Victor asserts that it sometimes fails on Linux, too?

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: Last time I looked into this I couldn't reproduce the failures on my Windows 10 VM, so it seems like an AppVeyor-specific problem. I'll take another look on Monday. Andrew, if you have a Windows environment, could you please try to run

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Ned Deily added the comment: Yury, are you still planning to address this? -- ___ Python tracker ___

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: Not at my computer right now, can do it tomorrow. -- ___ Python tracker ___

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

STINNER Victor added the comment: > I'm OK to skip it for now. Writing functional tests is super hard because > some buildbots are super slow and unpredictable. Would you mind to write a PR to skip the PR? So you will feel guilty and will try to remind to fix it!

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Yury Selivanov added the comment: I'm OK to skip it for now. Writing functional tests is super hard because some buildbots are super slow and unpredictable. -- ___ Python tracker

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

STINNER Victor added the comment: It would be nice to fix this bug before Python 3.7.0 final: either skip the test, or fix it. Flaky tests can be very annoying. For example, the full test suite is run to build a Red Hat package. If a single test fails, the package build

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

Change by Antoine Pitrou : -- nosy: -pitrou ___ Python tracker ___ ___ Python-bugs-list

[issue32458] test_asyncio: test_start_tls_server_1() fails randomly

STINNER Victor added the comment: test_start_tls_server_1() just failed on my Linux. It likely depends on the system load. -- nosy: +vstinner title: test_asyncio failures on Windows -> test_asyncio: test_start_tls_server_1() fails randomly