Kyle Stanley <aeros...@gmail.com> added the comment:
I was able to find a fix! Specifically, I think the issue was a subtle problem
with test_run_in_executor_cancel: it never properly shuts down the executor
prior to closing the event loop.
We do this in asyncio.run() (with the loop.shutdown_default_executor() that I
added last year), but it should be called explicitly when using loop.close()
directly. Otherwise, the resources of the executor may not be cleaned up
properly. I think the issue was being covered up because of the usage of daemon
threads in the ThreadPoolExecutor, but revealed itself when the workers were
converted to non-daemon threads since they require proper cleanup.
The change is very minimal:
```
def test_run_in_executor_cancel(self):
called = False
def patched_call_soon(*args):
nonlocal called
called = True
def run():
time.sleep(0.05)
f2 = self.loop.run_in_executor(None, run)
f2.cancel()
+ self.loop.run_until_complete(
+ self.loop.shutdown_default_executor())
self.loop.close()
self.loop.call_soon = patched_call_soon
self.loop.call_soon_threadsafe = patched_call_soon
time.sleep(0.4)
self.assertFalse(called)
```
Before change:
```
[aeros:~/repos/aeros-cpython]$ ./python -m test --fail-env-changed -R 3:3
test_asyncio -m
test.test_asyncio.test_events.EPollEventLoopTests.test_run_in_executor_cancel
0:00:00 load avg: 0.63 Run tests sequentially
0:00:00 load avg: 0.63 [1/1] test_asyncio
beginning 6 repetitions
123456
......
test_asyncio leaked [1, 1, 1] references, sum=3
test_asyncio leaked [2, 1, 1] memory blocks, sum=4
test_asyncio failed
== Tests result: FAILURE ==
1 test failed:
test_asyncio
Total duration: 3.2 sec
Tests result: FAILURE
```
After change:
```
[aeros:~/repos/aeros-cpython]$ ./python -m test --fail-env-changed -R 3:3
test_asyncio -m
test.test_asyncio.test_events.EPollEventLoopTests.test_run_in_executor_cancel
0:00:00 load avg: 0.24 Run tests sequentially
0:00:00 load avg: 0.24 [1/1] test_asyncio
beginning 6 repetitions
123456
......
== Tests result: SUCCESS ==
1 test OK.
Total duration: 3.5 sec
Tests result: SUCCESS
```
I'll also test the PR using the `test-with-buildbots` label to double check, it
should be attached to this issue within the next couple of hours.
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue40115>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
