[issue38856] asyncio ProactorEventLoop: wait_closed() can raise ConnectionResetError

[Bas Nijholt]

Bas Nijholt  added the comment:

I have noticed the following on Linux too:
```
Traceback (most recent call last):
  File "/config/custom_components/kef_custom/aiokef.py", line 327, in 
_disconnect
await self._writer.wait_closed()
  File "/usr/local/lib/python3.7/asyncio/streams.py", line 323, in wait_closed
await self._protocol._closed
  File "/config/custom_components/kef_custom/aiokef.py", line 299, in 
_send_message
await self._writer.drain()
  File "/usr/local/lib/python3.7/asyncio/streams.py", line 339, in drain
raise exc
  File "/usr/local/lib/python3.7/asyncio/selector_events.py", line 814, in 
_read_ready__data_received
data = self._sock.recv(self.max_size)
ConnectionResetError: [Errno 104] Connection reset by peer
```
after which every invocation of `await asyncio.open_connection` fails with 
`ConnectionRefusedError` until I restart the entire Python process.

IIUC, just ignoring the exception will not be enough.

This issue is about the `ProactorEventLoop` for Windows specifically, however, 
my process uses the default event loop on Linux.

--
nosy: +basnijholt

___
Python tracker 
<https://bugs.python.org/issue38856>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36281] OSError: handle is closed for ProcessPoolExecutor and run_in_executor

[Bas Nijholt]

Bas Nijholt  added the comment:

Using `git bisect` I've discovered the commit (b713adf27a) 
(https://github.com/python/cpython/commit/b713adf27a) that broke the code.

I've used one script:
```test.py
import sys
sys.path.append("/Users/basnijholt/Downloads/cpython/Lib/concurrent/futures/")
from random import random
from process import ProcessPoolExecutor
import asyncio

ioloop = asyncio.get_event_loop()

async def func(ioloop, executor):
result = await ioloop.run_in_executor(executor, random)
executor.shutdown(wait=False)  # bug doesn't occur when `wait=True`

if __name__ == "__main__":
executor = ProcessPoolExecutor()
task = ioloop.run_until_complete(func(ioloop, executor))
```
and `test2.py`
```
import pexpect
import sys

child = pexpect.spawn("python /Users/basnijholt/Downloads/cpython/test.py")
try:
child.expect(["OSError", "AssertionError"], timeout=1)
raise Exception
except pexpect.EOF as e:
sys.exit(0)
```

Then did
```
git checkout master
git reset --hard 9b6c60cbce  # bad commit
git bisect start
git bisect bad
git bisect good ad2c2d380e  # good commit
git bisect run python test2.py
```

I will see if I can fix it.

--

___
Python tracker 
<https://bugs.python.org/issue36281>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34075] asyncio: We should prohibit setting a ProcessPoolExecutor in with set_default_executor

[Bas Nijholt]

Bas Nijholt  added the comment:

I think this issue is related to  the problem in 
https://bugs.python.org/issue36281

If it indeed is the case, then the fix proposed here and implemented in 
https://github.com/python/cpython/commit/22d25085db2590932b3664ca32ab82c08f2eb2db
 won't really help.

--
nosy: +basnijholt

___
Python tracker 
<https://bugs.python.org/issue34075>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36281] OSError: handle is closed for ProcessPoolExecutor and run_in_executor

[Bas Nijholt]

Change by Bas Nijholt :


--
type:  -> crash

___
Python tracker 
<https://bugs.python.org/issue36281>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36281] OSError: handle is closed for ProcessPoolExecutor and run_in_executor

[Bas Nijholt]

New submission from Bas Nijholt :

The following code in Python 3.7.1
```
import random
import concurrent.futures
import asyncio

executor = concurrent.futures.ProcessPoolExecutor()
ioloop = asyncio.get_event_loop()

async def func():
result = await ioloop.run_in_executor(executor, random.random)
executor.shutdown(wait=False)  # bug doesn't occur when `wait=True`

task = ioloop.create_task(func())
```


prints the following error:
```
Exception in thread QueueManagerThread:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
  File "/opt/conda/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.7/concurrent/futures/process.py", line 368, in 
_queue_management_worker
thread_wakeup.clear()
  File "/opt/conda/lib/python3.7/concurrent/futures/process.py", line 92, in 
clear
while self._reader.poll():
  File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 255, in 
poll
self._check_closed()
  File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 136, in 
_check_closed
raise OSError("handle is closed")
OSError: handle is closed
```

I think this is related to https://bugs.python.org/issue34073 and 
https://bugs.python.org/issue34075

This happens in the Adaptive package 
https://adaptive.readthedocs.io/en/latest/docs.html#examples and the related 
issue is https://github.com/python-adaptive/adaptive/issues/156

--
components: asyncio
messages: 337868
nosy: asvetlov, basnijholt, yselivanov
priority: normal
severity: normal
status: open
title: OSError: handle is closed for ProcessPoolExecutor and run_in_executor
versions: Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue36281>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com