[issue33469] RuntimeError after closing loop that used run_in_executor

[Yury Selivanov]

Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[miss-islington]

miss-islington  added the comment:


New changeset a6d6bd70ac95a0f7bbfe07d4e60b43afcec370d2 by Miss Islington (bot) 
in branch '3.6':
bpo-33469: RuntimeError after closing loop that used run_in_executor (GH-7171)
https://github.com/python/cpython/commit/a6d6bd70ac95a0f7bbfe07d4e60b43afcec370d2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[miss-islington]

Change by miss-islington :


--
pull_requests: +6811

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset fdccfe09f0b10776645fdb04a0783d6864c32b21 by Yury Selivanov in 
branch 'master':
bpo-33469: RuntimeError after closing loop that used run_in_executor (GH-7171)
https://github.com/python/cpython/commit/fdccfe09f0b10776645fdb04a0783d6864c32b21


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[miss-islington]

Change by miss-islington :


--
pull_requests: +6812

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[miss-islington]

miss-islington  added the comment:


New changeset 8d8b86116fae91570c26fa48974b54986fbd1b72 by Miss Islington (bot) 
in branch '3.7':
bpo-33469: RuntimeError after closing loop that used run_in_executor (GH-7171)
https://github.com/python/cpython/commit/8d8b86116fae91570c26fa48974b54986fbd1b72


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[Yury Selivanov]

Change by Yury Selivanov :


--
keywords: +patch
pull_requests: +6806
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[Yury Selivanov]

Yury Selivanov  added the comment:

Ah, I see, the callback that tracks the state of the wrapped concurrent.Future 
doesn't check that the loop is closed and its future has been cancelled. I 
think this is a bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[Yury Selivanov]

Yury Selivanov  added the comment:

Hopefully asyncio.run() in Python 3.7 will handle this case correctly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[Hrvoje Nikšić]

Change by Hrvoje Nikšić :


--
components: +asyncio
nosy: +asvetlov, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33469] RuntimeError after closing loop that used run_in_executor

[Hrvoje Nikšić]

New submission from Hrvoje Nikšić :

Looking at a StackOverflow question[1], I was unable to find a way to correctly 
close an event loop that uses run_in_executor() for long-running tasks.

The question author tried to implement the following scenario:

1. create some tasks that use run_in_executor
2. run asyncio.wait(tasks, return_when=FIRST_EXCEPTION)
3. cancel pending tasks, if any
4. close the loop and continue with non-async work

However, when there are pending tasks from wait(), a RuntimeError is raised 
some time after step #4. In the test programs, it happens while waiting for the 
program to finish. I have attached a minimal example to reproduce the issue.

The immediate cause is that a callback installed by wrap_future() notices that 
the underlying concurrent.futures.Future is done and calls 
loop.call_soon_threadsafe() to copy the result to the asyncio.Future. 
call_soon_threadsafe() fails when the loop is closed.

This would be reasonable behavior if not for the fact that the code explicitly 
cancelled the asyncio future, and awaited it to ensure that the cancellation 
took effect. While it is clear that asyncio cannot interrupt a function already 
running in an executor, it should probably detach the connection between the 
concurrent future and the asyncio future, to prevent this kind of error (and 
possibly other problems).

For example, the _call_check_cancel closure in _chain_future could remove (or 
disable) the done_callback installed on source after the call to 
source.cancel(). Since at that point we know that destination (asyncio.Future) 
is already canceled, there is no longer value in invoking the done callback for 
source (concurrent.futures.Future).


[1]
https://stackoverflow.com/q/50279522/1600898

--
files: executor-cancel
messages: 316420
nosy: hniksic
priority: normal
severity: normal
status: open
title: RuntimeError after closing loop that used run_in_executor
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47583/executor-cancel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com