[issue20032] asyncio.Future.set_exception() creates a reference cycle

2015-12-09 Thread Jean-Louis Fuchs 

Jean-Louis Fuchs added the comment:

Just to let you know I hit this problem in my code, simplified version:

https://gist.github.com/ganwell/ce3718e5119c6e7e9b3e

Of course it is only a problem because I am a ref-counting stickler.

--
nosy: +Jean-Louis Fuchs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread STINNER Victor 

New submission from STINNER Victor:

asyncio.Future.set_exception(exc) sets the exception attribute to exc, but 
exc.__traceback__ refers to frames and the current frame probably referes to 
the future instance.

Tell me if I'm wrong, but it looks like a reference cycle:
fut -- fut.exception -- exception --exception.__traceback__ - traceback 
--traceback.tb_frame -- frame --frame.fb_locals -- fut

The frame class got a new clear() method in Python 3.4:
http://docs.python.org/dev/reference/datamodel.html#frame.clear

Maybe because of the PEP 442, the reference cycle is no more an issue. In fact, 
the following example calls fut destructor immediatly, at fut = None line.
---
import asyncio

fut = asyncio.Future()
try:
raise ValueError()
except Exception as err:
fut.set_exception(err)
fut = None
---

Attached patch breaks explicitly the reference cycle by scheduling a call to 
traceback.clear_frames() using call_soon(). The patch depends on 
asyncio_defer_format_tb.patch which is attached to the issue #19967.

--
files: asyncio_break_ref_cycle.patch
keywords: patch
messages: 206672
nosy: gvanrossum, haypo, pitrou
priority: normal
severity: normal
status: open
title: asyncio.Future.set_exception() creates a reference cycle
versions: Python 3.4
Added file: http://bugs.python.org/file33228/asyncio_break_ref_cycle.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread STINNER Victor 

STINNER Victor added the comment:

asyncio_break_ref_cycle.patch does not fix the issue on Python 3.3 (for Tulip).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread Guido van Rossum 

Guido van Rossum added the comment:

Do you have an example of code that behaves differently with this patch?  I 
can't find any.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread Antoine Pitrou 

Antoine Pitrou added the comment:

+self._loop.call_soon(traceback.clear_frames,
+ self._exception.__traceback__)

This will keep the traceback alive until called by the event loop, even if 
self._exception is cleared in the meantime...

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread STINNER Victor 

STINNER Victor added the comment:

 Do you have an example of code that behaves differently with this patch?  I 
 can't find any.

I didn't check in the Python standard library, but the reference cycle is 
obvious, and I hate such issue. It introduces tricky issues like memory leaks.

Here is an example to demonstrate the issue. The DELETE OBJECT message is 
never displayed, so the object is never deleted (memory leak).

Comment fut.set_exception(err) line to delete the object, or apply attached 
patch.

--
Added file: http://bugs.python.org/file33238/never_deleted.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread Guido van Rossum 

Guido van Rossum added the comment:

The cycle will be cleaned up (and the message printed) when the
garbage collector runs next. Your demo doesn't do anything else, so it
never allocates memory, so it never runs gc.collect(). But that's only
because it's a toy program.

Maybe it's time to look into
http://code.google.com/p/tulip/issues/detail?id=42 ? (It proposes to
run gc.collect() occasionally when the loop is idle.)

I am also concerned about Antoine's point -- the patch may actually
*prolong* the life of the traceback.

On Fri, Dec 20, 2013 at 2:15 PM, STINNER Victor rep...@bugs.python.org wrote:

 STINNER Victor added the comment:

 Do you have an example of code that behaves differently with this patch?  I 
 can't find any.

 I didn't check in the Python standard library, but the reference cycle is 
 obvious, and I hate such issue. It introduces tricky issues like memory leaks.

 Here is an example to demonstrate the issue. The DELETE OBJECT message is 
 never displayed, so the object is never deleted (memory leak).

 Comment fut.set_exception(err) line to delete the object, or apply attached 
 patch.

 --
 Added file: http://bugs.python.org/file33238/never_deleted.py

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue20032
 ___

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread Antoine Pitrou 

Antoine Pitrou added the comment:

 Maybe it's time to look into
 http://code.google.com/p/tulip/issues/detail?id=42 ? (It proposes to
 run gc.collect() occasionally when the loop is idle.)

Is it possible to break the cycle instead? Or is the graph of references
too complex for that?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread Guido van Rossum 

Guido van Rossum added the comment:

The only reasonable place to break the cycle seems to be the frame containing 
the set_exception() call -- but that could be app code.

Looking again at what the patch actually does I think it is too big a hammer 
anyway -- it would break debugging tools that preserve tracebacks and inspect 
the frames later.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread STINNER Victor 

STINNER Victor added the comment:

The cycle will be cleaned up (and the message printed) when the
garbage collector runs next.

Oh, ok. Using the following task, the object is correctly deleted.
---
@asyncio.coroutine
def idle():
while 1:
gc.collect()
yield from asyncio.sleep(0.1)

asyncio.Task(idle())
---

Maybe it's time to look into
http://code.google.com/p/tulip/issues/detail?id=42 ? (It proposes to
run gc.collect() occasionally when the loop is idle.)

I don't like such task. The issue can be documented, maybe with an example of 
call calling gc.collect() regulary? Such background task should be implemented 
in the application to control when the garbage collector is called.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20032] asyncio.Future.set_exception() creates a reference cycle

2013-12-20 Thread STINNER Victor 

Changes by STINNER Victor victor.stin...@gmail.com:


--
resolution:  - invalid
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20032
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com