Hi hubo, Wow, that was a very good analysis, thanks a lot for finding that out! Armin is working on fixing the bug.
Cheers, Carl Friedrich On June 28, 2018 7:51:58 AM GMT+02:00, hubo <h...@jiedaibao.com> wrote: >This problem is caused by caching GeneratorExit exception: > >https://bitbucket.org/pypy/pypy/src/8b43b50fbf61f46701d398bf514cf540201ffd03/pypy/interpreter/generator.py?at=py3.5&fileviewer=file-view-default#generator.py-454:457 > >In Python 3, an exception saves its traceback in __traceback__ >attribute. When an exception object with __traceback__ is raised again, >current frame is appended to the __traceback__. When the GeneratorExit >is raised inside a iterator, the __traceback__ attribute of this object >saves the frame, which prevents the iterator from been collected by GC. > >Since __traceback__ attribute is mutable, it is generally a dangerous >idea to reuse an exception object in Python 3, the problem includes: >creating memory leaks with frames, wrong traceback, etc. I think remove >the caching will solve the problem. > >It's quite easy to be validated with the following script: > >def test(): > try: > yield 1 > except BaseException as e: > global ex > ex = e > >for _ in range(10): > t = test() > next(t) > t.close() > >import traceback >traceback.print_tb(ex.__traceback__) > >You will see many frames in the traceback. > >Removing the __traceback__ attribute before each usage is not >acceptable, because the exception may be used concurrently in different >threads, or nested in a finalizer when GC is triggered inside the >generator close. > >2018-06-28 > >hubo > > > >发件人："hubo" <h...@jiedaibao.com> >发送时间：2018-06-28 13:06 >主题：[pypy-dev] Memory leaking with iterator.close() in PyPy3 >收件人："PyPy Developer Mailing List"<pypy-dev@python.org> >抄送： > >In PyPy3, when an iterator is closed with "close()" method, the >iterator leaks and cannot be collected. > >Execute the following script in PyPy3, the memory usage is increasing >very fast, and gc.collect() cannot collect the memory > > >def test(): > yield 1 > >while True: > t = test() > t.close() > > >The tested version: > >Python 3.5.3 (fdd60ed87e94, Apr 24 2018, 06:10:04) >[PyPy 6.0.0 with GCC 6.2.0 20160901] > >This is not reproduced in CPython 3.5 and PyPy2. > >2018-06-28 > > >hubo
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
