Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset 57f0af61da53 by Antoine Pitrou in branch 'default':
Issue #6695: Full garbage collection runs now clear the freelist of set objects.
http://hg.python.org/cpython/rev/57f0af61da53
--
nosy: +python-dev
Antoine Pitrou pit...@free.fr added the comment:
The clearing the dict and list freelists has been added independently, so I
committed the part of the patch pertaining to sets. Thank you!
--
resolution: - fixed
stage: - committed/rejected
status: open - closed
versions: +Python 3.3
Matthias Troffaes matthias.troff...@gmail.com added the comment:
Patch against current tip attached.
I can no longer reproduce the large memory leaks with the current tip (which is
of course wonderful!), so I guess the second part of the patch (freeing the
freelists during gc.collect) makes
Changes by Matthias Troffaes matthias.troff...@gmail.com:
Added file:
http://bugs.python.org/file17427/py3k-rev81387-clearfreelist-dict_set_list.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6695
Changes by Matthias Troffaes matthias.troff...@gmail.com:
Added file:
http://bugs.python.org/file17428/py3k-rev81387-clearfreelist-gc_collect.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6695
Changes by Matthias Troffaes matthias.troff...@gmail.com:
Added file:
http://bugs.python.org/file17429/py3k-rev81387-clearfreelist-time_gc_collect.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6695
Matthias Troffaes matthias.troff...@gmail.com added the comment:
I uploaded updates of the three relevant patches against the current revision
of the py3k branch, as the old patches no longer applied cleanly due to
whitespace changes.
To summarize:
* The first patch,
Changes by STINNER Victor victor.stin...@haypocalc.com:
--
nosy: +haypo
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6695
___
___
Changes by Skip Montanaro s...@pobox.com:
--
nosy: -skip.montanaro
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6695
___
___
Python-bugs-list
Matthias Troffaes matthias.troff...@gmail.com added the comment:
To aid the discussion, I attach another quick patch which reports the
time spent on PyXXX_ClearFreeList calls during highest generation
garbage collection (including gc.collect()). For simplicity, the timer
uses clock() so the
Skip Montanaro s...@pobox.com added the comment:
I thought GC was expected to eliminate reference cycles.
Antoine Of course, but it's also the de facto API when wanting to
Antoine reclaim memory.
When did that happen? I agree with Raymond. The cyclic gc should just
reclaim cycles.
Antoine Pitrou pit...@free.fr added the comment:
When did that happen? I agree with Raymond. The cyclic gc should just
reclaim cycles.
People don't care about referential cycles, they care about freeing some
memory (if memory was available in infinite quantities nobody would care
about
Antoine Pitrou pit...@free.fr added the comment:
The change was originally made in r60797, « Implemented Martin's
suggestion to clear the free lists during the garbage collection of the
highest generation ».
--
nosy: +christian.heimes, loewis
___
R. David Murray rdmur...@bitdance.com added the comment:
FWIW, I agree with Antoine here. I think user expectation is that when
garbage is collected, at least some freed memory will be returned to
the operating system. The normal user's conception of what garbage is
has nothing to do with
Changes by Raymond Hettinger rhettin...@users.sourceforge.net:
--
resolution: accepted -
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6695
___
Martin v. Löwis mar...@v.loewis.de added the comment:
I still stand by my suggestion to free memory as a side effect of
garbage collection. It may well be that an application will start
re-allocating blocks that soon end up in the free list again. OTOH, it
may also be that releasing those free
Gregory P. Smith g...@krypto.org added the comment:
+1 on the PyXXX_ClearFreeList patch and calling them from gc.collect()
as is done with the others.
I agree with Guido, don't add a tp_free_list slot as the common case
would be NULL.
Regarding gc clearing freelists: I agree with Antoine and
Antoine Pitrou pit...@free.fr added the comment:
The reason is that users expect gc.collect() to make its best to
diminish memory use. Clearing free lists can allow deallocting some
arenas which otherwise would still contain some used memory blocks. As
the comment says:
/* Clear all free lists
Raymond Hettinger rhettin...@users.sourceforge.net added the comment:
The reason is that users expect gc.collect() to make
its best to diminish memory use.
I thought GC was expected to eliminate reference cycles.
Perhaps there ought to be a separate API, such as
sys.clear_freelists(), to
Antoine Pitrou pit...@free.fr added the comment:
Le samedi 15 août 2009 à 22:06 +, Raymond Hettinger a écrit :
Raymond Hettinger rhettin...@users.sourceforge.net added the comment:
The reason is that users expect gc.collect() to make
its best to diminish memory use.
I thought GC
Matthias Troffaes matthias.troff...@gmail.com added the comment:
Thanks for the feedback!
Attaching a new patch which implements tp_free_list slot as suggested -
I hope I did it correctly. I've only implemented the new slot for dict
so far, but I'm happy to tp_free_list-ify the other freelist
Raymond Hettinger rhettin...@users.sourceforge.net added the comment:
Guido, do you want a slot assigned for this?
--
assignee: - gvanrossum
nosy: +gvanrossum, rhettinger
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6695
Guido van Rossum gu...@python.org added the comment:
A slot in every type object for this purpose seems wasteful; the large
majority of types won't have a free list. (Remember, each user-defined
class allocates a full type structure on the heap.)
--
assignee: gvanrossum -
Raymond Hettinger rhettin...@users.sourceforge.net added the comment:
Marking the PyXXX_ClearFreeList version as approved.
--
resolution: - accepted
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6695
Raymond Hettinger rhettin...@users.sourceforge.net added the comment:
Does anyone here know why GC calls the free_xxx functions? ISTM, they
cannot be involved in cycles. Free lists are kept by container objects
to speed-up allocation. Having GC call the free_xxx just slows down the
GC process
New submission from Matthias Troffaes matthias.troff...@gmail.com:
The Python C API provides PyXXX_ClearFreeList functions to allow the
float, int, etc... freelists to be freed, potentially releasing memory
to the OS earlier. Currently, there is no such API for the dict, set,
and list freelists.
Matthias Troffaes matthias.troff...@gmail.com added the comment:
I attach a second patch which also calls the new PyXXX_ClearFreeList
functions on garbage collection, during gc.collect().
--
Added file: http://bugs.python.org/file14709/py3k-clearfreelist-gc_collect.patch
Matthias Troffaes matthias.troff...@gmail.com added the comment:
I'm also attaching a test script to check the effect of the two patches
on gc.collect(). If many objects are allocated, space savings appear to
be relevant.
Before applying the patch (debug build on linux 64 bit):
Memory used
Skip Montanaro s...@pobox.com added the comment:
Instead of expanding the C API for each type which supports a free
list perhaps there should be a single call, say, PyObject_ClearFreeList,
which takes a pointer to the appropriate type object as an argument.
PyTypeObject can then grow a
29 matches
Mail list logo
