[issue12075] python3.2 memory leak when setting integer key in dictionary

[Jesús Cea Avión]

Changes by Jesús Cea Avión j...@jcea.es:


--
nosy: +jcea

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12075
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12075] python3.2 memory leak when setting integer key in dictionary

[kai zhu]

kai zhu kai...@ugcs.caltech.edu added the comment:

explicit gc.collect() doesn't seem to fix the leak in my application.
my current fix is to not re-instantiate the class attribute (which cost ~7mb) 
during reload  instead reference one created earlier.

i haven't pinpointed y, but i suspect its a corner case, which would rarely 
occur in general usage.  my class also inherits from subprocess.Popen, which 
has a __del__ method, which might interfere w/ collection (although gc.garbage 
says otherwise ;).

the reason i reload is that the script gets modified frequently, which the 
auto-build system will detect  reload.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12075
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12075] python3.2 memory leak when setting integer key in dictionary

[kai zhu]

New submission from kai zhu kai...@ugcs.caltech.edu:

i'm using the latest debian unstable python3.2 build on colinux (2011, may, 14)

## leak.py
##  import imp, leak; imp.reload(leak)
## will leak ~2.5mb per reload
## on i386 debian unstable machine (according to top).
## in my real world app (an automatic build system),
## i run out of memory after a number reloads :(
class Foo(object): pass
Foo.leaky_dictionary = {}
for aa in range(256):
  for bb in range(256):
Foo.leaky_dictionary[(bb  8) | aa] = None


$ python3.2
Python 3.2.1a0 (default, May  5 2011, 00:47:12)
[GCC 4.6.1 20110428 (prerelease)] on linux2
Type help, copyright, credits or license for more information.
 import imp, leak; imp.reload(leak) ## 11mb
module 'leak' from 'leak.py'
 import imp, leak; imp.reload(leak) ## 13mb
module 'leak' from 'leak.py'
 import imp, leak; imp.reload(leak) ## 16mb
module 'leak' from 'leak.py'
 import imp, leak; imp.reload(leak) ## 19mb
module 'leak' from 'leak.py'

--
messages: 135961
nosy: kaizhu
priority: normal
severity: normal
status: open
title: python3.2 memory leak when setting integer key in dictionary
type: resource usage
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12075
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12075] python3.2 memory leak when setting integer key in dictionary

[Ezio Melotti]

Ezio Melotti ezio.melo...@gmail.com added the comment:

I think this has to do with class attributes and reload():

wolf@hp:~/dev/py/py3k$ cat leak.py
class Foo: pass
Foo.l = list(range(65535))

wolf@hp:~/dev/py/py3k$ ./python 
Python 3.3a0 (default:4b122cac7ac5+, May 14 2011, 10:01:13) 
[GCC 4.5.2] on linux2
Type help, copyright, credits or license for more information.
 import imp, leak; imp.reload(leak)
module 'leak' from 'leak.py'
[189091 refs]
 imp.reload(leak)
module 'leak' from 'leak.py'
[254649 refs]
 imp.reload(leak)
module 'leak' from 'leak.py'
[320207 refs]
 imp.reload(leak)
module 'leak' from 'leak.py'
[385765 refs]
 import gc; gc.collect()
28
[123927 refs]

However calling gc.collect() explicitly seems to fix the problem.

--
nosy: +ezio.melotti
versions: +Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12075
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12075] python3.2 memory leak when setting integer key in dictionary

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc amaur...@gmail.com added the comment:

It's simply because all classes form a cycle (Foo - Foo.__mro__ - Foo)
A class and class attributes can only be freed with gc.collect().
Did you disable the garbage collector?

--
nosy: +amaury.forgeotdarc

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12075
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com