RE: Urgent : How to do memory leaks detection in python ?
Thanks for your inputs !!!
I have installed python v 2.5 on my Linux machine and executing the tool
again.
I would like to share the memory status( using free -m command ) before and
after the execution of the tool.
BEFORE EXECUTION
total used free sharedbuffers
cached
Mem: 1006148*858* 0 8 92
-/+ buffers/cache: 46960
Swap: 2047 0 2047
AFTER EXECUTION
===
total used free sharedbuffers
cached
Mem: 1006940 *66* 0 49846
-/+ buffers/cache: 44962
Swap: 2047 0 2047
I am unable to find out why *66 MB* system memory is left after tool
execution ? If python does not have memory leaks then where this memory is
going ?
I have explored few urls (as given below) related to memory leak in python :
http://www.nightmare.com/medusa/memory-leaks.html
http://mail.python.org/pipermail/tutor/1999-April/000162.html
Please comment !!!
-Original Message-
*From:* [EMAIL PROTECTED] [mailto:
[EMAIL PROTECTED] *On Behalf Of *tsuraan
*Sent:* 16 March 2008 8:27 AM
*To:* python-list@python.org
*Subject:* Re: Urgent : How to do memory leaks detection in python ?
Python doesn't have memory leaks.
Yeah, interesting bit of trivia: python is the world's only non-trivial
program that's totally free of bugs. Pretty exciting! But seriously,
python 2.4, at least, does have some pretty trivially exposed memory leaks
when working with strings. A simple example is this:
letters = [chr(c) for c in range(ord('a'), ord('z'))+range(ord('A'),
ord('Z'))]
ary = []
for a in letters:
... for b in letters:
... for c in letters:
...for d in letters:
... ary.append(a+b+c+d)
...
del(ary)
import gc
gc.collect()
0
The VM's memory usage will never drop from its high point of (on my
computer) ~200MB. Since you're using GIS data, this could be what you're
running into. I haven't been able to upgrade my systems to python 2.5, but
from my tests, that version did not have that memory leak. Nobody seems
interesting in backporting fixes from 2.5 to 2.4, so you're probably on your
own in that case as well, if upgrading to python 2.5 isn't an option or
isn't applicable to your situation.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Urgent : How to do memory leaks detection in python ?
On Mar 17, 2008, at 3:21 AM, Pradeep Rai wrote: Thanks for your inputs !!! I have installed python v 2.5 on my Linux machine and executing the tool again. I would like to share the memory status( using free -m command ) before and after the execution of the tool. BEFORE EXECUTION total used free shared buffers cached Mem: 1006148858 0 8 92 -/+ buffers/cache: 46960 Swap: 2047 0 2047 AFTER EXECUTION === total used free shared buffers cached Mem: 1006940 66 0 49846 -/+ buffers/cache: 44962 Swap: 2047 0 2047 I am unable to find out why 66 MB system memory is left after tool execution ? If python does not have memory leaks then where this memory is going ? the free you are looking at is not a good indication of 'actual memory available' in linux. the number you are intersted is this one: -/+ buffers/cache: 46960 vs -/+ buffers/cache: 44962 before execution you had 960 available for use by applications after execution you had 962 available. here is one of many pages explaining memory under linux: http://gentoo-wiki.com/FAQ_Linux_Memory_Management -- http://mail.python.org/mailman/listinfo/python-list
Re: Urgent : How to do memory leaks detection in python ?
Python doesn't have memory leaks.
Yeah, interesting bit of trivia: python is the world's only non-trivial
program that's totally free of bugs. Pretty exciting! But seriously,
python 2.4, at least, does have some pretty trivially exposed memory leaks
when working with strings. A simple example is this:
letters = [chr(c) for c in range(ord('a'), ord('z'))+range(ord('A'),
ord('Z'))]
ary = []
for a in letters:
... for b in letters:
... for c in letters:
...for d in letters:
... ary.append(a+b+c+d)
...
del(ary)
import gc
gc.collect()
0
The VM's memory usage will never drop from its high point of (on my
computer) ~200MB. Since you're using GIS data, this could be what you're
running into. I haven't been able to upgrade my systems to python 2.5, but
from my tests, that version did not have that memory leak. Nobody seems
interesting in backporting fixes from 2.5 to 2.4, so you're probably on your
own in that case as well, if upgrading to python 2.5 isn't an option or
isn't applicable to your situation.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Urgent : How to do memory leaks detection in python ?
2008/3/14, Pradeep Rai [EMAIL PROTECTED]: Dear All, I am working on the python tools that process a huge amount of GIS data. These tools encountering the problem of memory leaks. Please suggest what are the different ways to detect the memory leaks in python ? This is very critical problem for me. Help needed urgently. Thanks Regards, Pradeep Python doesn't have memory leaks. -- http://mail.python.org/mailman/listinfo/python-list
RE: Urgent : How to do memory leaks detection in python ?
This is not entirely true: Symptoms of increasing memory usage are either because a) you are keeping too much data around in user accessable memory (likely) b) you are creating self-referential structures that are not garbage collected (also likely) c) You have memory leaks in underlying C extension modules. For category a), you are on your own. For category b), you can use various methods in the gc module to print out everything that is still 'live'. You can also recompile python to build a list of all objects that are still live. For category c), a tool like valgrind or purify often helps. Create a simple example rather than trying to run it on your whole application. From: Michael Wieher [mailto:[EMAIL PROTECTED] Sent: Friday, March 14, 2008 10:16 AM To: python-list@python.org Subject: Re: Urgent : How to do memory leaks detection in python ? 2008/3/14, Pradeep Rai : Dear All, I am working on the python tools that process a huge amount of GIS data. These tools encountering the problem of memory leaks. Please suggest what are the different ways to detect the memory leaks in python ? This is very critical problem for me. Help needed urgently. Thanks Regards, Pradeep Python doesn't have memory leaks. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - This message is intended only for the personal and confidential use of the designated recipient(s) named above. If you are not the intended recipient of this message you are hereby notified that any review, dissemination, distribution or copying of this message is strictly prohibited. This communication is for information purposes only and should not be regarded as an offer to sell or as a solicitation of an offer to buy any financial product, an official confirmation of any transaction, or as an official statement of Lehman Brothers. Email transmission cannot be guaranteed to be secure or error-free. Therefore, we do not represent that this information is complete or accurate and it should not be relied upon as such. All information is subject to change without notice. IRS Circular 230 Disclosure: Please be advised that any discussion of U.S. tax matters contained within this communication (including any attachments) is not intended or written to be used and cannot be used for the purpose of (i) avoiding U.S. tax related penalties or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein. -- http://mail.python.org/mailman/listinfo/python-list
