Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Grant Edwards
On 2007-04-26, Martin v. Löwis <[EMAIL PROTECTED]> wrote: >> I'm a bit fuzzy on this, but I don't think there _is_ a >> practical way to "return memory to the OS" in many OSes. > > That's not true at all. Most C libraries these days manage > to return memory to the operating system. [...] Thanks

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Magnus Lycka
[EMAIL PROTECTED] wrote: > Our (python-)macro uses massively nested loops which are unfortunately > necessary. These loops perform complex calculations in this commercial > tool. To give you a quick overview how long this macros runs: > > The outer loop takes 5-7 hours for one cycle. Each cycle cr

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Martin v. Löwis
> The moment I close the application that launched the macro, my > ressources get freed. > > So is there a way to free my memory inside my nested loops? Yes. Most likely, it is your algorithm itself which is flawed (not Python, not the application that embeds Python): You keep, most likely, refer

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Martin v. Löwis
> I'm a bit fuzzy on this, but I don't think there _is_ a > practical way to "return memory to the OS" in many OSes. That's not true at all. Most C libraries these days manage to return memory to the operating system. On Win32 (which the OP is most likely to use), the operating system offers the

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread TimC
"Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > Maybe delegating the actual processing to an external python > process you feed through the macro might work. Thanks for the idea. I'll check it out soon and will report about possible improvements. -- http://mail.python.org/mailman/listinfo/pytho

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Paul McGuire
You might try looking at references between objects, especially if there are any cyclic refs. For instance, if you have a data structure in which child nodes have back refs to their parents, try changing these to use weakref's. This may help the garbage collector to better reclaim discarded objec

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Charles Sanders
Grant Edwards wrote: > Assuming the python interpreter free()s the memory, my > understanding is that on Unixes the memory is returned to the > pool used by malloc(), but is not returned to the OS since > there isn't a practical way to ensure that the memory at the > "end" of the data segment is no

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Diez B. Roggisch
TimC schrieb: > Donald 'Paddy' McCarthy <[EMAIL PROTECTED]> wrote > >> Could you split the program into one handling the outer loop and >> calling another program, with data transfer, to handle the inner >> loops? >> >> - Paddy. > > I'm afraid this isn't possible, because the python macro is call

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread TimC
Donald 'Paddy' McCarthy <[EMAIL PROTECTED]> wrote > Could you split the program into one handling the outer loop and > calling another program, with data transfer, to handle the inner > loops? > > - Paddy. I'm afraid this isn't possible, because the python macro is called and started from within

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Donn Cave
In article <[EMAIL PROTECTED]>, Steven Howe <[EMAIL PROTECTED]> wrote: > Interesting questions. What happens when an object is 'cleaned' up by > using the 'del' command. Does the memory space stay in the python > process, get handed back to the OS, or some combination of both? > I remember 'C' on

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Grant Edwards
On 2007-04-25, Steven Howe <[EMAIL PROTECTED]> wrote: >> I'm a bit fuzzy on this, but I don't think there _is_ a >> practical way to "return memory to the OS" in many OSes. For >> example in Unix the C library uses the sbrk() call to increase >> the size of the data segment when additional memory

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Donald 'Paddy' McCarthy
[EMAIL PROTECTED] wrote: > So I read quite a few things about this phenomenon in Python 2.4.x but > I can hardly believe that there is really no solution to my problem. > > We use a commercial tool that has a macro functionality. These macros > are written in python. So far nothing extraordinary.

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Steven Howe
Grant Edwards wrote: > On 2007-04-25, Chris Mellon <[EMAIL PROTECTED]> wrote: > > >> "Returning memory to the OS" doesn't affect exhaustion of your virtual >> address space. More likely, your nested loops are just creating more >> objects than is possible to hold within a single process. >>

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Grant Edwards
On 2007-04-25, Chris Mellon <[EMAIL PROTECTED]> wrote: > "Returning memory to the OS" doesn't affect exhaustion of your virtual > address space. More likely, your nested loops are just creating more > objects than is possible to hold within a single process. I'm a bit fuzzy on this, but I don't t

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Chris Mellon
On 25 Apr 2007 15:04:59 GMT, TimC <[EMAIL PROTECTED]> wrote: > Larry Bates <[EMAIL PROTECTED]> wrote: > > > Let's see for this I need to get out my crystal ball... > > > > If it is a commercial application, you should contact their tech > > support for a solution. The problem isn't specifically a

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread TimC
Larry Bates <[EMAIL PROTECTED]> wrote: > Let's see for this I need to get out my crystal ball... > > If it is a commercial application, you should contact their tech > support for a solution. The problem isn't specifically a Python > problem but rather an implementation problem with their app. >

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Larry Bates
[EMAIL PROTECTED] wrote: > So I read quite a few things about this phenomenon in Python 2.4.x but > I can hardly believe that there is really no solution to my problem. > > We use a commercial tool that has a macro functionality. These macros > are written in python. So far nothing extraordinary.

Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Durumdara
Hi! I got same problem with Lotus Domino + Python. The lotus COM objects are not freed, so I get out from memory. I solve this problem with a little trick: I make two py applications. The first is call the second as like another application. The first is put some work to the seconds datadir. Wh

Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread leuchte
So I read quite a few things about this phenomenon in Python 2.4.x but I can hardly believe that there is really no solution to my problem. We use a commercial tool that has a macro functionality. These macros are written in python. So far nothing extraordinary. Our (python-)macro uses massively