Re: Multiple interpreters retaining huge amounts of memory

2008-02-07 Thread Martin v. Löwis
> Each cycle leaks (or loses) 132k, which is a significant hit -- in my > real program the hit is around 800k/interpreter. > > I ran it through purify (after rebuilding python with the puremodule, no > pymalloc, no optimization, no threads, and debugging), and while the > results are somewhat ambi

RE: Multiple interpreters retaining huge amounts of memory

2008-02-07 Thread Bronner, Gregory
- From: Rhamphoryncus [mailto:[EMAIL PROTECTED] Sent: Thursday, February 07, 2008 12:38 PM To: python-list@python.org Subject: Re: Multiple interpreters retaining huge amounts of memory On Feb 2, 10:32 pm, Graham Dumpleton <[EMAIL PROTECTED]> wrote: > The multi interpreter feature has some limi

Re: Multiple interpreters retaining huge amounts of memory

2008-02-07 Thread Rhamphoryncus
On Feb 2, 10:32 pm, Graham Dumpleton <[EMAIL PROTECTED]> wrote: > The multi interpreter feature has some limitations, but if you know > what you are doing and your application can be run within those > limitations then it works fine. I've been wondering about this for a while. Given the severe li

Re: Multiple interpreters retaining huge amounts of memory

2008-02-06 Thread Martin v. Löwis
> What objects need to be shared across interpreters? > > My thought was to add an interpreter number to the PyThreadState > structure, to increment it when Py_NewInterpreter is called, and to > keep track of the interpreter that creates each object. On deletion, > all memory belonging to the

RE: Multiple interpreters retaining huge amounts of memory

2008-02-06 Thread Bronner, Gregory
would be freed. Thoughts? -Original Message- From: "Martin v. Löwis" [mailto:[EMAIL PROTECTED] Sent: Friday, February 01, 2008 8:34 PM To: python-list@python.org Subject: Re: Multiple interpreters retaining huge amounts of memory > Is there some way to track references per in

Re: Multiple interpreters retaining huge amounts of memory

2008-02-03 Thread Graham Dumpleton
On Feb 4, 10:03 am, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > >>> It means that > >>> environment variable separation for changes made unique to a sub > >>> interpreter is impossible. > >> That's not really true. You can't use os.environ for that, yes. > > > Which bit isn't really true? > > Th

Re: Multiple interpreters retaining huge amounts of memory

2008-02-03 Thread Martin v. Löwis
>>> It means that >>> environment variable separation for changes made unique to a sub >>> interpreter is impossible. >> That's not really true. You can't use os.environ for that, yes. > > Which bit isn't really true? The last sentence ("It means that..."). > When you do: > > os.environ['XYZ'

Re: Multiple interpreters retaining huge amounts of memory

2008-02-03 Thread Graham Dumpleton
On Feb 4, 7:13 am, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > You might also read section 'Application Environment Variables' of > > that document. This talks about the problem of leakage of environment > > variables between sub interpreters. There probably isn't much that one > > can do abou

Re: Multiple interpreters retaining huge amounts of memory

2008-02-03 Thread Martin v. Löwis
>> - objects can easily get shared across interpreters, and often are. >>This is particularly true for static variables that extensions keep, >>and for static type objects. > > Yep, but basically a problem with how people write C extension > modules. Ie., they don't write them with the fac

Re: Multiple interpreters retaining huge amounts of memory

2008-02-03 Thread Graham Dumpleton
Nice to see that your comments do come from some understanding of the issues. Been number of times in the past when people have gone off saying things about multiple interpreters, didn't really know what they were talking about and were just echoing what some one else had said. Some of the things b

Re: Multiple interpreters retaining huge amounts of memory

2008-02-02 Thread Martin v. Löwis
> If you are going to make a comment such as 'multi-interpreter feature > doesn't really work' you really should substantiate it by pointing to > where it is documented what the problems are or enumerate yourself > exactly what the issues are. There is already enough FUD being spread > around about

Re: Multiple interpreters retaining huge amounts of memory

2008-02-02 Thread Graham Dumpleton
On Feb 2, 12:34 pm, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > Is there some way to track references per interpreter, or to get the > > memory allocator to set up seperate arenas per interpreter so that it > > can remove all allocated memory when the interpreter exits? > > No. The multi-inter

Re: Multiple interpreters retaining huge amounts of memory

2008-02-01 Thread Martin v. Löwis
> Is there some way to track references per interpreter, or to get the > memory allocator to set up seperate arenas per interpreter so that it > can remove all allocated memory when the interpreter exits? No. The multi-interpreter feature doesn't really work, so you are basically on your own. If y