Possible to set cpython heap size?

2007-02-22 Thread Andy Watson
I have an application that scans and processes a bunch of text files. The content I'm pulling out and holding in memory is at least 200MB. I'd love to be able to tell the CPython virtual machine that I need a heap of, say 300MB up front rather than have it grow as needed. I've had a scan through

Re: Possible to set cpython heap size?

2007-02-22 Thread Diez B. Roggisch
Andy Watson wrote: > I have an application that scans and processes a bunch of text files. > The content I'm pulling out and holding in memory is at least 200MB. > > I'd love to be able to tell the CPython virtual machine that I need a > heap of, say 300MB up front rather than have it grow as nee

Re: Possible to set cpython heap size?

2007-02-22 Thread Andy Watson
> Why do you want that? And no, it is not possible. And to be honest: I have > no idea why e.g. the JVM allows for this. > > Diez The reason why is simply that I know roughly how much memory I'm going to need, and cpython seems to be taking a fair amount of time extending its heap as I read in co

Re: Possible to set cpython heap size?

2007-02-22 Thread Diez B. Roggisch
Andy Watson wrote: > > Why do you want that? And no, it is not possible. And to be honest: > I have >> no idea why e.g. the JVM allows for this. > > The reason why is simply that I know roughly how much memory I'm going > to need, and cpython seems to be taking a fair amount of time > extending

Re: Possible to set cpython heap size?

2007-02-22 Thread Irmen de Jong
Andy Watson wrote: > > Why do you want that? And no, it is not possible. And to be honest: > I have >> no idea why e.g. the JVM allows for this. >> >> Diez > > The reason why is simply that I know roughly how much memory I'm going > to need, and cpython seems to be taking a fair amount of time

Re: Possible to set cpython heap size?

2007-02-22 Thread Chris Mellon
On 22 Feb 2007 09:52:49 -0800, Andy Watson <[EMAIL PROTECTED]> wrote: > > Why do you want that? And no, it is not possible. And to be honest: > I have > > no idea why e.g. the JVM allows for this. > > > > Diez > > The reason why is simply that I know roughly how much memory I'm going > to need, an

Re: Possible to set cpython heap size?

2007-02-22 Thread Jussi Salmela
Andy Watson kirjoitti: > I have an application that scans and processes a bunch of text files. > The content I'm pulling out and holding in memory is at least 200MB. > > I'd love to be able to tell the CPython virtual machine that I need a > heap of, say 300MB up front rather than have it grow as

Re: Possible to set cpython heap size?

2007-02-22 Thread Andy Watson
On Feb 22, 10:53 am, a bunch of folks wrote: > Memory is basically free. This is true if you are simply scanning a file into memory. However, I'm storing the contents in some in-memory data structures and doing some data manipulation. This is my speculation: Several small objects per scanned

Re: Possible to set cpython heap size?

2007-02-22 Thread Chris Mellon
On 22 Feb 2007 11:28:52 -0800, Andy Watson <[EMAIL PROTECTED]> wrote: > On Feb 22, 10:53 am, a bunch of folks wrote: > > > Memory is basically free. > > This is true if you are simply scanning a file into memory. However, > I'm storing the contents in some in-memory data structures and doing > som

Re: Possible to set cpython heap size?

2007-02-22 Thread Martin v. Löwis
Andy Watson schrieb: > I have an application that scans and processes a bunch of text files. > The content I'm pulling out and holding in memory is at least 200MB. > > I'd love to be able to tell the CPython virtual machine that I need a > heap of, say 300MB up front rather than have it grow as ne

Re: Possible to set cpython heap size?

2007-02-23 Thread Andrew MacIntyre
Chris Mellon wrote: > On 22 Feb 2007 11:28:52 -0800, Andy Watson <[EMAIL PROTECTED]> wrote: >> On Feb 22, 10:53 am, a bunch of folks wrote: >> >>> Memory is basically free. >> This is true if you are simply scanning a file into memory. However, >> I'm storing the contents in some in-memory data st

Re: Possible to set cpython heap size?

2007-02-23 Thread Tony Nelson
In article <[EMAIL PROTECTED]>, "Andy Watson" <[EMAIL PROTECTED]> wrote: ... > If I could have a heap that is larger and does not need to be > dynamically extended, then the Python GC could work more efficiently. ... GC! If you're allocating lots of objects and holding on to them, GC will ru