Re: Please help with MemoryError

2010-02-14 Thread Steve Holden
rantingrick wrote: On Feb 12, 4:10 pm, Steve Holden st...@holdenweb.com wrote: Antoine Pitrou wrote: Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : On Feb 12, 4:10 pm, Steve Holden st...@holdenweb.com wrote: Antoine Pitrou wrote: Le Fri, 12 Feb 2010 17:14:57 +, Steven

Re: Please help with MemoryError

2010-02-14 Thread Steve Howell
On Feb 14, 10:32 am, Steve Holden st...@holdenweb.com wrote: rantingrick wrote: On Feb 12, 4:10 pm, Steve Holden st...@holdenweb.com wrote: Antoine Pitrou wrote: Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : On Feb 12, 4:10 pm, Steve Holden st...@holdenweb.com wrote:

Re: Please help with MemoryError

2010-02-14 Thread Steve Howell
On Feb 11, 5:50 pm, Steven D'Aprano st...@remove-this- cybersource.com.au wrote: On Thu, 11 Feb 2010 15:39:09 -0800, Jeremy wrote: My Python program now consumes over 2 GB of memory and then I get a MemoryError.  I know I am reading lots of files into memory, but not 2GB worth. 2.    

Re: Please help with MemoryError

2010-02-13 Thread rantingrick
On Feb 12, 4:10 pm, Steve Holden st...@holdenweb.com wrote: Antoine Pitrou wrote: Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : On Feb 12, 4:10 pm, Steve Holden st...@holdenweb.com wrote: Antoine Pitrou wrote: Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit :

Re: Please help with MemoryError

2010-02-12 Thread Tim Chase
Aahz wrote: Tim Chase python.l...@tim.thechases.com wrote: Just to add to the mix, I'd put the anydbm module on the gradient between using a file and using sqlite. It's a nice intermediate step between rolling your own file formats for data on disk, and having to write SQL since access is

Re: Please help with MemoryError

2010-02-12 Thread Aahz
In article mailman.2426.1265976954.28905.python-l...@python.org, Tim Chase python.l...@tim.thechases.com wrote: Aahz wrote: Tim Chase python.l...@tim.thechases.com wrote: Just to add to the mix, I'd put the anydbm module on the gradient between using a file and using sqlite. It's a nice

Re: Please help with MemoryError

2010-02-12 Thread Jeremy
On Feb 11, 6:50 pm, Steven D'Aprano st...@remove-this- cybersource.com.au wrote: On Thu, 11 Feb 2010 15:39:09 -0800, Jeremy wrote: My Python program now consumes over 2 GB of memory and then I get a MemoryError.  I know I am reading lots of files into memory, but not 2GB worth. Are you

Re: Please help with MemoryError

2010-02-12 Thread Tim Chase
Aahz wrote: Not quite. One critical difference between dbm and dicts is the need to remember to save changes by setting the key's valud again. Could you give an example of this? I'm not sure I understand what you're saying. Well, you're more likely to hit this by wrapping dbm with shelve

Re: Please help with MemoryError

2010-02-12 Thread Steven D'Aprano
On Fri, 12 Feb 2010 06:45:31 -0800, Jeremy wrote: You also confirmed what I thought was true that all variables are passed by reference so I don't need to worry about the data being copied (unless I do that explicitly). No, but yes. No, variables are not passed by reference, but yes, you

Re: Please help with MemoryError

2010-02-12 Thread John Posner
On 2/12/2010 12:14 PM, Steven D'Aprano wrote: On Fri, 12 Feb 2010 06:45:31 -0800, Jeremy wrote: You also confirmed what I thought was true that all variables are passed by reference so I don't need to worry about the data being copied (unless I do that explicitly). No, but yes. No,

Re: Please help with MemoryError

2010-02-12 Thread mk
John Posner wrote: http://effbot.org/zone/call-by-object.htm http://en.wikipedia.org/wiki/Evaluation_strategy [1] http://mail.python.org/pipermail/edu-sig/2008-May/008583.html Hmm how about call by label-value? That is, you change labels by assignment, but pass the value of the

Re: Please help with MemoryError

2010-02-12 Thread Mel
mk wrote: John Posner wrote: http://effbot.org/zone/call-by-object.htm http://en.wikipedia.org/wiki/Evaluation_strategy [1] http://mail.python.org/pipermail/edu-sig/2008-May/008583.html Hmm how about call by label-value? Nothing egregiously wrong with it.. maybe it's a bit

Re: Please help with MemoryError

2010-02-12 Thread Christian Heimes
mk wrote: Hmm how about call by label-value? Or call by guido? How do you like call like a dutch? :] -- http://mail.python.org/mailman/listinfo/python-list

Re: Please help with MemoryError

2010-02-12 Thread Alf P. Steinbach
* Christian Heimes: mk wrote: Hmm how about call by label-value? Or call by guido? How do you like call like a dutch? :] Just a note: it might be more clear to talk about pass by XXX than call by XXX. Unless you're talking about something else than argument passing. The standard

Re: Please help with MemoryError

2010-02-12 Thread Ethan Furman
mk wrote: John Posner wrote: http://effbot.org/zone/call-by-object.htm http://en.wikipedia.org/wiki/Evaluation_strategy [1] http://mail.python.org/pipermail/edu-sig/2008-May/008583.html Hmm how about call by label-value? That is, you change labels by assignment, but pass the

Re: Please help with MemoryError

2010-02-12 Thread Steven D'Aprano
On Fri, 12 Feb 2010 21:07:08 +0100, mk wrote: John Posner wrote: http://effbot.org/zone/call-by-object.htm http://en.wikipedia.org/wiki/Evaluation_strategy [1] http://mail.python.org/pipermail/edu-sig/2008-May/008583.html Hmm how about call by label-value? Python's calling

Re: Please help with MemoryError

2010-02-12 Thread Antoine Pitrou
Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : What Python does is called pass by sharing, or sometimes pass by object reference. It is exactly the same as what (e.g.) Ruby and Java do, except that confusingly the Ruby people call it pass by reference and the Java people call

Re: Please help with MemoryError

2010-02-12 Thread Steve Holden
Antoine Pitrou wrote: Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : What Python does is called pass by sharing, or sometimes pass by object reference. It is exactly the same as what (e.g.) Ruby and Java do, except that confusingly the Ruby people call it pass by reference and

Re: Please help with MemoryError

2010-02-12 Thread Alf P. Steinbach
* Antoine Pitrou: Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : What Python does is called pass by sharing, or sometimes pass by object reference. It is exactly the same as what (e.g.) Ruby and Java do, except that confusingly the Ruby people call it pass by reference and the

Re: Please help with MemoryError

2010-02-12 Thread Antoine Pitrou
Le Fri, 12 Feb 2010 17:10:01 -0500, Steve Holden a écrit : As has already been pointed out, if Python used call by reference then the following code would run without raising an AssertionError: def exchange(a, b): a, b = b, a x = 1 y = 2 exchange(x, y) assert (x == 2 and y == 1)

Re: Please help with MemoryError

2010-02-12 Thread Antoine Pitrou
Le Fri, 12 Feb 2010 23:12:06 +0100, Alf P. Steinbach a écrit : Steven talks about the standard meaning of pass by reference. See my answer to Steve's message. You can't postulate a standard meaning of pass by reference independently of the specificities of each language. For example a

Re: Please help with MemoryError

2010-02-12 Thread Alf P. Steinbach
* Antoine Pitrou: Le Fri, 12 Feb 2010 23:12:06 +0100, Alf P. Steinbach a écrit : Steven talks about the standard meaning of pass by reference. See my answer to Steve's message. You can't postulate a standard meaning of pass by reference independently of the specificities of each language.

Re: Please help with MemoryError

2010-02-12 Thread Gib Bogle
Steven D'Aprano wrote: def swap(a, b): a, b = b, a x = 1 y = 2 swap(x, y) assert (x == 2) and (y==1) Can't the same point be more simply made with this example: def setval(a): a = 12345 x = 1 setval(x) print x ? -- http://mail.python.org/mailman/listinfo/python-list

Re: Please help with MemoryError

2010-02-12 Thread Steve Holden
Gib Bogle wrote: Steven D'Aprano wrote: def swap(a, b): a, b = b, a x = 1 y = 2 swap(x, y) assert (x == 2) and (y==1) Can't the same point be more simply made with this example: def setval(a): a = 12345 x = 1 setval(x) print x Yes, and it will doubtless be subject

Re: Please help with MemoryError

2010-02-12 Thread Antoine Pitrou
Le Fri, 12 Feb 2010 23:49:38 +0100, Alf P. Steinbach a écrit : The main reason for not using that term for Python is that pass by reference has the extremely strong connotation of being able to implement 'swap'. But 'swap' is so easy to write as a one-line statement that it's foolish to

Re: Please help with MemoryError

2010-02-12 Thread Gregory Ewing
Steven D'Aprano wrote: Python's calling convention already has an well-established name, established over thirty years ago by the distinguished computer scientist Barbara Liskov, namely call-by-sharing. And she was mistaken in thinking it needed a new name. -- Greg --

Re: Please help with MemoryError

2010-02-12 Thread Robert Kern
On 2010-02-12 17:30 PM, Antoine Pitrou wrote: Le Fri, 12 Feb 2010 23:49:38 +0100, Alf P. Steinbach a écrit : The main reason for not using that term for Python is that pass by reference has the extremely strong connotation of being able to implement 'swap'. But 'swap' is so easy to write as

Re: Please help with MemoryError

2010-02-12 Thread Gregory Ewing
Antoine Pitrou wrote: It's just that assignment (=) means a different thing in Python than in non-object languages (or fake-object languages such as C++ or PHP): it rebinds instead of mutating in-place. If it mutated, you wouldn't have the AssertionError. It doesn't really have anything to

Please help with MemoryError

2010-02-11 Thread Jeremy
I have been using Python for several years now and have never run into memory errors… until now. My Python program now consumes over 2 GB of memory and then I get a MemoryError. I know I am reading lots of files into memory, but not 2GB worth. I thought I didn't have to worry about memory

Re: Please help with MemoryError

2010-02-11 Thread Stephen Hansen
On Thu, Feb 11, 2010 at 3:39 PM, Jeremy jlcon...@gmail.com wrote: My Python program now consumes over 2 GB of memory and then I get a MemoryError. I know I am reading lots of files into memory, but not 2GB worth. I thought I didn't have to worry about memory allocation in Python because of

Re: Please help with MemoryError

2010-02-11 Thread Alf P. Steinbach
* Jeremy: I have been using Python for several years now and have never run into memory errors… until now. My Python program now consumes over 2 GB of memory and then I get a MemoryError. I know I am reading lots of files into memory, but not 2GB worth. I thought I didn't have to worry about

Re: Please help with MemoryError

2010-02-11 Thread Jonathan Gardner
On Feb 11, 3:39 pm, Jeremy jlcon...@gmail.com wrote: I have been using Python for several years now and have never run into memory errors… until now. Yes, Python does a good job of making memory errors the least of your worries as a programmer. Maybe it's doing too good of a job... My

Re: Please help with MemoryError

2010-02-11 Thread Steven D'Aprano
On Thu, 11 Feb 2010 15:39:09 -0800, Jeremy wrote: My Python program now consumes over 2 GB of memory and then I get a MemoryError. I know I am reading lots of files into memory, but not 2GB worth. Are you sure? Keep in mind that Python has a comparatively high overhead due to its

Re: Please help with MemoryError

2010-02-11 Thread Tim Chase
Jonathan Gardner wrote: Don't use Python variables to store data long-term. Instead, setup a database or a file and use that. I'd first look at using a file, then using SQLite, and then a full-fledged database like PostgreSQL. Just to add to the mix, I'd put the anydbm module on the gradient

Re: Please help with MemoryError

2010-02-11 Thread Aahz
In article mailman.2418.1265942230.28905.python-l...@python.org, Tim Chase python.l...@tim.thechases.com wrote: Just to add to the mix, I'd put the anydbm module on the gradient between using a file and using sqlite. It's a nice intermediate step between rolling your own file formats for data