Re: Python garbage collector/memory manager behaving strangely

2012-11-15 Thread Thomas Rachel
Am 17.09.2012 04:28 schrieb Jadhav, Alok: Thanks Dave for clean explanation. I clearly understand what is going on now. I still need some suggestions from you on this. There are 2 reasons why I was using self.rawfile.read().split('|\n') instead of self.rawfile.readlines() - As you have seen, t

Re: Python garbage collector/memory manager behaving strangely

2012-11-14 Thread Dieter Maurer
a...@pythoncraft.com (Aahz) writes: > ... def readlines(f): lines = [] while "f is not empty": line = f.readline() if not line: break if len(line) > 2 and line[-2:] == '|\n': lines.append(lin

Re: Python garbage collector/memory manager behaving strangely

2012-11-14 Thread Aahz
In article <50570de3$0$29981$c3e8da3$54964...@news.astraweb.com>, Steven D'Aprano wrote: >On Mon, 17 Sep 2012 06:46:55 -0400, Dave Angel wrote: >> On 09/16/2012 11:25 PM, alex23 wrote: >>> >>> def readlines(f): >>> lines = [] >>> while "f is not empty": >>> line =

Re: Python garbage collector/memory manager behaving strangely

2012-10-31 Thread Robert Miles
On 9/16/2012 9:12 PM, Dave Angel wrote: On 09/16/2012 09:07 PM, Jadhav, Alok wrote: Hi Everyone, I have a simple program which reads a large file containing few million rows, parses each row (`numpy array`) and converts into an array of doubles (`python array`) and later writes into an `hdf5

Re: Python garbage collector/memory manager behaving strangely

2012-09-17 Thread Dave Angel
On 09/17/2012 07:47 AM, Steven D'Aprano wrote: > On Mon, 17 Sep 2012 06:46:55 -0400, Dave Angel wrote: > >> On 09/16/2012 11:25 PM, alex23 wrote: >>> def readlines(f): >>> lines = [] >>> while "f is not empty": >>> line = f.readline() >>> if not line: bre

Re: Python garbage collector/memory manager behaving strangely

2012-09-17 Thread Steven D'Aprano
On Mon, 17 Sep 2012 06:46:55 -0400, Dave Angel wrote: > On 09/16/2012 11:25 PM, alex23 wrote: >> def readlines(f): >> lines = [] >> while "f is not empty": >> line = f.readline() >> if not line: break >> if len(line) > 2 and line[-2:] == '|\n

RE: Python garbage collector/memory manager behaving strangely

2012-09-17 Thread Jadhav, Alok
garbage collector/memory manager behaving strangely On 09/16/2012 11:25 PM, alex23 wrote: > On Sep 17, 12:32 pm, "Jadhav, Alok" > wrote: >> - As you have seen, the line separator is not '\n' but its '|\n'. >> Sometimes the data itself has '\n

Re: Python garbage collector/memory manager behaving strangely

2012-09-17 Thread Dave Angel
On 09/16/2012 11:25 PM, alex23 wrote: > On Sep 17, 12:32 pm, "Jadhav, Alok" > wrote: >> - As you have seen, the line separator is not '\n' but its '|\n'. >> Sometimes the data itself has '\n' characters in the middle of the line >> and only way to find true end of the line is that previous charact

Re: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread 88888 Dihedral
alex23於 2012年9月17日星期一UTC+8上午11時25分06秒寫道: > On Sep 17, 12:32 pm, "Jadhav, Alok" > > wrote: > > > - As you have seen, the line separator is not '\n' but its '|\n'. > > > Sometimes the data itself has '\n' characters in the middle of the line > > > and only way to find true end of the line is tha

Re: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread alex23
On Sep 17, 12:32 pm, "Jadhav, Alok" wrote: > - As you have seen, the line separator is not '\n' but its '|\n'. > Sometimes the data itself has '\n' characters in the middle of the line > and only way to find true end of the line is that previous character > should be a bar '|'. I was not able spec

RE: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok
Angel [mailto:d...@davea.name] Sent: Monday, September 17, 2012 10:13 AM To: Jadhav, Alok Cc: python-list@python.org Subject: Re: Python garbage collector/memory manager behaving strangely On 09/16/2012 09:07 PM, Jadhav, Alok wrote: > Hi Everyone, > > > > I have a simple progra

RE: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok
ete file was reduced to one third of original time. Regards, Alok -Original Message- From: Dave Angel [mailto:d...@davea.name] Sent: Monday, September 17, 2012 10:13 AM To: Jadhav, Alok Cc: python-list@python.org Subject: Re: Python garbage collector/memory manager behaving strangel

Re: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Dave Angel
On 09/16/2012 09:07 PM, Jadhav, Alok wrote: > Hi Everyone, > > > > I have a simple program which reads a large file containing few million > rows, parses each row (`numpy array`) and converts into an array of > doubles (`python array`) and later writes into an `hdf5 file`. I repeat > this loop fo

Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok
Hi Everyone, I have a simple program which reads a large file containing few million rows, parses each row (`numpy array`) and converts into an array of doubles (`python array`) and later writes into an `hdf5 file`. I repeat this loop for multiple days. After reading each file, i delete all the