Re: Possible memory leak?

The reason it's done in width and heigth is that there is some other non-related processing functions that were going on in the mean time with the problem. I found the source of the slow-down, when 2 non-important lines of code were commented out, the program worked perfectly.

Re: Possible memory leak?

In <[EMAIL PROTECTED]>, Tuvas wrote: > The modified version of my code is now as follows: (Note, a few small > changes have been made to simplify things, however, these things don't > apply to a full-scale picture, so the shouldn't slow anything down in > the slightest.) > > def load_pic_data(wid

Re: Possible memory leak?

I have made one confirmation. The only identifiable difference that I have seen is that one runs on python 2.4.2, and the other 2.4.1. Oddly enough, it's the first one that's the one that is having more problems than the second... Why that is, I don't know. It still could be something else, but...

Re: Possible memory leak?

The times that I posted was the time that it took to perform ONE row iteration. As you can see, the time was going up, fairly dramatically. Why on earth could it be doing this? I understand the the time will fluctuate somewhat depending upon what else the CPU is doing, but, why is the base time inc

Re: Possible memory leak?

Steven D'Aprano wrote: > > Of course. I was just trying to make a point about string accumulation being > > O(n) and not O(n^2). > > But according to Fredrik, string accumulation is still quadratic, even > with the optimizations added to Python 2.4. Quoting: > > "it only means that if the interpre

Re: Possible memory leak?

On Wed, 25 Jan 2006 20:08:56 +0100, Giovanni Bajo wrote: > Steven D'Aprano wrote: > >> No, that's not correct. You are making a false >> assumption. > > Did you ever try to measure the code yourself? I'm still running Python 2.3, it would give misleading results. >> This is from the "What's N

Re: Possible memory leak?

On Wed, 25 Jan 2006 23:20:08 +, Giovanni Bajo wrote: > Of course. I was just trying to make a point about string accumulation being > O(n) and not O(n^2). But according to Fredrik, string accumulation is still quadratic, even with the optimizations added to Python 2.4. Quoting: "it only mea

Re: Possible memory leak?

[Fredrik Lundh] >> ... >> for the OP's problem, a PIL-based solution would probably be ~100 >> times faster than the array solution, but that's another story. [Tuvas] > What do you mean by a PIL based solution? The reason I need to get the > data into the string list is so I can pump it into PIL t

Re: Possible memory leak?

On Wed, 25 Jan 2006 13:17:25 -0800, Tuvas wrote: > Very interesting results with the last test. I guess I go back to my > other code then, even if it is only a hair faster, it's still faster... Can you say "premature optimization"? Have you timed your code with realistic data? Not somebody else'

Re: Possible memory leak?

Fredrik Lundh wrote: > since you know the length, you can preallocate the list > > def iters3(n): > L = [None]*n > for i in xrange(n): > L[i] = chr(i%64) > return "".join(L) > > or use a preallocated array > > def iters4(n): > L = array.array("B"

Re: Possible memory leak?

Fredrik Lundh wrote: > Giovanni Bajo wrote: > > > --- foo.py - > > def iters(n): > > s = '' > > for i in xrange(n): > > s += chr(i%64) > > return s > > > > def iters2(n): > > L = [] > > for i in xrange(n): > > L.append(chr(i%64)) > > return "".join(L

Re: Possible memory leak?

Giovanni Bajo wrote: > --- foo.py - > def iters(n): > s = '' > for i in xrange(n): > s += chr(i%64) > return s > > def iters2(n): > L = [] > for i in xrange(n): > L.append(chr(i%64)) > return "".join(L) > --- foo.py - > > So, look, it's even

Re: Possible memory leak?

Very interesting results with the last test. I guess I go back to my other code then, even if it is only a hair faster, it's still faster... It's odd, I just ran another test. There's 2 ways I can call my load_pic function, first of all, through taking a picture, secondly by loading a picture. Fo

Re: Possible memory leak?

Steven D'Aprano wrote: > No, that's not correct. You are making a false > assumption. Did you ever try to measure the code yourself? > This is from the "What's New" for Python 2.4: > > [quote] > String concatenations in statements of the form s = s + > "abc" and s += "abc" are now performed more

Re: Possible memory leak?

FYI, to all who asked, I was indeed just simply monitering the system memory. I changed my approach to one that uses arrays and simply joins the statements together at the end, it seems to have improved the speed. However, for some reason it still takes a small eternity to process on one computer,

Re: Possible memory leak?

Steven D'Aprano wrote: > > Steven D'Aprano wrote: > > > > > >>But the real killer is this one line: > >> > >>row=row+chr(num/64) > >> > >>Bad, bad BAD idea. Every time you add two strings together, Python > >>has to copy BOTH strings. As row gets huge, this takes longer and > >>longer to do. > > >

Re: Possible memory leak?

Replying to myself... the first step towards perdition... Steven D'Aprano wrote: > We really don't know what the optimization recognises, how it works, or > how fast a difference it makes. Of course, by "we" I mean "those of us who haven't seen and understood the CPython source code, or run de

Re: Possible memory leak?

Giovanni Bajo wrote: > Steven D'Aprano wrote: > > >>But the real killer is this one line: >> >>row=row+chr(num/64) >> >>Bad, bad BAD idea. Every time you add two strings together, Python >>has to copy BOTH strings. As row gets huge, this takes longer and >>longer to do. > > > This is no longer

Re: Possible memory leak?

Steven D'Aprano wrote: > But the real killer is this one line: > > row=row+chr(num/64) > > Bad, bad BAD idea. Every time you add two strings together, Python > has to copy BOTH strings. As row gets huge, this takes longer and > longer to do. This is no longer true as of CPython 2.4 though. I'm no

Re: Possible memory leak?

Tuvas wrote: > However, I would like to make a few more statements. The max image size > is 1024x1024. The length of time it takes to do the row statement > increased fairly substationally every time it was ran, in the order of > ~.5%. That seems small, but when you run it 1M times, the last > ope

Re: Possible memory leak?

Tuvas wrote: > Oh, I should also mention, I used a memory monitor and saw the amount > of memory being used go up with time, even when the function ended, > meaning I did the 10 128x128 pictures, never was any memory dealocated > until I exited the program. Are you really trying to say that the a

Re: Possible memory leak?

Oh, I should also mention, I used a memory monitor and saw the amount of memory being used go up with time, even when the function ended, meaning I did the 10 128x128 pictures, never was any memory dealocated until I exited the program. -- http://mail.python.org/mailman/listinfo/python-list

Re: Possible memory leak?

Hmm. The problem is that the required form for the image is as a string of characters to convert with the tkimage interface, at least, as I understood it. Perhaps an array would work, I'll try that when I get ahold of the computer in question (One thing required is a linux only library, and I don't

Re: Possible memory leak?

Tuvas wrote: > I have a function in a program that works something like this. > > def load_pic_data(width,heigth,inpdat, filt=TRUE): > data='' > total=0 > tnum=0 > size=100 > for y in range(0,heigth): > row='' > for x in range

Re: Possible memory leak?

In message <[EMAIL PROTECTED]>, Steven D'Aprano <[EMAIL PROTECTED]> writes >But the real killer is this one line: > >row=row+chr(num/64) > >Bad, bad BAD idea. Every time you add two strings together, Python has to >copy BOTH strings. As row gets huge, this takes longer and longer to do. > >A rule o

Re: Possible memory leak?

Paul Rubin writes: > Probably better to use StringIO or the array module. That's cStringIO of course. -- http://mail.python.org/mailman/listinfo/python-list

Re: Possible memory leak?

Steven D'Aprano <[EMAIL PROTECTED]> writes: > row = [] > # processing in a loop... > row.append(chr(num/64) > # finished processing > row = "".join(row) # convert to a string in one hit > print row Probably better to use StringIO or the array module. -- http://mail.python.org/mailman/listinfo/py

Re: Possible memory leak?

On Tue, 24 Jan 2006 13:51:36 -0800, Tuvas wrote: > The purpose of this part of a program is to take a 14 bit numerical > representation and convert it to an 8 bit representation. This will > later be displayed as an image. However, I've noticed the following > about this code. I was noticing when

Re: Possible memory leak?

Tuvas wrote: > I have a function in a program that works something like this. > > def load_pic_data(width,heigth,inpdat, filt=TRUE): > data='' > total=0 > tnum=0 > size=100 > for y in range(0,heigth): > row='' > for x in range

Possible memory leak?

I have a function in a program that works something like this. def load_pic_data(width,heigth,inpdat, filt=TRUE): data='' total=0 tnum=0 size=100 for y in range(0,heigth): row='' for x in range(0,width):