On Thu, 11 Oct 2007 07:02:10 +0000, thebjorn wrote: > On Oct 11, 8:53 am, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote: >> Even if `str.join()` would not convert the generator into a list first, >> you would have overallocation. You don't know the final string size >> beforehand so intermediate strings must get moved around in memory while >> concatenating. Worst case: all but the last string are already >> concatenated and the last one does not fit into the allocated memory >> anymore, so there is new memory allocates that can hold both strings -> >> double amount of memory needed. > > Perhaps realloc() could be used to avoid this? I'm guessing that's > what cStringIO does, although I'm too lazy to check (I don't have > source on this box). Perhaps a cStringIO.getvalue() implementation > that doesn't copy memory would solve the problem?
How could `realloc()` solve that problem? Doesn't `realloc()` copy the memory too if the current memory block can't hold the new size!? And `StringIO` has the very same problem, if the `getvalue()` method doesn't copy you have to make copies while writing to the `StringIO` object and the buffer is not large enough. Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list