Jeremy,
How did you get the string in memory in the first place?
If you read it from a file, perhaps you should change to
reading it from the file a line at the time and use
file.readline as your iterator.
fp=file(inputfile, 'r')
for line in fp:
...do your processing...
fp.close()
I don't think I would never read 400,000 lines as a single
string and then split it. Just a suggestion.
Larry Bates
Jeremy Sanders wrote:
> I have a large string containing lines of text separated by '\n'. I'm
> currently using text.splitlines(True) to break the text into lines, and
> I'm iterating over the resulting list.
>
> This is very slow (when using 400000 lines!). Other than dumping the
> string to a file, and reading it back using the file iterator, is there a
> way to quickly iterate over the lines?
>
> I tried using newpos=text.find('\n', pos), and returning the chopped text
> text[pos:newpos+1], but this is much slower than splitlines.
>
> Any ideas?
>
> Thanks
>
> Jeremy
>
--
http://mail.python.org/mailman/listinfo/python-list
