On Jan 21, 4:45 pm, Paul Rubin <http://[EMAIL PROTECTED]> wrote: > "p." <[EMAIL PROTECTED]> writes: > > 1. Has anyone done anything like this before, and if so, do you have > > any advice? > > Sort all the files with an external sort utility (e.g. unix sort), so > that records with the same key are all brought together. Then you can > process the files sequentially.
Seconded. Unix sort can do external sorting [1] so your program will work even if the files don't fit in memory. Once they are sorted, itertools (especially groupby) is your friend. George [1] http://en.wikipedia.org/wiki/External_sort -- http://mail.python.org/mailman/listinfo/python-list
