On 6/18/2010 7:01 PM, Scott Hess wrote: > > The old-school solution to this problem is an external sort (*). The > basic idea is that you process the incoming data in memory-sized > chunks (ie, fast), then write out sorted subfiles. Then you process > the sorted files in parallel, merging to the output. >
The other old-school solution involving preprocessing is block sorting or binning. You go through the data once and create a bunch of files of unsorted but grouped data. Then insert the data from the files in order. If the files are transaction-sized, each one will be adding to a fairly small range of values. Gerry _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
