Merge tables aren't an easy option here. There's a higher level data partitioning that we're putting into place, though, and it's been shown to help a lot. But I also want to get as much out of the file system as I can.
> The "solution" I use for duplicate filtering is to to create a > "load" table which loads a set of records, then compares those > records against the merge table for duplicates, deleting any found. > Then the load table is added to the merge table and the process is > repeated for the next batch of data. I don't think this will help us, but it's an interesting technique. We use staging tables to cut the load in a bunch of places. I think the true answer to this particular problem lies outside SQL and instead with a private index structure that is tuned for dealing with duplicates...it would help if the MyISAM engine was a little more clever about really large indexes. Best, chris -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
