Thanks for the tips and input. As always, your suggestions are appreciated by many more than just myself.
Gil > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Mike yearwood > Sent: Wednesday, April 02, 2008 11:59 AM > To: [EMAIL PROTECTED] > Subject: RE: [NF] RE: Microsoft New SQL OS >>> PostgreSQL info > > > Hi Gil > > > Message: 7 > > Date: Wed, 2 Apr 2008 10:13:07 -0400 > > From: "Gil Hale" <[EMAIL PROTECTED]> > > Subject: > To: [EMAIL PROTECTED] > > Message-ID: <[EMAIL PROTECTED]> > > Content-Type: text/plain; charset="us-ascii" > > > > SNIP!!! > > >I have no recorded time comparisons, just my gut feel based on years > of watching VFP import records from large csv files (millions of > records at a crack). And, for the record, when I do my imports into a > VFP table I leave any index/order set off, as having a table with an > active index/order will slow down an import process horribly with > large tables as VFP tries to rearrange imported records into the > active sort order... > > > > I don't think that's 100% accurate. If you have SET ORDER TO 0, but > still have a CDX, the cdx tags must be updated. If you have SET ORDER > TO something, I don't believe that slows down the APPEND FROM. - Can't > test it right now though. > > Regardless - here's something to consider. I hope you can apply this > to PostgreSQL. When importing large tables in I first APPEND the > source into a temporary table first, then do two things. Note the ORs > in the update command. In VFP they stop evaluating once the first one > is met. > > UPDATE t ; > SET ; > t.field1 = s.field1, ; > t.field2 = s.field2, ; > t.field3 = s.field3 ; > FROM ; > target t ; > inner join source s ; > on t.pk = s.pk ; > WHERE ; > t.field1 # s.field1 ; > OR t.field2 # s.field2 ; > OR t.field3 # s.field3 > > INSERT INTO ; > target ; > (; > field1,; > field2,; > field3) ; > SELECT ; > field1,; > field2,; > field3 ; > FROM ; > source s ; > WHERE ; > s.pk NOT IN ; > (SELECT pk FROM target) > > This works very well in VFP, because I don't update any indexes doing > the initial import. Then I only update few records (and their indexes) > during the update and append processes. If you could do those commands > directly with the csv files, that might be pretty good. > > Mike Yearwood > > [excessive quoting removed by server] _______________________________________________ Post Messages to: [email protected] Subscription Maintenance: http://leafe.com/mailman/listinfo/profox OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech Searchable Archive: http://leafe.com/archives/search/profox This message: http://leafe.com/archives/byMID/profox/[EMAIL PROTECTED] ** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
