Re: [PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-26 Thread CoL
hi, Sean Shanny wrote, On 4/22/2004 23:56: SELECT t1.id, t2.url FROM referral_temp t2 LEFT OUTER JOIN d_referral t1 ON t2.url = t1.referral_raw_url ORDER BY t1.id index on url (text) has no sense. Try to use and md5 (char(32) column) which contains the md5 hash of url field. and join these ones.

Re: [PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-23 Thread scott.marlowe
On Thu, 22 Apr 2004, Sean Shanny wrote: > I should have included this as well: > fsync | on > shared_buffers | 4000 > sort_mem | 64000 For purposes of loading only, you can try turning off fsync, assuming this is a virgin load an

Re: [PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-23 Thread Aaron Werman
t; <[EMAIL PROTECTED]> To: "Sean Shanny" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Friday, April 23, 2004 12:38 AM Subject: Re: [PERFORM] Looking for ideas on how to speed up warehouse loading > Sean Shanny wrote: > > explain analyze SELECT t1.id, t2.url FROM

Re: [PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-22 Thread Joe Conway
Sean Shanny wrote: explain analyze SELECT t1.id, t2.url FROM referral_temp t2 LEFT OUTER JOIN d_referral t1 ON t2.url = t1.referral_raw_url ORDER BY t1.id; What I would like to know is if there are better ways to do the join? I need to get all the rows back from the referral_temp table as they

Re: [PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-22 Thread Tom Lane
Sean Shanny <[EMAIL PROTECTED]> writes: > explain analyze SELECT t1.id, t2.url FROM referral_temp t2 LEFT OUTER > JOIN d_referral t1 ON t2.url = t1.referral_raw_url ORDER BY t1.id; > QUERY PLAN > ---

Re: [PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-22 Thread Nicholas Shanny
One other thing: we are running with a block size of 32K. Nick Shanny (Brother of above person) On Apr 22, 2004, at 7:30 PM, Sean Shanny wrote: I should have included this as well: show all; name |setting + add_missing

Re: [PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-22 Thread Sean Shanny
I should have included this as well: show all; name |setting + add_missing_from | on australian_timezones | off authentication_timeout | 60 check_function_bodies | on checkpoint_s

[PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-22 Thread Sean Shanny
To all, Essentials: Running 7.4.1 on OSX on a loaded G5 with dual procs, 8GB memory, direct attached via fibre channel to a fully optioned 3.5TB XRaid (14 spindles, 2 sets of 7 in RAID 5) box running RAID 50. Background: We are loading what are essentially xml based access logs from about 20