Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-08 Thread Craig A. James
Carlos, Now, yet another thing that you (Craig) seem to be missing: you're simply putting the expense of all this time under the expenses column in exchange for solving the particular problem... More like I was trying to keep my response short ;-). I think we're all in agreement on pretty mu

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-08 Thread Carlos Moreno
Ron wrote: Speak Their Language (tm) [ ... ] Do The Right Thing (tm) [...] Not Listening to Reason (tm), [...] fiscal or managerial irresponsibility.) And *here*, of all the instances, you don't put a (TM) sign ?? Tsk-tsk-tsk :-) Carlos -- ---(end of broadc

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-08 Thread Carlos Moreno
I would just like to note here that this is an example of inefficient strategy. [ ... ] Alex may have made the correct, rational choice, given the state of accounting at most corporations. Corporate accounting practices and the budgetary process give different weights to cash and labor.

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-08 Thread Ron
At 01:34 PM 3/8/2007, Craig A. James wrote: Rodrigo Madera wrote: I would just like to note here that this is an example of inefficient strategy. We could all agree (up to a certain economical point) that Alex saved the most expensive one thousand dollars of his life. I don't know the financial

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-08 Thread Craig A. James
Rodrigo Madera wrote: I would just like to note here that this is an example of inefficient strategy. We could all agree (up to a certain economical point) that Alex saved the most expensive one thousand dollars of his life. I don't know the financial status nor the size of your organization

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-08 Thread Rodrigo Madera
I would just like to note here that this is an example of inefficient strategy. We could all agree (up to a certain economical point) that Alex saved the most expensive one thousand dollars of his life. I don't know the financial status nor the size of your organization, but I'm sure that you ha

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-06 Thread Alex Deucher
On 3/6/07, Ron <[EMAIL PROTECTED]> wrote: At 10:25 AM 3/6/2007, Alex Deucher wrote: >On 3/5/07, Guido Neitzer <[EMAIL PROTECTED]> wrote: >>On 05.03.2007, at 19:56, Alex Deucher wrote: >> >> > Yes, I started setting that up this afternoon. I'm going to test that >> > tomorrow and post the results

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-06 Thread Ron
At 10:25 AM 3/6/2007, Alex Deucher wrote: On 3/5/07, Guido Neitzer <[EMAIL PROTECTED]> wrote: On 05.03.2007, at 19:56, Alex Deucher wrote: > Yes, I started setting that up this afternoon. I'm going to test that > tomorrow and post the results. Good - that may or may not give some insight in t

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-06 Thread Alex Deucher
On 3/5/07, Guido Neitzer <[EMAIL PROTECTED]> wrote: On 05.03.2007, at 19:56, Alex Deucher wrote: > Yes, I started setting that up this afternoon. I'm going to test that > tomorrow and post the results. Good - that may or may not give some insight in the actual bottleneck. You never know but it

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-05 Thread Guido Neitzer
On 05.03.2007, at 19:56, Alex Deucher wrote: Yes, I started setting that up this afternoon. I'm going to test that tomorrow and post the results. Good - that may or may not give some insight in the actual bottleneck. You never know but it seems to be one of the easiest to find out ... c

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-05 Thread Alex Deucher
On 3/5/07, Guido Neitzer <[EMAIL PROTECTED]> wrote: On 01.03.2007, at 13:40, Alex Deucher wrote: > I read several places that the SAN might be to blame, but > testing with bonnie and dd indicates that the SAN is actually almost > twice as fast as the scsi discs in the old sun server. I've tried

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-05 Thread Guido Neitzer
On 01.03.2007, at 13:40, Alex Deucher wrote: I read several places that the SAN might be to blame, but testing with bonnie and dd indicates that the SAN is actually almost twice as fast as the scsi discs in the old sun server. I've tried adjusting just about every option in the postgres config

[PERFORM] strange performance regression between 7.4 and 8.1

2007-03-05 Thread Alex Deucher
Hello, I have noticed a strange performance regression and I'm at a loss as to what's happening. We have a fairly large database (~16 GB). The original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB of ram running Solaris on local scsi discs. The new server is a sun Opteron box wi

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Alex Deucher
On 3/2/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Fri, 2 Mar 2007, Guido Neitzer wrote: > On 02.03.2007, at 14:20, Alex Deucher wrote: > >> Ah OK. I see what you are saying; thank you for clarifying. Yes, >> the SAN is configured for maximum capacity; it has large RAID 5 >> groups. As I sa

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Jeff Frost
On Fri, 2 Mar 2007, Guido Neitzer wrote: On 02.03.2007, at 14:20, Alex Deucher wrote: Ah OK. I see what you are saying; thank you for clarifying. Yes, the SAN is configured for maximum capacity; it has large RAID 5 groups. As I said earlier, we never intended to run a DB on the SAN, it jus

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Guido Neitzer
On 02.03.2007, at 14:20, Alex Deucher wrote: Ah OK. I see what you are saying; thank you for clarifying. Yes, the SAN is configured for maximum capacity; it has large RAID 5 groups. As I said earlier, we never intended to run a DB on the SAN, it just happened to come up, hence the configurat

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Alex Deucher
On 3/2/07, Ron <[EMAIL PROTECTED]> wrote: At 02:43 PM 3/2/2007, Alex Deucher wrote: >On 3/2/07, Ron <[EMAIL PROTECTED]> wrote: >> >>...and I still think looking closely at the actual physical layout of >>the tables in the SAN is likely to be worth it. > >How would I go about doing that? > >Alex

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Ron
At 02:43 PM 3/2/2007, Alex Deucher wrote: On 3/2/07, Ron <[EMAIL PROTECTED]> wrote: ...and I still think looking closely at the actual physical layout of the tables in the SAN is likely to be worth it. How would I go about doing that? Alex Hard for me to give specific advice when I don't k

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Alex Deucher
On 3/2/07, Ron <[EMAIL PROTECTED]> wrote: At 11:03 AM 3/2/2007, Alex Deucher wrote: >On 3/2/07, Ron <[EMAIL PROTECTED]> wrote: > >>May I suggest that it is possible that your schema, queries, etc were >>all optimized for pg 7.x running on the old HW? >>(explain analyze shows the old system taking

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Alvaro Herrera
Florian Weimer escribió: > Locale settings make a huge difference for sorting and LIKE queries. > We usually use the C locale and SQL_ASCII encoding, mostly for > performance reasons. (Proper UTF-8 can be enforced through > constraints if necessary.) Hmm, you are aware of varchar_pattern_ops and

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Anton Rommerskirchen
Am Donnerstag 01 März 2007 21:44 schrieb Alex Deucher: > Hello, > > I have noticed a strange performance regression and I'm at a loss as > to what's happening. We have a fairly large database (~16 GB). The > original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB > of ram running Sol

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Ron
At 11:03 AM 3/2/2007, Alex Deucher wrote: On 3/2/07, Ron <[EMAIL PROTECTED]> wrote: May I suggest that it is possible that your schema, queries, etc were all optimized for pg 7.x running on the old HW? (explain analyze shows the old system taking ~1/10 the time per row as well as estimating the

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Scott Marlowe
On Fri, 2007-03-02 at 10:03, Alex Deucher wrote: > On 3/2/07, Ron <[EMAIL PROTECTED]> wrote: > > At 10:16 AM 3/2/2007, Alex Deucher wrote: > > d= you went from local HD IO to a SAN > > (many differences hidden in that one line... ...and is the physical > > layout of tables and things like pg_xlo

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Alex Deucher
On 3/2/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Alex Deucher" <[EMAIL PROTECTED]> writes: > Anyway, new numbers after the analyze. > Unfortunately, they are improved, but still not great: Why are the index names different between the old and new servers? Is that just cosmetic, or is 8.2 actually

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Tom Lane
"Alex Deucher" <[EMAIL PROTECTED]> writes: > Anyway, new numbers after the analyze. > Unfortunately, they are improved, but still not great: Why are the index names different between the old and new servers? Is that just cosmetic, or is 8.2 actually picking a different (and less suitable) index fo

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Alex Deucher
On 3/2/07, Ron <[EMAIL PROTECTED]> wrote: At 10:16 AM 3/2/2007, Alex Deucher wrote: >On 3/2/07, Florian Weimer <[EMAIL PROTECTED]> wrote: >>* Alex Deucher: >> >> > I have noticed a strange performance regression and I'm at a loss as >> > to what's happening. We have a fairly large database (~16

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Ron
At 10:16 AM 3/2/2007, Alex Deucher wrote: On 3/2/07, Florian Weimer <[EMAIL PROTECTED]> wrote: * Alex Deucher: > I have noticed a strange performance regression and I'm at a loss as > to what's happening. We have a fairly large database (~16 GB). Sorry for asking, but is this a typo? Do you

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: > here are some examples. Analyze is still running on the new db, I'll > post results when that is done. Mostly what our apps do is prepared > row selects from different tables: > select c1,c2,c3,c4,c5 fro

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Ron
At 08:56 AM 3/2/2007, Carlos Moreno wrote: Florian Weimer wrote: * Alex Deucher: I have noticed a strange performance regression and I'm at a loss as to what's happening. We have a fairly large database (~16 GB). Sorry for asking, but is this a typo? Do you mean 16 *TB* instead of 16 *GB

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Alex Deucher
On 3/2/07, Florian Weimer <[EMAIL PROTECTED]> wrote: * Alex Deucher: > I have noticed a strange performance regression and I'm at a loss as > to what's happening. We have a fairly large database (~16 GB). Sorry for asking, but is this a typo? Do you mean 16 *TB* instead of 16 *GB*? If it's r

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Carlos Moreno
Florian Weimer wrote: * Alex Deucher: I have noticed a strange performance regression and I'm at a loss as to what's happening. We have a fairly large database (~16 GB). Sorry for asking, but is this a typo? Do you mean 16 *TB* instead of 16 *GB*? If it's really 16 GB, you should c

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-02 Thread Florian Weimer
* Alex Deucher: > I have noticed a strange performance regression and I'm at a loss as > to what's happening. We have a fairly large database (~16 GB). Sorry for asking, but is this a typo? Do you mean 16 *TB* instead of 16 *GB*? If it's really 16 GB, you should check if it's cheaper to buy mo

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: Alex Deucher wrote: > On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: >> \ >> >> Is the SAN being shared between the database servers and other >> >> servers? Maybe >> >> it was just random timing that gave you the poor write performanc

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Joshua D. Drake
Alex Deucher wrote: > On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: >> \ >> >> Is the SAN being shared between the database servers and other >> >> servers? Maybe >> >> it was just random timing that gave you the poor write performance on >> >> the old >> >> server which might be also yiel

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: > On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: >> On Thu, 1 Mar 2007, Alex Deucher wrote: >> >> >> >> Postgresql might be choosing a bad plan because your >> >> effective_cache_size >> >> >> is >> >> >>

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> >> Postgresql might be choosing a bad plan because your >> effective_cache_size >> >> is >> >> way off (it's the default now right?). Also, what was the block >>

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: \ >> Is the SAN being shared between the database servers and other >> servers? Maybe >> it was just random timing that gave you the poor write performance on >> the old >> server which might be also yielding occassional poor performance on >

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Joshua D. Drake
\ >> Is the SAN being shared between the database servers and other >> servers? Maybe >> it was just random timing that gave you the poor write performance on >> the old >> server which might be also yielding occassional poor performance on >> the new >> one. >> > > The direct attached scsi discs

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> >> Postgresql might be choosing a bad plan because your >> effective_cache_size >> >> is >> >> way off (it's the default now right?). Also, what was the block >> read/write >> > >> > yes it's set to the

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Ron
At 07:36 PM 3/1/2007, Jeff Frost wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> Postgresql might be choosing a bad plan because your effective_cache_size >> is >> way off (it's the default now right?). Also, what was the block read/write > > yes it's set to the default. > >> speed of the

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: >> Postgresql might be choosing a bad plan because your effective_cache_size >> is >> way off (it's the default now right?). Also, what was the block read/write > > yes it's set to the default. > >> speed of the SAN from your bonnie tests? Probably wa

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: here are some examples. Analyze is still running on the new db, I'll post results when that is done. Mostly what our apps do is prepared row selects from different tables: select c1,c2,c3,c4,c5 from t1 where c1='XXX'; old server: db=# EXPLAIN ANALYZE se

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Joshua D. Drake wrote: > Alex Deucher wrote: >> Hello, >> >> I have noticed a strange performance regression and I'm at a loss as >> to what's happening. We have a fairly large database (~16 GB). The >> original postgres 7.4

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: > On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: >> On Thu, 1 Mar 2007, Alex Deucher wrote: >> >> >> Vacuum? Analayze? default_statistics_target? How many shared_buffers? >> >> effective_cache_size? work_m

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> Vacuum? Analayze? default_statistics_target? How many shared_buffers? >> effective_cache_size? work_mem? >> > > I'm running the autovacuum process on the 8.1 serv

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> Vacuum? Analayze? default_statistics_target? How many shared_buffers? >> effective_cache_size? work_mem? >> > > I'm running the autovacuum process on the 8.1 server. vacuuming on > the old server was don

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: Vacuum? Analayze? default_statistics_target? How many shared_buffers? effective_cache_size? work_mem? I'm running the autovacuum process on the 8.1 server. vacuuming on the old server was done manually. default_statistics_target and effective_cache_si

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Joshua D. Drake wrote: > Alex Deucher wrote: >> Hello, >> >> I have noticed a strange performance regression and I'm at a loss as >> to what's happening. We have a fairly large databas

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Joshua D. Drake wrote: > Alex Deucher wrote: >> Hello, >> >> I have noticed a strange performance regression and I'm at a loss as >> to what's happening. We have a fairly large database (~16 GB). The >> original postgres 7.4

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: Alex Deucher wrote: > Hello, > > I have noticed a strange performance regression and I'm at a loss as > to what's happening. We have a fairly large database (~16 GB). The > original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Joshua D. Drake wrote: Alex Deucher wrote: Hello, I have noticed a strange performance regression and I'm at a loss as to what's happening. We have a fairly large database (~16 GB). The original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB of ram running So

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Joshua D. Drake
Alex Deucher wrote: > Hello, > > I have noticed a strange performance regression and I'm at a loss as > to what's happening. We have a fairly large database (~16 GB). The > original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB > of ram running Solaris on local scsi discs. The new

[PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
Hello, I have noticed a strange performance regression and I'm at a loss as to what's happening. We have a fairly large database (~16 GB). The original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB of ram running Solaris on local scsi discs. The new server is a sun Opteron box wi