Re: [HACKERS] raising the default default_statistics_target

2004-03-09 Thread Tom Lane
"scott.marlowe" <[EMAIL PROTECTED]> writes: > Hi Tom. I ran some very simple tests on analyze times and query plan > times on a very simple table, with data randomly distributed. The index > was on a date field, since that's what I was testing last. Thanks. > I also ran some quick tests on sm

Re: [HACKERS] raising the default default_statistics_target

2004-03-09 Thread scott.marlowe
On Sun, 7 Mar 2004, Tom Lane wrote: > Neil Conway <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> This is something we need to consider, but we'll need more evidence > >> before making a choice. One thing that we have very little data about > >> is how much difference it makes in the quality

Re: [HACKERS] raising the default default_statistics_target

2004-03-09 Thread Josh Berkus
Guys, > Hm, that had nothing to do with multi-column correlation though. > I'm at a loss to think of any work that matches with Josh's > recollection. H it's possible that early e-mails about Manfred's patch claimed to improve performance for multi-column indexes. But it's also pos

Re: [HACKERS] raising the default default_statistics_target

2004-03-09 Thread Tom Lane
Robert Treat <[EMAIL PROTECTED]> writes: > I think the thread you're thinking of is on or about this post: > http://archives.postgresql.org/pgsql-patches/2003-12/msg00039.php > Manfred Koizar produced a patch that modified index correlation by > sorting equal key values based on item pointers. The

Re: [HACKERS] raising the default default_statistics_target

2004-03-09 Thread Robert Treat
I think the thread you're thinking of is on or about this post: http://archives.postgresql.org/pgsql-patches/2003-12/msg00039.php Manfred Koizar produced a patch that modified index correlation by sorting equal key values based on item pointers. The patch went as far as getting accepted into the p

Re: [HACKERS] raising the default default_statistics_target

2004-03-08 Thread Josh Berkus
Tom, > Are you sure you're not thinking of stats for functional indexes? Positive.I even remember seeing that the patch was accepted. The patch specifically had to do with a multi-column correlation algorithm for improving the selectivity of multi-column indexes. Problem is, with 1400 post

Re: [HACKERS] raising the default default_statistics_target

2004-03-08 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Further, in 7.5 we'll be introducing correlated stats for multi-column > indexes (unless something's gone off with that?) > This was discussed on Hackers in October, a complete implementation was shown, > I thought it was committed at that time. If not

Re: [HACKERS] raising the default default_statistics_target

2004-03-08 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Do all the columns have to have the same number of statistics buckets? > > They do not, but the effort spent by ANALYZE is proportional to the > largest stats target among all the columns of the table. Could we use previous stats to

Re: [HACKERS] raising the default default_statistics_target

2004-03-08 Thread Josh Berkus
Tom, > > Further, in 7.5 we'll be introducing correlated stats for multi-column indexes > > (unless something's gone off with that?) This was discussed on Hackers in October, a complete implementation was shown, I thought it was committed at that time. If not, what happened to it? Dammit, i

Re: [HACKERS] raising the default default_statistics_target

2004-03-08 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Do all the columns have to have the same number of statistics buckets? They do not, but the effort spent by ANALYZE is proportional to the largest stats target among all the columns of the table. regards, tom lane -

Re: [HACKERS] raising the default default_statistics_target

2004-03-08 Thread Bruce Momjian
Tom Lane wrote: > Josh Berkus <[EMAIL PROTECTED]> writes: > > It is. I've found that "problem" queries, especially those caused by real, > > uneven distribution of data, require raising statistics to 150-400 in order > > to fix. This is much to high a level to assign as a default. > > That's

Re: [HACKERS] raising the default default_statistics_target

2004-03-08 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > It is. I've found that "problem" queries, especially those caused by real, > uneven distribution of data, require raising statistics to 150-400 in order > to fix. This is much to high a level to assign as a default. That's basically what's bothering m

Re: [HACKERS] raising the default default_statistics_target

2004-03-08 Thread Josh Berkus
Neil, > In the simple test I performed, raising the default_statistics_target > from 10 to 25 resulted in a 40% increase in the time to ANALYZE a > large table. (I picked 25 more or less at random -- would 15 or 20 be > better?) I find that very interesting, since I haven't found much higher i

Re: [HACKERS] raising the default default_statistics_target

2004-03-07 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> This is something we need to consider, but we'll need more evidence >> before making a choice. One thing that we have very little data about >> is how much difference it makes in the quality of planner choices. > Right, but is there a p

Re: [HACKERS] raising the default default_statistics_target

2004-03-07 Thread Neil Conway
Tom Lane wrote: This is something we need to consider, but we'll need more evidence before making a choice. One thing that we have very little data about is how much difference it makes in the quality of planner choices. Right, but is there a practical way to actually get this data? If the distri

Re: [HACKERS] raising the default default_statistics_target

2004-03-07 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > Any comments on whether increasing the default stats target is a good > idea for 7.5? (Details on the test I performed are included below) This is something we need to consider, but we'll need more evidence before making a choice. One thing that we have

[HACKERS] raising the default default_statistics_target

2004-03-07 Thread Neil Conway
From time to time, people on IRC ask for help with performance problems, and the cause of the difficulty is ultimately traced to a poor query plan that is chosen because default_statistics_target is too low. While there will always need to be *some* tuning of the statistics target by advanced u