Re: [HACKERS] Additional stats for Relations

2006-10-13 Thread NikhilS
Hi Jim, On 10/13/06, Jim C. Nasby <[EMAIL PROTECTED]> wrote: On Fri, Oct 13, 2006 at 06:17:47PM +0530, NikhilS wrote:> Currently a "select * from pg_statio_user_tables;" displays only > heap_blks_read, heap_blks_hit stats amongst others for the main relation. It> would be good to have the following

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Let me clarify, because that's not what I meant. Right now, there's not > even a shadow of a design for anything else, and this is a tough nut to > crack. I think you are not exactly measuring on a level playing field. On the textually-embedded-hints s

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Alvaro Herrera
Jim C. Nasby wrote: > So, does anyone out there have a plan for how we could give user's the > ability to control the planner at a per-table level in 8.3 or even 8.4? Per-table level? Some of the problems that have been put forward have to do with table combinations (for example selectivity of j

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 03:57:23PM -0700, Josh Berkus wrote: > Jim, > > > Well, that's not what I said (my point being that until the planner and > > stats are perfect you need a way to over-ride them)... but I've also > > never said hints would be faster or easier than stats modification (I > > s

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Josh Berkus
Csaba, > And upgrades are always causing breakage, I didn't have one upgrade > without some things to fix, so I would expect people is expecting that. > And that's true for Oracle too, our oracle guys always have something to > fix after an upgrade. And I repeat, I always had something to fix for

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Josh Berkus
Andreas, > I think we need to more precisely define the problems of our system with > point in time statistics > > -- no reaction to degree of other concurrent activity > -- no way to react to abnormal skew that only persists for a very short > duration > -- too late reaction to changing distribut

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Josh Berkus
Jim, > Well, that's not what I said (my point being that until the planner and > stats are perfect you need a way to over-ride them)... but I've also > never said hints would be faster or easier than stats modification (I > said I hope they would). Yes, you did. Repeatedly. On this and other th

Re: [HACKERS] [PATCHES] index advisor

2006-10-13 Thread Andrew Dunstan
Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> writes: >> Kai-Uwe Sattler wrote: >>> here is a patch implementing an index recommendation advisor for >>> 7.4.13. > >> You know we don't put new features on stable branches, right? > > I think you forgot the context --- somebody asked Kai-Uwe t

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> ... Is there any sort >> of "quiet mode" possible that would report only warnings? > [ it's already there ] So maybe we just ought to tweak the installation instructions to recommend use of --quiet? regards

Re: [HACKERS] [GENERAL] more anti-postgresql FUD

2006-10-13 Thread Dann Corbit
> -Original Message- > From: [EMAIL PROTECTED] [mailto:pgsql-general- > [EMAIL PROTECTED] On Behalf Of Thomas Kellerer > Sent: Friday, October 13, 2006 2:11 PM > To: pgsql-general@postgresql.org > Subject: Re: [GENERAL] more anti-postgresql FUD > > [EMAIL PROTECTED] wrote on 11.10.2006 16:

Re: [HACKERS] [PATCHES] index advisor

2006-10-13 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Kai-Uwe Sattler wrote: >> here is a patch implementing an index recommendation advisor for >> 7.4.13. > You know we don't put new features on stable branches, right? I think you forgot the context --- somebody asked Kai-Uwe to send what he had. There

Re: Hints (Was: [HACKERS] Index Tuning Features)

2006-10-13 Thread Josh Berkus
Tom, > We have also talked about solving the multi-column statistics problem > (which, at its core, is "which combinations of columns are worth > accumulating stats for?" --- you can't possibly store stats for every > combination!) by having what would amount to hints from the DBA saying > "keep s

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Peter Eisentraut
Peter Eisentraut wrote: > Yeah, it seems there are some portability problems with that code. > I'm looking into it. Fixed. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Martijn van Oosterhout
On Fri, Oct 13, 2006 at 01:24:52PM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > Hrm... I don't suppose there's a way to capture the critical warnings in > > a temporary file and then cat that at the end? (I'm assuming that it'll > > be nearly impossible to get a quite mod

Re: [HACKERS] Database Auditing

2006-10-13 Thread Marco Serantoni
I'm evaluating of use postgresql but for local law requirements is needed for the access of some kind of data (sensitive) a log of the accesses (Auditing) is a feature available in many databases but i've seen that lacks in PostgreSQL, there are already plans to implement it or patches already s

[HACKERS] problem with using O_DIRECT

2006-10-13 Thread Ye Qin
I tried to use O_DIRECT on Linux (SuSe) Kernel 2.6, but failed to make it run. For example, if I added the option in the "open" of BasicOpenFile(), I got the following error after typing "psql -l", psql: could not connect to server: Connection refused Is the server running locally and acce

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 09:36:14PM +0200, Peter Eisentraut wrote: > Jim C. Nasby wrote: > > > [EMAIL PROTECTED]:~/devel/pg82/pgsql$ ./configure --enable-foo --quiet > > > *** Option ignored: --enable-foo > > > [EMAIL PROTECTED]:~/devel/pg82/pgsql$ > > > > Odd, I can't get that on a very recent chec

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Peter Eisentraut
Jim C. Nasby wrote: > > [EMAIL PROTECTED]:~/devel/pg82/pgsql$ ./configure --enable-foo --quiet > > *** Option ignored: --enable-foo > > [EMAIL PROTECTED]:~/devel/pg82/pgsql$ > > Odd, I can't get that on a very recent checkout of HEAD: > [EMAIL PROTECTED]:51]~/pgsql/HEAD:156%./configure > --with-inc

Re: [HACKERS] [PATCHES] index advisor

2006-10-13 Thread Jim C. Nasby
Moving to -hackers... On Fri, Oct 13, 2006 at 03:07:50PM -0400, Andrew Dunstan wrote: > Kai-Uwe Sattler wrote: > >Hi, > >here is a patch implementing an index recommendation advisor for > >7.4.13. It consists of extensions to the planner and the catalog as > >well as a command line tool pg_advis

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 08:40:42PM +0200, Peter Eisentraut wrote: > Tom Lane wrote: > > Not sure what to do about that --- I doubt that raising this warning > > to error would be a good idea, seeing how firmly the upstream > > developers believe it shouldn't even be a warning. Is there any sort >

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Peter Eisentraut
Tom Lane wrote: > Not sure what to do about that --- I doubt that raising this warning > to error would be a good idea, seeing how firmly the upstream > developers believe it shouldn't even be a warning. Is there any sort > of "quiet mode" possible that would report only warnings? Would it > be a

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Stefan Kaltenbrunner
Jim C. Nasby wrote: > On Fri, Oct 13, 2006 at 10:23:31AM -0700, Jeff Davis wrote: >> Right. And I think the sane ideas are along the lines of estimate & cost >> corrections (like Tom is saying). > > Let me ask this... how long do you (and others) want to wait for those? well - we waited and got o

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Andrew Dunstan
Joshua D. Drake wrote: Jim C. Nasby wrote: On Fri, Oct 13, 2006 at 10:23:31AM -0700, Jeff Davis wrote: Right. And I think the sane ideas are along the lines of estimate & cost corrections (like Tom is saying). Let me ask this... how long do you (and others) want to wait for thos

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jeff Davis
On Fri, 2006-10-13 at 12:30 -0500, Jim C. Nasby wrote: > On Fri, Oct 13, 2006 at 10:23:31AM -0700, Jeff Davis wrote: > > Right. And I think the sane ideas are along the lines of estimate & cost > > corrections (like Tom is saying). > > Let me ask this... how long do you (and others) want to wait f

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread David Fetter
On Fri, Oct 13, 2006 at 12:30:24PM -0500, Jim C. Nasby wrote: > On Fri, Oct 13, 2006 at 10:23:31AM -0700, Jeff Davis wrote: > > Right. And I think the sane ideas are along the lines of estimate > > & cost corrections (like Tom is saying). > > Let me ask this... how long do you (and others) want to

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Bucky Jordan
> I completely agree that it's much better *in the long run* to improve > the planner and the statistics system so that we don't need hints. But > there's been no plan put forward for how to do that, which means we also > have no idea when some of these problems will be resolved. If someone > comes

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Joshua D. Drake
Jim C. Nasby wrote: > On Fri, Oct 13, 2006 at 10:23:31AM -0700, Jeff Davis wrote: >> Right. And I think the sane ideas are along the lines of estimate & cost >> corrections (like Tom is saying). > > Let me ask this... how long do you (and others) want to wait for those? > It's great that the plann

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 10:23:31AM -0700, Jeff Davis wrote: > Right. And I think the sane ideas are along the lines of estimate & cost > corrections (like Tom is saying). Let me ask this... how long do you (and others) want to wait for those? It's great that the planner is continually improving, b

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Hrm... I don't suppose there's a way to capture the critical warnings in > a temporary file and then cat that at the end? (I'm assuming that it'll > be nearly impossible to get a quite mode out of autoconf...) Hmm ... maybe we could just rearrange the s

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jeff Davis
On Fri, 2006-10-13 at 13:08 -0400, Bucky Jordan wrote: > > It sounds horrible to me too, and I'm the one that thought of it (or > at > > least I'm the one that introduced it to this thread). > > > > However, everything is relative. Since the other idea floating around > is > > to put the same hint

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Bucky Jordan
> -Original Message- > From: Jeff Davis [mailto:[EMAIL PROTECTED] > Sent: Friday, October 13, 2006 1:00 PM > To: Alvaro Herrera > Cc: Bucky Jordan; josh@agliodbs.com; Jim C. Nasby; pgsql- > [EMAIL PROTECTED] > Subject: Re: [HACKERS] [PERFORM] Hints proposal > > On Thu, 2006-10-12 at 18:02

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 12:45:23PM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > In the meantime, +1 to adding some whitespace around the warning... I'd > > suggest two blank lines before and after. > > I don't really see that that would accomplish anything. The problem

Re: [HACKERS] [PATCHES] array_accum aggregate

2006-10-13 Thread Tom Lane
Stephen Frost <[EMAIL PROTECTED]> writes: >>> Additionally, we'd have to be >>> able to mark the types as being polymorhpic along the same lines as >>> anyelement/anyarray. >> >> What for? > So that the finalfunc can be polymorphic along the lines of my suggested > aaccum_sfunc(anyarray,anyelement

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jeff Davis
On Thu, 2006-10-12 at 18:02 -0400, Alvaro Herrera wrote: > Bucky Jordan wrote: > > > What about using regular expressions, plus, if you have a function > > (views, or any other statement that is stored), you can assign a rule to > > that particular function. So you get matching, plus explicit sele

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Gregory Stark
Josh Berkus writes: >> > I actually think the way to attack this issue is to discuss the kinds >> > of errors the planner makes, and what tweaks we could do to correct >> > them. Here's the ones I'm aware of: >> > >> > -- Incorrect selectivity of WHERE clause >> > -- Incorrect selectivity of JOIN

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jeff Davis
On Thu, 2006-10-12 at 23:12 -0400, Christopher Browne wrote: > > No, I don't have any idea, except that it would be less push-back > > than changing a language that's embedded in client code. Also, I see > > no reason to think that a hint would not be obsolete upon a new > > release anyway. > > I

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > In the meantime, +1 to adding some whitespace around the warning... I'd > suggest two blank lines before and after. I don't really see that that would accomplish anything. The problem is exactly that configure emits many many lines of output which no o

Re: [HACKERS] Updated version of FAQ_Solaris

2006-10-13 Thread Zdenek Kotala
Peter Eisentraut napsal(a): Zdenek Kotala wrote: + The PostgreSQL 8.2 has implemented dtrace support. You can enable it by + the --enable-dtrace configure switch. If you want to compile a 64-bit code + with dtrace you must specify DTRACEFLAGS='-64', e.g. This is contrary to the documentation

Re: [HACKERS] [PATCHES] array_accum aggregate

2006-10-13 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > The other issue is, in the above scenario > > is it acceptable to modify the result of my_sfunc(null, 1) in the ,2 > > call? > > Yes, because the only place a nonnull value of the type could have come > from is a

Re: [HACKERS] Additional stats for Relations

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 06:17:47PM +0530, NikhilS wrote: > Currently a "select * from pg_statio_user_tables;" displays only > heap_blks_read, heap_blks_hit stats amongst others for the main relation. It > would be good to have the following stats collected too. I think these stats > can be used to

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > I completely agree that it's much better *in the long run* to improve > the planner and the statistics system so that we don't need hints. But > there's been no plan put forward for how to do that, which means we also > have no idea when some of these pr

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 11:25:28AM -0400, Alvaro Herrera wrote: > Tom Lane wrote: > > Martijn van Oosterhout writes: > > > I would like a pedantic mode, but the method of activation is a tricky > > > issue. You want it to be 'on' for normal users but 'off' for automatic > > > build scripts. > > >

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 01:58:22PM -0700, Josh Berkus wrote: > > Unless you've got a time machine or a team of coders in your back > > pocket, I don't see how the planner will suddenly become perfect in > > 8.4... > > Since you're not a core code contributor, I really don't see why you > continue

Re: [HACKERS] [PATCHES] array_accum aggregate

2006-10-13 Thread Tom Lane
Stephen Frost <[EMAIL PROTECTED]> writes: >> That's not really the flavor of solution I'd like to have. Ideally, >> it'd actually *work* to write >> my_ffunc(my_sfunc(my_sfunc(null, 1), 2)) > In general I like this idea but there are some complications, the main > one being where would the memory

Re: [HACKERS] [PATCHES] array_accum aggregate

2006-10-13 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > That's not really the flavor of solution I'd like to have. Ideally, > it'd actually *work* to write > > my_ffunc(my_sfunc(my_sfunc(null, 1), 2)) > > and get the same result as aggregating over the values 1 and 2. The > trick is to make sure that my_

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Alvaro Herrera
Tom Lane wrote: > Martijn van Oosterhout writes: > > I would like a pedantic mode, but the method of activation is a tricky > > issue. You want it to be 'on' for normal users but 'off' for automatic > > build scripts. > > A quiet mode would work better, because it could be on by default for > bo

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Zeugswetter Andreas ADI SD
> > Can you give us an example that had such a monstrous effect in Oracle, > > other than that the hint was a mistake in the first place ? > > Of course the hint was a mistake in the first place; the > little story I told was exactly an example of such a case. > The hint shouldn't have been p

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Csaba Nagy
> And remember, the places where PostgreSQL is getting used most > heavily are still the sort of environments where people will take a > lot of short cuts to achieve an immediate result, and be annoyed when > that short cut later turns out to have been expensive. Postgres will > get a black eye fr

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Tom Lane
Martijn van Oosterhout writes: > I would like a pedantic mode, but the method of activation is a tricky > issue. You want it to be 'on' for normal users but 'off' for automatic > build scripts. A quiet mode would work better, because it could be on by default for both cases. However, people may

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Andrew Sullivan
On Fri, Oct 13, 2006 at 10:41:36AM +0200, Zeugswetter Andreas ADI SD wrote: > Can you give us an example that had such a monstrous effect in Oracle, > other than that the hint was a mistake in the first place ? Of course the hint was a mistake in the first place; the little story I told was exactl

Re: [HACKERS] On status data and summaries

2006-10-13 Thread Bruce Momjian
Andrew Sullivan wrote: > On Thu, Oct 12, 2006 at 05:26:02PM -0400, Bruce Momjian wrote: > > What I think people wanted was something like that, but maintained > > during the development cycle, so they would know what features our being > > worked on, and by whom. > > I'd be happy to work on that.

Re: [HACKERS] [PATCHES] array_accum aggregate

2006-10-13 Thread Tom Lane
Martijn van Oosterhout writes: > What this really calls for is a type that users are forbidden to > interact with directly. Basically, the type may only be used by C > functions and such C functions may not appear in an SQL query. That's not really the flavor of solution I'd like to have. Ideall

Re: [HACKERS] On status data and summaries

2006-10-13 Thread Andrew Sullivan
On Thu, Oct 12, 2006 at 05:26:02PM -0400, Bruce Momjian wrote: > What I think people wanted was something like that, but maintained > during the development cycle, so they would know what features our being > worked on, and by whom. I'd be happy to work on that. > One great thing about the list I

[HACKERS] Additional stats for Relations

2006-10-13 Thread NikhilS
Hi, Currently a "select * from pg_statio_user_tables;" displays only heap_blks_read, heap_blks_hit stats amongst others for the main relation. It would be good to have the following stats collected too. I think these stats can be used to better statistically analyze/understand the block I/O activ

Re: [HACKERS] Hints (Was: Index Tuning Features)

2006-10-13 Thread Arturo Perez
In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (Andrew Sullivan) wrote: > On Thu, Oct 12, 2006 at 08:34:45AM +0200, Florian Weimer wrote: > > > > Some statistics are very hard to gather from a sample, e.g. the number > > of distinct values in a column. > > I like the suggestion, though, tha

Re: [HACKERS] Hints WAS: Index Tuning Features

2006-10-13 Thread Zeugswetter Andreas ADI SD
> And, to give you a starting point: the discussion has morphed into: > "What manual ways can we come up with for the DBA to > influence the planner and fix planner "bugs" which won't have > the fragility of query-based hints ala Oracle?" I see a few downsides though: 1. it lacks a way to try

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Zeugswetter Andreas ADI SD
> I actually think the way to attack this issue is to discuss the kinds of errors the planner makes, and what tweaks we could do to correct them. > Here's the ones I'm aware of: > > -- Incorrect selectivity of WHERE clause > -- Incorrect selectivity of JOIN > -- Wrong estimate of rows returned f

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Zeugswetter Andreas ADI SD
> The _other_ way it's going to be used is as a stealthy > alteration to regular behaviour, to solve a particular nasty > performance problem that happens to result on a given day. > And every single time I've seen anything like that done, the > long term effect is always monstrous. Funny, I

Re: [HACKERS] ./configure argument checking

2006-10-13 Thread Martijn van Oosterhout
On Thu, Oct 12, 2006 at 08:01:06PM -0500, Andrew Dunstan wrote: > > I think one idea is a "pedantic" mode that fails if an unrecognized > > option is supplied. > > I do not see any point at all in a special mode. If you know enough to > want to use it you should be able to protect yourself more di

Re: [HACKERS] [PATCHES] array_accum aggregate

2006-10-13 Thread Martijn van Oosterhout
On Thu, Oct 12, 2006 at 06:58:52PM -0400, Tom Lane wrote: > I wrote: > > aggregate_state would have no other uses in the system, and its input > > and output functions would raise an error, so type safety is assured > > --- there would be no way to call either the sfunc or ffunc "manually", > > exc