Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread David Fetter
On Tue, Sep 16, 2008 at 08:59:08PM -0400, Alvaro Herrera wrote: Simon Riggs wrote: Disabling autovacuum can have catastrophic effects, since it disables the ANALYZing of tables. Can we have a mode where we disable autoVACUUM yet enable autoANALYZE? You mean something like autovacuum

Re: [HACKERS] Proposal of SE-PostgreSQL patches (for CommitFest:Sep)

2008-09-17 Thread KaiGai Kohei
Greg Smith wrote: On Wed, 17 Sep 2008, Peter Eisentraut wrote: System-wide consistency in access controls could be nice to have in some cases. But is it really achievable? In the typical three-tier web application scenario, do you really have system-wide consistency? Can you configure

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Heikki Linnakangas
David Fetter wrote: On Tue, Sep 16, 2008 at 08:59:08PM -0400, Alvaro Herrera wrote: Simon Riggs wrote: Disabling autovacuum can have catastrophic effects, since it disables the ANALYZing of tables. Can we have a mode where we disable autoVACUUM yet enable autoANALYZE? You mean something like

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 10:09 +0300, Heikki Linnakangas wrote: David Fetter wrote: On Tue, Sep 16, 2008 at 08:59:08PM -0400, Alvaro Herrera wrote: Simon Riggs wrote: Disabling autovacuum can have catastrophic effects, since it disables the ANALYZing of tables. Can we have a mode where

Re: [HACKERS] EXEC_BACKEND

2008-09-17 Thread Simon Riggs
On Tue, 2008-09-16 at 15:53 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: We keep talking about EXEC_BACKEND mode, though until recently I had misunderstood what that meant. I also realised that I have more than once neglected to take it into account when writing a patch -

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Pavel Stehule
2008/9/17 Tom Lane [EMAIL PROTECTED]: Tatsuo Ishii [EMAIL PROTECTED] writes: Do we really have to make RECURSIVE a fully reserved keyword? According to the standard, RECURSIVE is a reserved keyword, I believe. Sure, but our general rule is to make keywords no more reserved than is

Re: [HACKERS] text search patch status update?

2008-09-17 Thread Heikki Linnakangas
Sushant Sinha wrote: Patch #2. I think this is a straigt forward bug fix. Yes, I think you're right. In hlCover(), *q is 0 when the only match is the first item in the text, and we shouldn't bail out with return false in that case. But there seems to be something else going on here as

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Andrew Chernow
Now, it's questionable how tense we need to be about that as long as event proc failure aborts calling of later event procs. That means that procs have to be robust against getting DESTROY with no CREATE calls in any case. Should we try to make that less uncertain? I attached a patch

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Andrew Chernow
Andrew Chernow wrote: Now, it's questionable how tense we need to be about that as long as event proc failure aborts calling of later event procs. That means that procs have to be robust against getting DESTROY with no CREATE calls in any case. Should we try to make that less

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Robert Haas
I am not sure, if these rule is good. Somebody who develop on postgresql should have a problems when they will be port to other databases in future. Reserved words in standards should be respected. I disagree. I have never ported an app written for PostgreSQL to another database system, and

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Tom Lane
Andrew Chernow [EMAIL PROTECTED] writes: Now, it's questionable how tense we need to be about that as long as event proc failure aborts calling of later event procs. That means that procs have to be robust against getting DESTROY with no CREATE calls in any case. Should we try to make that

Re: [HACKERS] New FSM patch

2008-09-17 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Heikki Linnakangas wrote: snip Let me describe this test case first: - The test program calls RecordAndGetPageWithFreeSpace in a tight loop, with random values. There's no activity to the heap. In normal usage, the time spent in RecordAndGetWithFreeSpace is

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tom Lane
Robert Haas [EMAIL PROTECTED] writes: I am not sure, if these rule is good. Somebody who develop on postgresql should have a problems when they will be port to other databases in future. Reserved words in standards should be respected. I disagree. I have never ported an app written for

Re: [HACKERS] Patch for SQL-standard negative valued year-month literals

2008-09-17 Thread Stephen R. van den Berg
Tom Lane wrote: Ron Mayer [EMAIL PROTECTED] writes: Tom Lane wrote: If I read SQL 200N's spec correctly select interval '-1 1:00:00'; should mean-1 days -1 hours, yet 8.3 sees it as -1 days +1 hours. I think we are kind of stuck on this one. If we change it, then how would one

Re: [HACKERS] Patch for SQL-standard negative valued year-month literals

2008-09-17 Thread Tom Lane
Stephen R. van den Berg [EMAIL PROTECTED] writes: Intervals are a scalar, not an addition of assorted values, alternating signs between fields would be wrong. Sorry, you're the one who's wrong on that. We've treated intervals as three independent fields for years now (and before that it was

Re: [HACKERS] per-table autovacuum configuration

2008-09-17 Thread Martin Pihlak
Alvaro Herrera wrote: any new thoughts on the matter? Perhaps someone is already working on it? It is still a wanted feature, but a couple of people have offered patches and I'm waiting for them ... Aha, good. I was considering taking a stab at it, but under the circumstances will wait

Re: [HACKERS] Proposal of SE-PostgreSQL patches (for CommitFest:Sep)

2008-09-17 Thread KaiGai Kohei
KaiGai Kohei wrote: Peter, thanks for your comments. Let's review: *) System-wide consistency in access controls could be nice to have in some cases. But is it really achievable? In the typical three-tier web application scenario, do you really have system-wide consistency? Can

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Andrew Chernow
Tom Lane wrote: Some thought would need to be given to how that interacts with RESULTCOPY. Presumably on the destination side, RESULTCOPY is the equivalent of RESULTCREATE, ie, don't DESTROY if you didn't get COPY. But what of the source side? Arguably you shouldn't invoke COPY on events that

Re: [HACKERS] NDirectFileRead and Write

2008-09-17 Thread Tom Lane
ITAGAKI Takahiro [EMAIL PROTECTED] writes: - NDirectFile{Read|Write} are renamed to BufFile{Read|Write}Count. - They are still declared in execdebug.h and buffile.c includes it. After some thought I concluded that the least ugly way to do this was to declare and define the variables in the same

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Tom Lane
Andrew Chernow [EMAIL PROTECTED] writes: Tom Lane wrote: Some thought would need to be given to how that interacts with RESULTCOPY. Presumably on the destination side, RESULTCOPY is the equivalent of RESULTCREATE, ie, don't DESTROY if you didn't get COPY. But what of the source side?

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Kevin Grittner
Tom Lane [EMAIL PROTECTED] wrote: Robert Haas [EMAIL PROTECTED] writes: I am not sure, if these rule is good. Somebody who develop on postgresql should have a problems when they will be port to other databases in future. Reserved words in standards should be respected. If people want to

Re: [HACKERS] Patch for SQL-standard negative valued year-month literals

2008-09-17 Thread Ron Mayer
Tom Lane wrote: Stephen R. van den Berg [EMAIL PROTECTED] writes: Intervals are a scalar, not an addition of assorted values, alternating signs between fields would be wrong. Sorry, you're the one who's wrong on that. We've treated intervals as three independent fields for years now (and

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Alvaro Herrera
Simon Riggs wrote: On Wed, 2008-09-17 at 10:09 +0300, Heikki Linnakangas wrote: Isn't autoanalyze a waste of time during a bulk load? Seems better to run ANALYZE manually at the end. Its not a waste of time because it catches tables immediately they have been loaded, not just at the

Re: Column level privileges was:(Re: [HACKERS] Extending grant insert on tables to sequences)

2008-09-17 Thread Stephen Frost
Jaime, * Stephen Frost ([EMAIL PROTECTED]) wrote: * Jaime Casanova ([EMAIL PROTECTED]) wrote: On 7/25/08, Stephen Frost [EMAIL PROTECTED] wrote: Yes, I'm working on it hi, any work on it? may i help? If you look at the commitfest, I've posted my WIP so far there. Most of the

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Heikki Linnakangas
Alvaro Herrera wrote: Simon Riggs wrote: On Wed, 2008-09-17 at 10:09 +0300, Heikki Linnakangas wrote: Isn't autoanalyze a waste of time during a bulk load? Seems better to run ANALYZE manually at the end. Its not a waste of time because it catches tables immediately they have been loaded, not

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Andrew Chernow
Andrew Chernow wrote: New patch following our discussion with updated docs. few logical errors). I don't think it makes sense to do it otherwise, it would be like calling free after a malloc failure. I can live with that definition, but please document it. To build on this analogy,

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tatsuo Ishii
Is physical_tlist optimization sensible for RecursiveScan? We seem to use it for every other Scan node type. To enable physical_tlist optimization, it seems build_physical_tlist, use_physical_tlist and disuse_physical_tlist need to be changed. build_physical_tlist and use_physical_tlist have

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Alvaro Herrera wrote: Why doesn't this new request conflict with that one? The problem back then was that a CREATE INDEX was waiting on the autoanalyze to finish, and the autoanalyze took a long time to finish because of vacuum_cost_delay. Now

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tom Lane
Tatsuo Ishii [EMAIL PROTECTED] writes: Is physical_tlist optimization sensible for RecursiveScan? We seem to use it for every other Scan node type. To enable physical_tlist optimization, it seems build_physical_tlist, use_physical_tlist and disuse_physical_tlist need to be changed.

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tatsuo Ishii
To enable physical_tlist optimization, it seems build_physical_tlist, use_physical_tlist and disuse_physical_tlist need to be changed. build_physical_tlist and use_physical_tlist have been already patched and only disuse_physical_tlist needs to be patched. Any other place I miss to

Re: [HACKERS] Patch for SQL-standard negative valued year-month literals

2008-09-17 Thread Stephen R. van den Berg
Tom Lane wrote: Stephen R. van den Berg [EMAIL PROTECTED] writes: Intervals are a scalar, not an addition of assorted values, alternating signs between fields would be wrong. Sorry, you're the one who's wrong on that. We've treated intervals as three independent fields for years now (and

Re: Column level privileges was:(Re: [HACKERS] Extending grant insert on tables to sequences)

2008-09-17 Thread Jaime Casanova
On 9/17/08, Stephen Frost [EMAIL PROTECTED] wrote: A piece which can be broken off pretty easily is adding support to track the columns used through to the executor so we can check the permissions in the right place. Jamie, have you had a chance to work on this? It's next on my list and

Re: [HACKERS] text search patch status update?

2008-09-17 Thread Teodor Sigaev
I remember about that but right now I havn't time to make final review. Sorry. Will return soon. Sushant Sinha wrote: Any status updates on the following patches? 1. Fragments in tsearch2 headlines: http://archives.postgresql.org/pgsql-hackers/2008-08/msg00043.php 2. Bug in hlCover:

[HACKERS] Patch for SQL-Standard Interval output and decoupling DateStyle from IntervalStyle

2008-09-17 Thread Ron Mayer
Tom Lane wrote: In fact, given that we are now somewhat SQL-compliant on interval input, a GUC that selected PG traditional, SQL-standard, or ISO 8601 interval output format seems like it could be a good idea. Short summary: The attached patch (1) adds a new GUC called IntervalStyle

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 10:52 -0400, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Alvaro Herrera wrote: Why doesn't this new request conflict with that one? The problem back then was that a CREATE INDEX was waiting on the autoanalyze to finish, and the autoanalyze took a

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Robert Haas
I tend to agree with Alvaro that there's not very much of a use case for an analyze-only autovacuum mode. Assuming that we get to the point of having a parallelizing pg_restore, it would be interesting to give it an option to include ANALYZE for each table it's loaded among the tasks that it

[HACKERS] proposal - function's default parametes

2008-09-17 Thread Pavel Stehule
Hello, before any implementation of named params we have to implement parameters with default values. I don't see any sense of named params without it. It's ortogonal to variadic functions (variadic functions are called when we put more parameters than functions has, dafault parameters are used

[HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
Hi, We've been profiling a large system (8 CPUs, 64 GB of memory, some dozens of disks) which seems rather more swamped than it should. Part of the problem seems to come from CleanupTempFiles, the second entry in oprofile output. This is the top 3 symbol report for a 2 minute oprofile run: $

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 16:25 -0400, Alvaro Herrera wrote: We've been profiling a large system (8 CPUs, 64 GB of memory, some dozens of disks) which seems rather more swamped than it should. Part of the problem seems to come from CleanupTempFiles, the second entry in oprofile output. I'm

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
Simon Riggs wrote: On Wed, 2008-09-17 at 16:25 -0400, Alvaro Herrera wrote: We've been profiling a large system (8 CPUs, 64 GB of memory, some dozens of disks) which seems rather more swamped than it should. Part of the problem seems to come from CleanupTempFiles, the second entry in

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
Alvaro Herrera wrote: Simon Riggs wrote: Simple solution is to have a state variable so you can see whether a backend has created an temp files in this transaction. Most don't, so I think the above two solutions are overkill. If we created any, scan for them, if not, don't. Just a simple

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Ah -- like this? +1, but there are two kinds of temp files in that module, and only one of them is relevant here. Call it something like have_xact_temporary_files to make things clearer. I concur that the explicit test on SizeVfdCache 0 is a waste of

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 17:34 -0400, Alvaro Herrera wrote: Simon Riggs wrote: On Wed, 2008-09-17 at 16:25 -0400, Alvaro Herrera wrote: We've been profiling a large system (8 CPUs, 64 GB of memory, some dozens of disks) which seems rather more swamped than it should. Part of the

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 17:34 -0400, Alvaro Herrera wrote: Ah -- like this? if test should include || isProcExit so you don't skip non-transactional temp files at proc exit when there weren't any created in the last transaction. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Ah -- like this? +1, but there are two kinds of temp files in that module, and only one of them is relevant here. Call it something like have_xact_temporary_files to make things clearer. Ok, so that's what I called it. Simon

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
BTW in testing this patch I was surprised by the fact that temp tables files are removed at checkpoint time, rather than when the transaction ends (at first I thought I had broken the removal of temp files). Is this a recent feature? (I verified that this continues to work fine for WITH HOLD

Re: [HACKERS] New FSM patch

2008-09-17 Thread Decibel!
On Sep 17, 2008, at 9:30 AM, Heikki Linnakangas wrote: I think we'd still need to WAL log operations that decrease the amount of free space on page. Otherwise, after we have partial vacuum, we might never revisit a page, and update the FSM, even though there's usable space on the page,

[HACKERS] 0x1A in control file on Windows

2008-09-17 Thread ITAGAKI Takahiro
I found a bug that pg_controldata ends with error if control files contain 0x1A (Ctrl+Z) on Windows. We probably need to add PG_BINARY when we open control files because 0x1A is an end-of-file marker on Windows. This fix needs to be applied in back versions (8.2, 8.3 and HEAD). Index:

[HACKERS] Where is Aggregation Attribute

2008-09-17 Thread Zhe He
I want to get the attribute from an Aggregation node, where is it? typedef struct Agg { Plan plan; AggStrategy aggstrategy; int numCols; /* number of grouping columns */ AttrNumber *grpColIdx; /* their indexes in the target list */ Oid *grpOperators; /* equality operators to compare with */

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tatsuo Ishii
Tom, thanks for the review. Here is an in-progress report. Patches against CVS HEAD attached. (uncommented items are work-in-progress). -- Tatsuo Ishii SRA OSS, Inc. Japan Tatsuo Ishii [EMAIL PROTECTED] writes: Included is the latest patches against CVS HEAD. I spent some time reading this

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: BTW in testing this patch I was surprised by the fact that temp tables files are removed at checkpoint time, [ blink... ] Doesn't look like that should happen. What is your test case? regards, tom lane -- Sent via

Re: [HACKERS] 0x1A in control file on Windows

2008-09-17 Thread Tom Lane
ITAGAKI Takahiro [EMAIL PROTECTED] writes: I found a bug that pg_controldata ends with error if control files contain 0x1A (Ctrl+Z) on Windows. We probably need to add PG_BINARY when we open control files because 0x1A is an end-of-file marker on Windows. Well, why is that a bug? If the