Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: I would suggest that at end of recovery we write the last LSN to the control file, so if we crash recover then we will always end archive recovery at the same place again should we re-enter it.

Re: [HACKERS] Auto-updated fields

2009-02-05 Thread Stephen R. van den Berg
Christopher Browne wrote: On Wed, Feb 4, 2009 at 1:23 PM, Bruce Momjian br...@momjian.us wrote: Robert Treat wrote: On Wednesday 21 January 2009 20:21:41 Bruce Momjian wrote: CREATE FUNCTION last_updated() RETURNS trigger AS $$ BEGIN NEW.last_update = CURRENT_TIMESTAMP; RETURN

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: I've changed the way minRecoveryPoint is updated now anyway, so it no longer happens every XLogFileRead(). Care to elucidate? I got rid of minSafeStartPoint, advancing minRecoveryPoint instead. And it's advanced

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 10:07 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: I would suggest that at end of recovery we write the last LSN to the control file, so if we crash recover then we will always

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 10:31 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: I've changed the way minRecoveryPoint is updated now anyway, so it no longer happens every XLogFileRead(). Care to elucidate? I got rid of

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 09:26 +, Simon Riggs wrote: This change seems speculative and also against what has previously been agreed with Tom. If he chooses not to comment on your changes, that's up to him, but I don't think you should remove things quietly that have been put there through

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 10:31 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2009-02-05 at 09:28 +0200, Heikki Linnakangas wrote: I got rid of minSafeStartPoint, advancing minRecoveryPoint instead. And it's advanced in XLogFlush instead of XLogFileRead. I'll post

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 11:46 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: So we might end up flushing more often *and* we will be doing it potentially in the code path of other users. For example, imagine a database that fits completely in shared buffers. If we update at every

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 11:46 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: So we might end up flushing more often *and* we will be doing it potentially in the code path of other users. For example, imagine a database that fits completely in shared buffers. If we

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 13:18 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2009-02-05 at 11:46 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: So we might end up flushing more often *and* we will be doing it potentially in the code path of other users. For example,

Re: [HACKERS] Synch Replication

2009-02-05 Thread Fujii Masao
Hi Niranjan, On Thu, Feb 5, 2009 at 3:34 PM, K, Niranjan (NSN - IN/Bangalore) niranja...@nsn.com wrote: I added the further logging codes into the patch, and uploaded the source. If you have time, please try it and report the results. Refer attached logs. Thanks, but I have not identified

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-02-05 at 13:18 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2009-02-05 at 11:46 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: So we might end up flushing more often *and* we will be doing it potentially in the code path of other users. For

Re: [HACKERS] Auto-updated fields

2009-02-05 Thread Andrew Dunstan
Christopher Browne wrote: I think this is a TODO, but not sure who is working on it or what needs to be done. The generic version in pagila is perhaps not generic enough: CREATE FUNCTION last_updated() RETURNS trigger AS $$ BEGIN NEW.last_update = CURRENT_TIMESTAMP; RETURN NEW;

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 14:18 +0200, Heikki Linnakangas wrote: when the control file is updated in XLogFlush, it's typically the bgwriter doing it as it cleans buffers ahead of the clock hand, not the startup process That is the key point. Let's do it your way. -- Simon Riggs

Re: [HACKERS] Auto-updated fields

2009-02-05 Thread Robert Haas
At the very least it should not have a hard-coded field name in it. You should pass the field name to be set as a parameter in the trigger setup. That's probably a lot more doable if the trigger is written in C, and in any case I think any prepackaged triggers we provide should be written in

[HACKERS] Re: [BUGS] BUG #4516: FOUND variable does not work after RETURN QUERY

2009-02-05 Thread Bruce Momjian
Pavel Stehule wrote: I am sending patch, that adds FOUND and GET DIAGNOSTICS support for RETURN QUERY statement Updated patch attached and applied. Thanks. --- Regards Pavel Stehule 2008/11/10 Andrew Gierth

[HACKERS] Fixing Grittner's planner issues

2009-02-05 Thread Tom Lane
I looked into the planning problems in HEAD discussed here: http://archives.postgresql.org/pgsql-hackers/2009-02/msg00120.php As I suspected, there are really two independent bugs there, though both are in the new semi/antijoin planning code. The first problem is the one already mentioned:

Re: [HACKERS] Fixing Grittner's planner issues

2009-02-05 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: SELECT ... FROM Case C LEFT OUTER JOIN CaseDispo CD ON (CD.caseNo = C.caseNo) AND (CD.countyNo = C.countyNo) AND (NOT (EXISTS (SELECT 1 FROM CaseDispo CD2 WHERE (CD2.caseNo = CD.caseNo) AND

Re: [HACKERS] Fixing Grittner's planner issues

2009-02-05 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: If it referred to any variables of the left side of the upper join (ie, C here) then we couldn't convert it to a separate join at all. The only thing that comes to mind for me that seems possibly helpful

Re: [HACKERS] Hot standby, recovery infra

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 21:54 +0200, Heikki Linnakangas wrote: - If bgwriter is performing a restartpoint when recovery ends, the startup checkpoint will be queued up behind the restartpoint. And since it uses the same smoothing logic as checkpoints, it can take quite some time for that to

[HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Alvaro Herrera
Hi, Right now, when autovacuum is turned on we always assume it's supposed to process all tables except those that have autovacuum_enabled=false. Now, sometimes it might make more sense to keep it enabled but have it only check for certain tables, and leave the majority of them disabled. For

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Joshua D. Drake
On Thu, 2009-02-05 at 17:45 -0300, Alvaro Herrera wrote: Hi, Right now, when autovacuum is turned on we always assume it's supposed to process all tables except those that have autovacuum_enabled=false. Now, sometimes it might make more sense to keep it enabled but have it only check for

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 17:45 -0300, Alvaro Herrera wrote: Right now, when autovacuum is turned on we always assume it's supposed to process all tables except those that have autovacuum_enabled=false. Now, sometimes it might make more sense to keep it enabled but have it only check for

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Right now, when autovacuum is turned on we always assume it's supposed to process all tables except those that have autovacuum_enabled=false. Now, sometimes it might make more sense to keep it enabled but have it only check for certain tables,

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Robert Haas
On Thu, Feb 5, 2009 at 3:45 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Right now, when autovacuum is turned on we always assume it's supposed to process all tables except those that have autovacuum_enabled=false. Now, sometimes it might make more sense to keep it enabled but have it

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Alvaro Herrera
Simon Riggs wrote: On Thu, 2009-02-05 at 17:45 -0300, Alvaro Herrera wrote: For this we'd have a separate GUC parameter, as in $SUBJECT (I'm not wedded to the name), and have the user set autovacuum_enabled=true via reloptions to enable it. I would prefer it if that behaviour was

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: It'd make more sense to put the effort into developing better scheduling control over autovacuum, such as a concept of maintenance windows. We need that as well, not instead of. People want to be able to specify (as an example) * autovac

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 18:25 -0300, Alvaro Herrera wrote: So you're not aware that we're doing away with pg_autovacuum for good? It's going to be replaced by reloptions, i.e. ALTER TABLE foo SET (autovacuum_enabled = false); Obviously there's no way to add a catchall setting. Seems like a

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Euler Taveira de Oliveira
Alvaro Herrera escreveu: Hi, Right now, when autovacuum is turned on we always assume it's supposed to process all tables except those that have autovacuum_enabled=false. Now, sometimes it might make more sense to keep it enabled but have it only check for certain tables, and leave the

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Alvaro Herrera
Tom Lane wrote: (BTW, autovac does vacuum tables to prevent wraparound even if you try to tell it to skip them, right?) Yes. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Alvaro Herrera
Simon Riggs wrote: On Thu, 2009-02-05 at 18:25 -0300, Alvaro Herrera wrote: So you're not aware that we're doing away with pg_autovacuum for good? It's going to be replaced by reloptions, i.e. ALTER TABLE foo SET (autovacuum_enabled = false); Obviously there's no way to add a

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: It'd make more sense to put the effort into developing better scheduling control over autovacuum, such as a concept of maintenance windows. We need that as well, not instead of. I disagree; adding

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 18:54 -0300, Alvaro Herrera wrote: Simon Riggs wrote: On Thu, 2009-02-05 at 18:25 -0300, Alvaro Herrera wrote: So you're not aware that we're doing away with pg_autovacuum for good? It's going to be replaced by reloptions, i.e. ALTER TABLE foo SET

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Joshua D. Drake
On Thu, 2009-02-05 at 17:08 -0500, Tom Lane wrote: I disagree; adding every frammish anyone could ever think of is not an overall improvement to the system. :) My feeling is that we should be trying to eliminate use-cases for cron-driven vacuuming, not trying to make sure that cron-driven

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Bruce Momjian
Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: It'd make more sense to put the effort into developing better scheduling control over autovacuum, such as a concept of maintenance windows. We need that as well, not instead

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 17:57 -0500, Bruce Momjian wrote: Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: It'd make more sense to put the effort into developing better scheduling control over autovacuum, such as a concept of

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 17:08 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Thu, 2009-02-05 at 16:29 -0500, Tom Lane wrote: It'd make more sense to put the effort into developing better scheduling control over autovacuum, such as a concept of maintenance windows.

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Greg Stark
On Thu, Feb 5, 2009 at 11:57 PM, Simon Riggs si...@2ndquadrant.com wrote: Writing an application maintenance utility in PL/pgSQL is much better than having to write it for all the different servers an application may need to run on. Welcome to the suction effect. If your scheduler is in the

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Robert Haas
Agreed, let's get this capability out in 8.4 and we can always adust it based on user demand. Oh, I agree to limiting what we do for 8,4, but we need more later. Thinking about this a little more, the biggest problem I have with this feature is that it makes autovacuum_enabled mean two

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Andrew Dunstan
Simon Riggs wrote: not trying to make sure that cron-driven scripts can do anything autovacuum can. I'm not in favour of limiting our capability to internal actions only. If we add a capability for scheduling work, we can easily make it capable of scheduling many kinds of work. Writing

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Josh Berkus
Alvaro, First off, with over 200 GUC variables currently active, in general we should be looking to *eliminate* variables rather than adding them. So my personal bar for endorsing a new GUC is set pretty high. Now, sometimes it might make more sense to keep it enabled but have it only

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Robert Haas
On Thu, Feb 5, 2009 at 7:13 PM, Robert Haas robertmh...@gmail.com wrote: Thinking about this a little more, the biggest problem I have with this feature is that it makes autovacuum_enabled mean two different things depending on context. But maybe we should change the name of the reloption to

Re: [HACKERS] autovacuum and reloptions

2009-02-05 Thread Euler Taveira de Oliveira
Alvaro Herrera escreveu: So here's what looks like a committable patch. Note to self: remember to remove src/include/catalog/pg_autovacuum.h and to bump catversion. Works for me. Just a few comments. (i) I don't like this construction by entries by changing storage parameters. I prefer

Re: [HACKERS] Synch Replication

2009-02-05 Thread Fujii Masao
Hi Niranjan, On Thu, Feb 5, 2009 at 10:50 PM, K, Niranjan (NSN - IN/Bangalore) niranja...@nsn.com wrote: I changed the code to dump all messages for replication, so please try it and report the result. The messages which the standby server receives are logged in the file (*1). Please send

Re: [HACKERS] 8.4 release planning

2009-02-05 Thread Fujii Masao
Hi, On Tue, Feb 3, 2009 at 5:08 AM, Bruce Momjian br...@momjian.us wrote: o Others We will focus on all the other items on the commit fest page, and that will determine our time-line for 8.4 beta, i.e. the first three items will not delay our beta release. Simon is assigned as

Re: [HACKERS] 8.4 release planning

2009-02-05 Thread Koichi Suzuki
I understand Simon is extremely busy on his own patch. I appreciate if anyone help the review. 2009/2/6 Fujii Masao masao.fu...@gmail.com: Hi, On Tue, Feb 3, 2009 at 5:08 AM, Bruce Momjian br...@momjian.us wrote: o Others We will focus on all the other items on the commit fest

Re: [HACKERS] Fixing Grittner's planner issues

2009-02-05 Thread Robert Haas
I think the only fix for this that's practical in the 8.4 time frame is to give up trying to flatten EXISTS/NOT EXISTS that occur within the ON condition of an outer join, ie, revert to 8.3's level of intelligence about this case. It seems like a general purpose solution would involve

[HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1522)

2009-02-05 Thread KaiGai Kohei
The following patches are updated ones: [1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1522.patch [2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1522.patch [3/5]

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Thu, 2009-02-05 at 19:23 -0500, Andrew Dunstan wrote: Simon Riggs wrote: not trying to make sure that cron-driven scripts can do anything autovacuum can. I'm not in favour of limiting our capability to internal actions only. If we add a capability for scheduling work, we

Re: [HACKERS] new GUC var: autovacuum_process_all_tables

2009-02-05 Thread Simon Riggs
On Fri, 2009-02-06 at 00:07 +, Greg Stark wrote: On Thu, Feb 5, 2009 at 11:57 PM, Simon Riggs si...@2ndquadrant.com wrote: Writing an application maintenance utility in PL/pgSQL is much better than having to write it for all the different servers an application may need to run on.