Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 11:44 +0200, Peter Eisentraut wrote: > On mån, 2009-12-14 at 08:54 +, Simon Riggs wrote: > > Wednesday because that seemed a good delay to allow review. Josh and I > > had discussed the value of getting patch into Alpha3, so that was my > > wish and aim. > > > > I'm not a

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 16:39 -0500, Tom Lane wrote: > Simon Riggs writes: > > What is the best way of restricting the hash table to a maximum size? > > There is nothing in dynahash that will enforce a maximum size against > calling code that's not cooperating; and I'll resist any attempt to > add

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Simon Riggs writes: > What is the best way of restricting the hash table to a maximum size? There is nothing in dynahash that will enforce a maximum size against calling code that's not cooperating; and I'll resist any attempt to add such a thing, because it would create a serialization point ac

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 15:24 -0500, Tom Lane wrote: > Simon Riggs writes: > > On Mon, 2009-12-14 at 20:32 +0200, Heikki Linnakangas wrote: > >>> I have ensured that they are always the same size, by definition, so no > >>> need to check. > >> > >> How did you ensure that? The hash table has no har

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Simon Riggs writes: > On Mon, 2009-12-14 at 20:32 +0200, Heikki Linnakangas wrote: >>> I have ensured that they are always the same size, by definition, so no >>> need to check. >> >> How did you ensure that? The hash table has no hard size limit. > The hash table is in shared memory and the ent

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 14:14 -0500, Tom Lane wrote: > Simon Riggs writes: > > On Mon, 2009-12-14 at 13:48 -0500, Tom Lane wrote: > >> Simon Riggs writes: > >>> * Disallow clustering system relations. This will definitely NOT work > >>> * for shared relations (we have no way to update pg_class row

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Simon Riggs writes: > On Mon, 2009-12-14 at 13:48 -0500, Tom Lane wrote: >> Simon Riggs writes: >>> * Disallow clustering system relations. This will definitely NOT work >>> * for shared relations (we have no way to update pg_class rows in other >>> * databases), nor for nailed-in-cache relation

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 13:48 -0500, Tom Lane wrote: > Simon Riggs writes: > > * Disallow clustering system relations. This will definitely NOT work > > * for shared relations (we have no way to update pg_class rows in other > > * databases), nor for nailed-in-cache relations (the relfilenode va

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 20:32 +0200, Heikki Linnakangas wrote: > >> * You removed this comment from KnownAssignedXidsInit: > >> > >> - /* > >> -* XXX: We should check that we don't exceed maxKnownAssignedXids. > >> -* Even though the hash table might hold a few more entries than that, > >>

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Simon Riggs writes: > * Disallow clustering system relations. This will definitely NOT work > * for shared relations (we have no way to update pg_class rows in other > * databases), nor for nailed-in-cache relations (the relfilenode values > * for those are hardwired, see relcache.c). It mig

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Sun, 2009-12-13 at 22:25 +, Simon Riggs wrote: > * Which exact tables are we talking about: just pg_class and the shared > catalogs? Everything else is in pg_class, so if we can find it we're OK? > formrdesc() tells me the list of nailed relations is: pg_database, > pg_class, pg_attribute,

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 18:02 +, Greg Stark wrote: > >> It doesn't seem any more wrong than using hash indexes right after > >> recovery, crash recovery or otherwise. It's certainly broken, but I > >> don't see much value in a partial fix. The bottom line is that hash > >> indexes should be WAL-l

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Heikki Linnakangas
Simon Riggs wrote: > On Mon, 2009-12-14 at 11:54 +0200, Heikki Linnakangas wrote: >> Simon Riggs wrote: >> * Are you planning to remove the recovery_connections setting before >> release? The documentation makes it sound like it's a temporary hack >> that we're not really sure is needed at all. Tha

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Heikki Linnakangas
Greg Smith wrote: > Heikki Linnakangas wrote: >> I note that if it was easy to run pgindent yourself on a patch before >> committing/submitting, we wouldn't need to have this discussion. I don't >> know hard it is to get it working right, but I recall I tried once and >> gave up. >> > What sort

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Greg Stark
On Mon, Dec 14, 2009 at 11:07 AM, Simon Riggs wrote: >> * vacuum_defer_cleanup_age is very hard to tune. You'll need an estimate >> on your transaction rate to begin with. Do we really want this setting >> in its current form? Does it make sense as PGC_USERSET, as if one >> backend uses a lower se

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Greg Smith
Heikki Linnakangas wrote: I note that if it was easy to run pgindent yourself on a patch before committing/submitting, we wouldn't need to have this discussion. I don't know hard it is to get it working right, but I recall I tried once and gave up. What sort of problems did you run into? --

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Bruce Momjian
Tom Lane wrote: > Robert Haas writes: > > On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs wrote: > >> Why is (1) important, and if it is important, why is it being mentioned > >> only now? Are we saying that all previous reviewers of my work (and > >> others') removed these without ever mentioning t

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 10:49 AM, Simon Riggs wrote: > On Mon, 2009-12-14 at 09:14 -0600, Kevin Grittner wrote: >> Simon Riggs wrote: >> >> > If we are going to run pgindent anyway, what is the point? >> >> Perhaps it would take less time to run this than to argue the point?: >> >> sed -e 's/[ \t

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread David Fetter
On Mon, Dec 14, 2009 at 11:09:49AM +0100, Magnus Hagander wrote: > On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas > wrote: > > * Please remove any spurious whitespace.  "git diff --color" makes > > them stand out like a sore thumb, in red. (pgindent will fix them > > but always better to fix th

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 09:14 -0600, Kevin Grittner wrote: > Simon Riggs wrote: > > > If we are going to run pgindent anyway, what is the point? > > Perhaps it would take less time to run this than to argue the point?: > > sed -e 's/[ \t][ \t]*$//' -e 's/ *\t/\t/g' * Not certain that is exac

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Heikki Linnakangas
Simon Riggs wrote: > On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: >> On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas >> wrote: >>> * Please remove any spurious whitespace. "git diff --color" makes them >>> stand out like a sore thumb, in red. (pgindent will fix them but always >>>

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Alvaro Herrera
Tom Lane escribió: > The whole thing would be a lot easier if someone would put together an > easily-installable version of pgindent. Bruce has posted the patches he > uses but I don't know what version of indent they're against... And we're still unclear on the typedef list that's going to be u

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 10:08 AM, Simon Riggs wrote: > On Mon, 2009-12-14 at 09:32 -0500, Robert Haas wrote: >> If every patch perfectly matched the pgident style, then the pgindent >> run would change nothing and we would all be VERY happy. > > I've made all whitespace changes to the patch. > > I

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Tom Lane
Robert Haas writes: > On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs wrote: >> Why is (1) important, and if it is important, why is it being mentioned >> only now? Are we saying that all previous reviewers of my work (and >> others') removed these without ever mentioning they had done so? > pgiden

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 13:56 +0100, Magnus Hagander wrote: > Same issue can be it in git - did you do a "git pull" before? You may > need merging with what's on there, and for that to work you must pull > before you can push. Found some merge conflicts and resolved them. I did fetch and merge at d

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Kevin Grittner
Simon Riggs wrote: > If we are going to run pgindent anyway, what is the point? Perhaps it would take less time to run this than to argue the point?: sed -e 's/[ \t][ \t]*$//' -e 's/ *\t/\t/g' * -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 09:32 -0500, Robert Haas wrote: > If every patch perfectly matched the pgident style, then the pgindent > run would change nothing and we would all be VERY happy. I've made all whitespace changes to the patch. I understand the reason for acting as you suggest, but we either

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 7:42 AM, Simon Riggs wrote: > There are no changes in this patch that are purely whitespace changes. > Those were removed prior to patch submission. This is all about code I > am adding or changing. My additions may disrupt their patches, but not > because of the whitespace

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Magnus Hagander
On Mon, Dec 14, 2009 at 13:47, Simon Riggs wrote: > On Mon, 2009-12-14 at 07:33 -0500, Robert Haas wrote: >> On Mon, Dec 14, 2009 at 7:22 AM, Simon Riggs wrote: >> > I am now unable to push these changes to the shared repo. What is >> > happening? >> >> Perhaps you need to run cvs update on your

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 07:33 -0500, Robert Haas wrote: > On Mon, Dec 14, 2009 at 7:22 AM, Simon Riggs wrote: > > I am now unable to push these changes to the shared repo. What is > > happening? > > Perhaps you need to run cvs update on your local copy? > > (I find this flavor the most useful: "cv

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 06:51 -0500, Robert Haas wrote: > On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs wrote: > > On Mon, 2009-12-14 at 06:21 -0500, Robert Haas wrote: > >> On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs wrote: > >> > On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: > >> >> O

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 7:22 AM, Simon Riggs wrote: > I am now unable to push these changes to the shared repo. What is > happening? Perhaps you need to run cvs update on your local copy? (I find this flavor the most useful: "cvs -q update -d" YMMV.) If that's not it, error message? ...Robert

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 11:07 +, Simon Riggs wrote: > Thanks for the further review, much appreciated. > > On Mon, 2009-12-14 at 11:54 +0200, Heikki Linnakangas wrote: > > Simon Riggs wrote: > > > Enclose latest version of Hot Standby. > > > * LockAcquireExtended needs a function comment. Or

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Magnus Hagander
On Mon, Dec 14, 2009 at 12:51, Robert Haas wrote: > On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs wrote: >> On Mon, 2009-12-14 at 06:21 -0500, Robert Haas wrote: >>> On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs wrote: >>> > On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: >>> >> On Mon,

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 6:35 AM, Simon Riggs wrote: > On Mon, 2009-12-14 at 06:21 -0500, Robert Haas wrote: >> On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs wrote: >> > On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: >> >> On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas >> >> wrote: >

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 06:21 -0500, Robert Haas wrote: > On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs wrote: > > On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: > >> On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas > >> wrote: > >> > * Please remove any spurious whitespace. "git diff -

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Robert Haas
On Mon, Dec 14, 2009 at 6:11 AM, Simon Riggs wrote: > On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: >> On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas >> wrote: >> > * Please remove any spurious whitespace.  "git diff --color" makes them >> > stand out like a sore thumb, in red. (pg

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Fujii Masao
On Mon, Dec 14, 2009 at 8:07 PM, Simon Riggs wrote: >> * Please remove any spurious whitespace.  "git diff --color" makes them >> stand out like a sore thumb, in red. (pgindent will fix them but always >> better to fix them before committing, IMO). > > What is your definition of spurious whitespac

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 11:09 +0100, Magnus Hagander wrote: > On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas > wrote: > > * Please remove any spurious whitespace. "git diff --color" makes them > > stand out like a sore thumb, in red. (pgindent will fix them but always > > better to fix them befo

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
Thanks for the further review, much appreciated. On Mon, 2009-12-14 at 11:54 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > Enclose latest version of Hot Standby. This is the "basic" patch, with > > no must-fix issues and no known bugs. Further additions will follow > > after commit, pr

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Magnus Hagander
On Mon, Dec 14, 2009 at 10:54, Heikki Linnakangas wrote: > * Please remove any spurious whitespace.  "git diff --color" makes them > stand out like a sore thumb, in red. (pgindent will fix them but always > better to fix them before committing, IMO). +1 in general, not particularly for this patch

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Heikki Linnakangas
Simon Riggs wrote: > Enclose latest version of Hot Standby. This is the "basic" patch, with > no must-fix issues and no known bugs. Further additions will follow > after commit, primarily around usability, which will include additional > control functions for use in testing. Various thoughts discus

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Peter Eisentraut
On mån, 2009-12-14 at 08:54 +, Simon Riggs wrote: > Wednesday because that seemed a good delay to allow review. Josh and I > had discussed the value of getting patch into Alpha3, so that was my > wish and aim. > > I'm not aware of any particular date for end of commitfest, though > possibly yo

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Simon Riggs
On Mon, 2009-12-14 at 10:11 +0200, Peter Eisentraut wrote: > On sön, 2009-12-13 at 19:20 +, Simon Riggs wrote: > > Barring resolving a few points and subject to even more testing, this > > is the version I expect to commit to CVS on Wednesday. > > To clarify: Did you pick Wednesday so that it

Re: [HACKERS] Hot Standby, release candidate?

2009-12-14 Thread Peter Eisentraut
On sön, 2009-12-13 at 19:20 +, Simon Riggs wrote: > Barring resolving a few points and subject to even more testing, this > is the version I expect to commit to CVS on Wednesday. To clarify: Did you pick Wednesday so that it gets included before the end of the commit fest (and thus into alpha3

Re: [HACKERS] Hot Standby, release candidate?

2009-12-13 Thread Simon Riggs
On Sun, 2009-12-13 at 15:45 -0500, Tom Lane wrote: > Simon Riggs writes: > > * NonTransactionalInvalidation logging has been removed following > > review, but AFAICS that means VACUUM FULL doesn't work correctly on > > catalog tables, which regrettably will be the only ones still standing > > even

Re: [HACKERS] Hot Standby, release candidate?

2009-12-13 Thread Tom Lane
Simon Riggs writes: > * NonTransactionalInvalidation logging has been removed following > review, but AFAICS that means VACUUM FULL doesn't work correctly on > catalog tables, which regrettably will be the only ones still standing > even after we apply VFI patch. Did I misunderstand the original i

Re: [HACKERS] Hot standby, recent changes

2009-12-07 Thread Simon Riggs
On Mon, 2009-12-07 at 10:02 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Sun, 2009-12-06 at 17:26 -0500, Robert Haas wrote: > > > >> For what it's worth, this doesn't seem particularly unlikely or > >> unusual to me. > > > > I don't know many people who shutdown both nodes of a hi

Re: [HACKERS] Hot standby, recent changes

2009-12-07 Thread Heikki Linnakangas
Simon Riggs wrote: > On Sun, 2009-12-06 at 17:26 -0500, Robert Haas wrote: > >> For what it's worth, this doesn't seem particularly unlikely or >> unusual to me. > > I don't know many people who shutdown both nodes of a highly available > application at the same time. If they did, I wouldn't expe

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 17:26 -0500, Robert Haas wrote: > For what it's worth, this doesn't seem particularly unlikely or > unusual to me. I don't know many people who shutdown both nodes of a highly available application at the same time. If they did, I wouldn't expect them to complain they couldn

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Dimitri Fontaine
Le 6 déc. 2009 à 23:26, Robert Haas a écrit : >>> Consider this scenario: >>> >>> 0. You have a master and a standby configured properly, and up and running. >>> 1. You shut down master for some reason. >>> 2. You restart standby. For some reason. Maybe by accident, or you want >>> to upgrade mino

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Robert Haas
On Sun, Dec 6, 2009 at 3:06 PM, Simon Riggs wrote: > On Sun, 2009-12-06 at 20:32 +0200, Heikki Linnakangas wrote: >> Simon Riggs wrote: >> > On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: >> >> 4. Need to handle the case where master is started up with >> >> wal_standby_info=true, sh

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 20:32 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: > >> 4. Need to handle the case where master is started up with > >> wal_standby_info=true, shut down, and restarted with > >> wal_standby_info=false, w

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Heikki Linnakangas
Simon Riggs wrote: > On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: >> 4. Need to handle the case where master is started up with >> wal_standby_info=true, shut down, and restarted with >> wal_standby_info=false, while the standby server runs continuously. And >> the code in StartupXL

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 10:51 +, Simon Riggs wrote: > > 5. You removed this comment from KnownAssignedXidsAdd: > > > > - /* > > -* XXX: We should check that we don't exceed maxKnownAssignedXids. > > -* Even though the hash table might hold a few more entries than that, > > -* we u

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 11:20 +, Simon Riggs wrote: > On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: > > > 3. The "Out of lock mem killer" in StandbyAcquireAccessExclusiveLock is > > quite harsh. It aborts all read-only transactions. It should be enough > > to kill just one random

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: > 3. The "Out of lock mem killer" in StandbyAcquireAccessExclusiveLock is > quite harsh. It aborts all read-only transactions. It should be enough > to kill just one random one, or maybe the one that's holding most locks. > Also, if ther

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: > 1. The XLogFlush() call you added to dbase_redo doesn't help where it > is. You need to call XLogFlush() after the *commit* record of the DROP > DATABASE. The idea is minimize the window where the files have already > been deleted but t

Re: [HACKERS] Hot standby, recent changes

2009-12-06 Thread Simon Riggs
On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote: > 2. "Allow RULEs ON INSERT, ON UPDATE and ON DELETE iff they generate > only SELECT statements that act INSTEAD OF the actual event." This > affects any read-only transaction, not just hot standby, so I think this > should be discussed

Re: [HACKERS] Hot standby, misc issues

2009-12-06 Thread Simon Riggs
On Sat, 2009-12-05 at 22:56 +0200, Heikki Linnakangas wrote: > So that RecordKnownAssignedTransactionIds() call needs to be put back. OK > BTW, if you want to resurrect the check in KnownAssignedXidsRemove(), > you also need to not complain before you reach the running-xacts record > and open up

Re: [HACKERS] Hot standby, misc issues

2009-12-05 Thread Heikki Linnakangas
Simon Riggs wrote: > On Fri, 2009-12-04 at 10:23 +0200, Heikki Linnakangas wrote: >>> @Heikki: Why is error checking in KnownAssignedXidsRemove() #ifdef'd >> out?? >> >> It's explained in the comment: >> /* XXX: This can still happen: If a transaction with a subtransaction >> * that haven't been

Re: [HACKERS] Hot standby, misc issues

2009-12-05 Thread Simon Riggs
On Fri, 2009-12-04 at 10:23 +0200, Heikki Linnakangas wrote: > > > @Heikki: Why is error checking in KnownAssignedXidsRemove() #ifdef'd > out?? > > It's explained in the comment: > /* XXX: This can still happen: If a transaction with a subtransaction > * that haven't been reported yet aborts, a

Re: [HACKERS] Hot Standby remaining issues

2009-12-04 Thread Kevin Grittner
Heikki Linnakangas wrote: > If the system is completely idle, and no WAL is written, we skip > xlog switches too, even if archive_timeout is set . It would be > pointless to create a stream of WAL files with no content except > for the XLOG_SWITCH records. It's not pointless if you want to mon

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 13:31 +0200, Heikki Linnakangas wrote: > b) seems much simpler to me. OK. Least ugly wins, but she ain't pretty. > > 2. (In HS recovery) When we see first commit record for the VF xid we > > commit the transaction in clog, yet maintain locks and KnownAssigned > > xids > >

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: > ISTM premature to remove all traces of VF from code. We may yet need it > for some reason, especially if doing so creates complex dependencies on > important features. Well, it's still in the repository. > So modified proposal looks like this > > 1. (In normal running) Provi

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 11:22 +0200, Heikki Linnakangas wrote: > > My answer is it doesn't, we will still have the problem noted above for > > catalog tables. So we still have a must-fix issue for HS, though that is > > no barrier to the new VF patch. > > I think the VACUUM FULL patch will have to

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 11:22 +0200, Heikki Linnakangas wrote: > Could you just mark the transaction as committed when you see the 1st > commit record, but leave the XID in the known-assigned list and not > release locks? That would emulate pretty closely what happens in the master. OK, works for m

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: > I've just reviewed the VACUUM FULL patch to see if it does all we need > it to do, i.e. does the removal of HS code match the new VF patch. Oh, good! > My answer is it doesn't, we will still have the problem noted above for > catalog tables. So we still have a must-fix issue

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote: > VACUUM FULL does a peculiar hack: once it's done moving tuples, and > before it truncates the relation, it calls RecordTransactionCommit to > mark the transaction as committed in clog and WAL, but the transaction > is still kept open i

Re: [HACKERS] Hot Standby remaining issues

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: > On Fri, 2009-12-04 at 10:37 +0200, Heikki Linnakangas wrote: >> Regarding this item from the wiki page: >>> The "standby delay" is measured as current timestamp - timestamp of last >>> replayed commit record. If there's little activity in the master, that can >>> lead to surp

Re: [HACKERS] Hot Standby remaining issues

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 10:37 +0200, Heikki Linnakangas wrote: > Regarding this item from the wiki page: > > The "standby delay" is measured as current timestamp - timestamp of last > > replayed commit record. If there's little activity in the master, that can > > lead to surprising results. For ex

Re: [HACKERS] Hot Standby remaining issues

2009-12-04 Thread Heikki Linnakangas
Regarding this item from the wiki page: > The "standby delay" is measured as current timestamp - timestamp of last > replayed commit record. If there's little activity in the master, that can > lead to surprising results. For example, imagine that max_standby_delay is > set to 8 hours. The stand

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-03 Thread Simon Riggs
On Wed, 2009-11-25 at 03:12 +, Greg Stark wrote: > On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > > As long as there's not anything the master actually does differently > > then I can't see where there'd be any performance testing to do. What's > > bothering me about this is that it seems

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Heikki Linnakangas
Simon Riggs wrote: > Hmm, what happens if someone enables wal_standby_info in postgresql.conf > while the server is shutdown. It would still be a valid starting point > in that case. Yeah, true. > I'll just make a note, I think. Yeah, a manual (or automatic, if you just wait) checkpoint will pro

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Simon Riggs
On Wed, 2009-12-02 at 16:41 +, Simon Riggs wrote: > On Tue, 2009-12-01 at 20:26 +0200, Heikki Linnakangas wrote: > > Simon Riggs wrote: > > > commit 02c3eadb766201db084b668daa271db4a900adc9 > > > Author: Simon Riggs > > > Date: Sat Nov 28 06:23:33 2009 + > > > > > > Added wal_standb

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Simon Riggs
On Tue, 2009-12-01 at 20:26 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > commit 02c3eadb766201db084b668daa271db4a900adc9 > > Author: Simon Riggs > > Date: Sat Nov 28 06:23:33 2009 + > > > > Added wal_standby_info GUC to turn RM_STANDBY_ID messages on/off. > > Various co

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Heikki Linnakangas
Simon Riggs wrote: > On Wed, 2009-12-02 at 12:49 +0200, Heikki Linnakangas wrote: > >> If a read-only transaction holds a lot of locks, consuming so much >> lock space that there's none left for the startup process to hold the >> lock it wants, it will abort and bring down postmaster. The patch >>

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Simon Riggs
On Wed, 2009-12-02 at 12:49 +0200, Heikki Linnakangas wrote: > If a read-only transaction holds a lot of locks, consuming so much > lock space that there's none left for the startup process to hold the > lock it wants, it will abort and bring down postmaster. The patch > attempts to kill any confl

Re: [HACKERS] Hot Standby remaining issues

2009-12-02 Thread Heikki Linnakangas
Heikki Linnakangas wrote: > Simon Riggs wrote: >> @@ -654,10 +656,13 @@ LockAcquire(const LOCKTAG *locktag, >> elog(PANIC, "lock table corrupted"); >> } >> LWLockRelease(partitionLock); >> -ereport(ERROR, >> -

Re: [HACKERS] Hot Standby remaining issues

2009-12-01 Thread Heikki Linnakangas
Simon Riggs wrote: > commit 02c3eadb766201db084b668daa271db4a900adc9 > Author: Simon Riggs > Date: Sat Nov 28 06:23:33 2009 + > > Added wal_standby_info GUC to turn RM_STANDBY_ID messages on/off. > Various comments added also. > This patch makes it unsafe to start hot standby mode

Re: [HACKERS] Hot Standby remaining issues

2009-11-30 Thread Heikki Linnakangas
Simon Riggs wrote: > @@ -654,10 +656,13 @@ LockAcquire(const LOCKTAG *locktag, > elog(PANIC, "lock table corrupted"); > } > LWLockRelease(partitionLock); > - ereport(ERROR, > - (errcode(ERRCODE_OUT_OF_

Re: [HACKERS] Hot Standby remaining issues

2009-11-27 Thread Simon Riggs
On Wed, 2009-11-25 at 13:00 +0200, Heikki Linnakangas wrote: > I've put up a wiki page with the issues I see with the patch as it > stands. They're roughly categorized by seriousness. > > http://wiki.postgresql.org/wiki/Hot_Standby_TODO > > New issues can and probably will still pop up, let's add

Re: [HACKERS] Hot Standby and cancelling idle queries

2009-11-27 Thread Simon Riggs
On Thu, 2009-11-26 at 08:30 +0200, Heikki Linnakangas wrote: > I suspect you missed the context of this change. It's about the code > in tablespc.c, to kill all backends that might have a temporary file > in a tablespace that's being dropped. It's not about tuple visibility > but temporary files.

Re: [HACKERS] Hot Standby and cancelling idle queries

2009-11-25 Thread Heikki Linnakangas
Simon Riggs wrote: > Recent change: > > An idle-in-transaction transaction can also hold a temporary file. Think > of an open cursor, for example. Therefore, remove the distinction > between CONFLICT_MODE_ERROR and CONFLICT_MODE_ERROR_IF_NOT_IDLE, > idle-in-transaction backends need to be killed t

Re: [HACKERS] Hot Standby and cancelling idle queries

2009-11-25 Thread Tom Lane
Simon Riggs writes: > An idle-in-transaction transaction can also hold a temporary file. Think > of an open cursor, for example. Therefore, remove the distinction > between CONFLICT_MODE_ERROR and CONFLICT_MODE_ERROR_IF_NOT_IDLE, > idle-in-transaction backends need to be killed too when a tablespa

Re: [HACKERS] Hot Standby remaining issues

2009-11-25 Thread Simon Riggs
On Wed, 2009-11-25 at 13:00 +0200, Heikki Linnakangas wrote: > I've put up a wiki page with the issues I see with the patch as it > stands. They're roughly categorized by seriousness. > > http://wiki.postgresql.org/wiki/Hot_Standby_TODO > > New issues can and probably will still pop up, let's add

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Greg Stark
On Wed, Nov 25, 2009 at 3:26 AM, Tom Lane wrote: > Greg Stark writes: >> Well the only thing that's been discussed is having vacuum require a >> minimum age before considering a transaction visible to all to reduce >> the chance of conflicts on cleanup records. > > [ shrug... ]  Call me Cassandra

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Wed, 2009-11-25 at 03:12 +, Greg Stark wrote: > On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > > As long as there's not anything the master actually does differently > > then I can't see where there'd be any performance testing to do. What's > > bothering me about this is that it seems

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Tom Lane
Greg Stark writes: > Well the only thing that's been discussed is having vacuum require a > minimum age before considering a transaction visible to all to reduce > the chance of conflicts on cleanup records. [ shrug... ] Call me Cassandra. I am not concerned about what has or has not been discu

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Greg Stark
On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > As long as there's not anything the master actually does differently > then I can't see where there'd be any performance testing to do.  What's > bothering me about this is that it seems likely that we'll find places > where the master has to do t

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Tom Lane
Simon Riggs writes: > Tom Lane wrote: >> There's no equivalent of XLogArchivingActive()? > We've tried hard to have it "just work". But I wonder whether we should > have a parameter to allow performance testing on the master? If nobody > finds any issues then we can remove it again, or at least m

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Sat, 2009-11-21 at 23:00 +0200, Heikki Linnakangas wrote: > Tom Lane wrote: > > Heikki Linnakangas writes: > >> Tom Lane wrote: > >>> There's no equivalent of XLogArchivingActive()? > > > >> XLogArchivingMode() == false enables us to skip WAL-logging in > >> operations like CLUSTER or COPY, wh

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote: > That causes some headaches for Hot Standby I say leave HS as it is and we can clean up when we do the VFectomy. It isn't really a headache, the code works easily enough. I agree its ugly and it should eventually be removed. Let's no

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Hannu Krosing
On Tue, 2009-11-24 at 09:24 +0200, Heikki Linnakangas wrote: > Greg Smith wrote: > > Heikki Linnakangas wrote: > >> So I guess what I'm asking is: Does anyone see any show-stoppers in > >> removing VACUUM FULL > > Here's the disclaimers attached to the new VACUUM REPLACE implementation > > from Ita

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Heikki Linnakangas
Greg Smith wrote: > Heikki Linnakangas wrote: >> So I guess what I'm asking is: Does anyone see any show-stoppers in >> removing VACUUM FULL > Here's the disclaimers attached to the new VACUUM REPLACE implementation > from Itagaki: > > "We still need traditional VACUUM FULL behavior for system cat

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Heikki Linnakangas
Itagaki Takahiro wrote: > VACUUM FULL is still reserved for system catalogs in my patch > because we cannot modify relfilenodes for the catalog tables. > Do you have solutions for it? Tom had an idea on that: http://archives.postgresql.org/message-id/19750.1252094...@sss.pgh.pa.us -- Heikki L

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Itagaki Takahiro
Heikki Linnakangas wrote: > So I guess what I'm asking is: Does anyone see any show-stoppers in > removing VACUUM FULL, and does anyone want to step up to the plate and > promise to do it before release? I'm working on "New VACUUM FULL" patch, that shrinks tables using CLUSTER-like rewrites. ht

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> Tom Lane wrote: >>> There's no equivalent of XLogArchivingActive()? > >> XLogArchivingMode() == false enables us to skip WAL-logging in >> operations like CLUSTER or COPY, which is a big optimization. I don't >> see anything like that in Hot Standby

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> There's no equivalent of XLogArchivingActive()? > XLogArchivingMode() == false enables us to skip WAL-logging in > operations like CLUSTER or COPY, which is a big optimization. I don't > see anything like that in Hot Standby. There is a few small th

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> Tom Lane wrote: >>> I don't see much problem with rejecting VAC FULL in a HS master, >>> whether or not it gets removed altogether. Why not just do that >>> rather than write a lot of kluges? > >> Hmm. At the moment, no action is required in the ma

<    1   2   3   4   5   6   7   8   9   10   >