Re: [HACKERS] Hot Backup with rsync fails at pg_clog if under load

2011-10-23 Thread Florian Pflug
On Oct23, 2011, at 22:48 , Daniel Farina wrote: > It doesn't seem meaningful for StartupCLOG (or, indeed, any of the > hot-standby path functionality) to be called before that code is > executed, but it is anyway right now. I think the idea is to check that the CLOG part which recovery *won't* ove

Re: [HACKERS] Hot Backup with rsync fails at pg_clog if under load

2011-10-23 Thread Florian Pflug
On Oct24, 2011, at 01:27 , Simon Riggs wrote: > FATAL: could not access status of transaction 21110784 > which, in pg_subtrans, is the first xid on a new subtrans page. So we > have missed zeroing a page. > > pg_control shows ... Latest checkpoint's oldestActiveXID: 2111 > which shows quite

[HACKERS] autovacuum and orphaned large objects

2011-10-23 Thread Euler Taveira de Oliveira
Hi, The main point of autovacuum is maintenance tasks. Currently, it executes VACUUM and ANALYZE commands. I want to propose that we incorporate vacuumlo functionality into it. While dealing with large objects (LO), we have lo contrib module that helps with LO maintenance but has some limitati

Re: [HACKERS] Hot Backup with rsync fails at pg_clog if under load

2011-10-23 Thread Simon Riggs
On Sun, Oct 23, 2011 at 9:48 PM, Daniel Farina wrote: > Having digged at this a little -- but not too much -- the problem > seems to be that postgres is reading the commit logs way, way too > early, that is to say, before it has played enough WAL to be > 'consistent' (the WAL between pg_start and

Re: [HACKERS] Hot Backup with rsync fails at pg_clog if under load

2011-10-23 Thread Simon Riggs
On Sun, Oct 16, 2011 at 2:33 AM, Chris Redekop wrote: > pg_subtrans: http://pastebin.com/qAXEHAQt I confirm this as a HS issue and will investigate from here. FATAL: could not access status of transaction 21110784 which, in pg_subtrans, is the first xid on a new subtrans page. So we have misse

Re: [HACKERS] EXECUTE tab completion

2011-10-23 Thread Tom Lane
Andreas Karlsson writes: > Thanks for cleaning up the code to some sanity, I should have done so > myself when I noticed the problem. > A new version is attached. Committed with minor adjustments --- I didn't see any need to make this wait for the next commitfest. regar

Re: [HACKERS] So, is COUNT(*) fast now?

2011-10-23 Thread Jeff Janes
On Fri, Oct 21, 2011 at 12:52 PM, Robert Haas wrote: > > Also, this line is kind of expensive: > >        if (!visibilitymap_test(scandesc->heapRelation, >                                ItemPointerGetBlockNumber(tid), >                                &node->ioss_VMBuffer)) > > Around 2%.  But I d

[HACKERS] Index only scans and visibilitymap.c

2011-10-23 Thread Jeff Janes
With index only scans, the comments in src/backend/access/heap/visibilitymap.c are probably out of date, starting with: "Currently, the visibility map is only used as a hint" Also, is there a discussion of how and why index-only scans is safe? i.e. what lock, if any, has to be held while nodeInd

Re: [HACKERS] Hot Backup with rsync fails at pg_clog if under load

2011-10-23 Thread Simon Riggs
On Sun, Oct 23, 2011 at 9:48 PM, Daniel Farina wrote: > On Mon, Oct 17, 2011 at 11:30 PM, Chris Redekop wrote: >> Well, on the other hand maybe there is something wrong with the data. >>  Here's the test/steps I just did - >> 1. I do the pg_basebackup when the master is under load, hot slave now

Re: [HACKERS] termination of backend waiting for sync rep generates a junk log message

2011-10-23 Thread Tom Lane
Robert Haas writes: > Well, there is a general problem that anything which throws an ERROR > too late in the commit path is Evil; and sync rep makes that worse to > the extent that it adds more stuff late in the commit path, but it > didn't invent the problem. BTW, it strikes me that if we want t

Re: [HACKERS] So, is COUNT(*) fast now?

2011-10-23 Thread Tom Lane
Jeff Janes writes: > count(*) and sum(1) do different things internally, and in my hands > sum(1) is ~10% slower. > I don't know how to dump the output of ExecBuildProjectionInfo into a > human readable form, so I don't know the basis of the difference. But > I wonder if using count(*) would lowe

Re: [HACKERS] Hot Backup with rsync fails at pg_clog if under load

2011-10-23 Thread Daniel Farina
On Mon, Oct 17, 2011 at 11:30 PM, Chris Redekop wrote: > Well, on the other hand maybe there is something wrong with the data. >  Here's the test/steps I just did - > 1. I do the pg_basebackup when the master is under load, hot slave now will > not start up but warm slave will. > 2. I start a warm

Re: [HACKERS] So, is COUNT(*) fast now?

2011-10-23 Thread Jeff Janes
On Fri, Oct 21, 2011 at 12:07 PM, Robert Haas wrote: > On Fri, Oct 21, 2011 at 2:33 PM, Tom Lane wrote: >>> I think HeapTupleSatisfiesMVCC is probably being skipped anyway in >>> this case, since all the heap pages should be PD_ALL_VISIBLE. >> >> Proves my point ;-) ... you're comparing a code pa

Re: [HACKERS] termination of backend waiting for sync rep generates a junk log message

2011-10-23 Thread Tom Lane
Robert Haas writes: > On Tue, Oct 18, 2011 at 11:27 PM, Tom Lane wrote: >> One thing worth asking is why we're willing to violate half a dozen >> different coding rules if we see ProcDiePending, yet we're perfectly >> happy to rely on the client understanding a WARNING for the >> QueryCancelPendi

Re: [HACKERS] Visual Studio 2010/Windows SDK 7.1 support

2011-10-23 Thread Brar Piening
thready wrote: [...] I don't know how to apply this patch. What's the exe that runs this patch? Will it ask me where the source folder root is when I run it? As the archives seem to be unreachable at the moment I'm copying text from the following message which I would otherwise ask you to read

Re: [HACKERS] [PATCH] Deferrable unique constraints vs join removal -- bug?

2011-10-23 Thread Tom Lane
Marti Raudsepp writes: > On Sun, Oct 23, 2011 at 06:44, Tom Lane wrote: >> Yeah, this seems like the right fix. > Oh, that sounds pretty obvious now that you mention it. :) > I will try to come up with a new patch in a few days (haven't had too > much time lately). Oh, I did it already.

Re: [HACKERS] [PATCH] Deferrable unique constraints vs join removal -- bug?

2011-10-23 Thread Marti Raudsepp
On Sun, Oct 23, 2011 at 06:44, Tom Lane wrote: > Robert Haas writes: >> I think maybe what we should do is add >> an "immediate" field to IndexOptInfo, mirroring the existing unique >> flag, and have get_relation_info() populate it from indimmediate, and >> then make relation_has_unique_index() d

Re: [HACKERS] Visual Studio 2010/Windows SDK 7.1 support

2011-10-23 Thread thready
Hi Brar, Thanks for sharing your work compiling Postgres with VS2010. I see that there's a patch file you created to fix the files that need fixing to do so, but I don't know how to apply this patch. What's the exe that runs this patch? Will it ask me where the source folder root is when I run i

Re: [HACKERS] synchronized snapshots

2011-10-23 Thread Thom Brown
On 23 October 2011 03:15, Tom Lane wrote: > Thom Brown writes: >> Can I ask why it doesn't return the same snapshot ID each time? >> Surely it can't change since you can only export the snapshot of a >> serializable or repeatable read transaction? > > No, that's incorrect.  You can export from a