Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Simon Riggs
On 7 May 2013 01:18, Stephen Frost sfr...@snowman.net wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: If anybody really wanted to fix pg_dump, they could do. If that was so important, why block this patch, but allow parallel pg_dump to be committed without it? Because parallel pg_dump

[HACKERS] [patch] PSQLDIR not passed to pg_regress in contrib/pg_upgrade/test.sh

2013-05-07 Thread Christoph Berg
make check-world in 9.3beta1 fails if you don't actually have 9.3 installed. In contrib/pg_upgrade/test.sh, it will try pg_regress --psqldir=/usr/lib/postgresql/9.3/bin which doesn't exist. +

[HACKERS] [patch] Adding EXTRA_REGRESS_OPTS to all pg_regress invocations

2013-05-07 Thread Christoph Berg
make check supports EXTRA_REGRESS_OPTS to pass extra options to pg_regress, but all the other places where pg_regress is used do not allow this. The attached patch adds EXTRA_REGRESS_OPTS to Makefile.global.in (for contrib modules) and two more special Makefiles (isolation and pg_upgrade). The

Re: [HACKERS] spoonbill vs. -HEAD

2013-05-07 Thread Stefan Kaltenbrunner
On 04/04/2013 02:18 AM, Tom Lane wrote: Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: On 04/03/2013 12:59 AM, Tom Lane wrote: BTW, on further thought it seems like maybe this is an OpenBSD bug, at least in part: what is evidently happening is that the temporary blockage of SIGINT during

[HACKERS] XLogFlush invoked about twice as much after 9.2 group commit enhancement

2013-05-07 Thread Amit Langote
Hello, I have been trying to understand how group commit implementation works the way it does after 9.2 group commit enhancement patch (9b38d46d9f5517dab67dda1dd0459683fc9cda9f on REL9_2_STABLE). I admit it's a pretty old commit though I seek some clarification as to how it provides the

Re: [HACKERS] [PATCH] add long options to pgbench (submission 1)

2013-05-07 Thread Fabien COELHO
Hello Robert, This very minor patch adds a corresponding long option to all short (one letter) options of pgbench. [...] I don't really have an opinion on whether this is worth doing, but we'd probably want to update all of our client utilities, not just pgbench, if we did. The current

[HACKERS] Fast promotion failure

2013-05-07 Thread Heikki Linnakangas
While testing the bug from the Assertion failure at standby promotion, I bumped into a different bug in fast promotion. When the first checkpoint after fast promotion is performed, there is no guarantee that the checkpointer process is running with the correct, new, ThisTimeLineID. In

Re: [HACKERS] Recovery target 'immediate'

2013-05-07 Thread Simon Riggs
On 3 May 2013 14:40, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03.05.2013 16:29, Bruce Momjian wrote: On Fri, May 3, 2013 at 01:02:08PM +0200, Cédric Villemain wrote: This changes the existing API which will confuse people that know it and invalidate everything written in

Re: [HACKERS] Recovery target 'immediate'

2013-05-07 Thread Heikki Linnakangas
On 07.05.2013 15:38, Simon Riggs wrote: On 3 May 2013 14:40, Heikki Linnakangashlinnakan...@vmware.com wrote: If we want to avoid adding a new option for this, how about a magic restore point called consistent or immediate: recovery_target_name='immediate' That would stop recovery right

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: All of which I think I agree with, but I don't agree with the conclusion that this larger window is somehow acceptable because there's a very small window (one which can't be made any smaller, today..) which exists today. The window isn't

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: On 7 May 2013 01:18, Stephen Frost sfr...@snowman.net wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: If anybody really wanted to fix pg_dump, they could do. If that was so important, why block this patch, but allow parallel pg_dump to be

Re: [HACKERS] Failing start-up archive recovery at Standby mode in PG9.2.4

2013-05-07 Thread Heikki Linnakangas
On 26.04.2013 11:51, KONDO Mitsumasa wrote: Hi, I discavered the problem cause. I think taht horiguchi's discovery is another problem... Problem has CreateRestartPoint. In recovery mode, PG should not WAL record. Because PG does not know latest WAL file location. But in this problem case,

[HACKERS] Re: [BUGS] BUG #8043: 9.2.4 doesn't open WAL files from archive, only looks in pg_xlog

2013-05-07 Thread Heikki Linnakangas
On 08.04.2013 18:58, Jeff Bohmer wrote: On Apr 6, 2013, at 1:24 PM, Jeff Janesjeff.ja...@gmail.com wrote: On Sat, Apr 6, 2013 at 1:24 AM, Heikki Linnakangas hlinnakan...@vmware.comwrote: Perhaps we should improve the documentation to make it more explicit that backup_label must be included

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Andres Freund
Hi, On 2013-05-07 08:54:54 -0400, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: All of which I think I agree with, but I don't agree with the conclusion that this larger window is somehow acceptable because there's a very small window (one which can't be made

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2013-05-07 08:54:54 -0400, Stephen Frost wrote: Agreed- but it also isn't currently possible to make it any smaller. Uh. Why not? I think this is what needs to be fixed instead of making the hole marginally smaller elsewhere. If we're

[HACKERS] XLogFlush invoked about twice as many times after 9.2 group commit enhancement

2013-05-07 Thread Amit Langote
Hello, I have been trying to understand how group commit implementation works the way it does after 9.2 group commit enhancement patch (9b38d46d9f5517dab67dda1dd0459683fc9cda9f on REL9_2_STABLE). I have observed some behavior in this regard that I could not understand. Profiling results show

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Greg Stark
On Tue, May 7, 2013 at 1:54 PM, Stephen Frost sfr...@snowman.net wrote: I believe the point that Tom is making is that we shouldn't paint ourselves into a corner by letting users provide old snapshots to pg_dump which haven't acquired any of the necessary locks. The goal, at least as I read

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Stephen Frost
Greg, * Greg Stark (st...@mit.edu) wrote: One natural way to do it would be to make an option to pg_dump which caused it to do all the normal pre-dump things it would normally do, then export a snapshot and wait for the user. (Alternately it could even create a prepared transaction which iirc

Re: [HACKERS] XLogFlush invoked about twice as many times after 9.2 group commit enhancement

2013-05-07 Thread Peter Geoghegan
On Tue, May 7, 2013 at 8:13 AM, Amit Langote amitlangot...@gmail.com wrote: Profiling results show that XLogFlush() is called about twice as many times after this patch while for XLogWrite() count remains about same as before. This patch modifies XLogFlush() such that it offers the said

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Tom Lane
Greg Stark st...@mit.edu writes: [ ideas about dumping some past state of a table ] If I try to access a table whose schema has changed then I might use the wrong tupledesc and see rows that don't decode properly. That would be a disaster. Can we protect against that by noticing that the

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Robert Haas
On Mon, May 6, 2013 at 1:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: Exported snapshots allow you to coordinate a number of actions together, so they all see a common view of the database. So this patch allows a very general approach to this, much more so than pg_dump allows currently since the

Re: [HACKERS] Recovery target 'immediate'

2013-05-07 Thread Fujii Masao
On Tue, May 7, 2013 at 9:38 PM, Simon Riggs si...@2ndquadrant.com wrote: On 3 May 2013 14:40, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03.05.2013 16:29, Bruce Momjian wrote: On Fri, May 3, 2013 at 01:02:08PM +0200, Cédric Villemain wrote: This changes the existing API which will

Re: [HACKERS] Recovery target 'immediate'

2013-05-07 Thread Simon Riggs
On 7 May 2013 13:50, Heikki Linnakangas hlinnakan...@vmware.com wrote: Can I suggest that we discuss a range of related changes together? So we have a roadmap of agreed changes in this area. That will be more efficient than discussing each one individually; often each one makes sense only as

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: On Mon, May 6, 2013 at 1:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm afraid that this is institutionalizing a design deficiency in pg_dump; namely that it takes its snapshot before acquiring locks. Ideally that would happen the other way around. I

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Andres Freund
On 2013-05-07 11:01:48 -0400, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2013-05-07 08:54:54 -0400, Stephen Frost wrote: Agreed- but it also isn't currently possible to make it any smaller. Uh. Why not? I think this is what needs to be fixed instead of

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Andres Freund
On 2013-05-07 16:50:52 +0100, Greg Stark wrote: What's the worst case for using an old snapshot? If I try to access a table that doesn't exist any longer I'll get an error. That doesn't really seem that bad for the use case I described. It's worse for the full table dump but for an explicit

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-07 Thread Robert Haas
On Mon, May 6, 2013 at 5:04 PM, Jeff Davis pg...@j-davis.com wrote: On Mon, 2013-05-06 at 15:31 -0400, Robert Haas wrote: On Wed, May 1, 2013 at 3:04 PM, Jeff Davis pg...@j-davis.com wrote: Regardless, you have a reasonable claim that my patch had effects that were not necessary. I have

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread bricklen
On Tue, May 7, 2013 at 10:02 AM, Dimitri Fontaine dimi...@2ndquadrant.frwrote: Rather than take some locks, you can now prevent the database objects from changing with an event trigger. pg_dump could install that event trigger in a preparing transaction, then do its work as currently, then

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-07 Thread Jeff Davis
On Tue, 2013-05-07 at 13:20 -0400, Robert Haas wrote: Hmm. Rereading your last email, I see your point: since we now have HEAP_XLOG_VISIBLE, this is much less of an issue than it would have been before. I'm still not convinced that simplifying that code is a good idea, but maybe it doesn't

Re: [HACKERS] [COMMITTERS] pgsql: Fix permission tests for views/tables proven empty by constraint

2013-05-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, May 1, 2013 at 6:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Fix permission tests for views/tables proven empty by constraint exclusion. I believe that this commit is responsible for the fact that the following test case now crashes the server:

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Andres Freund
On 2013-05-06 13:07:17 -0400, Tom Lane wrote: I'm afraid that this is institutionalizing a design deficiency in pg_dump; namely that it takes its snapshot before acquiring locks. I have suggested this before, but if pg_dump would use SELECT FOR SHARE in the queries it uses to build DDL it would

Re: [HACKERS] \watch stuck on execution of commands returning no tuples

2013-05-07 Thread Robert Haas
On Thu, May 2, 2013 at 7:53 PM, Bruce Momjian br...@momjian.us wrote: OK, what other database supports British spelling of commands? Can we call this a compatibility feature. ;-) The feature has been there since 2000: commit ebe0b236909732c75d665c73363bd4ac7a7aa138 Author:

Re: [HACKERS] Assertion failure at standby promotion

2013-05-07 Thread Heikki Linnakangas
On 06.05.2013 13:08, Heikki Linnakangas wrote: On 03.05.2013 18:17, Fujii Masao wrote: Hi, I got the following assertion failure when I promoted the standby. 2013-05-04 00:12:31 JST sby1 LOG: received promote request 2013-05-04 00:12:31 JST sby1 FATAL: terminating walreceiver process due to

Re: [HACKERS] \watch stuck on execution of commands returning no tuples

2013-05-07 Thread Ian Lawrence Barwick
2013/5/8 Robert Haas robertmh...@gmail.com: On Thu, May 2, 2013 at 7:53 PM, Bruce Momjian br...@momjian.us wrote: OK, what other database supports British spelling of commands? Can we call this a compatibility feature. ;-) The feature has been there since 2000: commit

Re: [HACKERS] \watch stuck on execution of commands returning no tuples

2013-05-07 Thread Robert Haas
On Tue, May 7, 2013 at 2:36 PM, Ian Lawrence Barwick barw...@gmail.com wrote: As a British native speaker involved in translating some PostgreSQL-related Japanese text, all I can say is yes please. (Although for true Japanese support, the grammar would have to be pretty much reversed, with the

Re: [HACKERS] pg_dump --snapshot

2013-05-07 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: It helps in that once we have the lock, things aren't changing under us. The closer we can keep that to when the transaction starts, the better.. If you look at my example the timing where we take the snapshot isn't the problem. While we wait

[HACKERS] issues with dropped columns in plpgsql code again

2013-05-07 Thread Pavel Stehule
Hello One user reported a issues with triggers related to dropped columns. When I checked his code, I found a more possible problems. He use a trigger in form $$ DECLARE somevar targettable; BEGIN somevar := NEW; // do some with somevar; RETURN somevar; END; $$ When I dropped column (I

Re: [HACKERS] issues with dropped columns in plpgsql code again

2013-05-07 Thread Pavel Stehule
sorry my test create table f1(a int, b int, c varchar, dropped_column numeric, d varchar); create or replace function f1_trg() returns trigger as $$ declare _f1_var f1; begin raise notice 'run trigger'; _f1_var := new; return _f1_var; end; $$ language plpgsql; create trigger xx before

Re: [HACKERS] XLogFlush invoked about twice as much after 9.2 group commit enhancement

2013-05-07 Thread Jeff Janes
On Tue, May 7, 2013 at 2:20 AM, Amit Langote amitlangot...@gmail.comwrote: Hello, I have been trying to understand how group commit implementation works the way it does after 9.2 group commit enhancement patch (9b38d46d9f5517dab67dda1dd0459683fc9cda9f on REL9_2_STABLE). I admit it's a

Re: [HACKERS] issues with dropped columns in plpgsql code again

2013-05-07 Thread Szymon Guz
On 7 May 2013 21:23, Pavel Stehule pavel.steh...@gmail.com wrote: sorry my test create table f1(a int, b int, c varchar, dropped_column numeric, d varchar); create or replace function f1_trg() returns trigger as $$ declare _f1_var f1; begin raise notice 'run trigger'; _f1_var :=

[HACKERS] local_preload_libraries logspam

2013-05-07 Thread Peter Geoghegan
It seems like an oversight to me that local_preload_libraries causes a new log message to appear each time a new connection is established. Is there any sympathy for the view that we should have a way of turning this off, or simply not log such messages? We could still have it appear at DEBUG2

Re: [HACKERS] about index inheritance

2013-05-07 Thread Robert Haas
On Mon, May 6, 2013 at 9:30 AM, Vincenzo Melandri vmelan...@imolinfo.it wrote: Hi guys, My first post here :) I stumbled into the same problem as this guy http://www.postgresql.org/message-id/4be2835a.5020...@cybertec.at , so since I have some spare time recently, I've set-up the development

Re: [HACKERS] Add some regression tests for SEQUENCE

2013-05-07 Thread Robins Tharakan
Hi, Have provided an updated patch as per Fabien's recent response on Commitfest site. Any and all feedback is appreciated. -- Robins Tharakan regress_sequence_v4.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Re: [COMMITTERS] pgsql: Fix permission tests for views/tables proven empty by constraint

2013-05-07 Thread Greg Stark
On Tue, May 7, 2013 at 6:59 PM, Tom Lane t...@sss.pgh.pa.us wrote: The main downside of this is that fully dummy appendrels (those with no live children at all, such as in your example) wouldn't be recognized by IS_DUMMY_PATH, so the quality of planning in the outer query would be slightly

Re: [HACKERS] Patch to add regression tests for SCHEMA

2013-05-07 Thread Robins Tharakan
Hi, Here is an updated patch that uses different schema / role names for different tests (as per commitfest site feedback). -- Robins Tharakan On 18 March 2013 17:16, robins thara...@gmail.com wrote: Hi, Attached is an updated patch that uses better schema / role names. -- Robins

Re: [HACKERS] local_preload_libraries logspam

2013-05-07 Thread Peter Eisentraut
On Tue, 2013-05-07 at 14:28 -0700, Peter Geoghegan wrote: It seems like an oversight to me that local_preload_libraries causes a new log message to appear each time a new connection is established. It is correct in my view (but perhaps I have just gotten used to it), but I wouldn't mind if you

Re: [HACKERS] Make targets of doc links used by phpPgAdmin static

2013-05-07 Thread Peter Eisentraut
On Tue, 2013-05-07 at 00:32 -0500, Karl O. Pinc wrote: Attached is a documentation patch against head which makes static the targets of the on-line PG html documentation that are referenced by the phpPgAdmin help system.e done -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [COMMITTERS] pgsql: Fix permission tests for views/tables proven empty by constraint

2013-05-07 Thread Tom Lane
Greg Stark st...@mit.edu writes: If we just reverted your fix and didn't fix it in 9.2 that would also fix the crash right? The bug was only that it leaked the fact that the view was provably empty from the definition? Well, it might fail to report a permissions violation when the

Re: [HACKERS] Failing start-up archive recovery at Standby mode in PG9.2.4

2013-05-07 Thread KONDO Mitsumasa
(2013/05/07 22:40), Heikki Linnakangas wrote: On 26.04.2013 11:51, KONDO Mitsumasa wrote: So I fix CreateRestartPoint at branching point of executing MinRecoveryPoint. It seems to fix this problem, but attached patch is plain. I didn't understand this. I committed a fix for the issue where

Re: [HACKERS] local_preload_libraries logspam

2013-05-07 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On Tue, 2013-05-07 at 14:28 -0700, Peter Geoghegan wrote: It seems like an oversight to me that local_preload_libraries causes a new log message to appear each time a new connection is established. It is correct in my view (but perhaps I have just

Re: [HACKERS] XLogFlush invoked about twice as many times after 9.2 group commit enhancement

2013-05-07 Thread Amit Langote
Why is that surprising? Most of those XLogFlush() calls will recheck the flushed-up-to point, and realize that another backend assumed the role of group commit leader, and flushed their WAL for them, so aside from the wait, the call to XLogFlush is cheap for that individual backend. It's

Re: [HACKERS] Make targets of doc links used by phpPgAdmin static

2013-05-07 Thread Alvaro Herrera
Peter Eisentraut wrote: On Tue, 2013-05-07 at 00:32 -0500, Karl O. Pinc wrote: Attached is a documentation patch against head which makes static the targets of the on-line PG html documentation that are referenced by the phpPgAdmin help system.e done I wonder about backpatching this to

Re: [HACKERS] local_preload_libraries logspam

2013-05-07 Thread Peter Geoghegan
On Tue, May 7, 2013 at 7:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: On the other hand, if we have it as DEBUG2 in the EXEC_BACKEND code path, I would be willing to argue that that's too low. If you're starting to feel a need to inquire into the backend's behavior, knowing about loaded modules

Re: [HACKERS] XLogFlush invoked about twice as many times after 9.2 group commit enhancement

2013-05-07 Thread Peter Geoghegan
On Tue, May 7, 2013 at 7:52 PM, Amit Langote amitlangot...@gmail.com wrote: So, any rise in number of XLogFlush() calls should roughly be accounted for by increased throughput. Am I right in interpreting it this way? I think so. There certainly isn't any question that the increased throughput

Re: [HACKERS] Patch to add regression tests for SCHEMA

2013-05-07 Thread Robert Haas
On Tue, May 7, 2013 at 7:26 PM, Robins Tharakan thara...@gmail.com wrote: Here is an updated patch that uses different schema / role names for different tests (as per commitfest site feedback). I'm not sure what's going on here. Reviews are to be posted to pgsql-hackers, and then linked from

Re: [HACKERS] XLogFlush invoked about twice as much after 9.2 group commit enhancement

2013-05-07 Thread Amit Langote
Are you sure you properly cleared out the stats between profiling sessions? Also, XLogFlush gets called by background processes like autovac, checkpointer and bgwriter, in addition to being called by committing processes. If one profiled session contained a checkpoint and other did not, or

Re: [HACKERS] XLogFlush invoked about twice as much after 9.2 group commit enhancement

2013-05-07 Thread Peter Geoghegan
On Tue, May 7, 2013 at 12:48 PM, Jeff Janes jeff.ja...@gmail.com wrote: Anyway, I don't see this behavior change when turning on wal_debug and looking in the logfiles for 'xlog flush request' messages. That could have everything to do with the hardware you're using. In general, the higher the

[HACKERS] remove src/tools/make_keywords?

2013-05-07 Thread Peter Eisentraut
It doesn't look as though it is used or usable. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch to add regression tests for SCHEMA

2013-05-07 Thread Robins Tharakan
Completely agree. Although the poster was kind enough to intimate me by email about his update there, but was wondering that if he hadn't, I wouldnt' have dreamt that there is a feedback on the site, two months down the line. -- Robins Tharakan On 8 May 2013 09:13, Robert Haas

Re: [HACKERS] Patch to add regression tests for SCHEMA

2013-05-07 Thread Fabien COELHO
Reviews are to be posted to pgsql-hackers, and then linked from the CommitFest site. Putting reviews only on the CommitFest site is bad practice. Indeed. Sorry, shame on me! I had not the original mail in my mailbox because I deleted it, I did not want to create a new thread because this