[HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Kyotaro HORIGUCHI
Hello. This message is a proposal of a pair of patches that enables the memory allocator for PGresult in libpq to be replaced. The comment at the the begging of pqexpbuffer.c says that libpq should not rely on palloc(). Besides, Tom Lane said that palloc should not be visible outside the

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Heikki Linnakangas
On 11.11.2011 11:18, Kyotaro HORIGUCHI wrote: The comment at the the begging of pqexpbuffer.c says that libpq should not rely on palloc(). Besides, Tom Lane said that palloc should not be visible outside the backend(*1) and I agree with it. *1:

Re: [HACKERS] Adding Node support in outfuncs.c and readfuncs.c

2011-11-11 Thread Dimitri Fontaine
Dimitri Fontaine dimi...@2ndquadrant.fr writes: What about adding that into src/tools/editors/pgsrc.el? Of course after having used it some more, I fixed a bug and added a function to start from the readfunc node name rather than the enum node number in the interactive calls. Please find

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread Dimitri Fontaine
David E. Wheeler da...@kineticode.com writes: EXTENSION= pgtap pgtap-core pgtap-schema See contrib/spi/Makefile for an exemple of that. make Makefile:79: *** multiple target patterns. Stop. So is that not supported? If not, why not? Can we see your Makefile? Regards, --

[HACKERS] SQLDA fix for ECPG

2011-11-11 Thread Boszormenyi Zoltan
Hi, I had a report about ECPG code crashing which involved a query using a date field. Attached is a one liner fix to make the date type's offset computed consistently across sqlda_common_total_size(), sqlda_compat_total_size() and sqlda_native_total_size(). This must have been a cut and paste

Re: [HACKERS] type privileges and default privileges

2011-11-11 Thread Robert Haas
On Thu, Nov 10, 2011 at 11:17 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Thu, Nov 10, 2011 at 10:52 PM, Stephen Frost sfr...@snowman.net wrote: Certainly a big one that people get caught by is our default of execute to public on functions..  

[HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Robert Haas
I noticed while playing around this morning that, in read committed mode, the following query - or any other straightforward query - calls GetSnapshotData() twice: SELECT 1; What happens is this: exec_simple_query() calls analyze_requires_snapshot() on the query. Since the query is a SELECT,

Re: [HACKERS] type privileges and default privileges

2011-11-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Oh, I didn't realize that you were proposing a database-wide setting; my point was just that the way the feature looks to the user doesn't have to dictate the catalog representation. I agree that the features don't have to define the catalog

Rename trace_userlocks? WAS Re: [HACKERS] LOCK_DEBUG is busted

2011-11-11 Thread Bruce Momjian
Robert Haas wrote: On Thu, Nov 10, 2011 at 5:07 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: It's possible to compile the source tree with LOCK_DEBUG defined, but the resulting postgres promptly dumps core, due to the fact that

Re: Rename trace_userlocks? WAS Re: [HACKERS] LOCK_DEBUG is busted

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 9:40 AM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Nov 10, 2011 at 5:07 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: It's possible to compile the source tree with LOCK_DEBUG defined, but

[HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Tom Lane
In another thread, Alvaro quoted from the manual: If for some reason autovacuum fails to clear old XIDs from a table, the system will begin to emit warning messages like this when the database's oldest XIDs reach ten million transactions from the wraparound point:

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Can't we arrange to retain the snapshot used for parse analysis / planning and reuse it for the portal that we create just afterwards? Possibly, but what if planning takes a long time? Also, I think you're ignoring the extended-query-protocol

[HACKERS] Hotel reservation for FOSDEM 2012 - Deadline: December 31th, 2011

2011-11-11 Thread Andreas 'ads' Scherbaum
Hi all, like the last years we will have a devroom at FOSDEM 2012. We also look forward to have a booth. We made a group reservation in the Agenda Louise hotel: Hotel Agenda Louise rue de Florence 6 B-1000 Brussels Tel: + 32.2.539.00.31 Fax: + 32.2.539.00.63 www.hotel-agenda.com This time,

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 10:01 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Can't we arrange to retain the snapshot used for parse analysis / planning and reuse it for the portal that we create just afterwards? Possibly, but what if planning takes a long

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: In another thread, Alvaro quoted from the manual:       If for some reason autovacuum fails to clear old XIDs from a table, the       system will begin to emit warning messages like this when the database's oldest       XIDs

Re: [HACKERS] Disable OpenSSL compression

2011-11-11 Thread Albe Laurenz
Tom Lane wrote: But in any case, my objection is that there's no adequate use-case for this GUC, because it's much more sensible to set it from the client side. We have too many GUCs already --- Josh B regularly goes on the warpath looking for ones we can remove. This one should never get in

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Alvaro Herrera
Excerpts from Robert Haas's message of vie nov 11 12:29:42 -0300 2011: On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: In another thread, Alvaro quoted from the manual:       If for some reason autovacuum fails to clear old XIDs from a table, the       system will

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: In another thread, Alvaro quoted from the manual: If for some reason autovacuum fails to clear old XIDs from a table, the system will begin to emit warning messages like

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Florian Pflug
On Nov11, 2011, at 16:18 , Robert Haas wrote: In the extend query protocol scenario, it seems to me that keeping the snapshot would be both wrong and a bad idea. It would be wrong because the user will (I think) expect the query can see all rows that were marked as committed prior to Execute

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Tom Lane
Florian Pflug f...@phlo.org writes: On Nov11, 2011, at 16:18 , Robert Haas wrote: In the extend query protocol scenario, it seems to me that keeping the snapshot would be both wrong and a bad idea. Hm, but that'd penalize clients who use the extended query protocol, which they have to if

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Florian Pflug
On Nov11, 2011, at 17:06 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: On Nov11, 2011, at 16:18 , Robert Haas wrote: In the extend query protocol scenario, it seems to me that keeping the snapshot would be both wrong and a bad idea. Hm, but that'd penalize clients who use the

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Robert Haas
On Nov 11, 2011, at 10:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: In another thread, Alvaro quoted from the manual: If for some reason autovacuum fails to clear old XIDs from a

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Heikki Linnakangas
On 11.11.2011 17:47, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: On Fri, Nov 11, 2011 at 9:59 AM, Tom Lanet...@sss.pgh.pa.us wrote: It occurs to me to wonder how this scenario will interact with the recent changes to let VACUUM skip pages. AFAIR there is not a way for a manual

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread David E. Wheeler
On Nov 11, 2011, at 4:27 AM, Dimitri Fontaine wrote: EXTENSION= pgtap pgtap-core pgtap-schema See contrib/spi/Makefile for an exemple of that. Okay, good, it *should* work then. make Makefile:79: *** multiple target patterns. Stop. So is that not supported? If not, why not?

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Heikki Linnakangas
On 11.11.2011 19:15, Heikki Linnakangas wrote: On 11.11.2011 17:47, Tom Lane wrote: (3) In disaster recovery scenarios, the last thing we want is to be imposing extra conditions on what an already-stressed DBA has to do to fix things; especially extra conditions that are different from the way

Re: [HACKERS] foreign key locks, 2nd attempt

2011-11-11 Thread David Kerr
On Thu, Nov 10, 2011 at 03:17:59PM -0500, Christopher Browne wrote: - On Sun, Nov 6, 2011 at 2:28 AM, Jeroen Vermeulen j...@xs4all.nl wrote: - On 2011-11-04 01:12, Alvaro Herrera wrote: - - I would like some opinions on the ideas on this patch, and on the patch - itself.  If someone wants more

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Tom Lane
Florian Pflug f...@phlo.org writes: Still, optimizing only the simple protocol seems weird. Would it be sane to decree that the statement snapshot lasts until Sync is received, in extended query mode? But frankly I do not like any of these proposals. Making fundamental changes in

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On second thought, if XID wraparound is close enough that the DBA has to log in to do manual vacuums to avoid it, relfrozenxid of the trouble-making tables are surely older than default vacuum_freeze_table_age, so plain VACUUM

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread David E. Wheeler
On Nov 11, 2011, at 9:20 AM, David E. Wheeler wrote: Can we see your Makefile? Right here: https://github.com/theory/pgtap/blob/master/Makefile Oy, this is stupidly my fault. I blame the late hours and the second stout. Thanks, David -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread Dimitri Fontaine
David E. Wheeler da...@kineticode.com writes: EXTENSION= pgtap pgtap-core pgtap-schema https://github.com/theory/pgtap/blob/master/Makefile I don't see the line above in the file you linked. The content of the Makefile I just read is not using multiple extensions at all, it's re

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread David E. Wheeler
On Nov 11, 2011, at 11:04 AM, Dimitri Fontaine wrote: EXTENSION= pgtap pgtap-core pgtap-schema https://github.com/theory/pgtap/blob/master/Makefile I don't see the line above in the file you linked. The content of the Makefile I just read is not using multiple extensions at all,

[HACKERS] Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Daniel Farina
On Thu, Nov 10, 2011 at 11:57 PM, Nikhil Sontakke nikkh...@gmail.com wrote: Hi, But if it's deemed to be a problem, I want to see a solution that's actually watertight.) After Daniel's hunch about pg_dump barfing due to such leftover entries proving out to be true, we have one credible

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Considering that GetSnapshotData() is the number-one consumer of CPU time on many profiling runs I've done, this seems needlessly inefficient. Can't we arrange to retain the snapshot used for parse analysis / planning and reuse it for the portal that

Re: [HACKERS] Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Alvaro Herrera
Excerpts from Daniel Farina's message of vie nov 11 16:08:01 -0300 2011: On Thu, Nov 10, 2011 at 11:57 PM, Nikhil Sontakke nikkh...@gmail.com wrote: Hi, But if it's deemed to be a problem, I want to see a solution that's actually watertight.) After Daniel's hunch about pg_dump

Re: [HACKERS] Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Daniel Farina's message of vie nov 11 16:08:01 -0300 2011: The 2004 report was in version 7.3, released in 2002. So it's Very Nearly a ten year old bug, and may very well go back further back in time. Wasn't 7.3 the release

Re: [HACKERS] reduce null bitmap size

2011-11-11 Thread YAMAMOTO Takashi
hi, y...@mwd.biglobe.ne.jp (YAMAMOTO Takashi) writes: how about making heap_form_tuple and its variants use smaller natts for tuples whose trailing columns are NULL? This idea has been proposed before, and rejected on the basis that it's unlikely to save enough to be worth the cycles

Re: [HACKERS] Disable OpenSSL compression

2011-11-11 Thread Bruce Momjian
Albe Laurenz wrote: Tom Lane wrote: But in any case, my objection is that there's no adequate use-case for this GUC, because it's much more sensible to set it from the client side. We have too many GUCs already --- Josh B regularly goes on the warpath looking for ones we can remove.

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Bruce Momjian
Heikki Linnakangas wrote: On 11.11.2011 17:47, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: On Fri, Nov 11, 2011 at 9:59 AM, Tom Lanet...@sss.pgh.pa.us wrote: It occurs to me to wonder how this scenario will interact with the recent changes to let VACUUM skip pages. AFAIR

[HACKERS] Working with git repo tagged versions

2011-11-11 Thread Robert Treat
Howdy folks, Occasionally I need to grab an older release from git based on a tag rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE branch. I used to know how to do this in CVS, but I find I tend to revert to grabbing tarballs now that we're on git. So, I'm wondering if anyone knows

Re: [HACKERS] Working with git repo tagged versions

2011-11-11 Thread Bruce Momjian
Robert Treat wrote: Howdy folks, Occasionally I need to grab an older release from git based on a tag rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE branch. I used to know how to do this in CVS, but I find I tend to revert to grabbing tarballs now that we're on git. So, I'm

Re: [HACKERS] Working with git repo tagged versions

2011-11-11 Thread Alvaro Herrera
Excerpts from Robert Treat's message of vie nov 11 17:18:43 -0300 2011: Howdy folks, Occasionally I need to grab an older release from git based on a tag rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE branch. I used to know how to do this in CVS, but I find I tend to

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 2:21 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: Considering that GetSnapshotData() is the number-one consumer of CPU time on many profiling runs I've done, this seems needlessly inefficient.  Can't we arrange to retain

Re: [HACKERS] Working with git repo tagged versions

2011-11-11 Thread Andrew Dunstan
On 11/11/2011 03:53 PM, Bruce Momjian wrote: Robert Treat wrote: Howdy folks, Occasionally I need to grab an older release from git based on a tag rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE branch. I used to know how to do this in CVS, but I find I tend to revert to

Re: [HACKERS] Working with git repo tagged versions

2011-11-11 Thread Tom Lane
Robert Treat r...@xzilla.net writes: Occasionally I need to grab an older release from git based on a tag rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE branch. I used to know how to do this in CVS, but I find I tend to revert to grabbing tarballs now that we're on git. So,

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Simon Riggs
On Fri, Nov 11, 2011 at 7:21 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: Considering that GetSnapshotData() is the number-one consumer of CPU time on many profiling runs I've done, this seems needlessly inefficient.  Can't we arrange to retain

Re: [HACKERS] foreign key locks, 2nd attempt

2011-11-11 Thread Josh Berkus
An UPDATE that modifies the key columns will be blocked, just as now. Same with a DELETE. OK, so it prevents non-key data modifications from spilling to the referred rows --- nice. Yes. Eliminates the leading cause of deadlocks in Postgres applications. -- Josh Berkus PostgreSQL Experts

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Tom, in that earlier thread you said you'd be doing something in this release about that. Can you say more about what that was, and will you be doing it still?

Re: [HACKERS] [GENERAL] VACUUM touching file but not updating relation

2011-11-11 Thread Tom Lane
Thom Brown t...@linux.com writes: On 11 November 2011 00:55, Tom Lane t...@sss.pgh.pa.us wrote: Thom Brown t...@linux.com writes: I just noticed that the VACUUM process touches a lot of relations (affects mtime) but for one file I looked at, it didn't change. This doesn't always happen, and

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 11.11.2011 11:18, Kyotaro HORIGUCHI wrote: The comment at the the begging of pqexpbuffer.c says that libpq should not rely on palloc(). Besides, Tom Lane said that palloc should not be visible outside the backend(*1) and I agree

[HACKERS] fix for psql's \dd version check

2011-11-11 Thread Josh Kupershmidt
Someone (me) didn't get the version check for part of psql's \dd command quite right. I was using a 9.2 client on a 9.1 server and got this when I ran \dd: ERROR: function pg_catalog.pg_opfamily_is_visible(oid) does not exist LINE 33: AND pg_catalog.pg_opfamily_is_visible(opf.oid) since

Re: [HACKERS] [GENERAL] VACUUM touching file but not updating relation

2011-11-11 Thread Thom Brown
On 11 November 2011 23:28, Tom Lane t...@sss.pgh.pa.us wrote: Thom Brown t...@linux.com writes: On 11 November 2011 00:55, Tom Lane t...@sss.pgh.pa.us wrote: Thom Brown t...@linux.com writes: I just noticed that the VACUUM process touches a lot of relations (affects mtime) but for one file I

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Heikki's idea is probably superior so far as PG backend usage is concerned in isolation, but I wonder if there are scenarios where a client application would like to be able to manage libpq's allocations. The answer to that is certainly 'yes'. It was one

Re: [HACKERS] foreign key locks, 2nd attempt

2011-11-11 Thread Jeroen Vermeulen
On 2011-11-12 00:30, David Kerr wrote: Is this being suggested in lieu of Alvaro's patch? because it seems to be adding complexity to the system (multiple types of primary key definitions) instead of just fixing an obvious problem (over-aggressive locking done on FK checks). It wouldn't be a

Re: [HACKERS] fix for psql's \dd version check

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 6:59 PM, Josh Kupershmidt schmi...@gmail.com wrote: Someone (me) didn't get the version check for part of psql's \dd command quite right. I was using a 9.2 client on a 9.1 server and got this when I ran \dd: ERROR:  function pg_catalog.pg_opfamily_is_visible(oid) does

Re: [HACKERS] Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Nikhil Sontakke
Wasn't 7.3 the release that introduced schemas in the first place? I think there's a very good chance that the older reports with similar symptoms are completely unrelated, anyhow. Tom Lane is reluctant and that should tell me something :) So unless the list feels that this should be fixed

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: Heikki's idea is probably superior so far as PG backend usage is concerned in isolation, but I wonder if there are scenarios where a client application would like to be able to manage libpq's allocations. The

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Robert Haas
On Sat, Nov 12, 2011 at 12:48 AM, Tom Lane t...@sss.pgh.pa.us wrote: AIUI Kyotaro-san is just suggesting that the app should be able to provide a substitute malloc function for use in allocating PGresult space (and not, I think, anything else that libpq allocates internally). Basically this

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Robert Haas
On Thu, Nov 10, 2011 at 7:00 AM, Nikhil Sontakke nikkh...@gmail.com wrote: PFA, a patch against git head. We take the AccessShareLock lock on the schema in DefineRelation now. Note that we do not want to interlock with other concurrent creations in the schema. We only want to interlock with