Re: [HACKERS] Set visibility map bit after HOT prune

2012-12-16 Thread Andres Freund
that condition in vacuumlazy, but thats not a problem... I don't immediately see a problem with HS, your logic seems to hold equally true there. The replay logic would need to be refined slightly, but it looks possible. That is without having checked the code... Greetings, Andres Freund -- Andres

Re: [HACKERS] Set visibility map bit after HOT prune

2012-12-16 Thread Andres Freund
wanted to make the patch more invasive than it had to be. FWIW I think that would be a pretty worthwile optimization - I have seen workloads where hot pruning lead to considerable contention. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] logical decoding - GetOldestXmin

2012-12-16 Thread Andres Freund
On 2012-12-15 01:19:26 +0100, Andres Freund wrote: On 2012-12-14 14:01:30 -0500, Robert Haas wrote: On Fri, Dec 14, 2012 at 6:46 AM, Andres Freund and...@2ndquadrant.com wrote: Just moving that tidbit inside the lock seems to be the pragmatic choice. GetOldestXmin is called

Re: [HACKERS] Set visibility map bit after HOT prune

2012-12-16 Thread Andres Freund
On 2012-12-16 16:25:03 +, Simon Riggs wrote: On 16 December 2012 14:41, Andres Freund and...@2ndquadrant.com wrote: On 2012-12-16 13:23:56 +0530, Pavan Deolasee wrote: Another idea could have been to NOT clear the visibility bit when a HOT update happens. Such tuple can get pruned

Re: [HACKERS] logical decoding - GetOldestXmin

2012-12-16 Thread Andres Freund
On 2012-12-16 16:44:04 +, Simon Riggs wrote: On 13 December 2012 20:03, Andres Freund and...@2ndquadrant.com wrote: Does anybody have an opinion on the attached patches? Especially 0001, which contains the procarray changes? It moves a computation of the sort of: result

Re: [HACKERS] XLByte* usage

2012-12-17 Thread Andres Freund
to invalid. I thought about that and had guessed you would be against it because it would cause useless diversion of the branches? Otherwise I am all for it. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] XLByte* usage

2012-12-17 Thread Andres Freund
. The individual bytes aren't necessarily laid out the same with two such uint32s as with one uint64. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] XLByte* usage

2012-12-18 Thread Andres Freund
On 2012-12-18 13:14:10 +0100, Dimitri Fontaine wrote: Andres Freund and...@2ndquadrant.com writes: In 2) unfortunately one has to make decision in which way to simplify negated XLByte(LT|LE) expressions. I tried to make that choice very careful and when over every change several times after

Re: [HACKERS] [ADMIN] Problems with enums after pg_upgrade

2012-12-18 Thread Andres Freund
. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] logical decoding - GetOldestXmin

2012-12-18 Thread Andres Freund
-- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-12-18 Thread Andres Freund
the OID (and in cases involving + * relfilenode swaps, it's not always clear which table OID to forget, + * anyway). + */ This and at least one other comment seems to be in too many locations now. Greetings, Andres Freund -- Andres Freund http://www

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2012-12-18 Thread Andres Freund
changeDependencyFor. I also added a regression test for this purpose. Ok. Are there no other depencencies towards indexes? I don't know of any right now, but I have the feeling there were some other cases. On Tue, Dec 11, 2012 at 12:28 AM, Andres Freund and...@2ndquadrant.comwrote: Some review comments

Re: [HACKERS] [ADMIN] Problems with enums after pg_upgrade

2012-12-19 Thread Andres Freund
at the end) to be able to change enums within transactions. That explains everything. You *CANNOT* do that. There are some pretty fundamental reasons why you are not allowed to add enums in a transaction. And even more reasons why deleting from enums isn't allowed at all. Greetings, Andres Freund

Re: [HACKERS] Review of Row Level Security

2012-12-19 Thread Andres Freund
and checking for unique violations. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-12-20 Thread Andres Freund
On 2012-12-20 02:54:48 +0100, Tomas Vondra wrote: On 19.12.2012 02:18, Andres Freund wrote: On 2012-12-17 00:31:00 +0100, Tomas Vondra wrote: I think except of the temp buffer issue mentioned below its ready. -DropRelFileNodeAllBuffers(RelFileNodeBackend rnode

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Andres Freund
for introducing InvalidTimeLineID soon... 0 as a invalid TimeLineID seems to spread and is annoying to grep for. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] ThisTimeLineID in checkpointer and bgwriter processes

2012-12-20 Thread Andres Freund
shouldn't be done in individual transactions because it would increase latency noticeably. So it seems it addresses a real performance problem. What do you dislike about it? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Parser Cruft in gram.y

2012-12-20 Thread Andres Freund
of extremly simply statements, so its not too surprising not to see a that big difference with that. Are we sure its really cache-misses and not just the actions performed in the grammar that make bison code show up in profiles? I remember the latter being the case... Greetings, Andres Freund

Re: [HACKERS] Parser Cruft in gram.y

2012-12-20 Thread Andres Freund
On 2012-12-20 15:45:47 +0100, Andres Freund wrote: On 2012-12-20 09:11:46 -0500, Robert Haas wrote: On Thu, Dec 20, 2012 at 8:55 AM, Simon Riggs si...@2ndquadrant.com wrote: On 18 December 2012 22:10, Robert Haas robertmh...@gmail.com wrote: Well that would be nice, but the problem

Re: [HACKERS] Parser Cruft in gram.y

2012-12-20 Thread Andres Freund
On 2012-12-20 15:51:37 +0100, Andres Freund wrote: On 2012-12-20 15:45:47 +0100, Andres Freund wrote: On 2012-12-20 09:11:46 -0500, Robert Haas wrote: On Thu, Dec 20, 2012 at 8:55 AM, Simon Riggs si...@2ndquadrant.com wrote: On 18 December 2012 22:10, Robert Haas robertmh

Re: [HACKERS] Parser Cruft in gram.y

2012-12-20 Thread Andres Freund
On 2012-12-20 16:04:49 +0100, Andres Freund wrote: On 2012-12-20 15:51:37 +0100, Andres Freund wrote: When doing a source/assembly annotation in SearchCatCache about half of the misses are attributed to the memcpy() directly at the beginning. Using a separate array for storing the arguments

Re: [HACKERS] Parser Cruft in gram.y

2012-12-20 Thread Andres Freund
in the profile. I quickly checked though, and the top three offenders for L1 caches are in bison not in the lexer... Its not implausible that the state transitions in the lexer are better cached and/or predicted... Greetings, Andres Freund -- Andres Freund http://www

Re: [HACKERS] ThisTimeLineID in checkpointer and bgwriter processes

2012-12-20 Thread Andres Freund
in recovery because we never write WAL. It's called from CreateRestartPoint. And the pre-inited files get used by the walreceiver and I would guess thats beneficial at elast for sync rep... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Feature Request: pg_replication_master()

2012-12-20 Thread Andres Freund
the node one step up or the top-level in the chain? Because I don't think we can do the latter without complicating the replication protocol noticably. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services

Re: [HACKERS] Parser Cruft in gram.y

2012-12-20 Thread Andres Freund
, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] need a function to extract list items from pg_node_tree

2012-12-21 Thread Andres Freund
catalog column that contains an expression list. Hm. Wouldn't it be better to create a pg_node_tree[] and use that in pg_attribute? Its already in the variable length part of pg_proc anyway... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-12-21 Thread Andres Freund
. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] Event Triggers: adding information

2012-12-21 Thread Andres Freund
not that different from whats done currently? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] foreign key locks

2012-12-22 Thread Andres Freund
pieces (the whole EPQ integration) that will require significant retesting once Alvaro has time to work on this again.. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers

Re: [HACKERS] pgcrypto seeding problem when ssl=on

2012-12-22 Thread Andres Freund
in the pid + gettimeofday()after the fork. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] buffer assertion tripping under repeat pgbench load

2012-12-23 Thread Andres Freund
(bufHdr-tag.rnode, InvalidBackendId, bufHdr-tag.forkNum), +PrivateRefCount[i], bufHdr-refcount); + } } } #endif Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development

Re: [HACKERS] Feature Request: pg_replication_master()

2012-12-24 Thread Andres Freund
. It also seems to be a good building block to do work on the other things asked for in this and related threads. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] fix bgworkers in EXEC_BACKEND

2012-12-27 Thread Andres Freund
) ??? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] fix bgworkers in EXEC_BACKEND

2012-12-27 Thread Andres Freund
On 2012-12-27 18:44:57 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I am still worried about the following scenario in the EXEC_BACKEND case: 1) postmaster starts 2) reads config 3) executes _PG_init for shared_preload_libraries 4) library 'abc' gets config value

Re: [HACKERS] XLByte* usage

2012-12-28 Thread Andres Freund
On 2012-12-28 14:59:50 -0300, Alvaro Herrera wrote: Andres Freund escribió: On 2012-12-17 13:16:47 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-12-17 12:47:41 -0500, Tom Lane wrote: But, if the day ever comes when 64 bits doesn't seem like enough, I bet

Re: [HACKERS] inconsistent time zone formats in log

2012-12-29 Thread Andres Freund
in pg_control/ControlFileData), the other is a TimestampTz. Those have completely different code paths for being printed (pg_strftime vs EncodeDateTime) ... I don't want to say its impossible or shouldn't be fixed, just that its not trivial to do so. Greetings, Andres Freund -- Andres Freund

[HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2012-12-29 Thread Andres Freund
where I would say its not really feasible. A shared table for event triggers sounds like it would be the far easier solution (9.4+ that is). Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services

[HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2013-01-03 Thread Andres Freund
those. Moving to a different tablespace, a rewriting ALTER TABLE, etc. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Andres Freund
to start with this. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Andres Freund
On 2013-01-07 02:21:26 -0800, John R Pierce wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I seem to remember somebody even posting a patch to that effect? I agree that it's probably too

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-07 Thread Andres Freund
that doesn't matter if all standbys already have built a snapshot, but the primary cannot know that. Having to issue a checkpoint while ensuring transactions are running just to get a standby up doesn't seem like a good idea to me :) Greetings, Andres Freund -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Andres Freund
Hi, It seems worth rereading the thread around http://archives.postgresql.org/message-id/CAAZKuFb59sABSa7gCG0vnVnGb-mJCUBBbrKiyPraNXHnis7KMw%40mail.gmail.com for people wanting to work on this. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Andres Freund
On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January 2013 12:39, Amit Kapila amit.kap...@huawei.com wrote

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Andres Freund
On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January 07

[HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Andres Freund
From: Andres Freund and...@2ndquadrant.com Subject: [PATCH] xlogreader-v4 In-Reply-To: Hi, this is the latest and obviously best version of xlogreader xlogdump with changes both from Heikki and me. Changes: * windows build support for pg_xlogdump * xlogdump moved to contrib * xlogdump option

[HACKERS] [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
From: Andres Freund and...@anarazel.de c.h already had parts of the assert support (StaticAssert*) and its the shared file between postgres.h and postgres_fe.h. This makes it easier to build frontend programs which have to do the hack. --- src/include/c.h | 65

[HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
From: Andres Freund and...@anarazel.de relpathbackend() (via some of its wrappers) is used in *_desc routines which we want to be useable without a backend environment arround. Change signature to return a 'const char *' to make misuse easier to detect. That necessicates also changing

[HACKERS] [PATCH 5/5] remove spurious space in running_xact's _desc function

2013-01-08 Thread Andres Freund
From: Andres Freund and...@anarazel.de --- src/backend/access/rmgrdesc/standbydesc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index c38892b..5fb6f54 100644 --- a/src/backend/access

[HACKERS] [PATCH 4/5] Add pg_xlogdump contrib module

2013-01-08 Thread Andres Freund
From: Andres Freund and...@anarazel.de Authors: Andres Freund, Heikki Linnakangas --- contrib/Makefile | 1 + contrib/pg_xlogdump/Makefile | 37 +++ contrib/pg_xlogdump/compat.c | 58 contrib/pg_xlogdump/pg_xlogdump.c | 654

Re: [HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Andres Freund
On 2013-01-08 20:09:42 +0100, Andres Freund wrote: From: Andres Freund and...@2ndquadrant.com Subject: [PATCH] xlogreader-v4 In-Reply-To: Hi, this is the latest and obviously best version of xlogreader xlogdump with changes both from Heikki and me. Changes: * windows build support

[HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 14:25:06 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: From: Andres Freund and...@anarazel.de c.h already had parts of the assert support (StaticAssert*) and its the shared file between postgres.h and postgres_fe.h. This makes it easier to build

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 14:28:14 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: maxpg From: Andres Freund and...@anarazel.de relpathbackend() (via some of its wrappers) is used in *_desc routines which we want to be useable without a backend environment arround. I'm 100

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 14:53:29 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:28:14 -0500, Tom Lane wrote: I'm 100% unimpressed with making relpathbackend return a pointer to a static buffer. Who's to say whether that won't create bugs due to overlapping

[HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 14:35:12 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:25:06 -0500, Tom Lane wrote: This patch seems unnecessary given that we already put a version of Assert() into postgres_fe.h. The problem is that some (including existing

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 15:27:23 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for adding it. I've said from the very beginning of this effort that it would be impossible

Re: [HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 17:36:19 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2013-01-08 14:35:12 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:25:06 -0500, Tom Lane wrote: This patch seems unnecessary given that we already put a version

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 22:47:43 +0200, Heikki Linnakangas wrote: On 08.01.2013 22:39, Andres Freund wrote: On 2013-01-08 15:27:23 -0500, Tom Lane wrote: Andres Freundand...@2ndquadrant.com writes: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 15:45:07 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: To what extent do you want palloc et al. emulation? Provide actual pools or just make redirect to malloc and provide the required symbols (at the very least CurrentMemoryContext)? I don't see any

[HACKERS] Re: [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 23:02:15 +0200, Heikki Linnakangas wrote: On 08.01.2013 23:00, Andres Freund wrote: Note that the xlogreader facility doesn't need any more emulation. I'm quite satisfied with that part of the patch now. However, the rmgr desc routines do, and I'm not very happy with those

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 17:28:33 -0500, Robert Haas wrote: On Tue, Jan 8, 2013 at 3:00 PM, Andres Freund and...@2ndquadrant.com wrote: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for adding it. FWIW, I'm with Tom on this one. Any meaningful code

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-09 Thread Andres Freund
On 2013-01-09 14:04:32 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:57 PM Andres Freund wrote: On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January

[HACKERS] [PATCH 2/2] use pg_malloc instead of an unchecked malloc in pg_resetxlog

2013-01-09 Thread Andres Freund
--- src/bin/pg_resetxlog/pg_resetxlog.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/bin/pg_resetxlog/pg_resetxlog.c b/src/bin/pg_resetxlog/pg_resetxlog.c index 8734f2c..60fb30c 100644 --- a/src/bin/pg_resetxlog/pg_resetxlog.c +++

[HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
Hi, As promised here's a patch to provide palloc emulation for frontend-ish environments. The patch: - makes palloc() into a real function so CurrentMemoryContext doesn't need to be provided - provides common pg_(malloc,malloc0, realloc, strdup, free) wrappers and removes various versions of

[HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
--- contrib/oid2name/oid2name.c| 52 + contrib/pg_upgrade/pg_upgrade.h| 5 +- contrib/pg_upgrade/util.c | 49 - contrib/pgbench/pgbench.c | 54 +- src/backend/utils/mmgr/mcxt.c | 78

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 13:46:53 +0200, Heikki Linnakangas wrote: On 09.01.2013 13:27, Andres Freund wrote: - makes palloc() into a real function so CurrentMemoryContext doesn't need to be provided I don't understand the need for this change. Can't you just: #define palloc(s) pg_malloc(s

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-09 Thread Andres Freund
On 2013-01-09 15:06:04 +0530, Amit Kapila wrote: On Wednesday, January 09, 2013 2:28 PM Andres Freund wrote: On 2013-01-09 14:04:32 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:57 PM Andres Freund wrote: On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
into multiple mails) It may be just me. But it may be others as well, so I figured I should raise the issue :) I am happy to comply with whatever others prefer. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
On 2013-01-09 22:23:25 +0900, Michael Paquier wrote: On Wed, Jan 9, 2013 at 9:54 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jan 9, 2013 at 1:47 PM, Andres Freund and...@2ndquadrant.com wrote: Yeah, so far it's also just my opinion in the other direction :) Hopefully, some

[HACKERS] Re: [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
to update the above comment. *headbang*. Gah. I hate these comments... Will update, but won't send anything again until more changes have accumulated. Thanks! Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 11:37:46 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-09 13:46:53 +0200, Heikki Linnakangas wrote: I don't understand the need for this change. Can't you just: #define palloc(s) pg_malloc(s) in the frontend context? Yes, that would

[HACKERS] Re: [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
On 2013-01-09 11:27:46 -0500, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Jan 9, 2013 at 1:47 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-01-09 13:34:12 +0100, Magnus Hagander wrote: Am I the only one who finds this way of posting patches really

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 11:57:35 -0500, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2013-01-09 11:37:46 -0500, Tom Lane wrote: I agree that what dirmod is doing is pretty ugly, but it's localized enough that I don't care particularly. (Really, the only reason it's a hack

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 12:32:20 -0500, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2013-01-09 11:57:35 -0500, Tom Lane wrote: My objection is that you're forcing *all* backend code to go through the trampoline. That probably has a negative impact on performance, and if you think

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
experience. I wonder whether it makes sense to inline the contents pstrdup() additionally? My gut feeling is not, but... I would like to move CurrentMemoryContext to memutils.h, but that seems to require too many changes. Greetings, Andres Freund -- Andres Freund http://www

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 15:07:10 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Well, I *did* benchmark it as noted elsewhere in the thread, but thats obviously just machine (E5520 x 2) with one rather restricted workload (pgbench -S -jc 40 -T60). At least its rather palloc heavy

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 17:28:35 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-09 15:43:19 -0500, Tom Lane wrote: I had thought of proposing that we code palloc() like this: void * palloc(Size size) { MemoryContext context = CurrentMemoryContext

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-10 Thread Andres Freund
On 2013-01-10 00:05:07 +0100, Andres Freund wrote: On 2013-01-09 17:28:35 -0500, Tom Lane wrote: (We know this doesn't matter, but gcc doesn't; this version of gcc apparently isn't doing much with the knowledge that elog won't return.) Afaics one reason for that is that we don't have any

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-10 Thread Andres Freund
On 2013-01-10 10:31:04 +0100, Andres Freund wrote: On 2013-01-10 00:05:07 +0100, Andres Freund wrote: On 2013-01-09 17:28:35 -0500, Tom Lane wrote: (We know this doesn't matter, but gcc doesn't; this version of gcc apparently isn't doing much with the knowledge that elog won't return

Re: [HACKERS] foreign key locks

2013-01-10 Thread Andres Freund
). Is that enough in case of a originally non-key update in read committed mode that turns into a key update due to a concurrent key update? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services

Re: [HACKERS] foreign key locks

2013-01-11 Thread Andres Freund
On 2013-01-11 12:11:47 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2013-01-10 18:00:40 -0300, Alvaro Herrera wrote: Here's version 28 of this patch. The only substantive change here from v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-11 Thread Andres Freund
On 2013-01-10 10:55:20 +0100, Andres Freund wrote: On 2013-01-10 10:31:04 +0100, Andres Freund wrote: On 2013-01-10 00:05:07 +0100, Andres Freund wrote: On 2013-01-09 17:28:35 -0500, Tom Lane wrote: (We know this doesn't matter, but gcc doesn't; this version of gcc apparently isn't

Re: [HACKERS] foreign key locks

2013-01-11 Thread Andres Freund
On 2013-01-11 13:10:49 -0300, Alvaro Herrera wrote: Andres Freund wrote: No, I was thinking about an update without triggers present. T0: CREATE TABLE tbl(id serial pk, name text unique, data text); T1: BEGIN; -- read committed T1: UPDATE tbl SET name = 'foo' WHERE name = 'blarg

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-11 Thread Andres Freund
On 2013-01-11 14:01:40 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: The attached patch: * adds configure checks for __VA_ARGS__ and __builtin_unreachable support * provides a pg_unreachable() macro that expands to __builtin_unreachable() or abort

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-11 Thread Andres Freund
On 2013-01-11 15:05:54 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: [ patch to mark elog() as non-returning if elevel = ERROR ] It strikes me that both in this patch, and in Peter's previous patch to do this for ereport(), there is an opportunity for improvement

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-11 Thread Andres Freund
On 2013-01-11 15:52:19 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-11 15:05:54 -0500, Tom Lane wrote: And another thing: what if the elevel argument isn't safe for multiple evaluation? No such hazard ever existed before these patches, so I'm not very

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-11 Thread Andres Freund
On 2013-01-11 16:16:58 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-11 15:52:19 -0500, Tom Lane wrote: I agree the scenario doesn't seem all that probable, but what scares me here is that if we use __builtin_constant_p(elevel) (elevel) = ERROR in some

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-11 Thread Andres Freund
On 2013-01-11 16:28:13 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-11 16:16:58 -0500, Tom Lane wrote: Uh ... because it's *not* unreachable if elevel ERROR. Otherwise we'd just mark errfinish as __attribute((noreturn)) and be done. Of course, that's

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-12 Thread Andres Freund
that would easy to fix, so no problem there. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-12 Thread Andres Freund
wonder whether __builtin_choose_expr is any better? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-13 Thread Andres Freund
On 2013-01-12 15:39:16 -0500, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 12.01.2013 20:42, Andres Freund wrote: I don't care for that too much in detail -- if errstart were to return false (it shouldn't, but if it did) this would be utterly broken

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-13 Thread Andres Freund
because the abort() didn't use to be there. And I think the danger youre alluding to above is a real one. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list

[HACKERS] Re: logical changeset generation v3 - comparison to Postgres-R change set format

2013-01-13 Thread Andres Freund
for far too little benefit. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-13 Thread Andres Freund
On 2013-01-12 18:15:17 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-12 13:16:56 -0500, Tom Lane wrote: However, using a do-block with a local variable is definitely something worth considering. I'm getting less enamored of the __builtin_constant_p idea

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-13 Thread Andres Freund
On 2013-01-12 16:36:39 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: It does *not* combine elog_start and elog_finish into one function if varargs are available although that brings a rather measurable size/performance benefit. Since you've apparently already done

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-13 Thread Andres Freund
On 2013-01-13 14:17:52 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: the numbers are: old definition: 10393.658ms, 5497912 bytes old definition + unreachable: 10011.102ms, 5469144 bytes stmt, two calls, unreachable

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-13 Thread Andres Freund
On 2013-01-13 15:44:58 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: And if you look at the disassembly of ERROR codepaths: I think your numbers are being twisted by -fno-omit-frame-pointer. What I get, with the __builtin_unreachable version of the macro, looks more

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-01-13 Thread Andres Freund
: postgres=# CREATE TABLE pg_catalog.test AS SELECT 1; SELECT 1 Time: 124.112 ms postgres=# DROP TABLE pg_catalog.test; ERROR: permission denied: test is a system catalog Time: 0.461 ms Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] passing diff options to pg_regress

2013-01-14 Thread Andres Freund
an environment variable, such as PG_REGRESS_DIFF_OPTS. The patch is very small. +ERANGE I usually do this by manually doing the diff, but thats pretty annoying. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training

Re: [HACKERS] Curious buildfarm failures

2013-01-14 Thread Andres Freund
in PostmasterMain (argc=6, argv=0x600d85e0) at postmaster.c:1244 #27 0x40577a30 in main (argc=6, argv=0x600d8010) at main.c:197 in the log. So it seems like it also could be related to locking changes although I don't immediately see why. Greetings, Andres Freund -- Andres Freund

<    2   3   4   5   6   7   8   9   10   11   >