Re: [HACKERS] Do we need so many hint bits?

2012-11-15 Thread Tom Lane
Simon Riggs writes: > On 15 November 2012 19:42, Jeff Davis wrote: >> many of the hint bits aren't terribly important > The truth is that nobody knows because there is no way of knowing. We had a discussion awhile back in which the idea of *no* hint bits was advocated, and someone (I think Robe

Re: [HACKERS] Do we need so many hint bits?

2012-11-15 Thread Tom Lane
Jeff Davis writes: > It occurred to me recently that many of the hint bits aren't terribly > important (at least it's not obvious to me). HEAP_XMIN_COMMITTED clearly > has a purpose, and we'd expect it to be used many times following the > initial CLOG lookup. Right. > But the other tuple hint b

Re: [HACKERS] [WIP] pg_ping utility

2012-11-15 Thread Phil Sorber
Thanks for the review. On Thu, Nov 15, 2012 at 9:23 PM, Michael Paquier wrote: > Hi Phil, > > I am currently looking at your patch. > A lot of people already had a look at at, but I hope I will be helpful in > finalizing it and hand it over to a committer. > > Strangely I got the following error

Re: [HACKERS] support for LDAP URLs

2012-11-15 Thread Peter Eisentraut
On Thu, 2012-11-15 at 14:44 -0500, Robert Haas wrote: > I think this is broadly reasonable, but I'm not sure this part is a > good idea: > > +#ifdef USE_LDAP > +#ifndef WIN32 > +/* We use a deprecated function to keep the codepath the same as > win32. */ > +#define LDAP_DEPRECATED 1 > +#include >

Re: [HACKERS] [WIP] pg_ping utility

2012-11-15 Thread Michael Paquier
On Fri, Nov 16, 2012 at 12:34 PM, Phil Sorber wrote: > Thanks for the review. > > On Thu, Nov 15, 2012 at 9:23 PM, Michael Paquier > wrote: > > Hi Phil, > > > > I am currently looking at your patch. > > A lot of people already had a look at at, but I hope I will be helpful in > > finalizing it a

Re: [HACKERS] Do we need so many hint bits?

2012-11-15 Thread Simon Riggs
On 15 November 2012 22:08, Tom Lane wrote: > Simon Riggs writes: >> On 15 November 2012 19:42, Jeff Davis wrote: >>> many of the hint bits aren't terribly important > >> The truth is that nobody knows because there is no way of knowing. > > We had a discussion awhile back in which the idea of *n

Re: [HACKERS] Do we need so many hint bits?

2012-11-15 Thread Pavan Deolasee
On Fri, Nov 16, 2012 at 8:51 AM, Tom Lane wrote: > Jeff Davis writes: > > > Removing those 3 hints would give us 3 more flag bits (eventually, after > > we are sure they aren't just leftover), and it would also reduce the > > chance that a page is dirtied for no other reason than to set them. >

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Simon Riggs
On 15 November 2012 05:25, Hannu Krosing wrote: > On 11/15/2012 09:48 AM, Craig Ringer wrote: >> >> If you want to prevent TRUNCATE, deny the privilege or add a trigger >> that aborts the command. > > You can abort the transaction but not skip action as currently it is only > possible to skip in R

Re: [HACKERS] logical changeset generation v3

2012-11-15 Thread Michael Paquier
Do you have a git repository or something where all the 14 patches are applied? I would like to test the feature globally. Sorry I recall that you put a link somewhere but I cannot remember its email... On Thu, Nov 15, 2012 at 6:34 PM, Andres Freund wrote: > Hi, > > On Thursday, November 15, 201

Re: [HACKERS] [WIP] pg_ping utility

2012-11-15 Thread Phil Sorber
On Thu, Nov 15, 2012 at 10:55 PM, Michael Paquier wrote: > Hum, it is not really consistent to use a magic number here, particularly in > the case where an additional state would be added in the enum PGPing. So why > not simply return PQPING_NO_ATTEMPT when there are incorrect options or you > sho

Re: [HACKERS] Do we need so many hint bits?

2012-11-15 Thread Simon Riggs
On 15 November 2012 22:21, Tom Lane wrote: >> Removing those 3 hints would give us 3 more flag bits (eventually, after >> we are sure they aren't just leftover), and it would also reduce the >> chance that a page is dirtied for no other reason than to set them. > > We aren't pressed for flag bits

Re: [HACKERS] WIP patch: add (PRE|POST)PROCESSOR options to COPY

2012-11-15 Thread Craig Ringer
On 11/16/2012 03:35 AM, Tom Lane wrote: > The biggest problem this patch has had from the very beginning is > overdesign, and this is more of the same. Let's please just define the > feature as "popen, not fopen, the given string" and have done. You can > put all the warning verbiage you want in t

Re: [HACKERS] Problem Observed in behavior of Create Index Concurrently and Hot Update

2012-11-15 Thread Pavan Deolasee
On Wed, Oct 31, 2012 at 2:40 PM, Simon Riggs wrote: > > > diff --git a/src/backend/utils/cache/relcache.c > > b/src/backend/utils/cache/relcache.c > > index a59950e..9cadb3f 100644 > > --- a/src/backend/utils/cache/relcache.c > > +++ b/src/backend/utils/cache/relcache.c > > @@ -3355,6 +3355,12 @@

Re: [HACKERS] [PATCH 05/14] Add a new relmapper.c function RelationMapFilenodeToOid that acts as a reverse of RelationMapOidToFilenode

2012-11-15 Thread Michael Paquier
Hi, This patch looks OK. I got 3 comments: 1) Why changing the OID of pg_class_tblspc_relfilenode_index from 3171 to 3455? It does not look necessary. 2) You should perhaps change the header of RelationMapFilenodeToOid so as not mentionning it as the opposite operation of RelationMapOidToFilenode

Re: [HACKERS] WIP patch: add (PRE|POST)PROCESSOR options to COPY

2012-11-15 Thread Tom Lane
Craig Ringer writes: > I'd still be much happier to have COPY ... FROM PROGRAM - or something - > to clearly make the two different, for clarity as much as security. I don't object to using a PROGRAM keyword rather than something inside the string to select this behavior.

Re: [HACKERS] gset updated patch

2012-11-15 Thread Karl O. Pinc
On 11/03/2012 01:45:36 PM, Pavel Stehule wrote: > Hello > > here is a updated patch This message does not appear to be threaded so I'm not sure I've read the whole back-history. Also, I don't really know what I'm doing. Never the less, I want to try to contribute to somebody else's patch so her

Re: [HACKERS] [WIP] pg_ping utility

2012-11-15 Thread Tom Lane
Phil Sorber writes: > On Thu, Nov 15, 2012 at 10:55 PM, Michael Paquier > wrote: >> Hum, it is not really consistent to use a magic number here, particularly in >> the case where an additional state would be added in the enum PGPing. So why >> not simply return PQPING_NO_ATTEMPT when there are in

Re: [HACKERS] autovacuum truncate exclusive lock round two

2012-11-15 Thread Amit Kapila
On Friday, November 16, 2012 4:09 AM Alvaro Herrera wrote: > Dimitri Fontaine wrote: > > Jan Wieck writes: > > > Use this lmgr feature inside count_nondeletable_pages() of > vacuumlazy.c to > > > periodically check, if there is a conflicting lock request waiting. > If not, > > > keep going. If the

Re: [HACKERS] Timing events WIP v1

2012-11-15 Thread Greg Smith
On 11/15/12 10:26 AM, Josh Berkus wrote: Extending this to save the key/value set and most of the other data I mentioned before is pretty straightforward. Why not use Hstore? Seriously? I just haven't done that part yet. The fact that hstore is the likely way to do it is why I said it's

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 11:28 PM Cédric Villemain wrote: > Le jeudi 15 novembre 2012 15:48:14, Amit kapila a écrit : > > On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: > > > > On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila > wrote: > > > Uh, no, I don't think that's a good idea.

Re: [HACKERS] gset updated patch

2012-11-15 Thread Pavel Stehule
Hello 2012/11/16 Karl O. Pinc : > On 11/03/2012 01:45:36 PM, Pavel Stehule wrote: >> Hello >> >> here is a updated patch > > This message does not appear to be threaded so I'm not > sure I've read the whole back-history. Also, I don't > really know what I'm doing. Never the less, I want > to try

Re: [HACKERS] Do we need so many hint bits?

2012-11-15 Thread Jeff Davis
On Thu, 2012-11-15 at 22:21 -0500, Tom Lane wrote: > We aren't pressed for flag bits particularly. I think the main > attraction of this idea is precisely to reduce unnecessary page dirties, > and so that leads me to suggest a variant: keep the four bits defined as > now, but do not attempt to set

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Jeff Davis
On Thu, 2012-11-15 at 17:50 +0100, Andres Freund wrote: > Me neither. I was thinking about letting the "user" allocate enough > memory like: > > binaryheap *heap = palloc(binaryheap_size(/*capacity*/ 10)); > binaryheap_init(heap, 10, comparator); > > But thats pretty ugly. Why not pass the alloc

[HACKERS] CommitFest #3 and upcoming schedule

2012-11-15 Thread Greg Smith
The third CommitFest for PostgreSQL 9.3 development is now officially active. If you have the time and interest to review one of the many patches submitted, claim it by adding yourself as a reviewer in the CommitFest application at https://commitfest.postgresql.org/action/commitfest_view?id=16

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Albe Laurenz
Peter Geoghegan wrote: > Some of you will be aware that I've tried to formulate a good general > recommendation for setting the value of commit_delay, in light of the > improvements made in 9.3. I previously asked for help finding a > methodology for optimising commit_delay [1]. The documentation r

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Craig Ringer
On 11/15/2012 03:44 PM, Darren Duncan wrote: > As they currently exist, triggers always fire based on certain SQL > syntax used, rather than on the semantics of what is actually going on. > That's not quite right. COPY fires INSERT triggers, despite never using an explicit INSERT statement. There

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Greg Smith
On 11/15/12 12:19 AM, Albe Laurenz wrote: If there is an agreement that half the sync time as reported by pg_test_fsync is a good value, would it make sense to have initdb test sync time and preset commit_delay? Peter has validated this against a good range of systems, but it would be optimist

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Craig Ringer
On 11/15/2012 04:56 PM, Greg Smith wrote: > > I would rather see this just turn into one of the things a more > general tuning tool knew how to do, executing against a fully setup > system. Having a useful implementation of commit_delay and useful docs > on it seems like enough of a jump forward fo

Re: [HACKERS] logical changeset generation v3

2012-11-15 Thread Andres Freund
Hi, On Thursday, November 15, 2012 05:08:26 AM Michael Paquier wrote: > Looks like cool stuff @-@ > I might be interested in looking at that a bit as I think I will hopefully > be hopefully be able to grab some time in the next couple of weeks. > Are some of those patches already submitted to a CF

[HACKERS] Timing events WIP v1

2012-11-15 Thread Greg Smith
Attached is a first WIP saving Timing Events via a new hook, grabbed by a new pg_timing_events contrib module. This only implements a small portion of the RFP spec I presented earlier this month, and is by no means finished code looking for a regular review. This is just enough framework to d

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Magnus Hagander
On Thu, Nov 15, 2012 at 9:56 AM, Greg Smith wrote: > On 11/15/12 12:19 AM, Albe Laurenz wrote: >> >> If there is an agreement that half the sync time as reported by >> pg_test_fsync is a good value, would it make sense to have initdb test >> sync time and preset commit_delay? > > > Peter has valid

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Michael Paquier
A row-level trigger for TRUNCATE does not really make sense, as it would mean that TRUNCATE needs to scan each tuple of the table it needs to interact with to fire its trigger, so it would more or less achieve the same performance as a plain "DELETE FROM table;". TRUNCATE is performant because it

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Hannu Krosing
On 11/15/2012 09:48 AM, Craig Ringer wrote: If you want to prevent TRUNCATE, deny the privilege or add a trigger that aborts the command. You can abort the transaction but not skip action as currently it is only possible to skip in ROW level triggers. So I'd modify this request to allow BEFORE

Re: [HACKERS] Materialized views WIP patch

2012-11-15 Thread Greg Smith
On 11/14/12 6:28 PM, Kevin Grittner wrote: - Documentation is incomplete. ... - There are no regression tests yet. Do you have any simple test cases you've been using you could attach? With epic new features like this, when things don't work it's hard to distinguish between "that just i

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 2:02 AM Atri Sharma wrote: On Thu, Nov 15, 2012 at 12:42 AM, Atri Sharma wrote: On Wed, Nov 7, 2012 at 9:47 PM, Merlin Moncure wrote: On Wed, Nov 7, 2012 at 6:01 AM, Amit Kapila wrote: >> >> Following the sig is a first cut at a patch (written by Atri) that >

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 10:04, Magnus Hagander wrote: >> I would rather see this just turn into one of the things a more general >> tuning tool knew how to do, executing against a fully setup system. Having a >> useful implementation of commit_delay and useful docs on it seems like >> enough of a jump

Re: [HACKERS] feature proposal - triggers by semantics

2012-11-15 Thread Craig Ringer
On 11/15/2012 06:25 PM, Hannu Krosing wrote: > On 11/15/2012 09:48 AM, Craig Ringer wrote: >> If you want to prevent TRUNCATE, deny the privilege or add a trigger >> that aborts the command. > You can abort the transaction but not skip action as currently it is only > possible to skip in ROW level

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote: The test is finished from myside. one more issue: > ... ./pg_basebackup -P -D ../../data_sub -X fetch -p 2303 pg_bas

Re: [HACKERS] Doc patch making firm recommendation for setting the value of commit_delay

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 08:56, Greg Smith wrote: > My main concern with this would be the relatively common practice of moving > the pg_xlog directory after initdb time. I probably should have increased the default number of seconds that pg_test_fsync runs for, in light of the fact that that can make

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 10.10.2012 17:26, Amit Kapila wrote: 36.+SendTimeLineHistory(TimeLineHistoryCmd *cmd) { .. if (nread<= 0) +ereport(ERROR, +(errcode_for_file_access(), + errmsg("could not read file \"%s\

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote: The test is finished from myside. one more issue: > ... ./pg_basebac

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 6:05 PM Heikki Linnakangas wrote: > On 15.11.2012 12:44, Heikki Linnakangas wrote: > > Here's an updated version of this patch, rebased with master, > > including the recent replication timeout changes, and some other > cleanup. > > > > On 12.10.2012 09:34, Amit Kapil

Re: [HACKERS] Hash id in pg_stat_statements

2012-11-15 Thread Magnus Hagander
On Tue, Oct 2, 2012 at 8:22 PM, Peter Geoghegan wrote: > On 2 October 2012 18:16, Tom Lane wrote >> 1. Why isn't something like md5() on the reported query text an equally >> good solution for users who want a query hash? > > Because that does not uniquely identify the entry. The very first > thi

Re: [HACKERS] Hash id in pg_stat_statements

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 13:10, Magnus Hagander wrote: >> Well, forgive me for pointing this out, but I did propose that the >> hash be a 64-bit value (which would have necessitated adopting >> hash_any() to produce 64-bit values), but you rejected the proposal. I >> arrived at the same probability for

[HACKERS] Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-15 Thread Amit kapila
On Monday, November 12, 2012 8:23 PM Fujii Masao wrote: On Fri, Nov 9, 2012 at 3:03 PM, Amit Kapila wrote: > On Thursday, November 08, 2012 10:42 PM Fujii Masao wrote: >> On Thu, Nov 8, 2012 at 5:53 PM, Amit Kapila >> wrote: >> > On Thursday, November 08, 2012 2:04 PM Heikki Linnakangas wrote: >>

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 12:08 AM, Amit Kapila wrote: > Okay. > So as Robert and Alvaro suggested to have it separate utility rather than > having options in pg_resetxlog to print MAX LSN seems to be quite > appropriate. > I am planning to update the patch to make it a separate utility as > pg_comp

Re: [HACKERS] [PATCH 08/14] Store the number of subtransactions in xl_running_xacts separately from toplevel xids

2012-11-15 Thread Simon Riggs
On 14 November 2012 22:17, Andres Freund wrote: > To avoid complicating logic we store both, the toplevel and the subxids, in > ->xip, first ->xcnt toplevel ones, and then ->subxcnt subxids. That looks good, not much change. Will apply in next few days. Please add me as committer and mark ready.

Re: [HACKERS] [PATCH 08/14] Store the number of subtransactions in xl_running_xacts separately from toplevel xids

2012-11-15 Thread Andres Freund
On 2012-11-15 09:07:23 -0300, Simon Riggs wrote: > On 14 November 2012 22:17, Andres Freund wrote: > > > To avoid complicating logic we store both, the toplevel and the subxids, in > > ->xip, first ->xcnt toplevel ones, and then ->subxcnt subxids. > > That looks good, not much change. Will apply i

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 03:17, Andres Freund wrote: Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integrated into xlog.c is rather hard and would require changes to rather int

Re: [HACKERS] add -Wlogical-op to standard compiler options?

2012-11-15 Thread Tom Lane
Peter Eisentraut writes: > I think it might be worth adding -Wlogical-op to the standard warning > options (for supported compilers, determined by configure test). Does that add any new warnings with the current source code, and if so what? regards, tom lane -- Sent vi

[HACKERS] Re: [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Andres Freund
On 2012-11-15 16:22:56 +0200, Heikki Linnakangas wrote: > On 15.11.2012 03:17, Andres Freund wrote: > > > >Features: > >- streaming reading/writing > >- filtering > >- reassembly of records > > > >Reusing the ReadRecord infrastructure in situations where the code that wants > >to do so is not tight

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Heikki Linnakangas
On 23.10.2012 00:29, Alvaro Herrera wrote: Here's an updated version of this patch, which also works in an EXEC_BACKEND environment. (I haven't tested this at all on Windows, but I don't see anything that would create a portability problem there.) Looks good at first glance. Fails on Windows,

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Alvaro Herrera
Heikki Linnakangas wrote: > On 15.11.2012 03:17, Andres Freund wrote: > > > >Features: > >- streaming reading/writing > >- filtering > >- reassembly of records > > > >Reusing the ReadRecord infrastructure in situations where the code that wants > >to do so is not tightly integrated into xlog.c is r

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Tom Lane
Heikki Linnakangas writes: > This is a fairly general issue, actually. Looking around, I can see at > least two similar cases in existing code, with BasicOpenFile, where we > will leak file descriptors on error: Um, don't we automatically clean those up during transaction abort? If we don't, we

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-15 Thread Amit kapila
On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila wrote: > Uh, no, I don't think that's a good idea. IMHO, what we should do is: > 1. Read postgresql.conf.auto and remember all the settings we saw. If we see > something funky like an inc

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Alvaro Herrera
Heikki Linnakangas escribió: > On 23.10.2012 00:29, Alvaro Herrera wrote: > >Here's an updated version of this patch, which also works in > >an EXEC_BACKEND environment. (I haven't tested this at all on Windows, > >but I don't see anything that would create a portability problem there.) > > Looks

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Robert Haas
On Wed, Nov 14, 2012 at 8:11 AM, Abhijit Menon-Sen wrote: > Comments? Suggestions? It could use a run through pgindent. And the header comments are separated by a blank line from the functions to which they are attached, which is not project style. + if (heap->size + 1 == heap->space) +

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Andres Freund
On 2012-11-15 11:50:37 -0300, Alvaro Herrera wrote: > Heikki Linnakangas wrote: > > On 15.11.2012 03:17, Andres Freund wrote: > > > > > >Features: > > >- streaming reading/writing > > >- filtering > > >- reassembly of records > > > > > >Reusing the ReadRecord infrastructure in situations where the

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 16:55, Tom Lane wrote: Heikki Linnakangas writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in existing code, with BasicOpenFile, where we will leak file descriptors on error: Um, don't we automatically clean those up during t

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Alvaro Herrera
Andres Freund wrote: > On 2012-11-15 11:50:37 -0300, Alvaro Herrera wrote: > > I worked a bit more on that patch of yours, but I neglected to submit > > it. Did you have something in particular that you wanted changed in it? > > Could you push your newest version to your git repository or simila

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Alvaro Herrera
Robert Haas escribió: > On Wed, Nov 14, 2012 at 8:11 AM, Abhijit Menon-Sen > wrote: > > Comments? Suggestions? > > It could use a run through pgindent. And the header comments are > separated by a blank line from the functions to which they are > attached, which is not project style. Also ther

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 17:10, Alvaro Herrera wrote: Heikki Linnakangas escribió: On 23.10.2012 00:29, Alvaro Herrera wrote: Here's an updated version of this patch, which also works in an EXEC_BACKEND environment. (I haven't tested this at all on Windows, but I don't see anything that would create a po

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Will Crawford
Assert(!"description of error") is an idiom I've seen more than once, although I'm not sure I understand why it's not written as Robert says with the condition in the brackets (or as a print to STDERR followed by abort() instead). On 15 November 2012 15:11, Robert Haas wrote: > On Wed, Nov 14,

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 16:50, Alvaro Herrera wrote: Heikki Linnakangas wrote: On 15.11.2012 03:17, Andres Freund wrote: Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integr

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 4:39 AM, Amit Kapila wrote: >>In each visibility function (except HeapTupleSatisfiesVacuum() ), an >> addition check has been added to check if the commit status of Xmin or Xmax >> of a tuple can be >retrieved from the cache. > > > > 1. From your explanation and code,

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Andrew Dunstan
On 11/15/2012 10:11 AM, Robert Haas wrote: + { + sift_down(heap, i); + } Project style is to omit braces for a single-line body. This comes up a few other places as well. I thought we modified that some years ago, although my memory of it is a bit hazy. Personall

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Robert Haas
On Tue, Oct 16, 2012 at 4:47 PM, Peter Geoghegan wrote: > The same basic strategy for sizing the tuplesort memtuples array in > also exists in tuplestore. I wonder if we should repeat this there? I > suppose that that could follow later. I think it'd be a good idea to either adjust tuplestore as

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 10:21 AM, Alvaro Herrera wrote: >> Other than the coding style issues, I think this looks fine. If you >> can fix that up, I believe I could go ahead and commit this, unless >> anyone else objects. > > Does this include the changes in nodeMergeappend.c? I think so. I was

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Andres Freund
On 2012-11-14 18:41:12 +0530, Abhijit Menon-Sen wrote: > There are two or three places in the Postgres source that implement heap > sort (e.g. tuplesort.c, nodeMergeAppend.c), and it's also needed by the > BDR code. It seemed reasonable to factor out the functionality. pg_dump also contains a bina

Re: [HACKERS] [PATCH 02/14] Add support for a generic wal reading facility dubbed XLogReader

2012-11-15 Thread Alvaro Herrera
Heikki Linnakangas wrote: > On 15.11.2012 16:50, Alvaro Herrera wrote: > >I worked a bit more on that patch of yours, but I neglected to submit > >it. Did you have something in particular that you wanted changed in it? > > Off the top of my head, there were a two open items with the patch > as I

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 9:27 PM Merlin Moncure wrote: > On Thu, Nov 15, 2012 at 4:39 AM, Amit Kapila > wrote: > >>In each visibility function (except HeapTupleSatisfiesVacuum() ), an > >> addition check has been added to check if the commit status of Xmin > or Xmax > >> of a tuple can be >r

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Alvaro Herrera
Andrew Dunstan escribió: > > On 11/15/2012 10:11 AM, Robert Haas wrote: > > >+{ > >+sift_down(heap, i); > >+} > > > >Project style is to omit braces for a single-line body. This comes up > >a few other places as well. > > I thought we modified that some years ago, although m

Re: [HACKERS] [PATCH 03/14] Add simple xlogdump tool

2012-11-15 Thread Peter Eisentraut
On 11/14/12 8:17 PM, Andres Freund wrote: > diff --git a/src/bin/Makefile b/src/bin/Makefile > index b4dfdba..9992f7a 100644 > --- a/src/bin/Makefile > +++ b/src/bin/Makefile > @@ -14,7 +14,7 @@ top_builddir = ../.. > include $(top_builddir)/src/Makefile.global > > SUBDIRS = initdb pg_ctl pg_du

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 11:22 AM, Andres Freund wrote: > On 2012-11-14 18:41:12 +0530, Abhijit Menon-Sen wrote: >> There are two or three places in the Postgres source that implement heap >> sort (e.g. tuplesort.c, nodeMergeAppend.c), and it's also needed by the >> BDR code. It seemed reasonable t

Re: [HACKERS] [PATCH 03/14] Add simple xlogdump tool

2012-11-15 Thread Andres Freund
On 2012-11-15 11:31:55 -0500, Peter Eisentraut wrote: > On 11/14/12 8:17 PM, Andres Freund wrote: > > diff --git a/src/bin/Makefile b/src/bin/Makefile > > index b4dfdba..9992f7a 100644 > > --- a/src/bin/Makefile > > +++ b/src/bin/Makefile > > @@ -14,7 +14,7 @@ top_builddir = ../.. > > include $(to

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Merlin Moncure
On Thu, Nov 15, 2012 at 10:25 AM, Amit Kapila wrote: >> IMNSHO. deferring non-critical i/o from foreground process to >> background process is generally good. > > Yes, in regard of deferring you are right. > However in this case may be when foreground process has to mark page dirty > due to hint-b

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Andres Freund
On 2012-11-15 11:37:16 -0500, Robert Haas wrote: > On Thu, Nov 15, 2012 at 11:22 AM, Andres Freund > wrote: > > On 2012-11-14 18:41:12 +0530, Abhijit Menon-Sen wrote: > >> There are two or three places in the Postgres source that implement heap > >> sort (e.g. tuplesort.c, nodeMergeAppend.c), and

Re: [HACKERS] [PATCH 03/14] Add simple xlogdump tool

2012-11-15 Thread Jeff Janes
On Wed, Nov 14, 2012 at 5:17 PM, Andres Freund wrote: > --- > src/bin/Makefile| 2 +- > src/bin/xlogdump/Makefile | 25 +++ > src/bin/xlogdump/xlogdump.c | 468 > > 3 files changed, 494 insertions(+), 1 deletion(-) > create mode 1006

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 11:50 AM, Andres Freund wrote: > Me neither. I was thinking about letting the "user" allocate enough > memory like: > > binaryheap *heap = palloc(binaryheap_size(/*capacity*/ 10)); > binaryheap_init(heap, 10, comparator); > > But thats pretty ugly. Yeah. I would vote agai

Re: [HACKERS] [PATCH 03/14] Add simple xlogdump tool

2012-11-15 Thread Andres Freund
On 2012-11-15 09:06:23 -0800, Jeff Janes wrote: > On Wed, Nov 14, 2012 at 5:17 PM, Andres Freund wrote: > > --- > > src/bin/Makefile| 2 +- > > src/bin/xlogdump/Makefile | 25 +++ > > src/bin/xlogdump/xlogdump.c | 468 > > > > 3 files

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 16:09, Robert Haas wrote: [Lots of complicated commentary on tuplesort variables] > Whew. In the attached version, I > updated the comment to reflect this, and also reversed the order of > the if/else block to put the shorter and more common case first, which > I think is bett

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 16:09, Robert Haas wrote: > I'm still not too sure why this part is OK: > > /* This may not be our first time through */ > if (newmemtupsize <= memtupsize) > return false; > > Suppose we enlarge the memtuples array by something other than a > multipl

[HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Peter Eisentraut
Independent of the discussion of how to edit configuration files from SQL, I had another idea how many of the use cases for this could be handled. We already have the ability to store in pg_db_role_setting configuration settings for specific user, specific database specific user, any database any

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-15 Thread Cédric Villemain
Le jeudi 15 novembre 2012 15:48:14, Amit kapila a écrit : > On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: > > On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila wrote: > > Uh, no, I don't think that's a good idea. IMHO, what we should do is: > > > > 1. Read postgresql.conf.auto and re

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 12:53 PM, Peter Eisentraut wrote: > Independent of the discussion of how to edit configuration files from > SQL, I had another idea how many of the use cases for this could be handled. > > We already have the ability to store in pg_db_role_setting configuration > settings f

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Cédric Villemain
Le jeudi 15 novembre 2012 18:53:15, Peter Eisentraut a écrit : > Independent of the discussion of how to edit configuration files from > SQL, I had another idea how many of the use cases for this could be > handled. > > We already have the ability to store in pg_db_role_setting configuration > set

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 12:14 PM, Peter Geoghegan wrote: > On 15 November 2012 16:09, Robert Haas wrote: > [Lots of complicated commentary on tuplesort variables] >> Whew. In the attached version, I >> updated the comment to reflect this, and also reversed the order of >> the if/else block to pu

[HACKERS] pg_ctl reload -o "...."

2012-11-15 Thread Jeff Janes
If I want to change a parameter that affects an auxiliary process (like bgwriter), I can usually get away with doing; pg_ctl restart -o '--setting=new' But sometimes I really need to avoid the restart, because it blows away shared_buffers or for other reasons. I can do pg_ctl reload, but that ig

Re: [HACKERS] Timing events WIP v1

2012-11-15 Thread Josh Berkus
> Extending this to save the key/value set and most of the other data I > mentioned before is pretty straightforward. Why not use Hstore? Seriously? It would require merging Hstore into core, but I don't necessarily see that as a bad thing. -- Josh Berkus PostgreSQL Experts Inc. http://pgexp

Re: [HACKERS] Materialized views WIP patch

2012-11-15 Thread Josh Berkus
Kevin, > Attached is a patch that is still WIP but that I think is getting > pretty close to completion. It is not intended to be the be-all and > end-all for materialized views, but the minimum useful feature set -- > which is all that I've had time to do for this release. In > particular, the vi

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-15 Thread Josh Berkus
> ApplySetting(InvalidOid, InvalidOid, relsetting, PGC_S_$SOMETHING); > > in postinit.c, and have some SQL command to modify this setting. > > The only thing you couldn't handle that way are SIGHUP settings, but the > often-cited use cases work_mem, logging, etc. would work. > > There would als

Re: [HACKERS] WIP patch: add (PRE|POST)PROCESSOR options to COPY

2012-11-15 Thread Robert Haas
On Wed, Nov 14, 2012 at 10:14 PM, Craig Ringer wrote: >> So? You're already handing the keys to the kingdom to anybody who can >> control the contents of that command line, even if it's only to point at >> the wrong program. And one man's "unexpected side-effect" is another >> man's "essential f

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 18:13, Robert Haas wrote: > Ah, yeah. I wondered in passing about that but forgot to follow up on > it. The problem specifically is that the intermediate result > memtupsize * newmemtuples might overflow. I believe that the old > memtupsize can never be more than 2^26 bytes,

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Robert Haas
On Thu, Nov 15, 2012 at 2:13 PM, Peter Geoghegan wrote: > On 15 November 2012 18:13, Robert Haas wrote: >> Ah, yeah. I wondered in passing about that but forgot to follow up on >> it. The problem specifically is that the intermediate result >> memtupsize * newmemtuples might overflow. I believ

Re: [HACKERS] pg_ctl reload -o "...."

2012-11-15 Thread Tom Lane
Jeff Janes writes: > Is there a reason pg_ctl reload shouldn't honor -o ? -o means "pass these switches on the postmaster's command line". If you're not restarting the postmaster, you don't get to change its command line. IMO setting stuff on the command line is pretty evil anyway. Adjusting pos

Re: [HACKERS] WIP patch: add (PRE|POST)PROCESSOR options to COPY

2012-11-15 Thread Tom Lane
Robert Haas writes: > Yeah. If we're going to do this at all, and I'm not convinced it's > worth the work, I think it's definitely good to support a variant > where we specify exactly the things that will be passed to exec(). > There's just too many ways to accidentally shoot yourself in the foot

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Peter Geoghegan
On 15 November 2012 19:16, Robert Haas wrote: > So what's next here? Do you want to work on these issue some more? > Or does Jeff? I'd like to see this go in, but I'm not sure I have the > bandwidth to do the legwork myself. I'll take another look. No elegant solution immediately occurs to me,

[HACKERS] pg_trgm partial-match

2012-11-15 Thread Fujii Masao
Hi, I'd like to propose to extend pg_trgm so that it can compare a partial-match query key to a GIN index. IOW, I'm thinking to implement the 'comparePartial' GIN method for pg_trgm. Currently, when the query key is less than three characters, we cannot use a GIN index (+ pg_trgm) efficiently, be

Re: [HACKERS] Dumping an Extension's Script

2012-11-15 Thread Robert Haas
On Mon, Nov 12, 2012 at 11:00 AM, Dimitri Fontaine wrote: > Please find attached to this email an RFC patch implementing the basics > of the pg_dump --extension-script option. After much discussion around > the concept of an inline extension, we decided last year that a good > first step would be

Re: [HACKERS] support for LDAP URLs

2012-11-15 Thread Robert Haas
On Mon, Nov 12, 2012 at 10:38 PM, Peter Eisentraut wrote: > Here is a patch to support RFC 2255 LDAP URLs in pg_hba.conf. So, > instead of, say > > host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" > ldapsearchattribute=uid > > you could write > > host ... ldap lapurl="ld

  1   2   >