Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 05:38:14 Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I'm all in favor of having some memory ordering primitives so that we can try to implement better algorithms, but if we use it here it amounts to a fairly significant escalation in the minimum

Re: [HACKERS] contrib: auth_delay module

2010-11-19 Thread KaiGai Kohei
(2010/11/18 2:17), Robert Haas wrote: On Wed, Nov 17, 2010 at 10:32 AM, Ross J. Reedstromreeds...@rice.edu wrote: On Tue, Nov 16, 2010 at 09:41:37PM -0500, Robert Haas wrote: On Tue, Nov 16, 2010 at 8:15 PM, KaiGai Koheikai...@ak.jp.nec.com wrote: If we don't need a PoC module for each new

Re: [HACKERS] Label switcher function

2010-11-19 Thread KaiGai Kohei
(2010/11/18 11:30), Robert Haas wrote: 2010/11/17 KaiGai Koheikai...@ak.jp.nec.com: I revised my patch as I attached. The hook function is modified and consolidated as follows: typedef enum FunctionCallEventType { FCET_BE_HOOKED, FCET_PREPARE, FCET_START,

Re: [HACKERS] SQL/MED estimated time of arrival?

2010-11-19 Thread Heikki Linnakangas
Some random comments on the patch: ReleaseConnection is a very generic name for a global function, would be good to prefix it with pgsqlfdw or something. Same with any other globally visible functions. Please use the built-in contain_mutable_functions(Node *) instead of custom

Re: [HACKERS] Fate of the old cvs repository

2010-11-19 Thread Magnus Hagander
On Sun, Oct 17, 2010 at 19:15, David Fetter da...@fetter.org wrote: On Sun, Oct 17, 2010 at 05:54:04PM +0200, Magnus Hagander wrote: So, it seems we're pretty firmly on git now, and I doubt we're ever going to shift back now :) That means I'd like to get the two CVS VMs shut down (that's

Re: [HACKERS] Isn't HANDLE 64 bits on Win64?

2010-11-19 Thread Tom Lane
Dave Page dp...@pgadmin.org writes: On Tue, Nov 16, 2010 at 10:01 AM, Magnus Hagander mag...@hagander.net wrote: On Tue, Nov 16, 2010 at 01:35, Tom Lane t...@sss.pgh.pa.us wrote: BTW, it seems like it'd be a good thing if we had a Win64 machine in the buildfarm. Yes. I actually thought we

Re: [HACKERS] Improving prep_buildtree used in VPATH builds

2010-11-19 Thread Alvaro Herrera
Excerpts from Greg Smith's message of vie nov 19 01:52:34 -0300 2010: I'd think that if configure takes longer than it has to because the system is heavily loaded, the amount compilation time is going to suffer from that would always dwarf this component of total build time. But if this

Re: [HACKERS] Changes to Linux OOM killer in 2.6.36

2010-11-19 Thread Kevin Grittner
Greg Smith wrote: oom_adj is deprecated, scheduled for removal in August 2010: That surprised me so I checked the URL. I believe you have a typo there and it's August, 2012. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Thu, Nov 18, 2010 at 11:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I'm all in favor of having some memory ordering primitives so that we can try to implement better algorithms, but if we use it here it amounts to a fairly significant escalation in

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 3:07 AM, Andres Freund and...@anarazel.de wrote: So the complicated case seems to be !defined(HAS_TEST_AND_SET) which uses spinlocks for that purpose - no idea where that is true these days. Me neither, which is exactly the problem. Under Tom's proposal, any

Re: [HACKERS] libpq changes for synchronous replication

2010-11-19 Thread Robert Haas
On Thu, Nov 18, 2010 at 7:43 AM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Nov 16, 2010 at 10:49 AM, Robert Haas robertmh...@gmail.com wrote: Just in a quick scan, I don't have any objection to v2 except that the protocol documentation is lacking. OK, I'll mark it Waiting on Author

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 15:16:24 Robert Haas wrote: On Fri, Nov 19, 2010 at 3:07 AM, Andres Freund and...@anarazel.de wrote: So the complicated case seems to be !defined(HAS_TEST_AND_SET) which uses spinlocks for that purpose - no idea where that is true these days. Me neither, which is

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 9:27 AM, Aidan Van Dyk ai...@highrise.ca wrote: On Fri, Nov 19, 2010 at 9:16 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Nov 19, 2010 at 3:07 AM, Andres Freund and...@anarazel.de wrote: So the complicated case seems to be !defined(HAS_TEST_AND_SET) which uses

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Aidan Van Dyk
On Fri, Nov 19, 2010 at 9:16 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Nov 19, 2010 at 3:07 AM, Andres Freund and...@anarazel.de wrote: So the complicated case seems to be !defined(HAS_TEST_AND_SET) which uses spinlocks for that purpose - no idea where that is true these days. Me

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 15:29:10 Andres Freund wrote: Besides, we can just jump into the kernel and back in that case (which the TAS implementation already does), that does more than just a fence... Or if you don't believe that is enough initialize a lock on the stack, lock and forget it...

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 9:29 AM, Andres Freund and...@anarazel.de wrote: On Friday 19 November 2010 15:16:24 Robert Haas wrote: On Fri, Nov 19, 2010 at 3:07 AM, Andres Freund and...@anarazel.de wrote: So the complicated case seems to be !defined(HAS_TEST_AND_SET) which uses spinlocks for

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 9:35 AM, Aidan Van Dyk ai...@highrise.ca wrote: On Fri, Nov 19, 2010 at 9:31 AM, Robert Haas robertmh...@gmail.com wrote: Just a small point of clarification - you need to have both that unknown archtecture, and that architecture has to have postgres process running

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 15:38:37 Robert Haas wrote: Eh, really? If there's a workaround for platforms for which we don't know what the appropriate read-fencing incantation is, then I'd feel more comfortable about doing this. But I don't see how to make that work. The whole problem here

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 15:14:58 Robert Haas wrote: On Thu, Nov 18, 2010 at 11:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I'm all in favor of having some memory ordering primitives so that we can try to implement better algorithms, but if we use

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 15:49:45 Robert Haas wrote: If we're going to work on memory primitives, I would much rather see us put that effort into, say, implementing more efficient LWLock algorithms to solve the bottlenecks that the MOSBENCH guys found, rather than spending it on trying to

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 9:51 AM, Andres Freund and...@anarazel.de wrote: On Friday 19 November 2010 15:49:45 Robert Haas wrote: If we're going to work on memory primitives, I would much rather see us put that effort into, say, implementing more efficient LWLock algorithms to solve the

Re: [HACKERS] Label switcher function

2010-11-19 Thread Robert Haas
2010/11/19 KaiGai Kohei kai...@ak.jp.nec.com: Indeed, the comment at middle of the fmgr_info_cxt_security() and just above definition of the fmgr_security_definer() are not correct. Did you notice anything else? I think I noticed a couple of places, but I didn't write down exactly which ones.

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Aidan Van Dyk
On Fri, Nov 19, 2010 at 9:49 AM, Andres Freund and...@anarazel.de wrote: Well, its not generally true - you are right there. But there is a wide range for syscalls available where its inherently true (which is what I sloppily referred to). And you are allowed to call a, although quite

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: If we're going to work on memory primitives, I would much rather see us put that effort into, say, implementing more efficient LWLock algorithms to solve the bottlenecks that the MOSBENCH guys found, rather than spending it on trying to avoid a minor

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 10:01 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: If we're going to work on memory primitives, I would much rather see us put that effort into, say, implementing more efficient LWLock algorithms to solve the bottlenecks that the

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Aidan Van Dyk
On Fri, Nov 19, 2010 at 9:31 AM, Robert Haas robertmh...@gmail.com wrote: Just a small point of clarification - you need to have both that unknown archtecture, and that architecture has to have postgres process running simultaneously on difference CPUs with different caches that are

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 15:58:39 Aidan Van Dyk wrote: On Fri, Nov 19, 2010 at 9:49 AM, Andres Freund and...@anarazel.de wrote: Well, its not generally true - you are right there. But there is a wide range for syscalls available where its inherently true (which is what I sloppily referred

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Markus Wanner
On 11/19/2010 03:58 PM, Aidan Van Dyk wrote: Well, it's not quite enough just to call into the kernel to serialize on some point of memory, because your point is to make sure that *this particular piece of memory* is coherent. Well, that certainly doesn't apply to full fences, that are not

Re: [HACKERS] duplicate connection failure messages

2010-11-19 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of vie nov 19 00:17:59 -0300 2010: Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mi nov 17 13:04:46 -0300 2010: OK, I doubt we want to add complexity to improve this, so I see our options as: o ignore the problem

Re: [HACKERS] libpq changes for synchronous replication

2010-11-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Nov 18, 2010 at 7:43 AM, Fujii Masao masao.fu...@gmail.com wrote: The patch is touching protocol.sgml as follows. Isn't this enough? How about some updates to the Message Flow section, especially the section on COPY Operations? Yeah. You're

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: ... The reason memory barriers solve the problem is because they'll be atomically released when we jump into the signal handler, but that is not true of a spin-lock or a semaphore. Hm, I wonder whether your concern is stemming from a wrong mental

[HACKERS] What do these terms mean in the SOURCE CODE?

2010-11-19 Thread Vaibhav Kaushal
I am going through the Executor code and come across the following terms quite often. Can someone tell me what do they mean (in a few (may be a couple of) sentences)? 1. Scan State 2. Plan State 3. Tuple Projection 4. EState 5. Qual 6. Expression They sound quite ambiguous in the source code,

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I completely agree, but I'm not too sure I want to drop support for any platform for which we haven't yet implemented such primitives. What's different about this case is that fall back to taking the spin lock is not a workable option. The point I was

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Markus Wanner mar...@bluegap.ch writes: Well, that certainly doesn't apply to full fences, that are not specific to a particular piece of memory. I'm thinking of 'mfence' on x86_64 or 'mf' on ia64. Hm, what do those do exactly? We've never had any such thing in the Intel-ish spinlock asm, but

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-19 Thread Pavel Stehule
I checked my tests and the most important is a remove a repeated detoast. postgres=# CREATE OR REPLACE FUNCTION public.filter01(text[], text, integer) RETURNS text[] LANGUAGE plpgsql AS $function$ DECLARE s text[] := '{}'; l int := 0; i int; v text; loc text[] = $1; BEGIN FOR i IN

Re: [HACKERS] duplicate connection failure messages

2010-11-19 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Bruce Momjian's message of vie nov 19 00:17:59 -0300 2010: Alvaro Herrera wrote: I think we should use inet_ntop where available to print the address. Good idea because inet_ntop() is thread-safe. Does that work on IPv6? You

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Markus Wanner
On 11/19/2010 04:51 PM, Tom Lane wrote: Hm, what do those do exactly? Performs a serializing operation on all load-from-memory and store-to-memory instructions that were issued prior the MFENCE instruction. [1] Given the memory ordering guarantees of x86, this instruction might only be relevant

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 16:51:00 Tom Lane wrote: Markus Wanner mar...@bluegap.ch writes: Well, that certainly doesn't apply to full fences, that are not specific to a particular piece of memory. I'm thinking of 'mfence' on x86_64 or 'mf' on ia64. Hm, what do those do exactly? We've

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Andres Freund and...@anarazel.de writes: Locked statments like 'lock xaddl;' guarantee that the specific operands (or their cachelines) are visible on all processors and are done atomically - but its not influencing the whole cache like mfence would. Where is this locking the whole cache

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 17:25:57 Tom Lane wrote: Andres Freund and...@anarazel.de writes: Locked statments like 'lock xaddl;' guarantee that the specific operands (or their cachelines) are visible on all processors and are done atomically - but its not influencing the whole cache like

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Andres Freund and...@anarazel.de writes: I was never talking about 'locking the whole cache' - I was talking about flushing/fencing it like a global read/write barrier would. And lock xchgb/xaddl does not imply anything for other cachelines but its own. If that's the case, why aren't the

Re: [HACKERS] libpq changes for synchronous replication

2010-11-19 Thread Alvaro Herrera
Excerpts from Tom Lane's message of vie nov 19 12:25:13 -0300 2010: Robert Haas robertmh...@gmail.com writes: On Thu, Nov 18, 2010 at 7:43 AM, Fujii Masao masao.fu...@gmail.com wrote: The patch is touching protocol.sgml as follows. Isn't this enough? How about some updates to the Message

Re: [HACKERS] libpq changes for synchronous replication

2010-11-19 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of vie nov 19 12:25:13 -0300 2010: Yeah. You're adding a new fundamental state to the protocol; it's not enough to bury that in the description of a message format. I don't think a whole lot of new verbiage is

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 10:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I completely agree, but I'm not too sure I want to drop support for any platform for which we haven't yet implemented such primitives. What's different about this case is that fall

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I think it would be useful to try to build up a library of primitives in this area. For this particular task, we really only need a write-with-fence primitive and a read-with-fence primitive. That's really entirely the wrong way to think about it.

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Florian Weimer
* Andres Freund: I was never talking about 'locking the whole cache' - I was talking about flushing/fencing it like a global read/write barrier would. And lock xchgb/xaddl does not imply anything for other cachelines but its own. My understanding is that once you've seen the result of an

Re: [HACKERS] contrib: auth_delay module

2010-11-19 Thread Ross J. Reedstrom
On Fri, Nov 19, 2010 at 04:57:03PM +0900, KaiGai Kohei wrote: (2010/11/18 2:17), Robert Haas wrote: If KaiGai updates the code per previous discussion, would you be willing to take a crack at adding documentation? P.S. Your email client seems to be setting the Reply-To address to a

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
I wrote: Markus Wanner mar...@bluegap.ch writes: Well, that certainly doesn't apply to full fences, that are not specific to a particular piece of memory. I'm thinking of 'mfence' on x86_64 or 'mf' on ia64. Hm, what do those do exactly? I poked around in the Intel manuals a bit. They do

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I think it would be useful to try to build up a library of primitives in this area. For this particular task, we really only need a write-with-fence primitive and a read-with-fence primitive. That's really

Re: [HACKERS] UNNEST ... WITH ORDINALITY (AND POSSIBLY OTHER STUFF)

2010-11-19 Thread Caleb.Welton
Note the standard also supports unnesting multiple arrays concurrently, the rule for handling arrays with different lengths is to use null padding of the shorter array. SELECT * FROM UNNEST( ARRAY[5,2,3,4], ARRAY['hello', 'world'] ) WITH ORDINALITY AS t(a,b,i); a b i

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: That's really entirely the wrong way to think about it. You need a fence primitive, full stop. It's a sequence point, not an operation in itself. I was taking it to mean something similar to the memory

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: What you typically need is a guarantee about the order in which writes become visible. In some cases you also need to guarantee the order of reads. Doesn't that suggest different primitives? -Kevin -- Sent via pgsql-hackers mailing list

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 18:46:00 Tom Lane wrote: I wrote: Markus Wanner mar...@bluegap.ch writes: Well, that certainly doesn't apply to full fences, that are not specific to a particular piece of memory. I'm thinking of 'mfence' on x86_64 or 'mf' on ia64. Hm, what do those do

Re: [HACKERS] UNNEST ... WITH ORDINALITY (AND POSSIBLY OTHER STUFF)

2010-11-19 Thread David Fetter
On Fri, Nov 19, 2010 at 01:48:06PM -0500, caleb.wel...@emc.com wrote: Note the standard also supports unnesting multiple arrays concurrently, the rule for handling arrays with different lengths is to use null padding of the shorter array. Interesting. I notice that our version doesn't support

[HACKERS] how correctly detoast a Datum value?

2010-11-19 Thread Pavel Stehule
Hello I try to explicitly detoast a plpgsql var, but I this code breaks a content. what is wrong? switch (datum-dtype) { case PLPGSQL_DTYPE_VAR: { PLpgSQL_var *var = (PLpgSQL_var *) datum;

Re: [HACKERS] UNNEST ... WITH ORDINALITY (AND POSSIBLY OTHER STUFF)

2010-11-19 Thread Alvaro Herrera
Excerpts from Caleb.Welton's message of vie nov 19 15:48:06 -0300 2010: Note the standard also supports unnesting multiple arrays concurrently, the rule for handling arrays with different lengths is to use null padding of the shorter array. SELECT * FROM UNNEST( ARRAY[5,2,3,4],

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Friday 19 November 2010 20:03:27 Andres Freund wrote: Which means something like (in intel's terminology) can happen: initially x = 0 P1: mov [_X], 1 P1: lock xchg Y, 1 P2. lock xchg [_Z], 1 P2: mov r1, [_X] A valid result is that r1 on P2 is 0. I think that is not biting pg

Re: [HACKERS] duplicate connection failure messages

2010-11-19 Thread Bruce Momjian
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Bruce Momjian's message of vie nov 19 00:17:59 -0300 2010: Alvaro Herrera wrote: I think we should use inet_ntop where available to print the address. Good idea because inet_ntop() is thread-safe. Does

Re: [HACKERS] directory archive format for pg_dump

2010-11-19 Thread Dimitri Fontaine
Hi, Sharing some thoughts after a first round of reviewing, where I only had time to read the patch itself. Joachim Wieland j...@mcknight.de writes: Since the compression is currently all down in the custom format backup code, the first thing I've done was refactoring the compression

Re: [HACKERS] duplicate connection failure messages

2010-11-19 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of vie nov 19 16:43:33 -0300 2010: Tom Lane wrote: I get the impression that you guys have forgotten the existence of src/backend/utils/adt/inet_net_ntop.c Yeah, that is nice, but we are calling this from libpq, not the backend. Let me work up a

Re: [HACKERS] duplicate connection failure messages

2010-11-19 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of vie nov 19 16:43:33 -0300 2010: Tom Lane wrote: I get the impression that you guys have forgotten the existence of src/backend/utils/adt/inet_net_ntop.c Yeah, that is nice, but we are calling this from libpq, not the

Re: [HACKERS] Changes to Linux OOM killer in 2.6.36

2010-11-19 Thread Alex Hunsaker
On Thu, Nov 18, 2010 at 19:43, Greg Smith g...@2ndquadrant.com wrote: Last month's new Linux kernel 2.6.36 includes a rewrite of the out of memory killer: http://lwn.net/Articles/391222/

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 1:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: However, for lock-free interactions I think this model isn't terribly helpful: it's not clear what is inside and what is outside the sync block, and forcing your code into that model doesn't improve either clarity or

Re: [HACKERS] UNNEST ... WITH ORDINALITY (AND POSSIBLY OTHER STUFF)

2010-11-19 Thread Caleb.Welton
The other aspect of the standard that the Postgres implementation does not currently support is the fact that unnest is supposed to be defined in terms of laterally derived subqueries, e.g. you should be able to unnest another element from a from list entry laterally on the left. CREATE TABLE

Re: [HACKERS] [PATCH] Custom code int(32|64) = text conversions out of performance reasons

2010-11-19 Thread Andres Freund
On Monday 15 November 2010 17:12:25 Robert Haas wrote: I notice that int8out isn't terribly consistent with int2out and int4out, in that it does an extra copy. Maybe that's justified given the greater potential memory wastage, but I'm not certain. One approach might be to pick some

Re: [HACKERS] directory archive format for pg_dump

2010-11-19 Thread José Arthur Benetasso Villanova
Hi Dimitri and Joachim. I've looked the patch too, and I want to share some thoughts too. I've used http://wiki.postgresql.org/wiki/Reviewing_a_Patch to guide my review. Submission review: I've apllied and compiled the patch successfully using the current master. Usability review: The dir

Re: [HACKERS] Changes to Linux OOM killer in 2.6.36

2010-11-19 Thread Greg Smith
Kevin Grittner wrote: Greg Smith wrote: oom_adj is deprecated, scheduled for removal in August 2010: That surprised me so I checked the URL. I believe you have a typo there and it's August, 2012. This is why I include references, so that when the cold medicine hits me in

[HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-19 Thread Joachim Wieland
Hi, I am seeing the following here on 9.0.1 on Linux x86-64: LOG: redo starts at 1F8/FC00E978 FATAL: too many KnownAssignedXids CONTEXT: xlog redo insert: rel 1663/16384/18373; tid 3829898/23 and this is the complete history: postgres was running as HS in foreground, Ctrl-C'ed it for a

Re: [HACKERS] directory archive format for pg_dump

2010-11-19 Thread Alvaro Herrera
Excerpts from José Arthur Benetasso Villanova's message of vie nov 19 18:28:03 -0300 2010: The md5.c and kwlookup.c reuse using a link doesn't look nice either. This way you need to compile twice, among others things, but I think that its temporary, right? Not sure what you mean here, but

Re: [HACKERS] directory archive format for pg_dump

2010-11-19 Thread Joachim Wieland
Hi Dimitri, thanks for reviewing my patch! On Fri, Nov 19, 2010 at 2:44 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: I think I'd like to see a separate patch for the new compression support. Sorry about that, I realize that's extra work… I guess it wouldn't be a very big deal but I also

Re: [HACKERS] directory archive format for pg_dump

2010-11-19 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: I think I'd like to see a separate patch for the new compression support. Sorry about that, I realize that's extra work… That part of the patch is likely to get rejected outright anyway, so I *strongly* recommend splitting it out. We have

Re: [HACKERS] duplicate connection failure messages

2010-11-19 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: I was wondering that. I am unclear if we need it though --- can we not assume inet_ntop() exists on all systems? We assumed inet_ntoa() did. The Single Unix Spec includes inet_ntoa but not inet_ntop. Of course, the buildfarm will tell us. The

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: But what about timings vs. random other stuff? Like in this case there's a problem if the signal arrives before the memory update to latch-is_set becomes visible. I don't know what we need to do to guarantee that. I don't believe there's an issue

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On Friday 19 November 2010 18:46:00 Tom Lane wrote: I poked around in the Intel manuals a bit. They do have mfence (also lfence and sfence) but so far as I can tell, those are only used to manage loads and stores that are issued by special instructions

Re: [HACKERS] Fwd: patch: format function - fixed oid

2010-11-19 Thread Jeff Janes
On Thu, Nov 18, 2010 at 11:54 PM, Pavel Stehule pavel.steh...@gmail.com wrote: -- Forwarded message -- From: Pavel Stehule pavel.steh...@gmail.com Date: 2010/11/18 Subject: Re: patch: format function, next generation To: Jeff Janes jeff.ja...@gmail.com Kopie:

Re: [HACKERS] directory archive format for pg_dump

2010-11-19 Thread Joachim Wieland
On Fri, Nov 19, 2010 at 11:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dimitri Fontaine dimi...@2ndquadrant.fr writes: I think I'd like to see a separate patch for the new compression support. Sorry about that, I realize that's extra work… That part of the patch is likely to get rejected

Re: Latches with weak memory ordering (Re: [HACKERS] max_wal_senders must die)

2010-11-19 Thread Andres Freund
On Saturday 20 November 2010 00:08:07 Tom Lane wrote: Andres Freund and...@anarazel.de writes: On Friday 19 November 2010 18:46:00 Tom Lane wrote: I poked around in the Intel manuals a bit. They do have mfence (also lfence and sfence) but so far as I can tell, those are only used to

Re: [HACKERS] Patch to add a primary key using an existing index

2010-11-19 Thread Steve Singer
On 10-11-07 01:54 PM, Gurjeet Singh wrote: Attached is the patch that extends the same feature for UNIQUE indexes. It also includes some doc changes for the ALTER TABLE command, but I could not verify the resulting changes since I don't have the doc-building infrastructure installed. Regards,

Re: [HACKERS] UNNEST ... WITH ORDINALITY (AND POSSIBLY OTHER STUFF)

2010-11-19 Thread David Fetter
On Fri, Nov 19, 2010 at 04:11:56PM -0500, caleb.wel...@emc.com wrote: The other aspect of the standard that the Postgres implementation does not currently support is the fact that unnest is supposed to be defined in terms of laterally derived subqueries, e.g. you should be able to unnest

Re: [HACKERS] [PATCH] Custom code int(32|64) = text conversions out of performance reasons

2010-11-19 Thread Robert Haas
On Fri, Nov 19, 2010 at 4:16 PM, Andres Freund and...@anarazel.de wrote: On Monday 15 November 2010 17:12:25 Robert Haas wrote: I notice that int8out isn't terribly consistent with int2out and int4out, in that it does an extra copy.   Maybe that's justified given the greater potential memory

Re: [HACKERS] Fix for seg picksplit function

2010-11-19 Thread Robert Haas
On Tue, Nov 16, 2010 at 6:07 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Tue, Nov 16, 2010 at 3:07 AM, Robert Haas robertmh...@gmail.com wrote: But on a broader note, I'm not very certain the sorting algorithm is sensible.  For example, suppose you have 10 segments that are exactly

[HACKERS] Fwd: What do these terms mean in the SOURCE CODE?

2010-11-19 Thread Vaibhav Kaushal
Is no one ready to help on this? :( -Vaibhav -- Forwarded message -- From: Vaibhav Kaushal vaibhavkaushal...@gmail.com Date: Fri, Nov 19, 2010 at 9:11 PM Subject: What do these terms mean in the SOURCE CODE? To: pgsql-hackers@postgresql.org I am going through the Executor code

Re: [HACKERS] directory archive format for pg_dump

2010-11-19 Thread Joachim Wieland
Hi Jose, 2010/11/19 José Arthur Benetasso Villanova jose.art...@gmail.com: The dir format generated in my database 60 files, with different sizes, and it looks very confusing. Is it possible to use the same trick as pigz and pbzip2, creating a concatenated file of streams? What pigz is

Re: [HACKERS] Isn't HANDLE 64 bits on Win64?

2010-11-19 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Tue, Nov 16, 2010 at 11:01, Magnus Hagander mag...@hagander.net wrote: So yes, it looks completely broken. I guess Windows doesn't actually *assign* you a handle larger than 2^32 until you actually ahve that many open handles. Typical values on my

Re: [HACKERS] UNNEST ... WITH ORDINALITY (AND POSSIBLY OTHER STUFF)

2010-11-19 Thread Itagaki Takahiro
On Sat, Nov 20, 2010 at 03:48, caleb.wel...@emc.com wrote: Note the standard also supports unnesting multiple arrays concurrently, the rule for handling arrays with different lengths is to use null padding of the shorter array.   UNNEST( ARRAY[5,2,3,4],           ARRAY['hello', 'world'] )

Re: [HACKERS] Isn't HANDLE 64 bits on Win64?

2010-11-19 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Tue, Nov 16, 2010 at 16:23, Tom Lane t...@sss.pgh.pa.us wrote: What's not clear to me is whether the section title means that only certain handles have this guarantee, and if so whether we have to worry about running into ones that don't. I