[HACKERS] Fwd: Patch for reserved connections for replication users

2013-10-07 Thread Amit Kapila
Sorry missed to keep -hackers in previous mail. From: Amit Kapila amit.kapil...@gmail.com Date: Mon, Oct 7, 2013 at 11:37 AM Subject: Re: Patch for reserved connections for replication users To: Robert Haas robertmh...@gmail.com, gibh...@zero-knowledge.org Cc: Andres Freund

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 06.10.2013 20:34, Kevin Grittner wrote: Note this comment, which I think was written by Heikki back when there was a lot more benchmarking and profiling of SSI going on: * Because a particular target might become obsolete, due to update to a new * version, before the reading

[HACKERS] old warning in docs

2013-10-07 Thread Andrew Dunstan
Given that we have not supported releases older than 8.3 for quite a while, do we need to keep this in extend.sgml any longer? caution para Changing varnamePG_CONFIG/varname only works when building against productnamePostgreSQL/productname 8.3 or later. With older

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-07 Thread Fabien COELHO
pgbench already offers two schedules of pgbench --initialize messaging, message-per-100k-rows and message-per-5s. A user too picky to find satisfaction in either option can filter the messages through grep, sed et al. We patched pgbench on two occasions during the 9.3 cycle to arrive at that

Re: [HACKERS] Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption

2013-10-07 Thread k...@rice.edu
On Mon, Oct 07, 2013 at 12:41:58AM +0200, Tomas Vondra wrote: 2. Consider using a simpler/faster hash function, like FNV[1] or Jenkins[2]. For fun, try not hashing those ints at all and see how that performs (that, I think, is what you get from HashSetint in Java/C#). I've used

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: So I don't think you can ever get a false conflict because of slot reuse. I spent some time looking at this, and I now agree. And if there's a hole in that thinking I can't see right now, the worst that will happen is some unnecessary

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Patch attached.  Any objections to applying that Real Soon Now? Oh, without the new line in predicate.h. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] logical changeset generation v6.2

2013-10-07 Thread Steve Singer
On 10/03/2013 04:00 PM, Andres Freund wrote: Ok, there were a couple of bugs because I thought mxacts wouldn't need to be supported. So far your testcase doesn't crash the database anymore - it spews some internal errors though, so I am not sure if it's entirely fixed for you. Thanks for

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Andres Freund
On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached.  Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe it's overly careful, but I personally slightly vote for applying it after the backbranch releases.

Re: [HACKERS] logical changeset generation v6.2

2013-10-07 Thread Andres Freund
On 2013-10-07 09:56:11 -0400, Steve Singer wrote: On 10/03/2013 04:00 PM, Andres Freund wrote: Ok, there were a couple of bugs because I thought mxacts wouldn't need to be supported. So far your testcase doesn't crash the database anymore - it spews some internal errors though, so I am not

Re: [HACKERS] ToDo: fast update of arrays with fixed length fields for PL/pgSQL

2013-10-07 Thread Pavel Stehule
Hello I fixed patch - there was a missing cast to domain when it was used (and all regress tests are ok now). a some performance tests set array_fast_update to off; postgres=# select fill_2d_array(300,300); fill_2d_array ─── 9 (1 row) Time: 33570.087 ms postgres=# set

Re: [HACKERS] ToDo: fast update of arrays with fixed length fields for PL/pgSQL

2013-10-07 Thread Andres Freund
On 2013-10-07 16:00:54 +0200, Pavel Stehule wrote: /* * We need to do subscript evaluation, which might require @@ -4321,6 +4322,14 @@ exec_assign_value(PLpgSQL_execstate *estate, oldarrayval =

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 16:58, Andres Freund wrote: On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached. Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe it's overly careful, but I personally slightly vote for

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 16:44, Kevin Grittner wrote: Heikki Linnakangashlinnakan...@vmware.com wrote: So I don't think you can ever get a false conflict because of slot reuse. I spent some time looking at this, and I now agree. And if there's a hole in that thinking I can't see right now, the worst

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Andres Freund
On 2013-10-07 17:07:16 +0300, Heikki Linnakangas wrote: On 07.10.2013 16:58, Andres Freund wrote: On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached. Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: Well, it's fairly harsh if the feature isn't working as advertised. Right -- there are people counting on serializable transaction to avoid data corruption (creation of data which violates the business rules which they believe are being

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached.  Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe it's overly careful, but I personally slightly vote

Re: [HACKERS] ToDo: fast update of arrays with fixed length fields for PL/pgSQL

2013-10-07 Thread Pavel Stehule
2013/10/7 Andres Freund and...@2ndquadrant.com On 2013-10-07 16:00:54 +0200, Pavel Stehule wrote: /* * We need to do subscript evaluation, which might require @@ -4321,6 +4322,14 @@ exec_assign_value(PLpgSQL_execstate *estate,

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-10-07 06:44:19 -0700, Kevin Grittner wrote: Patch attached.  Any objections to applying that Real Soon Now? (When, exactly is the deadline to make today's minor release cut-off?) Maybe it's overly

Re: [HACKERS] Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption

2013-10-07 Thread Atri Sharma
Sent from my iPad On 07-Oct-2013, at 4:11, Tomas Vondra t...@fuzzy.cz wrote: On 6.10.2013 20:37, Tomáš Janoušek wrote: Hi, On Sat, Oct 05, 2013 at 08:22:54PM +0200, Tomas Vondra wrote: I'm on 64-bit architecture and the example works with int32, which means the sizes should be about

Re: [HACKERS] old warning in docs

2013-10-07 Thread David Fetter
On Mon, Oct 07, 2013 at 07:51:44AM -0400, Andrew Dunstan wrote: Given that we have not supported releases older than 8.3 for quite a while, do we need to keep this in extend.sgml any longer? caution para Changing varnamePG_CONFIG/varname only works when building against

[HACKERS] segfault with contrib lo

2013-10-07 Thread Marc Cousin
I was using the lo contrib a few days ago and wasn't paying attention, and forgot the for each row in the create trigger command... PostgreSQL segfaulted, when the trigger tried to access the row's attributes. Please find attached a patch to control that the trigger is correctly defined (as in

Re: [HACKERS] space reserved for WAL record does not match what was written: panic on windows

2013-10-07 Thread Robert Haas
On Fri, Oct 4, 2013 at 8:19 AM, Andres Freund and...@2ndquadrant.com wrote: Could it be that MAXALIGN/TYPEALIGN doesn't really work for values bigger than 32bit? #define MAXALIGN(LEN) TYPEALIGN(MAXIMUM_ALIGNOF, (LEN)) #define TYPEALIGN(ALIGNVAL,LEN) \ (((intptr_t)

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2013-10-07 Thread Robert Haas
On Fri, Oct 4, 2013 at 3:20 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-04 15:15:36 -0400, Robert Haas wrote: Andres, are you (or is anyone) going to try to fix this assertion failure? I think short term replacing it by IsTransactionOrTransactionBlock() is the way to go.

Re: [HACKERS] Patch: FORCE_NULL option for copy COPY in CSV mode

2013-10-07 Thread Robert Haas
On Sat, Oct 5, 2013 at 7:38 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Sun, Sep 29, 2013 at 1:39 PM, Ian Lawrence Barwick barw...@gmail.com wrote: Hi, This patch implements the following TODO item: Allow COPY in CSV mode to control whether a quoted zero-length string is treated

Re: [HACKERS] Patch: FORCE_NULL option for copy COPY in CSV mode

2013-10-07 Thread Andrew Dunstan
On 10/07/2013 03:06 PM, Robert Haas wrote: Also if your use case is to treat empty strings as NULL (as per above documentation), can't it be handled with WITH NULL AS option. For example, something like: postgres=# COPY testnull FROM stdin with CSV NULL AS E''; Enter data to be copied

Re: [HACKERS] plpgsql.print_strict_params

2013-10-07 Thread Robert Haas
On Sat, Oct 5, 2013 at 6:15 PM, Marko Tiikkaja ma...@joh.to wrote: Something like the attached? Looks good to me. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption

2013-10-07 Thread Tomas Vondra
On 7.10.2013 14:50, k...@rice.edu wrote: On Mon, Oct 07, 2013 at 12:41:58AM +0200, Tomas Vondra wrote: 2. Consider using a simpler/faster hash function, like FNV[1] or Jenkins[2]. For fun, try not hashing those ints at all and see how that performs (that, I think, is what you get from

Re: [HACKERS] [COMMITTERS] pgsql: Add DISCARD SEQUENCES command.

2013-10-07 Thread Robert Haas
On Sat, Oct 5, 2013 at 8:10 AM, Michael Paquier michael.paqu...@gmail.com wrote: Here is the test case failing: =# create sequence foo; CREATE SEQUENCE =# select nextval('foo'); nextval - 1 (1 row) =# discard sequences ; DISCARD SEQUENCES =# select currval('foo');

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: There might be a problem if someone applies this fix while any prepared transactions are pending.  Still investigating the impact and possible fixes. I found a one-line fix for this.  It tested without problem. Pushed. -- Kevin Grittner EDB:

Re: [HACKERS] insert throw error when year field len 4 for timestamptz datatype

2013-10-07 Thread Robert Haas
On Mon, Oct 7, 2013 at 12:41 AM, Rushabh Lathia rushabh.lat...@gmail.com wrote: Hmm right it has some inconsistency when year length is 6. But the patch is based on assumption that 5-digit number is a year, because YMD and HMS require at least six digits. Now Year with 6-digit number its

Re: [HACKERS] pgbench progress report improvements - split 3 v2 - A

2013-10-07 Thread Robert Haas
On Sat, Oct 5, 2013 at 6:10 PM, Noah Misch n...@leadboat.com wrote: The sum of the squares of the latencies wraps after 2^63/(10^12 * avg_latency * nclients) seconds. That's unlikely to come up with the ordinary pgbench script, but one can reach it in a few hours when benchmarking a command

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Dan Ports
On Mon, Oct 07, 2013 at 12:26:37PM +0300, Heikki Linnakangas wrote: When updating a tuple, CheckTargetForConflictsIn() only marks a conflict if the transaction holding the predicate lock overlapped with the updating transaction. Ah, this is the bit I was forgetting. (I really ought to have

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 22:35, Kevin Grittner wrote: Kevin Grittnerkgri...@ymail.com wrote: There might be a problem if someone applies this fix while any prepared transactions are pending. Still investigating the impact and possible fixes. I found a one-line fix for this. It tested without

Re: [HACKERS] space reserved for WAL record does not match what was written: panic on windows

2013-10-07 Thread Andres Freund
On 2013-10-07 13:25:17 -0400, Robert Haas wrote: On Fri, Oct 4, 2013 at 8:19 AM, Andres Freund and...@2ndquadrant.com wrote: Could it be that MAXALIGN/TYPEALIGN doesn't really work for values bigger than 32bit? #define MAXALIGN(LEN) TYPEALIGN(MAXIMUM_ALIGNOF, (LEN))

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 23:45, Heikki Linnakangas wrote: On 07.10.2013 22:35, Kevin Grittner wrote: Kevin Grittnerkgri...@ymail.com wrote: There might be a problem if someone applies this fix while any prepared transactions are pending. Still investigating the impact and possible fixes. I found a

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2013-10-07 Thread Andres Freund
On 2013-10-07 15:02:36 -0400, Robert Haas wrote: On Fri, Oct 4, 2013 at 3:20 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-04 15:15:36 -0400, Robert Haas wrote: Andres, are you (or is anyone) going to try to fix this assertion failure? I think short term replacing it by

Re: [HACKERS] Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption

2013-10-07 Thread Tomas Vondra
Hi Atri! On 7.10.2013 16:56, Atri Sharma wrote: 3. Consider dropping buckets in favor of open addressing (linear probing, quadratic, whatever). This avoids another level of pointer indirection. OK, this sounds really interesting. It should be fairly straightforward for fixed-length data

Re: [HACKERS] space reserved for WAL record does not match what was written: panic on windows

2013-10-07 Thread Heikki Linnakangas
On 07.10.2013 23:47, Andres Freund wrote: On 2013-10-07 13:25:17 -0400, Robert Haas wrote: And does that indicate that intptr_t is the wrong type to be using here? No, I don't think so. intptr_t is defined to be a integer type to which you can cast a pointer, cast it back and still get the

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Peter Eisentraut
On Mon, 2013-10-07 at 23:49 +0300, Heikki Linnakangas wrote: On 07.10.2013 23:45, Heikki Linnakangas wrote: On 07.10.2013 22:35, Kevin Grittner wrote: Kevin Grittnerkgri...@ymail.com wrote: There might be a problem if someone applies this fix while any prepared transactions are pending.

Re: [HACKERS] [COMMITTERS] pgsql: Fix bugs in SSI tuple locking.

2013-10-07 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@iki.fi wrote: Fix bugs in SSI tuple locking. Thanks Heikki, both for these fixes and the discovery and discussion of the xmin issue! -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing

Re: [HACKERS] Bugfix and new feature for PGXS

2013-10-07 Thread Peter Eisentraut
On Sun, 2013-09-29 at 19:09 -0400, Andrew Dunstan wrote: On 09/03/2013 04:04 AM, Cédric Villemain wrote: Simple one, attached. I didn't document USE_VPATH, not sure how to explain that clearly. Just a remember that the doc is written and is waiting to be commited. There is also an issue

Re: [HACKERS] SSI freezing bug

2013-10-07 Thread Heikki Linnakangas
On 08.10.2013 03:25, Peter Eisentraut wrote: On Mon, 2013-10-07 at 23:49 +0300, Heikki Linnakangas wrote: To fix the bug that Hannu pointed out, we also need to apply these fixes: http://www.postgresql.org/message-id/52440266.5040...@vmware.com Per a chat with Bruce, I'm going to apply that

Re: [HACKERS] Bugfix and new feature for PGXS

2013-10-07 Thread Andrew Dunstan
On 10/07/2013 08:47 PM, Peter Eisentraut wrote: On Sun, 2013-09-29 at 19:09 -0400, Andrew Dunstan wrote: On 09/03/2013 04:04 AM, Cédric Villemain wrote: Simple one, attached. I didn't document USE_VPATH, not sure how to explain that clearly. Just a remember that the doc is written and is

Re: [HACKERS] Patch: FORCE_NULL option for copy COPY in CSV mode

2013-10-07 Thread Amit Kapila
On Tue, Oct 8, 2013 at 12:55 AM, Andrew Dunstan and...@dunslane.net wrote: On 10/07/2013 03:06 PM, Robert Haas wrote: Also if your use case is to treat empty strings as NULL (as per above documentation), can't it be handled with WITH NULL AS option. For example, something like: postgres=#

Re: [HACKERS] Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption

2013-10-07 Thread Atri Sharma
Hi Tomas, Consider the aspects associated with open addressing.Open addressing can quickly lead to growth in the main table.Also, chaining is a much cleaner way of collision resolution,IMHO. What do you mean by growth in the main table? Sorry, I should have been more verbose. AFAIK, Open

Re: [HACKERS] Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption

2013-10-07 Thread Claudio Freire
On Tue, Oct 8, 2013 at 1:23 AM, Atri Sharma atri.j...@gmail.com wrote: Consider the aspects associated with open addressing.Open addressing can quickly lead to growth in the main table.Also, chaining is a much cleaner way of collision resolution,IMHO. What do you mean by growth in the main