Re: [HACKERS] COPY and Volatile default expressions

2014-01-15 Thread Simon Riggs
On 16 April 2013 14:37, Simon Riggs si...@2ndquadrant.com wrote: On 16 April 2013 13:57, Heikki Linnakangas hlinnakan...@vmware.com wrote: You still need to check the args, if the function is nextval, otherwise you incorrectly perform the optimization for something like

Re: [HACKERS] COPY and Volatile default expressions

2013-06-28 Thread Simon Riggs
On 24 June 2013 10:21, Kohei KaiGai kai...@kaigai.gr.jp wrote: Hi Simon, I checked this patch. One thing I could comment on is, do you think it is a good idea to have oid of exception function list on contain_volatile_functions_walker()? The walker function is static thus here is no

Re: [HACKERS] COPY and Volatile default expressions

2013-06-24 Thread Kohei KaiGai
Hi Simon, I checked this patch. One thing I could comment on is, do you think it is a good idea to have oid of exception function list on contain_volatile_functions_walker()? The walker function is static thus here is no impact for other caller, and its context argument is unused. My proposition

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Simon Riggs
On 15 April 2013 21:53, Simon Riggs si...@2ndquadrant.com wrote: So I'll treat this as two separate cases: * add special case for sequences Patch attached. * use the NO SQL mechanism, as described, which implies no reads or writes of database state. We could test that, but its somewhat

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Heikki Linnakangas
On 16.04.2013 14:38, Simon Riggs wrote: On 15 April 2013 21:53, Simon Riggssi...@2ndquadrant.com wrote: So I'll treat this as two separate cases: * add special case for sequences Patch attached. + if (IsA(node, FuncExpr)) + { + FuncExpr *expr = (FuncExpr *)

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Simon Riggs
On 16 April 2013 13:57, Heikki Linnakangas hlinnakan...@vmware.com wrote: You still need to check the args, if the function is nextval, otherwise you incorrectly perform the optimization for something like nextval(myvolatilefunc()). Guess so. At least its an easy change. Thanks for checking.

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Bruce Momjian
On Tue, Apr 16, 2013 at 02:37:33PM +0100, Simon Riggs wrote: On 16 April 2013 13:57, Heikki Linnakangas hlinnakan...@vmware.com wrote: You still need to check the args, if the function is nextval, otherwise you incorrectly perform the optimization for something like

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: I found Simon's nextval()/COPY timings without this patch sobering. I assume he can apply this for 9.3, right? I believe it is a fix for a new 9.3 feature. It is not a fix, it is not for a 9.3 feature (the multi-insert thing went in in 9.2), and

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Bruce Momjian
On Tue, Apr 16, 2013 at 10:07:07AM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I found Simon's nextval()/COPY timings without this patch sobering. I assume he can apply this for 9.3, right? I believe it is a fix for a new 9.3 feature. It is not a fix, it is not for a

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Simon Riggs
On 16 April 2013 15:07, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: I found Simon's nextval()/COPY timings without this patch sobering. I assume he can apply this for 9.3, right? I believe it is a fix for a new 9.3 feature. It is not a fix, it is not for a 9.3

[HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile functions that wouldn't cause problems: any volatile function that

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 03:00:34PM +0100, Simon Riggs wrote: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Heikki Linnakangas
On 15.04.2013 17:00, Simon Riggs wrote: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile functions that wouldn't

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: I claim this is a common class, since sequence next_val functions and uuid generators meet that criteria and most common forms of auditing trigger, as well as any other form of data-reformatting trigger. Since this is a common case,

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile functions that

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:41, Heikki Linnakangas hlinnakan...@vmware.com wrote: What I'd like to do is to invent a new form of labelling that allows us to understand that COPY can still be optimised. It would be even nicer to detect at runtime, when a default expression or before trigger tries to

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: Do you have numbers on this, or ways to gather same? In other words, how do we know what resources (time, CPU cycles, disk seeks, etc.) are being consumed here? The multi-insert

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: Do you have numbers on this, or ways to gather same? In other words, how do we know what resources (time, CPU cycles, disk seeks,

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 11:49:42AM -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 05:04:16PM +0100, Simon Riggs wrote: On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: Do you have numbers on this, or ways to gather same? In other

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 17:08, David Fetter da...@fetter.org wrote: Loading data into a table with a SERIAL or UUID column is the main use case, so I'll measure that. The former is common enough a use case to optimize specifically, should the numbers come out right. Do you suppose that an in-core

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Hannu Krosing
On 04/15/2013 06:04 PM, Simon Riggs wrote: On 15 April 2013 16:55, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 15 April 2013 16:24, David Fetter da...@fetter.org wrote: Do you have numbers on this, or ways to gather same? In other words, how do we know

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
David Fetter da...@fetter.org writes: On Mon, Apr 15, 2013 at 05:04:16PM +0100, Simon Riggs wrote: Loading data into a table with a SERIAL or UUID column is the main use case, so I'll measure that. The former is common enough a use case to optimize specifically, should the numbers come out

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 17:04, Simon Riggs si...@2ndquadrant.com wrote: I will implement as a kluge, test and report the results. Test is COPY 1 million rows on a table with 2 columns, both bigint. Verified no checkpoints triggered during load. No other work active on database, tests condicted on

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 06:30:55PM +0100, Simon Riggs wrote: On 15 April 2013 17:04, Simon Riggs si...@2ndquadrant.com wrote: I will implement as a kluge, test and report the results. Test is COPY 1 million rows on a table with 2 columns, both bigint. Verified no checkpoints triggered

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 18:41, David Fetter da...@fetter.org wrote: The difference between HEAD and patch in the COPY, with sequence case is pretty remarkable. What's the patch? Attached. This is usable only for this test. It is not anywhere remotely close to being applied. -- Simon Riggs

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 07:04:55PM +0100, Simon Riggs wrote: On 15 April 2013 18:41, David Fetter da...@fetter.org wrote: The difference between HEAD and patch in the COPY, with sequence case is pretty remarkable. What's the patch? Attached. Thanks! :) This is usable only for this

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Robert Haas
On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: I claim this is a common class, since sequence next_val functions and uuid generators meet that criteria and most common forms of auditing trigger, as well as any other form of data-reformatting trigger. I don't believe that

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: I don't believe that it's a good idea to consider nextval() to be reorderable, so I'm not convinced by your argument here. Why not? I admit that I can't convince myself that it's

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Jaime Casanova
On Mon, Apr 15, 2013 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: OTOH, the notion that a UUID generator doesn't touch *any* database state seems like it might be worth treating as a general function property: it's simple to understand and applies to a lot of other volatile functions such

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 20:52, Robert Haas robertmh...@gmail.com wrote: On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: I claim this is a common class, since sequence next_val functions and uuid generators meet that criteria and most common forms of auditing trigger, as well as

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Robert Haas
On Mon, Apr 15, 2013 at 4:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think plenty of people would be upset if row serial numbers assigned with nextval() were not assigned in the order of the incoming rows. The argument that you can get gaps in the sequence in some corner cases (none of which

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 21:32, Jaime Casanova ja...@2ndquadrant.com wrote: On Mon, Apr 15, 2013 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: OTOH, the notion that a UUID generator doesn't touch *any* database state seems like it might be worth treating as a general function property: it's simple