Re: [HACKERS] newline conversion in SQL command strings

2012-09-19 Thread Heikki Linnakangas
On 20.09.2012 05:56, Peter Eisentraut wrote: I have received a number of bug reports about plsh choking on Windows-style line endings. The problem is that the user uses some Windows-based tool or other to execute an SQL command line this: CREATE FUNCTION foo() RETURNS something LANGUAGE plsh AS

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-19 Thread Amit Kapila
On Thursday, September 20, 2012 1:44 AM Simon Riggs wrote: On 12 September 2012 04:30, Amit Kapila wrote: > On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: > Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: > We have some use cases for this patch, whe

Re: [HACKERS] 64-bit API for large object

2012-09-19 Thread Kohei KaiGai
I checked this patch. It can be applied onto the latest master branch without any problems. My comments are below. 2012/9/11 Tatsuo Ishii : > Ok, here is the patch to implement 64-bit API for large object, to > allow to use up to 4TB large objects(or 16TB if BLCKSZ changed to > 32KB). The patch is

[HACKERS] newline conversion in SQL command strings

2012-09-19 Thread Peter Eisentraut
I have received a number of bug reports about plsh choking on Windows-style line endings. The problem is that the user uses some Windows-based tool or other to execute an SQL command line this: CREATE FUNCTION foo() RETURNS something LANGUAGE plsh AS $$ #!/bin/sh do something do something $$; w

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Florian Schoppmann
Tom Lane wrote: > florian.schoppm...@emc.com (Florian Schoppmann) writes: > > [VOLATILE function in WHERE clause *does* get optimized] > > I can't get excited about this. Any time you put a volatile function > into WHERE, you're playing with fire. The docs warn against it: > http://www.postgre

Re: [HACKERS] [COMMITTERS] pgsql: Fix bufmgr so CHECKPOINT_END_OF_RECOVERY behaves as a shutdown c

2012-09-19 Thread Andres Freund
On Tuesday, September 18, 2012 04:18:01 AM Robert Haas wrote: > >> Maybe what we should do is - if this is an end-of-recovery checkpoint > >> - *assert* that the BM_PERMANENT bit is set on every buffer we find. > >> That would provide a useful cross-check that we don't have a bug > >> similar to th

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread David Johnston
> -Original Message- > From: Kevin Grittner [mailto:kevin.gritt...@wicourts.gov] > Sent: Wednesday, September 19, 2012 5:51 PM > To: k...@rice.edu; David Johnston > Cc: 'Florian Schoppmann'; 'Robert Haas'; pgsql-hackers@postgresql.org; 'Tom > Lane' > Subject: RE: [HACKERS] Invalid optimizat

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Kevin Grittner
"David Johnston" wrote: > VOLATILE: "A Volatile function used in an ORDER BY or WHERE clause > without referencing any columns from the query itself (i.e., no > parameters or all constants) will be evaluated a single time and > the result treated as a constant (i.e., all rows will have > identic

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread David Johnston
> -Original Message- > >> | VOLATILE indicates that the function value can change even within a > >> | single table scan, so no optimizations can be made. > >> | Relatively few database functions are volatile in this sense; some > >> | examples are random(), [...] > > > What are the argu

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Kevin Grittner
"David Johnston" wrote: >> | VOLATILE indicates that the function value can change even >> | within a single table scan, so no optimizations can be made. >> | Relatively few database functions are volatile in this sense; >> | some examples are random(), [...] > What are the arguments against a

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread David Johnston
> -Original Message- > There really needs to be some way to specify that when an expression is > evaluated for each row in a set, a function used within that expression is not > optimized away for some rows. Fortunately we have a way: > > http://www.postgresql.org/docs/9.2/interactive/sql

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Kevin Grittner
"Kevin Grittner" wrote: > There is a workaround, if you don't mind ugly: Or, better: WITH source AS ( SELECT i, random() AS r FROM generate_series(1,10) AS i ) SELECT i FROM source, ( SELECT count(*) AS _n FROM source ) AS _stats WHERE r < 5::DOU

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Kevin Grittner
"k...@rice.edu" wrote: > On Wed, Sep 19, 2012 at 02:39:12PM -0500, Kevin Grittner wrote: >> In another thread, Tom Lane wrote: >>> 2. Apply the WHERE condition to each row from 1, and drop rows >>> that don't pass it. >> >> People expect that the results will be consistent with this >> model,

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-19 Thread Simon Riggs
On 12 September 2012 04:30, Amit Kapila wrote: > On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: > Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: > >>> We have some use cases for this patch, when can you post >>> a new version? I would test and review it

[HACKERS] CTE optimization fence on the todo list?

2012-09-19 Thread Daniel Browning
I would like to have the option of disabling the CTE optimization fence for certain CTEs and/or queries. Can that be added to the official todo list? If not, why not? I would find the option beneficial because large, complicated queries are often a lot clearer, simpler, and easier to read with

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread k...@rice.edu
On Wed, Sep 19, 2012 at 02:39:12PM -0500, Kevin Grittner wrote: > Tom Lane wrote: > > Robert Haas writes: > >> It still seems like awfully weird behavior. > > > > Why? The WHERE condition relates only to the output of the _stats > > subquery, so why shouldn't it be evaluated there, rather than

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Kevin Grittner
Tom Lane wrote: > Robert Haas writes: >> It still seems like awfully weird behavior. > > Why? The WHERE condition relates only to the output of the _stats > subquery, so why shouldn't it be evaluated there, rather than > after the join? In another thread, Tom Lane wrote: > It's easier to unde

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Robert Haas
On Wed, Sep 19, 2012 at 1:26 PM, Tom Lane wrote: > Robert Haas writes: >> It still seems like awfully weird behavior. > > Why? The WHERE condition relates only to the output of the _stats > subquery, so why shouldn't it be evaluated there, rather than after > the join? Because my mental model (

Re: [HACKERS] Removal of AcceptInvalidationMessages broke things

2012-09-19 Thread Robert Haas
On Wed, Sep 19, 2012 at 2:17 PM, Noah Misch wrote: > Sounds fine for now. I suspect the better change would be to make > AcceptInvalidationMessages() unconditional in LockRelationOid() and friends. > There's no reason to desire recent ACLs only when opening by name. I agree, on both counts. I t

Re: [HACKERS] Removal of AcceptInvalidationMessages broke things

2012-09-19 Thread Tom Lane
Noah Misch writes: > On Wed, Sep 19, 2012 at 01:17:17PM -0400, Tom Lane wrote: >> Since we have only a few hours before 9.2.1 is due to wrap, my >> inclination for a band-aid fix is to put back that code. There might be >> some more elegant answer, but we haven't got time to find it now. > Sound

Re: [HACKERS] Re: [COMMITTERS] pgsql: Properly set relpersistence for fake relcache entries.

2012-09-19 Thread Simon Riggs
On 19 September 2012 18:47, Tom Lane wrote: > BTW, what should our advice be for recovering from corruption due to > this bug? As far as the btree and GIN problems go, we can tell people > that REINDEX will fix it. And in 9.1, you don't really need to worry > about the visibility map being bad.

Re: [HACKERS] Removal of AcceptInvalidationMessages broke things

2012-09-19 Thread Noah Misch
On Wed, Sep 19, 2012 at 01:17:17PM -0400, Tom Lane wrote: > I looked into bug #7557, which demonstrates a case where a session fails > to notice a just-committed change in table permissions. > - /* > -* Check for shared-cache-inval messages before trying to open the > -* relation. This

Re: [HACKERS] Re: [COMMITTERS] pgsql: Properly set relpersistence for fake relcache entries.

2012-09-19 Thread Tom Lane
BTW, what should our advice be for recovering from corruption due to this bug? As far as the btree and GIN problems go, we can tell people that REINDEX will fix it. And in 9.1, you don't really need to worry about the visibility map being bad. But what do you do in 9.2, if you have a bad visibil

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Tom Lane
Robert Haas writes: > It still seems like awfully weird behavior. Why? The WHERE condition relates only to the output of the _stats subquery, so why shouldn't it be evaluated there, rather than after the join? In a green field I might agree that we should de-optimize such cases, but the problem

[HACKERS] Removal of AcceptInvalidationMessages broke things

2012-09-19 Thread Tom Lane
I looked into bug #7557, which demonstrates a case where a session fails to notice a just-committed change in table permissions. This is pretty obviously due to a failure to read the sinval message notifying other backends of the pg_class.relacl update. Some digging in the git history says it got

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Robert Haas
On Wed, Sep 19, 2012 at 12:34 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Sep 19, 2012 at 10:30 AM, Tom Lane wrote: >>> To do what you want, I'd suggest wrapping the join into a sub-select >>> with an "OFFSET 0" clause, which will serve as an optimization fence >>> that prevents the ran

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Tom Lane
Robert Haas writes: > On Wed, Sep 19, 2012 at 10:30 AM, Tom Lane wrote: >> To do what you want, I'd suggest wrapping the join into a sub-select >> with an "OFFSET 0" clause, which will serve as an optimization fence >> that prevents the random() call from being pushed down. > You've repeatedly o

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Robert Haas
On Wed, Sep 19, 2012 at 10:30 AM, Tom Lane wrote: > florian.schoppm...@emc.com (Florian Schoppmann) writes: >> In PostgreSQL 9.1 and 9.2 (possibly also in earlier versions), the query > >> --8<-- >> WITH source AS ( >> SELECT i FROM generate_series(1,10) AS i >> ) >> SELECT >> i >> FROM >>

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Merlin Moncure
On Wed, Sep 19, 2012 at 9:30 AM, Tom Lane wrote: > florian.schoppm...@emc.com (Florian Schoppmann) writes: >> In PostgreSQL 9.1 and 9.2 (possibly also in earlier versions), the query > >> --8<-- >> WITH source AS ( >> SELECT i FROM generate_series(1,10) AS i >> ) >> SELECT >> i >> FROM >>

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-19 Thread Tom Lane
florian.schoppm...@emc.com (Florian Schoppmann) writes: > In PostgreSQL 9.1 and 9.2 (possibly also in earlier versions), the query > --8<-- > WITH source AS ( > SELECT i FROM generate_series(1,10) AS i > ) > SELECT > i > FROM > source, ( > SELECT > count(*) AS _n >

Re: [HACKERS] ToDo: allow to get a number of processed rows by COPY statement

2012-09-19 Thread Heikki Linnakangas
On 16.08.2012 14:43, Pavel Stehule wrote: Hello here is updated patch The patch seems to be truncated, it ends with: *** a/src/test/regress/input/copy.source --- b/src/test/regress/input/copy.source *** *** 106,108 this is just a line full of junk that would error out if par

Re: [HACKERS] Reduce palloc's in numeric operations.

2012-09-19 Thread Heikki Linnakangas
On 14.09.2012 11:25, Kyotaro HORIGUCHI wrote: Hello, I will propose reduce palloc's in numeric operations. The numeric operations are slow by nature, but usually it is not a problem for on-disk operations. Altough the slowdown is enhanced on on-memory operations. I inspcted them and found some

Re: [HACKERS] proposal - assign result of query to psql variable

2012-09-19 Thread Shigeru HANADA
On Fri, Aug 10, 2012 at 3:21 AM, Pavel Stehule wrote: > there is new version of this patch > > * cleaned var list parser > * new regress tests > * support FETCH_COUNT > 0 Here are my review comments. Submission == The patch is formatted in context diff style, and it could be applied clea