Re: [HACKERS] Where do we stand on 9.3 bugs?

2014-01-13 Thread Tom Lane
Marti Raudsepp writes: > On Mon, Jan 13, 2014 at 5:16 PM, Tom Lane wrote: >> What remaining issues are there blocking a 9.3.3 release? > Well hardly a blocker since this has missed 2 releases already, but > I'm still hopeful to get many PGXS-based extensions to build again > without the dreaded

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Florian Pflug
On Jan14, 2014, at 00:33 , Craig Ringer wrote: > So I guess the question is: Is it worth all that hassle to remove a > misfeature you have to go out of your way to use? Is support for non-1 > lower bounds stopping us from doing something useful and important? Or > is it just an irritation that it

[HACKERS] Comment typo in src/include/access/gin_private.h

2014-01-13 Thread Etsuro Fujita
I ran into a typo in src/include/access/gin_private.h. Patch attached. Thanks, Best regards, Etsuro Fujita gin_private.h-typo.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/m

[HACKERS] Capturing EXPLAIN ANALYZE for a query without discarding the normal result set

2014-01-13 Thread Dave Cole
I apologise for dropping this out of nowhere. I had an idea about EXPLAIN ANALYZE that would be very useful for the system we are developing and supporting. It would be really cool if you could direct the EXPLAIN ANALYZE output to a temporary table so that the query being analyzed could execute n

Re: [HACKERS] Capturing EXPLAIN ANALYZE for a query without discarding the normal result set

2014-01-13 Thread Marti Raudsepp
On Tue, Jan 14, 2014 at 5:06 AM, Dave Cole wrote: > It would be really cool if you could direct the EXPLAIN ANALYZE output to a > temporary table so that the query being analyzed could execute normally. You can use the auto_explain contrib module to log the query plans of slow(er) queries: http:/

Re: [HACKERS] Capturing EXPLAIN ANALYZE for a query without discarding the normal result set

2014-01-13 Thread Tom Lane
Dave Cole writes: > It would be really cool if you could direct the EXPLAIN ANALYZE output to a > temporary table so that the query being analyzed could execute normally. What happens if the current transaction rolls back? If you want noninvasive explain data, contrib/auto_explain offers a solut

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Florian Pflug
On Jan14, 2014, at 02:10 , Kevin Grittner wrote: > The fact that some > day some new programmer might not be aware of all business rules, > or might choose to try to ignore them is the reason you add > constraints to columns and domains. Well, for columns and domains that seems easy. We could hav

[HACKERS] Soften pg_[start|stop]_backup to allow them on a standby?

2014-01-13 Thread Michael Paquier
Hi all, This is perhaps something that has already been discussed on hackers, I just could not find anything in the archives. Currently, pg_start_backup and pg_stop_backup cannot run on a standby because it is not possible to write a backup_label file to disk, because of the nature of a standby se

[HACKERS] Case sensitive mode in windows build option

2014-01-13 Thread Dilip kumar
As per current behavior if user want to build in debug mode in windows, then he need to give debug in capital letters (DEBUG), I think many user will always make mistake in giving this option, in my opinion we can make it case insensitive. I have attached a small patch for the same ( just conve

Re: [HACKERS] Case sensitive mode in windows build option

2014-01-13 Thread Craig Ringer
On 01/14/2014 11:49 AM, Dilip kumar wrote: > > > As per current behavior if user want to build in debug mode in windows, > then he need to give debug in capital letters (DEBUG), > > I think many user will always make mistake in giving this option, in my > opinion we can make it case insensitive

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-01-13 Thread Amit Kapila
On Tue, Jan 14, 2014 at 2:16 AM, Robert Haas wrote: > On Sat, Jan 11, 2014 at 1:08 AM, Amit Kapila wrote: >> Yes, currently this applies to update, what I have in mind is that >> in future if some one wants to use WAL compression for any other >> operation like 'full_page_writes', then it can be

Re: [HACKERS] [PATCH] Filter error log statements by sqlstate

2014-01-13 Thread Jeevan Chalke
On Mon, Jan 13, 2014 at 4:30 PM, Oskari Saarenmaa wrote: > Hi, > > > On 13/01/14 10:26, Jeevan Chalke wrote: > >> 1. Documentation is missing and thus becomes difficult to understand what >> exactly you are trying to do. Or in other words, user will be uncertain >> about using it more efficiently

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-13 Thread David Rowley
On Tue, Jan 14, 2014 at 2:00 PM, Florian Pflug wrote: > On Jan10, 2014, at 22:27 , David Rowley wrote: > > As the patch stands at the moment, I currently have a regression test > > which currently fails due to these extra zeros after the decimal point: > > > > -- This test currently fails due ex

Re: [HACKERS] [BUGS] surprising to_timestamp behavior

2014-01-13 Thread Jeevan Chalke
On Thu, Oct 31, 2013 at 10:50 AM, Jeevan Chalke < jeevan.cha...@enterprisedb.com> wrote: > > > > On Tue, Oct 29, 2013 at 11:05 PM, Robert Haas wrote: > >> On Tue, Oct 29, 2013 at 12:03 PM, Tom Lane wrote: >> > Robert Haas writes: >> >> It turns out that when you use the to_timestamp function, a

Re: [HACKERS] Proposal: variant of regclass

2014-01-13 Thread Michael Paquier
Hi, On Tue, Jan 14, 2014 at 4:28 PM, Yugo Nagata wrote: > Here is the patch to implement to_regclass, to_regproc, to_regoper, > and to_regtype. They are new functions similar to regclass, regproc, > regoper, and regtype except that if requested object is not found, > returns InvalidOid, rather th

Re: [HACKERS] [PATCH] Doc fix for VACUUM FREEZE

2014-01-13 Thread Amit Kapila
On Fri, Jan 3, 2014 at 9:02 PM, Peter Eisentraut wrote: > On 12/17/13, 8:16 PM, Maciek Sakrejda wrote: >> (now with patch--sorry about that) > > This patch doesn't apply. There are some recent changes around same place which broke this patch. Please find the modified patch attached with this mail

Re: [HACKERS] WIP patch (v2) for updatable security barrier views

2014-01-13 Thread Craig Ringer
On 01/09/2014 11:19 PM, Tom Lane wrote: > Dean Rasheed writes: >> My first thought was that it should just preprocess any security >> barrier quals in subquery_planner() in the same way as other quals are >> preprocessed. But thinking about it further, those quals are destined >> to become the qua

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Heikki Linnakangas
On 01/11/2014 12:40 AM, Peter Geoghegan wrote: My problem is that in general I'm not sold on the actual utility of making this kind of row locking work with exclusion constraints. I'm sincerely having a hard time thinking of a practical use-case (although, as I've said, I want to make it work wit

Re: [HACKERS] [PATCH] Filter error log statements by sqlstate

2014-01-13 Thread Jeevan Chalke
Hi Oskari, I had a quick look over the patch (Not compiled though). Here are few comments on the changes: 1. Documentation is missing and thus becomes difficult to understand what exactly you are trying to do. Or in other words, user will be uncertain about using it more efficiently. 2. Some more

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Heikki Linnakangas
On 01/11/2014 12:39 PM, Peter Geoghegan wrote: In any case, my patch is bound to win decisively for the other extreme, the insert-only case, because the overhead of doing an index scan first is always wasted there with your approach, and the overhead of extended btree leaf page locking has been s

Re: [HACKERS] ECPG regression tests generating warnings

2014-01-13 Thread Michael Meskes
On Sun, Jan 12, 2014 at 08:28:57AM -0800, Kevin Grittner wrote: > desc.pgc:55: WARNING: descriptor ""outdesc"" does not exist > desc.pgc:86: WARNING: descriptor ""outdesc"" does not exist Thanks, I didn't notice, fixed. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes do

Re: [HACKERS] WIP patch (v2) for updatable security barrier views

2014-01-13 Thread Dean Rasheed
On 12 January 2014 10:12, Craig Ringer wrote: > On 01/09/2014 06:48 PM, Dean Rasheed wrote: >> On 8 January 2014 10:19, Dean Rasheed wrote: >>> The assertion failure with inheritance and sublinks is a separate >>> issue --- adjust_appendrel_attrs() is not expecting to find any >>> unplanned subli

Re: [HACKERS] [PATCH] Filter error log statements by sqlstate

2014-01-13 Thread Oskari Saarenmaa
Hi, On 13/01/14 10:26, Jeevan Chalke wrote: 1. Documentation is missing and thus becomes difficult to understand what exactly you are trying to do. Or in other words, user will be uncertain about using it more efficiently. I figured I'd write documentation for this if it looks like a useful f

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-13 Thread Heikki Linnakangas
On 01/09/2014 10:55 PM, Josh Berkus wrote: On 01/09/2014 12:05 PM, Heikki Linnakangas wrote: Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the original server is

Re: [HACKERS] generic pseudotype IO functions?

2014-01-13 Thread Peter Eisentraut
On Mon, 2014-01-06 at 17:36 +0100, Andres Freund wrote: > FWIW, I am perfectly fine with duplicating the functions for now - I > just thought that that might not be the best way but I didn't (and > still > don't) have a strong opinion. Could we just put 0 in for the functions' OID and have code el

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Rajeev rastogi
> On Sun, Jan 12, Amit Kapila wrote: > >> How would that work? Would it be a tool in contrib? There already > >> is a timeout, so if a tool checked more frequently than the timeout, > >> it should work. The durable notification of the admin would happen > >> in the tool, right? > > > > Well, y

Re: [HACKERS] ISN extension bug? (with patch)

2014-01-13 Thread Heikki Linnakangas
On 01/03/2014 07:53 PM, Fabien COELHO wrote: If so, there is only the one-liner patch to consider. This patch doesn't apply anymore. Please submit an updated patch for the commit fest. In src/include/utils/elog.h there is an include for "utils/errcodes.h" which is generated somehow when co

Re: [HACKERS] WIP patch (v2) for updatable security barrier views

2014-01-13 Thread Craig Ringer
On 01/09/2014 11:19 PM, Tom Lane wrote: > Dean Rasheed writes: >> My first thought was that it should just preprocess any security >> barrier quals in subquery_planner() in the same way as other quals are >> preprocessed. But thinking about it further, those quals are destined >> to become the qua

Re: [HACKERS] nested hstore patch

2014-01-13 Thread Oleg Bartunov
Thank you, Erik ! Oleg On Mon, Jan 13, 2014 at 12:25 PM, Erik Rijkers wrote: > On Mon, January 13, 2014 00:24, Erik Rijkers wrote: >> On Sat, January 11, 2014 22:47, Andrew Dunstan wrote: >>> >>> On 01/11/2014 03:03 PM, Erik Rijkers wrote: On Sat, January 11, 2014 20:30, Peter Eisentraut wr

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Florian Pflug
On Jan12, 2014, at 04:18 , Josh Berkus wrote: > Thing is, when we talk about auto-degrade, we need to determine things > like "Is the replica down or is this just a network blip"? and take > action according to the user's desired configuration. This is not > something, realistically, that we can

[HACKERS] Where do we stand on 9.3 bugs?

2014-01-13 Thread Tom Lane
What remaining issues are there blocking a 9.3.3 release? I know that there were unresolved multixact issues when we put out 9.3.2 --- are those all dealt with now? What else do people see as release-blockers? regards, tom lane -- Sent via pgsql-hackers mailing list (p

Re: [HACKERS] generic pseudotype IO functions?

2014-01-13 Thread Tom Lane
Peter Eisentraut writes: > On Mon, 2014-01-06 at 17:36 +0100, Andres Freund wrote: >> FWIW, I am perfectly fine with duplicating the functions for now - I >> just thought that that might not be the best way but I didn't (and >> still don't) have a strong opinion. > Could we just put 0 in for the

Re: [HACKERS] Where do we stand on 9.3 bugs?

2014-01-13 Thread Alvaro Herrera
Tom Lane wrote: > What remaining issues are there blocking a 9.3.3 release? I know that > there were unresolved multixact issues when we put out 9.3.2 --- are > those all dealt with now? What else do people see as release-blockers? The only thing I'm aware still outstanding in multixact land is

Re: [HACKERS] Where do we stand on 9.3 bugs?

2014-01-13 Thread Andres Freund
On 2014-01-13 12:26:45 -0300, Alvaro Herrera wrote: > Tom Lane wrote: > > What remaining issues are there blocking a 9.3.3 release? I know that > > there were unresolved multixact issues when we put out 9.3.2 --- are > > those all dealt with now? What else do people see as release-blockers? > >

Re: [HACKERS] Where do we stand on 9.3 bugs?

2014-01-13 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-01-13 12:26:45 -0300, Alvaro Herrera wrote: > > Tom Lane wrote: > > > What remaining issues are there blocking a 9.3.3 release? I know that > > > there were unresolved multixact issues when we put out 9.3.2 --- are > > > those all dealt with now? What else do people

Re: [HACKERS] nested hstore patch

2014-01-13 Thread Alvaro Herrera
Andrew Dunstan wrote: > > On 01/13/2014 03:25 AM, Erik Rijkers wrote: > > >There are errors in the example expressions in "Table F-6. hstore Operators". > > > >Attached is a cumulative doc-patch (which includes the changes I sent > >earlier) which fixes these. > > > >I also attach an test perl p

Re: [HACKERS] nested hstore patch

2014-01-13 Thread Oleg Bartunov
Andrew, did you run perl script ? Actually, I found, that operator table needs to be fixed. Oleg On Mon, Jan 13, 2014 at 7:36 PM, Andrew Dunstan wrote: > > On 01/13/2014 03:25 AM, Erik Rijkers wrote: > >> There are errors in the example expressions in "Table F-6. hstore >> Operators". >> >> Att

Re: [HACKERS] Add CREATE support to event triggers

2014-01-13 Thread Alvaro Herrera
Alvaro Herrera escribió: > In an event trigger, the function pg_event_trigger_get_creation_commands() > returns the following JSON blob: After playing with this for a while, I realized something that must have seemed quite obvious to those paying attention: what this function is, is just a glorif

Re: [HACKERS] nested hstore patch

2014-01-13 Thread Andrew Dunstan
On 01/13/2014 11:03 AM, Alvaro Herrera wrote: Andrew Dunstan wrote: On 01/13/2014 03:25 AM, Erik Rijkers wrote: There are errors in the example expressions in "Table F-6. hstore Operators". Attached is a cumulative doc-patch (which includes the changes I sent earlier) which fixes these.

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Merlin Moncure
On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer wrote: > Implicit casts to text, anybody? This backward compatibility break orphaned the company I work for on 8.1 until last year and very nearly caused postgres to be summarily extirpated (only rescued at the last minute by my arrival). It cost hund

[HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Mel Gorman
Hi, I'm the chair for Linux Storage, Filesystem and Memory Management Summit 2014 (LSF/MM). A CFP was sent out last month (https://lwn.net/Articles/575681/) that you may have seen already. In recent years we have had at least one topic that was shared between all three tracks that was lead by a p

Re: [HACKERS] GIN improvements part 1: additional information

2014-01-13 Thread Alexander Korotkov
On Sat, Jan 11, 2014 at 6:15 AM, Tomas Vondra wrote: > On 8.1.2014 22:58, Alexander Korotkov wrote: > > Thanks for reporting. Fixed version is attached. > > I've tried to rerun the 'archie' benchmark with the current patch, and > once again I got > >PANIC: could not split GIN page, didn't fi

[HACKERS] KNN-GiST with recheck

2014-01-13 Thread Alexander Korotkov
Hackers! This patch was split from thread: http://www.postgresql.org/message-id/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=gon-...@mail.gmail.com I've split it to separate thead, because it's related to partial sort only conceptually not technically. Also I renamed it to "knn-gist-recheck" from

Re: [HACKERS] [PATCH] Add transforms feature

2014-01-13 Thread Robert Haas
On Fri, Jan 10, 2014 at 10:40 PM, Peter Eisentraut wrote: > On Wed, 2013-12-11 at 11:07 -0500, Tom Lane wrote: >> We should have learned by now that those are usually a bad idea. >> In this case, we've got changes in the behavior of function calling, >> which seems like not only a nightmare for de

Re: [HACKERS] nested hstore patch

2014-01-13 Thread Andrew Dunstan
On 01/13/2014 11:16 AM, Oleg Bartunov wrote: Andrew, did you run perl script ? Actually, I found, that operator table needs to be fixed. No. My build machine doesn't actually have DBD::Pg installed. Can you send me a patch if you don't want to push it yourself, or maybe Erik can send a

Re: [HACKERS] PoC: Partial sort

2014-01-13 Thread Alexander Korotkov
On Tue, Dec 31, 2013 at 5:41 AM, Andreas Karlsson wrote: > On 12/29/2013 08:24 AM, David Rowley wrote: > >> If it was possible to devise some way to reuse any >> previous tuplesortstate perhaps just inventing a reset method which >> clears out tuples, then we could see performance exceed the stan

Re: [HACKERS] Patch: show xid and xmin in pg_stat_activity and pg_stat_replication

2014-01-13 Thread Heikki Linnakangas
On 12/17/2013 04:58 PM, Christian Kruse wrote: attached you will find a patch for showing the current transaction id (xid) and the xmin of a backend in pg_stat_activty and the xmin in pg_stat_replication. Docs. When an admin is looking for a long-running transaction that's blocking vacuum, he

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Hannu Krosing
On 01/13/2014 04:12 PM, Florian Pflug wrote: > On Jan12, 2014, at 04:18 , Josh Berkus wrote: >> Thing is, when we talk about auto-degrade, we need to determine things >> like "Is the replica down or is this just a network blip"? and take >> action according to the user's desired configuration. Th

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Joshua D. Drake
On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the *easiest* place. Anything we do *inside* Postgres is going to have a really, really hard time dete

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Josh Berkus
Mel, > I'm the chair for Linux Storage, Filesystem and Memory Management Summit 2014 > (LSF/MM). A CFP was sent out last month (https://lwn.net/Articles/575681/) > that you may have seen already. > > In recent years we have had at least one topic that was shared between > all three tracks that wa

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Kevin Grittner
Josh Berkus wrote: >> Does the PostgreSQL community have a problem with recent >> kernels, particularly with respect to the storage, filesystem or >> memory management layers? > How about "don't add major IO behavior changes with no > backwards-compatibility switches"?  ;-) I notice, Josh, that

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Peter Geoghegan
On Mon, Jan 13, 2014 at 12:23 AM, Heikki Linnakangas wrote: > Exclusion constraints can be used to implement uniqueness checks with > SP-GiST or GiST indexes. For example, if you want to enforce that there are > no two tuples with the same x and y coordinates, ie. use a point as the key. > You cou

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Josh Berkus
On 01/13/2014 10:51 AM, Kevin Grittner wrote: >> How about "don't add major IO behavior changes with no >> backwards-compatibility switches"? ;-) > > I notice, Josh, that you didn't mention the problems many people > have run into with Transparent Huge Page defrag and with NUMA > access. Is that

Re: [HACKERS] Show lossy heap block info in EXPLAIN ANALYZE for bitmap heap scan

2014-01-13 Thread Robert Haas
On Thu, Jan 9, 2014 at 10:57 PM, Etsuro Fujita wrote: > I wrote: >> Robert Haas wrote: >> > Hmm, fair point. But I'm still not convinced that we really need to >> > add extra accounting for this. What's wrong with just reporting the >> > number of exact and lossy pages? > >> No. I intended to s

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2014-01-13 Thread Robert Haas
On Thu, Jan 9, 2014 at 11:06 PM, Amit Kapila wrote: > On Thu, Jan 9, 2014 at 12:26 AM, Robert Haas wrote: >> On Mon, Jan 6, 2014 at 11:37 PM, Amit Kapila wrote: >>> On Tue, Jan 7, 2014 at 12:52 AM, Robert Haas wrote: On Mon, Jan 6, 2014 at 9:48 AM, Amit Kapila wrote: >> Couldn't

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-13 Thread Robert Haas
On Thu, Jan 9, 2014 at 11:45 PM, Tom Lane wrote: > Greg Stark writes: >> On Thu, Jan 9, 2014 at 9:14 PM, Tom Lane wrote: >>> In short then, I think we should just add this to EXPLAIN and be done. >>> -1 for sticking the info into PlannedStmt or anything like that. > >> I'm confused. I thought I

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-13 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Currently the patch includes changes to prepare.c which is what seems > odd to me. I think it'd be fine to say, hey, I can't give you the > planning time in this EXPLAIN ANALYZE because I just used a cached > plan and did not re-plan. But saying, hey

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Robert Haas
On Mon, Jan 13, 2014 at 1:51 PM, Kevin Grittner wrote: > I notice, Josh, that you didn't mention the problems many people > have run into with Transparent Huge Page defrag and with NUMA > access. Amen to that. Actually, I think NUMA can be (mostly?) fixed by setting zone_reclaim_mode; is there s

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Robert Haas
On Mon, Jan 13, 2014 at 1:53 PM, Peter Geoghegan wrote: > On Mon, Jan 13, 2014 at 12:23 AM, Heikki Linnakangas > wrote: >> Exclusion constraints can be used to implement uniqueness checks with >> SP-GiST or GiST indexes. For example, if you want to enforce that there are >> no two tuples with the

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Claudio Freire
On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas wrote: > On a related note, there's also the problem of double-buffering. When > we read a page into shared_buffers, we leave a copy behind in the OS > buffers, and similarly on write-out. It's very unclear what to do > about this, since the kernel an

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-13 Thread Andreas Karlsson
On 01/13/2014 09:06 PM, Stephen Frost wrote: My thought, at least, was to always grab the planning time and then provide it for explain and/or explain analyze, and then for re-plan cases, indicate if a cached plan was returned, if a replan happened, and if a replan happened, what the old plan tim

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-13 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 9, 2014 at 11:45 PM, Tom Lane wrote: >> Uh, no, wasn't my suggestion. Doesn't that design imply measuring *every* >> planning cycle, explain or no? I was thinking more of just putting the >> timing calls into explain.c. > Currently the patch includes changes t

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 2:19 PM, Claudio Freire wrote: On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas wrote: On a related note, there's also the problem of double-buffering. When we read a page into shared_buffers, we leave a copy behind in the OS buffers, and similarly on write-out. It's very unclear what

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-13 Thread Robert Haas
On Fri, Jan 10, 2014 at 2:07 PM, Tom Lane wrote: > Florian Pflug writes: >> On Jan10, 2014, at 19:08 , Tom Lane wrote: >>> Although, having said that ... maybe "build your own aggregate" would >>> be a reasonable suggestion for people who need this? I grant that >>> it's going to be a minority

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Claudio Freire
On Mon, Jan 13, 2014 at 5:23 PM, Jim Nasby wrote: > On 1/13/14, 2:19 PM, Claudio Freire wrote: >> >> On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas >> wrote: >>> >>> On a related note, there's also the problem of double-buffering. When >>> we read a page into shared_buffers, we leave a copy behind

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-13 Thread Robert Haas
On Mon, Jan 13, 2014 at 3:23 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jan 9, 2014 at 11:45 PM, Tom Lane wrote: >>> Uh, no, wasn't my suggestion. Doesn't that design imply measuring *every* >>> planning cycle, explain or no? I was thinking more of just putting the >>> timing calls i

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 2:27 PM, Claudio Freire wrote: On Mon, Jan 13, 2014 at 5:23 PM, Jim Nasby wrote: On 1/13/14, 2:19 PM, Claudio Freire wrote: On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas wrote: On a related note, there's also the problem of double-buffering. When we read a page into shared_buff

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Claudio Freire
On Mon, Jan 13, 2014 at 5:32 PM, Jim Nasby wrote: >> >> That's my point. In terms of kernel-postgres interaction, it's fairly >> simple. >> >> What's not so simple, is figuring out what policy to use. Remember, >> you cannot tell the kernel to put some page in its page cache without >> reading it

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-13 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 13, 2014 at 3:23 PM, Tom Lane wrote: >> Meh. Why? This would only come into play for EXPLAIN EXECUTE stmtname. >> I don't think users would be surprised to see a report of minimal planning >> time for that. In fact, it might be a good thing, as it would make i

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Andres Freund
On 2014-01-13 15:15:16 -0500, Robert Haas wrote: > On Mon, Jan 13, 2014 at 1:51 PM, Kevin Grittner wrote: > > I notice, Josh, that you didn't mention the problems many people > > have run into with Transparent Huge Page defrag and with NUMA > > access. > > Amen to that. Actually, I think NUMA ca

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-01-13 Thread Robert Haas
On Sat, Jan 11, 2014 at 1:08 AM, Amit Kapila wrote: > Yes, currently this applies to update, what I have in mind is that > in future if some one wants to use WAL compression for any other > operation like 'full_page_writes', then it can be easily extendible. > > To be honest, I have not evaluated

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-13 Thread Robert Haas
On Mon, Jan 13, 2014 at 3:40 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Jan 13, 2014 at 3:23 PM, Tom Lane wrote: >>> Meh. Why? This would only come into play for EXPLAIN EXECUTE stmtname. >>> I don't think users would be surprised to see a report of minimal planning >>> time for that

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Peter Geoghegan
On Mon, Jan 13, 2014 at 12:17 PM, Robert Haas wrote: > For what it's worth, I agree with Heikki. There's probably nothing > sensible an upsert can do if it conflicts with more than one tuple, > but if it conflicts with just exactly one, it oughta be OK. If there is exactly one, *and* the existin

Re: [HACKERS] PoC: Partial sort

2014-01-13 Thread Marti Raudsepp
Hi Alexander, First, thanks a lot for working on this feature. This PostgreSQL shortcoming crops up in all the time in web applications that implement paging by multiple sorted columns. I've been trying it out in a few situations. I implemented a new enable_partialsort GUC to make it easier to tu

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jim Nasby
On 1/13/14, 2:37 PM, Claudio Freire wrote: On Mon, Jan 13, 2014 at 5:32 PM, Jim Nasby wrote: That's my point. In terms of kernel-postgres interaction, it's fairly simple. What's not so simple, is figuring out what policy to use. Remember, you cannot tell the kernel to put some page in its pag

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Heikki Linnakangas
On 01/13/2014 10:53 PM, Peter Geoghegan wrote: On Mon, Jan 13, 2014 at 12:17 PM, Robert Haas wrote: For what it's worth, I agree with Heikki. There's probably nothing sensible an upsert can do if it conflicts with more than one tuple, but if it conflicts with just exactly one, it oughta be OK.

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Andres Freund
On 2014-01-13 15:53:36 -0500, Trond Myklebust wrote: > > I've wondered before if there wouldn't be a chance for postgres to say > > "my dear OS, that the file range 0-8192 of file x contains y, no need to > > reread" and do that when we evict a page from s_b but I never dared to > > actually propos

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Jeff Janes
On Mon, Jan 13, 2014 at 12:32 PM, Jim Nasby wrote: > On 1/13/14, 2:27 PM, Claudio Freire wrote: > >> On Mon, Jan 13, 2014 at 5:23 PM, Jim Nasby wrote: >> >>> On 1/13/14, 2:19 PM, Claudio Freire wrote: >>> On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas wrote: > > On a r

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Robert Haas
On Mon, Jan 13, 2014 at 3:53 PM, Trond Myklebust wrote: > O_DIRECT was specifically designed to solve the problem of double buffering > between applications and the kernel. Why are you not able to use that in > these situations? O_DIRECT was apparently designed by a deranged monkey on some seri

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread James Bottomley
On Mon, 2014-01-13 at 14:32 -0600, Jim Nasby wrote: > On 1/13/14, 2:27 PM, Claudio Freire wrote: > > On Mon, Jan 13, 2014 at 5:23 PM, Jim Nasby wrote: > >> On 1/13/14, 2:19 PM, Claudio Freire wrote: > >>> > >>> On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas > >>> wrote: > > On a related n

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Trond Myklebust
On Jan 13, 2014, at 15:40, Andres Freund wrote: > On 2014-01-13 15:15:16 -0500, Robert Haas wrote: >> On Mon, Jan 13, 2014 at 1:51 PM, Kevin Grittner wrote: >>> I notice, Josh, that you didn't mention the problems many people >>> have run into with Transparent Huge Page defrag and with NUMA >>>

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Kevin Grittner
Josh Berkus wrote: > Wanna go to Collab? I don't think that works out for me, but thanks for suggesting it. I'd be happy to brainstorm with anyone who does go about issues to discuss; although the ones I keep running into have already been mentioned. Regarding the problems others have mentione

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Andres Freund
On 2014-01-13 12:34:35 -0800, James Bottomley wrote: > On Mon, 2014-01-13 at 14:32 -0600, Jim Nasby wrote: > > Well, if we were to collaborate with the kernel community on this then > > presumably we can do better than that for eviction... even to the > > extent of "here's some data from this range

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Jim Nasby
On 1/13/14, 12:21 PM, Joshua D. Drake wrote: On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the *easiest* place. Anything we do *inside* Postgres is

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Andres Freund
On 2014-01-13 15:14:21 -0600, Jim Nasby wrote: > On 1/13/14, 12:21 PM, Joshua D. Drake wrote: > > > >On 01/13/2014 10:12 AM, Hannu Krosing wrote: > In other words, if we're going to have auto-degrade, the most > intelligent place for it is in > RepMgr/HandyRep/OmniPITR/pgPoolII/whatever

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Jim Nasby
On 1/13/14, 10:40 AM, Merlin Moncure wrote: On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer wrote: >Implicit casts to text, anybody? This backward compatibility break orphaned the company I work for on 8.1 until last year and very nearly caused postgres to be summarily extirpated (only rescued a

Re: [HACKERS] [Lsf-pc] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Greg Stark
On Mon, Jan 13, 2014 at 9:12 PM, Andres Freund wrote: > For one, postgres doesn't use mmap for files (and can't without major > new interfaces). Frequently mmap()/madvise()/munmap()ing 8kb chunks has > horrible consequences for performance/scalability - very quickly you > contend on locks in the k

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Joshua D. Drake
On 01/13/2014 01:14 PM, Jim Nasby wrote: On 1/13/14, 12:21 PM, Joshua D. Drake wrote: On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the *easiest*

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread David Fetter
On Mon, Jan 13, 2014 at 10:40:57AM -0600, Merlin Moncure wrote: > On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer wrote: > > Implicit casts to text, anybody? > > This backward compatibility break orphaned the company I work for on > 8.1 until last year and very nearly caused postgres to be summaril

Re: [HACKERS] plpgsql.consistent_into

2014-01-13 Thread Jim Nasby
On 1/13/14, 1:44 AM, Pavel Stehule wrote: 2014/1/12 Florian Pflug mailto:f...@phlo.org>> On Jan12, 2014, at 22:37 , Pavel Stehule mailto:pavel.steh...@gmail.com>> wrote: > There is GUC for variable_conflict already too. In this case I would to > enable this functionality everyw

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Robert Haas
On Mon, Jan 13, 2014 at 4:45 PM, David Fetter wrote: > On Mon, Jan 13, 2014 at 10:40:57AM -0600, Merlin Moncure wrote: >> On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer wrote: >> > Implicit casts to text, anybody? >> >> This backward compatibility break orphaned the company I work for on >> 8.1 un

Re: [HACKERS] [Lsf-pc] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Josh Berkus
Everyone, I am looking for one or more hackers to go to Collab with me to discuss this. If you think that might be you, please let me know and I'll look for funding for your travel. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Syntax of INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Jim Nasby
On 1/12/14, 9:35 PM, Andreas Karlsson wrote: On 01/12/2014 11:20 PM, Peter Geoghegan wrote: On Sun, Jan 12, 2014 at 8:12 AM, Andreas Karlsson wrote: On 01/11/2014 11:42 PM, Peter Geoghegan wrote: I recently suggested that rather than RETURNING REJECTS, we could have a REJECTING clause, which

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-13 Thread Merlin Moncure
On Mon, Jan 13, 2014 at 3:45 PM, David Fetter wrote: > On Mon, Jan 13, 2014 at 10:40:57AM -0600, Merlin Moncure wrote: >> This project has no deprecation policy, > > I believe it actually does, although it's not a formal, written > policy. Would you like to help draft one up? Lack of 'formal, wr

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Mel Gorman
On Mon, Jan 13, 2014 at 03:15:16PM -0500, Robert Haas wrote: > On Mon, Jan 13, 2014 at 1:51 PM, Kevin Grittner wrote: > > I notice, Josh, that you didn't mention the problems many people > > have run into with Transparent Huge Page defrag and with NUMA > > access. > Ok, there are at least three

Re: [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Mel Gorman
On Mon, Jan 13, 2014 at 06:27:03PM -0200, Claudio Freire wrote: > On Mon, Jan 13, 2014 at 5:23 PM, Jim Nasby wrote: > > On 1/13/14, 2:19 PM, Claudio Freire wrote: > >> > >> On Mon, Jan 13, 2014 at 5:15 PM, Robert Haas > >> wrote: > >>> > >>> On a related note, there's also the problem of double-b

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Peter Geoghegan
On Mon, Jan 13, 2014 at 12:58 PM, Heikki Linnakangas wrote: > Well, even if you don't agree that locking all the conflicting rows for > update is sensible, it's still perfectly sensible to return the rejected > rows to the user. For example, you're inserting N rows, and if some of them > violate a

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Andres Freund
On 2014-01-13 14:19:56 -0800, James Bottomley wrote: > > Frequently mmap()/madvise()/munmap()ing 8kb chunks has > > horrible consequences for performance/scalability - very quickly you > > contend on locks in the kernel. > > Is this because of problems in the mmap_sem? It's been a while since I

Re: [Lsf-pc] [HACKERS] Linux kernel impact on PostgreSQL performance

2014-01-13 Thread Mel Gorman
On Mon, Jan 13, 2014 at 11:38:44PM +0100, Jan Kara wrote: > On Mon 13-01-14 22:26:45, Mel Gorman wrote: > > The flipside is also meant to hold true. If you know data will be needed > > in the near future then posix_fadvise(POSIX_FADV_WILLNEED). Glancing at > > the implementation it does a forced re

Re: [HACKERS] Syntax of INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-13 Thread Peter Geoghegan
On Mon, Jan 13, 2014 at 2:20 PM, Jim Nasby wrote: > Well, a common case for INSERT RETURNING is to get your set of surrogate > keys back; so I think users would want the ability to RETURN what finally > made it into the table. Your update can also have a RETURNING clause. I'm not necessarily that

  1   2   >