Re: [HACKERS] pgbench more operators & functions

2016-07-08 Thread Fabien COELHO
Here is a simple patch which adds a bunch of operators (bitwise: & | ^ ~, comparisons: =/== <>/!= < <= > >=, logical: and/&& or/|| xor/^^ not/!) and functions (exp ln if) to pgbench. I've tried to be pg's SQL compatible where appropriate. Also attached is a simple test script. Here is a si

Re: [HACKERS] [BUGS] BUG #14230: Wrong timeline returned by pg_stop_backup on a standby

2016-07-08 Thread Amit Kapila
On Thu, Jul 7, 2016 at 12:08 PM, Michael Paquier wrote: > On Thu, Jul 7, 2016 at 12:57 AM, Marco Nenciarini > wrote: >> After further analysis, the issue is that we retrieve the starttli from >> the ControlFile structure, but it was using ThisTimeLineID when writing >> the backup label. >> >> I'v

Re: [HACKERS] dumping database privileges broken in 9.6

2016-07-08 Thread Stephen Frost
* Noah Misch (n...@leadboat.com) wrote: > This PostgreSQL 9.6 open item is past due for your status update. Kindly send > a status update within 24 hours, and include a date for your subsequent status > update. Refer to the policy on open item ownership: > http://www.postgresql.org/message-id/201

Re: [HACKERS] bug in citext's upgrade script for parallel aggregates

2016-07-08 Thread David Rowley
On 30 June 2016 at 03:49, Robert Haas wrote: > On Sat, Jun 25, 2016 at 3:44 AM, Andreas Karlsson wrote: >> On 06/24/2016 01:31 PM, David Rowley wrote: >>> Seems there's a small error in the upgrade script for citext for 1.1 >>> to 1.2 which will cause min(citext) not to be parallel enabled. >>> >

Re: [HACKERS] dumping database privileges broken in 9.6

2016-07-08 Thread Noah Misch
On Wed, Jul 06, 2016 at 07:03:33PM -0400, Stephen Frost wrote: > * Noah Misch (n...@leadboat.com) wrote: > > On Wed, Jun 29, 2016 at 11:50:17AM -0400, Stephen Frost wrote: > > > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > > > > Do this: > > > > > > > > CREATE DATABASE test1; > >

[HACKERS] Re: [BUGS] BUG #14230: Wrong timeline returned by pg_stop_backup on a standby

2016-07-08 Thread Noah Misch
On Thu, Jul 07, 2016 at 03:38:26PM +0900, Michael Paquier wrote: > On Thu, Jul 7, 2016 at 12:57 AM, Marco Nenciarini > wrote: > > After further analysis, the issue is that we retrieve the starttli from > > the ControlFile structure, but it was using ThisTimeLineID when writing > > the backup label

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Michael Paquier
On Sat, Jul 9, 2016 at 4:02 AM, Pavel Stehule wrote: > > > 2016-07-08 20:39 GMT+02:00 Tom Lane : >> >> Alvaro Herrera writes: >> > As a separate concern, IMO having the source code in a \df+ column is >> > almost completely useless. >> >> Good point. It works okay for C/internal functions, but i

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-08 Thread Peter Geoghegan
On Thu, Jul 7, 2016 at 12:34 AM, Noah Misch wrote: >> How do you feel about adding testing to tuplesort.c not limited to >> hitting this bug (when Valgrind memcheck is used)? > > Sounds great, but again, not in the patch fixing this bug. Attached patch adds a CLUSTER external sort test case to th

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Gavin Flower
Please see comment at the bottom of this post. On 08/07/16 10:48, Pete Stevenson wrote: Good info, thanks for the note. Agreed that it is difficult to pull things apart to isolate these features for offload — so actually running experiments with offload is not possible, as you point out (and f

Re: [HACKERS] can we optimize STACK_DEPTH_SLOP

2016-07-08 Thread Tom Lane
Greg Stark writes: > Fwiw here's the pmap info from burbot (Linux Sparc64): > 136 48 48 rw---[ stack ] > 136 48 48 rw---[ stack ] > 136 48 48 rw---[ stack ] > 136 48 48 rw---[ stack ] > 136 56 56 rw---[ stack ] > 136 80

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Alvaro Herrera
Peter Geoghegan wrote: > Has anyone ever done any kind of write-up of the "TED" design that was > discussed during FOSDEM (I hope I recall the name it was given > correctly)? Apparently that's something that's been discussed a few > times among senior community members, and I think it has promise.

Re: [HACKERS] can we optimize STACK_DEPTH_SLOP

2016-07-08 Thread Greg Stark
On Fri, Jul 8, 2016 at 3:32 PM, Tom Lane wrote: > Hm, after reading the man page I don't quite see how that would help? > You'd have to already know the mapped stack address range in order to > call the function without getting ENOMEM. I had assumed unmapped pages would just return a 0 in the bi

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Pavel Stehule
2016-07-08 20:39 GMT+02:00 Tom Lane : > Alvaro Herrera writes: > > As a separate concern, IMO having the source code in a \df+ column is > > almost completely useless. > > Good point. It works okay for C/internal functions, but in those cases > it's usually redundant with the proname. For PL fu

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Merlin Moncure
On Thu, Jul 7, 2016 at 11:45 AM, Pete Stevenson wrote: > Hi postgresql hackers - > > I would like to find some analysis (published work, blog posts) on the > overheads affiliated with the guarantees provided by MVCC isolation. More > specifically, assuming the current workload is CPU bound (as o

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-08 Thread Andres Freund
On 2016-07-08 13:32:35 -0500, Kevin Grittner wrote: > On Fri, Jul 8, 2016 at 12:53 PM, Andres Freund wrote: > > On 2016-07-08 11:00:50 -0500, Kevin Grittner wrote: > >> On Wed, Jul 6, 2016 at 4:55 PM, Andres Freund wrote: > >> > >> > So I don't think that approach still allows old snapshot relate

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Peter Geoghegan
On Fri, Jul 8, 2016 at 11:44 AM, Tom Lane wrote: >> Sure, but we could *also* do it separately, splitting VACUUMs tasks of >> tuple freezing, page compaction, and index entry removal each into >> separate tasks. > > Uh ... wouldn't that tend to make things worse? The knocks on VACUUM are > too mu

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Kevin Grittner
On Thu, Jul 7, 2016 at 11:45 AM, Pete Stevenson wrote: > I would like to find some analysis (published work, blog posts) > on the overheads affiliated with the guarantees provided by MVCC > isolation. There are three levels of isolation implemented[1]; the incremental cost of SERIALIZABLE isolat

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> VACUUM in itself is an offloading optimization; the whole point of it >> is to do maintenance in a background process not foreground queries. > Well, if VACUUM worked so great, we wouldn't get so many trouble reports > with it. There's substantial impr

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-07-08 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: > At Fri, 10 Jun 2016 17:39:59 +0900, Michael Paquier > wrote in > > > On Thu, Jun 9, 2016 at 9:55 PM, Kyotaro HORIGUCHI > > wrote: > > Indeed, and you could just do the following to reproduce the failure > > with the recovery test suite, so I would suggest adding th

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Tom Lane
Alvaro Herrera writes: > As a separate concern, IMO having the source code in a \df+ column is > almost completely useless. Good point. It works okay for C/internal functions, but in those cases it's usually redundant with the proname. For PL functions it's a disaster formatting-wise, because t

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Pete Stevenson wrote: > >> Maybe I could figure out the lines of code that add versions into a > >> table and then those that collect old versions (they do get collected, > >> right?). Anyway, thought being I could profile while running TPC-C or > >> si

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-08 Thread Kevin Grittner
On Fri, Jul 8, 2016 at 12:53 PM, Andres Freund wrote: > On 2016-07-08 11:00:50 -0500, Kevin Grittner wrote: >> On Wed, Jul 6, 2016 at 4:55 PM, Andres Freund wrote: >> >> > So I don't think that approach still allows old snapshot related >> > cleanups for toast triggered vacuums? Is that an accep

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Tom Lane
Alvaro Herrera writes: > Pete Stevenson wrote: >> Maybe I could figure out the lines of code that add versions into a >> table and then those that collect old versions (they do get collected, >> right?). Anyway, thought being I could profile while running TPC-C or >> similar. I was hoping that som

Re: [HACKERS] strange explain in upstream - subplan 1 twice - is it bug?

2016-07-08 Thread Tom Lane
Robert Haas writes: > On Wed, Jun 1, 2016 at 7:29 AM, Pavel Stehule wrote: >> When I tested some queries, I found strange plan >> postgres=# explain analyze select s.nazev, o.nazev, o.pocet_obyvatel from >> (select nazev, array(select id from obce_pocet_obyvatel where okresy.id = >> okres_id orde

[HACKERS] Fix Error Message for allocate_recordbuf() Failure

2016-07-08 Thread Shoaib Lari
Hello, Attached is a patch for xlogreader.c for a more informative error message for allocate_recordbuf() failure. The patch details are: - Project name.: None - Uniquely identifiable file name, so we can tell difference between your v1 and v24.: src/backend/access/transam/xlogread

[HACKERS] Logical decoding

2016-07-08 Thread Joshua Bay
Hi, I have a question about logical decoding of Postgres. where are the entry points to logical decoding? Specifically, we want to know whether logical decoding happens immediately after commit, or whether there is a polling thread that scans the Write Ahead Log and then dumps to the special t

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-08 Thread Andres Freund
On 2016-07-08 11:00:50 -0500, Kevin Grittner wrote: > On Wed, Jul 6, 2016 at 4:55 PM, Andres Freund wrote: > > > So I don't think that approach still allows old snapshot related > > cleanups for toast triggered vacuums? Is that an acceptable > > restriction? > > What I would rather see is that

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Alvaro Herrera
Pete Stevenson wrote: > Maybe I could figure out the lines of code that add versions into a > table and then those that collect old versions (they do get collected, > right?). Anyway, thought being I could profile while running TPC-C or > similar. I was hoping that someone might be able to jump on

Re: [HACKERS] minor plpgsql doc patch

2016-07-08 Thread Fabien COELHO
The very minor patch attached improves the PL/pgSQL documentation about trigger functions. It moves the description common to both data change & database event triggers out of the first section and into a common header. It adds a link at the beginning of the sections to their corresponding gener

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Alvaro Herrera
Tom Lane wrote: > Magnus Hagander writes: > > On Friday, July 8, 2016, Michael Paquier wrote: > >> Fujii-san has reminded me of the fact that we do not show in \df+ the > >> parallel status of a function. The output of \df+ is already very > >> large, so I guess that any people mentally sane alre

Re: [HACKERS] minor plpgsql doc patch

2016-07-08 Thread Tom Lane
Fabien COELHO writes: > The very minor patch attached improves the PL/pgSQL documentation about > trigger functions. It moves the description common to both data change & > database event triggers out of the first section and into a common header. > It adds a link at the beginning of the sectio

Re: [HACKERS] doc: Incorrect return type of IsForeignScanParallelSafe in fdwhandler.sgml

2016-07-08 Thread Tom Lane
Etsuro Fujita writes: > I noticed that the return type of IsForeignScanParallelSafe described in > fdwhandler.sgml isn't correct; that should be bool, not Size. Please > find attached a small patch for that. Pushed, thanks! regards, tom lane -- Sent via pgsql-hack

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-08 Thread Kevin Grittner
On Wed, Jul 6, 2016 at 4:55 PM, Andres Freund wrote: > So I don't think that approach still allows old snapshot related > cleanups for toast triggered vacuums? Is that an acceptable > restriction? What I would rather see is that if the heap is vacuumed (whether or not by autovacuum) then the re

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Amit Kapila writes: > > On Fri, Jul 8, 2016 at 5:27 PM, Michael Paquier > > wrote: > >> Okay. Here we go. I named the column for the parallel information > >> "Parallelism". > > > Another option could be to name it as Parallel Mode. > > I'd go with just

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Magnus Hagander writes: > > On Friday, July 8, 2016, Michael Paquier wrote: > >> Fujii-san has reminded me of the fact that we do not show in \df+ the > >> parallel status of a function. The output of \df+ is already very > >> large, so I guess that any pe

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Tom Lane
Amit Kapila writes: > On Fri, Jul 8, 2016 at 5:27 PM, Michael Paquier > wrote: >> Okay. Here we go. I named the column for the parallel information >> "Parallelism". > Another option could be to name it as Parallel Mode. I'd go with just "Parallel", to keep it from being noticeably wider than

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Tom Lane
Magnus Hagander writes: > On Friday, July 8, 2016, Michael Paquier wrote: >> Fujii-san has reminded me of the fact that we do not show in \df+ the >> parallel status of a function. The output of \df+ is already very >> large, so I guess that any people mentally sane already use it with >> the exp

Re: [HACKERS] can we optimize STACK_DEPTH_SLOP

2016-07-08 Thread Tom Lane
Greg Stark writes: > Searching for info on ia64 turned up this interesting thread: > https://www.postgresql.org/message-id/21563.1289064886%40sss.pgh.pa.us Yeah, that's the same one I referenced upthread ;-) > From that discussion it seems we should probably run these tests with > -O0 because th

Re: [HACKERS] can we optimize STACK_DEPTH_SLOP

2016-07-08 Thread Greg Stark
On Fri, Jul 8, 2016 at 4:46 AM, Tom Lane wrote: > Based on what I'm seeing so far, really 100K ought to be more than plenty > of slop for most architectures, but I'm afraid to go there for IA64. Searching for info on ia64 turned up this interesting thread: https://www.postgresql.org/message-id/2

Re: [HACKERS] Reviewing freeze map code

2016-07-08 Thread Amit Kapila
On Thu, Jul 7, 2016 at 12:34 PM, Masahiko Sawada wrote: > Than you for reviewing! > > On Thu, Jul 7, 2016 at 7:06 AM, Andres Freund wrote: >> On 2016-07-05 23:37:59 +0900, Masahiko Sawada wrote: >>> diff --git a/src/backend/access/heap/heapam.c >>> b/src/backend/access/heap/heapam.c >>> index 57

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Pete Stevenson
Good info, thanks for the note. Agreed that it is difficult to pull things apart to isolate these features for offload — so actually running experiments with offload is not possible, as you point out (and for other reasons). Maybe I could figure out the lines of code that add versions into a tab

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Amit Kapila
On Fri, Jul 8, 2016 at 5:27 PM, Michael Paquier wrote: > On Fri, Jul 8, 2016 at 4:04 PM, Magnus Hagander wrote: >> On Friday, July 8, 2016, Michael Paquier wrote: >>> >>> Hi all, >>> >>> Fujii-san has reminded me of the fact that we do not show in \df+ the >>> parallel status of a function. The

[HACKERS] Re: [BUGS] BUG #14230: Wrong timeline returned by pg_stop_backup on a standby

2016-07-08 Thread Marco Nenciarini
On 08/07/16 13:10, Michael Paquier wrote: > On Fri, Jul 8, 2016 at 6:40 PM, Marco Nenciarini > wrote: >> The resulting backup is working perfectly, because Postgres has no use >> for pg_stop_backup LSN, but this can confuse any tool that uses the stop >> LSN to figure out which WAL files are neede

Re: [HACKERS] Header and comments describing routines in incorrect shape in visibilitymap.c

2016-07-08 Thread Michael Paquier
On Fri, Jul 8, 2016 at 8:31 AM, Michael Paquier wrote: > On Fri, Jul 8, 2016 at 1:18 AM, Tom Lane wrote: >> Alvaro Herrera writes: >>> Regarding the first hunk, I don't like these INTERFACE sections too >>> much; they get seriously outdated over the time and aren't all that >>> helpful anyway.

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Michael Paquier
On Fri, Jul 8, 2016 at 4:04 PM, Magnus Hagander wrote: > On Friday, July 8, 2016, Michael Paquier wrote: >> >> Hi all, >> >> Fujii-san has reminded me of the fact that we do not show in \df+ the >> parallel status of a function. The output of \df+ is already very >> large, so I guess that any peo

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Petr Jelinek
On 08/07/16 12:47, Craig Ringer wrote: On 8 July 2016 at 09:41, Robert Haas mailto:robertmh...@gmail.com>> wrote: If you want to add a column to a table, you say ALTER TABLE .. ADD COLUMN. If you want to add a column to an extension, you say ALTER EXTENSION .. ADD TABLE. If you wa

Re: [HACKERS] [BUGS] BUG #14230: Wrong timeline returned by pg_stop_backup on a standby

2016-07-08 Thread Michael Paquier
On Fri, Jul 8, 2016 at 6:40 PM, Marco Nenciarini wrote: > The resulting backup is working perfectly, because Postgres has no use > for pg_stop_backup LSN, but this can confuse any tool that uses the stop > LSN to figure out which WAL files are needed by the backup (in this case > the only file nee

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Craig Ringer
On 8 July 2016 at 11:18, Alvaro Herrera wrote: > Tom Lane wrote: > > Simon Riggs writes: > > > > pg_am has existed for decades without supporting DDL > > > > That argument has been obsoleted by events ;-) ... and in any case, the > > reason we went without CREATE ACCESS METHOD for so long was no

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Craig Ringer
On 8 July 2016 at 09:41, Robert Haas wrote: > > > Personally, I'm in the group of people that don't see the need for DDL. > > There are already many successful features that don't utilize DDL, such > as > > backup, advisory locks and some features that use DDL that don't really > need > > to suc

[HACKERS] doc: Incorrect return type of IsForeignScanParallelSafe in fdwhandler.sgml

2016-07-08 Thread Etsuro Fujita
Hi, I noticed that the return type of IsForeignScanParallelSafe described in fdwhandler.sgml isn't correct; that should be bool, not Size. Please find attached a small patch for that. Best regards, Etsuro Fujita diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml index

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Simon Riggs
On 8 July 2016 at 11:09, Petr Jelinek wrote: > > One interesting thing will be making sure we can replicate from physical > standby in the future as you mentioned elsewhere in the thread but I think > that should be possible as long as you define the catalogs on master (not > really sure yet if t

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Petr Jelinek
On 08/07/16 10:59, Simon Riggs wrote: On 8 July 2016 at 03:55, Tom Lane mailto:t...@sss.pgh.pa.us>> wrote: > One of my examples was full text search and it does have > DDL, but that was an anti-example; all the feedback I have is that it was > much easier to use before it had DDL and

[HACKERS] Re: [BUGS] BUG #14230: Wrong timeline returned by pg_stop_backup on a standby

2016-07-08 Thread Marco Nenciarini
On 07/07/16 08:38, Michael Paquier wrote: > On Thu, Jul 7, 2016 at 12:57 AM, Marco Nenciarini > wrote: >> After further analysis, the issue is that we retrieve the starttli from >> the ControlFile structure, but it was using ThisTimeLineID when writing >> the backup label. >> >> I've attached a ve

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Simon Riggs
On 8 July 2016 at 03:55, Tom Lane wrote: > > One of my examples was full text search and it does have > > DDL, but that was an anti-example; all the feedback I have is that it was > > much easier to use before it had DDL and that forcing it to use DDL > pretty > > much killed it for most users.

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Craig Ringer
On 8 July 2016 at 03:50, Pete Stevenson wrote: > Hi Simon - > > Thanks for the note. I think it's fair to say that I didn't provide enough > context, so let me try and elaborate on my question. > Please reply in-line in posts to make it easier to follow conversations with multiple people. > It

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Pavel Stehule
2016-07-08 9:00 GMT+02:00 Michael Paquier : > Hi all, > > Fujii-san has reminded me of the fact that we do not show in \df+ the > parallel status of a function. The output of \df+ is already very > large, so I guess that any people mentally sane already use it with > the expanded display mode, and

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Magnus Hagander
On Friday, July 8, 2016, Michael Paquier wrote: > Hi all, > > Fujii-san has reminded me of the fact that we do not show in \df+ the > parallel status of a function. The output of \df+ is already very > large, so I guess that any people mentally sane already use it with > the expanded display mode

[HACKERS] Showing parallel status in \df+

2016-07-08 Thread Michael Paquier
Hi all, Fujii-san has reminded me of the fact that we do not show in \df+ the parallel status of a function. The output of \df+ is already very large, so I guess that any people mentally sane already use it with the expanded display mode, and it may not matter adding more information. Thoughts abo