[HACKERS] [sqlsmith] Planner crash on foreign table join

2017-04-07 Thread Andreas Seltenreich
Hi, testing master at f0e44021df with a loopback postgres_fdw installed, I see lots of crashes on queries joining foreign tables with various expressions. Below is a reduced recipe for the regression database and a backtrace. regards, Andreas --8<---cut here---start-

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Masahiko Sawada
On Sat, Apr 8, 2017 at 1:09 PM, Masahiko Sawada wrote: > On Sat, Apr 8, 2017 at 3:37 AM, Andres Freund wrote: >> Hi, >> >> When I started writing this, there were the following reamining CF >> items, minus bugfix ones which aren't bound by the code freeze. >> >> I think it makes sense to go throu

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Tom Lane
Thomas Munro writes: > Here is an attempt at option 2 from the menu I posted above. Questions: > 1. Does anyone object to this extension of pg_blocking_pids()'s > remit? If so, I could make it a separate function (that was option > 3). It seems an entirely principle-free change in the functio

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Masahiko Sawada
On Sat, Apr 8, 2017 at 3:37 AM, Andres Freund wrote: > Hi, > > When I started writing this, there were the following reamining CF > items, minus bugfix ones which aren't bound by the code freeze. > > I think it makes sense to go through those and see whether it's > realistic to commit any of them.

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 9:24 PM, Thomas Munro wrote: > 2. Did I understand correctly that it is safe to scan the list of > SERIALIZABLEXACTs and access the possibleUnsafeConflicts list while > holding only SerializableXactHashLock, Yes. > and that 'inLink' is the correct link to be following?

Re: [HACKERS] [PATCH] Add GUCs for predicate lock promotion thresholds

2017-04-07 Thread Kevin Grittner
On Mon, Feb 27, 2017 at 7:35 AM, Dagfinn Ilmari Mannsåker wrote: > Kevin Grittner writes: > >> It occurred to me that it would make sense to allow these settings >> to be attached to a database or table (though *not* a role). I'll >> look at that when I get back if you don't get to it first. > >

Re: [HACKERS] valgrind errors around dsa.c

2017-04-07 Thread Thomas Munro
On Sat, Apr 8, 2017 at 8:57 AM, Thomas Munro wrote: > On Sat, Apr 8, 2017 at 4:49 AM, Andres Freund wrote: >> Hi, >> >> newly added tests exercise parallel bitmap scans. And they trigger >> valgrind errors: >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2017-04-07%2007%3A10%

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-07 Thread Joe Conway
On 04/07/2017 05:36 PM, Robert Haas wrote: > On Fri, Apr 7, 2017 at 5:22 PM, Joe Conway wrote: >> 1) commit the 0002 patch now before the feature freeze and follow up >>with the regression test patch when ready in a couple of days >> 2) hold off on both patches until ready >> 3) push both patc

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Thomas Munro
On Sat, Apr 8, 2017 at 9:47 AM, Kevin Grittner wrote: > On Fri, Apr 7, 2017 at 3:47 PM, Thomas Munro > wrote: >> On Sat, Apr 8, 2017 at 6:35 AM, Kevin Grittner wrote: >>> On Fri, Apr 7, 2017 at 12:52 PM, Andres Freund wrote: >>> I'd rather fix the issue, than remove the tests entirely. Se

Re: [HACKERS] Performance improvement for joins where outer side is unique

2017-04-07 Thread Tom Lane
David Rowley writes: [ unique_joins_2017-04-07b.patch ] It turned out that this patch wasn't as close to committable as I'd thought, but after a full day of whacking at it, I got to a place where I thought it was OK. So, pushed. [ and that's a wrap for v10 feature freeze, I think ]

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Claudio Freire
On Fri, Apr 7, 2017 at 10:06 PM, Claudio Freire wrote: >>> >> + if (seg->num_dead_tuples >= seg->max_dead_tuples) >>> >> + { >>> >> + /* >>> >> + * The segment is overflowing, so we must allocate >>> >> a new segment. >>> >> +

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-07 Thread Bruce Momjian
On Fri, Mar 24, 2017 at 07:01:46AM +0100, Fabien COELHO wrote: > > Hello Peter, > > >I think the fix belongs into the web site CSS, so there is nothing to > >commit into PostgreSQL here. > > Indeed, the changes were only for the "remove nesting" solution. > > >I will close the commit fest entry

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Claudio Freire
On Fri, Apr 7, 2017 at 10:12 PM, Andres Freund wrote: > On 2017-04-07 22:06:13 -0300, Claudio Freire wrote: >> On Fri, Apr 7, 2017 at 9:56 PM, Andres Freund wrote: >> > Hi, >> > >> > >> > On 2017-04-07 19:43:39 -0300, Claudio Freire wrote: >> >> On Fri, Apr 7, 2017 at 5:05 PM, Andres Freund wrot

Re: [HACKERS] Push down more UPDATEs/DELETEs in postgres_fdw

2017-04-07 Thread Robert Haas
On Wed, Mar 22, 2017 at 6:20 AM, Etsuro Fujita wrote: > On 2017/02/22 19:57, Rushabh Lathia wrote: >> Marked this as Ready for Committer. > > I noticed that this item in the CF app was incorrectly marked as Committed. > This patch isn't committed, so I returned it to the previous status. I also >

Re: [HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Keith Fiske
On Fri, Apr 7, 2017 at 8:41 PM, Tom Lane wrote: > Keith Fiske writes: > > On Fri, Apr 7, 2017 at 2:46 PM, Tom Lane wrote: > >> Joe Conway writes: > >>> Apparently INSERT and SELECT on the parent partitioned table skip > normal > >>> acl checks on the partitions. Is that intended behavior? > >

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Andres Freund
On 2017-04-07 22:06:13 -0300, Claudio Freire wrote: > On Fri, Apr 7, 2017 at 9:56 PM, Andres Freund wrote: > > Hi, > > > > > > On 2017-04-07 19:43:39 -0300, Claudio Freire wrote: > >> On Fri, Apr 7, 2017 at 5:05 PM, Andres Freund wrote: > >> > Hi, > >> > > >> > I've *not* read the history of this

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Claudio Freire
On Fri, Apr 7, 2017 at 9:56 PM, Andres Freund wrote: > Hi, > > > On 2017-04-07 19:43:39 -0300, Claudio Freire wrote: >> On Fri, Apr 7, 2017 at 5:05 PM, Andres Freund wrote: >> > Hi, >> > >> > I've *not* read the history of this thread. So I really might be >> > missing some context. >> > >> > >>

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Andres Freund
Hi, On 2017-04-07 19:43:39 -0300, Claudio Freire wrote: > On Fri, Apr 7, 2017 at 5:05 PM, Andres Freund wrote: > > Hi, > > > > I've *not* read the history of this thread. So I really might be > > missing some context. > > > > > >> From e37d29c26210a0f23cd2e9fe18a264312fecd383 Mon Sep 17 00:00:0

Re: [HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Tom Lane
Keith Fiske writes: > On Fri, Apr 7, 2017 at 2:46 PM, Tom Lane wrote: >> Joe Conway writes: >>> Apparently INSERT and SELECT on the parent partitioned table skip normal >>> acl checks on the partitions. Is that intended behavior? >> Yes, this matches normal inheritance behavior. > Should that

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-07 Thread Tom Lane
Alvaro Herrera writes: > Interesting. I wonder if it's possible that a relcache invalidation > would cause these values to get lost for some reason, because that would > be dangerous. > I suppose the rationale is that this shouldn't happen because any > operation that does things this way must h

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 5:22 PM, Joe Conway wrote: > On 04/07/2017 11:37 AM, Mike Palmiotto wrote: I found some missing bits in the 0002 patch -- new version attached. Will wait on new regression tests before committing, but I expect we'll have those by end of today and be able to co

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-04-07 Thread Peter Eisentraut
On 4/6/17 14:32, Pavel Stehule wrote: > I like to see any proposals about syntax or implementation. > > Using PRAGMA is one variant - introduced by PLpgSQL origin - Ada > language. The PRAGMA syntax can be used for PRAGMA autonomous with well > known syntax. It scales well - it supports function,

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 6:32 PM, Michael Paquier wrote: > On Sat, Apr 8, 2017 at 1:59 AM, Robert Haas wrote: >> On Fri, Apr 7, 2017 at 3:59 AM, Heikki Linnakangas wrote: >>> I think the "SCRAM" part is more important than "SHA-256", so -1 on that. >> >> I agree. The point here isn't that we're u

Re: [HACKERS] [COMMITTERS] pgsql: Improve 64bit atomics support.

2017-04-07 Thread Andres Freund
On 2017-04-07 16:36:09 -0700, Andres Freund wrote: > On 2017-04-07 19:55:21 -0300, Alvaro Herrera wrote: > > Andres Freund wrote: > > > Improve 64bit atomics support. > > > > > > When adding atomics back in b64d92f1a, I added 64bit support as > > > optional; there wasn't yet a direct user in sight

Re: [HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Keith Fiske
On Fri, Apr 7, 2017 at 2:46 PM, Tom Lane wrote: > Joe Conway writes: > > Apparently INSERT and SELECT on the parent partitioned table skip normal > > acl checks on the partitions. Is that intended behavior? > > Yes, this matches normal inheritance behavior. > > regards, t

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-07 Thread Alvaro Herrera
Alvaro Herrera wrote: > I suppose the rationale is that this shouldn't happen because any > operation that does things this way must hold an exclusive lock on the > relation. But that doesn't guarantee that the relcache entry is > completely stable, does it? If we can get proof of that, then thi

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-07 Thread Alvaro Herrera
I have claimed this patch as committer FWIW. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://ww

Re: [HACKERS] [COMMITTERS] pgsql: Improve 64bit atomics support.

2017-04-07 Thread Andres Freund
On 2017-04-07 19:55:21 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Improve 64bit atomics support. > > > > When adding atomics back in b64d92f1a, I added 64bit support as > > optional; there wasn't yet a direct user in sight. That turned out to > > be a bit short-sighted, it'd already

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-07 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: > The attached patch is quiiiccck-and-dirty-hack of Michael's patch > just as a PoC of my proposal quoted above. This also passes the > 006 test. The major changes are the following. > > - Moved sync_above and truncted_to into RelationData. Interesting. I wonder if it

Re: [HACKERS] Interval for launching the table sync worker

2017-04-07 Thread Peter Eisentraut
On 4/7/17 01:10, Masahiko Sawada wrote: > It's not critical but it could be problem. So I thought we should fix > it before the PostgreSQL 10 release. If it's not appropriate as an > open item I'll remove it. You wrote that you "sent" a patch, but I don't see a patch anywhere. I think a nonintrus

Re: [HACKERS] [COMMITTERS] pgsql: Improve 64bit atomics support.

2017-04-07 Thread Alvaro Herrera
Andres Freund wrote: > Improve 64bit atomics support. > > When adding atomics back in b64d92f1a, I added 64bit support as > optional; there wasn't yet a direct user in sight. That turned out to > be a bit short-sighted, it'd already have been useful a number of times. Seems like this killed an a

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Claudio Freire
On Fri, Apr 7, 2017 at 5:05 PM, Andres Freund wrote: > Hi, > > I've *not* read the history of this thread. So I really might be > missing some context. > > >> From e37d29c26210a0f23cd2e9fe18a264312fecd383 Mon Sep 17 00:00:00 2001 >> From: Claudio Freire >> Date: Mon, 12 Sep 2016 23:36:42 -0300 >

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Claudio Freire
On Fri, Apr 7, 2017 at 7:43 PM, Claudio Freire wrote: >>> + * Lookup in that structure proceeds sequentially in the list of segments, >>> + * and with a binary search within each segment. Since segment's size grows >>> + * exponentially, this retains O(N log N) lookup complexity. >> >> N log N is

Re: [HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Peter Eisentraut
On 4/7/17 16:50, Andres Freund wrote: > On 2017-04-07 22:47:55 +0200, Erik Rijkers wrote: >> monitoring.sgml has one tag missing > > Is that actually an issue? SGML allows skipping certain close tags, and > IIRC row is one them. The issue is a weird one. For some reason, older tool chains compl

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Michael Paquier
On Sat, Apr 8, 2017 at 1:59 AM, Robert Haas wrote: > On Fri, Apr 7, 2017 at 3:59 AM, Heikki Linnakangas wrote: >> I think the "SCRAM" part is more important than "SHA-256", so -1 on that. > > I agree. The point here isn't that we're using a better hashing > method, even if a lot of people *think

Re: [HACKERS] mvstats triggers 32bit warnings

2017-04-07 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > compiling on linux 32 bit I get a lot of warnings due to printf format. > I suspect most of them should just be cured by using %zd or %zu instead > of %ld. You're right, they are. Confirmed, and pushed fix using %zd. I suppose %zu would be "more correct", but this

Re: [HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Peter Eisentraut
On 4/7/17 16:47, Erik Rijkers wrote: > monitoring.sgml has one tag missing fixed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

2017-04-07 Thread Alvaro Herrera
Tomas Vondra wrote: > 1) bringetbitmap(PG_FUNCTION_ARGS) > > + /* > + * Estimate the approximate size of btup and allocate memory for btup. > + */ > + btupInitSize = bdesc->bd_tupdesc->natts * 16; > + btup = palloc(btupInitSize); > > What is the reasoning behind this estimate? I mean, this onl

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 3:47 PM, Thomas Munro wrote: > On Sat, Apr 8, 2017 at 6:35 AM, Kevin Grittner wrote: >> On Fri, Apr 7, 2017 at 12:52 PM, Andres Freund wrote: >> >>> I'd rather fix the issue, than remove the tests entirely. Seems quite >>> possible to handle blocking on Safesnapshot in a

[HACKERS] mvstats triggers 32bit warnings

2017-04-07 Thread Andres Freund
Hi, compiling on linux 32 bit I get a lot of warnings due to printf format. I suspect most of them should just be cured by using %zd or %zu instead of %ld. /home/andres/src/postgresql/src/backend/statistics/mvdistinct.c: In function ‘statext_ndistinct_deserialize’: /home/andres/src/postgresql/sr

Re: [HACKERS] Compilation warning with MSVC in pg_depend.c

2017-04-07 Thread Peter Eisentraut
On 4/6/17 23:03, Michael Paquier wrote: > Hi all, > > I am getting the following warning with MSVC 2010 for what is a result > of 3217327 (identity columns): > c:\users\michael\git\postgres\src\backend\catalog\pg_depend.c(615): > warning C4715: 'getOwnedSequence' : not all control paths return a

Re: [HACKERS] Making clausesel.c Smarter

2017-04-07 Thread Claudio Freire
On Fri, Apr 7, 2017 at 2:28 AM, David Rowley wrote: >> + if (rqlist->hibound == DEFAULT_INEQ_SEL || rqlist->lobound == >> DEFAULT_INEQ_SEL) >> + { >> + /* No point in checking null selectivity, DEFAULT_INEQ_SEL >> implies we have no stats */ >> + s2 = DEFAULT_RANGE_INEQ_SEL; >> + } >> + el

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-07 Thread Joe Conway
On 04/07/2017 11:37 AM, Mike Palmiotto wrote: >>> I found some missing bits in the 0002 patch -- new version attached. >>> Will wait on new regression tests before committing, but I expect we'll >>> have those by end of today and be able to commit the rest tomorrow. >> >> Attached are the regressio

Re: [HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Andres Freund
Hi, On 2017-04-07 23:00:01 +0200, Erik Rijkers wrote: > On 2017-04-07 22:50, Andres Freund wrote: > > On 2017-04-07 22:47:55 +0200, Erik Rijkers wrote: > > > monitoring.sgml has one tag missing > > > > Is that actually an issue? SGML allows skipping certain close tags, and > > IIRC row is one th

Re: [HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Erik Rijkers
On 2017-04-07 22:50, Andres Freund wrote: On 2017-04-07 22:47:55 +0200, Erik Rijkers wrote: monitoring.sgml has one tag missing Is that actually an issue? SGML allows skipping certain close tags, and IIRC row is one them. We'll probably move to xml at some point not too far away, but I don't

Re: [HACKERS] valgrind errors around dsa.c

2017-04-07 Thread Thomas Munro
On Sat, Apr 8, 2017 at 4:49 AM, Andres Freund wrote: > Hi, > > newly added tests exercise parallel bitmap scans. And they trigger > valgrind errors: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2017-04-07%2007%3A10%3A01 > > > ==4567== VALGRINDERROR-BEGIN > ==4567== Condition

Re: [HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Andres Freund
On 2017-04-07 22:47:55 +0200, Erik Rijkers wrote: > monitoring.sgml has one tag missing Is that actually an issue? SGML allows skipping certain close tags, and IIRC row is one them. We'll probably move to xml at some point not too far away, but I don't think it makes much sense to fix these one-

[HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Erik Rijkers
monitoring.sgml has one tag missing--- doc/src/sgml/monitoring.sgml.orig 2017-04-07 22:37:55.388708334 +0200 +++ doc/src/sgml/monitoring.sgml 2017-04-07 22:38:16.582047695 +0200 @@ -1275,6 +1275,7 @@ ProcArrayGroupUpdate Waiting for group leader to clear transaction

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Thomas Munro
On Sat, Apr 8, 2017 at 6:35 AM, Kevin Grittner wrote: > On Fri, Apr 7, 2017 at 12:52 PM, Andres Freund wrote: > >> I'd rather fix the issue, than remove the tests entirely. Seems quite >> possible to handle blocking on Safesnapshot in a similar manner as >> pg_blocking_pids? > > I'll see what I

Re: [HACKERS] BRIN desummarization writes junk WAL records

2017-04-07 Thread Alvaro Herrera
Tom Lane wrote: > The proximate cause of the exception seems to be that > brinSetHeapBlockItemptr is being passed pagesPerRange = 0, > which is problematic since HEAPBLK_TO_REVMAP_INDEX tries to > divide by that. Looking one level down, the bogus value > seems to be coming out of an xl_brin_desum

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > My offer to work with you on amcheck verification of WARM invariants > remains open. If nothing else, structuring things so that verification > is possible may clarify your design. Formalizing the preconditions, > postconditions, and legal states for on-disk structures mig

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Andres Freund
Hi, I've *not* read the history of this thread. So I really might be missing some context. > From e37d29c26210a0f23cd2e9fe18a264312fecd383 Mon Sep 17 00:00:00 2001 > From: Claudio Freire > Date: Mon, 12 Sep 2016 23:36:42 -0300 > Subject: [PATCH] Vacuum: allow using more than 1GB work mem > >

Re: [HACKERS] Undefined psql variables

2017-04-07 Thread Pavel Stehule
2017-04-07 21:04 GMT+02:00 Fabien COELHO : > > Hello Pavel, > > I wish I could have an explanation about why the :?varname (or some other >>> variant) syntax I suggested has a "namespace" issue. >>> >>> The advantage that I see is that although it is obviously ugly, it is >>> ugly >>> in the conti

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Peter Geoghegan
On Fri, Apr 7, 2017 at 12:28 PM, Alvaro Herrera wrote: > Peter Geoghegan wrote: >> On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund wrote: >> > Write Amplification Reduction Method (WARM) >> > - fair number of people don't think it's ready for v10. > > Given the number of votes against putting this

Re: [HACKERS] [sqlsmith] Unpinning error in parallel worker

2017-04-07 Thread Robert Haas
On Wed, Apr 5, 2017 at 10:18 AM, Robert Haas wrote: >>> Ugh, OK. I committed this, but I think this whole file needs a visit >>> from the message style police. >> >> Like this? > > I was thinking of maybe not creating two separate (translatable) > messages, and just using "could not attach to dyn

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 3:30 PM, Andres Freund wrote: > On 2017-04-07 16:28:03 -0300, Alvaro Herrera wrote: >> Peter Geoghegan wrote: >> > > - can't move to next fest because it's waiting-on-author, which doesn't >> > > allow that. Doesn't strike me as a useful restriction. >> > >> > I agree tha

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
On 2017-04-07 16:28:03 -0300, Alvaro Herrera wrote: > Peter Geoghegan wrote: > > > - can't move to next fest because it's waiting-on-author, which doesn't > > > allow that. Doesn't strike me as a useful restriction. > > > > I agree that that CF app restriction makes little sense. > > What the

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. Given the number of votes against putting this on pg10, I am going to back off from this patch now, with an ey

Re: [HACKERS] postgres_fdw bug in 9.6

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 2:33 PM, Robert Haas wrote: > On Mon, Apr 3, 2017 at 2:12 AM, Etsuro Fujita > wrote: >> On 2017/04/01 1:32, Jeff Janes wrote: >>> On Thu, Mar 23, 2017 at 5:20 AM, Etsuro Fujita >>> mailto:fujita.ets...@lab.ntt.co.jp>> wrote: >>> Done. Attached is a new version of the p

Re: [HACKERS] Undefined psql variables

2017-04-07 Thread Fabien COELHO
Hello Pavel, I wish I could have an explanation about why the :?varname (or some other variant) syntax I suggested has a "namespace" issue. The advantage that I see is that although it is obviously ugly, it is ugly in the continuity of the various :["'?]varname syntaxes already offered and it

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 2:53 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Andres Freund wrote: >>> Write Amplification Reduction Method (WARM) >>> - fair number of people don't think it's ready for v10. > >> I'm going over this one now with Pavan, with the intent of getting it in >> committable

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Tom Lane
Alvaro Herrera writes: > Andres Freund wrote: >> Write Amplification Reduction Method (WARM) >> - fair number of people don't think it's ready for v10. > I'm going over this one now with Pavan, with the intent of getting it in > committable shape. I have to agree with Andres that this is not som

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Merlin Moncure
On Fri, Apr 7, 2017 at 12:54 PM, Tom Lane wrote: > Tomas Vondra writes: >> On 04/07/2017 06:31 PM, Merlin Moncure wrote: >>> I think your math is off. Looking at your attachments, planning time >>> is 0.056ms, not 0.56ms. This is in no way relevant to performance on >>> the order of your measur

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
On 2017-04-07 15:45:33 -0300, Alvaro Herrera wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. > > I'm going over this one now with Pavan, with the intent of getting it in > committable shape. > > I may be biased, but the claimed pe

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Tom Lane
Andres Freund writes: > I think it makes sense to go through those and see whether it's > realistic to commit any of them. > Unique Joins > - Tom's discussing things with David, not sure. Working on this one today. > Generic type subscripting > - still some review back and forth > - probably sh

Re: [HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Tom Lane
Joe Conway writes: > Apparently INSERT and SELECT on the parent partitioned table skip normal > acl checks on the partitions. Is that intended behavior? Yes, this matches normal inheritance behavior. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Peter Geoghegan
On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund wrote: > Write Amplification Reduction Method (WARM) > - fair number of people don't think it's ready for v10. > - can't move to next fest because it's waiting-on-author, which doesn't > allow that. Doesn't strike me as a useful restriction. I agr

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Andres Freund wrote: > Unique Joins > - Tom's discussing things with David, not sure. This one was already included-and-removed from 9.6, Tom had said he'd give it priority during the current cycle as I recall. It seems unfair that it's still waiting for review on the last day of pg10's last com

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 1:37 PM, Andres Freund wrote: > I think it makes sense to go through those and see whether it's > realistic to commit any of them. > > Ready for Committer: > > Add GUCs for predicate lock promotion thresholds: > - claimed by Kevin, should be easy enough I was planning on p

[HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
Hi, When I started writing this, there were the following reamining CF items, minus bugfix ones which aren't bound by the code freeze. I think it makes sense to go through those and see whether it's realistic to commit any of them. Ready for Committer: Add GUCs for predicate lock promotion thre

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-07 Thread Mike Palmiotto
On Thu, Apr 6, 2017 at 5:52 PM, Joe Conway wrote: > On 04/06/2017 12:35 PM, Tom Lane wrote: >> Joe Conway writes: >>> Any thoughts on whether 0001a and 0001b ought to be backpatched? I'm >>> thinking not given the lack of past complaints but it might make sense >>> to do. >> >> I think 0001a abso

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 12:52 PM, Andres Freund wrote: > I'd rather fix the issue, than remove the tests entirely. Seems quite > possible to handle blocking on Safesnapshot in a similar manner as > pg_blocking_pids? I'll see what I can figure out. -- Kevin Grittner -- Sent via pgsql-hacker

Re: [HACKERS] postgres_fdw bug in 9.6

2017-04-07 Thread Robert Haas
On Mon, Apr 3, 2017 at 2:12 AM, Etsuro Fujita wrote: > On 2017/04/01 1:32, Jeff Janes wrote: >> On Thu, Mar 23, 2017 at 5:20 AM, Etsuro Fujita >> mailto:fujita.ets...@lab.ntt.co.jp>> wrote: >> Done. Attached is a new version of the patch. >> Is the fix for 9.6.3 going to be just a back port o

Re: [HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Keith Fiske
On Fri, Apr 7, 2017 at 2:05 PM, Joe Conway wrote: > Apparently INSERT and SELECT on the parent partitioned table skip normal > acl checks on the partitions. Is that intended behavior? > > 8<--- > test=# create user part_test; > CREATE ROLE > test=# > test=# create table t1

Re: [HACKERS] [PATCH] Remove unused argument in btree_xlog_split

2017-04-07 Thread Robert Haas
On Thu, Apr 6, 2017 at 8:21 AM, Aleksander Alekseev wrote: > Hi Robert, > >> Hmm. I don't see anything wrong with that, particularly, but it seems >> we also don't need the distinction between XLOG_BTREE_SPLIT_L and >> XLOG_BTREE_SPLIT_L_ROOT or likewise between XLOG_BTREE_SPLIT_R and >> XLOG_BTR

[HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Joe Conway
Apparently INSERT and SELECT on the parent partitioned table skip normal acl checks on the partitions. Is that intended behavior? 8<--- test=# create user part_test; CREATE ROLE test=# test=# create table t1 (id int) partition by range ((id % 4)); CREATE TABLE test=# create

Re: [HACKERS] UPDATE of partition key

2017-04-07 Thread Andres Freund
On 2017-04-07 13:55:51 -0400, Robert Haas wrote: > On Wed, Apr 5, 2017 at 5:54 AM, Amit Langote > wrote: > > Marked as ready for committer. > > Andres seems to have changed the status of this patch to "Needs > review" and then, 30 seconds later, to "Waiting on author" > there's no actual email on

Re: [HACKERS] UPDATE of partition key

2017-04-07 Thread Robert Haas
On Wed, Apr 5, 2017 at 5:54 AM, Amit Langote wrote: > Marked as ready for committer. Andres seems to have changed the status of this patch to "Needs review" and then, 30 seconds later, to "Waiting on author", but there's no actual email on the thread explaining what his concerns were. I'm going

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Tom Lane
Tomas Vondra writes: > On 04/07/2017 06:31 PM, Merlin Moncure wrote: >> I think your math is off. Looking at your attachments, planning time >> is 0.056ms, not 0.56ms. This is in no way relevant to performance on >> the order of your measured TPS. How are you measuring TPS? > Not sure where d

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Andres Freund
On 2017-04-07 12:49:22 -0500, Kevin Grittner wrote: > On Fri, Apr 7, 2017 at 12:28 PM, Tom Lane wrote: > > Andrew Dunstan writes: > >> On 04/07/2017 12:57 PM, Andres Freund wrote: > >>> I don't think any recent changes are supposed to affect deadlock > >>> detector behaviour? > > > >> Both these

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-07 Thread Andres Freund
On 2017-04-06 13:43:55 -0700, Andres Freund wrote: > On 2017-04-06 10:00:32 +0530, Dilip Kumar wrote: > > On Tue, Apr 4, 2017 at 5:51 AM, Dilip Kumar wrote: > > > Sure I can do that, In attached patch, I only fixed the problem of not > > > executing the bitmap test. Now, I will add few cases to c

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 12:28 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 04/07/2017 12:57 PM, Andres Freund wrote: >>> I don't think any recent changes are supposed to affect deadlock >>> detector behaviour? > >> Both these machines have CLOBBER_CACHE_ALWAYS set. And on both machines >> re

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-04-07 Thread Robert Haas
On Thu, Mar 9, 2017 at 5:49 PM, Robert Haas wrote: > However, I just realized that in > both this case and in the case of group XID clearing, we weren't > advertising a wait event for the PGSemaphoreLock calls that are part > of the group locking machinery. I think we should fix that, because a >

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Tomas Vondra
On 04/07/2017 06:31 PM, Merlin Moncure wrote: On Fri, Apr 7, 2017 at 5:16 AM, Prakash Itnal wrote: Hello, We currently use psotgres 9.3 in our products. Recently we upgraded to postgres 9.6. But with 9.6 we have seen a drastic reduction in throughput. After analyzing carefully I found that "pl

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Tom Lane
Andrew Dunstan writes: > On 04/07/2017 12:57 PM, Andres Freund wrote: >> I don't think any recent changes are supposed to affect deadlock >> detector behaviour? > Both these machines have CLOBBER_CACHE_ALWAYS set. And on both machines > recent changes have made the isolation tests run much much l

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Andrew Dunstan
On 04/07/2017 12:57 PM, Andres Freund wrote: > Hi, > > There's two machines that recently report changes in deadlock detector > output: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax&dt=2017-04-05%2018%3A58%3A04 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=friarbird&

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 3:59 AM, Heikki Linnakangas wrote: > I think the "SCRAM" part is more important than "SHA-256", so -1 on that. I agree. The point here isn't that we're using a better hashing method, even if a lot of people *think* that's the point. The point is we're using a modern algor

[HACKERS] pgbench --progress-timestamp no longer works correctly

2017-04-07 Thread Jeff Janes
--progress-timestamp is supposed to make -P report a Unix Epoch time stamp, for easy correlation with the entries in other log files (like the postgres server log file using %n). But that broke in this commit: commit 1d63f7d2d180c8708bc12710254eb7b45823440f Author: Tom Lane Date: Mon Jan 2 13:

[HACKERS] recent deadlock regression test failures

2017-04-07 Thread Andres Freund
Hi, There's two machines that recently report changes in deadlock detector output: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax&dt=2017-04-05%2018%3A58%3A04 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=friarbird&dt=2017-04-07%2004%3A20%3A01 both just failed twice in a

Re: [HACKERS] ExecPrepareExprList and per-query context

2017-04-07 Thread Tom Lane
Amit Langote writes: > Should ExecPrepareExprList also switch to estate->es_query_cxt? Good point; I'm surprised we haven't noted any failures from that. We surely want the entire result data structure to be in the same memory context. There are not very many callers right now, and I guess they

[HACKERS] valgrind errors around dsa.c

2017-04-07 Thread Andres Freund
Hi, newly added tests exercise parallel bitmap scans. And they trigger valgrind errors: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2017-04-07%2007%3A10%3A01 ==4567== VALGRINDERROR-BEGIN ==4567== Conditional jump or move depends on uninitialised value(s) ==4567==at 0x5F

Re: [HACKERS] Compiler warning in costsize.c

2017-04-07 Thread Tom Lane
Michael Paquier writes: > Bah. This actually fixes nothing. Attached is a different patch that > really addresses the problem, by removing the variable because we > don't want planner_rt_fetch() to run for non-Assert builds. I don't really like any of these fixes, because they take the code furth

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Merlin Moncure
On Fri, Apr 7, 2017 at 5:16 AM, Prakash Itnal wrote: > Hello, > > We currently use psotgres 9.3 in our products. Recently we upgraded to > postgres 9.6. But with 9.6 we have seen a drastic reduction in throughput. > After analyzing carefully I found that "planner time" in 9.6 is very high. > Below

Re: [HACKERS] Undefined psql variables

2017-04-07 Thread Pavel Stehule
2017-04-07 9:52 GMT+02:00 Fabien COELHO : > > Hello Corey, > > \if defined varname \if sql boolean expression to send to server \if compare value operator value >>> >>> I'm still thinking:-) >>> >>> Independently of the my aethetical complaint against having

Re: [HACKERS] Duplicate assignment in Unicode/convutils.pm

2017-04-07 Thread Heikki Linnakangas
On 04/07/2017 09:32 AM, Kyotaro HORIGUCHI wrote: Hi, I found that convutils.pl contains a harmless duplicate assignemnt. my $out = {f => $fname, l => $., code => hex($1), ucs => hex($2), comment => $4, direction => BOTH,

Re: [HACKERS] pg_basebackup: Allow use of arbitrary compression program

2017-04-07 Thread Jeff Janes
On Thu, Apr 6, 2017 at 7:04 PM, Michael Harris wrote: > Hello, > > Back in pg 9.2, we hacked a copy of pg_basebackup to add a command > line option which would allow the user to specify an arbitrary > external program (potentially including arguments) to be used to > compress the tar backup. > >

Re: [HACKERS] [PATCH] Document the order of changing certain settings when using hot-standby servers

2017-04-07 Thread Yorick Peterse
Done! It can be found at https://commitfest.postgresql.org/14/1110/ Thanks for reviewing thus far :) Yorick -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread David G. Johnston
On Fri, Apr 7, 2017 at 8:29 AM, Aleksander Alekseev < a.aleks...@postgrespro.ru> wrote: > Andres, Tatsuo, > > Thank you for sharing your thoughts. > > > -1 - I frequently just override earlier parameters by adding an > > include at the end of the file. Also, with postgresql.auto.conf it's > > eve

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Euler Taveira
2017-04-07 12:14 GMT-03:00 Aleksander Alekseev : > Recently I've discovered that if there are multiple values of the same > parameter in postgresql.conf PostgreSQL will silently use the last one. > It looks like not the best approach to me. For instance, user can find > the first value in the conf

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread David Steele
On 4/7/17 11:22 AM, Tatsuo Ishii wrote: >>> Recently I've discovered that if there are multiple values of the same >>> parameter in postgresql.conf PostgreSQL will silently use the last one. >>> It looks like not the best approach to me. For instance, user can find >>> the first value in the config

  1   2   >