Re: ALTER TABLE ADD COLUMN fast default

2018-04-02 Thread Andrew Dunstan
On Fri, Mar 30, 2018 at 10:08 AM, Andrew Dunstan wrote: > On Fri, Mar 30, 2018 at 5:00 AM, Andres Freund wrote: [ missing values are loaded in the TupleDesc by RelationBuildTupleDesc ] >>> > I'm still strongly object to do doing this unconditionally for queries >>> > that never need any of this

Re: Rewriting the test of pg_upgrade as a TAP test - take two

2018-04-02 Thread Michael Paquier
On Mon, Apr 02, 2018 at 10:35:09AM -0400, Peter Eisentraut wrote: > I took a quick look at that part. It appears to be quite invasive, more > than I would have hoped. Basically, it imposes that from now on all > program invocations must observe the bindir setting, which someone is > surely going

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-02 Thread Kyotaro HORIGUCHI
Hello. At Mon, 2 Apr 2018 16:11:12 -0300, Alvaro Herrera wrote in <20180402191112.wneiyj4v5upnfjst@alvherre.pgsql> > Why do we need AccessExclusiveLock on all children of a relation that we > want to scan to search for rows not satisfying the constraint? I think > it should be enough to get Sha

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-02 Thread Ashutosh Bapat
On Mon, Apr 2, 2018 at 1:40 AM, Andres Freund wrote: > Hi, > > On 2018-02-26 17:20:05 +0530, Ashutosh Bapat wrote: >> In a multi-level partitioned table, a parent whole-row reference gets >> translated into nested ConvertRowtypeExpr with child whole-row >> reference as the leaf. During the executi

Re: BUG #14941: Vacuum crashes

2018-04-02 Thread Michael Paquier
On Fri, Mar 30, 2018 at 05:19:46PM -0700, Andres Freund wrote: > On 2018-03-30 18:39:01 +, Bossart, Nathan wrote: >> I think the most straightforward approach for fixing this is to add >> skip-locked functionality in find_all_inheritors(), >> find_inheritance_children(), and vac_open_indexes(),

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-02 Thread Amit Langote
Fujita-san, On 2018/04/02 21:29, Etsuro Fujita wrote: > (2018/04/02 18:49), Amit Langote wrote: >> I looked at the new patch.  It looks good overall, although I have one >> question -- IIUC, BeginForeignInsert() performs actions that are >> equivalent of performing PlanForeignModify() + BeginForei

Re: Add default role 'pg_access_server_files'

2018-04-02 Thread Michael Paquier
On Mon, Apr 02, 2018 at 05:09:21PM -0400, Stephen Frost wrote: > * Michael Paquier (mich...@paquier.xyz) wrote: >> No refactoring for pg_file_unlink and its v1.1? > > I considered each function and thought about if it'd make sense to > refactor them or if they were simple enough that the additiona

Re: [HACKERS] [PATCH] Lockable views

2018-04-02 Thread Yugo Nagata
On Mon, 2 Apr 2018 18:32:53 +0900 Yugo Nagata wrote: > On Thu, 29 Mar 2018 17:26:36 -0700 > Andres Freund wrote: > > Thank you for your comments. I attach a patch to fix issues > you've pointed out. I found a typo in the documentation and attach the updated patch. Regards, > > > Hi, > > >

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-02 Thread Amit Langote
On 2018/04/02 21:26, Etsuro Fujita wrote: > (2018/03/30 22:28), Alvaro Herrera wrote: >> Etsuro Fujita wrote: >> >>> Now we have ON CONFLICT for partitioned tables, which requires the >>> conversion map to be computed in ExecInitPartitionInfo, so I updated the >>> patch so that we keep the former s

Re: WIP: Covering + unique indexes.

2018-04-02 Thread Peter Geoghegan
On Mon, Apr 2, 2018 at 4:27 PM, Alexander Korotkov wrote: > I thought abut that another time and I decided that it would be safer > to use 13th bit in index tuple flags. There are already attempt to > use whole 6 bytes of tid for not heap pointer information [1]. Thus, it > would be safe to use

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Peter Geoghegan
On Mon, Apr 2, 2018 at 7:54 PM, Robert Haas wrote: > Also, this really does make it impossible to write reliable programs. > Imagine that, while the server is running, somebody runs a program > which opens a file in the data directory, calls fsync() on it, and > closes it. If the fsync() fails, p

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 19:40:12 -0700, Peter Geoghegan wrote: > > So what happens if there's a concurrent insertion of a potentially > > matching tuple? > > It's not a special case. In all likelihood, you get a dup violation. > This is a behavior that I argued for from an early stage. Right. I think

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-02 Thread Peter Geoghegan
On Mon, Apr 2, 2018 at 8:11 PM, Pavan Deolasee wrote: > Honestly I don't think Peter ever raised concerns about the join, though I > could be missing early discussions when I wasn't paying attention. It's > there from day 1. Peter raised concerns about the two RTE stuff which was > necessitated wh

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-02 Thread Pavan Deolasee
On Tue, Apr 3, 2018 at 8:31 AM, Robert Haas wrote: > On Mon, Apr 2, 2018 at 10:40 PM, Peter Geoghegan wrote: > > On Mon, Apr 2, 2018 at 7:18 PM, Andres Freund > wrote: > >> I did a scan through this, as I hadn't been able to keep with the thread > >> previously. Sorry if some of the things ment

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-02 Thread Robert Haas
On Mon, Apr 2, 2018 at 10:40 PM, Peter Geoghegan wrote: > On Mon, Apr 2, 2018 at 7:18 PM, Andres Freund wrote: >> I did a scan through this, as I hadn't been able to keep with the thread >> previously. Sorry if some of the things mentioned here have been >> discussed previously. I am just reading

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Robert Haas
On Mon, Apr 2, 2018 at 2:53 PM, Anthony Iliopoulos wrote: > Given precisely that the dirty pages which cannot been written-out are > practically thrown away, the semantics of fsync() (after the 4.13 fixes) > are essentially correct: the first call indicates that a writeback error > indeed occurred

RE: Speedup of relation deletes during recovery

2018-04-02 Thread Tsunakawa, Takayuki
From: Jerry Sievers [mailto:gsiever...@comcast.net] > Wonder if this is the case for streaming standbys replaying truncates > also? Yes, As I wrote in my previous mail, TRUNCATE is worse than DROP TABLE. Regards Takayuki Tsunakawa

Re: Diagonal storage model

2018-04-02 Thread Michael Paquier
On Sun, Apr 01, 2018 at 03:48:07PM +0300, Konstantin Knizhnik wrote: > Attach please find patch with first prototype implementation. It > provides about 3.14 times improvement of performance at most of TPC-H > queries. Congratulations in finding a patch able to improve all workloads of Postgres i

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-02 Thread Peter Geoghegan
On Mon, Apr 2, 2018 at 7:18 PM, Andres Freund wrote: > I did a scan through this, as I hadn't been able to keep with the thread > previously. Sorry if some of the things mentioned here have been > discussed previously. I am just reading through the patch in its own > order, so please excuse if the

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-02 Thread Andres Freund
Hi, On 2018-03-30 12:10:04 +0100, Simon Riggs wrote: > On 29 March 2018 at 10:50, Simon Riggs wrote: > > On 28 March 2018 at 12:00, Pavan Deolasee wrote: > > > >> v27 attached, though review changes are in > >> the add-on 0005 patch. > > > > This all looks good now, thanks for making all of thos

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Thomas Munro
On Tue, Apr 3, 2018 at 3:03 AM, Craig Ringer wrote: > I see little benefit to not just PANICing unconditionally on EIO, really. It > shouldn't happen, and if it does, we want to be pretty conservative and > adopt a data-protective approach. > > I'm rather more worried by doing it on ENOSPC. Which

Re: 2018-03 Commitfest Summary (Andres #1)

2018-04-02 Thread Bruce Momjian
On Sat, Mar 31, 2018 at 10:08:34AM +0200, Fabien COELHO wrote: > However, it helps explain the disagreements about pgbench features: pgbench > internal developer-oriented use for postgres is somehow limited, and a lot > of new features are suggested with an external performance benchmarking use > i

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Peter Geoghegan
On Mon, Apr 2, 2018 at 5:05 PM, Andres Freund wrote: >>Even accepting that (I personally go with surprising over violation, as >>if my vote counted), it is highly unlikely that we will convince every >>kernel team to declare "What fools we've been!" and push a change... >>and even if they did, Pos

Re: check_ssl_key_file_permissions should be in be-secure-common.c

2018-04-02 Thread Michael Paquier
On Mon, Apr 02, 2018 at 11:39:57AM -0400, Peter Eisentraut wrote: > committed Thanks for the commit, Peter. -- Michael signature.asc Description: PGP signature

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Christophe Pettus
> On Apr 2, 2018, at 17:05, Andres Freund wrote: > > Don't we pretty much already have agreement in that? And Craig is the main > proponent of it? For sure on the second sentence; the first was not clear to me. -- -- Christophe Pettus x...@thebuild.com

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Andres Freund
On April 2, 2018 5:03:39 PM PDT, Christophe Pettus wrote: > >> On Apr 2, 2018, at 16:27, Craig Ringer wrote: >> >> They're undocumented and extremely surprising semantics that are >arguably a violation of the POSIX spec for fsync(), or at least a >surprising interpretation of it. > >Even accep

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Christophe Pettus
> On Apr 2, 2018, at 16:27, Craig Ringer wrote: > > They're undocumented and extremely surprising semantics that are arguably a > violation of the POSIX spec for fsync(), or at least a surprising > interpretation of it. Even accepting that (I personally go with surprising over violation, as i

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Craig Ringer
On 3 April 2018 at 07:05, Anthony Iliopoulos wrote: > Hi Stephen, > > On Mon, Apr 02, 2018 at 04:58:08PM -0400, Stephen Frost wrote: > > > > fsync() doesn't reflect the status of given pages, however, it reflects > > the status of the file descriptor, and as such the file, on which it's > > calle

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Andres Freund
On 2018-04-03 01:05:44 +0200, Anthony Iliopoulos wrote: > Would using sync_file_range() be helpful? Potential errors would only > apply to pages that cover the requested file ranges. There are a few > caveats though: To quote sync_file_range(2): Warning This system call is extremely

Re: Creating streaming replication standby

2018-04-02 Thread Tatsuo Ishii
> A complain that I have about such concepts is that it creates a > dependency between utilities which are usually classified by > distributions as client-side (pg_basebackup goes to postgresql-client > for Debian), and the server itself. There is perhaps room for an > extension though. If many v

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Anthony Iliopoulos
Hi Stephen, On Mon, Apr 02, 2018 at 04:58:08PM -0400, Stephen Frost wrote: > > fsync() doesn't reflect the status of given pages, however, it reflects > the status of the file descriptor, and as such the file, on which it's > called. This notion that fsync() is actually only responsible for the >

Re: Disabling memory display in EXPLAIN ANALYZE

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 17:01:08 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > i.e. memory consumption differs between environments. Which isn't > > surprising. > > > > I wonder if we could disable the display with a separate switch or tie > > it to !'COSTS OFF && TIMING

Re: Rethinking -L switch handling and construction of LDFLAGS

2018-04-02 Thread Tom Lane
So here's a draft patch for this. It fixes the originally complained-of issue about --with-libxml breaking the build on Macs. I've not tested it very far beyond that plus a sanity cross-check on Linux, but I doubt there's any point in further manual testing; we might as well just throw it at the

Re: Add default role 'pg_access_server_files'

2018-04-02 Thread Stephen Frost
Michael, all, * Michael Paquier (mich...@paquier.xyz) wrote: > On Sun, Apr 01, 2018 at 09:39:02AM -0400, Stephen Frost wrote: > > Thanks for checking. Attached is an updated version which also includes > > the changes for adminpack, done in a similar manner to how pgstattuple > > was updated, as

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-02 Thread Christophe Pettus
> On Apr 2, 2018, at 11:50, Andres Freund wrote: > I'm not convinced. I think it's perfectly reasonable to want to truncate > away data on the live node, but maintain it on an archival node. +1. We've had at least one specific request for this, in maintaining a data warehouse from an OLTP syst

Re: Disabling memory display in EXPLAIN ANALYZE

2018-04-02 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > i.e. memory consumption differs between environments. Which isn't > surprising. > > I wonder if we could disable the display with a separate switch or tie > it to !'COSTS OFF && TIMING OFF' or such? Yeah, I agree with having these suppress

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Stephen Frost
Greetings, * Anthony Iliopoulos (ail...@altatus.com) wrote: > On Mon, Apr 02, 2018 at 12:32:45PM -0700, Andres Freund wrote: > > On 2018-04-02 20:53:20 +0200, Anthony Iliopoulos wrote: > > > On Mon, Apr 02, 2018 at 11:13:46AM -0700, Andres Freund wrote: > > > > Throwing away the dirty pages *and*

Re: BRIN FSM vacuuming questions

2018-04-02 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> I noticed that several places in brin_pageops.c do this sort of thing: >> ... >> ie they put *one* page into the index's free space map and then invoke a >> scan of the index's entire FSM to ensure that that info is bubbled up to >> the top immediately.

Disabling memory display in EXPLAIN ANALYZE

2018-04-02 Thread Andres Freund
Hi, The, now reverted, MERGE patch used EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF) on a few queries. That doesn't seem like a bad idea when it's interesting to display the plan and run a query without (interesting) results. Otherwise one has to duplicate the query. But right now it triggers differ

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Anthony Iliopoulos
On Mon, Apr 02, 2018 at 12:32:45PM -0700, Andres Freund wrote: > On 2018-04-02 20:53:20 +0200, Anthony Iliopoulos wrote: > > On Mon, Apr 02, 2018 at 11:13:46AM -0700, Andres Freund wrote: > > > Throwing away the dirty pages *and* persisting the error seems a lot > > > more reasonable. Then provide

Re: BRIN FSM vacuuming questions

2018-04-02 Thread Alvaro Herrera
Tom Lane wrote: > I noticed that several places in brin_pageops.c do this sort of thing: > > if (extended) > { > Assert(BlockNumberIsValid(newblk)); > RecordPageWithFreeSpace(idxrel, newblk, freespace); > FreeSpaceMapVacuum(idxrel); > } >

Re: BUG #15112: Unable to run pg_upgrade with earthdistance extension

2018-04-02 Thread Bruce Momjian
On Fri, Mar 30, 2018 at 04:52:29PM -0400, Bruce Momjian wrote: > On Wed, Mar 14, 2018 at 11:12:26PM +, PG Bug reporting form wrote: > > The following bug has been logged on the website: > > > > Bug reference: 15112 > > Logged by: Keiko Oda > > Email address: keiko...@gmail.c

fdw_startup/tuple_cost vs. parallel_startup/tuple_cost

2018-04-02 Thread Robert Haas
postgres_fdw has values called fdw_startup_cost and fdw_tuple_cost that you can adjust in order to control the perceived cost of starting a remote query and schlepping a tuple over the network. Parallel query, similarly, has parallel_startup_cost and parallel_tuple_cost. Today, I noticed that on a

Re: Passing current_database to BackgroundWorkerInitializeConnection

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 14:33:54 -0500, Jeremy Finzel wrote: > Hmmm... not sure if I follow. My goal is to run a SQL statement every 10 > seconds (or what value is chosen) in a particular database, using a > background worker. Those are the two arguments. Am I missing some way to > implement this apa

Re: Passing current_database to BackgroundWorkerInitializeConnection

2018-04-02 Thread Jeremy Finzel
On Mon, Apr 2, 2018 at 2:27 PM, Andres Freund wrote: > Hi, > > On 2018-04-02 14:24:53 -0500, Jeremy Finzel wrote: > > Thank you, this makes sense. However, how can this be done since I can > > only pass one argument to bgw_main? Is there any way to do this without > > having to store the value

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Andres Freund
On 2018-04-02 20:53:20 +0200, Anthony Iliopoulos wrote: > On Mon, Apr 02, 2018 at 11:13:46AM -0700, Andres Freund wrote: > > Throwing away the dirty pages *and* persisting the error seems a lot > > more reasonable. Then provide a fcntl (or whatever) extension that can > > clear the error status in

Re: Passing current_database to BackgroundWorkerInitializeConnection

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 14:24:53 -0500, Jeremy Finzel wrote: > Thank you, this makes sense. However, how can this be done since I can > only pass one argument to bgw_main? Is there any way to do this without > having to store the value in shared memory? No (I mean you can store it in the filesystem o

Re: Passing current_database to BackgroundWorkerInitializeConnection

2018-04-02 Thread Jeremy Finzel
On Fri, Mar 30, 2018 at 5:37 PM, Andres Freund wrote: > > > On March 30, 2018 3:16:31 PM PDT, Jeremy Finzel wrote: > >> What do you mean with "current database"? Before you > >> BackgroundWorkerInitializeConnection() there is no such thing? > > > > > >My module is based directly off the worker_s

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-02 Thread Alvaro Herrera
Why do we need AccessExclusiveLock on all children of a relation that we want to scan to search for rows not satisfying the constraint? I think it should be enough to get ShareLock, which prevents INSERT, no? I have a feeling I'm missing something here, but I don't know what, and all tests pass w

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Anthony Iliopoulos
On Mon, Apr 02, 2018 at 11:13:46AM -0700, Andres Freund wrote: > Hi, > > On 2018-04-01 03:14:46 +0200, Anthony Iliopoulos wrote: > > On Sat, Mar 31, 2018 at 12:38:12PM -0400, Tom Lane wrote: > > > Craig Ringer writes: > > > > So we should just use the big hammer here. > > > > > > And bitch, loudl

Re: Feature Request - DDL deployment with logical replication

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 23:07:17 +0800, Craig Ringer wrote: > We then lack any mechanism by which you can NACK, saying "I can't apply > this". Sure, but nothing forces this mechanism to be in-band. > So upstream will wait indefinitely. I guess we just expect the user to > intervene and ROLLBACK if th

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 07:39:57 +0100, Simon Riggs wrote: > On 1 April 2018 at 21:01, Andres Freund wrote: > > >> *** > >> *** 111,116 CREATE PUBLICATION >> class="parameter">name > >> --- 111,121 > >> and so the default value for this option is > >> 'in

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 14:30:50 -0400, Peter Eisentraut wrote: > On 4/1/18 16:01, Andres Freund wrote: > > On 2018-01-25 14:21:15 +0100, Marco Nenciarini wrote: > >> + if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA) > >> + { > >> + > >> + /* > >> + * Check foreign

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-02 Thread Simon Riggs
On 2 April 2018 at 19:30, Peter Eisentraut wrote: >>> *** >>> *** 111,116 CREATE PUBLICATION >> class="parameter">name >>> --- 111,121 >>> and so the default value for this option is >>> 'insert, update, delete'. >>> >>> + >>> +

Re: disable SSL compression?

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 12:46:25 -0400, Tom Lane wrote: > It seemed like the attack you described wasn't all that dependent on > whether the data is compressed or not: if you can see the size of the > server's reply to "select ... where account_number = x", you can pretty > well tell the difference betw

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-02 Thread Peter Eisentraut
On 4/1/18 16:01, Andres Freund wrote: > On 2018-01-25 14:21:15 +0100, Marco Nenciarini wrote: >> +if (SessionReplicationRole != SESSION_REPLICATION_ROLE_REPLICA) >> +{ >> + >> +/* >> + * Check foreign key references. In CASCADE mode, this should >> be >> +

Re: disable SSL compression?

2018-04-02 Thread Andres Freund
Hi, On 2018-04-02 10:25:04 -0400, Robert Haas wrote: > In general, I'd expect compressing data to be beneficial for the > security of encryption because it should increase the entropy of the > encrypted bytes, but obviously it's not hard to hypothesize cases > where the opposite is true for one re

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Andres Freund
Hi, On 2018-04-01 03:14:46 +0200, Anthony Iliopoulos wrote: > On Sat, Mar 31, 2018 at 12:38:12PM -0400, Tom Lane wrote: > > Craig Ringer writes: > > > So we should just use the big hammer here. > > > > And bitch, loudly and publicly, about how broken this kernel behavior is. > > If we make enough

Re: [HACKERS] logical decoding of two-phase transactions

2018-04-02 Thread Andres Freund
On 2018-04-02 09:23:10 +0100, Simon Riggs wrote: > On 29 March 2018 at 23:24, Andres Freund wrote: > > >> I agree with the former, of course - docs are a must. I disagree with > >> the latter, though - there have been about no proposals how to do it > >> without the locking. If there are, I'd lik

Re: disable SSL compression?

2018-04-02 Thread Garick Hamlin
On Mon, Apr 02, 2018 at 12:46:25PM -0400, Tom Lane wrote: > Peter Eisentraut writes: > > I agree the attack is less likely to be applicable in typical database > > installations. I think we should move forward with considering protocol > > compression proposals, but any final result should put a

Re: disable SSL compression?

2018-04-02 Thread Peter Eisentraut
On 4/2/18 12:46, Tom Lane wrote: > Peter Eisentraut writes: >> I agree the attack is less likely to be applicable in typical database >> installations. I think we should move forward with considering protocol >> compression proposals, but any final result should put a warning in the >> documentat

Re: tab complete for procedures for \sf and \ef commands

2018-04-02 Thread Peter Eisentraut
On 4/1/18 04:14, Pavel Stehule wrote: > Hi > > small bugfix patch committed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: disable SSL compression?

2018-04-02 Thread Tom Lane
Peter Eisentraut writes: > I agree the attack is less likely to be applicable in typical database > installations. I think we should move forward with considering protocol > compression proposals, but any final result should put a warning in the > documentation that using compression is potential

Re: new function for tsquery creartion

2018-04-02 Thread Dmitry Ivanov
I've fixed a bug and added some tests and documentation. -- Dmitry Ivanov Postgres Professional: http://www.postgrespro.com The Russian Postgres Companydiff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 5abb1c46fb..c3b7be6e4e 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-02 Thread Alvaro Herrera
> /* > - * Check whether default partition has a row that would fit the > partition > - * being attached. > + * Check if the default partition contains a row that would belong in > the > + * partition being attached. >*/ > - defaultPartOid = > - g

Re: disable SSL compression?

2018-04-02 Thread Peter Eisentraut
On 4/2/18 11:05, Robert Haas wrote: > Ah. Yeah, that makes sense. Although to use PG as a vector, it seems > like the attacker would need to the ability to snoop network traffic > between the application server and PG. If both of those are > presumably inside the bank's network and yet an attack

Re: check_ssl_key_file_permissions should be in be-secure-common.c

2018-04-02 Thread Peter Eisentraut
On 4/2/18 02:51, Michael Paquier wrote: > It seems to me that this routine should be logically put into > be-secure-common.c so as future SSL implementations can use it. This > makes the code more consistent with the frontend refactoring that has > happened in f75a959. I would not have bothered a

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-04-02 Thread Robert Haas
On Wed, Mar 28, 2018 at 2:12 PM, Andres Freund wrote: > How will it break it? They'll see an invalid ctid and conclude that the > tuple is dead? Without any changes that's already something that can > happen if a later tuple in the chain has been pruned away. Sure, that > code won't realize it sh

Re: Feature Request - DDL deployment with logical replication

2018-04-02 Thread Craig Ringer
On 1 April 2018 at 00:57, Andres Freund wrote: > On 2018-03-31 22:13:42 +0800, Craig Ringer wrote: > > We'll still need a mechanism to transport them to downstreams (like WAL > > messages) and to send responses upstream. For responses I think we will > > finally want to add a backchannel to the l

Re: disable SSL compression?

2018-04-02 Thread Robert Haas
On Mon, Apr 2, 2018 at 10:52 AM, Peter Eisentraut wrote: > On 4/2/18 10:25, Robert Haas wrote: >> As I understand it on a brief review of the Google search >> results^W^W^Wliterature, the reason that was done was to prevent >> things like the CRIME attack, which apparently involves Javascript >> r

Re: Change RangeVarGetRelidExtended() to take flags argument?

2018-04-02 Thread Bossart, Nathan
On 3/30/18, 7:09 PM, "Andres Freund" wrote: > Pushed. Thanks! Nathan

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Craig Ringer
On 2 April 2018 at 02:24, Thomas Munro wrote: > > Maybe my drive-by assessment of those kernel routines is wrong and > someone will correct me, but I'm starting to think you might be better > to assume the worst on all systems. Perhaps a GUC that defaults to > panicking, so that users on those

Re: [HACKERS] Partition-wise aggregation/grouping

2018-04-02 Thread Robert Haas
On Thu, Mar 29, 2018 at 9:02 AM, Jeevan Chalke wrote: > Yep, I see the risk. Committed 0001 last week and 0002 just now. I don't really see 0003 a a critical need. If somebody demonstrates that this saves a meaningful amount of planning time, we can consider that part for a future release. --

Re: disable SSL compression?

2018-04-02 Thread Peter Eisentraut
On 4/2/18 10:25, Robert Haas wrote: > As I understand it on a brief review of the Google search > results^W^W^Wliterature, the reason that was done was to prevent > things like the CRIME attack, which apparently involves Javascript > running in your browser from deducing information that it shouldn

Re: Rewriting the test of pg_upgrade as a TAP test - take two

2018-04-02 Thread Peter Eisentraut
On 3/21/18 21:49, Michael Paquier wrote: > On Thu, Mar 22, 2018 at 09:42:35AM +0800, Craig Ringer wrote: >> I'm super excited by the idea of multi-version support in TAP, if that's >> what you mean. >> >> Why? Because I use TAP heavily in extensions. Especially replication >> extensions. Which like

Re: disable SSL compression?

2018-04-02 Thread Robert Haas
On Wed, Mar 28, 2018 at 7:16 PM, Andres Freund wrote: > +analysis of whether that's safe to do from a cryptographic POV. There's a > reason compression has been disabled for just about all SSL/TLS libraries. As I understand it on a brief review of the Google search results^W^W^Wliterature, the r

Re: Foreign keys and partitioned tables

2018-04-02 Thread Peter Eisentraut
Comments on the code: @@ -7226,12 +7235,23 @@ ATAddForeignKeyConstraint(AlteredTableInfo *tab, Relation rel, * numbers) */ if (pkrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE) + { + /* fix recursion in ATExecValidateConstraint to enable this case */ + if (fkconstrai

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Anthony Iliopoulos
On Sat, Mar 31, 2018 at 12:38:12PM -0400, Tom Lane wrote: > Craig Ringer writes: > > So we should just use the big hammer here. > > And bitch, loudly and publicly, about how broken this kernel behavior is. > If we make enough of a stink maybe it'll get fixed. It is not likely to be fixed (beyond

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Anthony Iliopoulos
On Sun, Apr 01, 2018 at 12:13:09AM +0800, Craig Ringer wrote: >On 31 March 2018 at 21:24, Anthony Iliopoulos <[1]ail...@altatus.com> >wrote: > > On Fri, Mar 30, 2018 at 10:18:14AM +1300, Thomas Munro wrote: > > > >> Yeah, I see why you want to PANIC. > > > > > > Indeed

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-04-02 Thread Anthony Iliopoulos
On Fri, Mar 30, 2018 at 10:18:14AM +1300, Thomas Munro wrote: > >> Yeah, I see why you want to PANIC. > > > > Indeed. Even doing that leaves question marks about all the kernel > > versions before v4.13, which at this point is pretty much everything > > out there, not even detecting this reliably.

Re: [HACKERS] logical decoding of two-phase transactions

2018-04-02 Thread Nikhil Sontakke
> > Quick thought: Should be simple to release lock when interacting with network. I don’t think this will be that simple. The network calls will typically happen from inside the plugins and we don’t want to make plugin authors responsible for that. > Could also have abort signal lockers.

Re: hot_standby_feedback vs excludeVacuum and snapshots

2018-04-02 Thread Amit Kapila
On Sun, Apr 1, 2018 at 3:30 PM, Simon Riggs wrote: > On 31 March 2018 at 14:21, Amit Kapila wrote: >> >> I think the vacuum assigns xids only if it needs to truncate some of >> the pages in the relation which happens towards the end of vacuum. >> So, it shouldn't hold back the xmin horizon for lo

Re: [PATCH] Verify Checksums during Basebackups

2018-04-02 Thread Michael Banck
Hi! On Sun, Apr 01, 2018 at 05:27:21PM +0200, Magnus Hagander wrote: > It might be a micro-optimisation, but ISTM that we shouldn't do the > basename(palloc(fn)) in is_heap_file() unless we actually need it -- so not > at the top of the function. (And surely "." and ".." should not occur here? > I

Re: Foreign keys and partitioned tables

2018-04-02 Thread Peter Eisentraut
On 3/31/18 18:21, Alvaro Herrera wrote: > Yeah, I started by putting what I thought was going to be just ALTER > TABLE in that test, then moved to the other file and added what I > thought were more complete tests there and failed to move stuff to > alter_table. Honestly, I think these should most

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-02 Thread Etsuro Fujita
(2018/04/02 18:49), Amit Langote wrote: 2. If I understand the description you provided in [1] correctly, the point of having ri_PartitionIsValid and ExecInitExtraPartitionInfo() is to avoid possibly-redundantly performing following two steps in ExecSetupPartitionTupleRouting() for *reused* parti

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-02 Thread Etsuro Fujita
(2018/03/30 22:28), Alvaro Herrera wrote: Etsuro Fujita wrote: Now we have ON CONFLICT for partitioned tables, which requires the conversion map to be computed in ExecInitPartitionInfo, so I updated the patch so that we keep the former step in ExecInitPartitionInfo and ExecSetupPartitionTupleRo

Re: Diagonal storage model

2018-04-02 Thread Andrey Borodin
> 2 апр. 2018 г., в 16:57, Ashutosh Bapat > написал(а): > On Mon, Apr 2, 2018 at 3:49 AM, Marko Tiikkaja wrote: >> >> I'm a little worried about the fact that even with this model we're still >> limited to only two dimensions. That's bound to cause problems sooner or >> later. > How about a

Re: [HACKERS] path toward faster partition pruning

2018-04-02 Thread David Rowley
On 2 April 2018 at 17:18, Amit Langote wrote: > On 2018/03/31 0:55, David Rowley wrote: >> explain (analyze, costs off, summary off, timing off) execute q1 (1,2,2,0); >>QUERY PLAN >> >> Result (actual rows=0 loops=1) >>One-Time Filter: false >> (2

Re: Diagonal storage model

2018-04-02 Thread Ashutosh Bapat
On Mon, Apr 2, 2018 at 3:49 AM, Marko Tiikkaja wrote: > On Sun, Apr 1, 2018 at 3:48 PM, Konstantin Knizhnik > wrote: >> >> I want to announce new model, "diagonal storage" which combines benefits >> of both approaches. >> The idea is very simple: we first store column 1 of first record, then >> c

Re: json(b)_to_tsvector with numeric values

2018-04-02 Thread Dmitry Dolgov
> On 2 April 2018 at 11:27, Arthur Zakirov wrote: > On Mon, Apr 02, 2018 at 11:41:12AM +0300, Oleg Bartunov wrote: >> On Mon, Apr 2, 2018 at 9:45 AM, Arthur Zakirov >> wrote: >> I found this bug, when working on presentation about FTS and it looked >> annoying, since it validates >> the consiste

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-02 Thread Amit Langote
Fujita-san, Thanks for updating the patch. On 2018/03/30 21:56, Etsuro Fujita wrote: > I modified the patch to use the existing API ExecForeignInsert instead of > that API and removed that API including this doc. OK. >> 2. If I understand the description you provided in [1] correctly, the >> po

Re: [HACKERS] [PATCH] Lockable views

2018-04-02 Thread Yugo Nagata
On Thu, 29 Mar 2018 17:26:36 -0700 Andres Freund wrote: Thank you for your comments. I attach a patch to fix issues you've pointed out. > Hi, > > On 2018-03-28 20:26:48 +0900, Yugo Nagata wrote: > > diff --git a/doc/src/sgml/ref/lock.sgml b/doc/src/sgml/ref/lock.sgml > > index b2c7203..96d477c

Re: json(b)_to_tsvector with numeric values

2018-04-02 Thread Arthur Zakirov
On Mon, Apr 02, 2018 at 11:41:12AM +0300, Oleg Bartunov wrote: > On Mon, Apr 2, 2018 at 9:45 AM, Arthur Zakirov > wrote: > I found this bug, when working on presentation about FTS and it looked > annoying, since it validates > the consistency of FTS.I think this is a bug, which needs to be fixed,

make installcheck-world in a clean environment

2018-04-02 Thread Alexander Lakhin
Hello, I am trying to perform "make installcheck-world" after "make clean" (or after installing a precompiled binaries), but it fails. The error I get is related to ECPG regression test: make -C test installcheck make[2]: Entering directory `/home/postgres/postgres/src/interfaces/ecpg/test' gc

Re: json(b)_to_tsvector with numeric values

2018-04-02 Thread Oleg Bartunov
Postgres Professional: http://www.postgrespro.com The Russian Postgres Company On Mon, Apr 2, 2018 at 9:45 AM, Arthur Zakirov wrote: > Hello Dmitry, > > Dmitry Dolgov <9erthali...@gmail.com> wrote: >> >> Any opinions about this suggestion? Can it be considered as a bug fix and >> included into t

Re: [HACKERS] logical decoding of two-phase transactions

2018-04-02 Thread Simon Riggs
On 29 March 2018 at 23:30, Petr Jelinek wrote: > On 29/03/18 23:58, Andres Freund wrote: >> On 2018-03-29 23:52:18 +0200, Tomas Vondra wrote: I have added details about this in src/backend/storage/lmgr/README as suggested by you. >>> >>> Thanks. I think the README is a good start, b

Re: [HACKERS] logical decoding of two-phase transactions

2018-04-02 Thread Simon Riggs
On 29 March 2018 at 23:24, Andres Freund wrote: >> I agree with the former, of course - docs are a must. I disagree with >> the latter, though - there have been about no proposals how to do it >> without the locking. If there are, I'd like to hear about it. > > I don't care. Either another soluti