Re: pie-in-sky idea: 'sensitive' function parameters

On 3 February 2018 at 11:46, Tom Lane wrote: > Chapman Flack writes: > > ... which led me to the idea of a function parameter > > declaration, putting the function definer in control of what > > bits should get redacted. > > +1 for thinking outside the

Re: [HACKERS] proposal: schema variables

Hi 2018-02-03 1:48 GMT+01:00 David G. Johnston : > ​I've done a non-compilation documentation review, the diff from the poc > patch and the diff from master are attached. > > Comments are inter-twined in the patch in xml comment format; though I > reiterate (some of?)

Re: pie-in-sky idea: 'sensitive' function parameters

Chapman Flack writes: > ... which led me to the idea of a function parameter > declaration, putting the function definer in control of what > bits should get redacted. +1 for thinking outside the box, but ... > Would anyone else see some value in this capability? Could it

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Feb 2, 2018 at 4:31 PM, Peter Geoghegan wrote: > On Fri, Feb 2, 2018 at 1:58 PM, Andres Freund wrote: >> Not saying you're wrong, but you should include a comment on why this is >> a benign warning. Presumably it's some padding memory somewhere, but >>

Re: JIT compiling with LLVM v9.1

On Mon, Jan 29, 2018 at 1:53 AM, Andres Freund wrote: >> https://git.postgresql.org/git/users/andresfreund/postgres.git There's a patch in there to change the scan order. I suggest that you rename the GUC "synchronize_seqscans" to something more generic like

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

Tomas Vondra writes: > BTW wouldn't it be possible to derive "traditional" proof in relational > algebra, similarly to other transforms? Perhaps. The patch depends on CTID, but you could probably model that as a primary key in a proof.

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

On 02/02/2018 03:26 PM, Tom Lane wrote: > Tomas Vondra writes: >> ISTM this patch got somewhat stuck as we're not quite sure the >> transformation is correct in all cases. Is my impression correct? > > Yeah, that's the core issue. > >> If yes, how to we convince

pie-in-sky idea: 'sensitive' function parameters

CREATE FUNCTION commence_primary_ignition(target text, password text) RETURNS void AS ...; SELECT commence_primary_ignition(target=>'Alderaan', password=>'1234'); /* 1234 could appear in logs, activity stats ... disturbing */ CREATE OR REPLACE FUNCTION commence_primary_ignition( target

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Feb 2, 2018 at 1:58 PM, Andres Freund wrote: > Not saying you're wrong, but you should include a comment on why this is > a benign warning. Presumably it's some padding memory somewhere, but > it's not obvious from the above bleat. Sure. This looks slightly more

Re: RelOptInfo -> Relation

Robert Haas writes: > Is there some good reason why get_relation_info() copies all of the > data that will be needed later in planning into the RelOptInfo instead > of just storing a pointer to the Relation directly into the > RelOptInfo? Of course, if we did that, then it

Re: Query running for very long time (server hanged) with parallel append

Please forgive my inexperience with the codebase, but as the guy who reported this bugger: https://www.postgresql.org/message-id/flat/151724453314.1238.409882538067070269%40wrigleys.postgresql.org#151724453314.1238.409882538067070...@wrigleys.postgresql.org, I thought I'd follow your hints, as

Re: Boolean partitions syntax

Robert Haas writes: > On Fri, Feb 2, 2018 at 4:40 PM, Peter Eisentraut > wrote: >> There might be other options, but one way to solve this would be to >> treat partition bounds as a general expression in the grammar and then >> check in

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Thu, Jan 25, 2018 at 6:21 PM, Thomas Munro wrote: > On Fri, Jan 26, 2018 at 9:38 AM, Claudio Freire > wrote: >> I had the tests running in a loop all day long, and I cannot reproduce >> that variance. >> >> Can you share your steps to

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

On Fri, Jan 26, 2018 at 1:28 AM, Amit Kapila wrote: > So, this means that in case of logical replication, it won't generate > the error this patch is trying to introduce. I think if we want to > handle this we need some changes in WAL and logical decoding as well. > >

Re: Boolean partitions syntax

On Fri, Feb 2, 2018 at 4:40 PM, Peter Eisentraut wrote: > There might be other options, but one way to solve this would be to > treat partition bounds as a general expression in the grammar and then > check in post-parse analysis that it's a constant. Yeah --

Draft release notes for 10.2 et al

... are posted at https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=bf641d3376018c40860f26167a60febff5bc1f51 They should appear in the website's development docs in a couple of hours, too. Please comment before Sunday if you see anything that could be improved.

Re: Boolean partitions syntax

Peter Eisentraut writes: > There might be other options, but one way to solve this would be to > treat partition bounds as a general expression in the grammar and then > check in post-parse analysis that it's a constant. That's pretty much what I said upthread.

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On 2018-02-02 13:35:59 -0800, Peter Geoghegan wrote: > On Fri, Feb 2, 2018 at 10:38 AM, Peter Geoghegan wrote: > > On Fri, Feb 2, 2018 at 10:37 AM, Robert Haas wrote: > >> If you could keep an eye on the buildfarm and investigate anything > >> that breaks, I

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Feb 2, 2018 at 10:38 AM, Peter Geoghegan wrote: > On Fri, Feb 2, 2018 at 10:37 AM, Robert Haas wrote: >> If you could keep an eye on the buildfarm and investigate anything >> that breaks, I would appreciate it. > I can keep an eye on it throughout

Re: [HACKERS] path toward faster partition pruning

On Fri, Feb 2, 2018 at 9:33 AM, Robert Haas wrote: >> Updated set of patches attached (patches 0002 onward mostly unchanged, >> except I incorporated the delta patch posted by David upthread). > > Committed 0001. Thanks. Some preliminary thoughts... Regarding 0002, I

Re: ERROR: too many dynamic shared memory segments

Robert Haas writes: > On Fri, Feb 2, 2018 at 3:01 PM, Tom Lane wrote: >> Do you mind if I change that Assert to a run-time test? > Hrm, I guess I could have done something like > shm_toc_lookup(pcxt->toc, PARALLEL_KEY_ERROR_QUEUE, (pcxt->nworkers == >

Re: [HACKERS] Bug in to_timestamp().

> On 1 February 2018 at 11:24, Arthur Zakirov wrote: > > I fixed those messages, but in a different manner. I think that an > unexpected character is unknown and neither space nor separator. And > better to say that was expected space/separator character. Sounds good,

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Feb 2, 2018 at 3:35 PM, Peter Geoghegan wrote: > On Fri, Feb 2, 2018 at 12:30 PM, Robert Haas wrote: >> For the record, I typically construct the list of reviewers by reading >> over the thread and adding all the people whose names I find there in >>

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Feb 2, 2018 at 12:30 PM, Robert Haas wrote: > For the record, I typically construct the list of reviewers by reading > over the thread and adding all the people whose names I find there in > chronological order, excluding things that are clearly not review > (like

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Feb 2, 2018 at 3:23 PM, Peter Geoghegan wrote: > On Fri, Feb 2, 2018 at 10:38 AM, Peter Geoghegan wrote: >> Thanks everyone > > I would like to acknowledge the assistance of Corey Huinker with early > testing of the patch (this took place in 2016, and much of

Re: Built-in connection pooling

Konstantin>I do not have explanation of performance degradation in case of this particular workload. A) Mongo Java Client uses a connection-pool of 100 connections by default. That is it does not follow "connection per client" (in YCSB terms), but it is capped by 100 connections. I think it can

Re: ERROR: too many dynamic shared memory segments

On Fri, Feb 2, 2018 at 3:01 PM, Tom Lane wrote: > Robert Haas writes: >> That turned out to produce more than one problem. I find that the >> select_parallel test then fails like this: >> ERROR: could not find key 18446744073709486082 in shm TOC at

Re: ERROR: too many dynamic shared memory segments

Robert Haas writes: > That turned out to produce more than one problem. I find that the > select_parallel test then fails like this: > ERROR: could not find key 18446744073709486082 in shm TOC at 0x10be98040 > The fix for that problem seems to be: > /* Recreate

Re: [HACKERS] make async slave to wait for lsn to be replayed

On 2 February 2018 at 18:46, Robert Haas wrote: > On Fri, Feb 2, 2018 at 3:46 AM, Simon Riggs wrote: >> In PG11, I propose the following command, sticking mostly to Ants' >> syntax, and allowing to wait for multiple events before it returns. It >>

Re: [HACKERS] make async slave to wait for lsn to be replayed

On Fri, Feb 2, 2018 at 3:46 AM, Simon Riggs wrote: > In PG11, I propose the following command, sticking mostly to Ants' > syntax, and allowing to wait for multiple events before it returns. It > doesn't hold snapshot and will not get cancelled by Hot Standby. > > WAIT FOR

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Feb 2, 2018 at 10:37 AM, Robert Haas wrote: > And that patch you attached is also, now, committed. > > If you could keep an eye on the buildfarm and investigate anything > that breaks, I would appreciate it. Fantastic! I can keep an eye on it throughout the day.

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Feb 2, 2018 at 11:16 AM, Peter Geoghegan wrote: > Attached patch has these changes. And that patch you attached is also, now, committed. If you could keep an eye on the buildfarm and investigate anything that breaks, I would appreciate it. -- Robert Haas EnterpriseDB:

Re: Add RANGE with values and exclusions clauses to the Window Functions

On Fri, Feb 2, 2018 at 9:26 AM, Oliver Ford wrote: > On Thu, Feb 1, 2018 at 1:46 AM, David G. Johnston > wrote: > > > The three callers of WinGetFuncArgInFrame don't use the isout argument; > they > > probably need to read that and a new isexcluded

Re: Temporary tables prevent autovacuum, leading to XID wraparound

On Fri, Feb 2, 2018 at 1:27 AM, Masahiko Sawada wrote: > Thank you for suggestion. It sounds more smarter. So it would be more > better if we vacuums database for anti-wraparound in ascending order > of relfrozenxid? Currently, we're doing it based on datfrozenxid. I was

Re: Built-in connection pooling

On 01.02.2018 23:28, Vladimir Sitnikov wrote: > config/pgjsonb-local.dat Do you use standard "workload" configuration values? (e.g. recordcount=1000, maxscanlength=100) Yes, I used default value for workload. For example, workload-A has the following settings: # Yahoo! Cloud System

Re: [HACKERS] Can ICU be used for a database's default sort order?

On Fri, Feb 2, 2018 at 4:22 AM, Andrey Borodin wrote: > I can try to do this before next CF. But ISTM that EDB and Postgres Pro > already have various flavours of similar feature. Maybe they are planning to > publish that? I would definitely review that patch. -- Peter

Re: Wait for parallel workers to attach

On Fri, Feb 2, 2018 at 6:11 AM, Robert Haas wrote: >> Fair enough, you can proceed with the patch. > > Committed. Now, on to the main event! Thank you both. -- Peter Geoghegan

Re: [HACKERS] Bug in to_timestamp().

On Wed, Jan 31, 2018 at 11:53 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > Why not write `is_separator_char` using `isprint`, `isalpha`, `isdigit` from > ctype.h? Something like: > > return isprint(*str) && !isalpha(*str) && !isdigit(*str) > > From what I see in the source code they do

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

On Thu, Feb 1, 2018 at 7:21 PM, Simon Riggs wrote: > Yes, it would be about 99% of the time. > > But you have it backwards - we are not assuming that case. That is the > only case that has risk - the one where an old WAL record starts at > exactly the place the latest one

Re: Query running for very long time (server hanged) with parallel append

On Fri, Feb 2, 2018 at 1:43 AM, Amit Khandekar wrote: > On 2 February 2018 at 03:50, Thomas Munro > wrote: >> Whatever logic bug might be causing the query to hang, it's not good >> that we're unable to SIGINT/SIGTERM our way out of this

Re: [HACKERS] [PATCH] Lockable views

On Thu, Feb 1, 2018 at 8:09 PM, Tatsuo Ishii wrote: > Initially I thought all base tables including ones in a subquery also > should be locked like you. But after some discussions with Yugo, I > agree that locking table in a subquery is less valuable for users (and > I am

Prefix operator for text and spgist support

Hi, Attached patch introduces prefix operator ^@ for text type. For 'a ^@ b' it returns true if 'a' starts with 'b'. Also there is spgist index support for this operator. It could be useful as an alternative for LIKE for 'something%' templates. Or even used in LIKE queries instead of ~>=~ and

Re: [HACKERS] path toward faster partition pruning

On Fri, Feb 2, 2018 at 1:04 AM, Amit Langote wrote: > Your proposed cleanup sounds much better, so I implemented it in the > attached updated 0001, while dropping the previously proposed > PartitionBoundCmpArg approach. > > Updated set of patches attached (patches

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

Tomas Vondra writes: > ISTM this patch got somewhat stuck as we're not quite sure the > transformation is correct in all cases. Is my impression correct? Yeah, that's the core issue. > If yes, how to we convince ourselves? Would some sort of automated testing >

Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently

On Thu, Feb 1, 2018 at 9:34 PM, Masahiko Sawada wrote: > On Mon, Jan 29, 2018 at 11:31 PM, Claudio Freire > wrote: >> On Mon, Jan 29, 2018 at 4:12 AM, Masahiko Sawada >> wrote: >>> On Sat, Jul 29, 2017 at 9:42 AM, Claudio

Re: Wait for parallel workers to attach

On Thu, Feb 1, 2018 at 9:48 PM, Amit Kapila wrote: > On Thu, Feb 1, 2018 at 9:09 PM, Robert Haas wrote: >> On Wed, Jan 31, 2018 at 10:08 PM, Amit Kapila >> wrote: >>> I think suggesting to use this API to wait "for a

Re: [HACKERS] Partition-wise aggregation/grouping

On Fri, Feb 2, 2018 at 8:25 AM, Jeevan Chalke wrote: >> The problem is that create_partition_agg_paths() is doing *exactly* >> same thing that add_paths_to_grouping_rel() is already doing inside >> the blocks that say if (grouped_rel->partial_pathlist). We don't

Re: [HACKERS] GnuTLS support

On Thu, Feb 1, 2018 at 5:08 AM, Christoph Berg wrote: > Re: Peter Eisentraut 2018-01-03 > <99680dba-cf63-8151-1de2-46ca93897...@2ndquadrant.com> >> One scenario is that if GnuTLS goes in, it's quite plausible that the >> PG11 packages for Debian and Ubuntu will use it by

Re: [HACKERS] Partition-wise aggregation/grouping

On Fri, Feb 2, 2018 at 1:41 AM, Robert Haas wrote: > On Thu, Feb 1, 2018 at 8:59 AM, Jeevan Chalke > wrote: > > I wrote a patch for this (on current HEAD) and attached separately here. > > Please have a look. > > Yes, this is approximately

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

On 01/04/2018 11:50 PM, Tom Lane wrote: > I wrote: >> Jim Nasby writes: >>> I've verified that the patch still applies and make check-world is clean. > >> Not any more :-(. Here's a v3 rebased over HEAD. No substantive >> change from v2. > > Again rebased up to HEAD;

Re: [HACKERS] Can ICU be used for a database's default sort order?

Hello! > 1 февр. 2018 г., в 19:09, Peter Eisentraut > написал(а): > > On 1/31/18 11:48, Vladimir Borodin wrote: >> Will it work only for a particular database? Or for a whole cluster >> during initdb also? Any chance to get this done in 11? > > I'm currently

Re: [HACKERS] Add support for tuple routing to foreign partitions

(2018/01/25 23:33), Stephen Frost wrote: I'm afraid a good bit of this patch is now failing to apply. I don't have much else to say except to echo the performance concern that Amit Langote raised about expanding the inheritence tree twice. To address that concern, I'm thinking to redesign the

Re: [HACKERS] Adding column_constraint description in ALTER TABLE synopsis

Greetings, * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > On 2018/01/23 8:57, Thomas Munro wrote: > > On Tue, Jan 23, 2018 at 12:41 PM, Thomas Munro > > wrote: > >> On Mon, Jan 15, 2018 at 2:32 PM, Stephen Frost wrote: > >>> If someone

Re: [PoC PATCH] Parallel dump to /dev/null

Hi. > 2 февр. 2018 г., в 13:25, Andrey Borodin написал(а): > >> Do folks here think that speedups would be worth it? > I've seen backup verification via COPY TO to /dev/null in production. These > copies were parallelized on the side of verification script. > Not sure

Re: [HACKERS] Add support for tuple routing to foreign partitions

(2018/02/02 19:33), Etsuro Fujita wrote: (2018/01/25 23:33), Stephen Frost wrote: I'm afraid a good bit of this patch is now failing to apply. I don't have much else to say except to echo the performance concern that Amit Langote raised about expanding the inheritence tree twice. To address

Re: JIT compiling with LLVM v9.1

On Friday, February 2, 2018 10:48:16 AM CET Pierre Ducroquet wrote: > On Monday, January 29, 2018 10:53:50 AM CET Andres Freund wrote: > > Hi, > > > > On 2018-01-23 23:20:38 -0800, Andres Freund wrote: > > > == Code == > > > > > > As the patchset is large (500kb) and I'm still quickly evolving

Re: non-bulk inserts and tuple routing

(2018/01/30 18:52), Etsuro Fujita wrote: (2018/01/30 18:39), Amit Langote wrote: Will wait for your comments before sending a new version then. Ok, I'll post my comments as soon as possible. * ExecInitPartitionResultRelInfo is called from ExecFindPartition, but we could call that another

Re: JIT compiling with LLVM v9.0

Hi, On 2018-02-02 18:22:34 +1300, Thomas Munro wrote: > The clang that was used for bitcode was the system /usr/bin/clang, > version 4.0. Is it a problem that I used that for compiling the > bitcode, but LLVM5 for JIT? I actually tried > CLANG=/usr/local/llvm50/bin/clang but ran into weird

Re: [HACKERS] path toward faster partition pruning

Hi David. On 2018/02/01 8:57, David Rowley wrote: > On 31 January 2018 at 21:03, Amit Langote > wrote: >> Update patch set attached. Thanks again. > > (My apologies for being slow to respond here. I've been on leave this > week and I'm off again next week. I now

Re: JIT compiling with LLVM v9.1

On Monday, January 29, 2018 10:53:50 AM CET Andres Freund wrote: > Hi, > > On 2018-01-23 23:20:38 -0800, Andres Freund wrote: > > == Code == > > > > As the patchset is large (500kb) and I'm still quickly evolving it, I do > > not yet want to attach it. The git tree is at > > > >

Re: [PoC PATCH] Parallel dump to /dev/null

Hi, Am Freitag, den 02.02.2018, 13:22 +0900 schrieb Michael Paquier: > On Thu, Feb 01, 2018 at 02:24:46PM +0100, Michael Banck wrote: > > dumping a database to /dev/null via pg_dump is (AFAIK) one recommended > > way to check for corruption. However, dumping to /dev/null is currently > > not

Re: Boolean partitions syntax

On 2018/01/29 14:57, Tom Lane wrote: > Amit Langote writes: >> Partition bound literals as captured gram.y don't have any type >> information attached. > > Isn't that design broken by definition? TRUE is not the same thing > as 't', nor as 'true'. Nor are 1 and

Re: [HACKERS] make async slave to wait for lsn to be replayed

Greetings, * Simon Riggs (si...@2ndquadrant.com) wrote: > On 22 January 2018 at 23:21, Stephen Frost wrote: > > >> It sounds reasonable. I can offer the following version. > >> > >> WAIT LSN lsn_number; > >> WAIT LSN lsn_number TIMEOUT delay; > >> WAIT LSN lsn_number

Re: JIT compiling with LLVM v9.0

On 2018-02-01 22:20:01 -0800, Jeff Davis wrote: > Thanks! That worked, but I had to remove the "-stdlib=libc++" also, > which was causing me problems. That'll be gone as soon as I finish the shlib thing. Will hope to have something over the weekend. Right now I'm at FOSDEM and need to prepare a

Re: JIT compiling with LLVM v9.0

Hi, On 2018-02-02 18:22:34 +1300, Thomas Munro wrote: > Is there something broken about my installation? I see simple > arithmetic expressions apparently compiling and working but I can > easily find stuff that breaks... so far I think it's anything > involving string literals: That definitely

Re: [HACKERS] Surjective functional indexes

On 1 February 2018 at 09:32, Simon Riggs wrote: > OK, thanks. Just checking my understanding and will add to the > comments in the patch. I'm feeling good about ths now, but for personal reasons won't be committing this until late Feb/early March. -- Simon Riggs

Re: [HACKERS] make async slave to wait for lsn to be replayed

On 30 October 2017 at 17:25, Ivan Kartyshov wrote: > It sounds reasonable. I can offer the following version. > > WAIT LSN lsn_number; > WAIT LSN lsn_number TIMEOUT delay; > WAIT LSN lsn_number INFINITE; > WAIT LSN lsn_number NOWAIT; > > > WAIT [token]

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

Hi Amit, Sorry for the delayed response. On Fri, Jan 26, 2018 at 11:58 AM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 12:44 PM, amul sul wrote: >> On Tue, Jan 23, 2018 at 7:01 PM, Amit Kapila wrote: >>> On Fri, Jan 12,