Misleading comment about single_copy, and some bikeshedding

2019-06-23 Thread Thomas Munro
Hi, /* * GatherMergePath runs several copies of a plan in parallel and collects * the results, preserving their common sort order. For gather merge, the * parallel leader always executes the plan too, so we don't need single_copy. */ typedef struct GatherMergePath The second sentence is not

Re: using explicit_bzero

2019-06-23 Thread Michael Paquier
On Sun, Jun 23, 2019 at 09:56:40PM +0200, Peter Eisentraut wrote: > On 2019-06-21 15:45, Dagfinn Ilmari Mannsåker wrote: >> Also, on Linux it requires libbsd: https://libbsd.freedesktop.org/ > > No, it's in glibc. From man: explicit_bzero() first appeared in glibc 2.25. -- Michael

Re: using explicit_bzero

2019-06-23 Thread Michael Paquier
On Sun, Jun 23, 2019 at 09:57:18PM +0200, Peter Eisentraut wrote: > On 2019-06-23 21:55, Peter Eisentraut wrote: >> On 2019-06-21 15:25, Tom Lane wrote: >>> Peter Eisentraut writes: +#ifndef HAVE_EXPLICIT_BZERO +#define explicit_bzero(b, len) bzero(b, len) +#endif >>> >>> This

Re: check_recovery_target_lsn() does a PG_CATCH without a throw

2019-06-23 Thread Michael Paquier
On Sun, Jun 23, 2019 at 07:21:02PM +0200, Peter Eisentraut wrote: > Updated patch for that. I have been looking at this patch set. Patch 0001 looks good to me. You are removing all traces of a set of timestamp keywords not supported anymore, and no objections from my side for this cleanup.

Re: Problem with default partition pruning

2019-06-23 Thread Alvaro Herrera
On 2019-Jun-24, shawn wang wrote: Hello, > Thank you for your reply. > You can see that the mail start time is February 22. So I looked at the > latest version at that time. I found that v11.2 was the newest branch at > the time. So I tried to merge this patch into the code, and I found that >

Re: Optimize partial TOAST decompression

2019-06-23 Thread Binguo Bao
> This is not correct: L bytes of compressed data do not always can be decoded into at least L bytes of data. At worst we have one control byte per 8 bytes of literal bytes. This means at most we need (L*9 + 8) / 8 bytes with current pglz format. Good catch! I've corrected the related code in the

Re: Plugging some testing holes

2019-06-23 Thread Michael Paquier
On Sun, Jun 23, 2019 at 06:15:06PM -0400, Andrew Dunstan wrote: > Alvaro pointed out to me recently that the buildfarm client doesn't have > any provision for running module tests like commit_ts and > snapshot_too_old that use NO_INSTALLCHECK. On looking into this a bit > more, I noticed that we

Re: Fix doc bug in logical replication.

2019-06-23 Thread Robert Treat
On Sun, Jun 23, 2019 at 1:25 PM Peter Eisentraut wrote: > > On 2019-04-12 19:52, Robert Treat wrote: > > It is clear to me that the docs are wrong, but I don't see anything > > inherently incorrect about the code itself. Do you have suggestions > > for how you would like to see the code comments

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-06-23 Thread Masahiko Sawada
On Thu, Jun 20, 2019 at 10:46 PM Joe Conway wrote: > > On 6/20/19 8:34 AM, Masahiko Sawada wrote: > > I think even if we provide the per table encryption we can have > > encryption keys per tablepaces. That is, all tables on the same > > tablespace encryption use the same encryption keys but user

Re: Problem with default partition pruning

2019-06-23 Thread shawn wang
Hi Alvaro, Thank you for your reply. You can see that the mail start time is February 22. So I looked at the latest version at that time. I found that v11.2 was the newest branch at the time. So I tried to merge this patch into the code, and I found that everything worked. So I tested on this

Re: Choosing values for multivariate MCV lists

2019-06-23 Thread Tomas Vondra
On Mon, Jun 24, 2019 at 01:42:32AM +0200, Tomas Vondra wrote: On Sun, Jun 23, 2019 at 10:23:19PM +0200, Tomas Vondra wrote: On Sun, Jun 23, 2019 at 08:48:26PM +0100, Dean Rasheed wrote: On Sat, 22 Jun 2019 at 15:10, Tomas Vondra wrote: One annoying thing I noticed is that the base_frequency

Re: Choosing values for multivariate MCV lists

2019-06-23 Thread Tomas Vondra
On Sat, Jun 22, 2019 at 04:10:52PM +0200, Tomas Vondra wrote: On Fri, Jun 21, 2019 at 08:50:33AM +0100, Dean Rasheed wrote: On Thu, 20 Jun 2019 at 23:35, Tomas Vondra wrote: On Thu, Jun 20, 2019 at 06:55:41AM +0100, Dean Rasheed wrote: I'm not sure it's easy to justify ordering by

Re: Choosing values for multivariate MCV lists

2019-06-23 Thread Tomas Vondra
On Sun, Jun 23, 2019 at 10:23:19PM +0200, Tomas Vondra wrote: On Sun, Jun 23, 2019 at 08:48:26PM +0100, Dean Rasheed wrote: On Sat, 22 Jun 2019 at 15:10, Tomas Vondra wrote: One annoying thing I noticed is that the base_frequency tends to end up being 0, most likely due to getting too small.

Plugging some testing holes

2019-06-23 Thread Andrew Dunstan
Alvaro pointed out to me recently that the buildfarm client doesn't have any provision for running module tests like commit_ts and snapshot_too_old that use NO_INSTALLCHECK. On looking into this a bit more, I noticed that we also don't run any TAP tests in src/test/modules. I'm adding some code

Re: Choosing values for multivariate MCV lists

2019-06-23 Thread Tomas Vondra
On Sun, Jun 23, 2019 at 08:48:26PM +0100, Dean Rasheed wrote: On Sat, 22 Jun 2019 at 15:10, Tomas Vondra wrote: One annoying thing I noticed is that the base_frequency tends to end up being 0, most likely due to getting too small. It's a bit strange, though, because with statistic target set

Re: unlogged sequences

2019-06-23 Thread Peter Eisentraut
On 2019-06-21 07:31, Michael Paquier wrote: > 1) Some SQL queries: > create unlogged sequence popo; > alter sequence popo increment 2; The problem is that the above command does a relation rewrite but the code doesn't know to copy the init fork of the sequence. That will need to be addressed. >

Re: Multivariate MCV stats can leak data to unprivileged users

2019-06-23 Thread Tomas Vondra
On Sun, Jun 23, 2019 at 06:56:53PM +0100, Dean Rasheed wrote: On Mon, 13 May 2019 at 23:36, Tomas Vondra wrote: On Fri, May 10, 2019 at 10:19:44AM +0100, Dean Rasheed wrote: >While working on 1aebfbea83c, I noticed that the new multivariate MCV >stats feature suffers from the same problem,

Re: using explicit_bzero

2019-06-23 Thread Peter Eisentraut
On 2019-06-23 21:55, Peter Eisentraut wrote: > On 2019-06-21 15:25, Tom Lane wrote: >> Peter Eisentraut writes: >>> +#ifndef HAVE_EXPLICIT_BZERO >>> +#define explicit_bzero(b, len) bzero(b, len) >>> +#endif >> >> This presumes that every platform has bzero, which is unsafe (POSIX >> doesn't

Re: using explicit_bzero

2019-06-23 Thread Peter Eisentraut
On 2019-06-21 15:45, Dagfinn Ilmari Mannsåker wrote: > Also, on Linux it requires libbsd: https://libbsd.freedesktop.org/ No, it's in glibc. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Choosing values for multivariate MCV lists

2019-06-23 Thread Dean Rasheed
On Sat, 22 Jun 2019 at 15:10, Tomas Vondra wrote: > One annoying thing I noticed is that the base_frequency tends to end up > being 0, most likely due to getting too small. It's a bit strange, though, > because with statistic target set to 10k the smallest frequency for a > single column is

Re: how to run encoding-dependent tests by default

2019-06-23 Thread Peter Eisentraut
On 2019-06-17 18:39, Andres Freund wrote: > Basically something like: > > \gset SELECT my_encodings_are_compatible() AS compatible > \if :compatible > test; > contents; > \endif Cool, that works out quite well. See attached patch. I flipped the logic around to make it \quit if not compatible.

Misleading comment in tuplesort_set_bound

2019-06-23 Thread James Coleman
While digging into the incremental sort patch, I noticed in tuplesort.c at the beginning of the function in $SUBJECT we have this comment and assertion: tuplesort_set_bound(Tuplesortstate *state, int64 bound) { /* Assert we're called before loading any tuples */ Assert(state->status ==

Re: Add CREATE DATABASE LOCALE option

2019-06-23 Thread Peter Eisentraut
On 2019-06-06 21:52, Alvaro Herrera wrote: > On 2019-Jun-06, Fabrízio de Royes Mello wrote: > Cool... would be nice also add some test cases. >>> >>> Right. Any suggestions where to put them? >> >> Hmm... good question... I thought we already have some regression tests for >> {CREATE|DROP}

Re: Multivariate MCV stats can leak data to unprivileged users

2019-06-23 Thread Dean Rasheed
On Mon, 13 May 2019 at 23:36, Tomas Vondra wrote: > > On Fri, May 10, 2019 at 10:19:44AM +0100, Dean Rasheed wrote: > >While working on 1aebfbea83c, I noticed that the new multivariate MCV > >stats feature suffers from the same problem, and also the original > >problems that were fixed in

Re: Fix doc bug in logical replication.

2019-06-23 Thread Peter Eisentraut
On 2019-04-12 19:52, Robert Treat wrote: > It is clear to me that the docs are wrong, but I don't see anything > inherently incorrect about the code itself. Do you have suggestions > for how you would like to see the code comments improved? The question is perhaps whether we want to document that

Re: check_recovery_target_lsn() does a PG_CATCH without a throw

2019-06-23 Thread Peter Eisentraut
On 2019-06-20 18:05, Andres Freund wrote: > Hi, > > On 2019-06-20 15:42:14 +0200, Peter Eisentraut wrote: >> On 2019-06-12 13:16, Peter Eisentraut wrote: >>> I haven't figured out the time zone issue yet, but I guess the solution >>> might involve moving some of the code from

Re: make \d pg_toast.foo show its indices ; and, \d toast show its main table ; and \d relkind=I show its partitions (and tablespace)

2019-06-23 Thread Justin Pryzby
I'm continuing this thread with an additional change to slash dee for partitioned indexes. postgres=# \d ttz_i_idx Partitioned index "public.ttz_i_idx" Column | Type | Key? | Definition +-+--+ i | integer | yes | i btree, for table "public.ttz" Number

Re: New vacuum option to do only freezing

2019-06-23 Thread Michael Paquier
On Thu, Jun 20, 2019 at 03:50:32PM +0900, Michael Paquier wrote: > And I finish with the attached. Thoughts? So, are there any objections with this patch? Or do people think that it's too late for v12 and that it is better to wait until v13 opens for business? -- Michael signature.asc

Refactoring base64 encoding and decoding into a safer interface

2019-06-23 Thread Michael Paquier
Hi all, After the issues behind CVE-2019-10164, it seems that we can do much better with the current interface of decoding and encoding functions for base64 in src/common/. The root issue is that the callers of pg_b64_decode() and pg_b64_encode() provide a buffer where the result gets stored

Re: Index Skip Scan

2019-06-23 Thread Tomas Vondra
Hi, I've done some initial review on v20 - just reading through the code, no tests at this point. Here are my comments: 1) config.sgml I'm not sure why the enable_indexskipscan section says This parameter requires that enable_indexonlyscan is on. I suppose it's the same thing as for

Re: Code comment change

2019-06-23 Thread Vik Fearing
On 23/06/2019 12:35, Thomas Munro wrote: > On Sun, Jun 23, 2019 at 9:21 PM Vik Fearing > wrote: >> There is some language in a code comment that has been bothering me for >> several years now. After pointing it out in a conversation off-list >> recently, I figured it was past time to do

Re: Index Skip Scan

2019-06-23 Thread Floris Van Nee
> Try the attached patch -- it has DEBUG1 traces with the contents of > index tuples from key points during index scans, allowing you to see > what's going on both as a B-Tree is descended, and as a range scan is > performed. It also shows details of the insertion scankey that is set > up within

Re: Code comment change

2019-06-23 Thread Thomas Munro
On Sun, Jun 23, 2019 at 9:21 PM Vik Fearing wrote: > There is some language in a code comment that has been bothering me for > several years now. After pointing it out in a conversation off-list > recently, I figured it was past time to do something about it. > > Patch attached. Pushed.

Re: Optimize partial TOAST decompression

2019-06-23 Thread Andrey Borodin
Hi, Binguo! > 2 июня 2019 г., в 19:48, Binguo Bao написал(а): > > Hi, hackers! > This seems to have a 10x improvement. If the number of toast data chunks is > more, I believe that patch can play a greater role, there are about 200 > related TOAST data chunks for each entry in the case.

Code comment change

2019-06-23 Thread Vik Fearing
There is some language in a code comment that has been bothering me for several years now. After pointing it out in a conversation off-list recently, I figured it was past time to do something about it. Patch attached. -- Vik Fearing +33 6 46 75 15 36