Re: proposal - patch: psql - sort_by_size

2019-06-29 Thread Fabien COELHO
Hello Pavel, \set SORT_BY_SIZE on \dt -- sorted by schema, name (size is not calculated and is not visible) \dt+ -- sorted by size Patch applies cleanly, compiles, runs. "make check" ok. doc build ok. There are no tests. Some infrastructure should be in place so that such features can be t

Re: Commitfest 2019-07, the first of five* for PostgreSQL 13

2019-06-29 Thread Thomas Munro
On Sat, Jun 29, 2019 at 9:47 AM David Steele wrote: > On 6/28/19 1:15 PM, Tom Lane wrote: > > Stephen Frost writes: > >> sh, don't look now, but there might be a "Resend email" button in > >> the archives now that you can click to have an email sent to you... > > > > Oooh, lovely. > > > >> (t

Re: proposal - patch: psql - sort_by_size

2019-06-29 Thread Pavel Stehule
so 29. 6. 2019 v 9:32 odesílatel Fabien COELHO napsal: > > Hello Pavel, > > > \set SORT_BY_SIZE on > > \dt -- sorted by schema, name (size is not calculated and is not visible) > > \dt+ -- sorted by size > > Patch applies cleanly, compiles, runs. "make check" ok. doc build ok. > > There are no te

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Julien Rouhaud
On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov wrote:> > On 29.06.2019 1:23, Julien Rouhaud wrote: > > But that kinda resembles stuff we already have - selectivity/cost. So > why shouldn't this be considered as part of costing? > > Yeah, I'm not entirely convinced that we need anything new here.

Re: Commitfest 2019-07, the first of five* for PostgreSQL 13

2019-06-29 Thread Magnus Hagander
On Sat, Jun 29, 2019 at 10:05 AM Thomas Munro wrote: > On Sat, Jun 29, 2019 at 9:47 AM David Steele wrote: > > On 6/28/19 1:15 PM, Tom Lane wrote: > > > Stephen Frost writes: > > >> sh, don't look now, but there might be a "Resend email" button in > > >> the archives now that you can click

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Tomas Vondra
On Sat, Jun 29, 2019 at 11:10:03AM +0200, Julien Rouhaud wrote: On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov wrote:> On 29.06.2019 1:23, Julien Rouhaud wrote: But that kinda resembles stuff we already have - selectivity/cost. So why shouldn't this be considered as part of costing? Yeah, I'

Re: Multivariate MCV list vs. statistics target

2019-06-29 Thread Tomas Vondra
On Fri, Jun 21, 2019 at 08:09:18AM +0100, Dean Rasheed wrote: On Thu, 20 Jun 2019 at 23:12, Tomas Vondra wrote: On Thu, Jun 20, 2019 at 08:08:44AM +0100, Dean Rasheed wrote: >On Tue, 18 Jun 2019 at 22:34, Tomas Vondra wrote: >> >> So I'm thinking we should allow tweaking the statistics for e

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Julien Rouhaud
On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra wrote: > > On Sat, Jun 29, 2019 at 11:10:03AM +0200, Julien Rouhaud wrote: > >On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov > >> -- patched > >> EXPLAIN ANALYZE SELECT * FROM test WHERE t LIKE '%1234%' AND t LIKE '%1%'; > >>

Re: Choosing values for multivariate MCV lists

2019-06-29 Thread Tomas Vondra
On Tue, Jun 25, 2019 at 11:18:19AM +0200, Tomas Vondra wrote: On Mon, Jun 24, 2019 at 02:54:01PM +0100, Dean Rasheed wrote: On Mon, 24 Jun 2019 at 00:42, Tomas Vondra wrote: On Sun, Jun 23, 2019 at 10:23:19PM +0200, Tomas Vondra wrote: On Sun, Jun 23, 2019 at 08:48:26PM +0100, Dean Rasheed w

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Tomas Vondra
On Sat, Jun 29, 2019 at 02:50:51PM +0200, Julien Rouhaud wrote: On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra wrote: On Sat, Jun 29, 2019 at 11:10:03AM +0200, Julien Rouhaud wrote: >On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov >> -- patched >> EXPLAIN ANALYZE SELECT * FROM test WHERE t LIKE

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Julien Rouhaud
On Sat, Jun 29, 2019 at 3:11 PM Tomas Vondra wrote: > > On Sat, Jun 29, 2019 at 02:50:51PM +0200, Julien Rouhaud wrote: > >On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra > >> A related issue is that during costing is too late to modify cardinality > >> estimates, so the 'Bitmap Index Scan' will be

Re: Optimize partial TOAST decompression

2019-06-29 Thread Andrey Borodin
Hi! Please, do not use top-posting, i.e. reply style where you quote whole message under your response. It makes reading of archives terse. > 24 июня 2019 г., в 7:53, Binguo Bao написал(а): > >> This is not correct: L bytes of compressed data do not always can be decoded >> into at least L byt

Re: TM format can mix encodings in to_char()

2019-06-29 Thread Tom Lane
Juanjo Santamaria Flecha writes: > It looks as if no work is left for this patch, so maybe updating the author > to Tom Lane (I'm just a repoter at this point, which it's fine) and the > status to ready for committer would better reflect its current status. Does > anyone think otherwise? Yeah,

Re: mcvstats serialization code is still shy of a load

2019-06-29 Thread Tomas Vondra
On Thu, Jun 27, 2019 at 01:26:32PM +0200, Tomas Vondra wrote: On Thu, Jun 27, 2019 at 12:04:30AM -0400, Tom Lane wrote: Tomas Vondra writes: OK. Attached is a patch ditching the alignment in serialized data. I've ditched the macros to access parts of serialized data, and everything gets copied

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Tomas Vondra
On Sat, Jun 29, 2019 at 03:28:11PM +0200, Julien Rouhaud wrote: On Sat, Jun 29, 2019 at 3:11 PM Tomas Vondra wrote: On Sat, Jun 29, 2019 at 02:50:51PM +0200, Julien Rouhaud wrote: >On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra >> A related issue is that during costing is too late to modify car

Re: Fix runtime errors from -fsanitize=undefined

2019-06-29 Thread didier
Hi, I tested this patch with clang 7 on master. - On unpatched master I can't reproduce errors with make check-world in: src/backend/access/heap/heapam.c src/backend/utils/cache/relcache.c (IIRC I triggered this one in a pg previous version) src/backend/utils/misc/guc.c - I have a hard to reprodu

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Alexander Korotkov
Hi! On Sat, Jun 29, 2019 at 1:52 AM Nikita Glukhov wrote: > We have a similar solution for this problem. The idea is to avoid full index > scan inside GIN itself when we have some GIN entries, and forcibly recheck > all tuples if triconsistent() returns GIN_MAYBE for the keys that emitted no > G

Re: [HACKERS] Regression tests vs existing users in an installation

2019-06-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> We could make the new subdirectory be something specific like >> "src/test/modules/test_rolenames", but I think very likely we'll be >> wanting some additional test scripts that we likewise deem unsafe to >> run during "installcheck

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Alexander Korotkov
On Sat, Jun 29, 2019 at 1:25 PM Tomas Vondra wrote: > A related issue is that during costing is too late to modify cardinality > estimates, so the 'Bitmap Index Scan' will be expected to return fewer > rows than it actually returns (after ignoring the full-scan quals). > Ignoring redundant quals (

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Alexander Korotkov
On Sat, Jun 29, 2019 at 3:51 PM Julien Rouhaud wrote: > On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra > wrote: > > > > On Sat, Jun 29, 2019 at 11:10:03AM +0200, Julien Rouhaud wrote: > > >On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov > > >> -- patched > > >> EXPLAIN ANALYZE SELECT * FROM test W

base backup client as auxiliary backend process

2019-06-29 Thread Peter Eisentraut
Setting up a standby instance is still quite complicated. You need to run pg_basebackup with all the right options. You need to make sure pg_basebackup has the right permissions for the target directories. The created instance has to be integrated into the operating system's start scripts. Ther

Increasing default value for effective_io_concurrency?

2019-06-29 Thread Tomas Vondra
Hi, I think we should consider changing the effective_io_concurrency default value, i.e. the guc that determines how many pages we try to prefetch in a couple of places (the most important being Bitmap Heap Scan). The default is 1 since forever, but from my experience hardly the right value, no

Where is SSPI auth username determined for TAP tests?

2019-06-29 Thread Tom Lane
bowerbird is failing the pg_dump regression tests with a lot of FATAL: SSPI authentication failed for user "regress_postgres" I think this is likely a consequence of ca129e58c0 having modified 010_dump_connstr.pl to use "regress_postgres" not "postgres" as the bootstrap superuser name in the sou

Re: UCT (Re: pgsql: Update time zone data files to tzdata release 2019a.)

2019-06-29 Thread Robert Haas
On Thu, Jun 27, 2019 at 1:58 PM Tom Lane wrote: > It's not really clear to me that the IANA folk intend those files to > be read as a list of preferred zone names. If they do, what are we > to make of the fact that no variant of "UTC" appears in them? I think their intent is key. We can't make

Re: Usage of epoch in txid_current

2019-06-29 Thread Robert Haas
On Mon, Jun 24, 2019 at 1:43 PM Alvaro Herrera wrote: > I think enlarging multixacts to 64 bits is a terrible idea. I would > rather get rid of multixacts completely; zheap is proposing not to use > multixacts at all, for example. But zedstore, at least as of the Saturday after PGCon, is proposi

Re: Where is SSPI auth username determined for TAP tests?

2019-06-29 Thread Michael Paquier
On Sat, Jun 29, 2019 at 04:36:51PM -0400, Tom Lane wrote: > I think this is likely a consequence of ca129e58c0 having modified > 010_dump_connstr.pl to use "regress_postgres" not "postgres" as the > bootstrap superuser name in the source cluster. I suppose I overlooked > some dependency on the use

Re: [HACKERS] proposal: schema variables

2019-06-29 Thread Pavel Stehule
pá 24. 5. 2019 v 19:12 odesílatel Pavel Stehule napsal: > Hi > > čt 9. 5. 2019 v 6:34 odesílatel Pavel Stehule > napsal: > >> Hi >> >> rebased patch >> > > rebase after pgindent > fresh rebase Regards Pavel > Regards > > Pavel > >> >> Regards >> >> Pavel >> >> >> schema-variables-20190630

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-06-29 Thread David Rowley
On Mon, 24 Jun 2019 at 23:12, David Rowley wrote: > > On Mon, 24 Jun 2019 at 22:16, Michael Paquier wrote: > > > > Don't take me bad, but I find the solution of defining and using a new > > callback to call the table AM callback not really elegant, and keeping > > all table AM callbacks called at

Re: proposal - patch: psql - sort_by_size

2019-06-29 Thread Fabien COELHO
Hello Pavel, \set SORT_BY_SIZE on \dt -- sorted by schema, name (size is not calculated and is not visible) \dt+ -- sorted by size Patch applies cleanly, compiles, runs. "make check" ok. doc build ok. There are no tests. Some infrastructure should be in place so that such features can be t

Re: Commitfest 2019-07, the first of five* for PostgreSQL 13

2019-06-29 Thread Thomas Munro
On Sat, Jun 29, 2019 at 9:47 AM David Steele wrote: > On 6/28/19 1:15 PM, Tom Lane wrote: > > Stephen Frost writes: > >> sh, don't look now, but there might be a "Resend email" button in > >> the archives now that you can click to have an email sent to you... > > > > Oooh, lovely. > > > >> (t

Re: proposal - patch: psql - sort_by_size

2019-06-29 Thread Pavel Stehule
so 29. 6. 2019 v 9:32 odesílatel Fabien COELHO napsal: > > Hello Pavel, > > > \set SORT_BY_SIZE on > > \dt -- sorted by schema, name (size is not calculated and is not visible) > > \dt+ -- sorted by size > > Patch applies cleanly, compiles, runs. "make check" ok. doc build ok. > > There are no te

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Julien Rouhaud
On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov wrote:> > On 29.06.2019 1:23, Julien Rouhaud wrote: > > But that kinda resembles stuff we already have - selectivity/cost. So > why shouldn't this be considered as part of costing? > > Yeah, I'm not entirely convinced that we need anything new here.

Re: Commitfest 2019-07, the first of five* for PostgreSQL 13

2019-06-29 Thread Magnus Hagander
On Sat, Jun 29, 2019 at 10:05 AM Thomas Munro wrote: > On Sat, Jun 29, 2019 at 9:47 AM David Steele wrote: > > On 6/28/19 1:15 PM, Tom Lane wrote: > > > Stephen Frost writes: > > >> sh, don't look now, but there might be a "Resend email" button in > > >> the archives now that you can click

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Tomas Vondra
On Sat, Jun 29, 2019 at 11:10:03AM +0200, Julien Rouhaud wrote: On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov wrote:> On 29.06.2019 1:23, Julien Rouhaud wrote: But that kinda resembles stuff we already have - selectivity/cost. So why shouldn't this be considered as part of costing? Yeah, I'

Re: Multivariate MCV list vs. statistics target

2019-06-29 Thread Tomas Vondra
On Fri, Jun 21, 2019 at 08:09:18AM +0100, Dean Rasheed wrote: On Thu, 20 Jun 2019 at 23:12, Tomas Vondra wrote: On Thu, Jun 20, 2019 at 08:08:44AM +0100, Dean Rasheed wrote: >On Tue, 18 Jun 2019 at 22:34, Tomas Vondra wrote: >> >> So I'm thinking we should allow tweaking the statistics for e

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Julien Rouhaud
On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra wrote: > > On Sat, Jun 29, 2019 at 11:10:03AM +0200, Julien Rouhaud wrote: > >On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov > >> -- patched > >> EXPLAIN ANALYZE SELECT * FROM test WHERE t LIKE '%1234%' AND t LIKE '%1%'; > >>

Re: Choosing values for multivariate MCV lists

2019-06-29 Thread Tomas Vondra
On Tue, Jun 25, 2019 at 11:18:19AM +0200, Tomas Vondra wrote: On Mon, Jun 24, 2019 at 02:54:01PM +0100, Dean Rasheed wrote: On Mon, 24 Jun 2019 at 00:42, Tomas Vondra wrote: On Sun, Jun 23, 2019 at 10:23:19PM +0200, Tomas Vondra wrote: On Sun, Jun 23, 2019 at 08:48:26PM +0100, Dean Rasheed w

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Tomas Vondra
On Sat, Jun 29, 2019 at 02:50:51PM +0200, Julien Rouhaud wrote: On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra wrote: On Sat, Jun 29, 2019 at 11:10:03AM +0200, Julien Rouhaud wrote: >On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov >> -- patched >> EXPLAIN ANALYZE SELECT * FROM test WHERE t LIKE

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Julien Rouhaud
On Sat, Jun 29, 2019 at 3:11 PM Tomas Vondra wrote: > > On Sat, Jun 29, 2019 at 02:50:51PM +0200, Julien Rouhaud wrote: > >On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra > >> A related issue is that during costing is too late to modify cardinality > >> estimates, so the 'Bitmap Index Scan' will be

Re: Optimize partial TOAST decompression

2019-06-29 Thread Andrey Borodin
Hi! Please, do not use top-posting, i.e. reply style where you quote whole message under your response. It makes reading of archives terse. > 24 июня 2019 г., в 7:53, Binguo Bao написал(а): > >> This is not correct: L bytes of compressed data do not always can be decoded >> into at least L byt

Re: TM format can mix encodings in to_char()

2019-06-29 Thread Tom Lane
Juanjo Santamaria Flecha writes: > It looks as if no work is left for this patch, so maybe updating the author > to Tom Lane (I'm just a repoter at this point, which it's fine) and the > status to ready for committer would better reflect its current status. Does > anyone think otherwise? Yeah,

Re: mcvstats serialization code is still shy of a load

2019-06-29 Thread Tomas Vondra
On Thu, Jun 27, 2019 at 01:26:32PM +0200, Tomas Vondra wrote: On Thu, Jun 27, 2019 at 12:04:30AM -0400, Tom Lane wrote: Tomas Vondra writes: OK. Attached is a patch ditching the alignment in serialized data. I've ditched the macros to access parts of serialized data, and everything gets copied

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Tomas Vondra
On Sat, Jun 29, 2019 at 03:28:11PM +0200, Julien Rouhaud wrote: On Sat, Jun 29, 2019 at 3:11 PM Tomas Vondra wrote: On Sat, Jun 29, 2019 at 02:50:51PM +0200, Julien Rouhaud wrote: >On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra >> A related issue is that during costing is too late to modify car

Re: Fix runtime errors from -fsanitize=undefined

2019-06-29 Thread didier
Hi, I tested this patch with clang 7 on master. - On unpatched master I can't reproduce errors with make check-world in: src/backend/access/heap/heapam.c src/backend/utils/cache/relcache.c (IIRC I triggered this one in a pg previous version) src/backend/utils/misc/guc.c - I have a hard to reprodu

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Alexander Korotkov
Hi! On Sat, Jun 29, 2019 at 1:52 AM Nikita Glukhov wrote: > We have a similar solution for this problem. The idea is to avoid full index > scan inside GIN itself when we have some GIN entries, and forcibly recheck > all tuples if triconsistent() returns GIN_MAYBE for the keys that emitted no > G

Re: [HACKERS] Regression tests vs existing users in an installation

2019-06-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> We could make the new subdirectory be something specific like >> "src/test/modules/test_rolenames", but I think very likely we'll be >> wanting some additional test scripts that we likewise deem unsafe to >> run during "installcheck

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Alexander Korotkov
On Sat, Jun 29, 2019 at 1:25 PM Tomas Vondra wrote: > A related issue is that during costing is too late to modify cardinality > estimates, so the 'Bitmap Index Scan' will be expected to return fewer > rows than it actually returns (after ignoring the full-scan quals). > Ignoring redundant quals (

Re: Avoid full GIN index scan when possible

2019-06-29 Thread Alexander Korotkov
On Sat, Jun 29, 2019 at 3:51 PM Julien Rouhaud wrote: > On Sat, Jun 29, 2019 at 12:25 PM Tomas Vondra > wrote: > > > > On Sat, Jun 29, 2019 at 11:10:03AM +0200, Julien Rouhaud wrote: > > >On Sat, Jun 29, 2019 at 12:51 AM Nikita Glukhov > > >> -- patched > > >> EXPLAIN ANALYZE SELECT * FROM test W

base backup client as auxiliary backend process

2019-06-29 Thread Peter Eisentraut
Setting up a standby instance is still quite complicated. You need to run pg_basebackup with all the right options. You need to make sure pg_basebackup has the right permissions for the target directories. The created instance has to be integrated into the operating system's start scripts. Ther

Increasing default value for effective_io_concurrency?

2019-06-29 Thread Tomas Vondra
Hi, I think we should consider changing the effective_io_concurrency default value, i.e. the guc that determines how many pages we try to prefetch in a couple of places (the most important being Bitmap Heap Scan). The default is 1 since forever, but from my experience hardly the right value, no

Where is SSPI auth username determined for TAP tests?

2019-06-29 Thread Tom Lane
bowerbird is failing the pg_dump regression tests with a lot of FATAL: SSPI authentication failed for user "regress_postgres" I think this is likely a consequence of ca129e58c0 having modified 010_dump_connstr.pl to use "regress_postgres" not "postgres" as the bootstrap superuser name in the sou

Re: UCT (Re: pgsql: Update time zone data files to tzdata release 2019a.)

2019-06-29 Thread Robert Haas
On Thu, Jun 27, 2019 at 1:58 PM Tom Lane wrote: > It's not really clear to me that the IANA folk intend those files to > be read as a list of preferred zone names. If they do, what are we > to make of the fact that no variant of "UTC" appears in them? I think their intent is key. We can't make

Re: Usage of epoch in txid_current

2019-06-29 Thread Robert Haas
On Mon, Jun 24, 2019 at 1:43 PM Alvaro Herrera wrote: > I think enlarging multixacts to 64 bits is a terrible idea. I would > rather get rid of multixacts completely; zheap is proposing not to use > multixacts at all, for example. But zedstore, at least as of the Saturday after PGCon, is proposi

Re: Where is SSPI auth username determined for TAP tests?

2019-06-29 Thread Michael Paquier
On Sat, Jun 29, 2019 at 04:36:51PM -0400, Tom Lane wrote: > I think this is likely a consequence of ca129e58c0 having modified > 010_dump_connstr.pl to use "regress_postgres" not "postgres" as the > bootstrap superuser name in the source cluster. I suppose I overlooked > some dependency on the use

Re: [HACKERS] proposal: schema variables

2019-06-29 Thread Pavel Stehule
pá 24. 5. 2019 v 19:12 odesílatel Pavel Stehule napsal: > Hi > > čt 9. 5. 2019 v 6:34 odesílatel Pavel Stehule > napsal: > >> Hi >> >> rebased patch >> > > rebase after pgindent > fresh rebase Regards Pavel > Regards > > Pavel > >> >> Regards >> >> Pavel >> >> >> schema-variables-20190630

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-06-29 Thread David Rowley
On Mon, 24 Jun 2019 at 23:12, David Rowley wrote: > > On Mon, 24 Jun 2019 at 22:16, Michael Paquier wrote: > > > > Don't take me bad, but I find the solution of defining and using a new > > callback to call the table AM callback not really elegant, and keeping > > all table AM callbacks called at