Re: pgbench - use pg logging capabilities

2020-01-09 Thread Fabien COELHO
Bonjour Michaël, TBH, my recommendation would be to drop *all* of these likely() and unlikely() calls. What evidence have you got that those are meaningfully improving the quality of the generated code? And if they're buried inside macros, they certainly aren't doing anything useful in terms

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2020-01-09 Thread Mahendra Singh Thalor
On Thu, 9 Jan 2020 at 09:36, Michael Paquier wrote: > > On Wed, Jan 08, 2020 at 10:56:01AM +0900, Michael Paquier wrote: > > And done this way as per the attached. I am of course open to > > objections or better ideas, though this looks formulation looks pretty > > good to me. Robert? > > Just

Re: CREATE ROUTINE MAPPING

2020-01-09 Thread Tom Lane
[ I wasn't paying much attention to this thread at the time, but Kyotaro-san just drew my attention to it again ] Masahiko Sawada writes: > I agree that this feature covers A and B as the first step. But I'm > concerned that for D (and maybe for C?) the volatility of mapped > function could be

Re: [Logical Replication] TRAP: FailedAssertion("rel->rd_rel->relreplident == REPLICA_IDENTITY_DEFAULT || rel->rd_rel->relreplident == REPLICA_IDENTITY_FULL || rel->rd_rel->relreplident == REPLICA_IDE

2020-01-09 Thread Dilip Kumar
On Fri, Jan 10, 2020 at 10:31 AM Michael Paquier wrote: > > On Fri, Jan 10, 2020 at 07:30:34AM +0530, Dilip Kumar wrote: > > On Thu, 9 Jan 2020 at 10:43 PM, Andres Freund wrote: > >> There's not much point in having this assert, right? Given that it > >> covers all choices? Seems better to just

Re: remove some STATUS_* symbols

2020-01-09 Thread Michael Paquier
On Thu, Jan 09, 2020 at 11:15:08AM +0100, Peter Eisentraut wrote: > You mean put he subsequent GrantLock() calls into LockCheckConflicts()? That > would technically save some duplicate code, but it seems weird, because > LockCheckConflicts() is notionally a read-only function that shouldn't >

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-01-09 Thread Dilip Kumar
On Mon, Dec 30, 2019 at 3:43 PM Amit Kapila wrote: > > On Sun, Dec 29, 2019 at 1:34 PM Dilip Kumar wrote: > > > > I have observed some more issues > > > > 1. Currently, In ReorderBufferCommit, it is always expected that > > whenever we get REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM, we must > >

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2020-01-09 Thread Tom Lane
Noah Misch writes: > On Thu, Jan 09, 2020 at 12:45:41AM -0500, Tom Lane wrote: >> (1) Why did we get a crash and not some more-decipherable out-of-resources >> error? Can we improve that experience? > By default, 32-bit AIX binaries have maxdata:0x. Specifying > maxdata:0x1000

Re: [HACKERS] Block level parallel vacuum

2020-01-09 Thread Masahiko Sawada
On Thu, 9 Jan 2020 at 19:33, Amit Kapila wrote: > > On Thu, Jan 9, 2020 at 10:41 AM Masahiko Sawada > wrote: > > > > On Wed, 8 Jan 2020 at 22:16, Amit Kapila wrote: > > > > > > > > > What do you think of the attached? Sawada-san, kindly verify the > > > changes and let me know your opinion. >

Re: [HACKERS] Block level parallel vacuum

2020-01-09 Thread Mahendra Singh Thalor
On Thu, 9 Jan 2020 at 17:31, Sergei Kornilov wrote: > > Hello > > I noticed that parallel vacuum uses min_parallel_index_scan_size GUC to skip > small indexes but this is not mentioned in documentation for both vacuum > command and GUC itself. > > + /* Determine the number of parallel

Re: [Logical Replication] TRAP: FailedAssertion("rel->rd_rel->relreplident == REPLICA_IDENTITY_DEFAULT || rel->rd_rel->relreplident == REPLICA_IDENTITY_FULL || rel->rd_rel->relreplident == REPLICA_IDE

2020-01-09 Thread Michael Paquier
On Fri, Jan 10, 2020 at 07:30:34AM +0530, Dilip Kumar wrote: > On Thu, 9 Jan 2020 at 10:43 PM, Andres Freund wrote: >> There's not much point in having this assert, right? Given that it >> covers all choices? Seems better to just drop it. > > Yeah right! Refreshing my mind on that... The two

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-01-09 Thread Dilip Kumar
On Sat, Jan 4, 2020 at 4:07 PM Amit Kapila wrote: > > On Mon, Dec 30, 2019 at 3:11 PM Dilip Kumar wrote: > > > > On Thu, Dec 12, 2019 at 9:44 AM Dilip Kumar wrote: > > > > 0002-Issue-individual-invalidations-with-wal_level-log > > > >

Re: pgbench - use pg logging capabilities

2020-01-09 Thread Michael Paquier
On Thu, Jan 09, 2020 at 08:09:29PM -0500, Tom Lane wrote: > TBH, my recommendation would be to drop *all* of these likely() > and unlikely() calls. What evidence have you got that those are > meaningfully improving the quality of the generated code? And if > they're buried inside macros, they

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2020-01-09 Thread Amit Kapila
On Fri, Jan 10, 2020 at 6:10 AM Tom Lane wrote: > > I wrote: > > ReorderBuffer: 223302560 total in 26995 blocks; 7056 free (3 > > chunks); 223295504 used > > > The test case is only inserting 50K fairly-short rows, so this seems > > like an unreasonable amount of memory to be consuming

Re: base backup client as auxiliary backend process

2020-01-09 Thread Masahiko Sawada
On Fri, 22 Nov 2019 at 19:22, Peter Eisentraut wrote: > > On 2019-11-15 14:52, Sergei Kornilov wrote: > >> I looked into this. It seems trivial to make walsender create and use a > >> temporary replication slot by default if no permanent replication slot > >> is specified. This is basically the

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2020-01-09 Thread Melanie Plageman
On Tue, Jan 7, 2020 at 4:14 PM Thomas Munro wrote: > * I am uneasy about BarrierArriveExplicitAndWait() (a variant of > BarrierArriveAndWait() that lets you skip directly to a given phase?); > perhaps you only needed that for a circular phase system, which you > could do with modular phase

Re: [Logical Replication] TRAP: FailedAssertion("rel->rd_rel->relreplident == REPLICA_IDENTITY_DEFAULT || rel->rd_rel->relreplident == REPLICA_IDENTITY_FULL || rel->rd_rel->relreplident == REPLICA_IDE

2020-01-09 Thread Dilip Kumar
On Thu, 9 Jan 2020 at 10:43 PM, Andres Freund wrote: > Hi, > > On 2020-01-09 13:17:59 +0530, Dilip Kumar wrote: > > I am able to reproduce the failure, I think the assert in the > > 'logicalrep_write_insert' is not correct. IMHO even if the replica > > identity is set to NOTHING we should be

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2020-01-09 Thread Noah Misch
On Thu, Jan 09, 2020 at 12:45:41AM -0500, Tom Lane wrote: > Noah Misch writes: > > Even so, a web search for "extend_brk" led to the answer. By default, > > 32-bit > > AIX binaries get only 256M of RAM for stack and sbrk. The new regression > > test > > used more than that, hence this crash.

Re: table partitioning and access privileges

2020-01-09 Thread Fujii Masao
On Tue, Jan 7, 2020 at 5:15 PM Amit Langote wrote: > > On Fri, Dec 27, 2019 at 4:26 AM Tom Lane wrote: > > Fujii Masao writes: > > > My customer reported me that the queries through a partitioned table > > > ignore each partition's SELECT, INSERT, UPDATE, and DELETE privileges, > > > on the

Re: backup manifests

2020-01-09 Thread David Steele
Hi Robert, On 1/7/20 6:33 PM, Stephen Frost wrote: > These are issues that we've thought > about and worried about over the years of pgbackrest and with that > experience we've come down on the side that a JSON-based format would be > an altogether better design. That's why we're advocating

RE: [PATCH] Resolve Parallel Hash Join Performance Issue

2020-01-09 Thread Deng, Gang
Regarding to the reason of setting bit was not cheap anymore in parallel join. As I explain in my original mail, it is because 'false sharing cache coherence'. In short word, setting of the bit will cause the whole cache line (64 bytes) dirty. So that all CPU cores contain the cache line have

Re: pgbench - use pg logging capabilities

2020-01-09 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Jan-09, Fabien COELHO wrote: >> -if (unlikely(__pg_log_level <= PG_LOG_DEBUG)) >> +if (pg_log_debug_level) >> { > Umm ... I find the original exceedingly ugly, but the new line is > totally impenetrable. So, I had not been paying any attention to

RE: [PATCH] Resolve Parallel Hash Join Performance Issue

2020-01-09 Thread Deng, Gang
Thank you for the comment. Yes, I agree the alternative of using '(!parallel)', so that no need to test the bit. Will someone submit patch to for it accordingly? -Original Message- From: Thomas Munro Sent: Thursday, January 9, 2020 6:04 PM To: Deng, Gang Cc:

Re: pgbench - use pg logging capabilities

2020-01-09 Thread Michael Paquier
On Thu, Jan 09, 2020 at 09:27:42PM -0300, Alvaro Herrera wrote: > On 2020-Jan-09, Fabien COELHO wrote: >> -if (unlikely(__pg_log_level <= PG_LOG_DEBUG)) >> +if (pg_log_debug_level) >> { > > Umm ... I find the original exceedingly ugly, but the new line is > totally impenetrable.

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2020-01-09 Thread Tom Lane
I wrote: > ReorderBuffer: 223302560 total in 26995 blocks; 7056 free (3 > chunks); 223295504 used > The test case is only inserting 50K fairly-short rows, so this seems > like an unreasonable amount of memory to be consuming for that; and > even if you think it's reasonable, it clearly

Re: pgbench - use pg logging capabilities

2020-01-09 Thread Alvaro Herrera
On 2020-Jan-09, Fabien COELHO wrote: > - if (unlikely(__pg_log_level <= PG_LOG_DEBUG)) > + if (pg_log_debug_level) > { Umm ... I find the original exceedingly ugly, but the new line is totally impenetrable. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2020-01-09 Thread Alvaro Herrera
I looked at this a little while and was bothered by the perl changes; it seems out of place to have RecursiveCopy be thinking about tablespaces, which is way out of its league. So I rewrote that to use a callback: the PostgresNode code passes a callback that's in charge to handle the case of a

Re: pgbench - use pg logging capabilities

2020-01-09 Thread Michael Paquier
On Thu, Jan 09, 2020 at 10:28:21AM +0100, Fabien COELHO wrote: > Yep, I thought of it, but I was not very keen on having a malloc/free cycle > just for one debug message. However under debug this is probably not an > issue. Consistency is more important here IMO, so applied. > Your patch works

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2020-01-09 Thread Tom Lane
Amit Kapila writes: > On Thu, Jan 9, 2020 at 11:15 AM Tom Lane wrote: >> Noah Misch writes: >>> Even so, a web search for "extend_brk" led to the answer. By default, >>> 32-bit >>> AIX binaries get only 256M of RAM for stack and sbrk. The new regression >>> test >>> used more than that,

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2020-01-09 Thread cary huang
On Mon, Jan 6, 2020 at 4:43 AM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: > On Sat, 4 Jan 2020 at 15:11, cary huang wrote: > >> > >> Hello Sawada and all > >> > >> I would like to elaborate more on Sehrope and Sawada's discussion on > passing NULL IV in "pg_cipher_encrypt/decrypt"

Re: Setting min/max TLS protocol in clientside libpq

2020-01-09 Thread Daniel Gustafsson
Thanks for review everyone! A v2 of the patch which I believe addresses all concerns raised is attached. > On 6 Jan 2020, at 07:01, Michael Paquier wrote: > > On Thu, Jan 02, 2020 at 09:46:44PM +, cary huang wrote: >> I agree with Arthur that it makes sense to check the validity of >>

Re: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2020-01-09 Thread Andrew Dunstan
On Fri, Jan 10, 2020 at 8:32 AM Tom Lane wrote: > > Andrew Dunstan writes: > > On Fri, Jan 10, 2020 at 1:21 AM Robert Haas wrote: > >> I share the concern about the security issue here. I can't testify to > >> whether Christoph's whole analysis is here, but as a general point, > >>

Re: pgbench - rework variable management

2020-01-09 Thread Fabien COELHO
Patch v4 is a just a rebase. Patch v5 is a rebase with some adjustements. -- Fabien.diff --git a/src/bin/pgbench/Makefile b/src/bin/pgbench/Makefile index f402fe7b91..e1d3ef9bb3 100644 --- a/src/bin/pgbench/Makefile +++ b/src/bin/pgbench/Makefile @@ -10,6 +10,7 @@ include

Re: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2020-01-09 Thread Tom Lane
Andrew Dunstan writes: > On Fri, Jan 10, 2020 at 1:21 AM Robert Haas wrote: >> I share the concern about the security issue here. I can't testify to >> whether Christoph's whole analysis is here, but as a general point, >> non-superusers can't be allowed to do things that cause the server to >>

Re: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2020-01-09 Thread Daniel Gustafsson
> On 9 Jan 2020, at 22:38, Andrew Dunstan > wrote: > I'm not (yet) > convinced that there is any significant security threat here. This > doesn't give the user or indeed any postgres code any access to the > contents of these files. But if there is a consensus to restrict this > I'll do it.

Re: pgsql: Add basic TAP tests for psql's tab-completion logic.

2020-01-09 Thread Tom Lane
I wrote: > Peter Eisentraut writes: >> I think he means something like >> make check with_readline=no >> not for the actual build. > Oh, I see. I'd rather not codify that though, because it risks > problems if that symbol ever gets used any other way. I was > thinking of making the test

Re: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2020-01-09 Thread Andrew Dunstan
On Fri, Jan 10, 2020 at 1:21 AM Robert Haas wrote: > > On Thu, Jan 9, 2020 at 5:30 AM Christoph Berg wrote: > > I have some concerns about security, though. It's true that the > > sslcert/sslkey options can only be set/modified by superusers when > > "password_required" is set. But when

Re: Coding in WalSndWaitForWal

2020-01-09 Thread Tom Lane
Alvaro Herrera writes: > However, we already have a large number of proc_exit() calls in switch > blocks that are not followed by breaks. In fact, the majority are > already like that. Oh, hmm ... consistency is good. regards, tom lane

Re: our checks for read-only queries are not great

2020-01-09 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 9, 2020 at 3:07 PM Tom Lane wrote: >> You could argue about exactly how to extend that to non-spec >> utility commands, but for the most part allowing them seems >> to make sense if DML is allowed. > But I think we allow them on all tables, not just temp

Re: our checks for read-only queries are not great

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 3:37 PM Robert Haas wrote: > > You could argue about exactly how to extend that to non-spec > > utility commands, but for the most part allowing them seems > > to make sense if DML is allowed. > > But I think we allow them on all tables, not just temp tables, so I > don't

Re: our checks for read-only queries are not great

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 3:07 PM Tom Lane wrote: > Robert Haas writes: > > Maybe the SQL standard has something to say about this? > > [ pokes around ... ] Yeah, it does, and I'd say it's pretty clearly > in agreement with what Peter did, so far as DML ops go. For instance, > this bit from

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 3:18 PM Tom Lane wrote: > * ISTM that that's assuming that the DBA and the sysadmin are the same > person (or at least hold identical views on this subject). In many > installations it'd only be root who has control over what's in that > directory, and I don't think it's

Re: [WIP] UNNEST(REFCURSOR): allowing SELECT to consume data from a REFCURSOR

2020-01-09 Thread Dent John
> On 9 Jan 2020, at 17:43, Daniel Verite wrote: > > […] > (using plain text instead of HTML messages might help with that). Thanks. I’ll do that next time. > […] > * unnest-refcursor-v3.patch needs a slight rebase because this chunk > in the Makefile fails > - regexp.o regproc.o

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Tom Lane
Stephen Frost writes: > I am not particularly concerned about that backwards compatibility issue > and I don’t intend to base my argument on it, but I would use that case to > point out that we have long had the ability to install trusted C functions > into the backend as a DB owner, without

Re: Coding in WalSndWaitForWal

2020-01-09 Thread Alvaro Herrera
On 2020-Jan-09, Tom Lane wrote: > Alvaro Herrera writes: > > In modern times, we define pg_attribute_noreturn() like this: > > > /* GCC, Sunpro and XLC support aligned, packed and noreturn */ > > #if defined(__GNUC__) || defined(__SUNPRO_C) || defined(__IBMC__) > > #define

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Stephen Frost
Greetings, On Thu, Jan 9, 2020 at 14:48 Tom Lane wrote: > Robert Haas writes: > > So, if I understand correctly, the patch you are proposing to commit > > has a new system role, and if you've got that system role, then you > > can install extensions. > > Install *trusted* extensions, correct.

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Tom Lane
Stephen Frost writes: > So I'm at a loss for why there is this insistence on a default role and > a superuser-explicit-granting based approach that goes beyond "is it > installed on the filesystem?" and "is it marked as trusted?". Okay, so it seems like we're down to just this one point of

Re: our checks for read-only queries are not great

2020-01-09 Thread Tom Lane
Robert Haas writes: > Maybe the SQL standard has something to say about this? [ pokes around ... ] Yeah, it does, and I'd say it's pretty clearly in agreement with what Peter did, so far as DML ops go. For instance, this bit from SQL99's description of DELETE: 1) If the access mode

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 2:48 PM Tom Lane wrote: > I agree that if we dropped the proviso about DB owners, it would be > a cleaner design. I included that only for backwards compatibility > with the existing behavior that DB owners can install trusted PLs. > If we can agree that we're willing to

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Jan 9, 2020 at 1:35 PM Tom Lane wrote: > > Robert Haas writes: > > > Again, as I said upthread, Tom had the exact feature about which I am > > > talking in the first version of the patch. That is a strong argument > > > in favor

Re: our checks for read-only queries are not great

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 2:24 PM Tom Lane wrote: > Robert Haas writes: > > I'd be really interested to hear if anyone knows the history behind > > allowing CLUSTER, REINDEX, VACUUM, and some operations on temp tables. > > It seems to have been that way for a long time. I wonder if it was a > >

Re: Coding in WalSndWaitForWal

2020-01-09 Thread Tom Lane
Alvaro Herrera writes: > In modern times, we define pg_attribute_noreturn() like this: > /* GCC, Sunpro and XLC support aligned, packed and noreturn */ > #if defined(__GNUC__) || defined(__SUNPRO_C) || defined(__IBMC__) > #define pg_attribute_noreturn() __attribute__((noreturn)) > #define

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Tom Lane
Robert Haas writes: > So, if I understand correctly, the patch you are proposing to commit > has a new system role, and if you've got that system role, then you > can install extensions. Install *trusted* extensions, correct. The patch as it stands also allows DB owners to install trusted

Re: Coding in WalSndWaitForWal

2020-01-09 Thread Alvaro Herrera
On 2019-Nov-12, Andres Freund wrote: > We still have the curious > proc_exit(0); > abort();/* keep the compiler > quiet */ > > pattern in WalSndShutdown() - wouldn't the right approach to instead > proc_exit() with pg_attribute_noreturn()?

Re: our checks for read-only queries are not great

2020-01-09 Thread Tom Lane
Robert Haas writes: > I'd be really interested to hear if anyone knows the history behind > allowing CLUSTER, REINDEX, VACUUM, and some operations on temp tables. > It seems to have been that way for a long time. I wonder if it was a > deliberate choice or something that just happened

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 1:35 PM Tom Lane wrote: > Robert Haas writes: > > Again, as I said upthread, Tom had the exact feature about which I am > > talking in the first version of the patch. That is a strong argument > > in favor of it being practical. It's also a pretty good argument that > > it

Re: our checks for read-only queries are not great

2020-01-09 Thread Robert Haas
On Wed, Jan 8, 2020 at 5:57 PM Stephen Frost wrote: > Yeah, I don't like the WEAKLY_READ_ONLY idea either- explicitly having > COMMAND_OK_IN_X seems cleaner. Done that way. v2 attached. > Would we be able to make a rule of "can't change on-disk stuff, except > for things in temporary schemas"

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Tom Lane
Robert Haas writes: > Again, as I said upthread, Tom had the exact feature about which I am > talking in the first version of the patch. That is a strong argument > in favor of it being practical. It's also a pretty good argument that > it is at least potentially useful, because Tom doesn't

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 11:30 AM Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > On Thu, Jan 9, 2020 at 10:09 AM Stephen Frost wrote: > > > [ wall of text ] > > This really isn't helpful. Sorry. That being said, I'm pretty tired of writing emails that say the same thing

Re: Add pg_file_sync() to adminpack

2020-01-09 Thread Tom Lane
Julien Rouhaud writes: > On Thu, Jan 9, 2020 at 6:16 PM Stephen Frost wrote: >> Why would you expect that when it isn't the case for the filesystem >> itself..? > Just a usual habit with durable property. I tend to agree with Stephen on this, mainly because the point of these adminpack

Re: [WIP] UNNEST(REFCURSOR): allowing SELECT to consume data from a REFCURSOR

2020-01-09 Thread Daniel Verite
Dent John wrote: > I’ve made a revision of this patch. Some comments: * the commitfest app did not extract up the patch from the mail, possibly because it's buried in the MIME structure of the mail (using plain text instead of HTML messages might help with that). The patch has no

Re: Add pg_file_sync() to adminpack

2020-01-09 Thread Julien Rouhaud
On Thu, Jan 9, 2020 at 6:16 PM Stephen Frost wrote: > > Greetings, > > * Julien Rouhaud (rjuju...@gmail.com) wrote: > > On Thu, Jan 9, 2020 at 7:43 AM Fujii Masao wrote: > > > On Wed, Dec 25, 2019 at 11:11 PM Julien Rouhaud > > > wrote: > > > > On Wed, Dec 25, 2019 at 2:01 PM Fujii Masao > >

Re: Make autovacuum sort tables in descending order of xid_age

2020-01-09 Thread Robert Haas
On Thu, Dec 12, 2019 at 2:26 PM David Fetter wrote: > > I wonder if you might add information about table size, table changes, > > and bloat to your RelFrozenXidAge struct and modify rfxa_comparator to > > use a heuristic to cost the (age, size, bloat, changed) grouping and > > sort on that cost,

Re: Add pg_file_sync() to adminpack

2020-01-09 Thread Stephen Frost
Greetings, * Julien Rouhaud (rjuju...@gmail.com) wrote: > On Thu, Jan 9, 2020 at 7:43 AM Fujii Masao wrote: > > On Wed, Dec 25, 2019 at 11:11 PM Julien Rouhaud wrote: > > > On Wed, Dec 25, 2019 at 2:01 PM Fujii Masao wrote: > > > > I'd like to propose to add pg_file_sync() function into > > >

Re: [Logical Replication] TRAP: FailedAssertion("rel->rd_rel->relreplident == REPLICA_IDENTITY_DEFAULT || rel->rd_rel->relreplident == REPLICA_IDENTITY_FULL || rel->rd_rel->relreplident == REPLICA_IDE

2020-01-09 Thread Andres Freund
Hi, On 2020-01-09 13:17:59 +0530, Dilip Kumar wrote: > I am able to reproduce the failure, I think the assert in the > 'logicalrep_write_insert' is not correct. IMHO even if the replica > identity is set to NOTHING we should be able to replicate INSERT? > > This will fix the issue. > > diff

Re: [HACKERS] pg_shmem_allocations view

2020-01-09 Thread Andres Freund
Hi, On 2020-01-09 11:13:46 -0500, Robert Haas wrote: > On Wed, Jan 8, 2020 at 2:30 AM Kyotaro Horiguchi > wrote: > > I agree to the patch as-is. Thanks for the explanatin. > > OK. Thanks for the review, and for your thoughtful consideration of my > comments. > > I went ahead and committed

Re: [Proposal] Global temporary tables

2020-01-09 Thread Tomas Vondra
On Thu, Jan 09, 2020 at 02:17:08PM +0300, Konstantin Knizhnik wrote: On 06.01.2020 8:04, 曾文旌(义从) wrote: In the previous communication 1 we agreed on the general direction 1.1 gtt use local (private) buffer 1.2 no replica access in first version 2 We feel that gtt needs to maintain

Re: Recognizing superuser in pg_hba.conf

2020-01-09 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > But, again, we already *have* a way of solving this problem: use > quotes. As Simon pointed out, your proposed solution isn't really a > solution at all, because & can appear in role names. It probably > won't, but there probably also

Re: Recognizing superuser in pg_hba.conf

2020-01-09 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 9, 2020 at 11:06 AM Tom Lane wrote: >> The problem is that we keep deciding that okay, it probably won't hurt >> anybody if this particular thing-that-ought-to-be-a-reserved-word isn't >> really reserved. > But, again, we already *have* a way of solving this

Re: [Proposal] Global temporary tables

2020-01-09 Thread Tomas Vondra
On Thu, Jan 09, 2020 at 06:07:46PM +0300, Konstantin Knizhnik wrote: On 06.01.2020 14:01, Tomas Vondra wrote: On Mon, Jan 06, 2020 at 01:04:15PM +0800, 曾文旌(义从) wrote: In the previous communication 1 we agreed on the general direction 1.1 gtt use local (private) buffer 1.2 no replica access

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Jan 9, 2020 at 10:09 AM Stephen Frost wrote: > > [ wall of text ] This really isn't helpful. > I don't see anything in here I really disagree with, but nor do I > understand why any of it means that giving superusers the ability

Re: Recognizing superuser in pg_hba.conf

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 11:06 AM Tom Lane wrote: > The problem is that we keep deciding that okay, it probably won't hurt > anybody if this particular thing-that-ought-to-be-a-reserved-word isn't > really reserved. Your exercise in justifying that for "superuser" is > not unlike every other

Re: pgsql: Add basic TAP tests for psql's tab-completion logic.

2020-01-09 Thread Tom Lane
Peter Eisentraut writes: > On 2020-01-09 16:50, Tom Lane wrote: >> That would disable psql's tab completion, command editing, and history >> altogether, which I doubt is what you want for production builds. >> If we conclude we can't work around the testing issues for ancient >> libedit, probably

Re: [HACKERS] pg_shmem_allocations view

2020-01-09 Thread Robert Haas
On Wed, Jan 8, 2020 at 2:30 AM Kyotaro Horiguchi wrote: > I agree to the patch as-is. Thanks for the explanatin. OK. Thanks for the review, and for your thoughtful consideration of my comments. I went ahead and committed this. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The

Re: pgsql: Add basic TAP tests for psql's tab-completion logic.

2020-01-09 Thread Peter Eisentraut
On 2020-01-09 16:50, Tom Lane wrote: Christoph Berg writes: Fwiw if libedit in xenial is Just Broken and fixing the tests would just codify the brokenness without adding any value, I'll just disable that test in that particular build. It looks like setting with_readline=no on the make command

Re: Recognizing superuser in pg_hba.conf

2020-01-09 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 9, 2020 at 10:06 AM Tom Lane wrote: >> What I'm basically objecting to is the pseudo-reservedness. If we >> don't want to dodge that with special syntax, we should dodge it >> by making sure the keywords are actually reserved names. > ... > But I think I'm

Re: pgsql: Add basic TAP tests for psql's tab-completion logic.

2020-01-09 Thread Tom Lane
Christoph Berg writes: > Fwiw if libedit in xenial is Just Broken and fixing the tests would > just codify the brokenness without adding any value, I'll just disable > that test in that particular build. It looks like setting > with_readline=no on the make command line should work. That would

Re: Patch to document base64 encoding

2020-01-09 Thread Karl O. Pinc
On Thu, 9 Jan 2020 08:27:28 +0100 (CET) Fabien COELHO wrote: > > Another option would be to use "bytes bytea". > > > (The current patch uses "string bytea".) > > This would probably also require some re-wording throughout. > I like it, but this is only my own limited opinion, and I'm not a

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2020-01-09 Thread Alvaro Herrera
On 2020-Jan-09, Tom Lane wrote: > Alvaro Herrera writes: > > Hmm, so why not revert the test only in the back branches, given that > > it's not so onerous in master? > > I grow tired of repeating myself, but: it's purely accidental that this > test passes in master for the existing set of

Re: Libpq support to connect to standby server as priority

2020-01-09 Thread Alvaro Herrera
On 2020-Jan-06, tsunakawa.ta...@fujitsu.com wrote: > Let me check my understanding. Are you proposing these? > > * The canonical libpq connection parameter is target_session_attr = {primary > | standby | prefer-standby}. Leave and document read-write as a synonym for > primary. > > * When

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2020-01-09 Thread Tom Lane
Alvaro Herrera writes: > Hmm, so why not revert the test only in the back branches, given that > it's not so onerous in master? I grow tired of repeating myself, but: it's purely accidental that this test passes in master for the existing set of buildfarm members. If I have to do so to prove my

Re: Recognizing superuser in pg_hba.conf

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 10:06 AM Tom Lane wrote: > What I'm basically objecting to is the pseudo-reservedness. If we > don't want to dodge that with special syntax, we should dodge it > by making sure the keywords are actually reserved names. You know, as I was reading this email, I got to

Re: Recognizing superuser in pg_hba.conf

2020-01-09 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > What I'm basically objecting to is the pseudo-reservedness. If we > don't want to dodge that with special syntax, we should dodge it > by making sure the keywords are actually reserved names. In other > words, add a "pg_" prefix, as somebody

Re: Removing pg_pltemplate and creating "trustable" extensions

2020-01-09 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 8, 2020 at 6:09 PM Stephen Frost wrote: > > > To me, this seems more accidental than the natural fallout of good design. > > > > I disagree that the privilege design for FDWs was accidental. > > That seems like a stronger

Re: [Proposal] Global temporary tables

2020-01-09 Thread Konstantin Knizhnik
On 06.01.2020 14:01, Tomas Vondra wrote: On Mon, Jan 06, 2020 at 01:04:15PM +0800, 曾文旌(义从) wrote: In the previous communication 1 we agreed on the general direction 1.1 gtt use local (private) buffer 1.2 no replica access in first version OK, good. 2 We feel that gtt needs to maintain

Re: Recognizing superuser in pg_hba.conf

2020-01-09 Thread Tom Lane
Simon Riggs writes: > On Wed, 8 Jan 2020 at 23:55, Vik Fearing > wrote: >> What is being proposed is what is in the Subject and the original >> patch. The other patch is because Tom didn't like "the continuing creep >> of pseudo-reserved database and user names" so I wrote a patch to mark >>

Re: pgsql: Add basic TAP tests for psql's tab-completion logic.

2020-01-09 Thread Christoph Berg
Re: Robert Haas 2020-01-09 > > So that raises the question: why does xenial's version of libedit > > not match either its documentation or the distributed source code? > > Because it doesn't. > > The level of effort you've put into this is extremely impressive, but > I can't shake the feeling

Re: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 5:30 AM Christoph Berg wrote: > I have some concerns about security, though. It's true that the > sslcert/sslkey options can only be set/modified by superusers when > "password_required" is set. But when password_required is not set, any > user and create user mappings that

Re: pgsql: Add basic TAP tests for psql's tab-completion logic.

2020-01-09 Thread Robert Haas
On Tue, Jan 7, 2020 at 5:29 PM Tom Lane wrote: > So that raises the question: why does xenial's version of libedit > not match either its documentation or the distributed source code? > Because it doesn't. The level of effort you've put into this is extremely impressive, but I can't shake the

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2020-01-09 Thread Alvaro Herrera
On 2020-Jan-09, Amit Kapila wrote: > In HEAD, we have a guc variable 'logical_decoding_work_mem' by which > we can control the memory usage of changes and we have used that, but > for back branches, we don't have such a control. > After the latest changes by Noah, the tern and mandrill both are

Re: [PATCH] Resolve Parallel Hash Join Performance Issue

2020-01-09 Thread Tom Lane
Thomas Munro writes: > Right, I see. The funny thing is that the match bit is not even used > in this query (it's used for right and full hash join, and those > aren't supported for parallel joins yet). Hmm. So, instead of the > test you proposed, an alternative would be to use if (!parallel).

Re: pgsql: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2020-01-09 Thread Christoph Berg
Re: Robert Haas 2020-01-09 > Does this mean that a non-superuser can induce postgres_fdw to read an > arbitrary file from the local filesystem? Yes, see my comments in the "Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings" thread. Christoph

Re: Fixing parallel make of libpq

2020-01-09 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > My thoughts about the patch: > > 1) Changing from an "|"-style dependency to a plain dependency seems > like a semantics change. I've never been totally clear on the > difference though. I think Peter introduced our use of the "|" style, > so maybe he can comment.

Re: pgsql: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 3:11 AM Andrew Dunstan wrote: > Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings > > This allows different users to authenticate with different certificates. > > Author: Craig Ringer > >

Re: Fixing parallel make of libpq

2020-01-09 Thread Tom Lane
ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: > Michael Paquier writes: >> Hmm. That logically makes sense. Isn't that a side effect of 7143b3e >> then? Now, FWIW, I am not able to reproduce it here, after trying on >> two different machines, various parallel job

Re: Add pg_file_sync() to adminpack

2020-01-09 Thread Julien Rouhaud
On Thu, Jan 9, 2020 at 7:31 AM Fujii Masao wrote: > > On Mon, Jan 6, 2020 at 3:42 PM Michael Paquier wrote: > > > > On Mon, Jan 06, 2020 at 03:20:13PM +0900, Arthur Zakirov wrote: > > > It isn't case if a file doesn't exist. But if there are no permissions on > > > the file: > > > > > > PANIC:

Re: Add pg_file_sync() to adminpack

2020-01-09 Thread Julien Rouhaud
On Thu, Jan 9, 2020 at 7:43 AM Fujii Masao wrote: > > On Wed, Dec 25, 2019 at 11:11 PM Julien Rouhaud wrote: > > > > On Wed, Dec 25, 2019 at 2:01 PM Fujii Masao wrote: > > > > > > Hi, > > > > > > I'd like to propose to add pg_file_sync() function into contrib/adminpack. > > > This function

Re: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2020-01-09 Thread Christoph Berg
Re: To Andrew Dunstan 2020-01-09 <20200109103014.ga4...@msg.df7cb.de> > I believe the options are still used in that case > for creating connections, even when that means the remote server isn't > set up for cert auth, which needs password_required=false to succeed. They are indeed:

Re: Error message inconsistency

2020-01-09 Thread MBeena Emerson
Hi Mahendra, Thanks for the patch. I am not sure but maybe the relation name should also be added to the following test case? create table t4 (id int); insert into t4 values (1); ALTER TABLE t4 ADD CONSTRAINT c1 CHECK (id > 10) NOT VALID; -- succeeds ALTER TABLE t4 VALIDATE CONSTRAINT c1;

Re: [HACKERS] Block level parallel vacuum

2020-01-09 Thread Sergei Kornilov
Hello I noticed that parallel vacuum uses min_parallel_index_scan_size GUC to skip small indexes but this is not mentioned in documentation for both vacuum command and GUC itself. + /* Determine the number of parallel workers to launch */ + if (lps->lvshared->for_cleanup) + {

Re: [Proposal] Global temporary tables

2020-01-09 Thread Konstantin Knizhnik
On 06.01.2020 8:04, 曾文旌(义从) wrote: In the previous communication 1 we agreed on the general direction 1.1 gtt use local (private) buffer 1.2 no replica access in first version 2 We feel that gtt needs to maintain statistics, but there is no agreement on what it will be done. 3 Still no

  1   2   >