Re: [PATCH] pg_hba.conf error messages for logical replication connections

2021-01-29 Thread Amit Kapila
On Sat, Jan 30, 2021 at 12:24 AM Paul Martinez wrote: > > On Thu, Jan 28, 2021 at 8:17 PM Amit Kapila wrote: > > > > What exactly are you bothered about here? Is the database name not > > present in the message your concern or the message uses 'replication' > > but actually it doesn't relate to

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Peter Geoghegan
On Fri, Jan 29, 2021 at 6:44 PM Tom Lane wrote: > Pointer width is interesting, but really it's a solved problem > compared to these. What about USE_FLOAT8_BYVAL? -- Peter Geoghegan

Re: logical replication worker accesses catalogs in error context callback

2021-01-29 Thread Andres Freund
Hi, On 2021-01-28 11:14:03 +0530, Amit Kapila wrote: > On Wed, Jan 27, 2021 at 9:38 AM Bharath Rupireddy > wrote: > > > > On Wed, Jan 27, 2021 at 7:48 AM Zhihong Yu wrote: > > > > Thanks for pointing to the changes in the commit message. I corrected > > them. Attaching v4 patch set, consider it

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Tom Lane
Peter Geoghegan writes: > Broad trends have made it easier to write portable C code, but that > doesn't apply to 32-bit machines, I imagine. Including even the > extremely low power 32-bit chips that are not yet fully obsolete, like > the Raspberry Pi Zero's chip. Meh. To my mind, the most

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Peter Geoghegan
On Fri, Jan 29, 2021 at 6:33 PM Michael Paquier wrote: > Those 32-bit modules are still being sold actively by the RPI > foundation, and used as cheap machines for education purposes, so I > think that it is still useful for Postgres to have active buildfarm > members for 32-bit architectures.

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Peter Geoghegan
On Fri, Jan 29, 2021 at 5:53 PM Tom Lane wrote: > Hmph ... three of my five buildfarm animals are 32-bit, plus I > have got 32-bit OSes for my Raspberry Pi ;-). Admittedly, none > of those represent hardware someone would put a serious database > on today. But in terms of testing diversity, I

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Michael Paquier
On Fri, Jan 29, 2021 at 08:53:49PM -0500, Tom Lane wrote: > Peter Geoghegan writes: >> I don't know anybody that still runs Postgres >> (or anything like it) on a 32-bit platform. I think that Michael >> Paquier owns a Raspberry Pi zero, but that hardly counts. hamster died a couple of years

Re: LogwrtResult contended spinlock

2021-01-29 Thread Andres Freund
Hi, On 2021-01-29 22:47:56 -0300, Alvaro Herrera wrote: > So I tried this, but -- perhaps not suprisingly -- I can't get it to > work properly; the synchronization fails. What do you mean by "synchronization fails"? > Strangely, all tests work for me, but the pg_upgrade one in particular >

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-29 Thread Michael Paquier
On Fri, Jan 29, 2021 at 08:56:47PM +0300, Alexey Kondratov wrote: > On 2021-01-28 14:42, Alexey Kondratov wrote: >> No, you are right, we are doing REINDEX with AccessExclusiveLock as it >> was before. This part is a more specific one. It only applies to >> partitioned indexes, which do not hold

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-29 Thread Peter Geoghegan
On Thu, Jan 28, 2021 at 10:16 AM Michail Nikolaev wrote: > > I wonder if it would help to not actually use the LP_DEAD bit for > > this. Instead, you could use the currently-unused-in-indexes > > LP_REDIRECT bit. > > Hm… Sound very promising - an additional bit is a lot in this situation. Yeah,

Re: LogwrtResult contended spinlock

2021-01-29 Thread Andres Freund
Hi, On 2021-01-29 12:40:18 -0300, Alvaro Herrera wrote: > On 2020-Aug-31, Andres Freund wrote: > > > Wouldn't the better fix here be to allow reading of individual members > > without a lock? E.g. by wrapping each in a 64bit atomic. > > So I've been playing with this and I'm annoyed about

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Tom Lane
Peter Geoghegan writes: > On Fri, Jan 29, 2021 at 4:01 PM Tom Lane wrote: >> We do still have x86 in the buildfarm, as well as some other >> 32-bit platforms, so I don't agree that it's that much less >> tested than non-mainstream 64-bit platforms. > I don't know anybody that still runs

Re: LogwrtResult contended spinlock

2021-01-29 Thread Alvaro Herrera
So I tried this, but -- perhaps not suprisingly -- I can't get it to work properly; the synchronization fails. I suspect I need some barriers, but I tried adding a few (probably some that are not really necessary) and that didn't have the expected effect. Strangely, all tests work for me, but

Re: Support for NSS as a libpq TLS backend

2021-01-29 Thread Michael Paquier
On Fri, Jan 29, 2021 at 02:13:30PM +0100, Daniel Gustafsson wrote: > I'm still not convinced that adding --with-ssl=openssl is worth it before the > rest of NSS goes in (and more importantly, *if* it goes in). > > On the one hand, we already have pluggable (for some value of) support for > adding

Re: New IndexAM API controlling index vacuum strategies

2021-01-29 Thread Peter Geoghegan
On Tue, Jan 26, 2021 at 10:59 PM Masahiko Sawada wrote: > What value is set to fillfactor? 90, same as before. > That's very good. I'm happy that this patch efficiently utilizes > bottom-up index deletion feature. Me too! > Looking at the relation size growth, there is almost no difference >

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Peter Geoghegan
On Fri, Jan 29, 2021 at 4:01 PM Tom Lane wrote: > > It's probably much riskier to use 32-bit x86 today than > > it is to use (say) POWER8, or some other contemporary minority > > platform. > > We do still have x86 in the buildfarm, as well as some other > 32-bit platforms, so I don't agree that

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-29 Thread Bharath Rupireddy
On Sat, Jan 30, 2021 at 12:14 AM Fujii Masao wrote: > + /* > +* It doesn't make sense to show this entry in the > output with a > +* NULL server_name as it will be closed at the xact > end. > +*/ > +

Re: Proposal: Save user's original authenticated identity for logging

2021-01-29 Thread Jacob Champion
On Fri, 2021-01-29 at 18:40 -0500, Tom Lane wrote: > Ah. So basically, this comes into play when you consider that some > outside-the-database entity is your "real" authenticated identity. > That seems reasonable when using Kerberos or the like, though it's > not real meaningful for traditional

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Tom Lane
Peter Geoghegan writes: > On Fri, Jan 29, 2021 at 12:12 PM Tom Lane wrote: >> I got annoyed (not for the first time) by the fact that the >> partitioned_rels field of AppendPath and MergeAppendPath is a list of >> Relids, i.e., Bitmapsets. This is problematic because a Bitmapset is >> not a

Re: Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Peter Geoghegan
On Fri, Jan 29, 2021 at 12:12 PM Tom Lane wrote: > I got annoyed (not for the first time) by the fact that the > partitioned_rels field of AppendPath and MergeAppendPath is a list of > Relids, i.e., Bitmapsets. This is problematic because a Bitmapset is > not a type of Node, and thus a List of

Re: Proposal: Save user's original authenticated identity for logging

2021-01-29 Thread Tom Lane
Jacob Champion writes: > On Fri, 2021-01-29 at 17:30 -0500, Tom Lane wrote: >> What happens if ALTER USER RENAME is done while the session is still >> alive? > IMO the authenticated identity should be write-once. Especially since > one of my goals is to have greater auditability into events as

Re: Proposal: Save user's original authenticated identity for logging

2021-01-29 Thread Jacob Champion
On Fri, 2021-01-29 at 17:30 -0500, Tom Lane wrote: > What happens if ALTER USER RENAME is done while the session is still > alive? IMO the authenticated identity should be write-once. Especially since one of my goals is to have greater auditability into events as they've actually happened. So

Re: Proposal: Save user's original authenticated identity for logging

2021-01-29 Thread Jacob Champion
On Fri, 2021-01-29 at 17:01 -0500, Stephen Frost wrote: > > - for LDAP, the bind DN is discarded entirely; > > We don't support pg_ident.conf-style entries for LDAP, meaning that the > user provided has to match what we check, so I'm not sure what would be > improved with this change..? For

Re: [sqlsmith] Failed assertion during partition pruning

2021-01-29 Thread Tom Lane
Zhihong Yu writes: > I wonder if the (failed) assertion should be converted to an if statement: As I said, I'm now thinking it's not the Assert that's faulty. If I'm right about that, it's likely that the mistaken labeling of these paths has other consequences beyond triggering this assertion.

Re: [sqlsmith] Failed assertion during partition pruning

2021-01-29 Thread Zhihong Yu
Hi, I wonder if the (failed) assertion should be converted to an if statement: diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c index fac921eea5..d646f08a07 100644 --- a/src/backend/partitioning/partprune.c +++ b/src/backend/partitioning/partprune.c @@

Re: Key management with tests

2021-01-29 Thread Stephen Frost
Greetings, * Masahiko Sawada (sawada.m...@gmail.com) wrote: > On Fri, Jan 29, 2021 at 5:22 AM Bruce Momjian wrote: > > On Thu, Jan 28, 2021 at 02:41:09PM -0500, Tom Kincaid wrote: > > > I would also like to add a "not wanted" entry for this feature on the > > > TODO list, baaed on the

Re: Proposal: Save user's original authenticated identity for logging

2021-01-29 Thread Tom Lane
Stephen Frost writes: > * Jacob Champion (pchamp...@vmware.com) wrote: >> I propose that every auth method should store the string it uses to >> identify a user -- what I'll call an "authenticated identity" -- into >> one central location in Port, after authentication succeeds but before >> any

Re: Proposal: Save user's original authenticated identity for logging

2021-01-29 Thread Stephen Frost
Greetings, * Jacob Champion (pchamp...@vmware.com) wrote: > First, the context: recently I've been digging into the use of third- > party authentication systems with Postgres. One sticking point is the > need to have a Postgres role corresponding to the third-party user > identity, which becomes

Re: Assertion fail with window function and partitioned tables

2021-01-29 Thread Jaime Casanova
On Thu, Jan 28, 2021 at 9:45 PM Jaime Casanova wrote: > > Hi, > > Just found another crash. > > Seems that commit a929e17e5a8c9b751b66002c8a89fdebdacfe194 broke something. > Attached is a minimal case and the stack trace. > Hi, Seems this is the same that Andreas reported in

Re: [sqlsmith] Failed assertion during partition pruning

2021-01-29 Thread Tom Lane
I wrote: >> What it looks like to me is that the code for setting up run-time >> partition pruning has failed to consider the possibility of nested >> partitioning: it's expecting that every partitioned table will have >> at least one direct child that is a leaf. I'm not sure though >> whether

Should we make Bitmapsets a kind of Node?

2021-01-29 Thread Tom Lane
I got annoyed (not for the first time) by the fact that the partitioned_rels field of AppendPath and MergeAppendPath is a list of Relids, i.e., Bitmapsets. This is problematic because a Bitmapset is not a type of Node, and thus a List of them is really an invalid data structure. The main

Re: Support for NSS as a libpq TLS backend

2021-01-29 Thread Jacob Champion
On Fri, 2021-01-29 at 13:57 +0100, Daniel Gustafsson wrote: > > On 21 Jan 2021, at 06:21, Michael Paquier wrote: > > I really need to study more the choide of the options chosen for > > NSS_InitContext()... But based on the docs I can read on the matter I > > think that saving nsscontext in

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-29 Thread Fujii Masao
On 2021/01/29 19:45, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 1:24 PM Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 1:17 PM Fujii Masao wrote: But if the issue is only the inconsistency of test results, we can go with the option (2)? Even with (2), we can make the test stable

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-29 Thread Alexey Kondratov
On 2021-01-28 14:42, Alexey Kondratov wrote: On 2021-01-28 00:36, Alvaro Herrera wrote: I didn't look at the patch closely enough to understand why you're trying to do something like CLUSTER, VACUUM FULL or REINDEX without holding full AccessExclusiveLock on the relation. But do keep in mind

Re: Dumping/restoring fails on inherited generated column

2021-01-29 Thread Tom Lane
Peter Eisentraut writes: > I've had another go at this, and I've found a solution that appears to > address all the issues I'm aware of. It's all very similar to the > previously discussed patches. The main difference is that previous > patches had attempted to use something like

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-29 Thread Alexander Korotkov
On Thu, Jan 21, 2021 at 11:14 PM Pavel Stehule wrote: >> Looks good, I've applied it, thanks. > > I tested last set of patches > > 1. There is no problem with patching and compilation > 2. make check-world passed > 3. build doc without problems > 4. I have not any objections against implemented

Re: LogwrtResult contended spinlock

2021-01-29 Thread Alvaro Herrera
On 2020-Aug-31, Andres Freund wrote: > Wouldn't the better fix here be to allow reading of individual members > without a lock? E.g. by wrapping each in a 64bit atomic. So I've been playing with this and I'm annoyed about having two datatypes to represent Write/Flush positions: typedef struct

Re: Allow matching whole DN from a client certificate

2021-01-29 Thread Andrew Dunstan
On 1/28/21 5:10 PM, Andrew Dunstan wrote: > >> (I'd still recommend switching to use the RFC >> flag to OpenSSL, to ease future improvements.) There should be a bunch >> of warning documentation saying not to do anything more complex unless >> you're an expert, and that the exact regular

Re: Is Recovery actually paused?

2021-01-29 Thread Yugo NAGATA
On Fri, 29 Jan 2021 16:33:32 +0530 Dilip Kumar wrote: > On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > > > On Thu, 28 Jan 2021 09:55:42 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > > > > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA

Re: Allow matching whole DN from a client certificate

2021-01-29 Thread Andrew Dunstan
On 1/29/21 8:18 AM, Daniel Gustafsson wrote: >> On 28 Jan 2021, at 23:10, Andrew Dunstan wrote: >> On 1/28/21 11:39 AM, Jacob Champion wrote: >>> Unfortunately I don't really know what that solution should look like. >>> A DSL for filtering on RDNs would be a lot of work, but it could >>>

Re: Phrase search vs. multi-lexeme tokens

2021-01-29 Thread Alexander Korotkov
On Tue, Jan 26, 2021 at 4:31 AM Neil Chen wrote: > On Mon, Jan 25, 2021 at 11:25 PM Alexander Korotkov > wrote: >> >> >> BTW, you mentioned you read the documentation. Do you think it needs >> to be adjusted accordingly to the patch? >> > > Yes, I checked section 8.11, section 9.13 and Chapter

Re: Printing backtrace of postgres processes

2021-01-29 Thread vignesh C
Thanks Bharath for your review comments. Please find my comments inline below. On Thu, Jan 28, 2021 at 7:40 PM Bharath Rupireddy wrote: > > On Thu, Jan 28, 2021 at 5:22 PM vignesh C wrote: > > Thanks for the comments, I have fixed and attached an updated patch > > with the fixes for the same. >

Re: Allow matching whole DN from a client certificate

2021-01-29 Thread Daniel Gustafsson
> On 28 Jan 2021, at 23:10, Andrew Dunstan wrote: > On 1/28/21 11:39 AM, Jacob Champion wrote: >> >> Unfortunately I don't really know what that solution should look like. >> A DSL for filtering on RDNs would be a lot of work, but it could >> potentially allow LDAP to be mapped through pg_ident

Re: Dumping/restoring fails on inherited generated column

2021-01-29 Thread Peter Eisentraut
I've had another go at this, and I've found a solution that appears to address all the issues I'm aware of. It's all very similar to the previously discussed patches. The main difference is that previous patches had attempted to use something like tbinfo->attislocal to determine whether a

Re: Support for NSS as a libpq TLS backend

2021-01-29 Thread Daniel Gustafsson
> On 29 Jan 2021, at 07:01, Michael Paquier wrote: > > On Fri, Jan 29, 2021 at 12:20:21AM +0100, Daniel Gustafsson wrote: >> SSL is admittedly an obsolete technical term, but it's one that enough people >> have decided is interchangeable with TLS that it's not a hill worth dying on >> IMHO.

Re: Is Recovery actually paused?

2021-01-29 Thread Dilip Kumar
On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > On Thu, 28 Jan 2021 09:55:42 +0530 > Dilip Kumar wrote: > > > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > > > > > > > On Wed, 27 Jan 2021 13:29:23 +0530 > > > >

Re: Single transaction in the tablesync worker?

2021-01-29 Thread Peter Smith
On Thu, Jan 28, 2021 at 9:37 PM Amit Kapila wrote: > > On Thu, Jan 28, 2021 at 12:32 PM Peter Smith wrote: > > > > On Wed, Jan 27, 2021 at 2:53 PM Amit Kapila wrote: > > > > > > On Sat, Jan 23, 2021 at 5:56 PM Amit Kapila > > > wrote: > > > > > > > > On Sat, Jan 23, 2021 at 4:55 AM Peter

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-29 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 1:24 PM Bharath Rupireddy wrote: > > On Fri, Jan 29, 2021 at 1:17 PM Fujii Masao > wrote: > > >> But if the issue is only the inconsistency of test results, > > >> we can go with the option (2)? Even with (2), we can make the test > > >> stable by removing "valid" column

Re: Single transaction in the tablesync worker?

2021-01-29 Thread Peter Smith
Hi Amit. PSA the v22 patch for the Tablesync Solution1. Differences from v21: + Patch is rebased to latest OSS HEAD @ 29/Jan. + Includes new code as suggested [ak0128] to ensure no dangling slots at Drop/AlterSubscription. + Removes the slot/origin cleanup down by process interrupt logic

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-01-29 Thread Joe Conway
On 1/29/21 12:13 AM, Ian Lawrence Barwick wrote: > 2021年1月28日(木) 17:18 Peter Eisentraut: > I'm not convinced the current behavior is wrong.  Is there some > practical use case that is affected by this behavior? > >   > I was poking around at the function with a view to using it for

Re: Is Recovery actually paused?

2021-01-29 Thread Yugo NAGATA
On Thu, 28 Jan 2021 09:55:42 +0530 Dilip Kumar wrote: > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > > > > > On Wed, 27 Jan 2021 13:29:23 +0530 > > > Dilip Kumar wrote: > > > > > > > On Wed, Jan 27, 2021 at 12:50 PM Masahiko

Re: [PoC] Non-volatile WAL buffer

2021-01-29 Thread Takashi Menjo
Hi Tomas, I'd answer your questions. (Not all for now, sorry.) > Do I understand correctly that the patch removes "regular" WAL buffers and instead writes the data into the non-volatile PMEM buffer, without writing that to the WAL segments at all (unless in archiving mode)? > Firstly, I guess

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-01-29 Thread Masahiro Ikeda
On 2021-01-27 00:14, David G. Johnston wrote: On Mon, Jan 25, 2021 at 11:56 PM Masahiro Ikeda wrote: (wal_write) The number of times WAL buffers were written out to disk via XLogWrite Thanks. I thought it's better to omit "The" and "XLogWrite" because other views' description omits

Re: Snapbuild woes followup

2021-01-29 Thread Amit Kapila
On Tue, Jan 26, 2021 at 2:18 AM Andres Freund wrote: > > Hi, > > On 2021-01-25 12:00:08 -0800, Andres Freund wrote: > > > > /* > > > >* For backward compatibility reasons this has to be stored in the > > > > wrongly > > > >* named field. Will be fixed in next major version. > > > >

[WIP] Reduce likelihood of fdw prepared statement collisions

2021-01-29 Thread Marco
Hi All, I realize using foreign data wrappers with transaction pooling may not be strictly supported, but for the most part they work. I am however occasionally noticing errors stemming from the prepared statement names created by the fdw modify code colliding between sessions/DBs. Would the

Re: Key management with tests

2021-01-29 Thread Masahiko Sawada
On Fri, Jan 29, 2021 at 5:22 AM Bruce Momjian wrote: > > On Thu, Jan 28, 2021 at 02:41:09PM -0500, Tom Kincaid wrote: > > I would also like to add a "not wanted" entry for this feature on the > > TODO list, baaed on the feature's limited usefulness, but I already > > asked about that