Re: Some RELKIND macro refactoring

2021-10-31 Thread Peter Eisentraut
Small rebase of this patch set. From a5ecfa6cfb3963ff069b591254975525ac91b2e1 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 16 Aug 2021 14:25:59 +0200 Subject: [PATCH v3 1/2] pg_dump: Remove redundant relkind checks It is checked in flagInhTables() which relkinds can have parents.

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-10-31 Thread Fujii Masao
On 2021/10/31 18:05, Etsuro Fujita wrote: Hi, As I said before [1], I’m working on $SUBJECT. Attached is a WIP patch for that. Thanks for the patch! The patch is pretty simple: if a server option added by the patch “parallel_commit” is enabled, Could you tell me why the parameter is n

Re: Non-decimal integer literals

2021-10-31 Thread Peter Eisentraut
On 28.09.21 17:30, Peter Eisentraut wrote: On 09.09.21 16:08, Vik Fearing wrote: Even without that point, this patch *is* going to break valid queries, because every one of those cases is a valid number-followed-by-identifier today, Ah, true that.  So if this does go in, we may as well add t

Re: Added schema level support for publication.

2021-10-31 Thread Masahiko Sawada
On Fri, Oct 29, 2021 at 1:35 PM Amit Kapila wrote: > > On Thu, Oct 28, 2021 at 9:55 AM vignesh C wrote: > > > > Thanks for committing the patch, please find the remaining patches attached. > > Thanks Hou Zhijie and Greg Nancarrow for sharing a few comments > > offline, I have fixed those in the a

Re: Adding CI to our tree

2021-10-31 Thread Andres Freund
Hi, Attached is an updated version of the CI patches. Changes: - more optional features are enabled on various platforms, including building with openssl on windows - added somewhat minimal, README explaining how CI can enabled in a repository - some cleanup to the windows crash reporting sup

Re: Time to drop plpython2?

2021-10-31 Thread Peter Eisentraut
On 31.10.21 19:45, Andres Freund wrote: To me it seems time to drop plpython2 support. Supporting plpython2 until ~7 years after python2 is EOL is already quite long... It'd be one thing if it were completely painless, but imo it's not. I was about to list better plpython2/3 support (including t

Re: Use -fvisibility=hidden for shared libraries

2021-10-31 Thread Andres Freund
Hi, On 2021-11-01 05:55:40 +0100, Laurenz Albe wrote: > I see that at least clang and gcc support this flag. Yes - and have for a long time. 2004 or so. > Could the reduced number of exported functions be a problem, if someone > relies on some function being exported? It may not be worth worry

Re: Missing include in be-secure-openssl.c?

2021-10-31 Thread Michael Paquier
On Sun, Oct 31, 2021 at 06:45:47PM -0400, Tom Lane wrote: > I observe that the OpenSSL docs say you are supposed to > #include > when using these functions. We are including that header in > some other modules, but not here. I speculate that we've gotten > away with that so far because of indire

Re: Added schema level support for publication.

2021-10-31 Thread Greg Nancarrow
On Fri, Oct 29, 2021 at 3:35 PM Amit Kapila wrote: > > Sawada-San, others, what do you think? Is it really useful to have such a > view? > > One point to think is if we introduce such a view then how it should > behave for schema objects? Do we need to display only schemas or > additionally all t

Re: Use -fvisibility=hidden for shared libraries

2021-10-31 Thread Laurenz Albe
On Sun, 2021-10-31 at 19:03 -0700, Andres Freund wrote: > Currently postgres builds extension shared libraries (i.e. pretty much > everything but libpq) with global visibilty. I.e. symbols that are not > static will be visible from outside the shared library. > > On the one platform where that beh

Re: pg_receivewal starting position

2021-10-31 Thread Michael Paquier
On Fri, Oct 29, 2021 at 10:13:44AM +0200, Ronan Dunklau wrote: > We could use a single query on the primary (using the primary's checkpoint > LSN > instead) but it feels a bit convoluted just to avoid a query on the standby. Cheating with pg_walfile_name() running on the primary is fine by me.

make update-po problem with USE_PGXS

2021-10-31 Thread wangsh.f...@fujitsu.com
Hi, I'm developing an extension, after modifying some source file(add some message like gettext('x')) and execute > $ make USE_PGXS=1 update-po > make: Nothing to be done for 'update-po'. I feel strange because I think *.po.new files should be created. After some research, I find in $pginst

Re: Opclass parameters of indexes lost after REINDEX CONCURRENTLY[

2021-10-31 Thread Michael Paquier
On Sat, Oct 30, 2021 at 09:26:35PM +0900, Michael Paquier wrote: > Yeah, you are right that it would be better here to use > get_attoptions() to grab a copy of each attribute's option directly > from the catalogs. We also do that for predicates and expressions. While looking again at this one thi

Re: inefficient loop in StandbyReleaseLockList()

2021-10-31 Thread Kyotaro Horiguchi
At Sun, 31 Oct 2021 16:55:01 -0400, Tom Lane wrote in > I wrote: > > Pushed the patch as given. I've not yet reviewed other list_delete_first > > callers, but I'll take a look. (I seem to remember that I did survey > > them while writing 1cff1b95a, but I evidently missed that this code > > coul

Use -fvisibility=hidden for shared libraries

2021-10-31 Thread Andres Freund
Hi, Currently postgres builds extension shared libraries (i.e. pretty much everything but libpq) with global visibilty. I.e. symbols that are not static will be visible from outside the shared library. On the one platform where that behaviour is not available, namely windows, we emulate it by ana

Re: Skipping logical replication transactions on subscriber side

2021-10-31 Thread Masahiko Sawada
On Sat, Oct 30, 2021 at 12:21 PM Amit Kapila wrote: > > On Fri, Oct 29, 2021 at 4:49 PM Amit Kapila wrote: > > > > On Fri, Oct 29, 2021 at 10:54 AM Masahiko Sawada > > wrote: > > > > > > I'm not sure it's better to drop apply worker stats after resetting > > > skip xid (i.g., after skipping the

Re: Skipping logical replication transactions on subscriber side

2021-10-31 Thread Masahiko Sawada
On Fri, Oct 29, 2021 at 8:20 PM Amit Kapila wrote: > > On Fri, Oct 29, 2021 at 10:54 AM Masahiko Sawada > wrote: > > > > On Thu, Oct 28, 2021 at 7:40 PM Amit Kapila wrote: > > > > > > On Thu, Oct 28, 2021 at 10:36 AM Masahiko Sawada > > > wrote: > > > > > > > > On Wed, Oct 27, 2021 at 7:02 PM

Re: parallel vacuum comments

2021-10-31 Thread Masahiko Sawada
On Sun, Oct 31, 2021 at 6:21 AM Andres Freund wrote: > > Hi, > > Due to bug #17245: [1] I spent a considerably amount of time looking at vacuum > related code. And I found a few things that I think could stand improvement: > > - There's pretty much no tests. This is way way too complicated feature

Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

2021-10-31 Thread Kyotaro Horiguchi
At Fri, 29 Oct 2021 22:25:04 +0530, Bharath Rupireddy wrote in > On Mon, Oct 11, 2021 at 9:55 AM Bharath Rupireddy > wrote: > > > > On Mon, Oct 11, 2021 at 8:21 AM torikoshia > > wrote: > > > If we can use debuggers, it's possible to know the memory contexts e.g. > > > using MemoryContextStat

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-10-31 Thread Mark Dilger
> On Oct 29, 2021, at 4:46 PM, Jeff Davis wrote: > > But I don't think the concept of role ownership has zero potential > confusion, either. For instance, I could certainly imagine a user A > creating a role B (and therefore owning it), and then doing "GRANT A TO > B". Is there a reason to do

Missing include in be-secure-openssl.c?

2021-10-31 Thread Tom Lane
hamerkop has been failing recently [1] with c:\\build-farm-local\\buildroot\\head\\pgsql.build\\src\\backend\\libpq\\be-secure-openssl.c(67): error C2143: \215\\\225\266\203G\203\211\201[: ')' \202\252 '(' \202\314\221O\202\311\202\240\202\350\202\334\202\271\202\361\201B [c:\\build-farm-loc

Re: row filtering for logical replication

2021-10-31 Thread Peter Smith
The v34* patch set is temporarily broken. It was impacted quite a lot by the recently committed "schema publication" patch [1]. We are actively fixing the full v34* patch set and will re-post it here as soon as the re-base hurdles can be overcome. Meanwhile, the small tab-complete patch (which i

Re: Add additional information to src/test/ssl/README

2021-10-31 Thread Tom Lane
Daniel Gustafsson writes: > On 31 Oct 2021, at 19:28, Tom Lane wrote: >> Here's a quick stab at rearranging src/test/perl/README so that the >> initial section is all how-to-run-the-tests info, and advice about >> writing new tests starts after that. Your point about PG_TEST_NOCLEAN >> could be

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Michael Banck
Hi, On Sun, Oct 31, 2021 at 01:16:33PM -0700, Andres Freund wrote: > On 2021-10-31 15:43:57 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2021-10-31 10:59:19 -0400, Tom Lane wrote: > > >> No DBA would be likely to consider it as anything but log spam. > > > > > I don't agree at all.

Re: Added schema level support for publication.

2021-10-31 Thread Tomas Vondra
Hi, On 10/28/21 04:41, Amit Kapila wrote: On Mon, Oct 25, 2021 at 3:09 PM Amit Kapila wrote: On Mon, Oct 25, 2021 at 1:11 PM vignesh C wrote: I have fixed this in the v47 version attached. Thanks, the first patch in the series "Allow publishing the tables of schema." looks good to me. U

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Tomas Vondra
On 10/31/21 21:16, Andres Freund wrote: Hi, On 2021-10-31 15:43:57 -0400, Tom Lane wrote: Andres Freund writes: On 2021-10-31 10:59:19 -0400, Tom Lane wrote: No DBA would be likely to consider it as anything but log spam. I don't agree at all. No postgres instance should be run without lo

Re: inefficient loop in StandbyReleaseLockList()

2021-10-31 Thread Tom Lane
I wrote: > Pushed the patch as given. I've not yet reviewed other list_delete_first > callers, but I'll take a look. (I seem to remember that I did survey > them while writing 1cff1b95a, but I evidently missed that this code > could be dealing with a list long enough to be problematic.) I looked

small change to comment for ATExecDetachPartition

2021-10-31 Thread Zhihong Yu
Hi, I was going over: commit 71f4c8c6f74ba021e55d35b1128d22fb8c6e1629 Author: Alvaro Herrera Date: Thu Mar 25 18:00:28 2021 -0300 ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY It seems there were some missing words in one of the comments. See patch attached. Cheers detach-partitio

Re: logical decoding and replication of sequences

2021-10-31 Thread Tomas Vondra
On 9/25/21 22:05, Hannu Krosing wrote: Just a note for some design decisions 1) By default, sequences are treated non-transactionally, i.e. sent to the output plugin right away. If our aim is just to make sure that all user-visible data in *transactional* tables is consistent with sequence s

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Jan Wieck
On 10/31/21 16:16, Andres Freund wrote: Hi, On 2021-10-31 15:43:57 -0400, Tom Lane wrote: Andres Freund writes: > On 2021-10-31 10:59:19 -0400, Tom Lane wrote: >> No DBA would be likely to consider it as anything but log spam. > I don't agree at all. No postgres instance should be run without

Re: inefficient loop in StandbyReleaseLockList()

2021-10-31 Thread Andres Freund
Hi, On 2021-10-31 15:38:15 -0400, Tom Lane wrote: > Yeah, there's no expectation that this data structure needs to be kept > consistent after an error; and I'm not real sure that the existing > code could claim to satisfy such a requirement if we did need it. To be clear, I was making that commen

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Andres Freund
Hi, On 2021-10-31 15:43:57 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2021-10-31 10:59:19 -0400, Tom Lane wrote: > >> No DBA would be likely to consider it as anything but log spam. > > > I don't agree at all. No postgres instance should be run without > > log_checkpoints enabled. Per

Re: Add additional information to src/test/ssl/README

2021-10-31 Thread Daniel Gustafsson
> On 31 Oct 2021, at 19:28, Tom Lane wrote: > > I wrote: >> Daniel Gustafsson writes: >>> Wouldn't it make more sense to start collecting troubleshooting advice in >>> src/test/perl/README and instead refer to that in the boilerplate? I notice >>> that we don't document for example PG_TEST_NOCL

Re: Time to drop plpython2?

2021-10-31 Thread Thomas Munro
On Mon, Nov 1, 2021 at 8:05 AM Andres Freund wrote: > > Yeah, it's a bit hard to justify continuing support in HEAD. +1, it's dropping out of distros, it'd be unsupportable without unreasonable effort.

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Tom Lane
Andres Freund writes: > On 2021-10-31 10:59:19 -0400, Tom Lane wrote: >> No DBA would be likely to consider it as anything but log spam. > I don't agree at all. No postgres instance should be run without > log_checkpoints enabled. Performance is poor if checkpoints are > triggered by anything but

Re: inefficient loop in StandbyReleaseLockList()

2021-10-31 Thread Tom Lane
"Bossart, Nathan" writes: > On 10/28/21, 11:53 PM, "Michael Paquier" wrote: >> Actually, as the list of recovery locks is saved in TopMemoryContext, >> wouldn't it be better to keep a per-cell deletion of the list, which >> would mean that we'd better do the operation in the reverse order to >> m

Re: Time to drop plpython2?

2021-10-31 Thread Andres Freund
Hi, On 2021-10-31 14:49:22 -0400, Tom Lane wrote: > Andres Freund writes: > > To me it seems time to drop plpython2 support. Supporting plpython2 > > until ~7 years after python2 is EOL is already quite long... It'd be one > > thing if it were completely painless, but imo it's not. > > 7 years?

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Andres Freund
Hi, On 2021-10-31 10:59:19 -0400, Tom Lane wrote: > The general policy at the moment is that a normally-functioning server > should emit *no* log traffic by default (other than a few messages > at startup and shutdown). log_checkpoints is a particularly poor > candidate for an exception to that p

Re: emit recovery stats via a new file or a new hook

2021-10-31 Thread Andres Freund
Hi, On 2021-10-31 19:06:07 +0530, Bharath Rupireddy wrote: > It is sometimes super important to be able to answer customer > questions like: What was the total time taken by the last recovery of > the server? What was the time taken by each phase of recovery/redo > processing of the startup proces

Re: Time to drop plpython2?

2021-10-31 Thread Tom Lane
Andres Freund writes: > To me it seems time to drop plpython2 support. Supporting plpython2 > until ~7 years after python2 is EOL is already quite long... It'd be one > thing if it were completely painless, but imo it's not. 7 years? Oh, you're envisioning dropping plpython2 in HEAD but keeping

Time to drop plpython2?

2021-10-31 Thread Andres Freund
Hi, The last release of python2 was 2.7.18 released 2020-04-20, with support already having ended before that 2020-01-01. To me it seems time to drop plpython2 support. Supporting plpython2 until ~7 years after python2 is EOL is already quite long... It'd be one thing if it were completely painle

Re: Add additional information to src/test/ssl/README

2021-10-31 Thread Tom Lane
I wrote: > Daniel Gustafsson writes: >> Wouldn't it make more sense to start collecting troubleshooting advice in >> src/test/perl/README and instead refer to that in the boilerplate? I notice >> that we don't document for example PG_TEST_NOCLEAN anywhere (which admittedly >> is my fault), a trub

Logical insert/update/delete WAL records for custom table AMs

2021-10-31 Thread Jeff Davis
Attached is a WIP patch to add new WAL records to represent a logical insert, update, or delete. These records do not do anything at REDO time, they are only processed during logical decoding/replication. These are intended to be used by a custom table AM, like my columnar compression extension[0]

Re: Feature request for adoptive indexes

2021-10-31 Thread Tomas Vondra
On 10/31/21 16:48, Pavel Borisov wrote:   4 columns: 106 ms   6 columns: 109 ms So there's like 3% difference between the two cases, and even that might be just noise. This is consistent with the two indexes being about the same size. I also don't think we can get g

Re: Feature request for adoptive indexes

2021-10-31 Thread Pavel Borisov
> > 4 columns: 106 ms > 6 columns: 109 ms > > So there's like 3% difference between the two cases, and even that might > be just noise. This is consistent with the two indexes being about the > same size. > I also don't think we can get great speedup in the mentioned case, so it is not urgently

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Tom Lane
Bharath Rupireddy writes: > How about we enable it out of the box? No. The general policy at the moment is that a normally-functioning server should emit *no* log traffic by default (other than a few messages at startup and shutdown). log_checkpoints is a particularly poor candidate for an exce

should we enable log_checkpoints out of the box?

2021-10-31 Thread Bharath Rupireddy
Hi, It seems the checkpoint stats, that are emitted to server logs when the GUC log_checkpoints is enabled, are so important that a postgres database provider would ever want to disable the GUC. Right now, the GUC is disabled by default and a postgres database provider needs to enable it explicitl

emit recovery stats via a new file or a new hook

2021-10-31 Thread Bharath Rupireddy
Hi, It is sometimes super important to be able to answer customer questions like: What was the total time taken by the last recovery of the server? What was the time taken by each phase of recovery/redo processing of the startup process? Why did the recovery take so long? We've encountered these q

Synchronizing slots from primary to standby

2021-10-31 Thread Peter Eisentraut
I want to reactivate $subject. I took Petr Jelinek's patch from [0], rebased it, added a bit of testing. It basically works, but as mentioned in [0], there are various issues to work out. The idea is that the standby runs a background worker to periodically fetch replication slot information

postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-10-31 Thread Etsuro Fujita
Hi, As I said before [1], I’m working on $SUBJECT. Attached is a WIP patch for that. The patch is pretty simple: if a server option added by the patch “parallel_commit” is enabled, 1) asynchronously send COMMIT TRANSACTION (RELEASE SAVEPOINT) to all remote servers involved in a local (sub)transa