Re: Multiple Query IDs for a rewritten parse tree

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 05:51:56PM +0100, Dmitry Dolgov wrote: > > On Fri, Jan 21, 2022 at 11:33:22AM +0500, Andrey V. Lepikhov wrote: > > On 1/9/22 5:49 AM, Tom Lane wrote: > > > The idea I'd been vaguely thinking about is to allow attaching a list > > > of query-hash nodes to a Query, where

Re: GUC flags

2022-01-28 Thread Michael Paquier
On Thu, Jan 27, 2022 at 10:36:21PM -0600, Justin Pryzby wrote: > Maybe you misunderstood - I'm not reading the file specified by > current_setting('config_file'). Rather, I'm reading > tmp_check/data/postgresql.conf, which is copied from the sample conf. > Do you see an issue with that ? Yes, as

Re: make MaxBackends available in _PG_init

2022-01-28 Thread Michael Paquier
On Fri, Jan 28, 2022 at 08:33:42PM -0800, Nathan Bossart wrote: > Hm. Perhaps we should understand the full scope of the problem first. > What else besides MaxBackends and the shared memory GUCs won't be properly > initialized when the shared_preload_libraries' _PG_init() functions are > called?

Re: Schema variables - new implementation for Postgres 15

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 07:51:08AM +0100, Pavel Stehule wrote: > st 26. 1. 2022 v 8:23 odesílatel Julien Rouhaud napsal: > > > + The ON TRANSACTION END RESET > > + clause causes the session variable to be reset to its default value > > when > > + the transaction is committed

Re: make MaxBackends available in _PG_init

2022-01-28 Thread Nathan Bossart
On Sat, Jan 29, 2022 at 11:19:12AM +0900, Michael Paquier wrote: > On Thu, Jan 27, 2022 at 10:18:15AM -0800, Nathan Bossart wrote: >> Alright. I think the comment adjustments still apply, so I split those out >> to a new patch. > > Looks fine after a second look, so applied. Thanks! > As of

Re: refactoring basebackup.c

2022-01-28 Thread Jeevan Ladhe
On Sat, Jan 29, 2022 at 1:20 AM Robert Haas wrote: > On Fri, Jan 28, 2022 at 12:48 PM Jeevan Ladhe > wrote: > > I have attached the latest rebased version of the LZ4 server-side > compression > > patch on the recent commits. This patch also introduces the compression > level > > and adds a tap

Re: Why is INSERT-driven autovacuuming based on pg_class.reltuples?

2022-01-28 Thread Peter Geoghegan
On Thu, Jan 27, 2022 at 11:22 PM Laurenz Albe wrote: > What would you suggest instead? pg_stat_all_tables.n_live_tup? I'm not sure, except that I assume that it'll have to come from the statistics collector, not from pg_class. I think that this bug stemmed from the fact that

Re: Bugs in pgoutput.c

2022-01-28 Thread Amit Kapila
On Thu, Jan 6, 2022 at 3:42 AM Tom Lane wrote: > > Commit 6ce16088b caused me to look at pgoutput.c's handling of > cache invalidations, and I was pretty appalled by what I found. > > * rel_sync_cache_relation_cb does the wrong thing when called for > a cache flush (i.e., relid == 0). Instead of

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 10:20:07AM -0800, Andres Freund wrote: > > On 2022-01-28 21:56:36 +0800, Julien Rouhaud wrote: > > I think having a new option for vacuumdb is the right move. > > Can't we pass the option via the connection string, e.g. something > PGOPTIONS='-c

Re: Printing backtrace of postgres processes

2022-01-28 Thread vignesh C
On Fri, Jan 28, 2022 at 1:54 PM Bharath Rupireddy wrote: > > On Thu, Jan 27, 2022 at 10:45 AM vignesh C wrote: > > > > On Wed, Jan 26, 2022 at 11:07 AM Bharath Rupireddy > > wrote: > > > > > > On Tue, Jan 25, 2022 at 12:00 PM vignesh C wrote: > > > > Thanks for the comments, attached v17 patch

Re: make MaxBackends available in _PG_init

2022-01-28 Thread Michael Paquier
On Thu, Jan 27, 2022 at 10:18:15AM -0800, Nathan Bossart wrote: > Alright. I think the comment adjustments still apply, so I split those out > to a new patch. Looks fine after a second look, so applied. As of the issues of this thread, we really have two things to think about: 1) How do we want

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-01-28 Thread Michael Paquier
On Fri, Jan 28, 2022 at 06:32:27PM +0900, Kyotaro Horiguchi wrote: > End-of-recovery checkpoint is requested as CHECKPOINT_WAIT, which > seems to me to mean the state is always DB_IN_ARCHIVE_RECOVERY while > the checkpoint is running? With the patch, yes, we would keep the control file under

Re: pg_upgrade should truncate/remove its logs before running

2022-01-28 Thread Michael Paquier
On Fri, Jan 28, 2022 at 06:27:29PM -0500, Andrew Dunstan wrote: > I have committed this. But it will take time to get every buildfarm own > to upgrade. Thanks for that. > I will try to make a new release ASAP. And thanks for that, as well. -- Michael signature.asc Description: PGP signature

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-28 Thread Andres Freund
On 2022-01-28 16:36:32 -0800, Andres Freund wrote: > On 2022-01-28 18:43:57 -0500, James Coleman wrote: > > Alternatively I see pg_attribute_aligned, but that's not defined > > (AFAICT) on clang, for example, so I'm not sure that'd be acceptable? > > clang should have it (it defines __GNUC__).

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-28 Thread Andres Freund
Hi, On 2022-01-28 18:43:57 -0500, James Coleman wrote: > Alternatively I see pg_attribute_aligned, but that's not defined > (AFAICT) on clang, for example, so I'm not sure that'd be acceptable? clang should have it (it defines __GNUC__). The problem would be msvc, I think. Not sure if there's a

Re: row filtering for logical replication

2022-01-28 Thread Andres Freund
Hi, Are there any recent performance evaluations of the overhead of row filters? I think it'd be good to get some numbers comparing: 1) $workload with master 2) $workload with patch, but no row filters 3) $workload with patch, row filter matching everything 4) $workload with patch, row filter

Re: warn if GUC set to an invalid shared library

2022-01-28 Thread Justin Pryzby
Thanks for loooking On Fri, Jan 28, 2022 at 11:36:20PM +, Cary Huang wrote: > This is fine as this is what these patches are aiming to provide. However, > when I try to restart the server, it fails to start because abc.so and xyz.so > do not exist. Setting the parameters

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-28 Thread James Coleman
On Thu, Jan 20, 2022 at 8:15 AM James Coleman wrote: > > On Wed, Jan 19, 2022 at 10:12 PM Andres Freund wrote: > > > > Hi, > > > > On 2022-01-19 21:23:12 -0500, James Coleman wrote: > > > { oid => '3537', descr => 'get identification of SQL object', > > > diff --git a/src/include/storage/proc.h

Re: archive modules

2022-01-28 Thread Nathan Bossart
On Fri, Jan 28, 2022 at 03:20:41PM -0500, Robert Haas wrote: > On Fri, Jan 28, 2022 at 3:01 PM Nathan Bossart > wrote: >> I discussed the two main deficiencies I'm aware of with basic_archive >> earlier [0]. The first one is the issue with "incovenient" server crashes >> (mentioned below). > >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Andres Freund
Hi, On 2021-12-07 20:06:22 +0530, Bharath Rupireddy wrote: > One concern is that we don't want to increase the size of pg_controldata by > more than the typical block size (of 8K) to avoid any torn-writes. The limit is 512 bytes (a disk sector), not 8K. There are plenty devices with 4K sectors

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Andres Freund
Hi, On 2021-12-08 03:04:23 +0100, Tomas Vondra wrote: > On 12/8/21 02:54, Bharath Rupireddy wrote: > >> I'm not sure about adding it to control data, though. That doesn't seem > >> like a very good match for something that's mostly for monitoring. > > > > Having it in the control data file

Re: warn if GUC set to an invalid shared library

2022-01-28 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested Hello I tested the patches on master branch on Ubuntu 18.04 and

Re: support for MERGE

2022-01-28 Thread Erik Rijkers
Op 28-01-2022 om 21:27 schreef Alvaro Herrera: MERGE, v10. I am much more comfortable with this version; I have removed a bunch of temporary hacks and cleaned up the interactions with table AM and executor, which is something that had been bothering me for a while. The complete set of changes

Re: pg_upgrade should truncate/remove its logs before running

2022-01-28 Thread Andrew Dunstan
On 1/28/22 08:42, Michael Paquier wrote: > On Wed, Jan 26, 2022 at 11:00:28AM +0900, Michael Paquier wrote: >> Bleh. This would point to the old data directory, so this needs to be >> "$self->{pgsql}/src/bin/pg_upgrade/tmp_check/data/pg_upgrade_output.d/log/*.log" >> to point to the upgraded

Re: support for MERGE

2022-01-28 Thread Justin Pryzby
On Fri, Jan 28, 2022 at 05:27:37PM -0300, Alvaro Herrera wrote: > The one thing I'm a bit bothered about is the fact > that we expose a lot of executor functions previously static. I am now > wondering if it would be better to move the MERGE executor support > functions into nodeModifyTable.c,

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-28 Thread Andres Freund
Hi, On 2022-01-24 16:38:54 +0400, Pavel Borisov wrote: > +64-bit Transaction ID's (XID) > += > + > +A limited number (N = 2^32) of XID's required to do vacuum freeze to prevent > +wraparound every N/2 transactions. This causes performance degradation due > +to the need

Re: support for MERGE

2022-01-28 Thread Zhihong Yu
On Fri, Jan 28, 2022 at 2:19 PM Andres Freund wrote: > Hi, > > On 2022-01-28 17:27:37 -0300, Alvaro Herrera wrote: > > MERGE, v10. I am much more comfortable with this version; I have > > removed a bunch of temporary hacks and cleaned up the interactions with > > table AM and executor, which is

Re: support for MERGE

2022-01-28 Thread Andres Freund
Hi, On 2022-01-28 17:27:37 -0300, Alvaro Herrera wrote: > MERGE, v10. I am much more comfortable with this version; I have > removed a bunch of temporary hacks and cleaned up the interactions with > table AM and executor, which is something that had been bothering me for > a while. The complete

Re: archive modules

2022-01-28 Thread Robert Haas
On Fri, Jan 28, 2022 at 3:01 PM Nathan Bossart wrote: > I discussed the two main deficiencies I'm aware of with basic_archive > earlier [0]. The first one is the issue with "incovenient" server crashes > (mentioned below). Seems easy enough to rectify, if it's just a matter of

Re: archive modules

2022-01-28 Thread Nathan Bossart
On Fri, Jan 28, 2022 at 02:06:50PM -0500, Robert Haas wrote: > I've committed 0001 now. I don't see anything particularly wrong with > the rest of this either, but here are a few comments: Thanks! > - I wonder whether it might be better to promote the basic archiving > module to contrib (as

Re: refactoring basebackup.c

2022-01-28 Thread Robert Haas
On Fri, Jan 28, 2022 at 12:48 PM Jeevan Ladhe wrote: > I have attached the latest rebased version of the LZ4 server-side compression > patch on the recent commits. This patch also introduces the compression level > and adds a tap test. In view of this morning's commit of

Re: archive modules

2022-01-28 Thread Robert Haas
On Thu, Jan 13, 2022 at 2:38 PM Bossart, Nathan wrote: > Here is another rebase for cfbot. I've committed 0001 now. I don't see anything particularly wrong with the rest of this either, but here are a few comments: - I wonder whether it might be better to promote the basic archiving module to

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2022-01-28 Thread Andres Freund
Hi, On 2022-01-28 21:56:36 +0800, Julien Rouhaud wrote: > I think having a new option for vacuumdb is the right move. Can't we pass the option via the connection string, e.g. something PGOPTIONS='-c binary_upgrade_mode=true'? That seems to scale better than to add it gradually to multiple tools.

Re: refactoring basebackup.c

2022-01-28 Thread Jeevan Ladhe
Hi Robert, I have attached the latest rebased version of the LZ4 server-side compression patch on the recent commits. This patch also introduces the compression level and adds a tap test. Also, while adding the lz4 case in the pg_verifybackup/t/008_untar.pl, I found an unused variable

Re: Server-side base backup: why superuser, not pg_write_server_files?

2022-01-28 Thread Dagfinn Ilmari Mannsåker
On Fri, 28 Jan 2022, at 17:33, Robert Haas wrote: > LGTM. Committed. Thanks! - ilmari

Re: Server-side base backup: why superuser, not pg_write_server_files?

2022-01-28 Thread Robert Haas
On Fri, Jan 28, 2022 at 12:16 PM Dagfinn Ilmari Mannsåker wrote: > Or now. Patch attached. LGTM. Committed. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Server-side base backup: why superuser, not pg_write_server_files?

2022-01-28 Thread Dagfinn Ilmari Mannsåker
Dagfinn Ilmari Mannsåker writes: > Robert Haas writes: > >> On Fri, Jan 28, 2022 at 5:58 AM Dagfinn Ilmari Mannsåker >> wrote: >>> I just noticed that the new server-side base backup feature requires >>> superuser privileges (which is only documented in the pg_basebackup >>> manual, not in the

Re: Multiple Query IDs for a rewritten parse tree

2022-01-28 Thread Dmitry Dolgov
> On Fri, Jan 21, 2022 at 11:33:22AM +0500, Andrey V. Lepikhov wrote: > On 1/9/22 5:49 AM, Tom Lane wrote: > > The idea I'd been vaguely thinking about is to allow attaching a list > > of query-hash nodes to a Query, where each node would contain a "tag" > > identifying the specific hash

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 08:21:52PM +0530, Bharath Rupireddy wrote: > > I don't think we need to change pg_upgrade's ControlData controldata; > structure as the information may not be needed there and the while > loop there specifically parses/searches for the required > pg_controldata output

Re: Support tab completion for upper character inputs in psql

2022-01-28 Thread Tom Lane
"tanghy.f...@fujitsu.com" writes: > I did some tests on it and here are something cases I feel we need to confirm > whether they are suitable. > 1) postgres=# create table atest(id int, "iD" int, "ID" int); > 2) CREATE TABLE > 3) postgres=# alter table atest rename i[TAB] > 4) id"iD" > 5)

Re: Remove extra includes of "access/xloginsert.h" when "access/xlog.h" is included

2022-01-28 Thread Alvaro Herrera
On 2022-Jan-28, Bharath Rupireddy wrote: > Hi, > > It seems like there are some instances where xloginsert.h is included > right after xlog.h but xlog.h has already included xloginsert.h. > Unless I'm missing something badly, we can safely remove including > xloginsert.h after xlog.h. Attempting

Re: Server-side base backup: why superuser, not pg_write_server_files?

2022-01-28 Thread Dagfinn Ilmari Mannsåker
Robert Haas writes: > On Fri, Jan 28, 2022 at 5:58 AM Dagfinn Ilmari Mannsåker > wrote: >> I just noticed that the new server-side base backup feature requires >> superuser privileges (which is only documented in the pg_basebackup >> manual, not in the streaming replication protocol

Re: Support for NSS as a libpq TLS backend

2022-01-28 Thread Daniel Gustafsson
> On 28 Jan 2022, at 15:30, Robert Haas wrote: > > On Fri, Jan 28, 2022 at 9:08 AM Daniel Gustafsson wrote: >>> Kinda makes me question the wisdom of starting to depend on NSS. When >>> openssl >>> docs are vastly outshining a library's, that library really should start to >>> ask itself some

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Bharath Rupireddy
On Fri, Jan 28, 2022 at 2:20 PM Julien Rouhaud wrote: > PG_CONTROL_VERSION is different from catversion. You should update it in this > patch. My bad. Updated it. > But Horiguchi-san was also mentioning that pg_upgrade/controldata.c needs some > modifications if you change the format (thus the

Re: A test for replay of regression tests

2022-01-28 Thread Andrew Dunstan
On 1/27/22 18:24, Thomas Munro wrote: > On Fri, Jan 28, 2022 at 12:03 PM Andres Freund wrote: >> Revert "graceful shutdown" changes for Windows, in back branches only. > FTR I'm actively working on a fix for that one for master now (see > that other thread where the POC survived Alexander's

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-28 Thread Pavel Borisov
> > Did you consider using 4 bytes for pd_xid_base and another 4 bytes for > (pd_xid_base/pd_multi_base)? This would allow calculating XMIN/XMAX > as: > > XMIN = (t_min_extra_bits << 32) | t_xmin > XMAX = (t_max_extra_bits << 32) | t_xmax > > ... and save 8 extra bytes in the pd_special area. Or

Re: warn if GUC set to an invalid shared library

2022-01-28 Thread Robert Haas
On Tue, Dec 28, 2021 at 12:45 PM Justin Pryzby wrote: > 0002 adds context when failing to start. > > 2021-12-27 17:01:12.996 CST postmaster[1403] WARNING: could not load > library: $libdir/plugins/asdf: cannot open shared object file: No such file > or directory > 2021-12-27

Re: Make relfile tombstone files conditional on WAL level

2022-01-28 Thread Dilip Kumar
On Wed, Jan 19, 2022 at 10:37 AM Dilip Kumar wrote: > > On Thu, Jan 6, 2022 at 7:22 PM Robert Haas wrote: >> >> On Thu, Jan 6, 2022 at 3:47 AM Thomas Munro wrote: >> > Another problem is that relfilenodes are normally allocated with >> > GetNewOidWithIndex(), and initially match a relation's

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-28 Thread Aleksander Alekseev
Hi Pavel, > Please feel free to discuss readme and your opinions on the current patch and > proposed changes [1]. Just a quick question about this design choice: > On-disk tuple format remains unchanged. 32-bit t_xmin and t_xmax store the > lower parts of 64-bit XMIN and XMAX values. Each heap

Re: Support for NSS as a libpq TLS backend

2022-01-28 Thread Robert Haas
On Fri, Jan 28, 2022 at 9:08 AM Daniel Gustafsson wrote: > > Kinda makes me question the wisdom of starting to depend on NSS. When > > openssl > > docs are vastly outshining a library's, that library really should start to > > ask itself some hard questions. Yeah, OpenSSL is very poor, so being

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade*

2022-01-28 Thread Julien Rouhaud
On Fri, Jan 28, 2022 at 03:06:57PM +0100, Denis Laxalde wrote: > Julien Rouhaud a écrit : > > I think having a new option for vacuumdb is the right move. > > > > It seems unlikely that any cron or similar on the host will try to run some > > concurrent vacuumdb, but we still have to enforce that

Remove extra includes of "access/xloginsert.h" when "access/xlog.h" is included

2022-01-28 Thread Bharath Rupireddy
Hi, It seems like there are some instances where xloginsert.h is included right after xlog.h but xlog.h has already included xloginsert.h. Unless I'm missing something badly, we can safely remove including xloginsert.h after xlog.h. Attempting to post a patch to remove the extra xloginsert.h

Re: Support escape sequence for cluster_name in postgres_fdw.application_name

2022-01-28 Thread Robert Haas
On Thu, Jan 27, 2022 at 3:10 AM Kyotaro Horiguchi wrote: > Is the reason for 'C' in upper-case to avoid possible conflict with > 'c' of log_line_prefix? I'm not sure that preventive measure is worth > doing. Looking the escape-sequence spec alone, it seems to me rather > strange that an

Re: Support for NSS as a libpq TLS backend

2022-01-28 Thread Daniel Gustafsson
>>> Can we propose a patch to document them? Don't want to get bitten by this >>> suddenly changing... >> >> I can certainly propose something on their mailinglist, but I unfortunately >> wouldn't get my hopes up too high as NSS and documentation aren't exactly >> best >> friends (the in-tree

Re: Parameter for planner estimate of recursive queries

2022-01-28 Thread Robert Haas
On Tue, Jan 25, 2022 at 4:44 AM Peter Eisentraut wrote: > On the one hand, this smells like a planner hint. But on the other > hand, it doesn't look like we will come up with proper graph-aware > selectivity estimation system any time soon, so just having all graph > OLTP queries suck until then

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2022-01-28 Thread Denis Laxalde
Julien Rouhaud a écrit : I think having a new option for vacuumdb is the right move. It seems unlikely that any cron or similar on the host will try to run some concurrent vacuumdb, but we still have to enforce that only the one executed by pg_upgrade can succeed. I guess it could be an

Re: Unlogged relations and WAL-logging

2022-01-28 Thread Robert Haas
On Thu, Jan 27, 2022 at 2:32 PM Heikki Linnakangas wrote: > Unlogged relations are not WAL-logged, but creating the init-fork is. > There are a few things around that seem sloppy: > > 1. In index_build(), we do this: > > >*/ > > if (indexRelation->rd_rel->relpersistence ==

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 11:02:46AM +0100, Denis Laxalde wrote: > > I tried that simple change first: > > diff --git a/src/backend/access/transam/xlog.c > b/src/backend/access/transam/xlog.c > index dfe2a0bcce..8feab0cb96 100644 > --- a/src/backend/access/transam/xlog.c > +++

Re: row filtering for logical replication

2022-01-28 Thread Alvaro Herrera
I just pushed a change to tab-complete because of a comment in the column-list patch series. I checked and your v72-0002 does not conflict, but it doesn't fully work either; AFAICT you'll have to change it so that the WHERE clause appears in the COMPLETE_WITH(",") line I just added. As far as I

Re: Server-side base backup: why superuser, not pg_write_server_files?

2022-01-28 Thread Robert Haas
On Fri, Jan 28, 2022 at 5:58 AM Dagfinn Ilmari Mannsåker wrote: > I just noticed that the new server-side base backup feature requires > superuser privileges (which is only documented in the pg_basebackup > manual, not in the streaming replication protocol specification). > > Isn't this the kind

Re: refactoring basebackup.c

2022-01-28 Thread Robert Haas
On Fri, Jan 28, 2022 at 3:54 AM Dipesh Pandit wrote: > Thanks. This makes sense. > > +#ifdef HAVE_LIBZ > + /* > +* If the user has requested a server compressed archive along with > archive > +* extraction at client then we need to decompress it. > +*/ > + if (format == 'p' &&

Re: pg_upgrade should truncate/remove its logs before running

2022-01-28 Thread Michael Paquier
On Wed, Jan 26, 2022 at 11:00:28AM +0900, Michael Paquier wrote: > Bleh. This would point to the old data directory, so this needs to be > "$self->{pgsql}/src/bin/pg_upgrade/tmp_check/data/pg_upgrade_output.d/log/*.log" > to point to the upgraded cluster. Please note that I have sent a patch to

Re: Parameter for planner estimate of recursive queries

2022-01-28 Thread Hamid Akhtar
On Tue, 25 Jan 2022 at 14:44, Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 31.12.21 15:10, Simon Riggs wrote: > >> The factor 10 is a reasonably safe assumption and helps avoid worst > >> case behavior in bigger graph queries. However, the factor 10 is way > >> too large for

Re: UNIQUE null treatment option

2022-01-28 Thread Pavel Borisov
> > Makes sense. Here is an updated patch with this change. > > I didn't end up renaming anynullkeys. I came up with names like > "anyalwaysdistinctkeys", but in the end that felt too abstract, and > moreover, it would require rewriting a bunch of code comments that refer > to null values in

Re: Logical replication timeout problem

2022-01-28 Thread Fabrice Chapuis
Thanks for your new fix Wang. TimestampTz ping_time = TimestampTzPlusMilliseconds(sendTime, wal_sender_timeout / 2); shouldn't we use receiver_timeout in place of wal_sender_timeout because de problem comes from the consummer. On Wed, Jan 26, 2022 at 4:37 AM wangw.f...@fujitsu.com <

Re: BeginCopyTo - remove switching to old memory context in between COPY TO command processing

2022-01-28 Thread Michael Paquier
On Fri, Jan 28, 2022 at 03:41:11PM +0530, Bharath Rupireddy wrote: > While reviewing patch at [1], it has been found that the memory > context switch to oldcontext from cstate->copycontext in between > BeginCopyTo is not correct because the intention of the copycontext is > to use it through the

Re: BeginCopyTo - remove switching to old memory context in between COPY TO command processing

2022-01-28 Thread Japin Li
On Fri, 28 Jan 2022 at 18:11, Bharath Rupireddy wrote: > Hi, > > While reviewing patch at [1], it has been found that the memory > context switch to oldcontext from cstate->copycontext in between > BeginCopyTo is not correct because the intention of the copycontext is > to use it through the

Server-side base backup: why superuser, not pg_write_server_files?

2022-01-28 Thread Dagfinn Ilmari Mannsåker
Hi Hackers, I just noticed that the new server-side base backup feature requires superuser privileges (which is only documented in the pg_basebackup manual, not in the streaming replication protocol specification). Isn't this the kind of thing the pg_write_server_files role was created for, so

Re: refactoring basebackup.c

2022-01-28 Thread tushar
On 1/27/22 11:12 PM, Robert Haas wrote: Well what's weird here is that you are using both --gzip and also --compress. Those both control the same behavior, so it's a surprising idea to specify both. But I guess if someone does, we should make the second one fully override the first one. Here's a

BeginCopyTo - remove switching to old memory context in between COPY TO command processing

2022-01-28 Thread Bharath Rupireddy
Hi, While reviewing patch at [1], it has been found that the memory context switch to oldcontext from cstate->copycontext in between BeginCopyTo is not correct because the intention of the copycontext is to use it through the copy command processing. It looks like a thinko from the commit c532d1

Re: TAP test to cover "EndOfLogTLI != replayTLI" case

2022-01-28 Thread Amul Sul
On Thu, Jan 27, 2022 at 12:01 PM Michael Paquier wrote: > > On Thu, Jan 20, 2022 at 12:13:08PM +0530, Amul Sul wrote: > > I found the cause for the test failing on window -- is due to the > > custom archive command setting which wasn't setting the correct window > > archive directory path. > >

Re: row filtering for logical replication

2022-01-28 Thread Greg Nancarrow
On Fri, Jan 28, 2022 at 2:26 PM houzj.f...@fujitsu.com wrote: > > Attach the V72 patch set which did the above changes. > Thanks for updating the patch set. One thing I noticed, in the patch commit comment it says: Psql commands \dRp+ and \d will display any row filters. However, "\d" by

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2022-01-28 Thread Denis Laxalde
Hi, Julien Rouhaud a écrit : On Wed, 27 Jan 2021 11:25:11 +0100 Denis Laxalde wrote: Andres Freund a écrit : b) when in binary upgrade mode / -b, error out on all wal writes in sessions that don't explicitly set a session-level GUC to allow writes. It should be enough to add an

Re: Suppressing useless wakeups in walreceiver

2022-01-28 Thread Thomas Munro
On Fri, Jan 28, 2022 at 8:26 PM Kyotaro Horiguchi wrote: > At Thu, 27 Jan 2022 23:50:04 +1300, Thomas Munro > wrote in > > While working on WaitEventSet-ifying various codepaths, I found it > > strange that walreceiver wakes up 10 times per second while idle. > > Here's a draft patch to compute

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-01-28 Thread Kyotaro Horiguchi
At Thu, 27 Jan 2022 10:47:20 -0800, Nathan Bossart wrote in > On Thu, Jan 27, 2022 at 02:06:40PM +0900, Michael Paquier wrote: > > So, I have been checking this idea in details, and spotted what looks > > like one issue in CreateRestartPoint(), as of: > > /* > > * Update pg_control,

Re: Add header support to text format and matching feature

2022-01-28 Thread Peter Eisentraut
On 31.12.21 18:36, Rémi Lapeyre wrote: Here’s an updated version of the patch that takes into account the changes in d1029bb5a2. The actual code is the same as v10 which was already marked as ready for committer. I have committed the 0001 patch. I will work on the 0002 patch next. I notice

Re: refactoring basebackup.c

2022-01-28 Thread Dipesh Pandit
Hi, > I made a pass over these patches today and made a bunch of minor > corrections. New version attached. The two biggest things I changed > are (1) s/gzip_extractor/gzip_compressor/, because I feel like you > extract an archive like a tarfile, but that is not what is happening > here, this is

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 01:49:19PM +0530, Bharath Rupireddy wrote: > On Thu, Jan 27, 2022 at 3:23 PM Sergey Dudoladov > wrote: > > > Here's v2, rebased onto the latest master. > > > > I've reviewed this patch. The patch builds against the master (commit > >

pg_log_backend_memory_contexts - remove unnecessary test case

2022-01-28 Thread Bharath Rupireddy
Hi, While reviewing another patch [1], I found an unnecessary test case of pg_log_backend_memory_contexts, that is, executing the function after granting the permission to the role and checking the permissions with the has_function_privilege. IMO, it's unnecessary as we have the test cases

Re: RFC: Logging plan of the running query

2022-01-28 Thread torikoshia
On 2022-01-27 20:18, Fujii Masao wrote: Here are another review comments: Thanks for reviewing! +LOG: plan of the query running on backend with PID 17793 is: This seems not the same as what actually logged. Modified. + ereport(WARNING, +

Re: Column Filtering in Logical Replication

2022-01-28 Thread Peter Smith
Here are some review comments for the v17-0001 patch. ~~~ 1. Commit message If no column list is specified, all the columns are replicated, as previously Missing period (.) at the end of that sentence. ~~~ 2. doc/src/sgml/catalogs.sgml + + This is an array of values that

Re: Printing backtrace of postgres processes

2022-01-28 Thread Bharath Rupireddy
On Thu, Jan 27, 2022 at 10:45 AM vignesh C wrote: > > On Wed, Jan 26, 2022 at 11:07 AM Bharath Rupireddy > wrote: > > > > On Tue, Jan 25, 2022 at 12:00 PM vignesh C wrote: > > > Thanks for the comments, attached v17 patch has the fix for the same. > > > > Have a minor comment on v17: > > > >

Re: Is there a way (except from server logs) to know the kind of on-going/last checkpoint?

2022-01-28 Thread Bharath Rupireddy
On Thu, Jan 27, 2022 at 3:23 PM Sergey Dudoladov wrote: > > Here's v2, rebased onto the latest master. > > I've reviewed this patch. The patch builds against the master (commit > e9d4001ec592bcc9a3332547cb1b0211e8794f38) and passes all the tests. > The patch does what it intends to do, namely