Re: Implementing Incremental View Maintenance

2021-11-24 Thread Yugo NAGATA
On Wed, 24 Nov 2021 04:31:25 + "r.takahash...@fujitsu.com" wrote: > > > ivm=# create table t (c1 int, c2 int); > > > CREATE TABLE > > > ivm=# create incremental materialized view ivm_t as select distinct c1 > > > from t; > > > NOTICE: created index "ivm_t_index" on materialized view "ivm_t"

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-24 Thread Greg Nancarrow
On Tue, Oct 5, 2021 at 7:07 PM Dilip Kumar wrote: > > Patch details: > 0001 to 0006 implements an approach1 > 0007 removes the code of pg_class scanning and adds the directory scan. > I had a scan through the patches, though have not yet actually run any tests to try to better gauge their benefit

Re: Deduplicate code updating ControleFile's DBState.

2021-11-24 Thread Michael Paquier
On Thu, Nov 25, 2021 at 10:21:40AM +0530, Amul Sul wrote: > Thanks for the inputs -- moved timestamp setting inside update_controlfile(). I have not check the performance implication of that with a micro benchmark or the like, but I can get behind 0001 on consistency grounds between the backend a

Re: Implementing Incremental View Maintenance

2021-11-24 Thread Yugo NAGATA
Hello Takahashi-san, On Wed, 24 Nov 2021 04:27:13 + "r.takahash...@fujitsu.com" wrote: > Hi Nagata-san, > > > Sorry for late reply. > > > > However, even if we create triggers recursively on the parents or children, > > we would still > > need more consideration. This is because we will

Re: [BUG]Missing REPLICA IDENTITY check when DROP NOT NULL

2021-11-24 Thread Michael Paquier
On Thu, Nov 25, 2021 at 10:44:53AM +0530, Dilip Kumar wrote: > Good catch, your patch looks fine to me and solves the reported problem. And applied down to 10. A couple of comments in the same area did not get the call, though. -- Michael signature.asc Description: PGP signature

Re: prevent immature WAL streaming

2021-11-24 Thread Amul Sul
On Wed, Nov 24, 2021 at 2:10 AM Alvaro Herrera wrote: > > On 2021-Nov-23, Tom Lane wrote: > > > We're *still* not out of the woods with 026_overwrite_contrecord.pl, > > as we are continuing to see occasional "mismatching overwritten LSN" > > failures, further down in the test where it tries to sta

Re: pg_dump, pg_basebackup don't error out with wrong option for "--format"

2021-11-24 Thread Bharath Rupireddy
On Thu, Nov 25, 2021 at 11:02 AM Dilip Kumar wrote: > > On Thu, Nov 25, 2021 at 10:44 AM Bharath Rupireddy > wrote: > > > > Hi, > > > > I noticed that the pg_dump and pg_basebackup are not erroring out when > > "--fo"/"--for"/"--form"/"--forma"/" are specified(use cases 1 - 4, 7 - > > 9) whereas

Re: pg_dump, pg_basebackup don't error out with wrong option for "--format"

2021-11-24 Thread Dilip Kumar
On Thu, Nov 25, 2021 at 10:44 AM Bharath Rupireddy wrote: > > Hi, > > I noticed that the pg_dump and pg_basebackup are not erroring out when > "--fo"/"--for"/"--form"/"--forma"/" are specified(use cases 1 - 4, 7 - > 9) whereas it fails if a pattern that doesn't match with "format" is > specified (

Re: [BUG]Missing REPLICA IDENTITY check when DROP NOT NULL

2021-11-24 Thread Dilip Kumar
On Thu, Nov 25, 2021 at 8:21 AM tanghy.f...@fujitsu.com wrote: > > On Wed, Nov 24, 2021 7:29 PM, Michael Paquier wrote: > > Thanks for your comment. I agree with you. > I have fixed it and attached v2 patch. Good catch, your patch looks fine to me and solves the reported problem. -- Regards,

pg_dump, pg_basebackup don't error out with wrong option for "--format"

2021-11-24 Thread Bharath Rupireddy
Hi, I noticed that the pg_dump and pg_basebackup are not erroring out when "--fo"/"--for"/"--form"/"--forma"/" are specified(use cases 1 - 4, 7 - 9) whereas it fails if a pattern that doesn't match with "format" is specified (use case 5, 10). This seems to be true only for "--format" option, for o

Re: Skipping logical replication transactions on subscriber side

2021-11-24 Thread Amit Kapila
On Wed, Nov 24, 2021 at 5:14 PM Masahiko Sawada wrote: > > Changed. I've removed first_error_time as per discussion on the thread > for adding xact stats. > We also agreed to change the column names to start with last_error_* [1]. Is there a reason to not make those changes? Do you think that we

Re: pg_replslotdata - a tool for displaying replication slot information

2021-11-24 Thread Bharath Rupireddy
On Wed, Nov 24, 2021 at 9:40 PM Japin Li wrote: > > > On Wed, 24 Nov 2021 at 23:59, Bharath Rupireddy > wrote: > > On Wed, Nov 24, 2021 at 9:09 PM Bharath Rupireddy > >> > Thoughts? > >> > >> Attaching the rebased v2 patch. > > > > On windows the previous patches were failing, fixed that in the

Re: Deduplicate code updating ControleFile's DBState.

2021-11-24 Thread Amul Sul
On Thu, Nov 11, 2021 at 1:30 AM Bossart, Nathan wrote: > > On 10/1/21, 10:40 PM, "Michael Paquier" wrote: > > On Fri, Oct 01, 2021 at 05:47:45PM +, Bossart, Nathan wrote: > >> I'm inclined to agree that anything that calls update_controlfile() > >> should update the timestamp. > > > > pg_cont

Re: Non-superuser subscription owners

2021-11-24 Thread Amit Kapila
On Thu, Nov 25, 2021 at 6:00 AM Jeff Davis wrote: > > On Fri, 2021-11-19 at 16:45 -0800, Mark Dilger wrote: > > Renamed as 0001 in version 3, as it is the only remaining patch. For > > anyone who reviewed the older patch set, please note that I made some > > changes to the src/test/subscription/t

Re: Reduce function call costs on ELF platforms

2021-11-24 Thread Andres Freund
Hi, On 2021-11-24 17:55:03 -0500, Andrew Dunstan wrote: > On 11/24/21 13:55, Andres Freund wrote: > > On 2021-11-23 17:28:08 +0100, Peter Eisentraut wrote: > >> On 22.11.21 23:32, Tom Lane wrote: > The easier approach for this class of issues is to use the linker option > -Bsymbolic. > >

Re: row filtering for logical replication

2021-11-24 Thread Peter Smith
On Tue, Nov 23, 2021 at 10:52 PM Amit Kapila wrote: > > On Tue, Nov 23, 2021 at 4:58 PM Ajin Cherian wrote: > > > > Attaching a new patchset v41 which includes changes by both Peter and > > myself. > > > ... > I suggest at this stage we can combine 0001, 0003, and 0004. Then move > pg_dump and p

Re: row filtering for logical replication

2021-11-24 Thread Peter Smith
On Thu, Nov 25, 2021 at 12:03 AM Amit Kapila wrote: > > On Tue, Nov 23, 2021 at 4:58 PM Ajin Cherian wrote: > > > > Attaching a new patchset v41 which includes changes by both Peter and > > myself. > > ... > Few more comments: > = ... > 0005 > 2. > +typedef struct { > + Relation

Re: row filtering for logical replication

2021-11-24 Thread Peter Smith
On Wed, Nov 24, 2021 at 8:52 PM vignesh C wrote: > > On Tue, Nov 23, 2021 at 4:58 PM Ajin Cherian wrote: > > > > Attaching a new patchset v41 which includes changes by both Peter and > > myself. > > > > Patches v40-0005 and v40-0006 have been merged to create patch > > v41-0005 which reduces the

Re: row filtering for logical replication

2021-11-24 Thread Peter Smith
On Tue, Nov 23, 2021 at 4:40 PM tanghy.f...@fujitsu.com wrote: > > On Thursday, November 18, 2021 9:34 AM Peter Smith > wrote: > > > > PSA new set of v40* patches. > > > > Besides, a small comment on 0004 patch: > > +* Multiple row-filter expressions for the same publication >

Re: row filtering for logical replication

2021-11-24 Thread Peter Smith
On Tue, Nov 23, 2021 at 8:02 PM Amit Kapila wrote: > > On Thu, Nov 18, 2021 at 11:02 AM Peter Smith wrote: > > > > On Mon, Nov 15, 2021 at 9:31 PM Amit Kapila wrote: > > > > > > > > 4. I think we should add some comments in pgoutput_row_filter() as to > > > why we are caching the row_filter here

Re: Parallel vacuum workers prevent the oldest xmin from advancing

2021-11-24 Thread Amit Kapila
On Wed, Nov 24, 2021 at 7:46 PM John Naylor wrote: > > On Wed, Nov 17, 2021 at 7:28 AM Amit Kapila wrote: > > > > > The patch looks good to me. But I can't come up with a stable test for > > > this. It seems to be hard without stopping and resuming parallel > > > vacuum workers. Do you have any g

Re: row filtering for logical replication

2021-11-24 Thread Peter Smith
On Thu, Nov 18, 2021 at 9:35 PM Greg Nancarrow wrote: > > On Thu, Nov 18, 2021 at 12:33 PM Peter Smith wrote: > > > > PSA new set of v40* patches. > > > > Thanks for the patch updates. > > A couple of comments so far: > > (1) compilation warning > WIth the patches applied, there's a single compil

RE: [BUG]Missing REPLICA IDENTITY check when DROP NOT NULL

2021-11-24 Thread tanghy.f...@fujitsu.com
On Wed, Nov 24, 2021 7:29 PM, Michael Paquier wrote: > > On Wed, Nov 24, 2021 at 07:04:51AM +, tanghy.f...@fujitsu.com wrote: > > create table tbl (a int not null unique); > > alter table tbl replica identity using INDEX tbl_a_key; > > alter table tbl alter column a drop not null; > > insert

RE: pg_get_publication_tables() output duplicate relid

2021-11-24 Thread houzj.f...@fujitsu.com
On Wed, Nov 24, 2021 4:48 PM Amit Kapila wrote: > On Mon, Nov 22, 2021 at 12:55 PM Amit Langote > wrote: > > > > On Fri, Nov 19, 2021 at 2:28 PM Amit Kapila > wrote: > > > On Fri, Nov 19, 2021 at 7:19 AM Amit Langote > wrote: > > > > As in, > > > > do we know of any replication (initial/stream

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2021-11-24 Thread Justin Pryzby
Thanks for working on this. I was just trying to find something like "pg_stat_checkpointer". You wrote beentry++ at the start of two loops, but I think that's wrong; it should be at the end, as in the rest of the file (or as a loop increment). BackendStatusArray[0] is actually used (even though i

Re: Non-superuser subscription owners

2021-11-24 Thread Jeff Davis
On Fri, 2021-11-19 at 16:45 -0800, Mark Dilger wrote: > Renamed as 0001 in version 3, as it is the only remaining patch. For > anyone who reviewed the older patch set, please note that I made some > changes to the src/test/subscription/t/026_nosuperuser.pl test case > relative to the prior version

Re: Reduce function call costs on ELF platforms

2021-11-24 Thread Andrew Dunstan
On 11/24/21 13:55, Andres Freund wrote: > Hi, > > On 2021-11-23 17:28:08 +0100, Peter Eisentraut wrote: >> On 22.11.21 23:32, Tom Lane wrote: The easier approach for this class of issues is to use the linker option -Bsymbolic. >>> I don't recall details, but we've previously rejected th

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2021-11-24 Thread Michael Paquier
On Wed, Nov 24, 2021 at 02:12:19PM -0800, SATYANARAYANA NARLAPURAM wrote: > While an exclusive backup is in progress if Postgres restarts, postgres > runs the recovery from the checkpoint identified by the label file instead > of the control file. This can cause long recovery or even sometimes fail

Re: internal subtransactions, memory contexts, resource owners

2021-11-24 Thread Tom Lane
Chapman Flack writes: > By inspection of plperl and plpython, it looks like the canonical pattern > for a PL using internal subtransactions is: > save CurrentMemoryContext > save CurrentResourceOwner > BeginInternalSubTransaction > reimpose the saved memory context > // but not the saved resource

Postgres restart in the middle of exclusive backup and the presence of backup_label file

2021-11-24 Thread SATYANARAYANA NARLAPURAM
Hi Hackers, While an exclusive backup is in progress if Postgres restarts, postgres runs the recovery from the checkpoint identified by the label file instead of the control file. This can cause long recovery or even sometimes fail to recover as the WAL records corresponding to that checkpoint loc

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2021-11-24 Thread Melanie Plageman
On Fri, Nov 19, 2021 at 11:49 AM Andres Freund wrote: > > +/* > > + * On modern systems this is really just *counter++. On some older systems > > + * there might be more to it, due to inability to read and write 64 bit > > values > > + * atomically. > > + */ > > +static inline void inc_counter(p

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2021-11-24 Thread Bossart, Nathan
On 11/23/21, 4:59 PM, "Mark Dilger" wrote: >> On Nov 23, 2021, at 4:51 PM, Bossart, Nathan wrote: >> >> This is a good point. Right now, you'd have to manually inspect the >> infomask field to determine the exact form of the invalid combination. >> My only worry with this is that we'd want to ma

internal subtransactions, memory contexts, resource owners

2021-11-24 Thread Chapman Flack
Hi, By inspection of plperl and plpython, it looks like the canonical pattern for a PL using internal subtransactions is: save CurrentMemoryContext save CurrentResourceOwner BeginInternalSubTransaction reimpose the saved memory context // but not the saved resource owner ... (RollbackAnd)?Releas

Re: Post-CVE Wishlist

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 3:14 PM Tom Lane wrote: > Robert Haas writes: > > On Wed, Nov 24, 2021 at 2:53 PM Tom Lane wrote: > >> One other point to be made here is that it seems like a stretch to call > >> these particular bugs "high-severity". > > > Well, I was referring to the CVSS score, which

Re: Post-CVE Wishlist

2021-11-24 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 24, 2021 at 2:53 PM Tom Lane wrote: >> One other point to be made here is that it seems like a stretch to call >> these particular bugs "high-severity". > Well, I was referring to the CVSS score, which was in the "high" range. The server CVE is; the client CVE,

Re: Post-CVE Wishlist

2021-11-24 Thread Jacob Champion
On Wed, 2021-11-24 at 14:01 -0500, Robert Haas wrote: > The bar for actually ripping it out > on an expedited time scale would be proving not only that it's broken > in multiple ways, but that it's so badly broken that it can't be fixed > with any reasonable amount of effort. And I just don't see o

Re: Post-CVE Wishlist

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 2:53 PM Tom Lane wrote: > One other point to be made here is that it seems like a stretch to call > these particular bugs "high-severity". Well, I was referring to the CVSS score, which was in the "high" range. > Given what we learned about > the difficulty of exploiting

Re: Post-CVE Wishlist

2021-11-24 Thread Jacob Champion
On Wed, 2021-11-24 at 14:03 -0500, Tom Lane wrote: > > I don't buy the idea that, because we have fixed that particular > > vulnerability, we've rendered this entire class of bugs "hypothetical". > > There will be more code and more clients. There will always be bugs. > > I'd rather the bugs that p

Re: Post-CVE Wishlist

2021-11-24 Thread Tom Lane
Robert Haas writes: > I think it would take an overwhelming amount of evidence to convince > the project to remove support for the current method. One or even two > or three high-severity bugs will probably not convince the project to > do more than spend more studying that code and trying to tigh

Re: Split xlog.c

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 12:16 PM Heikki Linnakangas wrote: > And here's another rebase, now that Robert got rid of ReadRecPtr and > EndRecPtr. In general, I think 0001 is a good idea, but the comment that says "Set the XLP_FIRST_IS_OVERWRITE_CONTRECORD flag on the page header" seems to me to be t

Re: Should rename "startup process" to something else?

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 1:54 PM Alvaro Herrera wrote: > I don't object to an underscore, but it looks a bit uglier to me. > AFAIK the main problem with uuid-ossp was that it is used as an identifier, so > it required quoting, which won't be the case with this process name. I agree, and for that r

Re: Should rename "startup process" to something else?

2021-11-24 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Nov-23, Tom Lane wrote: >> Bad memories of uuid-ossp float up ... can we use underscore? > I don't object to an underscore, but it looks a bit uglier to me. > AFAIK the main problem with uuid-ossp was that it is used as an identifier, so > it required quoting, whi

Re: Post-CVE Wishlist

2021-11-24 Thread Tom Lane
Jacob Champion writes: > What I'm trying to convince you of is that this pattern of beginning a > TLS conversation is known to be particularly error-prone, across > multiple protocols and implementations. I think this is supported by > the fact that at least three independent client libraries made

Re: Post-CVE Wishlist

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 1:29 PM Jacob Champion wrote: > What I'm trying to convince you of is that this pattern of beginning a > TLS conversation is known to be particularly error-prone, across > multiple protocols and implementations. I think this is supported by > the fact that at least three in

Re: [PATCH] Partial foreign key updates in referential integrity triggers

2021-11-24 Thread Peter Eisentraut
On 05.01.21 22:40, Paul Martinez wrote: CREATE TABLE tenants (id serial PRIMARY KEY); CREATE TABLE users ( tenant_id int REFERENCES tenants ON DELETE CASCADE, id serial, PRIMARY KEY (tenant_id, id), ); CREATE TABLE posts ( tenant_id int REFERENCES tenants ON DELETE CASCADE, id

Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 1:51 PM Alvaro Herrera wrote: > > There is a buildfarm failure on lapwing which looks likely to be > > attributable to this commit, but I don't understand what has gone > > wrong exactly. The error message that is reported is: > > No, this is an unrelated problem; Tom repor

Re: Reduce function call costs on ELF platforms

2021-11-24 Thread Andres Freund
Hi, On 2021-11-23 17:28:08 +0100, Peter Eisentraut wrote: > On 22.11.21 23:32, Tom Lane wrote: > > > The easier approach for this class of issues is to use the linker option > > > -Bsymbolic. > > I don't recall details, but we've previously rejected the idea of > > trying to use -Bsymbolic widely;

Re: Should rename "startup process" to something else?

2021-11-24 Thread Alvaro Herrera
On 2021-Nov-23, Tom Lane wrote: > Alvaro Herrera writes: > > On 2021-Nov-20, Andrew Dunstan wrote: > >> Maybe something along those lines but using a dash/hyphen would work: > >> e.g. wal-replayer > > > Yeah, the idea of a dash occurred to me too. > > Bad memories of uuid-ossp float up ... can

Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.

2021-11-24 Thread Alvaro Herrera
On 2021-Nov-24, Robert Haas wrote: > On Wed, Nov 24, 2021 at 11:28 AM Robert Haas wrote: > > xlog.c: Remove global variables ReadRecPtr and EndRecPtr. > > https://git.postgresql.org/pg/commitdiff/d2ddfa681db27a138acb63c8defa8cc6fa588922 > > There is a buildfarm failure on lapwing which looks lik

Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.

2021-11-24 Thread Tom Lane
Robert Haas writes: > 2021-11-24 17:07:41.428 UTC [1449:6] FATAL: mismatching overwritten > LSN 0/1FFE014 -> 0/1FFE000 > 2021-11-24 17:07:41.428 UTC [1449:7] CONTEXT: WAL redo at 0/224 > for XLOG/OVERWRITE_CONTRECORD: lsn 0/1FFE014; time 2021-11-24 > 17:07:41.127049+00 > Beyond the fact tha

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Peter Geoghegan
On Wed, Nov 24, 2021 at 9:53 AM Alvaro Herrera wrote: > OK, this makes a lot more sense. I wasn't aware of ae7291ac (and I > wasn't aware of the significance of 8523492d either, but that's not > really relevant here.) Thanks for hearing me out about the significance of 8523492d. Having the righ

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-24 Thread Tom Lane
Laurenz Albe writes: > On Tue, 2021-11-23 at 16:41 -0500, Tom Lane wrote: >> A bit of polishing later, maybe like the attached. > That looks good to me. Pushed, thanks. regards, tom lane

Re: BUFFERS enabled by default in EXPLAIN (ANALYZE)

2021-11-24 Thread Michael Christofides
I think it *should* be enabled for planning, since that makes the default > easier to understand and document, and it makes a user's use of "explain" > easier. I’d be keen to see BUFFERS off by default with EXPLAIN, and on by default with EXPLAIN ANALYZE. The SUMMARY flag was implemented that wa

Re: Post-CVE Wishlist

2021-11-24 Thread Jacob Champion
On Wed, 2021-11-24 at 09:40 -0500, Robert Haas wrote: > I am not really persuaded by Jacob's argument that, had this only > worked the other way from the start, this bug wouldn't have occurred. > That's just a tautology, because we can only have bugs in the code we > write, not the code we didn't w

Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 11:28 AM Robert Haas wrote: > xlog.c: Remove global variables ReadRecPtr and EndRecPtr. > https://git.postgresql.org/pg/commitdiff/d2ddfa681db27a138acb63c8defa8cc6fa588922 There is a buildfarm failure on lapwing which looks likely to be attributable to this commit, but I d

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Alvaro Herrera
On 2021-Nov-24, Peter Geoghegan wrote: > TIDs (ItemPointerData structs) are of course not the same thing as > line pointers (ItemIdData structs). There is a tendency to refer to > the latter as "item pointers" all the same, which was confusing. I > personally corrected/normalized this in commit ae

Re: Python 3.11 vs. Postgres

2021-11-24 Thread Tom Lane
Peter Eisentraut writes: > On 24.11.21 04:07, Tom Lane wrote: >> According to [1], we need to stop including Python's . > See attached patch. The minimum Python version for this change is 2.4, > which is the oldest version supported by PG10, so we can backpatch this > to all live branches. LG

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Peter Geoghegan
On Wed, Nov 24, 2021 at 7:16 AM Alvaro Herrera wrote: > Sorry to reply to myself, but I realized that I forgot to return to the > main point of this thread. If we agree that "an LP_DEAD item pointer > does not point to any item" (an assertion that gives a precise meaning > to both those terms), t

Re: Minor documentation fix - missing blank space

2021-11-24 Thread Heikki Linnakangas
On 24/11/2021 16:49, Japin Li wrote: When I read the documentation about Overview of PostgreSQL Internals - Executor [1], I find there is a missing space in the documentation. diff --git a/doc/src/sgml/arch-dev.sgml b/doc/src/sgml/arch-dev.sgml index 7aff059e82..c2be28fac8 100644 --- a/doc/src/

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-24 Thread Robert Haas
On Tue, Nov 23, 2021 at 1:36 PM Robert Haas wrote: > Attached please find the test case not for commit as > v2-0001-dubious-test-case.patch; the fix, for commit and now with a > proper commit message as > v2-0002-Fix-corner-case-failure-to-detect-improper-timeli.patch; and > the back-ported test c

Re: Synchronizing slots from primary to standby

2021-11-24 Thread Dimitri Fontaine
Hi all, Peter Eisentraut writes: > I want to reactivate $subject. I took Petr Jelinek's patch from [0], > rebased it, added a bit of testing. It basically works, but as > mentioned in [0], there are various issues to work out. Thanks for working on that topic, I believe it's an important part

Re: psql - add SHOW_ALL_RESULTS option

2021-11-24 Thread Fabien COELHO
Hello Daniel, This patch still has a few of the problems reported earlier this year. The patch fails to apply and the thread seems to have taken a nap. I'm not napping:-) I just do not have enough time available this month. I intend to work on the patch in the next CF (January). AFAICR th

Re: pg_replslotdata - a tool for displaying replication slot information

2021-11-24 Thread Japin Li
On Wed, 24 Nov 2021 at 23:59, Bharath Rupireddy wrote: > On Wed, Nov 24, 2021 at 9:09 PM Bharath Rupireddy >> > Thoughts? >> >> Attaching the rebased v2 patch. > > On windows the previous patches were failing, fixed that in the v3 > patch. I'm really sorry for the noise. > Cool! When I try to

Re: pg_replslotdata - a tool for displaying replication slot information

2021-11-24 Thread Bharath Rupireddy
On Wed, Nov 24, 2021 at 9:09 PM Bharath Rupireddy wrote: > > On Tue, Nov 23, 2021 at 10:39 AM Bharath Rupireddy > wrote: > > > > Hi, > > > > The replication slots data is stored in binary format on the disk under the > > pg_replslot/<> directory which isn't human readable. If the > > server is

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 9:51 AM Alvaro Herrera wrote: > On 2021-Nov-24, Robert Haas wrote: > > Hmm. I think in my model an item and an item pointer and a line > > pointer are all the same thing, but a TID is different. When I talk > > about a TID, I mean the location of an item pointer, not its co

Re: pg_replslotdata - a tool for displaying replication slot information

2021-11-24 Thread Bharath Rupireddy
On Tue, Nov 23, 2021 at 10:39 AM Bharath Rupireddy wrote: > > Hi, > > The replication slots data is stored in binary format on the disk under the > pg_replslot/<> directory which isn't human readable. If the server > is crashed/down (for whatever reasons) and unable to come up, currently > ther

Re: Windows build warnings

2021-11-24 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > =?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: >> Should we change the compiler checks for attributes in c.h to include >> `|| __has_attribute(…)`, so that we automatically get them on compilers >> that support that (particularly clang)? > > clang already #defines GCC, no?

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Alvaro Herrera
On 2021-Nov-24, Alvaro Herrera wrote: > On 2021-Nov-24, Robert Haas wrote: > > > Hmm. I think in my model an item and an item pointer and a line > > pointer are all the same thing, but a TID is different. When I talk > > about a TID, I mean the location of an item pointer, not its contents. > > S

Re: Windows build warnings

2021-11-24 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > Should we change the compiler checks for attributes in c.h to include > `|| __has_attribute(…)`, so that we automatically get them on compilers > that support that (particularly clang)? clang already #defines GCC, no? re

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Alvaro Herrera
On 2021-Nov-24, Robert Haas wrote: > Hmm. I think in my model an item and an item pointer and a line > pointer are all the same thing, but a TID is different. When I talk > about a TID, I mean the location of an item pointer, not its contents. > So a TID is what tells me that I want block 5 and th

Minor documentation fix - missing blank space

2021-11-24 Thread Japin Li
Hi, When I read the documentation about Overview of PostgreSQL Internals - Executor [1], I find there is a missing space in the documentation. diff --git a/doc/src/sgml/arch-dev.sgml b/doc/src/sgml/arch-dev.sgml index 7aff059e82..c2be28fac8 100644 --- a/doc/src/sgml/arch-dev.sgml +++ b/doc/src

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 9:37 AM Alvaro Herrera wrote: > My mental model is that "tuple" (in the narrow context of heap vacuum) > is the variable-size on-disk representation of a row in a page; "line > pointer" is the fixed-size struct at the bottom of each page that > contains location, size and f

Re: Post-CVE Wishlist

2021-11-24 Thread Robert Haas
On Tue, Nov 23, 2021 at 5:41 PM Heikki Linnakangas wrote: > All that said, I'm not sure how serious I am about this. I think it > would work, and it wouldn't even be very complicated, but it feels > hacky, and that's not a good thing with anything security related. And > the starttls-style negotia

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Alvaro Herrera
On 2021-Nov-24, Robert Haas wrote: > On Wed, Nov 24, 2021 at 7:48 AM Masahiko Sawada wrote: > > I think it's more consistent if we change it to one side. I prefer > > "dead items". > > I feel like "items" is quite a generic word, so I think I would prefer > TIDs. But it's probably not a big deal

Re: Parallel vacuum workers prevent the oldest xmin from advancing

2021-11-24 Thread John Naylor
On Wed, Nov 17, 2021 at 7:28 AM Amit Kapila wrote: > > > The patch looks good to me. But I can't come up with a stable test for > > this. It seems to be hard without stopping and resuming parallel > > vacuum workers. Do you have any good idea? > > > > No, let's wait for a day or so to see if anybo

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Robert Haas
On Wed, Nov 24, 2021 at 7:48 AM Masahiko Sawada wrote: > I think it's more consistent if we change it to one side. I prefer "dead > items". I feel like "items" is quite a generic word, so I think I would prefer TIDs. But it's probably not a big deal. -- Robert Haas EDB: http://www.enterprisedb

Re: Support for NSS as a libpq TLS backend

2021-11-24 Thread Joshua Brindle
On Wed, Nov 24, 2021 at 8:46 AM Joshua Brindle wrote: > > On Wed, Nov 24, 2021 at 6:59 AM Daniel Gustafsson wrote: > > > > > On 23 Nov 2021, at 23:39, Joshua Brindle > > > wrote: > > > > > It no longer happens with v49, since it was a null deref of the pr_fd > > > which no longer happens. > > >

Re: Support for NSS as a libpq TLS backend

2021-11-24 Thread Joshua Brindle
On Wed, Nov 24, 2021 at 6:59 AM Daniel Gustafsson wrote: > > > On 23 Nov 2021, at 23:39, Joshua Brindle > > wrote: > > > It no longer happens with v49, since it was a null deref of the pr_fd > > which no longer happens. > > > > I'll continue testing now, so far it's looking better. > > Great, th

Re: row filtering for logical replication

2021-11-24 Thread Amit Kapila
On Tue, Nov 23, 2021 at 4:58 PM Ajin Cherian wrote: > > Attaching a new patchset v41 which includes changes by both Peter and myself. > > Patches v40-0005 and v40-0006 have been merged to create patch > v41-0005 which reduces the patches to 6 again. > This patch-set contains changes addressing the

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-24 Thread Masahiko Sawada
On Wed, Nov 24, 2021 at 2:46 PM Peter Geoghegan wrote: > > Attached patch performs polishing within vacuumlazy.c, as follow-up > work to the refactoring work in Postgres 14. This mainly consists of > changing references of dead tuples to dead items, which reflects the > fact that VACUUM no longer

Re: VS2022: Support Visual Studio 2022 on Windows

2021-11-24 Thread Dave Page
Hi On Wed, Nov 24, 2021 at 11:36 AM Michael Paquier wrote: > On Wed, Nov 24, 2021 at 10:00:19AM +, Dave Page wrote: > > It's extremely unlikely that we'd shift to such a new version for PG15. > We > > build many components aside from PostgreSQL, and need to use the same > > toolchain for all

Re: Support for NSS as a libpq TLS backend

2021-11-24 Thread Daniel Gustafsson
> On 23 Nov 2021, at 23:39, Joshua Brindle > wrote: > It no longer happens with v49, since it was a null deref of the pr_fd > which no longer happens. > > I'll continue testing now, so far it's looking better. Great, thanks for confirming. I'm still keen on knowing how you triggered the segfa

Re: Teach pg_receivewal to use lz4 compression

2021-11-24 Thread Jeevan Ladhe
On Wed, Nov 24, 2021 at 10:55 AM Michael Paquier wrote: > On Mon, Nov 22, 2021 at 09:02:47AM -0500, Robert Haas wrote: > > On Mon, Nov 22, 2021 at 12:46 AM Jeevan Ladhe > > wrote: > >> Fair enough. But, still I have a doubt in mind what benefit would that > >> really bring to us here, because we

Re: Skipping logical replication transactions on subscriber side

2021-11-24 Thread Masahiko Sawada
On Wed, Nov 17, 2021 at 8:14 PM Amit Kapila wrote: > > On Tue, Nov 16, 2021 at 12:01 PM Masahiko Sawada > wrote: > > > > Right. I've fixed this issue and attached an updated patch. > > > > Few comments/questions: > = > 1. > + > + The pg_stat_subscription_workers view will

Re: VS2022: Support Visual Studio 2022 on Windows

2021-11-24 Thread Michael Paquier
On Wed, Nov 24, 2021 at 10:00:19AM +, Dave Page wrote: > It's extremely unlikely that we'd shift to such a new version for PG15. We > build many components aside from PostgreSQL, and need to use the same > toolchain for all of them (we've had very painful experiences with mix n > match CRT vers

Re: [BUG]Missing REPLICA IDENTITY check when DROP NOT NULL

2021-11-24 Thread Michael Paquier
On Wed, Nov 24, 2021 at 07:04:51AM +, tanghy.f...@fujitsu.com wrote: > create table tbl (a int not null unique); > alter table tbl replica identity using INDEX tbl_a_key; > alter table tbl alter column a drop not null; > insert into tbl values (null); Oops. Yes, that's obviously not good. >

Re: Windows build warnings

2021-11-24 Thread Dagfinn Ilmari Mannsåker
Daniel Gustafsson writes: > On 22 Nov 2021, at 16:40, Tom Lane wrote: > >> I can't find anything that is providing a non-empty definition of >> PG_USED_FOR_ASSERTS_ONLY (a/k/a pg_attribute_unused) for anything >> except GCC. > > It's supported in clang as well per the documentation [0] in at l

Re: VS2022: Support Visual Studio 2022 on Windows

2021-11-24 Thread Dave Page
Hi On Wed, Nov 24, 2021 at 9:12 AM Hans Buschmann wrote: > Hello Michael, > > thanks for your hard work and quick response! > It is very convenient to only use VS2022 for Windows from now on... > > >Diff unrelated to your patch. > > Sorry for the copysoft problem from the first version. > > >Gla

Re: row filtering for logical replication

2021-11-24 Thread vignesh C
On Tue, Nov 23, 2021 at 4:58 PM Ajin Cherian wrote: > > Attaching a new patchset v41 which includes changes by both Peter and myself. > > Patches v40-0005 and v40-0006 have been merged to create patch > v41-0005 which reduces the patches to 6 again. Few comments: 1) I'm not sure if we will be abl

AW: VS2022: Support Visual Studio 2022 on Windows

2021-11-24 Thread Hans Buschmann
Hello Michael, thanks for your hard work and quick response! It is very convenient to only use VS2022 for Windows from now on... >Diff unrelated to your patch. Sorry for the copysoft problem from the first version. >Glad to see that we should have nothing to do about locales this >time.

Re: rename SnapBuild* macros in slot.c

2021-11-24 Thread Amit Kapila
On Tue, Nov 23, 2021 at 8:12 AM Amit Kapila wrote: > > On Mon, Nov 22, 2021 at 7:47 PM Alvaro Herrera > wrote: > > > > On 2021-Nov-22, Amit Kapila wrote: > > > > > +1 for this change. This seems to be introduced by commit ec5896aed3 > > > [1] and I think it is just a typo to name these macros st

Re: parallel vacuum comments

2021-11-24 Thread Amit Kapila
On Wed, Nov 24, 2021 at 12:16 PM Masahiko Sawada wrote: > > On Wed, Nov 24, 2021 at 1:34 PM Amit Kapila wrote: > > > > On Wed, Nov 24, 2021 at 7:43 AM Masahiko Sawada > > wrote: > > > > > > On Mon, Nov 22, 2021 at 1:48 PM Amit Kapila > > > wrote: > > > > > > > > On Fri, Nov 19, 2021 at 7:55 A

Re: Skipping logical replication transactions on subscriber side

2021-11-24 Thread Amit Kapila
On Wed, Nov 24, 2021 at 1:50 PM Masahiko Sawada wrote: > > On Wed, Nov 24, 2021 at 12:13 PM Amit Kapila wrote: > > > > On Wed, Nov 24, 2021 at 7:51 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Nov 18, 2021 at 12:52 PM houzj.f...@fujitsu.com > > > wrote: > > > > > > > > On Tuesday, Novembe

Re: pg_get_publication_tables() output duplicate relid

2021-11-24 Thread Amit Kapila
On Mon, Nov 22, 2021 at 12:55 PM Amit Langote wrote: > > On Fri, Nov 19, 2021 at 2:28 PM Amit Kapila wrote: > > On Fri, Nov 19, 2021 at 7:19 AM Amit Langote > > wrote: > > > As in, > > > do we know of any replication (initial/streaming) misbehavior caused > > > by the duplicate partition OIDs

Re: pg_get_publication_tables() output duplicate relid

2021-11-24 Thread Amit Kapila
On Wed, Nov 24, 2021 at 12:02 PM Amit Langote wrote: > > On Tue, Nov 23, 2021 at 12:21 PM Amit Kapila wrote: > > Isn't it better to document this case and explain what > > users can expect instead of trying to design a solution around this? > > I thought about the problems you've described and it

Re: Post-CVE Wishlist

2021-11-24 Thread Heikki Linnakangas
On 24/11/2021 07:09, Michael Paquier wrote: On Tue, Nov 23, 2021 at 02:18:30PM -0500, Tom Lane wrote: Jacob Champion writes: = Client-Side Auth Selection = The second request is for the client to stop fully trusting the server during the authentication phase. If I tell libpq to use a client ce

Re: Skipping logical replication transactions on subscriber side

2021-11-24 Thread Masahiko Sawada
On Wed, Nov 24, 2021 at 12:13 PM Amit Kapila wrote: > > On Wed, Nov 24, 2021 at 7:51 AM Masahiko Sawada wrote: > > > > On Thu, Nov 18, 2021 at 12:52 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada > > > wrote: > > > > Right. I've fixed thi