Re: Optionally automatically disable logical replication subscriptions on error

2022-02-23 Thread Masahiko Sawada
On Tue, Feb 22, 2022 at 3:03 PM Peter Smith wrote: > > ~~~ > > 1. worker.c - comment > > + subform = (Form_pg_subscription) GETSTRUCT(tup); > + > + /* > + * We would not be here unless this subscription's disableonerr field was > + * true, but check whether that field has changed in the interim.

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-02-23 Thread Kyotaro Horiguchi
At Wed, 23 Feb 2022 02:58:07 +, "Imseih (AWS), Sami" wrote in > >Ooh, nice find and diagnosys. I can confirm that the test fails as you > >described without the code fix, and doesn't fail with it. > > >I attach the same patch, with the test file put in its final place > >

RE: Design of pg_stat_subscription_workers vs pgstats

2022-02-23 Thread tanghy.f...@fujitsu.com
On Thu, Feb 24, 2022 9:33 AM Masahiko Sawada wrote: > > Thank you for the comments! I've attached the latest version patch > that incorporated all comments I got so far. The primary change from > the previous version is that the subscription statistics live globally > rather than per-database. >

Re: speed up a logical replica setup

2022-02-23 Thread Amit Kapila
On Mon, Feb 21, 2022 at 5:41 PM Euler Taveira wrote: > > Logical replication has been used to migration with minimal downtime. However, > if you are dealing with a big database, the amount of required resources (disk > -- due to WAL retention) increases as the backlog (WAL) increases. Unless you

Re: Per-table storage parameters for TableAM/IndexAM extensions

2022-02-23 Thread Sadhuprasad Patro
On Fri, Feb 18, 2022 at 10:48 PM Simon Riggs wrote: > > On Thu, 17 Feb 2022 at 17:55, Sadhuprasad Patro wrote: > > > > > On Sat, Feb 12, 2022 at 2:35 AM Robert Haas wrote: > > >> > > >> > > >> Imagine that I am using the "foo" tableam with "compression=lots" and > > >> I want to switch to the

Re: pg_stat_get_replication_slot and pg_stat_get_subscription_worker incorrectly marked as proretset

2022-02-23 Thread Michael Paquier
On Wed, Feb 23, 2022 at 09:52:02AM +0530, Amit Kapila wrote: > Thanks, so you are okay with me pushing that patch just to HEAD. Yes, I am fine with that. I am wondering about patching the second function though, to avoid any risk of forgetting it, but I am fine to leave that to your judgement.

Re: Patch a potential memory leak in describeOneTableDetails()

2022-02-23 Thread Kyotaro Horiguchi
At Thu, 24 Feb 2022 14:44:56 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 22 Feb 2022 09:59:09 +0100, Daniel Gustafsson wrote > in > > The section in question was added to our docs 22 years ago, to make it > > reflect > > the current operating systems in use maybe just not mentioning

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-23 Thread Etsuro Fujita
On Wed, Feb 23, 2022 at 3:30 PM Etsuro Fujita wrote: > Attached is an updated patch. > Barring objections, I’ll commit > the patch. I have committed the patch. I think the 0003 patch needs rebase. I'll update the patch. Best regards, Etsuro Fujita

Re: Patch a potential memory leak in describeOneTableDetails()

2022-02-23 Thread Kyotaro Horiguchi
At Tue, 22 Feb 2022 09:59:09 +0100, Daniel Gustafsson wrote in > > On 22 Feb 2022, at 07:14, Kyotaro Horiguchi wrote: > > > Anyway, don't we use the -ftabstop option flag to silence compiler? > > The warning is contradicting our convention. The attached adds that > > flag. > > Isn't this

Re: remove more archiving overhead

2022-02-23 Thread Kyotaro Horiguchi
At Mon, 21 Feb 2022 17:19:48 -0800, Nathan Bossart wrote in > I also spent some time investigating whether durably renaming the archive > status files was even necessary. In theory, it shouldn't be. If a crash > happens before the rename is persisted, we might try to re-archive files, > but

Re: Typo in pgbench messages.

2022-02-23 Thread KAWAMOTO Masaya
Thanks for your comment! Sorry, I did not attach the patch file. This patch focas on master branch. Best regards, On Thu, 24 Feb 2022 12:15:55 +0900 (JST) Tatsuo Ishii wrote: > > Hi, > > > > I found messages inserted a space before the "%" in pgbench. > > I think this is typo because there

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-23 Thread Amit Kapila
On Thu, Feb 24, 2022 at 7:03 AM Masahiko Sawada wrote: > > > ~~~ > > > > 6. doc/src/sgml/monitoring.sgml - why two counters? > > > > Please forgive this noob question... > > > > I see there are 2 error_count columns (one for each kind of worker) > > but I was wondering why it is useful for users

Re: Typo in pgbench messages.

2022-02-23 Thread Tatsuo Ishii
> Hi, > > I found messages inserted a space before the "%" in pgbench. > I think this is typo because there are no space before the "%" in other > messages. > What do you think? I think you are right. In English there's should be no space between number and "%". AFAIK other parts of

Re: Support logical replication of DDLs

2022-02-23 Thread Zheng Li
Hi Aleksander, >That's great! Thanks! >Maybe the first implementation shouldn't be perfect as long as known >limitations are documented and the future improvements are unlikely to >break anything for the users. Committing an MVP and iterating on this is >much simpler in terms of development and

Re: row filtering for logical replication

2022-02-23 Thread Peter Smith
On Thu, Feb 24, 2022 at 1:33 PM Amit Kapila wrote: > > On Thu, Feb 24, 2022 at 7:57 AM Peter Smith wrote: > > > > I noticed that there was a build-farm failure on the machine 'komodoensis' > > [1] > > > > # Failed test 'check replicated rows to tab_rowfilter_toast' > > # at

Typo in pgbench messages.

2022-02-23 Thread KAWAMOTO Masaya
Hi, I found messages inserted a space before the "%" in pgbench. I think this is typo because there are no space before the "%" in other messages. What do you think? diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index f166a77e3a..4ebe5e6ea4 100644 ---

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-23 Thread osumi.takami...@fujitsu.com
On Wednesday, February 23, 2022 6:52 PM Tang, Haiying/唐 海英 wrote: > I have a comment on v21 patch. > > I wonder if we really need subscription s2 in 028_disable_on_error.pl. I > think for > subscription s2, we only tested some normal cases(which could be tested with > s1), and didn't test any

Re: Extract epoch from Interval weird behavior

2022-02-23 Thread Joseph Koshakow
On Wed, Feb 23, 2022 at 7:42 PM Joseph Koshakow wrote: > > Hi all, > > I noticed something odd when going through some > of the Interval code. The DAYS_PER_YEAR constant > is defined in src/include/datatype/timestamp.h. > > #define DAYS_PER_YEAR365.25/* assumes leap year every four years

RE: [Proposal] Add foreign-server health checks infrastructure

2022-02-23 Thread kuroda.hay...@fujitsu.com
Dear Fujii-san, Thank you for your quick reviewing! I attached new version. I found previous patches have wrong name. Sorry. > The connection check timer is re-scheduled repeatedly even while the backend > is > in idle state or is running a local transaction that doesn't access to any >

Re: row filtering for logical replication

2022-02-23 Thread Amit Kapila
On Thu, Feb 24, 2022 at 7:57 AM Peter Smith wrote: > > I noticed that there was a build-farm failure on the machine 'komodoensis' [1] > > # Failed test 'check replicated rows to tab_rowfilter_toast' > # at t/028_row_filter.pl line 687. > # got: '' > # expected: 't|1' > # Looks

RE: row filtering for logical replication

2022-02-23 Thread Shinoda, Noriyoshi (PN Japan FSIP)
> You can give it for multiple tables. See below as an example: Thank you very much. I understood. Regards, Noriyoshi Shinoda -Original Message- From: Amit Kapila Sent: Thursday, February 24, 2022 11:25 AM To: Shinoda, Noriyoshi (PN Japan FSIP) Cc: Euler Taveira ;

Re: row filtering for logical replication

2022-02-23 Thread Peter Smith
I noticed that there was a build-farm failure on the machine 'komodoensis' [1] # Failed test 'check replicated rows to tab_rowfilter_toast' # at t/028_row_filter.pl line 687. # got: '' # expected: 't|1' # Looks like you failed 1 test of 20. [18:21:24] t/028_row_filter.pl

Re: row filtering for logical replication

2022-02-23 Thread Amit Kapila
On Thu, Feb 24, 2022 at 7:43 AM Shinoda, Noriyoshi (PN Japan FSIP) wrote: > > Hi, > Thank you for developing of the great feature. > If multiple tables are specified when creating a PUBLICATION, > is it supposed that the WHERE clause condition is given to only one table? > You can give it for

RE: row filtering for logical replication

2022-02-23 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hi, Thank you for developing of the great feature. If multiple tables are specified when creating a PUBLICATION, is it supposed that the WHERE clause condition is given to only one table? I attached the operation log below. --- operation log --- postgres=> CREATE TABLE data1(c1 INT PRIMARY

Re: Failed transaction statistics to measure the logical replication progress

2022-02-23 Thread Masahiko Sawada
On Mon, Feb 21, 2022 at 12:45 PM osumi.takami...@fujitsu.com wrote: > > On Saturday, February 19, 2022 12:00 AM osumi.takami...@fujitsu.com > wrote: > > On Friday, February 18, 2022 3:34 PM Tang, Haiying/唐 海英 > > wrote: > > > On Wed, Jan 12, 2022 8:35 PM osumi.takami...@fujitsu.com > > >

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Tom Lane
Peter Eisentraut writes: > On 23.02.22 23:58, Tom Lane wrote: >> Peter Eisentraut writes: >>> libpq TAP tests should be in src/interfaces/libpq/t/. >> That's failing to account for the fact that a libpq test can't >> really be a pure-perl TAP test; you need some C code to drive the >> library.

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-23 Thread Masahiko Sawada
Hi, On Wed, Feb 23, 2022 at 12:08 PM Peter Smith wrote: > > Hi. Below are my review comments for the v1 patch. Thank you for the comments! I've attached the latest version patch that incorporated all comments I got so far. The primary change from the previous version is that the subscription

Extract epoch from Interval weird behavior

2022-02-23 Thread Joseph Koshakow
Hi all, I noticed something odd when going through some of the Interval code. The DAYS_PER_YEAR constant is defined in src/include/datatype/timestamp.h. > #define DAYS_PER_YEAR365.25/* assumes leap year every four years */ We execute the EXTRACT and date_part functions in

[PATCH] Expose port->authn_id to extensions and triggers

2022-02-23 Thread Jacob Champion
Hi all, Stephen pointed out [1] that the authenticated identity that's stored in MyProcPort can't be retrieved by extensions or triggers. Attached is a patch that provides both a C API and a SQL function for retrieving it. GetAuthenticatedIdentityString() is a mouthful but I wanted to

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Peter Eisentraut
On 23.02.22 23:58, Tom Lane wrote: Peter Eisentraut writes: On 23.02.22 21:30, Andres Freund wrote: Where would we want that test to live? Right now we have the slightly odd convention that some tap tests live in src/test/{misc,modules,...}. But e.g. frontend binary ones are below src/bin/.

Re: [PATCH] Enable SSL library detection via PQsslAttribute

2022-02-23 Thread Jacob Champion
On Wed, 2022-02-23 at 14:11 -0500, Andrew Dunstan wrote: > On 2/23/22 13:38, Jacob Champion wrote: > > > > If this looks good, I'm not sure how best to test it in the regression > > suite. I see that libpq has an installcheck recipe that compiles a test > > executable for URI parsing; should I

Re: libpq async duplicate error results

2022-02-23 Thread Tom Lane
I wrote: > However ... in the wake of 618c16707, I wonder if we should consider > an alternative definition, which is to NOT clear errorMessage when > starting a new pipelined query. (That would be this patch minus > the addition to pqPipelineProcessQueue.) Thanks to 618c16707, > the returned

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Tom Lane
Peter Eisentraut writes: > On 23.02.22 21:30, Andres Freund wrote: >> Where would we want that test to live? Right now we have the slightly odd >> convention that some tap tests live in src/test/{misc,modules,...}. But >> e.g. frontend binary ones are below src/bin/. > libpq TAP tests should be

Re: Some optimisations for numeric division

2022-02-23 Thread Tom Lane
Dean Rasheed writes: > On Wed, 23 Feb 2022 at 20:55, Tom Lane wrote: >> I'm +1 on 0001 and 0002, but 0003 feels >> a bit ad-hoc. It certainly *looks* weird for the allegedly faster >> function to be handing off to the allegedly slower one. I also wonder >> if we're leaving anything on the

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Peter Eisentraut
On 23.02.22 21:30, Andres Freund wrote: hen verifying that the meson port actually runs all perl based tests I came across src/interfaces/libpq/test/regress.pl. Instead of running tests yet another way, it seems better to convert it to a tap test. I hope others agree? Where would we want that

Re: Some optimisations for numeric division

2022-02-23 Thread Dean Rasheed
On Wed, 23 Feb 2022 at 20:55, Tom Lane wrote: > > I took a quick look through these (just eyeball, didn't try to verify > your performance statements). Thanks for looking! > I'm +1 on 0001 and 0002, but 0003 feels > a bit ad-hoc. It certainly *looks* weird for the allegedly faster > function

Re: Proposal: Support custom authentication methods using hooks

2022-02-23 Thread samay sharma
Hi all, On Thu, Feb 17, 2022 at 11:25 AM samay sharma wrote: > Hi all, > > I wanted to submit a patch to expose 2 new hooks (one for the > authentication check and another one for error reporting) in auth.c. These > will allow users to implement their own authentication methods for Postgres >

Re: logical decoding and replication of sequences

2022-02-23 Thread Euler Taveira
On Wed, Feb 23, 2022, at 4:18 PM, Tomas Vondra wrote: > On 2/23/22 18:33, Euler Taveira wrote: > > On Wed, Feb 23, 2022, at 1:07 PM, Tomas Vondra wrote: > >> Maybe, but I don't think it's very common to have that many > >> schemas added to the same publication. And it probably does not > >> make

Re: reallocing without oom check in pg_regress

2022-02-23 Thread Tom Lane
Daniel Gustafsson writes: > In pg_regress we realloc() with the destination and source pointer being > equal, > without checking for OOM. While a fairly unlikely source of errors, is there > a > reason not to use pg_realloc() there for hygiene? Yeah, looks like oversight to me.

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Tom Lane
Alvaro Herrera writes: > On 2022-Feb-23, Andres Freund wrote: >> Separately: I don't really understand why we do the whole if USE_PGXS dance >> in >> src/test/modules? > Yeah, it seems a bit silly. I'm not opposed to making these makefiles > unconditionally do the PGXS thing -- or the non-PGXS

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Andrew Dunstan
On 2/23/22 16:16, Andres Freund wrote: > Hi, > > On 2022-02-23 15:57:08 -0500, Tom Lane wrote: >> Andrew Dunstan writes: >>> On 2/23/22 15:30, Andres Freund wrote: Perhaps we should just rename src/test/modules/libpq_pipeline to src/test/modules/libpq and move uri-regress in there as

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Alvaro Herrera
On 2022-Feb-23, Andres Freund wrote: > Separately: I don't really understand why we do the whole if USE_PGXS dance in > src/test/modules? Yeah, it seems a bit silly. I'm not opposed to making these makefiles unconditionally do the PGXS thing -- or the non-PGXS thing, if that makes it easier to

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Alvaro Herrera
On 2022-Feb-23, Andres Freund wrote: > When verifying that the meson port actually runs all perl based tests I came > across src/interfaces/libpq/test/regress.pl. Instead of running tests yet > another way, it seems better to convert it to a tap test. > > I hope others agree? WFM. > Perhaps

reallocing without oom check in pg_regress

2022-02-23 Thread Daniel Gustafsson
In pg_regress we realloc() with the destination and source pointer being equal, without checking for OOM. While a fairly unlikely source of errors, is there a reason not to use pg_realloc() there for hygiene? -- Daniel Gustafsson https://vmware.com/ pg_regress_realloc.diff

GiST index build missing smgrimmedsync()?

2022-02-23 Thread Melanie Plageman
I brought this up in [1] but wanted to start a dedicated thread. Since 16fa9b2b30a357 GiST indexes are not built in shared buffers. However, smgrimmedsync() is not done anywhere and skipFsync=true is always passed to smgrwrite() and smgrextend(). So, if a checkpoint starts and finishes after

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Andres Freund
Hi, On 2022-02-23 15:57:08 -0500, Tom Lane wrote: > Andrew Dunstan writes: > > On 2/23/22 15:30, Andres Freund wrote: > >> Perhaps we should just rename src/test/modules/libpq_pipeline to > >> src/test/modules/libpq and move uri-regress in there as well? > > > WFM > > +1 Cool. One question

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Tom Lane
Andrew Dunstan writes: > On 2/23/22 15:30, Andres Freund wrote: >> Perhaps we should just rename src/test/modules/libpq_pipeline to >> src/test/modules/libpq and move uri-regress in there as well? > WFM +1 regards, tom lane

Re: Some optimisations for numeric division

2022-02-23 Thread Tom Lane
Dean Rasheed writes: > Attached are 3 small patches that improve the performance of numeric > division. Patch 0002 seems to have the biggest impact, but I think > they're all worth including, since they're quite simple changes, with > noticeable performance gains. I took a quick look through

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Daniel Gustafsson
> On 23 Feb 2022, at 21:30, Andres Freund wrote: > When verifying that the meson port actually runs all perl based tests I came > across src/interfaces/libpq/test/regress.pl. Instead of running tests yet > another way, it seems better to convert it to a tap test. > > I hope others agree? Many

Re: convert libpq uri-regress tests to tap test

2022-02-23 Thread Andrew Dunstan
On 2/23/22 15:30, Andres Freund wrote: > Hi, > > When verifying that the meson port actually runs all perl based tests I came > across src/interfaces/libpq/test/regress.pl. Instead of running tests yet > another way, it seems better to convert it to a tap test. > > I hope others agree? yes

convert libpq uri-regress tests to tap test

2022-02-23 Thread Andres Freund
Hi, When verifying that the meson port actually runs all perl based tests I came across src/interfaces/libpq/test/regress.pl. Instead of running tests yet another way, it seems better to convert it to a tap test. I hope others agree? Where would we want that test to live? Right now we have the

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Joel Jacobson
On Wed, Feb 23, 2022, at 15:23, Andrew Dunstan wrote: > Yeah, I have several others on bitbucket that might be of interest (in > varying states of doneness): Thanks, added. /Joel

Re: Trap errors from streaming child in pg_basebackup to exit early

2022-02-23 Thread Daniel Gustafsson
> On 22 Feb 2022, at 02:13, Michael Paquier wrote: > The patch set looks fine overall. Thanks for reviewing and testing, I pushed this today after taking another round at it. -- Daniel Gustafsson https://vmware.com/

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Matthias van de Meent
On Wed, 23 Feb 2022 at 15:24, Nitin Jadhav wrote: > > > At least for pg_stat_progress_checkpoint, storing only a timestamp in > > the pg_stat storage (instead of repeatedly updating the field as a > > duration) seems to provide much more precise measures of 'time > > elapsed' for other sessions

Re: fix crash with Python 3.11

2022-02-23 Thread Tom Lane
I wrote: > We could provide a simplified API in which SPI_start_transaction is > automatically included for either success or failure, so that the caller > doesn't have the delayed-cleanup responsibility. I'm not actually sure > that there is any situation where that couldn't be done, but putting

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Justin Pryzby
+ Whenever the checkpoint operation is running, the + pg_stat_progress_checkpoint view will contain a + single row indicating the progress of the checkpoint. The tables below Maybe it should show a single row , unless the checkpointer isn't running at all (like in single user mode). +

Re: Use generation context to speed up tuplesorts

2022-02-23 Thread Tomas Vondra
On 2/23/22 03:25, Andres Freund wrote: > Hi, > > On 2022-02-18 12:10:51 +1300, David Rowley wrote: >> The other way I thought to fix it was by changing the logic for when >> generation blocks are freed. In the problem case mentioned above, the >> block being freed is the current block (which was

Re: logical decoding and replication of sequences

2022-02-23 Thread Tomas Vondra
On 2/23/22 18:33, Euler Taveira wrote: > On Wed, Feb 23, 2022, at 1:07 PM, Tomas Vondra wrote: >> Maybe, but I don't think it's very common to have that many >> schemas added to the same publication. And it probably does not >> make much difference whether you have 1000 or 2000 items in the >>

Re: Synchronizing slots from primary to standby

2022-02-23 Thread James Coleman
On Fri, Feb 18, 2022 at 5:23 PM Andres Freund wrote: > > Hi, > > On 2022-02-11 15:28:19 +0100, Peter Eisentraut wrote: > > On 05.02.22 20:59, Andres Freund wrote: > > > On 2022-01-03 14:46:52 +0100, Peter Eisentraut wrote: > > > > From ec00dc6ab8bafefc00e9b1c78ac9348b643b8a87 Mon Sep 17 00:00:00

Re: [PATCH] Enable SSL library detection via PQsslAttribute

2022-02-23 Thread Andrew Dunstan
On 2/23/22 13:38, Jacob Champion wrote: > Hello, > > As part of the NSS work it came up [1] that clients don't have a good > way to ask libpq what SSL library it was compiled with, unless they > already have a connection pointer so that they can call > PQsslAttribute(conn, "library"). This poses

Re: CLUSTER on partitioned index

2022-02-23 Thread Justin Pryzby
On Tue, Jul 20, 2021 at 08:27:02PM -0400, Alvaro Herrera wrote: > I have to wonder if there really *is* a use case for CLUSTER in the > first place on regular tables, let alone on partitioned tables, which > are likely to be large and thus take a lot of time. What justifies > spending so much

Re: Add index scan progress to pg_stat_progress_vacuum

2022-02-23 Thread Peter Geoghegan
On Wed, Feb 23, 2022 at 10:02 AM Imseih (AWS), Sami wrote: > If the failsafe kicks in midway through a vacuum, the number indexes_total > will not be reset to 0. If INDEX_CLEANUP is turned off, then the value will > be 0 at the start of the vacuum. The way that this works with num_index_scans

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Alvaro Herrera
I think the change to ImmediateCheckpointRequested() makes no sense. Before this patch, that function merely inquires whether there's an immediate checkpoint queued. After this patch, it ... changes a progress-reporting flag? I think it would make more sense to make the progress-report flag

[PATCH] Enable SSL library detection via PQsslAttribute

2022-02-23 Thread Jacob Champion
Hello, As part of the NSS work it came up [1] that clients don't have a good way to ask libpq what SSL library it was compiled with, unless they already have a connection pointer so that they can call PQsslAttribute(conn, "library"). This poses a chicken-and-egg problem: with the NSS proposal,

Re: Add index scan progress to pg_stat_progress_vacuum

2022-02-23 Thread Imseih (AWS), Sami
+ + + indexes_total bigint + + + The number of indexes to be processed in the + vacuuming indexes or cleaning up indexes phase + of the vacuum. + + + + + +

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-23 Thread Nathan Bossart
On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: > To address that, I'd like to propose the possibility to include files in hba > and ident configuration files. This was already discussed in the past, and in > my understanding this is mostly wanted, while some people expressed

Re: logical decoding and replication of sequences

2022-02-23 Thread Euler Taveira
On Wed, Feb 23, 2022, at 1:07 PM, Tomas Vondra wrote: > Maybe, but I don't think it's very common to have that many schemas > added to the same publication. And it probably does not make much > difference whether you have 1000 or 2000 items in the list - either both > are acceptable or

Re: Frontend error logging style

2022-02-23 Thread Euler Taveira
On Wed, Feb 23, 2022, at 12:30 PM, Tom Lane wrote: > Updating the patch is going to be a bit tedious, so I'm not > going to do it without buy-in that this solution would be > okay to commit. Any objections? No. I was confused by the multiple log functions spread in the client programs while I was

Re: PATCH: add "--config-file=" option to pg_rewind

2022-02-23 Thread Jacob Champion
On Wed, 2022-02-23 at 16:28 +0100, Gunnar "Nick" Bluth wrote: > I'm extremely unhappy that I called it "--config-file" here, while > "postgres" itself wants "--config_file". But as the other dashed options > of pg_rewind are, well, dashed and not underscored, it seemed to be > better that the

Re: bailing out in tap tests nearly always a bad idea

2022-02-23 Thread Andrew Dunstan
On 2/23/22 11:40, Andres Freund wrote: > Hi, > > On 2022-02-23 08:43:38 -0500, Andrew Dunstan wrote: >> On 2/22/22 15:54, Andres Freund wrote: >>> On 2022-02-22 15:10:30 -0500, Andrew Dunstan wrote: I'll be surprised if we can't come up with something cleaner than that. >>> Suggestions? >>

Re: Frontend error logging style

2022-02-23 Thread Andres Freund
Hi, On 2022-02-23 10:30:02 -0500, Tom Lane wrote: > So I now propose modifying yesterday's patch thus: > > * Reinstantiate the PG_LOG_FATAL enum value, add support macros > pg_log_fatal, pg_log_fatal_hint, pg_log_fatal_detail. > > * Define pg_fatal as pg_log_fatal + exit(1). (This would

Re: bailing out in tap tests nearly always a bad idea

2022-02-23 Thread Andres Freund
Hi, On 2022-02-23 08:43:38 -0500, Andrew Dunstan wrote: > > On 2/22/22 15:54, Andres Freund wrote: > > On 2022-02-22 15:10:30 -0500, Andrew Dunstan wrote: > >> I'll be surprised if we can't come up with something cleaner than that. > > Suggestions? > > > If we just have the sig handler actions

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Ashutosh Sharma
+ if ((ckpt_flags & +(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY)) == 0) + { This code (present at multiple places) looks a little ugly to me, what we can do instead is add a macro probably named IsShutdownCheckpoint() which does the above check and use it in

Re: logical decoding and replication of sequences

2022-02-23 Thread Tomas Vondra
On 2/23/22 12:10, Amit Kapila wrote: > On Wed, Feb 23, 2022 at 4:54 AM Tomas Vondra > wrote: >> >> 7) This updates psql to do roughly the same thing as for tables, so \dRp >> now list sequences added either directly or through schema, so you might >> get footer like this: >> >> \dRp+

Re: Time to drop plpython2?

2022-02-23 Thread Mark Wong
On Tue, Feb 22, 2022 at 08:50:07PM -0500, Tom Lane wrote: > Mark Wong writes: > > Take 3. :) > > > I've upgraded everyone to the v14 buildfarm scripts and made sure the > > --test passed on HEAD on each one. So I hopefully didn't miss any > > (other than the one EOL OpenSUSE version that I will

Re: making pg_regress less noisy by removing boilerplate

2022-02-23 Thread Andrew Dunstan
On 2/22/22 17:06, Andres Freund wrote: >> What's involved in moving to require Unix socket support? > It works today (the CI scripts use it on windows for example). > > But it's awkward because make_temp_sockdir() defaults to /tmp/ if TMPDIR isn't > set. Which it is not by default on windows.

Re: Frontend error logging style

2022-02-23 Thread Tom Lane
I wrote: > Andres Freund writes: >> What about adding a pg_fatal() that's pg_log_fatal() + exit()? That keeps >> pg_log_* stuff "log only", but adds something adjacent enough to hopefully >> reduce future misunderstandings? > I'd be okay with that, except that pg_upgrade already has a pg_fatal >

PATCH: add "--config-file=" option to pg_rewind

2022-02-23 Thread Gunnar "Nick" Bluth
Hi! During a Patroni PR discussion (https://github.com/zalando/patroni/pull/2225), we realised that if one wants to use the "-c" option on a typical Debian/Ubuntu installation (where the config resides below /etc/postgresql/), pg_rewind needs a way to be told where the postgresql.conf

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Joe Conway
On 2/23/22 09:52, Aleksander Alekseev wrote: > What about scanning for "PG_MODULE_MAGIC"? An extension can be written without using C at all. BTW some extensions [1] are written in Rust these days. Sure, but scanning for PG_MODULE_MAGIC may well pick up repos that would otherwise have been

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Aleksander Alekseev
Hi Joe, > What about scanning for "PG_MODULE_MAGIC"? An extension can be written without using C at all. BTW some extensions [1] are written in Rust these days. [1]: https://github.com/timescale/timescaledb-toolkit -- Best regards, Aleksander Alekseev

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Joe Conway
On 2/23/22 09:33, Euler Taveira wrote: On Wed, Feb 23, 2022, at 6:00 AM, Joel Jacobson wrote: On Fri, Feb 11, 2022, at 04:46, Noah Misch wrote: > How did you make the list?  (I'd imagine doing it by searching for > repositories containing evidence like \bpgxs\b matches.) Searching Github for

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Euler Taveira
On Wed, Feb 23, 2022, at 6:00 AM, Joel Jacobson wrote: > On Fri, Feb 11, 2022, at 04:46, Noah Misch wrote: > > How did you make the list? (I'd imagine doing it by searching for > > repositories containing evidence like \bpgxs\b matches.) > > Searching Github for repos with a *.control file in

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
> At least for pg_stat_progress_checkpoint, storing only a timestamp in > the pg_stat storage (instead of repeatedly updating the field as a > duration) seems to provide much more precise measures of 'time > elapsed' for other sessions if one step of the checkpoint is taking a > long time. I am

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Andrew Dunstan
On 2/23/22 03:52, Joel Jacobson wrote: > >> (e.g. I see three >> instances of >> blackhole_fdw listed, but not the original repo, which is not even on >> GitHub) etc.. > Thanks, I've fixed blackhole_fdw manually: > > - https://bitbucket.org/adunstan/blackhole_fdw > -

Re: bailing out in tap tests nearly always a bad idea

2022-02-23 Thread Andrew Dunstan
On 2/22/22 15:54, Andres Freund wrote: > On 2022-02-22 15:10:30 -0500, Andrew Dunstan wrote: >> I'll be surprised if we can't come up with something cleaner than that. > Suggestions? If we just have the sig handler actions as:     diag("died: $_[0]");     done_testing();    we get:     ok 1

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
> On what basis have you classified the above into the various types of > checkpoints? AFAIK, the first two types are based on what triggered > the checkpoint (whether it was the checkpoint_timeout or maz_wal_size > settings) while the third type indicates the force checkpoint that can > happen

Some optimisations for numeric division

2022-02-23 Thread Dean Rasheed
Attached are 3 small patches that improve the performance of numeric division. Patch 0002 seems to have the biggest impact, but I think they're all worth including, since they're quite simple changes, with noticeable performance gains. Patch 0001 vectorises the inner loop of div_var_fast(). This

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
> I will make use of pgstat_progress_update_multi_param() in the next > patch to replace multiple calls to checkpoint_progress_update_param(). Fixed. --- > > The other progress tables use [type]_total as column names for counter > > targets (e.g. backup_total for backup_streamed, heap_blks_total

Re: C++ Trigger Framework

2022-02-23 Thread Shmuel Kamensky
Hi Nathan, Thanks for the reply. I did indeed read that document and it's a great place to get started. But it doesn't quite answer my questions. Do you personally have experience writing software in C++ that interacts with Postgres? On Tue, Feb 22, 2022, 23:07 Nathan Bossart wrote: > On Tue,

Re: logical decoding and replication of sequences

2022-02-23 Thread Amit Kapila
On Wed, Feb 23, 2022 at 4:54 AM Tomas Vondra wrote: > > 7) This updates psql to do roughly the same thing as for tables, so \dRp > now list sequences added either directly or through schema, so you might > get footer like this: > > \dRp+ testpub_mix > ... > Tables: >

Re: Using Test::More test functions for pg_rewind

2022-02-23 Thread Daniel Gustafsson
> On 21 Feb 2022, at 16:17, Andrew Dunstan wrote: > Looks OK. Thanks, pushed. -- Daniel Gustafsson https://vmware.com/

[PATCH] Add native windows on arm64 support

2022-02-23 Thread Niyas Sait
Hello, I have created a patch that adds support for building Postgres for the windows/arm64 platform using the MSVC toolchain. Following changes have been included 1. Extend MSVC scripts to handle ARM64 platform. 2. Add arm64 definition of spin_delay function. 3. Exclude arm_acle.h import with

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-23 Thread tanghy.f...@fujitsu.com
Hi Osumi-san, I have a comment on v21 patch. I wonder if we really need subscription s2 in 028_disable_on_error.pl. I think for subscription s2, we only tested some normal cases(which could be tested with s1), and didn't test any error case, which means it wouldn't be automatically disabled.

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Joel Jacobson
On Fri, Feb 11, 2022, at 04:46, Noah Misch wrote: > How did you make the list? (I'd imagine doing it by searching for > repositories containing evidence like \bpgxs\b matches.) Searching Github for repos with a *.control file in the root dir and a Makefile containing ^PGXS Hmm, now that you

Re: Logical replication timeout problem

2022-02-23 Thread Amit Kapila
On Tue, Feb 22, 2022 at 9:17 AM wangw.f...@fujitsu.com wrote: > > On Fri, Feb 18, 2022 at 10:51 AM Ajin Cherian wrote: > > Some comments: > Thanks for your review. > > > I see you only track skipped Inserts/Updates and Deletes. What about > > DDL operations that are skipped, what about

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Joel Jacobson
On Fri, Feb 11, 2022, at 01:22, Ian Lawrence Barwick wrote: > More precisely it's a list of all? the repositories with PostgreSQL > extensions on > GitHub? Yes, the ambition is to list all repos at GitHub, and to manually add repos hosted at GitLab and other places. > OTOH not everything is on

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Joel Jacobson
On Tue, Feb 22, 2022, at 09:13, Aleksander Alekseev wrote: > Hi Joel, > >> I've compiled a list of all* PostgreSQL EXTENSIONs in the world: > > The list is great. Thanks for doing this! Glad to hear! > Just curious, is it a one-time experiment or you are going to keep the > list in an actual