Incremental backup from a streaming replication standby

2024-06-28 Thread Laurenz Albe
I played around with incremental backup yesterday and tried $subject The WAL summarizer is running on the standby server, but when I try to take an incremental backup, I get an error that I understand to mean that WAL summarizing hasn't caught up yet. I am not sure if that is working as

Re: Backporting BackgroundPsql

2024-06-28 Thread Pavan Deolasee
On Thu, Jun 27, 2024 at 10:05 PM Heikki Linnakangas wrote: > > Ok, I pushed commits to backport BackgroundPsql down to v12. I used > "option 2", i.e. I changed background_psql() to return the new > BackgroundPsql object. > > Don't we need to add install and uninstall rules for the new module,

Re: Inval reliability, especially for inplace updates

2024-06-28 Thread Noah Misch
On Mon, Jun 17, 2024 at 04:58:54PM -0700, Noah Misch wrote: > attached v2 patch stack. Rebased. This applies on top of three patches from https://postgr.es/m/20240629024251.03.nmi...@google.com. I'm attaching those to placate cfbot, but this thread is for review of the last patch only. Author:

Re: race condition in pg_class

2024-06-28 Thread Noah Misch
On Fri, Jun 28, 2024 at 01:17:22AM -0400, Tom Lane wrote: > Noah Misch writes: > > Pushed. Buildfarm member prion is failing the new inplace-inval.spec, > > almost > > surely because prion uses -DCATCACHE_FORCE_RELEASE and inplace-inval.spec is > > testing an extant failure to inval a cache

Re: cfbot update: Using GitHub for patch review

2024-06-28 Thread Thomas Munro
On Sat, Jun 29, 2024 at 1:10 AM Ashutosh Bapat wrote: > I need to sign in to github to add my review comments. So those who do not > have a github account can not use it for review. But I don't think that can > be fixed. We need a way to know who left review comments. I don't think Jelte was

Re: Restart pg_usleep when interrupted

2024-06-28 Thread Sami Imseih
Thanks for the feedback! > On Jun 28, 2024, at 4:34 PM, Tom Lane wrote: > > Sami Imseih writes: >> Reattaching the patch. > > I feel like this is fundamentally a wrong solution, for the reasons > cited in the comment for pg_usleep: long sleeps are a bad idea > because of the resulting

Re: Restart pg_usleep when interrupted

2024-06-28 Thread Thomas Munro
On Sat, Jun 29, 2024 at 9:34 AM Tom Lane wrote: > Therefore, rather than "improving" pg_usleep (and uglifying its API), > the right answer is to fix parallel vacuum leaders to not depend on > pg_usleep in the first place. A better idea might be to use > pg_sleep() or equivalent code. In case

Re: Add LSN <-> time conversion functionality

2024-06-28 Thread Melanie Plageman
On Wed, Jun 26, 2024 at 10:04 PM Melanie Plageman wrote: > > I've implemented these review points in the attached v4. I realized the docs had a compilation error. Attached v5 fixes that as well as three bugs I found while using this patch set more intensely today. I see Michael has been working

Re: Confine vacuum skip logic to lazy_scan_skip

2024-06-28 Thread Melanie Plageman
On Sun, Mar 17, 2024 at 2:53 AM Thomas Munro wrote: > > On Tue, Mar 12, 2024 at 10:03 AM Melanie Plageman > wrote: > > I've rebased the attached v10 over top of the changes to > > lazy_scan_heap() Heikki just committed and over the v6 streaming read > > patch set. I started testing them and see

Re: Restart pg_usleep when interrupted

2024-06-28 Thread Tom Lane
Sami Imseih writes: > Reattaching the patch. I feel like this is fundamentally a wrong solution, for the reasons cited in the comment for pg_usleep: long sleeps are a bad idea because of the resulting uncertainty about whether we'll respond to interrupts and such promptly. An example here is

Re: Restart pg_usleep when interrupted

2024-06-28 Thread Sami Imseih
> We want patches to be in the pgsql-hackers archives, not temporarily > accessible via some link. > > …Robert > Moving to another email going forward. Reattaching the patch. 0001-Handle-Sleep-interrupts.patch Description: Binary data Regards, Sami Imseih Amazon Web Services

Doc: Move standalone backup section, mention -X argument

2024-06-28 Thread David G. Johnston
A documentation comment came in [1] causing me to review some of our backup documentation and I left the current content and location of the standalone backups was odd. I propose to move it to a better place, under file system backups. Adding to commitfest. David J. [1]

Re: Restart pg_usleep when interrupted

2024-06-28 Thread Imseih (AWS), Sami
> I think you need to find a way to disable this "Attachment protected > by Amazon" thing: Yes, I am looking into that. I only noticed after sending the email. Sorry about that. Sami

Re: Restart pg_usleep when interrupted

2024-06-28 Thread Robert Haas
Hi, I think you need to find a way to disable this "Attachment protected by Amazon" thing: http://postgr.es/m/01000190606e3d2a-116ead16-84d2-4449-8d18-5053da66b1f4-000...@email.amazonses.com We want patches to be in the pgsql-hackers archives, not temporarily accessible via some link.

Re: Track the amount of time waiting due to cost_delay

2024-06-28 Thread Imseih (AWS), Sami
> 46ebdfe164 will interrupt the leaders sleep every time a parallel workers > reports > progress, and we currently don't handle interrupts by restarting the sleep > with > the remaining time. nanosleep does provide the ability to restart with the > remaining > time [1], but I don't think it's

Restart pg_usleep when interrupted

2024-06-28 Thread Imseih (AWS), Sami
Attachment protected by Amazon: [0001-Handle-Sleep-interrupts.patch] https://us-east-1.secure-attach.amazon.com/fcdc82ce-7887-4aa1-af9e-c1161a6b1d6f/bc81fa24-41de-48f9-a767-a6d15801754b Amazon has replaced attachment with download link. Downloads will be available until July 28, 2024, 19:59

Re: Converting README documentation to Markdown

2024-06-28 Thread Peter Eisentraut
On 28.06.24 11:56, Jelte Fennema-Nio wrote: On Fri, 28 Jun 2024 at 09:38, Peter Eisentraut wrote: Getting that right in Markdown can be quite tricky. I agree that in some cases it's tricky. But what's the worst case that can happen when you get it wrong? It renders weird on github.com. I

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-28 Thread Peter Geoghegan
On Thu, Jun 27, 2024 at 11:06 PM wrote: > OK. I would like to understand more about your proposed patch. I > have also registered as a reviewer in the commitfests entry. Great! > Although I haven't looked on your patch yet, if it's difficult to know > how it can optimize during the planning

Re: Check lateral references within PHVs for memoize cache keys

2024-06-28 Thread Andrei Lepikhov
On 6/18/24 08:47, Richard Guo wrote: On Mon, Mar 18, 2024 at 4:36 PM Richard Guo wrote: Here is another rebase over master so it applies again. I also added a commit message to help review. Nothing else has changed. AFAIU currently we do not add Memoize nodes on top of join relation paths.

Re: Why is infinite_recurse test suddenly failing?

2024-06-28 Thread Alexander Lakhin
Hello hackers, 15.08.2019 10:17, Thomas Munro wrote: On Thu, Aug 15, 2019 at 5:49 PM Tom Lane wrote: So that leads to the thought that "the infinite_recurse test is fine if it runs by itself, but it tends to fall over if there are concurrently-running backends". I have absolutely no idea how

Re: remaining sql/json patches

2024-06-28 Thread Amit Langote
Hi Alexander, On Fri, Jun 28, 2024 at 5:00 PM Alexander Lakhin wrote: > > Hi Amit, > > 28.06.2024 09:15, Amit Langote wrote: > > Hi Alexander, > > > > > > Thanks for the report. Yeah, those comments that got added in > > 7081ac46ace are obsolete. > > > > Thanks for paying attention to that! > >

Re: cfbot update: Using GitHub for patch review

2024-06-28 Thread Ashutosh Bapat
On Fri, Jun 21, 2024 at 8:06 PM Jelte Fennema-Nio wrote: > I recently got write access to the cfbot repo[1] and machine from > Thomas. And I deployed a few improvements this week. The most > significant one is that it is now much easier to use GitHub as part of > your patch review workflow. > >

Re: walsender.c comment with no context is hard to understand

2024-06-28 Thread Bertrand Drouvot
Hi, On Fri, Jun 28, 2024 at 03:15:22PM +0530, Amit Kapila wrote: > On Fri, Jun 28, 2024 at 12:55 PM Peter Smith wrote: > > > > On Fri, Jun 28, 2024 at 4:18 PM Amit Kapila wrote: > > > > > > On Fri, Jun 28, 2024 at 5:15 AM Peter Smith wrote: > > > > > > > > On Thu, Jun 27, 2024 at 3:44 PM

Re: Converting README documentation to Markdown

2024-06-28 Thread Tatsuo Ishii
> I've been thinking about this some more. I think the most value here > would be to just improve the plain-text formatting, so that there are > consistent list styles, header styles, indentation, some of the > ambiguities cleared up -- much of which your 0001 patch does. You > might as well be

Re: pgindent exit status if a file encounters an error

2024-06-28 Thread Ashutosh Bapat
On Wed, Jun 26, 2024 at 8:54 PM Tom Lane wrote: > Ashutosh Bapat writes: > > The usage help mentions exit code 2 specifically while describing --check > > option but it doesn't mention exit code 1. Neither does the README. So I > > don't think we need to document exit code 3 anywhere. Please

Re: Test to dump and restore objects left behind by regression

2024-06-28 Thread Ashutosh Bapat
Sorry for delay, but here's next version of the patchset for review. On Thu, Jun 6, 2024 at 5:07 AM Michael Paquier wrote: > On Wed, Jun 05, 2024 at 05:09:58PM +0530, Ashutosh Bapat wrote: > > Thanks for the suggestion. I didn't understand the dependency with the > > buildfarm module. Will the

Re: SQL:2011 application time

2024-06-28 Thread Robert Haas
On Thu, Jun 27, 2024 at 5:56 PM Paul Jungwirth wrote: > I did add a relperiods column, but I have a mostly-complete branch here (not > included in the > patches) that does without. Not maintaining that new column is simpler for > sure. The consequence is > that the relcache must scan for

Re: Parallel heap vacuum

2024-06-28 Thread Amit Kapila
On Fri, Jun 28, 2024 at 9:44 AM Masahiko Sawada wrote: > > # Benchmark results > > * Test-1: parallel heap scan on the table without indexes > > I created 20GB table, made garbage on the table, and run vacuum while > changing parallel degree: > > create unlogged table test (a int) with

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-28 Thread Matthias van de Meent
On Fri, 28 Jun 2024 at 10:59, Jelte Fennema-Nio wrote: > > On Fri, 28 Jun 2024 at 00:41, Peter Geoghegan wrote: > > Typically, no, it won't be. But there's really no telling for sure. > > The access patterns for a composite index on '(a, b)' with a qual > > "WHERE b = 5" are identical to a qual

Re: pgsql: Add more SQL/JSON constructor functions

2024-06-28 Thread Amit Langote
On Fri, Jun 28, 2024 at 3:14 PM jian he wrote: > On Thu, Jun 27, 2024 at 7:48 PM Amit Langote wrote: > > > > > > I've attempted that in the attached 0001, which removes > > > JsonExpr.coercion_expr and a bunch of code around it. > > > > > > 0002 is now the original patch minus the changes to

Re: Showing applied extended statistics in explain Part 2

2024-06-28 Thread Tatsuro Yamada
Hi Tomas and All, Attached file is a new patch including: 6) Add stats option to explain command 7) The patch really needs some docs (partly) >4) Add new node (resolve errors in cfbot and prepared statement) I tried adding a new node in pathnode.h, but it doesn't work well. So, it

Re: pg_createsubscriber: drop pre-existing subscriptions from the converted node

2024-06-28 Thread Amit Kapila
On Thu, Jun 27, 2024 at 11:47 AM Hayato Kuroda (Fujitsu) wrote: > > > It seems disabling subscriptions on the primary can make the primary > > stop functioning for some duration of time. I feel we need some > > solution where after converting to subscriber, we disable and drop > > pre-existing

Re: recoveryCheck/008_fsm_truncation is failing on dodo in v14- (due to slow fsync?)

2024-06-28 Thread Robins Tharakan
On Sun, 23 Jun 2024 at 22:30, Alexander Lakhin wrote: > Unfortunately, the buildfarm log doesn't contain regress_log_002_limits, > but I managed to reproduce the failure on that my device, when it's storage > as slow as: > $ dd if=/dev/zero of=./test count=1024 oflag=dsync bs=128k > 1024+0

Re: Converting README documentation to Markdown

2024-06-28 Thread Jelte Fennema-Nio
On Fri, 28 Jun 2024 at 09:38, Peter Eisentraut wrote: > Getting that right in Markdown can be quite tricky. I agree that in some cases it's tricky. But what's the worst case that can happen when you get it wrong? It renders weird on github.com. Luckily there's a "code" button to go to the plain

Re: walsender.c comment with no context is hard to understand

2024-06-28 Thread Amit Kapila
On Fri, Jun 28, 2024 at 12:55 PM Peter Smith wrote: > > On Fri, Jun 28, 2024 at 4:18 PM Amit Kapila wrote: > > > > On Fri, Jun 28, 2024 at 5:15 AM Peter Smith wrote: > > > > > > On Thu, Jun 27, 2024 at 3:44 PM Michael Paquier > > > wrote: > > > > > > > > On Wed, Jun 26, 2024 at 02:30:26PM

Re: Improve EXPLAIN output for multicolumn B-Tree Index

2024-06-28 Thread Jelte Fennema-Nio
On Fri, 28 Jun 2024 at 00:41, Peter Geoghegan wrote: > Typically, no, it won't be. But there's really no telling for sure. > The access patterns for a composite index on '(a, b)' with a qual > "WHERE b = 5" are identical to a qual explicitly written "WHERE a = > any() AND b = 5". Hmm, that's

Re: remaining sql/json patches

2024-06-28 Thread Alexander Lakhin
Hi Amit, 28.06.2024 09:15, Amit Langote wrote: Hi Alexander, Thanks for the report. Yeah, those comments that got added in 7081ac46ace are obsolete. Thanks for paying attention to that! Could you also look at comments for transformJsonObjectAgg() and transformJsonArrayAgg(), aren't they

Re: Converting README documentation to Markdown

2024-06-28 Thread Peter Eisentraut
On 15.05.24 14:26, Daniel Gustafsson wrote: Another aspect of platform/flavour was to make the markdown version easy to maintain for hackers writing content. Requiring the minimum amount of markup seems like the developer-friendly way here to keep productivity as well as document quality high.

Re: Flush pgstats file during checkpoints

2024-06-28 Thread Konstantin Knizhnik
On 18/06/2024 9:01 am, Michael Paquier wrote: Hi all, On HEAD, xlog.c has the following comment, which has been on my own TODO list for a couple of weeks now: * TODO: With a bit of extra work we could just start with a pgstat file * associated with the checkpoint redo location

Re: walsender.c comment with no context is hard to understand

2024-06-28 Thread Peter Smith
On Fri, Jun 28, 2024 at 4:18 PM Amit Kapila wrote: > > On Fri, Jun 28, 2024 at 5:15 AM Peter Smith wrote: > > > > On Thu, Jun 27, 2024 at 3:44 PM Michael Paquier wrote: > > > > > > On Wed, Jun 26, 2024 at 02:30:26PM +0530, vignesh C wrote: > > > > On Mon, 3 Jun 2024 at 11:21, Peter Smith

Re: Support "Right Semi Join" plan shapes

2024-06-28 Thread Richard Guo
On Fri, Jun 28, 2024 at 2:54 PM Richard Guo wrote: > On Mon, Jun 24, 2024 at 5:59 PM Richard Guo wrote: > > I noticed that this patch changes the plan of a query in join.sql from > > a semi join to right semi join, compromising the original purpose of > > this query, which was to test the fix

Re: Document use of ldapurl with LDAP simple bind

2024-06-28 Thread Peter Eisentraut
On 24.05.24 20:54, Jacob Champion wrote: Our documentation implies that the ldapurl setting in pg_hba is used for search+bind mode only. It was pointed out to me recently that this is not true, and if you're dealing with simple bind on a non-standard scheme or port, then ldapurl makes the HBA

Re: Adminpack removal

2024-06-28 Thread Philippe BEAUDOIN
Le 27/06/2024 à 10:38, Matthias van de Meent a écrit : On Thu, 27 Jun 2024, 07:34 Philippe BEAUDOIN, wrote: Hi, I have just tested PG17 beta1 with the E-Maj solution I maintain. The only issue I found is the removal of the adminpack contrib. In the emaj extension, which is the heart of the

Re: Support "Right Semi Join" plan shapes

2024-06-28 Thread Richard Guo
On Mon, Jun 24, 2024 at 5:59 PM Richard Guo wrote: > I noticed that this patch changes the plan of a query in join.sql from > a semi join to right semi join, compromising the original purpose of > this query, which was to test the fix for neqjoinsel's behavior for > semijoins (see commit

Re: Showing applied extended statistics in explain Part 2

2024-06-28 Thread Tatsuro Yamada
Hi Tomas, The attached patch does not correspond to the above comment. > But it does solve some of the issues mentioned in previous threads. > Oops, I made a mistake sending a patch on my previous email. Attached patch is the right patch. Regards, Tatsuro Yamada

Re: Support LIKE with nondeterministic collations

2024-06-28 Thread Peter Eisentraut
Here is an updated patch for this. I have added some more documentation based on the discussions, including some examples taken directly from the emails here. One thing I have been struggling with a bit is the correct use of LIKE_FALSE versus LIKE_ABORT in the MatchText() code. I have made

Re: walsender.c comment with no context is hard to understand

2024-06-28 Thread Amit Kapila
On Fri, Jun 28, 2024 at 5:15 AM Peter Smith wrote: > > On Thu, Jun 27, 2024 at 3:44 PM Michael Paquier wrote: > > > > On Wed, Jun 26, 2024 at 02:30:26PM +0530, vignesh C wrote: > > > On Mon, 3 Jun 2024 at 11:21, Peter Smith wrote: > > >> Perhaps the comment should say something like it used to:

Re: remaining sql/json patches

2024-06-28 Thread Amit Langote
Hi Alexander, On Wed, Jun 26, 2024 at 8:00 PM Alexander Lakhin wrote: > > Hello, > > I'm not sure I've chosen the most appropriate thread for reporting the > issue, but maybe you would like to look at code comments related to > SQL/JSON constructors: > > * Transform JSON_ARRAY() constructor. >

Re: pgsql: Add more SQL/JSON constructor functions

2024-06-28 Thread jian he
On Thu, Jun 27, 2024 at 7:48 PM Amit Langote wrote: > > > > I've attempted that in the attached 0001, which removes > > JsonExpr.coercion_expr and a bunch of code around it. > > > > 0002 is now the original patch minus the changes to make > > JSON_EXISTS(), JSON_QUERY(), and JSON_VALUE() behave