Re: WAL Insertion Lock Improvements

2023-04-09 Thread Michael Paquier
On Mon, Feb 20, 2023 at 09:49:48PM -0800, Nathan Bossart wrote: > I'm marking this as ready-for-committer. I think a couple of the comments > could use some small adjustments, but that probably doesn't need to hold up > this patch. Apologies. I was planning to have a thorough look at this patch

Re: PGDOCS - function pg_get_publication_tables is not documented?

2023-04-09 Thread Amit Kapila
On Sun, Apr 9, 2023 at 7:35 AM Yu Shi (Fujitsu) wrote: > > On Fri, Mar 24, 2023 6:26 AM Tom Lane wrote: > > > > I do see a docs change that I think would be worth making: get > > rid of the explicit mention of it in create_subscription.sgml > > in favor of using that view. > > > > I agree and I

Re: CREATE SUBSCRIPTION -- add missing tab-completes

2023-04-09 Thread Amit Kapila
On Sun, Apr 9, 2023 at 11:33 AM Gregory Stark (as CFM) wrote: > > On Fri, 7 Apr 2023 at 01:28, Amit Kapila wrote: > > > > On Wed, Apr 5, 2023 at 5:58 AM Peter Smith wrote: > > > > > PSA patches to add those tab completions. > > > > LGTM, so pushed. > > I moved this to the next CF but actually I

Re: Direct I/O

2023-04-09 Thread Andres Freund
Hi, On 2023-04-10 00:17:12 +1200, Thomas Munro wrote: > I think there are two separate bad phenomena. > > 1. A concurrent modification of the user space buffer while writing > breaks the checksum so you can't read the data back in, as . I can > reproduce that with a stand-alone program,

RE: Fix the description of GUC "max_locks_per_transaction" and "max_pred_locks_per_transaction" in guc_table.c

2023-04-09 Thread Wei Wang (Fujitsu)
On Sat, Apr 8, 2023 at 1:32 AM Tom Lane wrote: > "wangw.f...@fujitsu.com" writes: > > On Tues, Apr 4, 2023 at 23:48 PM Tom Lane wrote: > >> I like the "per eligible process" wording, at least for guc_tables.c; > >> or maybe it could be "per server process"? That would be more > >> accurate and

RE: Partial aggregates pushdown

2023-04-09 Thread fujii.y...@df.mitsubishielectric.co.jp
Hi Mr.Momjian, Mr.Lane, Mr.Freund. Thank you for advices. > From: Bruce Momjian > > > > 2. Automation of creating definition of partialaggfuncs In > > > > development of v17, I manually create definition of > > > > partialaggfuncs for avg, min, max, sum, > > > count. > > > > I am concerned that

Re: Commitfest 2023-03 starting tomorrow!

2023-04-09 Thread Tom Lane
Greg Stark writes: > Not added: > * Fix improper qual pushdown after applying outer join identity 3 I already made an open item for that one. regards, tom lane

DecodeInterval fixes

2023-04-09 Thread Joseph Koshakow
Hi all, This patch does three things in the DecodeInterval function: 1) Removes dead code for handling unit type RESERVE. There used to be a unit called "invalid" that was of type RESERVE. At some point that unit was removed and there were no more units of type RESERVE. Therefore, the code for

Re: Commitfest 2023-03 starting tomorrow!

2023-04-09 Thread Greg Stark
So here's the list of CF entries that I thought *might* still get committed either because they're an Open Issue or they're one of those other categories. I had intended to go through and add them all to the Open Issues but it turns out I only feel confident in a couple of them qualifying for

Re: Show various offset arrays for heap WAL records

2023-04-09 Thread Peter Geoghegan
On Fri, Apr 7, 2023 at 4:46 PM Peter Geoghegan wrote: > Pushed that one too. I noticed that the nbtree VACUUM and DELETE record types have their update/xl_btree_update arrays output incorrectly. We cannot use the generic array_desc() approach with xl_btree_update elements, because they're

Re: Commitfest 2023-03 starting tomorrow!

2023-04-09 Thread Michael Paquier
On Sat, Apr 08, 2023 at 09:50:49PM -0400, Tom Lane wrote: > I think that's largely independent. We don't look back at closed-out CFs > as a kind of TODO list; anything that's left behind there is basically > never going to be seen again, until the author makes a new CF entry. > > Anything that's

Re: Direct I/O

2023-04-09 Thread Thomas Munro
On Mon, Apr 10, 2023 at 8:43 AM Tom Lane wrote: > Boy, it's hard to look at that trace and not call it a filesystem bug. Agreed. > Given the apparent dependency on COW, I wonder if this has something > to do with getting confused about which copy is current? Yeah, I suppose it would require

Re: Commitfest 2023-03 starting tomorrow!

2023-04-09 Thread Michael Paquier
On Fri, Apr 07, 2023 at 10:40:01PM -0400, Kirk Wolak wrote: > I got no response to my point that the backquote solution is cumbersome > because I have to use* psql in both windows* > *and in linux environments* (realizing I am the odd duck in this group). > But my fall back was a common script

Re: Direct I/O

2023-04-09 Thread Noah Misch
On Sun, Apr 09, 2023 at 02:45:16PM -0700, Andres Freund wrote: > On 2023-04-08 21:29:54 -0700, Noah Misch wrote: > > On Sat, Apr 08, 2023 at 11:08:16AM -0700, Andres Freund wrote: > > > On 2023-04-07 23:04:08 -0700, Andres Freund wrote: > > > > If you look at log_newpage_range(), it's not

Re: Parallel Full Hash Join

2023-04-09 Thread Michael Paquier
On Sat, Apr 08, 2023 at 02:19:54PM -0400, Melanie Plageman wrote: > Another worker attached to the batch barrier, saw that it was in > PHJ_BATCH_SCAN, marked it done and detached. This is fine. > BarrierArriveAndDetachExceptLast() is meant to ensure no one waits > (deadlock hazard) and that at

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2023-04-09 Thread Michael Paquier
On Sat, Apr 08, 2023 at 04:24:35PM +0200, Matthias van de Meent wrote: > Thanks a lot! I'll post the separation of record construction and > write-out to xlog in a future thread for 17. Thanks! Creating a new thread makes sense. > One remaining question: Considering that the changes and checks

Re: Add index scan progress to pg_stat_progress_vacuum

2023-04-09 Thread Michael Paquier
On Fri, Apr 07, 2023 at 12:01:17PM -0700, Andres Freund wrote: > Why would it mean that? Parallel workers are updated together with the leader, > so there's no compatibility issue? My point is that the callback system would still need to be maintained in a stable branch, and, while useful, it

Re: Add index scan progress to pg_stat_progress_vacuum

2023-04-09 Thread Michael Paquier
On Fri, Apr 07, 2023 at 07:27:17PM +, Imseih (AWS), Sami wrote: > If called by a worker, it will send a 'P' message to the front end > passing both the progress index, i.e. PROGRESS_VACUUM_INDEXES_PROCESSED > And the value to increment by, i.e. 1 for index vacuum progress. > > With that, the

Re: Direct I/O

2023-04-09 Thread Andres Freund
Hi, On 2023-04-08 21:29:54 -0700, Noah Misch wrote: > On Sat, Apr 08, 2023 at 11:08:16AM -0700, Andres Freund wrote: > > On 2023-04-07 23:04:08 -0700, Andres Freund wrote: > > > There were some failures in CI (e.g. [1] (and perhaps also bf, didn't yet > > > check), about "no unpinned buffers

Re: differential code coverage

2023-04-09 Thread Andres Freund
Hi, On 2023-04-04 09:03:45 -0700, Andres Freund wrote: > For quite a while I'd been wishing to see *differential* code coverage, to see > what changed in code coverage between two major releases. Unfortunately lcov > didn't provide that. A few months ago a PR for it has been merged into lcov >

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-04-09 Thread Sandro Santilli
On Mon, Apr 03, 2023 at 09:26:25AM +0700, Yurii Rashkovskii wrote: > I want to chime in on the issue of lower-number releases that are released > after higher-number releases. The way I see this particular problem is that > we always put upgrade SQL files in release "packages," and they obviously

Re: Direct I/O

2023-04-09 Thread Tom Lane
Thomas Munro writes: > we have a page at offset 638976, and we can find all system calls that > touched that offset: > [pid 26031] 23:26:48.521123 pwritev(50, > [{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > iov_len=8192}], 1, 638976) = 8192 > [pid 26040]

Re: cataloguing NOT NULL constraints

2023-04-09 Thread Tom Lane
Alvaro Herrera writes: >> I'm inclined to think that this idea of suppressing the implied >> NOT NULL from PRIMARY KEY is a nonstarter and we should just >> go ahead and make such a constraint. Another idea could be for >> pg_dump to emit the NOT NULL, load data, do the ALTER ADD PRIMARY >> KEY,

Re: cataloguing NOT NULL constraints

2023-04-09 Thread Alvaro Herrera
On 2023-Apr-09, Tom Lane wrote: > In the new dispensation, pg_dump omits the NOT NULL clauses. > Great, you say, that makes the output more like what the user wrote. > I'm not so sure. This means that the ALTER TABLE will be compelled > to perform a full-table scan to verify that there are no

Re: Doc: Rework contrib appendix -- informative titles, tweaked sentences

2023-04-09 Thread Alvaro Herrera
On 2023-Apr-09, Noah Misch wrote: > On Thu, Mar 30, 2023 at 01:27:05AM -0500, Karl O. Pinc wrote: > > Your point seems valid but this is above my station. > > I have no idea as to how to best resolve this, or even how to make the > > resolution happen now that the change has been committed. > >

Re: Doc: Rework contrib appendix -- informative titles, tweaked sentences

2023-04-09 Thread Noah Misch
On Thu, Mar 30, 2023 at 01:27:05AM -0500, Karl O. Pinc wrote: > On Wed, 29 Mar 2023 21:32:05 -0700 > Noah Misch wrote: > > > On Sun, Jan 22, 2023 at 02:42:46PM -0600, Karl O. Pinc wrote: > > > v10-0001-List-trusted-and-obsolete-extensions.patch > > > > > + > > > + These modules and

Re: Direct I/O

2023-04-09 Thread Andrew Dunstan
On 2023-04-09 Su 09:14, Andrew Dunstan wrote: On 2023-04-09 Su 08:39, Thomas Munro wrote: On Sun, Apr 9, 2023 at 11:25 PM Andrew Dunstan wrote: Didn't seem to make any difference. Thanks for testing. I think it's COW (and I think that implies also checksums?) that needs to be turned off,

Re: cataloguing NOT NULL constraints

2023-04-09 Thread Tom Lane
Andres Freund writes: > I think > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo=2023-04-07%2021%3A16%3A04 > might point out a problem with the pg_dump or pg_upgrade backward compat > paths: Yeah, this patch has broken every single upgrade-from-back-branch test. I think there's

Re: Direct I/O

2023-04-09 Thread Andrew Dunstan
On 2023-04-09 Su 08:39, Thomas Munro wrote: On Sun, Apr 9, 2023 at 11:25 PM Andrew Dunstan wrote: Didn't seem to make any difference. Thanks for testing. I think it's COW (and I think that implies also checksums?) that needs to be turned off, at least based on experiments here. Googling

Re: Direct I/O

2023-04-09 Thread Thomas Munro
On Sun, Apr 9, 2023 at 11:25 PM Andrew Dunstan wrote: > Didn't seem to make any difference. Thanks for testing. I think it's COW (and I think that implies also checksums?) that needs to be turned off, at least based on experiments here.

Re: Direct I/O

2023-04-09 Thread Thomas Munro
On Sun, Apr 9, 2023 at 4:52 PM Thomas Munro wrote: > Here, btrfs seems to be taking a different path that I can't quite > make out... I see no warning/error about a checksum failure like [1], > and we apparently managed to read something other than a mix of the > old and new page contents

Re: Direct I/O

2023-04-09 Thread Andrew Dunstan
On 2023-04-08 Sa 18:50, Thomas Munro wrote: On Sun, Apr 9, 2023 at 10:17 AM Andrew Dunstan wrote: I can run the test in isolation, and it's get an error reliably. Random idea: it looks like you have compression enabled. What if you turn it off in the directory where the test runs?

Re: CREATE SUBSCRIPTION -- add missing tab-completes

2023-04-09 Thread Gregory Stark (as CFM)
On Fri, 7 Apr 2023 at 01:28, Amit Kapila wrote: > > On Wed, Apr 5, 2023 at 5:58 AM Peter Smith wrote: > > > PSA patches to add those tab completions. > > LGTM, so pushed. I moved this to the next CF but actually I just noticed the thread starts with the original patch being pushed. Maybe we