Re: problems with "Shared Memory and Semaphores" section of docs

2024-05-17 Thread Nathan Bossart
ry_size_in_huge_pages will not be. shared_memory_size_in_huge_pages is computed at runtime and can be viewed with "postgres -C" before actually trying to start the server [0]. [0] https://www.postgresql.org/docs/devel/kernel-resources.html#LINUX-HUGE-PAGES -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: problems with "Shared Memory and Semaphores" section of docs

2024-05-17 Thread Nathan Bossart
On Fri, May 17, 2024 at 12:48:37PM -0500, Nathan Bossart wrote: > On Fri, May 17, 2024 at 01:09:55PM -0400, Tom Lane wrote: >> Nathan Bossart writes: >>> At a bare minimum, we should probably fix the obvious problems, but I >>> wonder if we could simplify this sec

Re: problems with "Shared Memory and Semaphores" section of docs

2024-05-17 Thread Nathan Bossart
On Fri, May 17, 2024 at 01:09:55PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> [ many, many problems in documented formulas ] > >> At a bare minimum, we should probably fix the obvious problems, but I >> wonder if we could simplify this section a bit, too. &g

problems with "Shared Memory and Semaphores" section of docs

2024-05-17 Thread Nathan Bossart
(moving to a new thread) On Thu, May 16, 2024 at 09:16:46PM -0500, Nathan Bossart wrote: > On Thu, May 16, 2024 at 04:37:10PM +, Imseih (AWS), Sami wrote: >> Also, Not sure if I am mistaken here, but the "+ 5" in the existing docs >> seems wrong. >> >>

Re: improve performance of pg_dump --binary-upgrade

2024-05-17 Thread Nathan Bossart
rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From ded5e61ff631c2d02835fdba941068dcd86741ce Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 18 Apr 2024 15:15:19 -0500 Subject: [PATCH v5 1/2] Remove is_index parameter from binary_upgrade_set_pg_class_o

Re: allow changing autovacuum_max_workers without restarting

2024-05-16 Thread Nathan Bossart
, the WAL summarizer was added. On top of this, IIUC you actually need even more semaphores if your system doesn't support atomics, and from a quick skim this doesn't seem to be covered in this documentation. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-05-16 Thread Nathan Bossart
Here is a rebased version of 0002, which I intend to commit once v18 development begins. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From e9cba5e4303c7fa5ad2d7d5deb23fe0b1c740b09 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 7 May 2024 14:35:34 -0500 Subj

Re: optimizing pg_upgrade's once-in-each-database steps

2024-05-16 Thread Nathan Bossart
many queries? Most of those seem like just checks. This was the idea behind 347758b. It may be possible to do more along these lines. IMO parallelizing will still be useful even if we do combine more of the steps. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

optimizing pg_upgrade's once-in-each-database steps

2024-05-16 Thread Nathan Bossart
d probably create some generic tooling that each relevant step would provide a set of callback functions. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 05a9903295cb3b57ca9144217e89f0aac27277b5 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 15 May 2024 12:07:10

Re: race condition when writing pg_control

2024-05-16 Thread Nathan Bossart
n track down a copy of the control file with the bad checksum. Other than searching for any new code that isn't doing the appropriate locking, maybe we could search the buildfarm for any other occurrences. I also seem some threads concerning whether the way we are reading/writing the control fil

Re: recovery modules

2024-05-15 Thread Nathan Bossart
rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 046d6e4b13d3a6b15df1245f3154969f7553594d Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 15 Feb 2023 14:28:53 -0800 Subject: [PATCH v22 1/5] introduce routine for checking mutually exclusive string G

Re: Why does pgindent's README say to download typedefs.list from the buildfarm?

2024-05-15 Thread Nathan Bossart
On Wed, May 15, 2024 at 04:52:19PM -0400, Robert Haas wrote: > On Wed, May 15, 2024 at 4:50 PM Tom Lane wrote: >> At this point my OCD got the better of me and I did a little >> additional wordsmithing. How about the attached? > > No objections here. +1 -- Nathan Bossar

Re: Why does pgindent's README say to download typedefs.list from the buildfarm?

2024-05-15 Thread Nathan Bossart
On Wed, May 15, 2024 at 04:07:18PM -0400, Robert Haas wrote: > On Wed, May 15, 2024 at 3:30 PM Nathan Bossart > wrote: >> This is much cleaner, thanks. The only thing that stands out to me is that >> the "once per release cycle" section should probably say

Re: More performance improvements for pg_dump in binary upgrade mode

2024-05-15 Thread Nathan Bossart
ost soon, too. v18 should have a lot of good stuff for pg_upgrade... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: An improved README experience for PostgreSQL

2024-05-15 Thread Nathan Bossart
On Wed, May 15, 2024 at 07:23:19AM +0200, Peter Eisentraut wrote: > I think for CONTRIBUTING.md, a better link would be > <https://www.postgresql.org/developer/>. WFM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 74de0e89bea2802bf699397837ebf77252a0e06b Mon

Re: Why does pgindent's README say to download typedefs.list from the buildfarm?

2024-05-15 Thread Nathan Bossart
an indent run after downloading the typedef file. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: An improved README experience for PostgreSQL

2024-05-14 Thread Nathan Bossart
e out of the way but yet >> updatable by any committer. > > +1 for .github/, that was my first reaction as well after reading the > link Peter posted. Here's an updated patch that uses .github/. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From d6e27e9acf65bf77c54e229

Re: An improved README experience for PostgreSQL

2024-05-14 Thread Nathan Bossart
On Tue, May 14, 2024 at 10:05:01AM +0200, Peter Eisentraut wrote: > On 13.05.24 17:26, Nathan Bossart wrote: >> On Sun, May 12, 2024 at 05:17:42PM +0200, Peter Eisentraut wrote: >> > I don't know, I find these files kind of "yelling". It's fine to have a >> &g

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-05-13 Thread Nathan Bossart
Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: An improved README experience for PostgreSQL

2024-05-13 Thread Nathan Bossart
ion centralized. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: An improved README experience for PostgreSQL

2024-05-13 Thread Nathan Bossart
efault-community-health-file This was the intent of my patch. There might be a few others that we could use, but I figured we could start with the low-hanging fruit that would have the most impact on the GitHub experience. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-05-10 Thread Nathan Bossart
On Wed, May 08, 2024 at 11:01:01AM +0900, Michael Paquier wrote: > On Tue, May 07, 2024 at 02:39:42PM -0500, Nathan Bossart wrote: >> Okay, phew. We can still do something like v3-0002 for v18. I'll give >> Michael a chance to comment on 0001 before committing/back-patch

Re: add --no-sync to pg_upgrade's calls to pg_dump and pg_dumpall

2024-05-09 Thread Nathan Bossart
ide significant improvements. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: ALTER EXTENSION SET SCHEMA versus dependent types

2024-05-08 Thread Nathan Bossart
On Wed, May 08, 2024 at 07:57:55PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Agreed. Another option could be to just annotate the arguments with the >> parameter names. > > At the call sites you mean? Sure, I can do that. Yes. -- Nathan Bossart Amazo

Re: ALTER EXTENSION SET SCHEMA versus dependent types

2024-05-08 Thread Nathan Bossart
On Wed, May 08, 2024 at 07:42:18PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Looks reasonable to me. The added test coverage seems particularly >> valuable. If I really wanted to nitpick, I might complain about the three >> consecutive Boolean parameters for Alter

Re: ALTER EXTENSION SET SCHEMA versus dependent types

2024-05-08 Thread Nathan Bossart
yOid, nspOid, true, false, true, + objsMoved); difficult to interpret. But that's not necessarily the fault of this patch and probably needn't block it. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add --no-sync to pg_upgrade's calls to pg_dump and pg_dumpall

2024-05-08 Thread Nathan Bossart
On Wed, May 08, 2024 at 10:09:46AM +0200, Peter Eisentraut wrote: > On 03.05.24 19:13, Nathan Bossart wrote: >> This is likely small potatoes compared to some of the other >> pg_upgrade-related improvements I've proposed [0] [1] or plan to propose, >> but this is easy enough

Re: New GUC autovacuum_max_threshold ?

2024-05-07 Thread Nathan Bossart
folks may opt to set this value super low, I think that's more likely to lead to some interesting secondary effects. If the default is high, hopefully these secondary effects will be minimized or avoided. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-05-07 Thread Nathan Bossart
On Tue, May 07, 2024 at 03:02:01PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Tue, May 07, 2024 at 01:44:16PM -0400, Tom Lane wrote: >>> +1 to include that, as it offers a defense if someone invokes this >>> function directly. In HEAD we could then rip o

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-05-07 Thread Nathan Bossart
On Tue, May 07, 2024 at 01:44:16PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> char relpersist = seqrel->rd_rel->relpersistence; > >> if (relpersist == RELPERSISTENCE_PERMANENT || >> (relpersist == RELPERSISTENCE_UNLOG

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-05-07 Thread Nathan Bossart
On Sat, May 04, 2024 at 06:45:32PM +0900, Michael Paquier wrote: > On Fri, May 03, 2024 at 05:22:06PM -0400, Tom Lane wrote: >> Nathan Bossart writes: >>> IIUC this would cause other sessions' temporary sequences to appear in the >>> view. Is that desirable? >>

Re: allow changing autovacuum_max_workers without restarting

2024-05-07 Thread Nathan Bossart
ed, but that seems >> unlikely to be a problem for the systems that will run Postgres v18. > > I agree with this. Here's what this might look like. I chose an upper limit of 1024, which seems like it "ought to be enough for anybody," at least for now. -- Nathan Bossart Ama

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-05-03 Thread Nathan Bossart
I'd track separately from the back-patchable fix proposed in this thread. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

add --no-sync to pg_upgrade's calls to pg_dump and pg_dumpall

2024-05-03 Thread Nathan Bossart
%40nathanxps13 [1] https://postgr.es/m/20240503025140.GA1227404%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 998613046fb5ff5c6618964c821109b573bfa89b Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Fri, 3 May 2024 10:35:21 -0500 Subject: [PATCH v1 1/1]

improve performance of pg_dump with many sequences

2024-05-02 Thread Nathan Bossart
. Furthermore, the query in 0003 is a bit goofy because it needs to dance around a bug reported elsewhere [1]. [0] https://postgr.es/m/20240418041712.GA3441570%40nathanxps13 [1] https://postgr.es/m/20240501005730.GA594666%40nathanxps13 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: allow changing autovacuum_max_workers without restarting

2024-05-02 Thread Nathan Bossart
ay, 256 slots might require a few additional kilobytes of shared memory, most of which will go unused, but that seems unlikely to be a problem for the systems that will run Postgres v18. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-04-30 Thread Nathan Bossart
On Tue, Apr 30, 2024 at 08:13:17PM -0500, Nathan Bossart wrote: > Good point. I'll work on a patch along these lines, then. This ended up being easier than I expected. While unlogged sequences are only supported on v15 and above, temporary sequences have been around since v7.2,

Re: pg_sequence_last_value() for unlogged sequences on standbys

2024-04-30 Thread Nathan Bossart
On Tue, Apr 30, 2024 at 09:06:04PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> If you create an unlogged sequence on a primary, pg_sequence_last_value() >> for that sequence on a standby will error like so: >> postgres=# select pg_sequence_last_value('test'::reg

pg_sequence_last_value() for unlogged sequences on standbys

2024-04-30 Thread Nathan Bossart
R to align with what you see when you try to access unlogged relations on a standby (i.e., "cannot access temporary or unlogged relations during recovery"). Thoughts? [0] https://postgr.es/m/CAAaqYe8JL8Et2DoO0RRjGaMvy7-C6eDH-2wHXK-gp3dOssvBkQ%40mail.gmail.com -- Nathan Bossart Amazon

Re: New committers: Melanie Plageman, Richard Guo

2024-04-26 Thread Nathan Bossart
success and few reverts! Congratulations to both! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: New GUC autovacuum_max_threshold ?

2024-04-25 Thread Nathan Bossart
um_analyze_threshold. Is that okay? Or do we need to introduce a "limit" GUC for each? I guess the question is whether we anticipate any need to have different values for these limits, which might be unlikely. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: New GUC autovacuum_max_threshold ?

2024-04-25 Thread Nathan Bossart
On Thu, Apr 25, 2024 at 09:13:07AM +0200, Frédéric Yhuel wrote: > Le 24/04/2024 à 21:57, Nathan Bossart a écrit : >> Yeah, I'm having trouble following the proposed mechanics for this new GUC, >> and it's difficult to understand how users would choose a value. If we >> just w

Re: Extend ALTER DEFAULT PRIVILEGES for large objects

2024-04-24 Thread Nathan Bossart
rested in resolving any remaining reasons folks are using large objects over TOAST. I see a couple of reasons listed in the docs [0] that might be worth examining. [0] https://www.postgresql.org/docs/devel/lo-intro.html -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: A varint implementation for PG?

2024-04-24 Thread Nathan Bossart
[0], which I just wanted to link here for posterity. If there are no plans, I might give it a try, but otherwise I'm happy to help review. [0] https://postgr.es/m/20221004234952.anrguppx5owewb6n%40awork3.anarazel.de -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Partitioned tables and [un]loggedness

2024-04-24 Thread Nathan Bossart
ses UNLOGGED. Could we add LOGGED for CREATE TABLE? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: New GUC autovacuum_max_threshold ?

2024-04-24 Thread Nathan Bossart
, I'm very sorry for the useless noise. > > I rooted around in the hackers archive and couldn't find any threads > on this specific proposal. I copied some other hackers I knew of who > have worked on this problem and thought about it in the past, in case > they know of some exist

Re: Why does pgindent's README say to download typedefs.list from the buildfarm?

2024-04-23 Thread Nathan Bossart
On Mon, Apr 22, 2024 at 03:20:10PM -0500, Nathan Bossart wrote: > On Mon, Apr 22, 2024 at 04:08:08PM -0400, Tom Lane wrote: >> The problem with the README is that it describes that process, >> rather than the now-typical workflow of incrementally keeping >> the tree indent

Re: Popcount optimization using AVX512

2024-04-23 Thread Nathan Bossart
On Thu, Apr 18, 2024 at 05:13:58PM -0500, Nathan Bossart wrote: > Makes sense, thanks. I'm planning to commit this fix sometime early next > week. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Why does pgindent's README say to download typedefs.list from the buildfarm?

2024-04-22 Thread Nathan Bossart
to do the full-monty process, but you're right that > the README needs to explain the incremental method as being > the one most developers would usually use. > > Want to write some text? Yup, I'll put something together. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Why does pgindent's README say to download typedefs.list from the buildfarm?

2024-04-22 Thread Nathan Bossart
added until 2010 (commit 1604057), so maybe this is just leftover from back then. Could we remove this note now? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: improve performance of pg_dump --binary-upgrade

2024-04-22 Thread Nathan Bossart
I noticed that there are some existing examples of this sort of thing in pg_dump (e.g., commit d5e8930), so I adjusted the patch to match the surrounding style a bit better. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 80dd3233730422298ffe3b4523a7c58d7e9b55b9 Mon Sep 17

Re: allow changing autovacuum_max_workers without restarting

2024-04-19 Thread Nathan Bossart
urdened > with solving the general problem here. But I do think the general > problem is worth some discussion. I certainly don't want to hold up $SUBJECT for a larger rewrite of autovacuum scheduling, but I also don't want to shy away from a larger rewrite if it's an idea whose time has come

Re: allow changing autovacuum_max_workers without restarting

2024-04-19 Thread Nathan Bossart
n in the current state, users should think > about > these settings. However, it seems even important if this value is to be > dynamically adjusted. I don't necessarily disagree that it might be worth mentioning these parameters, but I would argue that this should be proposed in a separate

Re: Popcount optimization using AVX512

2024-04-18 Thread Nathan Bossart
e no CPU's with avx512bw/avx512popcnt > without avx512f. Unfortunately though, avx512popcnt does not mean > avx512bw (I think the deprecated Xeon Phi processors falls in this > category) which is why we need both. Makes sense, thanks. I'm planning to commit this fix sometime early next wee

Re: Popcount optimization using AVX512

2024-04-18 Thread Nathan Bossart
ssary to also check for avx512f? At the moment, we are assuming that's supported if the other AVX-512 instructions are available. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From e04c348eb389c6aa1597ac35d57b5e7ae7075381 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Nathan Bossart
of lines > away), and a comment on the if (oids == NULL) block explaining the caching. Added. Thanks for reviewing! Unfortunately, this one will have to sit for a couple months... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 6797e4e1fc1a63f710ed0a409b1337880eb91ad4 M

Re: Popcount optimization using AVX512

2024-04-18 Thread Nathan Bossart
rect. It needs to check all the 3 bits (XMM/YMM and ZMM). The > way it is written is now is in-correct. Thanks for the feedback. I've attached an updated patch. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From d20b19804a17d9f6eab1d40de7e9fb10488ac6b0 Mon Sep 17 00:00:00 20

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Nathan Bossart
On Thu, Apr 18, 2024 at 10:23:01AM -0500, Nathan Bossart wrote: > On Thu, Apr 18, 2024 at 09:24:53AM +0200, Daniel Gustafsson wrote: >> That does indeed seem like a saner approach. Since we look up the relkind we >> can also remove the is_index parameter to binary_upgrade_se

Re: Popcount optimization using AVX512

2024-04-18 Thread Nathan Bossart
nd AVX state [0]. I don't know how likely it is that 0xe0 would succeed but 0xe6 wouldn't, but we might as well make it correct. [0] https://en.wikipedia.org/wiki/Control_register#cite_ref-23 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Nathan Bossart
On Thu, Apr 18, 2024 at 09:24:53AM +0200, Daniel Gustafsson wrote: >> On 18 Apr 2024, at 06:17, Nathan Bossart wrote: > >> The attached work-in-progress patch speeds up 'pg_dump --binary-upgrade' >> for this case. Instead of executing the query in every call to the >&g

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Nathan Bossart
obvious low-memory alternative would be to make a prepared statement, > though that does nothing to cut down on the roundtrips. > > I think this is a good trade off. Cool. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

improve performance of pg_dump --binary-upgrade

2024-04-17 Thread Nathan Bossart
/3612876.1689443232%40sss.pgh.pa.us -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 27b4a3249dd97376f13a7c99505330ab7cd78e3f Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 17 Apr 2024 22:55:27 -0500 Subject: [PATCH v1 1/1] Improve performance of pg_dump --binary-upgrade. --- src/

Re: Popcount optimization using AVX512

2024-04-17 Thread Nathan Bossart
attached a one-line patch to fix this. [0] https://github.com/pgvector/pgvector/pull/519#issuecomment-2062804463 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/src/port/pg_popcount_avx512_choose.c b/src/port/pg_popcount_avx512_choose.c index ae3fa3d306..cc3e89e096

Re: Statistics Import and Export

2024-04-17 Thread Nathan Bossart
n dumpTableData(). However, that seems to prevent the stats from getting exported in the --schema-only/--binary-upgrade scenario, which presents a problem for pg_upgrade. ISTM we'll need some extra hacks to get this to work as desired. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: An improved README experience for PostgreSQL

2024-04-16 Thread Nathan Bossart
On Wed, Feb 28, 2024 at 02:21:49PM -0600, Nathan Bossart wrote: > I see many projects have files like SECURITY.md, CODE_OF_CONDUCT.md, and > CONTRIBUTING.md, and I think it would be relatively easy to add content to > each of those for PostgreSQL, even if they just link elsewhere. Here'

Re: SET ROLE documentation improvement

2024-04-15 Thread Nathan Bossart
On Fri, Apr 12, 2024 at 09:54:24AM +0900, Michael Paquier wrote: > On Thu, Apr 11, 2024 at 02:21:49PM -0500, Nathan Bossart wrote: >> No objections here. I'll give this a few days for others to comment. I'm >> not particularly interested in back-patching this since it's arguabl

Re: allow changing autovacuum_max_workers without restarting

2024-04-15 Thread Nathan Bossart
On Mon, Apr 15, 2024 at 11:28:33AM -0500, Nathan Bossart wrote: > On Mon, Apr 15, 2024 at 08:33:33AM -0500, Justin Pryzby wrote: >> On Wed, Apr 10, 2024 at 04:23:44PM -0500, Nathan Bossart wrote: >>> The proof-of-concept patch keeps autovacuum_max_workers as the maximum &

Re: allow changing autovacuum_max_workers without restarting

2024-04-15 Thread Nathan Bossart
On Mon, Apr 15, 2024 at 08:33:33AM -0500, Justin Pryzby wrote: > On Wed, Apr 10, 2024 at 04:23:44PM -0500, Nathan Bossart wrote: >> The proof-of-concept patch keeps autovacuum_max_workers as the maximum >> number of slots to reserve for workers, but I think we should in

Re: allow changing autovacuum_max_workers without restarting

2024-04-14 Thread Nathan Bossart
Here is a first attempt at a proper patch set based on the discussion thus far. I've split it up into several small patches for ease of review, which is probably a bit excessive. If this ever makes it to commit, they could likely be combined. -- Nathan Bossart Amazon Web Services: https

Re: allow changing autovacuum_max_workers without restarting

2024-04-13 Thread Nathan Bossart
s/m/27574.1581015893%40sss.pgh.pa.us -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: further improving roles_is_member_of()

2024-04-12 Thread Nathan Bossart
On Thu, Apr 11, 2024 at 11:16:33PM -0500, Nathan Bossart wrote: > As shown in the attached work-in-progress patch, this actually ends up > removing more code than it adds, and it seems to provide similar results to > HEAD (using the benchmark from the previous thread [0]). I inten

Re: allow changing autovacuum_max_workers without restarting

2024-04-12 Thread Nathan Bossart
;autovacuum_max_worker_slots" might be worth considering, too. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

further improving roles_is_member_of()

2024-04-11 Thread Nathan Bossart
ached work-in-progress patch, this actually ends up removing more code than it adds, and it seems to provide similar results to HEAD (using the benchmark from the previous thread [0]). I intend to test it with many more roles to see if it's better in more extreme cases. [0] https://postgr.es/m/34118

Re: allow changing autovacuum_max_workers without restarting

2024-04-11 Thread Nathan Bossart
we want to get these parameters tangled up like >> this, though... > > This will be confusing to describe and we will be reserving autovac workers > implicitly, rather than explicitly with a new GUC. Yeah, that's probably a good reason to give autovacuum its own worker pool. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: SET ROLE documentation improvement

2024-04-11 Thread Nathan Bossart
On Thu, Apr 11, 2024 at 01:38:37PM -0400, Robert Haas wrote: > On Thu, Apr 11, 2024 at 1:03 PM Nathan Bossart > wrote: >> While it's fresh on my mind, I very hastily hacked together a draft of what >> I believe is the remaining work. > > That looks fine to me. And if ot

Re: SET ROLE documentation improvement

2024-04-11 Thread Nathan Bossart
On Thu, Apr 11, 2024 at 11:48:30AM -0500, Nathan Bossart wrote: > On Thu, Apr 11, 2024 at 11:36:52AM -0400, Robert Haas wrote: >> I suggest that we close the existing CF entry as committed and >> somebody can start a new one for whatever remains. I think that will >> be less

Re: SET ROLE documentation improvement

2024-04-11 Thread Nathan Bossart
On Thu, Apr 11, 2024 at 11:36:52AM -0400, Robert Haas wrote: > I suggest that we close the existing CF entry as committed and > somebody can start a new one for whatever remains. I think that will > be less confusing. Done: https://commitfest.postgresql.org/48/4923/. -- Nathan Bossa

Re: allow changing autovacuum_max_workers without restarting

2024-04-11 Thread Nathan Bossart
On Thu, Apr 11, 2024 at 09:42:40AM -0500, Nathan Bossart wrote: > On Thu, Apr 11, 2024 at 02:24:18PM +, Imseih (AWS), Sami wrote: >> max_worker_processes defines a pool of max # of background workers allowed. >> parallel workers and extensions that spin up background work

Re: allow changing autovacuum_max_workers without restarting

2024-04-11 Thread Nathan Bossart
ern with this approach is that other background workers could use up all the slots and prevent autovacuum workers from starting, unless of course we reserve autovacuum_max_workers slots for _only_ autovacuum workers. I'm not sure if we want to get these parameters tangled up like this, though... --

Re: Allow non-superuser to cancel superuser tasks.

2024-04-11 Thread Nathan Bossart
mber(procNumber); + + if (!ret) + return B_INVALID; + + return ret->st_backendType; +} I'm not sure we really need to introduce a new function for this. I avoided using it in my example snippet above. But, maybe it'll come in handy down the road... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Allow non-superuser to cancel superuser tasks.

2024-04-11 Thread Nathan Bossart
ing entry? https://commitfest.postgresql.org/48/ -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: SET ROLE documentation improvement

2024-04-11 Thread Nathan Bossart
rivileges" note. * Note that SET ROLE and SET SESSION AUTHORIZATION are exceptions. While I think these are good changes, I don't sense any urgency here, so I'm treating this as v18 material at this point. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: recovery modules

2024-04-10 Thread Nathan Bossart
rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 5897631d5f09032565d92d5b8547baf3d24eef87 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 15 Feb 2023 14:28:53 -0800 Subject: [PATCH v21 1/5] introduce routine for checking mutually exclusive string G

allow changing autovacuum_max_workers without restarting

2024-04-10 Thread Nathan Bossart
drops below the new parameter's value. I don't think we should kill existing workers, or anything else like that. TBH I've been sitting on this idea for a while now, only because I think it has a slim chance of acceptance, but IMHO this is a simple change that could help many users. -- Nathan

Re: Allow non-superuser to cancel superuser tasks.

2024-04-10 Thread Nathan Bossart
gt;>> because autovacuum workers operate like regular backends. This name >>> can also be confused with the autovacuum launcher. >> >> Ok. What would be a good choice? Is `pg_signal_autovacuum_worker` good >> enough? > > Sounds fine to me. Perhaps others have an opinion about that? WFM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Fixup some StringInfo usages

2024-04-08 Thread Nathan Bossart
o identify where all of > these were introduced. All are new to PG17. > > Any objections? Looks reasonable to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Popcount optimization using AVX512

2024-04-07 Thread Nathan Bossart
On Sun, Apr 07, 2024 at 08:23:32PM -0500, Nathan Bossart wrote: > The Intel documentation for _mm256_undefined_si256() [0] > indicates that it is intended to return "undefined elements," so it seems > like the use of an uninitialized variable might be intentional. See also http

Re: Popcount optimization using AVX512

2024-04-07 Thread Nathan Bossart
Anyway, we can certainly just dismiss this warning if it > doesn't correspond to any real problem in our code. > But I thought I'd raise the question. That's probably the right thing to do, unless there's some action we can take to suppress this warning. [0] https://www.intel.com/content/www

Re: Popcount optimization using AVX512

2024-04-06 Thread Nathan Bossart
On Sat, Apr 06, 2024 at 02:41:01PM -0500, Nathan Bossart wrote: > Here is what I have staged for commit, which I intend to do shortly. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Popcount optimization using AVX512

2024-04-06 Thread Nathan Bossart
On Sat, Apr 06, 2024 at 02:51:39PM +1300, David Rowley wrote: > On Sat, 6 Apr 2024 at 14:17, Nathan Bossart wrote: >> On Sat, Apr 06, 2024 at 12:08:14PM +1300, David Rowley wrote: >> > Won't Valgrind complain about this? >> > >> > +pg_popcount_avx512(const c

Re: Popcount optimization using AVX512

2024-04-05 Thread Nathan Bossart
nted to avoid because of the weird function overhead juggling) or find another way to do a partial load into an __m512i. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Allow non-superuser to cancel superuser tasks.

2024-04-05 Thread Nathan Bossart
ot following what you mean by this. Are you suggesting that we should keep the existing superuser message for the autovacuum workers? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: WIP Incremental JSON Parser

2024-04-05 Thread Nathan Bossart
On Fri, Apr 05, 2024 at 10:15:45AM -0400, Andrew Dunstan wrote: > On 2024-04-04 Th 15:54, Nathan Bossart wrote: >> On Thu, Apr 04, 2024 at 03:31:12PM -0400, Andrew Dunstan wrote: >> > Does the attached patch fix it for you? >> It clears up 2 of the 3 warnings for me:

Re: Popcount optimization using AVX512

2024-04-05 Thread Nathan Bossart
On Fri, Apr 05, 2024 at 07:58:44AM -0500, Nathan Bossart wrote: > On Fri, Apr 05, 2024 at 10:33:27AM +0300, Ants Aasma wrote: >> The main issue I saw was that clang was able to peel off the first >> iteration of the loop and then eliminate the mask assignment and >>

Re: Popcount optimization using AVX512

2024-04-05 Thread Nathan Bossart
would make sense to > have the extra code if it gives a noticeable benefit for large cases. Yeah, I did see this, but I also wasn't sure if it was worth further complicating the code. I can test with and without your fix and see if it makes any difference in the benchmarks. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Allow non-superuser to cancel superuser tasks.

2024-04-05 Thread Nathan Bossart
sers distinguishing autovacuum workers from other superuser backends, _but_ if roles with pg_signal_autovacuum can't even figure out the PIDs for the autovacuum workers, then this feature seems kind-of useless. Perhaps we should allow roles with privileges of pg_signal_autovacuum to see the autovacuum workers in pg_stat_activity. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Popcount optimization using AVX512

2024-04-04 Thread Nathan Bossart
Here is an updated patch set. IMHO this is in decent shape and is approaching committable. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From df59d3e78604e4530f5096bafc08ac94e13d82d2 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 27 Mar 2024 16:39:24 -0500 Subj

Re: WIP Incremental JSON Parser

2024-04-04 Thread Nathan Bossart
may be used uninitialized in this function [-Werror=maybe-uninitialized] 2018 | if (lex->incremental && !lex->inc_state->is_last_chunk && | -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Allow non-superuser to cancel superuser tasks.

2024-04-04 Thread Nathan Bossart
; else if (!has_privs_of_role(GetUserId(), proc->roleId) && >!has_privs_of_role(GetUserId(), > ROLE_PG_SIGNAL_BACKEND)) > { > return SIGNAL_BACKEND_NOPERMISSION; > } There's no need for the explicit superuser() check in the pg_signal_autovacuum section. That's built into has_privs_of_role() already. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: WIP Incremental JSON Parser

2024-04-04 Thread Nathan Bossart
p; (total_len == dummy_lex.input_length); | ~^~~~~~~~~~~~ -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

  1   2   3   4   5   6   7   8   9   10   >