Re: --sync-method isn't documented to take an argument

2023-10-04 Thread Nathan Bossart
ing too much otherwise. I matched the adjacent options as you suggested. Perhaps unsurprisingly, the inclusion of class="parameter" is not the only inconsistency. I also found that pg_upgrade adds the before the argument name! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: --sync-method isn't documented to take an argument

2023-10-04 Thread Nathan Bossart
On Wed, Oct 04, 2023 at 11:51:32AM -0400, Robert Haas wrote: > On Wed, Oct 4, 2023 at 11:27 AM Nathan Bossart > wrote: >> Here's a patch. I didn't address the class="parameter" stuff at all. I >> figured it would be best to handle that separately. > > I g

Re: --sync-method isn't documented to take an argument

2023-10-04 Thread Nathan Bossart
address the class="parameter" stuff at all. I figured it would be best to handle that separately. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml index 8a09c5c438..d43c91575c 100644 --- a/doc/src/sgml/r

Re: should frontend tools use syncfs() ?

2023-10-04 Thread Nathan Bossart
2. Maybe it could be put there? I will give this a try. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: --sync-method isn't documented to take an argument

2023-10-04 Thread Nathan Bossart
ke this: > > --pgdata=directory > --filenode=filenode > > etc. > > This one should be something like this: > > --sync-method=method Whoops. Thanks for pointing this out. I'll get this fixed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: stopgap fix for signal handling during restore_command

2023-10-04 Thread Nathan Bossart
On Sun, Oct 01, 2023 at 08:50:15PM +0200, Peter Eisentraut wrote: > Is this still being contemplated? What is the status of this? I'll plan on committing this in the next couple of days. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: ALTER ROLE documentation improvement

2023-09-25 Thread Nathan Bossart
s attribute." msgstr "Der Bootstrap-Benutzer muss das %s-Attribut haben." No need to update the translation files. Those are updated separately in the pgtranslation repo. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Improve the log message output of basic_archive when basic_archive.archive_directory parameter is not set

2023-09-25 Thread Nathan Bossart
th v16? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: SET ROLE documentation improvement

2023-09-25 Thread Nathan Bossart
On Fri, Sep 15, 2023 at 02:36:16PM -0700, Yurii Rashkovskii wrote: > On Fri, Sep 15, 2023 at 1:47 PM Nathan Bossart > wrote: >> I think another issue is that the aforementioned note doesn't mention the >> new SET option added in 3d14e17. > > How do you think we sho

Re: Add 'worker_type' to pg_stat_subscription

2023-09-25 Thread Nathan Bossart
On Thu, Sep 21, 2023 at 04:01:20PM +0530, Amit Kapila wrote: > The changes looks good to me, though I haven't tested it. But feel > free to commit if you are fine with this patch. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: bug fix and documentation improvement about vacuumdb

2023-09-23 Thread Nathan Bossart
I propose we > commit > to fix this, with a backpatch to v16 where it was introduced. LGTM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add 'worker_type' to pg_stat_subscription

2023-09-20 Thread Nathan Bossart
posted. Right, I'll take care of it before committing. I'm trying to make sure I don't forget. :) -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Improvements in pg_dump/pg_restore toc format and performances

2023-09-20 Thread Nathan Bossart
ze); } deps[depIdx] = depId; Also, can we make depId more locally scoped? I have yet to look into 0004 still. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add 'worker_type' to pg_stat_subscription

2023-09-20 Thread Nathan Bossart
t; I checked pg_stat_activity.backend_type, there is nothing called main > or leader backend even when the backend is involved in parallel query. Okay. Here is v9 of the patch with this change. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From b8268a4c95ff217742bc2e127f74f67c9a417233 M

Re: should frontend tools use syncfs() ?

2023-09-20 Thread Nathan Bossart
me for a variable in xlog module. +1 I think we should also consider renaming things like SYNC_METHOD_FSYNC to WAL_SYNC_METHOD_FSYNC, and sync_method_options to wal_sync_method_options. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Inefficiency in parallel pg_restore with many tables

2023-09-19 Thread Nathan Bossart
On Mon, Sep 18, 2023 at 02:22:32PM -0700, Nathan Bossart wrote: > For now, I've committed 0001 and 0002. I intend to commit the others soon. I've committed the rest of the patches. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Inefficiency in parallel pg_restore with many tables

2023-09-18 Thread Nathan Bossart
nt. I won't have much time for this for the next few weeks, so if someone else wants to pick it up, please feel free. [0] https://postgr.es/m/20230408191007.7lysd42euafwl74f%40awork3.anarazel.de [1] https://github.com/anarazel/postgres/commits/drop-homegrown-msvc -- Nathan Bossart Amazon Web Se

Re: Inefficiency in parallel pg_restore with many tables

2023-09-18 Thread Nathan Bossart
On Mon, Sep 18, 2023 at 09:23:20PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> For now, I've committed 0001 and 0002. I intend to commit the others soon. > > bowerbird is unhappy with this. I suppose you missed out updating > the src/tools/msvc/ scripts. (Weren't we a

Re: Improvements in pg_dump/pg_restore toc format and performances

2023-09-18 Thread Nathan Bossart
On Mon, Sep 18, 2023 at 02:52:47PM -0700, Nathan Bossart wrote: > On Thu, Jul 27, 2023 at 10:51:11AM +0200, Pierre Ducroquet wrote: >> I ended up writing several patches that shaved some time for pg_restore -l, >> and reduced the toc.dat size. > > I've only just

Re: Improvements in pg_dump/pg_restore toc format and performances

2023-09-18 Thread Nathan Bossart
toul) 202M22.8s 0.94s > #4 (string list) 181M 23.1s 0.87s At a glance, the size improvements in 0004 look the most interesting to me. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Inefficiency in parallel pg_restore with many tables

2023-09-18 Thread Nathan Bossart
For now, I've committed 0001 and 0002. I intend to commit the others soon. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add 'worker_type' to pg_stat_subscription

2023-09-18 Thread Nathan Bossart
ed" that perhaps should > say "logical replication leader apply worker for subscription \"%s\" > has started". Such changes don't belong in this stats patch. +1 [0] https://postgr.es/m/attachment/150345/v7-0001-Add-worker-type-to-pg_stat_subscription.patch -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add 'worker_type' to pg_stat_subscription

2023-09-16 Thread Nathan Bossart
sibility for confusion with the documentation's use of "leader apply worker", but I haven't touched that for now. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 42c7b1910af29a8543ed65b2150e5eedae34a594 Mon Sep 17 00:00:00 2001 From: Nathan Bossa

Re: ALTER ROLE documentation improvement

2023-09-15 Thread Nathan Bossart
quot;. Perhaps we should standardize on one name. [0] https://www.postgresql.org/docs/devel/glossary.html -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: SET ROLE documentation improvement

2023-09-15 Thread Nathan Bossart
n user is a superuser, any role can be selected.) But I don't see a big issue with clarifying things further as you propose. I think another issue is that the aforementioned note doesn't mention the new SET option added in 3d14e17. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add 'worker_type' to pg_stat_subscription

2023-09-15 Thread Nathan Bossart
On Thu, Sep 14, 2023 at 03:04:19PM -0700, Nathan Bossart wrote: > The only reason I didn't apply this already is because IMHO we should > adjust the worker types and the documentation for the view to be > consistent. For example, the docs say "leader apply worker" but the vie

Re: bug fix and documentation improvement about vacuumdb

2023-09-15 Thread Nathan Bossart
ely readable. These sorts of things bug me, too (see 2af3336). -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Improve the log message output of basic_archive when basic_archive.archive_directory parameter is not set

2023-09-15 Thread Nathan Bossart
rchive-modules.sgml, so >> that needs to be updated as well. This also means you can't backpatch >> this change, or you risk breaking external software that implements this >> interface. > > Absolutely, this is master only for v17. +1 -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add 'worker_type' to pg_stat_subscription

2023-09-14 Thread Nathan Bossart
On Wed, Sep 13, 2023 at 09:59:04AM -0700, Nathan Bossart wrote: > On Wed, Sep 13, 2023 at 05:06:28PM +0300, Maxim Orlov wrote: >> So, should we mark this thread as RfC? > > I've done so. Barring additional feedback, I intend to commit this in the > next few days. I did

Re: bug fix and documentation improvement about vacuumdb

2023-09-14 Thread Nathan Bossart
pecified schema. > > Maybe, but the point of vacuumdb is to analyze a database so I'm not sure who > would expect anything else than vacuuming everything but the excluded schema > when specifying -N. What else could "vacuumdb -N foo" be interpreted to do > that can b

Re: Inefficiency in parallel pg_restore with many tables

2023-09-13 Thread Nathan Bossart
On Wed, Sep 13, 2023 at 08:01:39PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Upon closer inspection, I found a rather nasty problem. The qsort >> comparator expects a TocEntry **, but the binaryheap comparator expects a >> TocEntry *, and we simply pass

Re: Inefficiency in parallel pg_restore with many tables

2023-09-13 Thread Nathan Bossart
On Wed, Sep 13, 2023 at 11:34:50AM -0700, Nathan Bossart wrote: > On Sun, Sep 10, 2023 at 12:35:10PM -0400, Tom Lane wrote: >> Other than those nitpicks, I like v6. I'll mark this RfC. > > Great. I've posted a v8 with your comments addressed in order to get one > more round

Re: Inefficiency in parallel pg_restore with many tables

2023-09-13 Thread Nathan Bossart
erwise some compilers might complain about the result possibly > not fitting in "j". Sure. IMO it's a tad more readable, too. > Other than those nitpicks, I like v6. I'll mark this RfC. Great. I've posted a v8 with your comments addressed in order to get one more round of cfbot cover

Re: Add 'worker_type' to pg_stat_subscription

2023-09-13 Thread Nathan Bossart
On Wed, Sep 13, 2023 at 09:59:04AM -0700, Nathan Bossart wrote: > On Wed, Sep 13, 2023 at 05:06:28PM +0300, Maxim Orlov wrote: >> So, should we mark this thread as RfC? > > I've done so. Barring additional feedback, I intend to commit this in the > next few days. Note t

Re: Add 'worker_type' to pg_stat_subscription

2023-09-13 Thread Nathan Bossart
the code accordingly. If we just rely on an ERROR in the default section, we might miss it if there isn't a relevant test. > So, should we mark this thread as RfC? I've done so. Barring additional feedback, I intend to commit this in the next few days. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-09-08 Thread Nathan Bossart
On Fri, Sep 08, 2023 at 10:52:10AM -0400, Bruce Momjian wrote: > On Thu, Sep 7, 2023 at 09:21:07PM -0700, Nathan Bossart wrote: >> On Thu, Sep 07, 2023 at 07:13:44PM -0400, Bruce Momjian wrote: >> > On Thu, Sep 7, 2023 at 02:54:13PM -0700, Nathan Bossart wrote: >> >&g

Re: Adding a pg_get_owned_sequence function?

2023-09-08 Thread Nathan Bossart
until it's actually gone (and maybe not even then). There really isn't > a serious middle-ground where deprecation is helpful given our yearly > release cycle and long major version support period. Fair point. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-09-07 Thread Nathan Bossart
On Thu, Sep 07, 2023 at 07:13:44PM -0400, Bruce Momjian wrote: > On Thu, Sep 7, 2023 at 02:54:13PM -0700, Nathan Bossart wrote: >> IMO the phrase "open a port" is kind of nonstandard. I think we should say >> something along the lines of >> >> If list

Re: Add 'worker_type' to pg_stat_subscription

2023-09-07 Thread Nathan Bossart
ker with this type, perhaps it would be better to actually list it as "unknown". Thoughts? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 85b826752b1b10176809918975ce7b495dadd3db Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 7 Sep 2023 15:24:23 -0700 Subject: [P

Re: Document that server will start even if it's unable to open some TCP/IP ports

2023-09-07 Thread Nathan Bossart
ten_addresses is not empty, the server will start only if it can listen on at least one of the specified addresses. A warning will be emitted for any addresses that the server cannot listen on. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: should frontend tools use syncfs() ?

2023-09-06 Thread Nathan Bossart
On Tue, Sep 05, 2023 at 05:08:53PM -0700, Nathan Bossart wrote: > I've committed 0001 for now. I plan to commit the rest in the next couple > of days. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: should frontend tools use syncfs() ?

2023-09-05 Thread Nathan Bossart
I've committed 0001 for now. I plan to commit the rest in the next couple of days. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add 'worker_type' to pg_stat_subscription

2023-09-05 Thread Nathan Bossart
On Wed, Sep 06, 2023 at 09:02:21AM +1200, Peter Smith wrote: > On Sat, Sep 2, 2023 at 7:41 AM Nathan Bossart > wrote: >> I see that the table refers to "leader apply workers". Would those show up >> as parallel apply workers in the view? Can we add anot

Re: Replace known_assigned_xids_lck by memory barrier

2023-09-05 Thread Nathan Bossart
On Fri, Sep 01, 2023 at 03:53:54PM -0400, Robert Haas wrote: > I'm not an expert on this code but I looked at this patch briefly and > it seems OK to me. Thanks for taking a look. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Inefficiency in parallel pg_restore with many tables

2023-09-04 Thread Nathan Bossart
On Sat, Sep 02, 2023 at 11:55:21AM -0700, Nathan Bossart wrote: > I ended up hacking together a (nowhere near committable) patch to see how > hard it would be to allow using any type with binaryheap. It doesn't seem > too bad. I spent some more time on this patch and made the

Re: Inefficiency in parallel pg_restore with many tables

2023-09-03 Thread Nathan Bossart
On Sun, Sep 03, 2023 at 12:04:00PM +0200, Alvaro Herrera wrote: > On 2023-Sep-02, Nathan Bossart wrote: >> I ended up hacking together a (nowhere near committable) patch to see how >> hard it would be to allow using any type with binaryheap. It doesn't seem >> too bad.

Re: Inefficiency in parallel pg_restore with many tables

2023-09-02 Thread Nathan Bossart
On Fri, Sep 01, 2023 at 01:52:48PM -0700, Nathan Bossart wrote: > On Fri, Sep 01, 2023 at 04:00:44PM -0400, Robert Haas wrote: >> In hindsight, I think that making binaryheap depend on Datum was a bad >> idea. I think that was my idea, and I think it wasn't very smart. >> Co

Re: Inefficiency in parallel pg_restore with many tables

2023-09-01 Thread Nathan Bossart
directly in the array thanks to the use of Datum. However, it _would_ allow us to abandon this frontend/backend void */Datum kludge, which is something. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add 'worker_type' to pg_stat_subscription

2023-09-01 Thread Nathan Bossart
ly workers". Would those show up as parallel apply workers in the view? Can we add another worker type for those? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Replace known_assigned_xids_lck by memory barrier

2023-09-01 Thread Nathan Bossart
On Wed, Aug 16, 2023 at 01:07:15PM -0700, Nathan Bossart wrote: > Ah, that explains it. v5 of the patch is attached. Barring additional feedback, I plan to commit this patch in the current commitfest. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: should frontend tools use syncfs() ?

2023-09-01 Thread Nathan Bossart
On Fri, Sep 01, 2023 at 01:19:13PM -0500, Justin Pryzby wrote: > What about (per git grep no-sync doc) pg_receivewal? I don't think it's applicable there, either. IIUC that option specifies whether to sync the data as it is streamed over. -- Nathan Bossart Amazon Web Services: ht

Re: should frontend tools use syncfs() ?

2023-09-01 Thread Nathan Bossart
inct error when syncfs is not supported > than when it's truely recognized. Later versions of the patch should have this. > The patch should handle pg_dumpall, too. It looks like pg_dumpall only ever fsyncs a single file, so I don't think it is really needed there. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Adding a pg_get_owned_sequence function?

2023-09-01 Thread Nathan Bossart
On Fri, Sep 01, 2023 at 01:31:43PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> I wonder if it'd be possible to just remove pg_get_serial_sequence(). > > A quick search at http://codesearch.debian.net/ finds uses of it > in packages like gdal, qgis, rails, ... We c

Re: Inefficiency in parallel pg_restore with many tables

2023-09-01 Thread Nathan Bossart
On Fri, Sep 01, 2023 at 01:41:41PM -0400, Tom Lane wrote: > I've not actually looked at any of these patchsets after the first one. > I have added myself as a reviewer and will hopefully get to it within > a week or so. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Inefficiency in parallel pg_restore with many tables

2023-09-01 Thread Nathan Bossart
On Tue, Jul 25, 2023 at 11:53:36AM -0700, Nathan Bossart wrote: > Here is a polished patch set for this approach. I've also added a 0004 > that replaces the open-coded heap in pg_dump_sort.c with a binaryheap. > IMHO these patches are in decent shape. I'm hoping to commit these patche

Re: Adding a pg_get_owned_sequence function?

2023-09-01 Thread Nathan Bossart
But I suppose it wouldn't be too much trouble to keep it around for anyone who happens to need it. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: should frontend tools use syncfs() ?

2023-08-31 Thread Nathan Bossart
worth worrying about. > +command_ok([ 'initdb', '-S', $datadir, '--sync-method', 'fsync' ], > + 'sync method fsync'); > > Removing this one may be fine, actually, because we test the sync > paths on other places like pg_dump. Done. > This split is OK by me, so WFM. Cool. -

Re: should frontend tools use syncfs() ?

2023-08-31 Thread Nathan Bossart
dds DataDirSyncMethod and uses it for recovery_init_sync_method. The second adds syncfs() support in file_utils.c. And the third adds the ability to specify syncfs in the frontend utilities. WDYT? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From e627b8ea4dcc01ac9e2fe3a21e3b3cc109b815e4 Mon Sep 17 0

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-30 Thread Nathan Bossart
On Wed, Aug 30, 2023 at 10:56:22AM -0400, Robert Haas wrote: > On Wed, Aug 30, 2023 at 10:27 AM Nathan Bossart > wrote: >> I'm about to spend way too much time writing the commit message for 0002, >> but I plan to commit both patches sometime today. > > Thanks!

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-30 Thread Nathan Bossart
tly, I don't have any comments on the proposed patches. I'm about to spend way too much time writing the commit message for 0002, but I plan to commit both patches sometime today. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-29 Thread Nathan Bossart
On Wed, Aug 30, 2023 at 08:22:27AM +0900, Michael Paquier wrote: > On Tue, Aug 29, 2023 at 09:46:55AM -0700, Nathan Bossart wrote: >> This was my first reaction [0]. I was concerned about renaming the >> exported functions so close to release, so I was suggesting that we hold >

Re: Wrong usage of pqMsg_Close message code?

2023-08-29 Thread Nathan Bossart
On Wed, Aug 30, 2023 at 07:56:33AM +0900, Michael Paquier wrote: > On Tue, Aug 29, 2023 at 02:11:06PM -0700, Nathan Bossart wrote: >> I plan to commit the attached patch shortly. > > WFM. Committed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: should frontend tools use syncfs() ?

2023-08-29 Thread Nathan Bossart
cause compiler warnings until they are implemented. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From f5ecb3c8b5d397c94a9f8ab9a053b3a0cfd7f854 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 21 Aug 2023 19:05:14 -0700 Subject: [PATCH v7 1/2] move PG_TEMP_FILE*

Re: Is pg_regress --use-existing used by anyone or is it broken?

2023-08-29 Thread Nathan Bossart
think I've ever used it. AFAICT it was added with hot standby mode (efc16ea) to support 'make standbycheck', which was removed last year (4483b2cf). Unless someone claims to be using it, it's probably fine to just remove it. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Wrong usage of pqMsg_Close message code?

2023-08-29 Thread Nathan Bossart
On Tue, Aug 29, 2023 at 09:15:55AM -0700, Nathan Bossart wrote: > Thanks for the report. I'll get this fixed up. My guess is that this was > leftover from an earlier version of the patch that used the same macro for > identical protocol characters. I plan to commit the attached patc

Re: tablecmds.c/MergeAttributes() cleanup

2023-08-29 Thread Nathan Bossart
On Tue, Aug 29, 2023 at 08:44:02PM +0200, Alvaro Herrera wrote: > On 2023-Aug-29, Nathan Bossart wrote: >> My compiler is complaining about 1fa9241b: >> >> ../postgresql/src/backend/commands/sequence.c: In function ‘DefineSequence’: >> ../postgresql/src/backend/comman

Re: tablecmds.c/MergeAttributes() cleanup

2023-08-29 Thread Nathan Bossart
Elts, coldef); | ^~~~ This went away when I added a default case that ERROR'd or initialized coldef to NULL. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-29 Thread Nathan Bossart
tions so close to release, so I was suggesting that we hold off on that part until v17. If there isn't a concern with renaming these functions in v16, I can proceed with something more like v2. [0] https://postgr.es/m/20230824161913.GA1394441%40nathanxps13.lan -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Wrong usage of pqMsg_Close message code?

2023-08-29 Thread Nathan Bossart
hings) out in the first round to avoid prolonging the naming discussion, but we can continue to add to protocol.h. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: should frontend tools use syncfs() ?

2023-08-29 Thread Nathan Bossart
rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 1bb28cfe5d40670386ae663e14c8854dc1b5486d Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 21 Aug 2023 19:05:14 -0700 Subject: [PATCH v6 1/2] move PG_TEMP_FILE* macros to file_utils.h --- src/backend/bac

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-25 Thread Nathan Bossart
On Fri, Aug 25, 2023 at 08:32:51AM -0700, Nathan Bossart wrote: > On second thought, renaming these exported functions so close to release is > probably not a great idea. I should probably skip back-patching that one. > Or I could have the existing functions call the new on

Re: initdb caching during tests

2023-08-25 Thread Nathan Bossart
On Thu, Aug 24, 2023 at 03:10:00PM -0700, Andres Freund wrote: > Cool. Pushed that way. I just noticed the tests running about 30% faster on my machine due to this. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-25 Thread Nathan Bossart
gt; to call out that "subxact_count" will "only increase if a transaction is > performing writes". Also to link > the reader to the subtransactions doc [2]. I'd rather keep this patch focused on fixing the bug, given we are so close to the v16 release. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-25 Thread Nathan Bossart
ea. I should probably skip back-patching that one. Or I could have the existing functions call the new ones in v16 for backward compatibility... -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add timing information to pg_upgrade

2023-08-24 Thread Nathan Bossart
On Wed, Aug 23, 2023 at 09:35:20AM -0700, Nathan Bossart wrote: > On Tue, Aug 22, 2023 at 07:06:23AM -0700, Nathan Bossart wrote: >> On Tue, Aug 22, 2023 at 11:49:33AM +0200, Peter Eisentraut wrote: >>> Let's change MESSAGE_WIDTH to 62 in v16, and then pursue the larger >>&

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-24 Thread Nathan Bossart
On Wed, Aug 23, 2023 at 07:51:40PM -0700, Nathan Bossart wrote: > On Wed, Aug 23, 2023 at 07:32:06PM -0700, Nathan Bossart wrote: >> On Thu, Aug 24, 2023 at 10:22:49AM +0900, Ian Lawrence Barwick wrote: >>> Looking at the code, this is happening because >>> "

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-23 Thread Nathan Bossart
On Wed, Aug 23, 2023 at 07:32:06PM -0700, Nathan Bossart wrote: > On Thu, Aug 24, 2023 at 10:22:49AM +0900, Ian Lawrence Barwick wrote: >> Looking at the code, this is happening because >> "pgstat_fetch_stat_local_beentry()" >> expects to be passed the backend ID

Re: pg_stat_get_backend_subxact() and backend IDs?

2023-08-23 Thread Nathan Bossart
could probably modify pgstat_fetch_stat_beentry() to use this new function. I suspect we'll want to work on the naming, too. Maybe we could name them pg_stat_fetch_local_beentry_by_index() and pg_stat_fetch_local_beentry_by_backendid(). -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add timing information to pg_upgrade

2023-08-23 Thread Nathan Bossart
On Tue, Aug 22, 2023 at 07:06:23AM -0700, Nathan Bossart wrote: > On Tue, Aug 22, 2023 at 11:49:33AM +0200, Peter Eisentraut wrote: >> Let's change MESSAGE_WIDTH to 62 in v16, and then pursue the larger >> restructuring leisurely. > > Sounds good. I plan to commit this w

Re: [PATCH] Add function to_oct

2023-08-23 Thread Nathan Bossart
On Tue, Aug 22, 2023 at 04:57:02PM +0200, Peter Eisentraut wrote: > On 22.08.23 16:26, Nathan Bossart wrote: >> > I don't understand the reason for this handling of negative values. I >> > would >> > expect that, say, to_hex(-1234) would return '-' || to_hex(1

Re: Using defines for protocol characters

2023-08-22 Thread Nathan Bossart
On Thu, Aug 17, 2023 at 09:52:03AM -0700, Nathan Bossart wrote: > On Thu, Aug 17, 2023 at 12:22:24PM -0400, Robert Haas wrote: >> I think this is going to be a major improvement in terms of readability! >> >> I'm pretty happy with this version, too. We may be running out

Re: should frontend tools use syncfs() ?

2023-08-22 Thread Nathan Bossart
On Tue, Aug 22, 2023 at 12:53:53PM +0900, Michael Paquier wrote: > On Mon, Aug 21, 2023 at 07:06:32PM -0700, Nathan Bossart wrote: >> This would look something like the attached patch. I think this is nicer. >> With this patch, we don't have to choose between including fd.h o

Re: [PATCH] Add function to_oct

2023-08-22 Thread Nathan Bossart
On Tue, Aug 22, 2023 at 04:20:02PM +0200, Peter Eisentraut wrote: > On 20.08.23 17:25, Nathan Bossart wrote: >> > Doing a quick test, shows that this changes the current behaviour, >> > because all inputs are now treated as 64-bit: >> > >> > HEAD

Re: add timing information to pg_upgrade

2023-08-22 Thread Nathan Bossart
On Tue, Aug 22, 2023 at 11:49:33AM +0200, Peter Eisentraut wrote: > Let's change MESSAGE_WIDTH to 62 in v16, and then pursue the larger > restructuring leisurely. Sounds good. I plan to commit this within the next couple of days. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: should frontend tools use syncfs() ?

2023-08-21 Thread Nathan Bossart
On Tue, Aug 22, 2023 at 10:50:01AM +0900, Michael Paquier wrote: > On Mon, Aug 21, 2023 at 06:44:07PM -0700, Nathan Bossart wrote: >> I'm hoping there's a simpler path forward here. pg_rewind only needs the >> following lines from fd.h: >> >> /* Filename co

Re: should frontend tools use syncfs() ?

2023-08-21 Thread Nathan Bossart
.h: /* Filename components */ #define PG_TEMP_FILES_DIR "pgsql_tmp" #define PG_TEMP_FILE_PREFIX "pgsql_tmp" Maybe we could move these to file_utils.h instead. WDYT? -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Optimize Arm64 crc32 implementation in PostgreSQL

2023-08-21 Thread Nathan Bossart
src/port/pg_crc32c_armv8.c), but AFAICT Postgres has no such optimization for CRC-32. The CRC-32 macros have a comment indicating that they are currently only used in ltree and hstore, so there might not be terribly much demand for hardware acceleration, though. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: [PATCH] Add function to_oct

2023-08-21 Thread Nathan Bossart
s, even though I understand the words "to bin" > could be interpreted differently. (Looking elsewhere for precedents, > Python has bin(), oct() and hex() functions.) I renamed it to to_bin(). > Also, I think the convention is to always list functions > alphabetically, so to_oct(

Re: [PATCH] Add function to_oct

2023-08-20 Thread Nathan Bossart
gestion and noted that the output is the two's complement representation [0]. [0] https://postgr.es/m/CAEZATCVbkL1ynqpsKiTDpch34%3DSCr5nnau%3DnfNmiy2nM3SJHtw%40mail.gmail.com -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From a03c06d0b26ccbd49bda55c2efab565ab9fa90db Mon Sep 17 0

Re: [PATCH] Add function to_oct

2023-08-20 Thread Nathan Bossart
> Assert(base <= 16); > > do > { > *--ptr = digits[value % base]; > value /= base; > } while (ptr > buf && value); > > return cstring_to_text_with_len(ptr, end - ptr); > } I will use this in v8. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: SLRUs in the main buffer pool - Page Header definitions

2023-08-18 Thread Nathan Bossart
e SLRU > mechanism, we don't have any objection to it either. I think I agree with Stephen. We routinely make changes that require updates to extensions, and I doubt anyone is terribly wild about maintaining two SLRU systems for several years. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: should frontend tools use syncfs() ?

2023-08-18 Thread Nathan Bossart
On Thu, Aug 17, 2023 at 12:50:31PM +0900, Michael Paquier wrote: > On Wed, Aug 16, 2023 at 08:17:05AM -0700, Nathan Bossart wrote: >> The patch does have the following note: >> >> +On Linux, syncfs may be used instead to ask the >> +operating system to

Re: Using defines for protocol characters

2023-08-17 Thread Nathan Bossart
more! Awesome. If we are indeed out of things to argue about, I'll plan on committing this sometime next week. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Using defines for protocol characters

2023-08-17 Thread Nathan Bossart
codes. I'm OK to do things incrementally so it's fine by > me to not add them now, just noticing on the way what could be added > to this new header. Cool, thanks. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 69f94689857a469333fecbfd2057868140796516 Mon Sep 17 00:0

Re: [PATCH] Add function to_oct

2023-08-17 Thread Nathan Bossart
; own palloc. > > Or maybe better, save the starting pointer, compute the length at the end, > and call cstring_to_text_with_len()? (It seems we wouldn't need > the nul-terminator then, either.) Works for me. I did it that way in v7. -- Nathan Bossart Amazon Web Services: http

Re: run pgindent on a regular basis / scripted manner

2023-08-17 Thread Nathan Bossart
On Wed, Aug 16, 2023 at 01:15:55PM -0700, Peter Geoghegan wrote: > On Tue, Aug 15, 2023 at 1:31 PM Nathan Bossart > wrote: >> Should we add those? Patch attached. > > I think that that makes sense. Committed. > I just don't want to normalize updating > .git-blame-ign

Re: Replace known_assigned_xids_lck by memory barrier

2023-08-16 Thread Nathan Bossart
use only > the startup process can remove and add xids to KnownAssignedXids" Ah, that explains it. v5 of the patch is attached. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 6038182ac96226f0eec43b1c6ab2e060e8b1e3a8 Mon Sep 17 00:00:00 2001 From: Nathan Bossa

Re: Using defines for protocol characters

2023-08-16 Thread Nathan Bossart
because > these defines should be usable by third parties. I moved the definitions out to a separate file in v6. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From fc54d1655fe662abe99a4605bdff2d1b65f0dc2a Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 16 A

Re: Replace known_assigned_xids_lck by memory barrier

2023-08-16 Thread Nathan Bossart
read barrier before the array entries are inspected. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 11de5076adc060c0dde32e8538714301f9b98d02 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 16 Aug 2023 10:52:56 -0700 Subject: [PATCH v4 1/1] Replace known_as

Re: should frontend tools use syncfs() ?

2023-08-16 Thread Nathan Bossart
On Wed, Aug 16, 2023 at 08:17:05AM -0700, Nathan Bossart wrote: > On Wed, Aug 16, 2023 at 08:10:10AM +0900, Michael Paquier wrote: >> + pg_log_error("could not synchronize file system for file \"%s\": >> %m", path); >> + (void) close(fd); >

<    4   5   6   7   8   9   10   11   12   13   >