Re: Generate pg_stat_get_xact*() functions with Macros

2023-01-11 Thread Drouvot, Bertrand
Hi, On 1/11/23 11:59 PM, Andres Freund wrote: Hi, Michael, CCing you because of the point about PG_STAT_GET_DBENTRY_FLOAT8 below. On 2023-01-05 14:48:39 +0100, Drouvot, Bertrand wrote: While at it, I also took the opportunity to create the macros for pg_stat_get_xact_function_total_time(),

Re: Using WaitEventSet in the postmaster

2023-01-11 Thread Thomas Munro
On Thu, Jan 12, 2023 at 7:57 PM Thomas Munro wrote: > On Thu, Jan 12, 2023 at 7:27 PM Tom Lane wrote: > > skink seems to have found a problem: > > > > ==2011873== VALGRINDERROR-BEGIN > > ==2011873== Syscall param epoll_wait(events) points to unaddressable byte(s) > > ==2011873==at 0x4D8DC73:

Re: [PATCH] Const'ify the arguments of ilist.c/ilist.h functions

2023-01-11 Thread Peter Eisentraut
On 07.01.23 08:21, Peter Eisentraut wrote: On 23.11.22 14:57, Aleksander Alekseev wrote: Hi Andres, Thanks for the review! I don't think it is correct for any of these to add const. The only reason it works is because of casting etc. Fair enough. PFA the corrected patch v2. This patch

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-11 Thread Drouvot, Bertrand
Hi, On 1/12/23 5:44 AM, Michael Paquier wrote: On Wed, Jan 11, 2023 at 07:17:47PM +0530, Bharath Rupireddy wrote: Note that the raw page on the table might differ not just in page LSN but also in other fields, for instance see heap_mask for instance. It masks lsn, checksum, hint bits, unused

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-11 Thread Will Mortensen
Hi Andres, On Wed, Jan 11, 2023 at 12:33 PM Andres Freund wrote: > I think such a function would still have to integrate enough with the lock > manager infrastructure to participate in the deadlock detector. Otherwise I > think you'd trivially end up with loads of deadlocks. Could you elaborate

RE: Fix pg_publication_tables to exclude generated columns

2023-01-11 Thread shiy.f...@fujitsu.com
On Wed, Jan 11, 2023 2:40 PM Amit Kapila wrote: > > On Wed, Jan 11, 2023 at 10:07 AM Tom Lane wrote: > > > > Amit Kapila writes: > > >> On Mon, Jan 9, 2023 11:06 PM Tom Lane wrote: > > >>> We could just not fix it in the back branches. I'd argue that this is > > >>> as much a definition

Re: Using WaitEventSet in the postmaster

2023-01-11 Thread Thomas Munro
On Thu, Jan 12, 2023 at 7:27 PM Tom Lane wrote: > skink seems to have found a problem: > > ==2011873== VALGRINDERROR-BEGIN > ==2011873== Syscall param epoll_wait(events) points to unaddressable byte(s) > ==2011873==at 0x4D8DC73: epoll_wait (epoll_wait.c:30) > ==2011873==by 0x55CA49:

Re: On login trigger: take three

2023-01-11 Thread Pavel Stehule
Hi po 19. 12. 2022 v 10:40 odesílatel Mikhail Gribkov napsal: > Hi Ted, > > > bq. in to the system > > 'in to' -> into > > bq. Any bugs in a trigger procedure > > Any bugs -> Any bug > > Thanks! Fixed typos in the attached v35. > > > + if (event == EVT_Login) > > +

Re: Fix condition in shm_toc and remove unused function shm_toc_freespace.

2023-01-11 Thread Zhang Mingli
Hi, On Jan 12, 2023, 14:34 +0800, Zhang Mingli , wrote: > Hi, hackers > > Some conditions in shm_toc_insert and shm_toc_allocate are bogus, like: > if (toc_bytes + nbytes > total_bytes || toc_bytes + nbytes < > toc_bytes) > Remove the condition `toc_bytes + nbytes < toc_bytes` and

Re: Small miscellaneus fixes (Part II)

2023-01-11 Thread John Naylor
On Thu, Jan 12, 2023 at 12:34 PM Justin Pryzby wrote: > > On Thu, Jan 12, 2023 at 12:15:24PM +0700, John Naylor wrote: > > On Fri, Dec 23, 2022 at 8:08 AM Justin Pryzby wrote: > > > > > Makes sense now (in your first message, you said that the problem was > > > with "sign", and the patch didn't

Re: Common function for percent placeholder replacement

2023-01-11 Thread Peter Eisentraut
On 11.01.23 19:54, Nathan Bossart wrote: On Wed, Jan 11, 2023 at 11:09:01AM +0100, Peter Eisentraut wrote: committed with that fixed While rebasing my recovery modules patch set, I noticed a couple of small things that might be worth cleaning up. committed, thanks

Fix condition in shm_toc and remove unused function shm_toc_freespace.

2023-01-11 Thread Zhang Mingli
Hi, hackers Some conditions in shm_toc_insert and shm_toc_allocate are bogus, like: if (toc_bytes + nbytes > total_bytes || toc_bytes + nbytes < toc_bytes) Remove the condition `toc_bytes + nbytes < toc_bytes` and take a sizeof(shm_entry) into account in shm_toc_allocate though

Re: recovery modules

2023-01-11 Thread Michael Paquier
On Wed, Jan 11, 2023 at 11:29:01AM -0800, Nathan Bossart wrote: > I'm having trouble thinking of any practical advantage of providing the > redo LSN and TLI. If the main use-case is removing older archives as the > documentation indicates, it seems better to provide the file name so that > you

Re: Using WaitEventSet in the postmaster

2023-01-11 Thread Tom Lane
Thomas Munro writes: > Pushed, after a few very minor adjustments, mostly comments. Thanks > for the reviews and pointers. I think there are quite a lot of > refactoring and refinement opportunities unlocked by this change (I > have some draft proposals already), but for now I'll keep an eye on

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-11 Thread Michael Paquier
On Wed, Jan 11, 2023 at 06:59:18PM +0530, Bharath Rupireddy wrote: > I've done it that way for pg_get_wal_fpi_info. If this format looks > okay, I can propose to do the same for other functions (for > backpatching too) in a separate thread though. My vote would be to make that happen first, to

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-11 Thread Masahiko Sawada
On Wed, Jan 11, 2023 at 12:13 PM John Naylor wrote: > > On Tue, Jan 10, 2023 at 7:08 PM Masahiko Sawada wrote: > > > It looks no problem in terms of vacuum integration, although I've not > > fully tested yet. TID store uses the radix tree as the main storage, > > and with the template radix

Re: pgsql: Doc: add XML ID attributes to and tags.

2023-01-11 Thread Brar Piening
On 12.01.2023 at 00:05, Tom Lane wrote: Peter Eisentraut writes: Any reason the new ids in create_database.sgml deviate from the normal naming schemes used everywhere else? Is it to preserve the existing create-database-strategy? Maybe we should rename that one and make the new ones

Re: Small miscellaneus fixes (Part II)

2023-01-11 Thread Justin Pryzby
On Thu, Jan 12, 2023 at 12:15:24PM +0700, John Naylor wrote: > On Fri, Dec 23, 2022 at 8:08 AM Justin Pryzby wrote: > > > Makes sense now (in your first message, you said that the problem was > > with "sign", and the patch didn't address the actual problem in > > IS_PLUS()). > > > > One can look

Re: [EXTERNAL] Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2023-01-11 Thread Michael Paquier
On Wed, Jan 11, 2023 at 03:22:35PM +0100, Jelte Fennema wrote: > The main uncertainty I have is if the case insensitivity check is > actually needed in check_role. It seems like a case insensitive > check against the database user shouldn't actually be necessary. > I only understand the need for

Re: resend from mailing list archive doesn't working

2023-01-11 Thread Pavel Stehule
čt 12. 1. 2023 v 6:24 odesílatel Justin Pryzby napsal: > On Thu, Jan 12, 2023 at 06:20:16AM +0100, Pavel Stehule wrote: > > Hi > > > > I need to resend > > > https://www.postgresql.org/message-id/CALte62yFXQvRrA47unpedfcn%3DGoE_VyvxcKkqj2NUhenK__qgA%40mail.gmail.com > > > > Unfortunately I

Re: resend from mailing list archive doesn't working

2023-01-11 Thread Justin Pryzby
On Thu, Jan 12, 2023 at 06:20:16AM +0100, Pavel Stehule wrote: > Hi > > I need to resend > https://www.postgresql.org/message-id/CALte62yFXQvRrA47unpedfcn%3DGoE_VyvxcKkqj2NUhenK__qgA%40mail.gmail.com > > Unfortunately I didn't get any mail. It looks like you're already on the "CC" list, and so

Re: Remove source code display from \df+?

2023-01-11 Thread Pavel Stehule
st 11. 1. 2023 v 22:11 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > st 11. 1. 2023 v 19:31 odesílatel Magnus Hagander > > napsal: > >> This is only about Internal and C, isn't it? Isn't the oid of these > >> static, and identified by INTERNALlanguageId and ClanguageId >

resend from mailing list archive doesn't working

2023-01-11 Thread Pavel Stehule
Hi I need to resend https://www.postgresql.org/message-id/CALte62yFXQvRrA47unpedfcn%3DGoE_VyvxcKkqj2NUhenK__qgA%40mail.gmail.com Unfortunately I didn't get any mail. Regards Pavel

Re: Small miscellaneus fixes (Part II)

2023-01-11 Thread John Naylor
On Fri, Dec 23, 2022 at 8:08 AM Justin Pryzby wrote: > Makes sense now (in your first message, you said that the problem was > with "sign", and the patch didn't address the actual problem in > IS_PLUS()). > > One can look and find that the unreachable code was introduced at > 7a3e7b64a. > > With

Re: on placeholder entries in view rule action query's range table

2023-01-11 Thread Justin Pryzby
On Wed, Jan 11, 2023 at 10:45:33PM -0500, Tom Lane wrote: > Amit Langote writes: > > On Thu, Jan 12, 2023 at 10:06 AM Tom Lane wrote: > >> I've pushed this with some cleanup --- aside from fixing > >> outfuncs/readfuncs, I did some more work on the comments, which > >> I think you were too

Re: allowing for control over SET ROLE

2023-01-11 Thread Noah Misch
On Wed, Jan 11, 2023 at 03:13:29PM -0500, Robert Haas wrote: > On Wed, Jan 11, 2023 at 10:16 AM Noah Misch wrote: > > I still think docs for the SET option itself should give a sense of the > > diversity of things it's intended to control. It could be simple. A bunch > > of > > the sites

Re: Spinlock is missing when updating two_phase of ReplicationSlot

2023-01-11 Thread Masahiko Sawada
On Thu, Jan 12, 2023 at 1:42 PM Michael Paquier wrote: > > On Wed, Jan 11, 2023 at 02:36:17PM +0900, Michael Paquier wrote: > > Looks right to me, the paths updating the data related to the slots > > are careful about that, even when it comes to fetching a slot from > > MyReplicationSlot. I have

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-11 Thread Amit Kapila
On Thu, Jan 12, 2023 at 9:54 AM Peter Smith wrote: > > > doc/src/sgml/monitoring.sgml > > 5. pg_stat_subscription > > @@ -3198,11 +3198,22 @@ SELECT pid, wait_event_type, wait_event FROM > pg_stat_activity WHERE wait_event i > > > > + apply_leader_pid integer > + > +

Re: Cygwin cleanup

2023-01-11 Thread Justin Pryzby
On Wed, Jan 11, 2023 at 10:39:49PM -0600, Justin Pryzby wrote: > Here's my latest copy of the patch. > + # due to resource constraints we don't run this task by default for now > + trigger_type: manual Now, with trigger_type commented, so Thomas doesn't have to click "trigger" for me. >From

Re: on placeholder entries in view rule action query's range table

2023-01-11 Thread Amit Langote
On Thu, Jan 12, 2023 at 12:45 PM Tom Lane wrote: > Amit Langote writes: > > On Thu, Jan 12, 2023 at 10:06 AM Tom Lane wrote: > >> I've pushed this with some cleanup --- aside from fixing > >> outfuncs/readfuncs, I did some more work on the comments, which > >> I think you were too sloppy

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-11 Thread Bharath Rupireddy
On Thu, Jan 12, 2023 at 10:14 AM Michael Paquier wrote: > > On Wed, Jan 11, 2023 at 07:17:47PM +0530, Bharath Rupireddy wrote: > > Note that the raw page on the table might differ not just in page LSN > > but also in other fields, for instance see heap_mask for instance. It > > masks lsn,

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-11 Thread Michael Paquier
On Wed, Jan 11, 2023 at 07:17:47PM +0530, Bharath Rupireddy wrote: > Note that the raw page on the table might differ not just in page LSN > but also in other fields, for instance see heap_mask for instance. It > masks lsn, checksum, hint bits, unused space etc. before verifying FPI > consistency

Re: Spinlock is missing when updating two_phase of ReplicationSlot

2023-01-11 Thread Michael Paquier
On Wed, Jan 11, 2023 at 02:36:17PM +0900, Michael Paquier wrote: > Looks right to me, the paths updating the data related to the slots > are careful about that, even when it comes to fetching a slot from > MyReplicationSlot. I have been looking around the slot code to see if > there are other

Re: Cygwin cleanup

2023-01-11 Thread Justin Pryzby
On Sat, Jan 07, 2023 at 12:39:11AM +1300, Thomas Munro wrote: > On Wed, Nov 9, 2022 at 2:04 PM Justin Pryzby wrote: > > +data_sync_retry = on > > Sharing with the list some clues that Justin and I figured out about > what that part is doing. Without it, you get failures like: > > PANIC:

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-11 Thread Peter Smith
Hi, here are some review comments for patch v78-0001. == General 1. (terminology) AFAIK everywhere until now we’ve been referring everywhere (docs/comments/code) to the parent apply worker as the "leader apply worker". Not the "main apply worker". Not the "apply leader worker". Not any

Re: Using WaitEventSet in the postmaster

2023-01-11 Thread Thomas Munro
On Wed, Jan 11, 2023 at 4:07 PM Thomas Munro wrote: > Thanks. Here's v10. I'll wait a bit longer to see if anyone else has > feedback. Pushed, after a few very minor adjustments, mostly comments. Thanks for the reviews and pointers. I think there are quite a lot of refactoring and

Re: Fixing a couple of buglets in how VACUUM sets visibility map bits

2023-01-11 Thread Peter Geoghegan
On Mon, Jan 9, 2023 at 12:58 PM Peter Geoghegan wrote: > On Mon, Jan 9, 2023 at 11:57 AM Andres Freund wrote: > > Afaict we'll need to backpatch this all the way? > > I thought that we probably wouldn't need to, at first. But I now think > that we really have to. Attached is v4. This is almost

Re: on placeholder entries in view rule action query's range table

2023-01-11 Thread Tom Lane
Amit Langote writes: > On Thu, Jan 12, 2023 at 10:06 AM Tom Lane wrote: >> I've pushed this with some cleanup --- aside from fixing >> outfuncs/readfuncs, I did some more work on the comments, which >> I think you were too sloppy about. > Thanks a lot for the fixes. It looks like we're not

Re: Blocking execution of SECURITY INVOKER

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 18:16:32 -0800, Jeff Davis wrote: > Motivation > == > > SECURITY INVOKER is dangerous, particularly for administrators. There > are numerous ways to put code in a place that's likely to be executed: > triggers, views calling functions, logically-replicated tables,

Re: How to generate the new expected out file.

2023-01-11 Thread Tom Lane
Amit Kapila writes: > On Thu, Jan 12, 2023 at 4:39 AM Andres Freund wrote: >> On 2023-01-05 16:22:01 +0530, Amit Kapila wrote: >>> You can run the tests and copy the required changes from >>> src/test/regress/output/interval.out to >>> src/test/regress/expected/interval.out >> Wonder if we

Re: ATTACH PARTITION seems to ignore column generation status

2023-01-11 Thread Tom Lane
Amit Langote writes: > I noticed a typo in the doc additions, which I've attached a fix for. Doh, right, pushed. regards, tom lane

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-11 Thread shveta malik
On Wed, Jan 11, 2023 at 6:16 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shveta, > > Thanks for reviewing! PSA new version. > > > 1. > > + errmsg("min_apply_delay must not be set when streaming = parallel"))); > > we give the same error msg for both the cases: > > a. when subscription is created

Re: How to generate the new expected out file.

2023-01-11 Thread Amit Kapila
On Thu, Jan 12, 2023 at 4:39 AM Andres Freund wrote: > > On 2023-01-05 16:22:01 +0530, Amit Kapila wrote: > > On Thu, Jan 5, 2023 at 4:12 PM jian he wrote: > > > Hi. > > > > > > I changed the src/test/regress/sql/interval.sql, How can I generate the > > > new

RE: Allow logical replication to copy tables in binary format

2023-01-11 Thread Takamichi Osumi (Fujitsu)
On Wednesday, January 11, 2023 7:45 PM Melih Mutlu wrote: > Thanks for your review. > Done. Hi, minor comments on v5. (1) publisher's version check + /* If the publisher is v16 or later, copy in binary format.*/ + server_version = walrcv_server_version(LogRepWorkerWalRcvConn); +

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-11 Thread Kyotaro Horiguchi
At Wed, 11 Jan 2023 12:46:24 +, "Hayato Kuroda (Fujitsu)" wrote in > them. Which version is better? Some comments by a quick loock, different from the above. + CONNECTION 'host=192.168.1.50 port=5432 user=foo dbname=foodb' I understand that we (not PG people, but IT people) are

Re: ATTACH PARTITION seems to ignore column generation status

2023-01-11 Thread Amit Langote
On Thu, Jan 12, 2023 at 5:58 AM Tom Lane wrote: > Amit Langote writes: > > I've updated your disallow-generated-child-columns-2.patch to do this, > > and have also merged the delta post that I had attached with my last > > email, whose contents you sound to agree with. > > Pushed with some

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-11 Thread Ted Yu
On Wed, Jan 11, 2023 at 6:54 PM Amit Kapila wrote: > On Wed, Jan 11, 2023 at 9:31 AM Ted Yu wrote: > > > > On Tue, Jan 10, 2023 at 7:55 PM houzj.f...@fujitsu.com < > houzj.f...@fujitsu.com> wrote: > >> > >> On Wednesday, January 11, 2023 10:21 AM Ted Yu > wrote: > >> > /* First time

Re: releasing ParallelApplyTxnHash when pa_launch_parallel_worker returns NULL

2023-01-11 Thread Amit Kapila
On Wed, Jan 11, 2023 at 9:31 AM Ted Yu wrote: > > On Tue, Jan 10, 2023 at 7:55 PM houzj.f...@fujitsu.com > wrote: >> >> On Wednesday, January 11, 2023 10:21 AM Ted Yu wrote: >> > /* First time through, initialize parallel apply worker state >> > hashtable. */ >> > if

Re: on placeholder entries in view rule action query's range table

2023-01-11 Thread Amit Langote
On Thu, Jan 12, 2023 at 10:06 AM Tom Lane wrote: > Amit Langote writes: > > On Mon, Jan 9, 2023 at 5:58 AM Tom Lane wrote: > >> Conceivably we could make it work by allowing RTE_SUBQUERY RTEs to > >> carry a relation OID and associated RTEPermissionInfo, so that when a > >> view's RTE_RELATION

Blocking execution of SECURITY INVOKER

2023-01-11 Thread Jeff Davis
Motivation == SECURITY INVOKER is dangerous, particularly for administrators. There are numerous ways to put code in a place that's likely to be executed: triggers, views calling functions, logically-replicated tables, casts, search path and function resolution tricks, etc. If this code

RE: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2023-01-11 Thread Yoshikazu Imai (Fujitsu)
Hi. While investigating the codes in RelationCopyStorageUsingBuffer, I wonder that there is any reason to use RBM_NORMAL for acquiring destination buffer. I think we can use RBM_ZERO_AND_LOCK here instead of RBM_NORMAL. When we use RBM_NORMAL, a destination block is read by smgrread and it's

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2023-01-11 Thread Peter Geoghegan
On Wed, Jan 11, 2023 at 4:44 PM Andres Freund wrote: > > Attached patch fixes up these issues. It's almost totally mechanical. > > Looks better, thanks! Pushed that just now. Thanks -- Peter Geoghegan

Re: No Callbacks on FATAL

2023-01-11 Thread Tom Lane
Andres Freund writes: > On 2023-01-11 18:10:33 -0500, Tom Lane wrote: >> It's intended behavior, and I seriously doubt that it ever worked >> differently. > Hm? MemoryContextDelete() unconditionally calls the > callbacks. ShutdownPostgres() calls AbortOutOfAnyTransaction(). So if there's > an

Re: on placeholder entries in view rule action query's range table

2023-01-11 Thread Tom Lane
Amit Langote writes: > On Mon, Jan 9, 2023 at 5:58 AM Tom Lane wrote: >> Conceivably we could make it work by allowing RTE_SUBQUERY RTEs to >> carry a relation OID and associated RTEPermissionInfo, so that when a >> view's RTE_RELATION RTE is transmuted to an RTE_SUBQUERY RTE it still >> carries

Re: low wal_retrieve_retry_interval causes missed signals on Windows

2023-01-11 Thread Nathan Bossart
On Wed, Jan 11, 2023 at 04:40:14PM -0800, Andres Freund wrote: > On 2023-01-11 15:26:45 -0800, Nathan Bossart wrote: >> On Wed, Jan 11, 2023 at 12:48:36PM -0800, Andres Freund wrote: >> > Given that we check for interrupts in other parts of recovery with >> > HandleStartupProcInterrupt(), which

Re: No Callbacks on FATAL

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 18:10:33 -0500, Tom Lane wrote: > Ed Behn writes: > > I'm using a callback function that is called when a memory context is > > deleted to remove a temporary file. This works fine when the transaction > > ends normally or raises an ERROR. However, when a FATAL event happens,

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-11 Thread David G. Johnston
On Wed, Jan 11, 2023 at 2:16 PM Robert Haas wrote: > On Wed, Jan 11, 2023 at 4:00 PM Tom Lane wrote: > > Robert Haas writes: > > > If you want to make safe a SECURITY DEFINER function written using sql > > > or plpgsql, you either have to schema-qualify every single reference > > > or, more

Re: Switching XLog source from archive to streaming when primary available

2023-01-11 Thread Nathan Bossart
On Tue, Oct 18, 2022 at 07:31:33AM +0530, Bharath Rupireddy wrote: > In summary, the standby state machine in WaitForWALToBecomeAvailable() > exhausts all the WAL in pg_wal before switching to streaming after > failing to fetch from archive. The v8 patch proposed upthread deviates > from this

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 16:06:31 -0800, Peter Geoghegan wrote: > On Mon, Jan 9, 2023 at 2:18 PM Peter Geoghegan wrote: > > I'll try to get back to it this week. > > Attached patch fixes up these issues. It's almost totally mechanical. Looks better, thanks! > (Ended up using "git diff

Re: low wal_retrieve_retry_interval causes missed signals on Windows

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 15:26:45 -0800, Nathan Bossart wrote: > On Wed, Jan 11, 2023 at 12:48:36PM -0800, Andres Freund wrote: > > Given that we check for interrupts in other parts of recovery with > > HandleStartupProcInterrupt(), which doesn't interact with latches, isn't the > > actual bug that

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 17:26:19 -0700, David G. Johnston wrote: > Should we just add "ring_buffers" to the existing "shared_buffers" and > "temp_buffers" settings? The different types of ring buffers have different sizes, for good reasons. So I don't see that working well. I also think it'd be more

Re: Use windows VMs instead of windows containers on the CI

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 17:21:21 -0600, Justin Pryzby wrote: > On Tue, Jan 10, 2023 at 03:20:18PM +0300, Nazir Bilal Yavuz wrote: > > Hi, > > > > I propose using windows VMs instead of containers, the patch is attached. > > Currently, windows containers are used on the CI, but these container images >

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread David G. Johnston
On Wed, Jan 11, 2023 at 2:39 PM Andres Freund wrote: > Hi, > > On 2023-01-11 16:18:34 -0500, Tom Lane wrote: > > Peter Geoghegan writes: > > > On Wed, Jan 11, 2023 at 11:18 AM Andres Freund > wrote: > > >> I don't like that - it's also quite useful to disable use of > ringbuffers when > > >>

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2023-01-11 Thread Peter Geoghegan
On Mon, Jan 9, 2023 at 2:18 PM Peter Geoghegan wrote: > I'll try to get back to it this week. Attached patch fixes up these issues. It's almost totally mechanical. (Ended up using "git diff --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space" with this, per your recent tip, which did

Re: Refactor recordExtObjInitPriv()

2023-01-11 Thread Nathan Bossart
On Tue, Dec 27, 2022 at 09:56:10AM +0100, Peter Eisentraut wrote: > Refactor recordExtObjInitPriv(): Instead of half a dozen of > mostly-duplicate conditional branches, write one common one that can handle > most catalogs. We already have all the information we need, such as which > system

Re: Rework of collation code, extensibility

2023-01-11 Thread Jeff Davis
On Wed, 2023-01-11 at 15:08 +0100, Peter Eisentraut wrote: > I think the refactoring that you proposed in the thread "Refactor to > introduce pg_strcoll()." was on a sensible track.  Maybe we should > try > to get that done. Those should be patches 0001-0003 in this thread (now at v6), which

Re: mprove tab completion for ALTER EXTENSION ADD/DROP

2023-01-11 Thread Michael Paquier
On Wed, Jan 11, 2023 at 10:29:25PM +0530, vignesh C wrote: > I too felt keeping it simpler is better. How about using the simple > first version of patch itself? Okay, I have just done that, then, after checking that all the object types were covered (28). Note that PROCEDURAL LANGUAGE has been

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-11 Thread Peter Geoghegan
On Wed, Jan 11, 2023 at 3:06 PM Andres Freund wrote: > > + * We can't use TransactionIdDidAbort here because it won't treat > > transactions > > + * that were in progress during a crash as aborted by now. We determine > > that > > + * transactions aborted/crashed through process of elimination

Re: low wal_retrieve_retry_interval causes missed signals on Windows

2023-01-11 Thread Nathan Bossart
On Wed, Jan 11, 2023 at 12:48:36PM -0800, Andres Freund wrote: > Given that we check for interrupts in other parts of recovery with > HandleStartupProcInterrupt(), which doesn't interact with latches, isn't the > actual bug that HandleStartupProcInterrupt() doesn't contain the same black > magic

Re: Use windows VMs instead of windows containers on the CI

2023-01-11 Thread Justin Pryzby
On Tue, Jan 10, 2023 at 03:20:18PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > I propose using windows VMs instead of containers, the patch is attached. > Currently, windows containers are used on the CI, but these container images > are needs to get pulled on every CI run, also they are slow to

Re: Show various offset arrays for heap WAL records

2023-01-11 Thread Peter Geoghegan
On Wed, Jan 11, 2023 at 3:00 PM Andres Freund wrote: > What are your thoughts about the place for the helper functions? You're ok > with rmgrdesc_utils.[ch]? Yeah, that seems okay. We may well need to put more stuff in that file. We're overdue a big overhaul of the rmgr output, so that

Re: No Callbacks on FATAL

2023-01-11 Thread Tom Lane
Ed Behn writes: > I'm using a callback function that is called when a memory context is > deleted to remove a temporary file. This works fine when the transaction > ends normally or raises an ERROR. However, when a FATAL event happens, the > callback is not run. Is this a bug or intended

Re: How to generate the new expected out file.

2023-01-11 Thread Andres Freund
Hi, On 2023-01-05 16:22:01 +0530, Amit Kapila wrote: > On Thu, Jan 5, 2023 at 4:12 PM jian he wrote: > > Hi. > > > > I changed the src/test/regress/sql/interval.sql, How can I generate the new > > src/test/regress/expected/interval.out file. > > > > You can run the tests and copy the required

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 14:29:25 -0800, Peter Geoghegan wrote: > On Sat, Jan 7, 2023 at 7:25 PM Andres Freund wrote: > > Probably a good idea, although it doesn't neatly fit right now. > > I'll leave it for now. > > Attached is v2, which changes things based on your feedback. Would > like to get

Re: pgsql: Doc: add XML ID attributes to and tags.

2023-01-11 Thread Tom Lane
Peter Eisentraut writes: > On 09.01.23 21:08, Tom Lane wrote: >> Doc: add XML ID attributes to and tags. > Any reason the new ids in create_database.sgml deviate from the normal > naming schemes used everywhere else? Is it to preserve the existing > create-database-strategy? Maybe we

Re: No Callbacks on FATAL

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 17:47:28 -0500, Ed Behn wrote: > I'm developing a module that implements Haskell as a procedural language ( > https://www.postgresql.org/about/news/plhaskell-v10-released-2519/) > > I'm using a callback function that is called when a memory context is > deleted to remove a

Re: Show various offset arrays for heap WAL records

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 14:53:54 -0800, Peter Geoghegan wrote: > On Tue, Jan 10, 2023 at 11:35 AM Andres Freund wrote: > > Nontrivial, I'm afraid. We don't pass any relevant parameters to rm_desc: > > void(*rm_desc) (StringInfo buf, XLogReaderState > > *record); > > > > so we'd

Re: Generate pg_stat_get_xact*() functions with Macros

2023-01-11 Thread Andres Freund
Hi, Michael, CCing you because of the point about PG_STAT_GET_DBENTRY_FLOAT8 below. On 2023-01-05 14:48:39 +0100, Drouvot, Bertrand wrote: > While at it, I also took the opportunity to create the macros for > pg_stat_get_xact_function_total_time(), > pg_stat_get_xact_function_self_time() and

Re: Show various offset arrays for heap WAL records

2023-01-11 Thread Peter Geoghegan
On Tue, Jan 10, 2023 at 11:35 AM Andres Freund wrote: > Nontrivial, I'm afraid. We don't pass any relevant parameters to rm_desc: > void(*rm_desc) (StringInfo buf, XLogReaderState *record); > > so we'd need to patch all of them. That might be worth doing at some point, > but I

Re: Add SHELL_EXIT_CODE to psql

2023-01-11 Thread Corey Huinker
> > > > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: > > Conflict was due to the doc patch applying id tags to psql variable names. I've rebased and added my own id tags to the two new variables. From 9e2827a6f955e7cebf87ca538fab113a359951b4 Mon Sep 17 00:00:00

No Callbacks on FATAL

2023-01-11 Thread Ed Behn
I'm developing a module that implements Haskell as a procedural language ( https://www.postgresql.org/about/news/plhaskell-v10-released-2519/) I'm using a callback function that is called when a memory context is deleted to remove a temporary file. This works fine when the transaction ends

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-11 Thread Peter Geoghegan
On Sat, Jan 7, 2023 at 7:25 PM Andres Freund wrote: > Probably a good idea, although it doesn't neatly fit right now. I'll leave it for now. Attached is v2, which changes things based on your feedback. Would like to get this out of the way soon. > > Also, does amcheck's get_xid_status() need a

Re: [PATCH] psql: Add tab-complete for optional view parameters

2023-01-11 Thread Thomas Munro
On Thu, Jan 12, 2023 at 5:50 AM vignesh C wrote: > For some reason CFBot is not able to apply the patch, please have a > look and post an updated version if required, also check and handle > Dean Rasheed and Jim Jones comment if applicable: > === Applying patches on top of PostgreSQL commit ID >

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-11 Thread Magnus Hagander
On Wed, Jan 11, 2023 at 8:06 PM Jacob Champion wrote: > On Wed, Jan 11, 2023 at 10:23 AM Magnus Hagander > wrote: > > Sorry to jump in (very) late in this game. So first, I like this general > approach :) > > Thanks! > > > It feels icky to have to add configure tests just to make a test work. >

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-11 Thread Justin Pryzby
> Subject: [PATCH v45 4/5] Add system view tracking IO ops per backend type The patch can/will fail with: CREATE TABLESPACE test_io_shared_stats_tblspc LOCATION ''; +WARNING: tablespaces created by regression test cases should have names starting with "regress_" CREATE TABLESPACE test_stats

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 22:30:42 +0100, Tomas Vondra wrote: > On 1/11/23 21:58, Andres Freund wrote: > > If you're thinking of decoding changes in parallel (rather than streaming > > out > > large changes before commit when possible), you'd only be able to do that in > > cases when transaction haven't

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Tomas Vondra
On 1/11/23 21:12, Andres Freund wrote: > Hi, > > > Heikki, CCed you due to the point about 2c03216d8311 below. > > > On 2023-01-10 19:32:12 +0100, Tomas Vondra wrote: >> 0001 is a fix for the pre-existing issue in logicalmsg_decode, >> attempting to build a snapshot before getting into a

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 16:18:34 -0500, Tom Lane wrote: > Peter Geoghegan writes: > > On Wed, Jan 11, 2023 at 11:18 AM Andres Freund wrote: > >> I don't like that - it's also quite useful to disable use of ringbuffers > >> when > >> you actually need to clean up indexes. Especially when we have a

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Tomas Vondra
On 1/11/23 21:58, Andres Freund wrote: > Hi, > > On 2023-01-11 15:41:45 -0500, Robert Haas wrote: >> I wonder, then, what happens if somebody wants to do parallel apply. That >> would seem to require some relaxation of this rule, but then doesn't that >> break what this patch wants to do? >

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Jan 11, 2023 at 11:18 AM Andres Freund wrote: >> I don't like that - it's also quite useful to disable use of ringbuffers when >> you actually need to clean up indexes. Especially when we have a lot of dead >> tuples we'll rescan indexes over and over... >

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-11 Thread Robert Haas
On Wed, Jan 11, 2023 at 4:00 PM Tom Lane wrote: > Robert Haas writes: > > If you want to make safe a SECURITY DEFINER function written using sql > > or plpgsql, you either have to schema-qualify every single reference > > or, more realistically, attach a SET clause to the function to set the > >

Re: doc: mentioned CREATE+ATTACH PARTITION as an alternative to CREATE TABLE..PARTITION OF

2023-01-11 Thread Robert Haas
On Wed, Jan 11, 2023 at 10:48 AM Robert Treat wrote: > > @Robert: I wonder why shouldn't CREATE..PARTITION OF *also* be patched > > to first create a table, and then attach the partition, transparently > > doing what everyone would want, without having to re-read the updated > > docs or know to

Re: Remove source code display from \df+?

2023-01-11 Thread Tom Lane
Pavel Stehule writes: > st 11. 1. 2023 v 19:31 odesílatel Magnus Hagander > napsal: >> This is only about Internal and C, isn't it? Isn't the oid of these >> static, and identified by INTERNALlanguageId and ClanguageId respectively? >> So we could just have the query show the prosrc column if

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 14:38:34 -0600, Justin Pryzby wrote: > On Wed, Jan 11, 2023 at 10:58:54AM -0800, Andres Freund wrote: > > Some ideas: > > > > USE_RING_BUFFERS on|off > > REUSE_BUFFERS on|off > > +1 for either of these. Then I'd go for REUSE_BUFFERS. What made you prefer it over

Re: Can we let extensions change their dumped catalog schemas?

2023-01-11 Thread Tom Lane
Jacob Champion writes: > On Tue, Jan 10, 2023 at 7:53 PM Tom Lane wrote: >> I also fear that it will break >> a bunch of use-cases that work fine today, which are exactly the >> ones for which we originally defined pg_dump as *not* committing >> to a particular extension version. > Right, I

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-11 Thread Tom Lane
Robert Haas writes: > If you want to make safe a SECURITY DEFINER function written using sql > or plpgsql, you either have to schema-qualify every single reference > or, more realistically, attach a SET clause to the function to set the > search_path to a sane value during the time that the

Re: ATTACH PARTITION seems to ignore column generation status

2023-01-11 Thread Tom Lane
Amit Langote writes: > I've updated your disallow-generated-child-columns-2.patch to do this, > and have also merged the delta post that I had attached with my last > email, whose contents you sound to agree with. Pushed with some further work to improve the handling of multiple- inheritance

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 15:41:45 -0500, Robert Haas wrote: > I wonder, then, what happens if somebody wants to do parallel apply. That > would seem to require some relaxation of this rule, but then doesn't that > break what this patch wants to do? I don't think it'd pose a direct problem - presumably

Re: low wal_retrieve_retry_interval causes missed signals on Windows

2023-01-11 Thread Andres Freund
Hi, On 2023-01-10 22:11:16 -0800, Nathan Bossart wrote: > The attached patch fixes this by always calling WaitLatch(), even if > wal_retrieve_retry_interval milliseconds have already elapsed and the > timeout is 0. It doesn't seem right to call WaitLatch() just for that purpose - nor necessarily

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Robert Haas
On Wed, Jan 11, 2023 at 3:28 PM Andres Freund wrote: > On 2023-01-11 15:23:18 -0500, Robert Haas wrote: > > Yeah, I meant if #1 had committed and then #2 started to do its thing. > > I was worried that decoding might reach the nextval operations in > > transaction #2 before it replayed #1. > > >

  1   2   >