Re: Invalidate the subscription worker in cases where a user loses their superuser status

2023-10-12 Thread Amit Kapila
On Fri, Oct 13, 2023 at 10:04 AM vignesh C wrote: > > On Thu, 12 Oct 2023 at 11:10, Amit Kapila wrote: > > > > On Sun, Oct 8, 2023 at 8:22 AM vignesh C wrote: > > > > > > > --- a/src/include/catalog/pg_subscription.h > > +++ b/src/include/catalog/pg_subscription.h > > @@ -127,6 +127,7 @@

Re: pg_upgrade's interaction with pg_resetwal seems confusing

2023-10-12 Thread Amit Kapila
On Fri, Oct 13, 2023 at 10:37 AM Dilip Kumar wrote: > > On Fri, Oct 13, 2023 at 9:29 AM Amit Kapila wrote: > > > > On Fri, Oct 13, 2023 at 12:00 AM Robert Haas wrote: > > > > > > On Thu, Oct 12, 2023 at 7:17 AM Amit Kapila > > > wrote: > > > > Now, as mentioned in the first paragraph, it

Re: PostgreSQL domains and NOT NULL constraint

2023-10-12 Thread Tom Lane
Vik Fearing writes: > Regardless of what the spec may or may not say about v1.d, it still > remains that nulls should not be allowed in a *base table* if the domain > says nulls are not allowed. Not mentioned in this thread but the > constraints are also applied when CASTing to the domain.

Re: Tab completion for AT TIME ZONE

2023-10-12 Thread Michael Paquier
On Fri, Oct 13, 2023 at 03:07:25AM +0200, Vik Fearing wrote: > The SQL committee already has another operator starting with AT which is AT > LOCAL. The other patch was the reason why I looked at this one. At the end, I've made peace with Dagfinn's argument two messages ago, and applied the patch

Re: pg_upgrade's interaction with pg_resetwal seems confusing

2023-10-12 Thread Dilip Kumar
On Fri, Oct 13, 2023 at 9:29 AM Amit Kapila wrote: > > On Fri, Oct 13, 2023 at 12:00 AM Robert Haas wrote: > > > > On Thu, Oct 12, 2023 at 7:17 AM Amit Kapila wrote: > > > Now, as mentioned in the first paragraph, it seems we anyway don't > > > need to reset the WAL at the end when setting the

Re: Add support for AT LOCAL

2023-10-12 Thread Michael Paquier
On Fri, Oct 13, 2023 at 07:03:20AM +0200, Vik Fearing wrote: > Thank you, Michael君! No pb, ヴィックさん。 -- Michael signature.asc Description: PGP signature

Re: Add support for AT LOCAL

2023-10-12 Thread Vik Fearing
On 10/13/23 05:07, Michael Paquier wrote: On Fri, Oct 13, 2023 at 02:20:59AM +0200, Vik Fearing wrote: On 10/10/23 05:34, Michael Paquier wrote: I am attaching a v5 that addresses the documentation bits, could you look at the business with date.c? Here is a v6 Thanks for the new version.

Re: Invalidate the subscription worker in cases where a user loses their superuser status

2023-10-12 Thread vignesh C
On Thu, 12 Oct 2023 at 11:10, Amit Kapila wrote: > > On Sun, Oct 8, 2023 at 8:22 AM vignesh C wrote: > > > > --- a/src/include/catalog/pg_subscription.h > +++ b/src/include/catalog/pg_subscription.h > @@ -127,6 +127,7 @@ typedef struct Subscription > * skipped */ > char*name; /* Name of

Re: Add support for AT LOCAL

2023-10-12 Thread Michael Paquier
On Fri, Oct 13, 2023 at 02:20:59AM +0200, Vik Fearing wrote: > On 10/10/23 05:34, Michael Paquier wrote: > > I am attaching a v5 that addresses the documentation bits, could you > > look at the business with date.c? > > Here is a v6 Thanks for the new version. > which hopefully addresses all of

Re: pg_upgrade's interaction with pg_resetwal seems confusing

2023-10-12 Thread Amit Kapila
On Fri, Oct 13, 2023 at 12:00 AM Robert Haas wrote: > > On Thu, Oct 12, 2023 at 7:17 AM Amit Kapila wrote: > > Now, as mentioned in the first paragraph, it seems we anyway don't > > need to reset the WAL at the end when setting the next OID for the new > > cluster with the -o option. If that is

Re: Wait events for delayed checkpoints

2023-10-12 Thread Thomas Munro
On Fri, Oct 13, 2023 at 2:19 PM Robert Haas wrote: > On Thu, Oct 12, 2023 at 7:09 PM Michael Paquier wrote: > > HaveVirtualXIDsDelayingChkpt() does immediately a LWLockAcquire() > > which would itself report a wait event for ProcArrayLock, overwriting > > this new one, no? > > Ah, right: the

Re: PGDOCS - add more links in the pub/sub reference pages

2023-10-12 Thread Peter Smith
On Thu, Oct 12, 2023 at 3:44 PM Amit Kapila wrote: > > On Mon, Oct 9, 2023 at 12:15 PM Peter Smith wrote: > > > > On Mon, Oct 9, 2023 at 3:32 PM Amit Kapila wrote: > > > > > > > In v1, I used the same pattern as on the CREATE SUBSCRIPTION page, > > which doesn't look like those... > > > > Yeah,

Re: [dynahash] do not refill the hashkey after hash_search

2023-10-12 Thread Nathan Bossart
On Thu, Sep 14, 2023 at 04:28:26PM +0800, Junwang Zhao wrote: > Add a v2 with some change to fix warnings about unused-parameter. > > I will add this to Commit Fest. This looks reasonable to me. I've marked the commitfest entry as ready-for-committer. I will plan on committing it in a couple

Re: Removing unneeded self joins

2023-10-12 Thread Andrei Lepikhov
On 12/10/2023 18:32, Alexander Korotkov wrote: On Thu, Oct 5, 2023 at 12:17 PM Andrei Lepikhov We have almost the results we wanted to have. But in the last explain you can see that nothing happened with the OR clause. We should use the expression mutator instead of walker to handle such

Re: PostgreSQL domains and NOT NULL constraint

2023-10-12 Thread Vik Fearing
On 10/13/23 02:44, Tom Lane wrote: Vik Fearing writes: On 10/12/23 15:54, Tom Lane wrote: There's been some discussion of treating the output of such a join, subselect, etc as being of the domain's base type not the domain proper. That'd solve this particular issue since then we'd decide we

Re: Improve the log message output of basic_archive when basic_archive.archive_directory parameter is not set

2023-10-12 Thread Nathan Bossart
On Tue, Sep 26, 2023 at 08:13:45AM +0200, Daniel Gustafsson wrote: >> On 26 Sep 2023, at 00:20, Nathan Bossart wrote: >> >> On Thu, Sep 21, 2023 at 11:18:00AM +0900, bt23nguyent wrote: >>> -basic_archive_configured(ArchiveModuleState *state) >>> +basic_archive_configured(ArchiveModuleState

Re: Some performance degradation in REL_16 vs REL_15

2023-10-12 Thread Andres Freund
Hi, On 2023-10-12 11:00:22 +0300, Anton A. Melnikov wrote: > Found that simple test pgbench -c20 -T20 -j8 gives approximately > for REL_15_STABLE at 5143f76: 336+-1 TPS > and > for REL_16_STABLE at 4ac7635f: 324+-1 TPS > > The performance drop is approximately 3,5% while the corrected standard

Re: PostgreSQL domains and NOT NULL constraint

2023-10-12 Thread Tom Lane
Vik Fearing writes: > On 10/12/23 15:54, Tom Lane wrote: >> There's been some discussion of treating the output of such a join, >> subselect, etc as being of the domain's base type not the domain >> proper. That'd solve this particular issue since then we'd decide >> we have to cast the base

Re: PostgreSQL domains and NOT NULL constraint

2023-10-12 Thread Vik Fearing
On 10/12/23 15:54, Tom Lane wrote: Erki Eessaar writes: PostgreSQL's CREATE DOMAIN documentation (section Notes) describes a way how one can add NULL's to a column that has a domain with the NOT NULL constraint. https://www.postgresql.org/docs/current/sql-createdomain.html To me it seems very

Re: Wait events for delayed checkpoints

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 7:09 PM Michael Paquier wrote: > On Thu, Oct 12, 2023 at 01:32:29PM -0400, Robert Haas wrote: > > IPC seems right to me. Yeah, a timeout is being used, but as you say, > > that's an implementation detail. > > > > +1 for the idea, too. > > Agreed that timeout makes little

Re: On login trigger: take three

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 6:54 PM Alexander Korotkov wrote: > On Thu, Oct 12, 2023 at 8:35 PM Robert Haas wrote: > > Doesn't that mean that if you create the first login trigger in a > > database and leave the transaction open, nobody can connect to that > > database until the transaction ends? >

Re: Tab completion for AT TIME ZONE

2023-10-12 Thread Vik Fearing
On 10/12/23 10:27, Dagfinn Ilmari Mannsåker wrote: Michael Paquier writes: On Fri, Apr 14, 2023 at 12:05:25PM +0200, Jim Jones wrote: The patch applies cleanly and it does what it is proposing. - and it's IMHO a very nice addition. I've marked the CF entry as "Ready for Committer". +/*

Re: SQL:2011 application time

2023-10-12 Thread Vik Fearing
On 10/11/23 05:47, Paul Jungwirth wrote: +SELECT pg_get_indexdef(conindid, 0, true) FROM pg_constraint WHERE conname = 'temporal_rng_pk'; +    pg_get_indexdef +--- + CREATE UNIQUE INDEX

Re: Some performance degradation in REL_16 vs REL_15

2023-10-12 Thread Michael Paquier
On Thu, Oct 12, 2023 at 09:20:36PM +1300, David Rowley wrote: > It would be interesting to know what's to blame here and if you can > attribute it to a certain commit. +1. -- Michael signature.asc Description: PGP signature

Re: CHECK Constraint Deferrable

2023-10-12 Thread Vik Fearing
On 10/10/23 15:12, Robert Haas wrote: On Mon, Oct 9, 2023 at 5:07 PM David G. Johnston wrote: 2. I don't think it's a good idea for the same patch to try to solve two problems unless they are so closely related that solving one without solving the other is not sensible. A NOT NULL constraint

Re: Add support for AT LOCAL

2023-10-12 Thread Vik Fearing
On 10/10/23 05:34, Michael Paquier wrote: I am attaching a v5 that addresses the documentation bits, could you look at the business with date.c? Here is a v6 which hopefully addresses all of your concerns. -- Vik Fearing From 042ce9b581ca3b17afbf229d209ca59addb6c9a2 Mon Sep 17 00:00:00 2001

Re: Test 026_overwrite_contrecord fails on very slow machines (under Valgrind)

2023-10-12 Thread Andres Freund
Hi, On 2023-10-12 14:00:00 +0300, Alexander Lakhin wrote: > So to fail on the test, skink should perform at least twice slower than > usual The machine skink is hosted on runs numerous buildfarm animals (24 I think right now, about to be 28). While it has plenty resources (16 cores/32 threads,

Re: LLVM 16 (opaque pointers)

2023-10-12 Thread Andres Freund
Hi, On 2023-10-11 21:59:50 +1300, Thomas Munro wrote: > +#else > + LLVMPassBuilderOptionsRef options; > + LLVMErrorRef err; > + int compile_optlevel; > + char *passes; > + > + if (context->base.flags & PGJIT_OPT3) > + compile_optlevel = 3;

Re: Test 026_overwrite_contrecord fails on very slow machines (under Valgrind)

2023-10-12 Thread Michael Paquier
On Thu, Oct 12, 2023 at 02:00:00PM +0300, Alexander Lakhin wrote: > So to fail on the test, skink should perform at least twice slower than > usual, and may be it's an extraordinary condition indeed, but on the other > hand, may be increase checkpoint_timeout as already done in several tests >

Re: Making aggregate deserialization (and WAL receive) functions slightly faster

2023-10-12 Thread David Rowley
On Wed, 11 Oct 2023 at 08:52, Tom Lane wrote: > > David Rowley writes: > > I've attached a slightly more worked on patch that makes maxlen == 0 > > mean read-only. Unsure if a macro is worthwhile there or not. > > A few thoughts: Thank you for the review. I spent more time on this and did end

Re: interval_ops shall stop using btequalimage (deduplication)

2023-10-12 Thread Peter Geoghegan
On Thu, Oct 12, 2023 at 4:10 PM Noah Misch wrote: > > exactly one case like that post-fix (interval_ops is at least the only > > affected core code opfamily), so why not point that out directly with > > a HINT? A HINT could go a long way towards putting the problem in > > context, without really

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-10-12 Thread Michael Paquier
On Thu, Oct 12, 2023 at 10:41:39AM -0400, David Steele wrote: > After some more thought, I think we could massage the "pg_control in > backup_label" method into something that could be back patched, with more > advanced features (e.g. error on backup_label and pg_control both present on > initial

Re: A new strategy for pull-up correlated ANY_SUBLINK

2023-10-12 Thread Alena Rybakina
On 12.10.2023 10:52, Andy Fan wrote: Unfortunately, I found a request when sublink did not pull-up, as in the examples above. I couldn't quite figure out why. I'm not sure what you mean with the "above", I guess it should be the "below"? Yes, you are right) explain

Re: interval_ops shall stop using btequalimage (deduplication)

2023-10-12 Thread Noah Misch
On Wed, Oct 11, 2023 at 01:00:44PM -0700, Peter Geoghegan wrote: > On Wed, Oct 11, 2023 at 11:38 AM Noah Misch wrote: > > Interesting. So, >99% of interval-type indexes, even ones WITH > > (deduplicate_items=off), will get amcheck failures. The <1% of exceptions > > might include indexes having

Re: Wait events for delayed checkpoints

2023-10-12 Thread Michael Paquier
On Thu, Oct 12, 2023 at 01:32:29PM -0400, Robert Haas wrote: > IPC seems right to me. Yeah, a timeout is being used, but as you say, > that's an implementation detail. > > +1 for the idea, too. Agreed that timeout makes little sense in this context, and IPC looks correct. +

Re: On login trigger: take three

2023-10-12 Thread Alexander Korotkov
On Thu, Oct 12, 2023 at 8:35 PM Robert Haas wrote: > On Tue, Oct 10, 2023 at 3:43 PM Alexander Korotkov > wrote: > > Yep, in v43 it worked that way. One transaction has to wait for > > another finishing update of pg_database tuple, then fails. This is > > obviously ridiculous. Since

BRIN minmax multi - incorrect distance for infinite timestamp/date

2023-10-12 Thread Tomas Vondra
Hi, Ashutosh Bapat reported me off-list a possible issue in how BRIN minmax-multi calculate distance for infinite timestamp/date values. The current code does this: if (TIMESTAMP_NOT_FINITE(dt1) || TIMESTAMP_NOT_FINITE(dt2)) PG_RETURN_FLOAT8(0); so means infinite values are "very

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread David G. Johnston
On Thu, Oct 12, 2023 at 2:58 PM Nikita Malakhov wrote: > Why pg_upgrade cannot be used? > We document both a pg_dump/pg_restore migration and a pg_upgrade one (not to mention that logical backup and restore would cause the oids to change). It seems odd to have a feature that requires

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread Nikita Malakhov
Hi, Textual representation requires a long text field because it could contain schema, arguments, it is difficult and not effective to be saved as part of the data, and must be parsed to retrieve function oid. By using direct oid (actually, a value of the regprocedure field) we avoid it and

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

2023-10-12 Thread Peter Geoghegan
On Thu, Oct 12, 2023 at 1:10 PM Robert Haas wrote: > On Thu, Oct 12, 2023 at 12:01 PM Peter Geoghegan wrote: > > No objections from me. > > Here is a doc-only patch that I think could be back-patched as far as > emergency mode exists. It combines all of the wording changes to the > documentation

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread David G. Johnston
On Thu, Oct 12, 2023 at 1:31 PM Nikita Malakhov wrote: > About using surrogate key - this feature is more for data generated by > the DBMS itself, i.e. data processed by some extension and saved > and re-processed automatically or by user's request, but without bothering > user with these

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread Nikita Malakhov
Hi, I've already implemented preserving PG_PROC oids during pg_upgrade in a way like relfilenodes, etc, actually, it is quite simple, and on the first look there are no any problems. About using surrogate key - this feature is more for data generated by the DBMS itself, i.e. data processed by

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 3:36 PM David G. Johnston wrote: > Every catalog has both a natural and a surrogate key. Developers get to use > the surrogate key while end-users get to use the natural one (i.e., the one > they provided). I see no reason to change that specification. I agree with

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 12:01 PM Peter Geoghegan wrote: > No objections from me. Here is a doc-only patch that I think could be back-patched as far as emergency mode exists. It combines all of the wording changes to the documentation from v1-v3 of the previous version, but without changing the

building 32bit windows version

2023-10-12 Thread Dave Cramer
Greetings, I've been running into challenges building 32 bit windows version. I suspect there are no build farms and nobody really builds this. The reason I need these is to be able to build 32 bit dll's for ODBC. At one time EDB used to provide binaries but that doesn't appear to be the case.

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread David G. Johnston
On Thu, Oct 12, 2023 at 11:43 AM Robert Haas wrote: > On Thu, Oct 12, 2023 at 2:38 PM David G. Johnston > wrote: > > It's more like a lot number or surveying tract than an postal address. > Useful for a single party, the builder or the government, but not something > you give out to other

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 2:38 PM David G. Johnston wrote: > It's more like a lot number or surveying tract than an postal address. > Useful for a single party, the builder or the government, but not something > you give out to other people so they can find you. > > Whether or not we copy over

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread David G. Johnston
On Thu, Oct 12, 2023, 11:21 Robert Haas wrote: > > The pg_upgrade experience right now is a bit as if you woke up in the > morning and found that city officials came by during the night and > renumbered your house, thus changing your address. Then, they sent > change of address forms to everyone

Re: pg_upgrade's interaction with pg_resetwal seems confusing

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 7:17 AM Amit Kapila wrote: > Now, as mentioned in the first paragraph, it seems we anyway don't > need to reset the WAL at the end when setting the next OID for the new > cluster with the -o option. If that is true, then I think even without > slots work it will be helpful

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 10:35 AM Tom Lane wrote: > You have the burden of proof backwards. That would add a great deal > of new mechanism, and you haven't provided even one reason why it'd > be worth doing. "A great deal of new mechanism" seems like a slight exaggeration. We preserve a bunch of

Re: New WAL record to detect the checkpoint redo location

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 3:27 AM Michael Paquier wrote: > I have looked at 0001, for now.. And it looks OK to me. Cool. I've committed that one. Thanks for the review. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread David G. Johnston
On Thu, Oct 12, 2023 at 7:36 AM Tom Lane wrote: > Nikita Malakhov writes: > > Please advise on the idea of preserving pg_proc oids during pg_upgrade, > in > > a way like relfilenodes, type id and so on. What are possible downsides > of > > such a solution? > > You have the burden of proof

Re: Special-case executor expression steps for common combinations

2023-10-12 Thread Andres Freund
Hi, On 2023-10-12 13:24:27 +0300, Heikki Linnakangas wrote: > On 12/10/2023 12:48, Daniel Gustafsson wrote: > > The attached patch adds special-case expression steps for common sets of > > steps > > in the executor to shave a few cycles off during execution, and make the JIT > > generated code

Re: On login trigger: take three

2023-10-12 Thread Robert Haas
On Tue, Oct 10, 2023 at 3:43 PM Alexander Korotkov wrote: > Yep, in v43 it worked that way. One transaction has to wait for > another finishing update of pg_database tuple, then fails. This is > obviously ridiculous. Since overlapping setters of flag will have to > wait anyway, I changed lock

Re: Wait events for delayed checkpoints

2023-10-12 Thread Robert Haas
On Wed, Oct 11, 2023 at 9:13 PM Thomas Munro wrote: > You can't tell if your checkpointer is spending a lot of time waiting > around for flags in delayChkptFlags to clear. Trivial patch to add > that. I've managed to see it a few times when checkpointing > repeatedly with a heavy pgbench

Re: Eager page freeze criteria clarification

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 11:50 AM Melanie Plageman wrote: > I was under the impression that we decided we still had to consider > the number of clean pages dirtied as well as the number of pages > unfrozen. The number of pages frozen and unfrozen over a time period > gives us some idea of if we

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread David G. Johnston
On Thu, Oct 12, 2023 at 9:57 AM Nikita Malakhov wrote: > Say, we have data processed by some user function and we want to keep > reference to this function > in our data. > Then you need to keep the user-visible identifier of said function (schema+name+input argument types - you'd probably want

Re: Separate memory contexts for relcache and catcache

2023-10-12 Thread Andres Freund
Hi, On 2023-08-09 15:02:31 +0300, Melih Mutlu wrote: > To quickly show how pg_backend_memory_contexts would look like, I did the > following: > > -Create some tables: > SELECT 'BEGIN;' UNION ALL SELECT format('CREATE TABLE %1$s(id serial > primary key, data text not null unique)', 'test_'||g.i)

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread Nikita Malakhov
Hi! Say, we have data processed by some user function and we want to keep reference to this function in our data. In this case we have two ways - first - store string output of regprocedure, which is not very convenient, and the second - store its OID, which requires slight modification of

Re: Performance degradation on concurrent COPY into a single relation in PG16.

2023-10-12 Thread Andres Freund
Hi, On 2023-10-12 11:44:09 -0400, Tom Lane wrote: > Andres Freund writes: > >> On 2023-09-25 15:42:26 -0400, Tom Lane wrote: > >>> I just did a git bisect run to discover when the failure documented > >>> in bug #18130 [1] started. And the answer is commit 82a4edabd. > > > Uh, huh. The

Re: Parent/child context relation in pg_get_backend_memory_contexts()

2023-10-12 Thread Andres Freund
Hi, On 2023-08-04 21:16:49 +0300, Melih Mutlu wrote: > Melih Mutlu , 16 Haz 2023 Cum, 17:03 tarihinde şunu > yazdı: > > > With this change, here's a query to find how much space used by each > > context including its children: > > > > > WITH RECURSIVE cte AS ( > > > SELECT id, total_bytes,

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

2023-10-12 Thread Peter Geoghegan
On Thu, Oct 12, 2023 at 8:54 AM Robert Haas wrote: > - I find the use of the word "generate" in error messages slightly > odd. I think it's reasonable given the existing precedent, but the > word I would have picked is "assign", which I see is what Aleksander > actually had in v1. How would

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

2023-10-12 Thread Robert Haas
On Wed, Oct 4, 2023 at 8:07 AM Peter Geoghegan wrote: > If you're willing to take over as committer here, I'll let the issue > of backpatching go. > > I only ask that you note why you've not backpatched in the commit message. Will do, but see also the last point below. I have looked over these

Re: Eager page freeze criteria clarification

2023-10-12 Thread Melanie Plageman
On Wed, Oct 11, 2023 at 8:43 PM Andres Freund wrote: > > Robert, Melanie and I spent an evening discussing this topic around > pgconf.nyc. Here are, mildly revised, notes from that: Thanks for taking notes! > The main thing we are worried about is repeated freezing / unfreezing of > pages

Re: Performance degradation on concurrent COPY into a single relation in PG16.

2023-10-12 Thread Tom Lane
Andres Freund writes: >> On 2023-09-25 15:42:26 -0400, Tom Lane wrote: >>> I just did a git bisect run to discover when the failure documented >>> in bug #18130 [1] started. And the answer is commit 82a4edabd. > Uh, huh. The problem is that COPY uses a single BulkInsertState for multiple >

Re: logical decoding and replication of sequences, take 2

2023-10-12 Thread Tomas Vondra
On 7/25/23 12:20, Amit Kapila wrote: > ... > > I have used the debugger to reproduce this as it needs quite some > coordination. I just wanted to see if the sequence can go backward and > didn't catch up completely before the sequence state is marked > 'ready'. On the publisher side, I created a

Re: logical decoding and replication of sequences, take 2

2023-10-12 Thread Tomas Vondra
On 9/13/23 15:18, Ashutosh Bapat wrote: > On Fri, Aug 18, 2023 at 4:28 PM Amit Kapila wrote: >> >> On Fri, Aug 18, 2023 at 10:37 AM Amit Kapila wrote: >>> >>> On Thu, Aug 17, 2023 at 7:13 PM Ashutosh Bapat >>> wrote: On Wed, Aug 16, 2023 at 7:56 PM Tomas Vondra wrote: >

Re: logical decoding and replication of sequences, take 2

2023-10-12 Thread Tomas Vondra
On 9/20/23 11:53, Dilip Kumar wrote: > On Wed, Aug 16, 2023 at 7:57 PM Tomas Vondra > wrote: >> > > I was reading through 0001, I noticed this comment in > ReorderBufferSequenceIsTransactional() function > > + * To decide if a sequence change should be handled as transactional or > applied

Re: Lowering the default wal_blocksize to 4K

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 9:57 AM Ants Aasma wrote: > This reminds me that xlp_tli is not being used to its full potential right > now either. We only check that it's not going backwards, but there is at > least one not very hard to hit way to get postgres to silently replay on the > wrong

Re: PostgreSQL domains and NOT NULL constraint

2023-10-12 Thread Tom Lane
Erki Eessaar writes: > PostgreSQL's CREATE DOMAIN documentation (section Notes) describes a way how > one can add NULL's to a column that has a domain with the NOT NULL constraint. > https://www.postgresql.org/docs/current/sql-createdomain.html > To me it seems very strange and amounts to a bug

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-10-12 Thread David Steele
On 10/12/23 09:58, David Steele wrote: On Thu, Oct 12, 2023 at 12:25:34PM +1300, Thomas Munro wrote: I'm planning to push 0002 (retries in frontend programs, which is where this thread began) and 0004 (add missing locks to SQL functions), including back-patches as far as 12, in a day or so.

Re: Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread Tom Lane
Nikita Malakhov writes: > Please advise on the idea of preserving pg_proc oids during pg_upgrade, in > a way like relfilenodes, type id and so on. What are possible downsides of > such a solution? You have the burden of proof backwards. That would add a great deal of new mechanism, and you

Re: Eager page freeze criteria clarification

2023-10-12 Thread Robert Haas
Thanks for these notes. On Wed, Oct 11, 2023 at 8:43 PM Andres Freund wrote: > - We also discussed an idea by Robert to track the number of times we need to > dirty a page when unfreezing and to compare that to the number of pages > dirtied overall (IIRC), but I don't think we really came to

Re: The danger of deleting backup_label

2023-10-12 Thread David Steele
On 10/11/23 18:22, Michael Paquier wrote: On Tue, Oct 10, 2023 at 05:06:45PM -0400, David Steele wrote: That fails because there is a check to make sure the checkpoint is valid when pg_control is loaded. Another possibility is to use a special LSN like we use for unlogged tables. Anything >=

Pro et contra of preserving pg_proc oids during pg_upgrade

2023-10-12 Thread Nikita Malakhov
Hi hackers! Please advise on the idea of preserving pg_proc oids during pg_upgrade, in a way like relfilenodes, type id and so on. What are possible downsides of such a solution? Thanks! -- Regards, Nikita Malakhov Postgres Professional The Russian Postgres Company https://postgrespro.ru/

Re: The danger of deleting backup_label

2023-10-12 Thread David Steele
Hi Thomas, On 10/11/23 18:10, Thomas Munro wrote: Even though I spent a whole bunch of time trying to figure out how to make concurrent reads of the control file sufficiently atomic for backups (pg_basebackup and low level filesystem tools), and we explored multiple avenues with varying

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-10-12 Thread David Steele
On 10/11/23 21:10, Michael Paquier wrote: On Thu, Oct 12, 2023 at 12:25:34PM +1300, Thomas Munro wrote: I'm planning to push 0002 (retries in frontend programs, which is where this thread began) and 0004 (add missing locks to SQL functions), including back-patches as far as 12, in a day or

Re: Lowering the default wal_blocksize to 4K

2023-10-12 Thread Ants Aasma
On Thu, 12 Oct 2023 at 16:36, Robert Haas wrote: > On Wed, Oct 11, 2023 at 4:28 PM Thomas Munro > wrote: > > That leaves only the segments where a record starts exactly on the > > first usable byte of a segment, which is why I was trying to think of > > a way to cover that case too. I

Re: Problem, partition pruning for prepared statement with IS NULL clause.

2023-10-12 Thread Sergei Glukhov
On 10/12/23 16:27, David Rowley wrote: I've now also pushed the fix for the incorrect logic for nullkeys in ExecInitPruningContext(). Thanks! Regards, Gluh

Re: Lowering the default wal_blocksize to 4K

2023-10-12 Thread Robert Haas
On Wed, Oct 11, 2023 at 6:11 PM Andres Freund wrote: > I think the question is what the point of the crosschecks in long page headers > is. It's pretty easy to see what the point of the xlp_sysid check is - make it > less likely to accidentally replay WAL from a different system. It's much >

Re: [RFC] Add jit deform_counter

2023-10-12 Thread Daniel Gustafsson
> On 12 Oct 2023, at 15:37, Nazir Bilal Yavuz wrote: > > Hi, > > On Fri, 8 Sept 2023 at 20:22, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> >>> On Fri, Sep 08, 2023 at 03:34:42PM +0200, Daniel Gustafsson wrote: On 5 Sep 2023, at 16:37, Daniel Gustafsson wrote: >>> I've gone over

Re: [RFC] Add jit deform_counter

2023-10-12 Thread Nazir Bilal Yavuz
Hi, On Fri, 8 Sept 2023 at 20:22, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Fri, Sep 08, 2023 at 03:34:42PM +0200, Daniel Gustafsson wrote: > > > On 5 Sep 2023, at 16:37, Daniel Gustafsson wrote: > > > > > I've gone over this version of the patch and I think it's ready to go in. > >

Re: Lowering the default wal_blocksize to 4K

2023-10-12 Thread Robert Haas
On Wed, Oct 11, 2023 at 4:28 PM Thomas Munro wrote: > That leaves only the segments where a record starts exactly on the > first usable byte of a segment, which is why I was trying to think of > a way to cover that case too. I suggested we could notice and insert > a new record at that place.

PostgreSQL domains and NOT NULL constraint

2023-10-12 Thread Erki Eessaar
Hello PostgreSQL's CREATE DOMAIN documentation (section Notes) describes a way how one can add NULL's to a column that has a domain with the NOT NULL constraint. https://www.postgresql.org/docs/current/sql-createdomain.html To me it seems very strange and amounts to a bug because it defeats the

Re: RFC: Logging plan of the running query

2023-10-12 Thread torikoshia
On 2023-10-11 16:22, Ashutosh Bapat wrote: Like many others I think this feature is useful to debug a long running query. Sorry for jumping late into this. I have a few of high level comments Thanks for your comments! There is a lot of similarity between what this feature does and what

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2023-10-12 Thread Dilip Kumar
On Wed, Oct 11, 2023 at 5:57 PM Dilip Kumar wrote: > > On Wed, Oct 11, 2023 at 4:34 PM Dilip Kumar wrote: > In my last email, I forgot to give the link from where I have taken > the base path for dividing the buffer pool in banks so giving the same > here[1]. And looking at this again it seems

Re: Problem, partition pruning for prepared statement with IS NULL clause.

2023-10-12 Thread David Rowley
On Mon, 9 Oct 2023 at 12:26, David Rowley wrote: > > On Sat, 7 Oct 2023 at 03:11, Sergei Glukhov wrote: > > I noticed that combination of prepared statement with generic plan and > > 'IS NULL' clause could lead partition pruning to crash. > > > Test case: > > -- > > set plan_cache_mode to

Re: Special-case executor expression steps for common combinations

2023-10-12 Thread David Rowley
On Thu, 12 Oct 2023 at 22:54, Daniel Gustafsson wrote: > EEOP_FUNCEXPR_STRICT_* (10M iterations): > master : (7503.317, 7553.691, 7634.524) > patched : (7422.756, 7455.120, 7492.393) > > pgbench: > master : (3653.83, 3792.97, 3863.70) > patched : (3743.04, 3830.02, 3869.80) > >

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-12 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Thanks for reviewing! New patch is available at [1]. > > Some more comments: > 1. Let's restruture binary_upgrade_validate_wal_logical_end() a bit. > First, let's change its name to binary_upgrade_slot_has_pending_wal() > or something like that. Then move the context creation and

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-10-12 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Thanks for your suggestion! PSA new version. > The other problem is that pg_resetwal removes all pre-existing WAL > files which in this case could lead to the removal of the WAL file > corresponding to restart_lsn. This is because at least the shutdown > checkpoint record will be

Re: Removing unneeded self joins

2023-10-12 Thread Alexander Korotkov
On Thu, Oct 5, 2023 at 12:17 PM Andrei Lepikhov wrote: > On 4/10/2023 14:34, Alexander Korotkov wrote: > > > Relid replacement machinery is the most contradictory code here. We used > > > a utilitarian approach and implemented a simplistic variant. > > > > > > 2) It would be nice to skip the

pg_upgrade's interaction with pg_resetwal seems confusing

2023-10-12 Thread Amit Kapila
In pg_upgrade, we reset WAL archives (remove WAL), transaction id, etc. in copy_xact_xlog_xid() for the new cluster. Then, we create new objects in the new cluster, and again towards the end of the upgrade we invoke pg_resetwal with the -o option to reset the next OID. Now, along with resetting

Test 026_overwrite_contrecord fails on very slow machines (under Valgrind)

2023-10-12 Thread Alexander Lakhin
Hello hackers, While investigating the recent skink failure [1], I've reproduced this failure under Valgrind on a slow machine and found that this happens due to the last checkpoint recorded in the segment 2, that is removed in the test: The failure log contains: 2023-10-10 19:10:08.212 UTC

Re: Server crash on RHEL 9/s390x platform against PG16

2023-10-12 Thread Suraj Kharage
Here is clang version: [edb@9428da9d2137]$ clang --version clang version 15.0.7 (Red Hat 15.0.7-2.el9) Target: s390x-ibm-linux-gnu Thread model: posix InstalledDir: /usr/bin Let me know if any further information is needed. On Mon, Oct 9, 2023 at 8:21 AM Suraj Kharage wrote: > It looks

Re: [PATCH] Compression dictionaries for JSONB

2023-10-12 Thread Aleksander Alekseev
Hi hackers, I would like to continue discussing compression dictionaries. > So I summarized the requirements we agreed on so far and ended up with > the following list: [...] Again, here is the summary of our current agreements, at least how I understand them. Please feel free to correct me

Re: Special-case executor expression steps for common combinations

2023-10-12 Thread Heikki Linnakangas
On 12/10/2023 12:48, Daniel Gustafsson wrote: The attached patch adds special-case expression steps for common sets of steps in the executor to shave a few cycles off during execution, and make the JIT generated code simpler. * Adds EEOP_FUNCEXPR_STRICT_1 and EEOP_FUNCEXPR_STRICT_2 for function

Re: Use virtual tuple slot for Unique node

2023-10-12 Thread Ashutosh Bapat
On Tue, Oct 10, 2023 at 2:23 PM David Rowley wrote: > > On Wed, 27 Sept 2023 at 20:01, David Rowley wrote: > > > > On Sat, 23 Sept 2023 at 03:15, Heikki Linnakangas wrote: > > > So not a win in this case. Could you peek at the outer slot type, and > > > use the same kind of slot for the

Re: Logging parallel worker draught

2023-10-12 Thread Benoit Lobréau
On 10/11/23 17:26, Imseih (AWS), Sami wrote: Thank you for resurrecting this thread. Well, if you read Benoit's earlier proposal at [1] you'll see that he does propose to have some cumulative stats; this LOG line he proposes here is not a substitute for stats, but rather a complement. I don't

Re: cataloguing NOT NULL constraints

2023-10-12 Thread Alexander Lakhin
Hi Alvaro, 25.08.2023 14:38, Alvaro Herrera wrote: I have now pushed this again. Hopefully it'll stick this time. I've discovered that that commit added several recursive functions, and some of them are not protected from stack overflow. Namely, with "max_locks_per_transaction = 600" and

Special-case executor expression steps for common combinations

2023-10-12 Thread Daniel Gustafsson
The attached patch adds special-case expression steps for common sets of steps in the executor to shave a few cycles off during execution, and make the JIT generated code simpler. * Adds EEOP_FUNCEXPR_STRICT_1 and EEOP_FUNCEXPR_STRICT_2 for function calls of strict functions with 1 or 2

  1   2   >