Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread Kyotaro Horiguchi
At Fri, 17 Jun 2022 15:54:13 +0900 (JST), Kyotaro Horiguchi wrote in > > Or we could add a timeout.c API that specifies the timeout? > > I sometimes wanted this, But I don't see a simple way to sort multiple > relative timeouts in absolute time order. Maybe we can skip > GetCurrentTimestamp on

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread Kyotaro Horiguchi
At Thu, 16 Jun 2022 23:24:56 -0700, Andres Freund wrote in > The remaining difference looks like it's largely caused by the > enable_timeout_after(IDLE_STATS_UPDATE_TIMEOUT, ...) introduced as part of the > pgstats patch. It's only really visible when I pin a single connection pgbench > to the sa

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread Andres Freund
Hi, On 2022-06-16 22:22:28 -0700, Andres Freund wrote: > On 2022-06-17 16:53:31 +1200, David Rowley wrote: > > On Fri, 17 Jun 2022 at 15:33, Peter Geoghegan wrote: > > > Have you tried this with the insert benchmark [1]? > > > > I was mostly focusing on the performance of the hashed saop feature

RE: Replica Identity check of partition table on subscriber

2022-06-16 Thread shiy.f...@fujitsu.com
On Fri Jun 17, 2022 11:06 AM shiy.f...@fujitsu.com wrote: > > Attached the new version of patch set. I also moved the partitioned table > check > in logicalrep_rel_mark_updatable() to check_relation_updatable() as > discussed > [2]. > Attached back-branch patches of the first patch. Regards,

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread Andres Freund
Hi, On 2022-06-17 16:53:31 +1200, David Rowley wrote: > On Fri, 17 Jun 2022 at 15:33, Peter Geoghegan wrote: > > Have you tried this with the insert benchmark [1]? > > I was mostly focusing on the performance of the hashed saop feature > after having removed the additional fields that pushed Exp

Re: libpq: Remove redundant null pointer checks before free()

2022-06-16 Thread Tom Lane
Michael Paquier writes: > On Thu, Jun 16, 2022 at 10:07:33PM +0200, Peter Eisentraut wrote: >> calls, where the "if" part is unnecessary. This is of course pretty >> harmless, but some functions like scram_free() and freePGconn() have become >> so bulky that it becomes annoying. So while I was d

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread David Rowley
On Fri, 17 Jun 2022 at 15:33, Peter Geoghegan wrote: > Have you tried this with the insert benchmark [1]? I was mostly focusing on the performance of the hashed saop feature after having removed the additional fields that pushed ExprEvalStep over 64 bytes in 14. I agree it would be good to do fu

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread Peter Geoghegan
On Thu, Jun 16, 2022 at 7:15 PM David Rowley wrote: > So, there appears to be no performance regression due to the extra > indirection. There's maybe even some gains due to the smaller step > size. Have you tried this with the insert benchmark [1]? I've run it myself in the past (when working on

Re: libpq: Remove redundant null pointer checks before free()

2022-06-16 Thread Michael Paquier
On Thu, Jun 16, 2022 at 10:07:33PM +0200, Peter Eisentraut wrote: > calls, where the "if" part is unnecessary. This is of course pretty > harmless, but some functions like scram_free() and freePGconn() have become > so bulky that it becomes annoying. So while I was doing some work in that > area

RE: Replica Identity check of partition table on subscriber

2022-06-16 Thread shiy.f...@fujitsu.com
On Thu, Jun 16, 2022 2:13 PM Amit Langote wrote: > > Hi, > > On Thu, Jun 16, 2022 at 2:07 PM shiy.f...@fujitsu.com > wrote: > > On Wed, Jun 15, 2022 8:30 PM Amit Kapila > wrote: > > > I have pushed the first bug-fix patch today. > > > > Attached the remaining patches which are rebased. > > Th

Re: CREATE TABLE ( .. STORAGE ..)

2022-06-16 Thread Kyotaro Horiguchi
Thanks! I have been annoyed sometimes by the lack of this feature. At Thu, 16 Jun 2022 16:40:55 +0300, Aleksander Alekseev wrote in > Hi Matthias, > > > Apart from this comment on the format of the patch, the result seems solid. > > Many thanks. > > > When updating a patchset generally we tr

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread David Rowley
On Fri, 17 Jun 2022 at 11:31, Andres Freund wrote: > hashedscalararrayop/EEOP_HASHED_SCALARARRAYOP is 64 bytes, even though the > limit is 40 bytes. > commit 50e17ad281b8d1c1b410c9833955bc80fbad4078 > Author: David Rowley > Date: 2021-04-08 23:51:22 +1200 > > Speedup ScalarArrayOpExpr eval

Re: pg_upgrade (12->14) fails on aggregate

2022-06-16 Thread Justin Pryzby
On Wed, Jun 15, 2022 at 03:32:04PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > But Petr has a point - pg_upgrade should aspire to catch errors in --check, > > rather than starting and then leaving a mess behind for the user to clean up > > Agreed; pg_upgrade has historically tried to find

Re: fix stats_fetch_consistency value in postgresql.conf.sample

2022-06-16 Thread Kyotaro Horiguchi
At Thu, 16 Jun 2022 08:23:07 -0500, Justin Pryzby wrote in > On Thu, Jun 16, 2022 at 05:19:46PM +0900, Kyotaro Horiguchi wrote: > > At Sat, 11 Jun 2022 09:41:37 -0500, Justin Pryzby > > wrote in > > > Note that this gives: > > > > > > guc.c:7573:9: warning: ‘dst’ may be used uninitialized in

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread Andres Freund
Hi, On 2022-06-16 16:31:30 -0700, Andres Freund wrote: > The EEOP_JSONEXPR stuff was added during 15 development in: > > commit 1a36bc9dba8eae90963a586d37b6457b32b2fed4 > Author: Andrew Dunstan > Date: 2022-03-03 13:11:14 -0500 > > SQL/JSON query functions I'm quite confused about part of

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread Tom Lane
Andres Freund writes: > However, jsonexpr/EEOP_JSONEXPR is 296 bytes, and > hashedscalararrayop/EEOP_HASHED_SCALARARRAYOP is 64 bytes, even though the > limit is 40 bytes. Oops. > Maybe it's worth sticking a StaticAssert() for the struct size > somewhere. Indeed. I thought we had one already.

PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-16 Thread Andres Freund
Hi, Mark Callaghan reported a regression in 15, in the post linked here (with comments in the twitter thread, hence linked here) https://twitter.com/MarkCallaghanDB/status/1537475430227161098 A differential flame graph shows increased time spent doing memory allocations, below ExecInitExpr(): htt

Re: better page-level checksums

2022-06-16 Thread Bruce Momjian
On Tue, Jun 14, 2022 at 01:42:55PM -0400, Robert Haas wrote: > Hmm, but on the other hand, if you imagine a scenario in which the > "storage system extra blob" is actually a nonce for TDE, you need to > be able to find it before you've decrypted the rest of the page. If > pd_checksum gives you the

Re: real/float example for testlibpq3

2022-06-16 Thread Mark Wong
On Thu, Jun 16, 2022 at 03:41:50PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Tue, Jun 14, 2022 at 1:40 PM Mark Wong wrote: > >> I've created a function for each data type with the idea that an example > >> for handling a specific data type can be more easily reviewed by looking > >> in

Re: Nothing is using StrategyNotifyBgWriter() anymore

2022-06-16 Thread Nasby, Jim
Answering my own question... I now see that the wakeup does in fact happen in StrategyGetBuffer(). Sorry for the noise. On 6/16/22, 5:32 PM, "Jim Nasby" wrote: While browsing through some of the clock-sweep code I noticed that the only place StrategyNotifyBgWriter() is called now is i

Re: [PROPOSAL] Detecting plan changes with plan_id in pg_stat_activity

2022-06-16 Thread Imseih (AWS), Sami
>AFAICS you're proposing to add an identifier for a specific plan, but no > way to >know what that plan was? How are users supposed to use the information if > they >know something changed but don't know what changed exactly? I see this as a start to do more useful things with plans

Re: Typo in ro.po file?

2022-06-16 Thread Bruce Momjian
On Wed, Jun 15, 2022 at 09:29:02AM +0200, Peter Eisentraut wrote: > On 14.06.22 05:34, Peter Smith wrote: > > FWIW, I stumbled on this obscure possible typo (?) in > > src/pl/plperl/po/ro.po: > > > > ~~~ > > > > #: plperl.c:788 > > msgid "while parsing Perl initialization" > > msgstr "în timpul

libpq: Remove redundant null pointer checks before free()

2022-06-16 Thread Peter Eisentraut
libpq contains a lot of if (foo) free(foo); calls, where the "if" part is unnecessary. This is of course pretty harmless, but some functions like scram_free() and freePGconn() have become so bulky that it becomes annoying. So while I was doing some work in that area I undertook

Re: SGML doc file references

2022-06-16 Thread Peter Eisentraut
On 16.06.22 19:30, Josh Soref wrote: I'm reading the docs (I'm trying to figure out some replication things) and I was wondering why the file references [1] don't match the file names. I think it was never a goal to absolutely make them match all the time, so a lot of the differences might be

Re: real/float example for testlibpq3

2022-06-16 Thread Tom Lane
Robert Haas writes: > On Tue, Jun 14, 2022 at 1:40 PM Mark Wong wrote: >> I've created a function for each data type with the idea that an example >> for handling a specific data type can be more easily reviewed by looking >> in a single place. >> I've added examples for REAL, TIMESTAMP WITHOUT T

Re: real/float example for testlibpq3

2022-06-16 Thread Robert Haas
On Tue, Jun 14, 2022 at 1:40 PM Mark Wong wrote: > Checking in for quick feedback to see if this refactor makes sense. > > I've created a function for each data type with the idea that an example > for handling a specific data type can be more easily reviewed by looking > in a single place. > > I'

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-06-16 Thread Andrew Dunstan
On 2022-06-16 Th 00:56, Masahiko Sawada wrote: > > I've attached an updated version patch that changes the configure > script. I'm still studying how to support AVX2 on msvc build. Also, > added more regression tests. I think you would need to add '/arch:AVX2' to the compiler flags in MSBuildPr

Re: better page-level checksums

2022-06-16 Thread Robert Haas
On Wed, Jun 15, 2022 at 5:53 PM Peter Geoghegan wrote: > I think that it's worth doing the following exercise (humor me): Why > wouldn't it be okay to just encrypt the tuple space and the line > pointer array, leaving both the page header and page special area > unencrypted? What kind of user woul

SGML doc file references

2022-06-16 Thread Josh Soref
Hi, I'm reading the docs (I'm trying to figure out some replication things) and I was wondering why the file references [1] don't match the file names. Most of the inconsistent items are for `obsolete-*` where the filename is actually `appendix-obsolete-*`. But, oddly, afaict, they were introduced

Re: SLRUs in the main buffer pool, redux

2022-06-16 Thread Konstantin Knizhnik
On 28.05.2022 04:13, Thomas Munro wrote: On Fri, May 27, 2022 at 11:24 PM Thomas Munro wrote: Rebased, debugged and fleshed out a tiny bit more, but still with plenty of TODO notes and questions. I will talk about this idea at PGCon, so I figured it'd help to have a patch that actually appli

Re: Modest proposal to extend TableAM API for controlling cluster commands

2022-06-16 Thread Mark Dilger
> On Jun 16, 2022, at 12:27 AM, Andres Freund wrote: > >> I don't think I should have to do so. It's like saying, "I think I should >> have freedom of speech", and you say, "well, I'm not sure about that; tell >> me what you want to say, and I'll decide if I'm going to let you say it".' >> Th

Re: Skipping logical replication transactions on subscriber side

2022-06-16 Thread Robert Haas
On Thu, Jun 16, 2022 at 3:26 AM Masahiko Sawada wrote: > FWIW, looking at the manual, there might have > been a solution for AIX to specify -qalign=natural compiler option in > order to enforce the alignment of double to 8. Well if that can work it sure seems better. -- Robert Haas EDB: http://

Re: Modest proposal to extend TableAM API for controlling cluster commands

2022-06-16 Thread Mark Dilger
> On Jun 16, 2022, at 12:28 AM, David G. Johnston > wrote: > > But you are basically proposing a reworking of the existing system into one > that makes pretty much any SQL Command something that a TAM can treat as > being an optional request by the user; Yes, and I think I'm perfectly corr

Re: Tightening behaviour for non-immutable behaviour in immutable functions

2022-06-16 Thread Greg Stark
On Mon, 13 Jun 2022 at 16:50, Greg Stark wrote: > > For that matter the gotchas are a bit convoluted > > Perhaps we should automatically fix up the current search patch and > attach it to functions where necessary for users instead of just > whingeing at them So I reviewed my own

Re: CREATE TABLE ( .. STORAGE ..)

2022-06-16 Thread Aleksander Alekseev
Hi Matthias, > Apart from this comment on the format of the patch, the result seems solid. Many thanks. > When updating a patchset generally we try to keep the patches > self-contained, and update patches as opposed to adding incremental > patches to the set. My reasoning was to separate my cha

Re: fix stats_fetch_consistency value in postgresql.conf.sample

2022-06-16 Thread Justin Pryzby
On Thu, Jun 16, 2022 at 05:19:46PM +0900, Kyotaro Horiguchi wrote: > At Sat, 11 Jun 2022 09:41:37 -0500, Justin Pryzby > wrote in > > Note that this gives: > > > > guc.c:7573:9: warning: ‘dst’ may be used uninitialized in this function > > [-Wmaybe-uninitialized] > > Mmm. I don't have an idea

Re: Replica Identity check of partition table on subscriber

2022-06-16 Thread Amit Langote
On Thu, Jun 16, 2022 at 9:28 PM Amit Kapila wrote: > On Thu, Jun 16, 2022 at 5:24 PM Amit Langote wrote: > > On Thu, Jun 16, 2022 at 6:42 PM Amit Kapila wrote: > > > On Fri, Jun 10, 2022 at 2:26 PM Amit Langote > > > wrote: > > > > @@ -1735,6 +1735,13 @@ apply_handle_insert_internal(ApplyExecu

Re: Ignoring BRIN for HOT udpates seems broken

2022-06-16 Thread Tomas Vondra
On 6/6/22 09:28, Michael Paquier wrote: > On Mon, Jun 06, 2022 at 09:08:08AM +0200, Tomas Vondra wrote: >> Attached is a patch reverting both commits (5753d4ee32 and fe60b67250). >> This changes the IndexAmRoutine struct, so it's an ABI break. That's not >> great post-beta :-( In principle we might

Re: Replica Identity check of partition table on subscriber

2022-06-16 Thread Amit Kapila
On Thu, Jun 16, 2022 at 5:24 PM Amit Langote wrote: > > On Thu, Jun 16, 2022 at 6:42 PM Amit Kapila wrote: > > On Fri, Jun 10, 2022 at 2:26 PM Amit Langote > > wrote: > > > @@ -1735,6 +1735,13 @@ apply_handle_insert_internal(ApplyExecutionData > > > *edata, > > > static void > > > check_rela

Re: PGDOCS - Integer configuration parameters should say "(integer)"

2022-06-16 Thread Michael Paquier
On Thu, Jun 16, 2022 at 07:22:15PM +1000, Peter Smith wrote: > It looked like a small mistake to me; there are only 3 of (int), > versus 148 of (integer). Grepping around, that's correct. Will fix. -- Michael signature.asc Description: PGP signature

Re: Replica Identity check of partition table on subscriber

2022-06-16 Thread Amit Langote
On Thu, Jun 16, 2022 at 6:42 PM Amit Kapila wrote: > On Fri, Jun 10, 2022 at 2:26 PM Amit Langote wrote: > > @@ -1735,6 +1735,13 @@ apply_handle_insert_internal(ApplyExecutionData > > *edata, > > static void > > check_relation_updatable(LogicalRepRelMapEntry *rel) > > { > > + /* > > +*

Re: Handle infinite recursion in logical replication setup

2022-06-16 Thread vignesh C
On Wed, Jun 15, 2022 at 12:13 PM Peter Smith wrote: > > Here are some review comments for patch v20-0004. > > == > > 1. General > > I thought that it is better to refer to the > subscription/publications/table "on" the node, rather than "in" the > node. Most of the review comments below are re

Re: Handle infinite recursion in logical replication setup

2022-06-16 Thread vignesh C
On Wed, Jun 15, 2022 at 12:11 PM Peter Smith wrote: > > Here are some review comments for patch v20-0003. > > == > > 1. Commit message > > In case, table t1 has a unique key, it will lead to a unique key > violation and replication won't proceed. > > SUGGESTION > If table t1 has a unique key,

Re: Replica Identity check of partition table on subscriber

2022-06-16 Thread Amit Kapila
On Fri, Jun 10, 2022 at 2:26 PM Amit Langote wrote: > > @@ -1735,6 +1735,13 @@ apply_handle_insert_internal(ApplyExecutionData *edata, > static void > check_relation_updatable(LogicalRepRelMapEntry *rel) > { > + /* > +* If it is a partitioned table, we don't check it, we will check its >

PGDOCS - Integer configuration parameters should say "(integer)"

2022-06-16 Thread Peter Smith
FYI, I happened to notice in the PG docs there are a few integer configuration parameters that describe themselves as type "(int)" instead of "(integer)". It looked like a small mistake to me; there are only 3 of (int), versus 148 of (integer). ~~~ doc\src\sgml\auth-delay.sgml: 29 30:

Re: CREATE TABLE ( .. STORAGE ..)

2022-06-16 Thread Matthias van de Meent
On Wed, 15 Jun 2022 at 16:51, Aleksander Alekseev wrote: > > Hi hackers, > > I noticed that cfbot is not entirely happy with the patch, so I rebased it. > > > I see that COMPRESSION and STORAGE now are handled slightly > > differently in the grammar. Maybe we could standardize that a bit > > more;

Re: Prevent writes on large objects in read-only transactions

2022-06-16 Thread Michael Paquier
On Thu, Jun 16, 2022 at 03:42:06PM +0900, Yugo NAGATA wrote: > I am not sure if we should use PreventCommandIfReadOnly in lo_write() > because there are other public functions that write to catalogs but there > are not the similar restrictions in such functions. I think it is caller's > responsibil

Re: Replica Identity check of partition table on subscriber

2022-06-16 Thread Amit Kapila
On Thu, Jun 16, 2022 at 12:30 PM Amit Langote wrote: > > On Thu, Jun 16, 2022 at 3:45 PM Amit Kapila wrote: > > On Thu, Jun 16, 2022 at 11:43 AM Amit Langote > > wrote: > > > + * Don't throw any error here just mark the relation entry as not > > > updatable, > > > + * as replica identity is on

Re: fix stats_fetch_consistency value in postgresql.conf.sample

2022-06-16 Thread Kyotaro Horiguchi
At Thu, 16 Jun 2022 12:07:03 +0900, Michael Paquier wrote in > On Sat, Jun 11, 2022 at 09:41:37AM -0500, Justin Pryzby wrote: > > Note that this gives: > > > > guc.c:7573:9: warning: ‘dst’ may be used uninitialized in this function > > [-Wmaybe-uninitialized] > > > > with gcc version 9.2.1 20

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-06-16 Thread John Naylor
On Thu, Jun 16, 2022 at 11:57 AM Masahiko Sawada wrote: > I've attached an updated version patch that changes the configure > script. I'm still studying how to support AVX2 on msvc build. Also, > added more regression tests. Thanks for the update, I will take a closer look at the patch in the nea

Re: Modest proposal to extend TableAM API for controlling cluster commands

2022-06-16 Thread David G. Johnston
On Wed, Jun 15, 2022 at 11:23 PM Mark Dilger wrote: > > > On Jun 15, 2022, at 8:50 PM, David G. Johnston < > david.g.johns...@gmail.com> wrote: > > > > On Wed, Jun 15, 2022 at 8:18 PM Andres Freund > wrote: > > > If a simple callback like > > > relation_supports_cluster(Relation rel) is too simp

Re: Modest proposal to extend TableAM API for controlling cluster commands

2022-06-16 Thread Andres Freund
Hi, On 2022-06-15 22:23:36 -0700, Mark Dilger wrote: > I'm not entirely against you on that, but it makes me cringe that we impose > design decisions like that on any and all future TAMs. It seems better to > me to let the TAM author decide to emit an error, warning, notice, or > whatever, as the

Re: Skipping logical replication transactions on subscriber side

2022-06-16 Thread Masahiko Sawada
On Thu, Jun 16, 2022 at 2:27 AM Robert Haas wrote: > > On Tue, Jun 14, 2022 at 3:54 AM Masahiko Sawada wrote: > > > AFAICS, we could do that by: > > > > > > 1. De-supporting platforms that have this problem, or > > > 2. Introducing new typalign values, as Noah proposed back on April 2, or > > > 3

Re: Add header support to text format and matching feature

2022-06-16 Thread Peter Eisentraut
On 15.06.22 13:50, Daniel Verite wrote: The other errors in the list above are more about the format itself, with options that make sense only for one format. So the way we're supposed to understand ERRCODE_FEATURE_NOT_SUPPORTED in these other cases is that such format does not support such featu

Re: Replica Identity check of partition table on subscriber

2022-06-16 Thread Amit Langote
On Thu, Jun 16, 2022 at 3:45 PM Amit Kapila wrote: > On Thu, Jun 16, 2022 at 11:43 AM Amit Langote wrote: > > + * Don't throw any error here just mark the relation entry as not > > updatable, > > + * as replica identity is only for updates and deletes but inserts can be > > + * replicated even w