Re: Typo about subxip in comments

2022-11-14 Thread Amit Kapila
On Sun, Nov 13, 2022 at 8:32 PM Japin Li wrote: > > On Sat, 12 Nov 2022 at 12:12, Amit Kapila wrote: > > > > I don't know if that is an improvement. I think we should stick to > > your initial proposed change. > > Agreed. Let's focus on the initial proposed change. > Pushed. -- With Regards,

Re: Time delayed LR (WAS Re: logical replication restrictions)

2022-11-14 Thread Amit Kapila
On Mon, Nov 14, 2022 at 6:52 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Amit, > > > > > It seems to me Kuroda-San has proposed this change [1] to fix the test > > > > but it is not clear to me why such a change is required. Why can't > > > > CHECK_FOR_INTERRUPTS() after waiting, followed by the

Re: Avoid overhead open-close indexes (catalog updates)

2022-11-14 Thread Michael Paquier
On Tue, Nov 15, 2022 at 09:57:26AM +0900, Michael Paquier wrote: > Anyway, multi-inserts are going to be solution better than > CatalogTupleInsertWithInfo() in some cases, because we would just > generate one WAL record of N inserts rather than N records with one > INSERT each. Something that you

Re: PGDOCS - Logical replication GUCs - added some xrefs

2022-11-14 Thread Peter Smith
On Sun, Nov 13, 2022 at 11:47 AM vignesh C wrote: > > On Mon, 24 Oct 2022 at 13:15, Peter Smith wrote: > > > > Hi hackers. > > > > There is a docs Logical Replication section "31.10 Configuration > > Settings" [1] which describes some logical replication GUCs, and > > details on how they

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Dilip Kumar
On Mon, Nov 14, 2022 at 10:18 PM David G. Johnston wrote: >> Do you have a view on this point? >> > > NULL when overflowed seems like the opposite of the desired effect, calling > attention to the exceptional status. Make it a text column and write > "overflow" or "###" as appropriate.

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Dilip Kumar
On Tue, Nov 15, 2022 at 2:47 AM Andres Freund wrote: > > First, it's not good to have a cliff that you can't see coming - presumbly > you'd want to warn *before* you regularly reach PGPROC_MAX_CACHED_SUBXIDS > subxids, rather when the shit has hit the fan already. I agree with the point that it

RE: Assertion failure in SnapBuildInitialSnapshot()

2022-11-14 Thread Takamichi Osumi (Fujitsu)
Hi, On Tuesday, November 15, 2022 10:26 AM Andres Freund wrote: > On 2022-11-10 16:04:40 +0530, Amit Kapila wrote: > > I don't have any good ideas on how to proceed with this. Any thoughts > > on this would be helpful? > > One thing worth doing might be to convert the assertion path into an

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-14 Thread Nathan Bossart
On Mon, Nov 14, 2022 at 03:40:04PM -0800, Nathan Bossart wrote: > Thanks for taking a look! Here is a rebased version of the patch set. Oops, apparently object_aclcheck() cannot be used for pg_class. Here is another version that uses pg_class_aclcheck() instead. I'm not sure how I missed this

Re: Schema variables - new implementation for Postgres 15

2022-11-14 Thread Pavel Stehule
po 14. 11. 2022 v 8:00 odesílatel Sergey Shinderuk < s.shinde...@postgrespro.ru> napsal: > On 13.11.2022 20:59, Pavel Stehule wrote: > > fresh rebase > > Hello, > > Sorry, I haven't been following this thread, but I'd like to report a > memory management bug. I couldn't apply the latest patches,

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-14 Thread Masahiko Sawada
On Mon, Nov 14, 2022 at 10:00 PM John Naylor wrote: > > On Mon, Nov 14, 2022 at 3:44 PM Masahiko Sawada wrote: > > > > 0004 patch is a new patch supporting a pointer tagging of the node > > kind. Also, it introduces rt_node_ptr we discussed so that internal > > functions use it rather than

Re: Suppressing useless wakeups in walreceiver

2022-11-14 Thread Nathan Bossart
Another option might be to just force initial reply/feedback messages when streaming starts. The attached patch improves src/test/recovery test runtime just like the previous one I posted. On Mon, Nov 14, 2022 at 11:01:27AM -0800, Nathan Bossart wrote: > I wonder if we > ought to do something

Re: libpq compression (part 2)

2022-11-14 Thread Andrey Borodin
On Sat, Nov 12, 2022 at 8:04 PM Andrey Borodin wrote: > > While testing patch some more I observe unpleasant segfaults: > > #27 0x0b08fda2 in lz4_decompress (d_stream=0x18cf82a0, > src=0x7feae4fa505d, src_size=92, > (gdb) select-frame 27 > (gdb) info locals > ds = 0x18cf82a0 > decPtr =

Re: Assertion failure in SnapBuildInitialSnapshot()

2022-11-14 Thread Thomas Munro
On Tue, Nov 15, 2022 at 3:38 PM Andres Freund wrote: > On 2022-11-14 17:25:31 -0800, Andres Freund wrote: > Another theory: I dimly remember Thomas mentioning that there's some different > behaviour of xlogreader during shutdown as part of the v15 changes. I don't > quite remember what the

Re: Direct I/O

2022-11-14 Thread Andres Freund
Hi, On 2022-11-10 14:26:20 +0700, John Naylor wrote: > On Tue, Nov 1, 2022 at 2:37 PM Thomas Munro wrote: > > > Memory alignment patches: > > > > Direct I/O generally needs to be done to/from VM page-aligned > > addresses, but only "standard" 4KB pages, even when larger VM pages > > are in use

Re: Assertion failure in SnapBuildInitialSnapshot()

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 17:25:31 -0800, Andres Freund wrote: > Hm, also, shouldn't the patch adding CRS_USE_SNAPSHOT have copied more of > SnapBuildExportSnapshot()? Why aren't the error checks for > SnapBuildExportSnapshot() needed? Why don't we need to set XactReadOnly? Which > transactions are we

Re: List of Bitmapset (was Re: ExecRTCheckPerms() and many prunable partitions)

2022-11-14 Thread Amit Langote
On Mon, Nov 14, 2022 at 11:57 PM Tom Lane wrote: > Amit Langote writes: > > On Sat, Nov 12, 2022 at 1:46 AM Tom Lane wrote: > >> The main thing I was wondering about in connection with that > >> was whether to assume that there could be other future applications > >> of the logic to perform

Re: [BUG] Logical replica crash if there was an error in a function.

2022-11-14 Thread Tom Lane
"Anton A. Melnikov" writes: > On 02.11.2022 21:02, Tom Lane wrote: >> I don't think the cost of that test case is justified by the tiny >> probability that it'd ever catch anything. > Seems it is possible to do a test without these remarks. The attached > test uses existing nodes and checks the

Re: [BUG] Logical replica crash if there was an error in a function.

2022-11-14 Thread Anton A. Melnikov
Hello! On 02.11.2022 21:02, Tom Lane wrote: So I'm now good with the idea of just not failing. I don't like the patch as presented though. First, the cfbot is quite rightly complaining about the "uninitialized variable" warning it draws. Second, I don't see a good reason to tie the change to

Re: Assertion failure in SnapBuildInitialSnapshot()

2022-11-14 Thread Andres Freund
Hi, On 2022-11-10 16:04:40 +0530, Amit Kapila wrote: > I don't have any good ideas on how to proceed with this. Any thoughts > on this would be helpful? One thing worth doing might be to convert the assertion path into an elog(), mentioning both xids (or add a framework for things like

Re: A new strategy for pull-up correlated ANY_SUBLINK

2022-11-14 Thread Richard Guo
On Sun, Nov 13, 2022 at 6:45 AM Tom Lane wrote: > Looking again at that contain_vars_of_level restriction, I think the > reason for it was just to avoid making a FROM subquery that has outer > references, and the reason we needed to avoid that was merely that we > didn't have LATERAL at the

Re: Avoid overhead open-close indexes (catalog updates)

2022-11-14 Thread Michael Paquier
On Sat, Nov 12, 2022 at 11:03:46AM -0300, Ranier Vilela wrote: > I think complexity doesn't pay off. > For example, CopyStatistics not knowing how many tuples will be processed. > IMHO, this step is right now. > CatalogTupleInsertWithInfo offers considerable improvement without > introducing bugs

Re: Collation version tracking for macOS

2022-11-14 Thread Jeff Davis
I looked at v6. * We'll need some clearer instructions on how to build/install extra ICU versions that might not be provided by the distribution packaging. For instance, I got a cryptic error until I used --enable-rpath, which might not be obvious to all users. * Can we have a better error

Re: Meson doesn't define HAVE_LOCALE_T for mscv

2022-11-14 Thread Andres Freund
Hi, On 2022-11-10 10:59:41 +0100, Juan José Santamaría Flecha wrote: > Meson doesn't see the redefinition of locale_t done > in src/include/port/win32_port.h, so is not defining > HAVE_LOCALE_T, HAVE_WCSTOMBS_L nor HAVE_MBSTOWCS_L as the > current src/tools/msvc/build.pl script does. > > Please

Re: Multitable insert syntax support on Postgres?

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 21:06:09 -0300, Alexandre hadjinlian guerra wrote: > Hello > Are there any plans to incorporate a formal syntax multitable/conditional > insert , similar to the syntax below? snowflake does have the same feature > > https://oracle-base.com/articles/9i/multitable-inserts > >

Re: Error on missing Python module in Meson setup

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 14:23:02 +0100, Daniel Gustafsson wrote: > When setting up a postgres tree with Meson on an almost empty Debian 11 VM I > hit an error on "meson setup -Ddebug=true build ." like this: > > Program python3 found: YES (/usr/bin/python3) > meson.build:987:2: ERROR: Unknown

Multitable insert syntax support on Postgres?

2022-11-14 Thread Alexandre hadjinlian guerra
Hello Are there any plans to incorporate a formal syntax multitable/conditional insert , similar to the syntax below? snowflake does have the same feature https://oracle-base.com/articles/9i/multitable-inserts Today, im resorting to a function that receives the necessary parameters from the

Re: pg_basebackup's --gzip switch misbehaves

2022-11-14 Thread Michael Paquier
On Mon, Nov 14, 2022 at 03:27:14PM +0100, Daniel Gustafsson wrote: > Ugh, yes, that's what it should say. A split sounds fine by me. On top of what Tom has mentioned, I have spotted two small-ish things. -This module is available from CPAN or an operating system package. +This module is

Re: Support logical replication of DDLs

2022-11-14 Thread Peter Smith
Here are some review comments for v32-0002 == 1. Commit message Comment says: While creating a publication, we register a command end trigger that deparses the DDL as a JSON blob, and WAL logs it. The event trigger is automatically removed at the time of drop publication. SUGGESTION

Re: [RFC] building postgres with meson - v12

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 17:16:46 -0600, Justin Pryzby wrote: > On Sun, Aug 28, 2022 at 01:37:41PM -0700, Andres Freund wrote: > > > You're running tap tests via a python script. There's no problem with > > > that, but it's different from what's done by the existing makefiles. > > > I was able to

Re: meson oddities

2022-11-14 Thread Andres Freund
Hi, On 2022-11-15 08:22:59 +0900, Michael Paquier wrote: > I pass down some custom CFLAGS to the meson command as well, and these find > their way to LDFLAGS on top of CFLAGS for the user-defined entries. I would > not have expected that, either. We effectively do that with autoconf as well,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-14 Thread Michael Paquier
On Mon, Nov 14, 2022 at 03:47:27PM +0800, Julien Rouhaud wrote: > On Mon, Nov 14, 2022 at 02:40:37PM +0900, Michael Paquier wrote: >> If the caller passes NULL for *linectx as the initial line context, >> just create it as we do now. If *linectx is not NULL, just reuse it. >> That may be cleaner

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-14 Thread Nathan Bossart
On Fri, Oct 14, 2022 at 07:37:38PM -0400, Corey Huinker wrote: > Patch applies. > Passes make check and make check-world. > Test coverage seems adequate. > > Coding is very clear and very much in the style of the existing code. Any > quibbles I have with the coding style are ones I have with the

Re: Suppressing useless wakeups in walreceiver

2022-11-14 Thread Thomas Munro
On Tue, Nov 15, 2022 at 12:14 PM Nathan Bossart wrote: > On Tue, Nov 15, 2022 at 09:42:26AM +1300, Thomas Munro wrote: > > That works for 020_pg_receivewal. I wonder if there are also tests > > that stream a bit of WAL first and then do wait_for_catchup that were > > previously benefiting from

Re: meson oddities

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 17:41:54 -0500, Andrew Dunstan wrote: > Here's a couple of things I've noticed. > > > andrew@ub22:HEAD $ inst.meson/bin/pg_config --libdir --ldflags > /home/andrew/pgl/pg_head/root/HEAD/inst.meson/lib/x86_64-linux-gnu > -fuse-ld=lld -DCOPY_PARSE_PLAN_TREES

Re: meson oddities

2022-11-14 Thread Michael Paquier
On Mon, Nov 14, 2022 at 05:41:54PM -0500, Andrew Dunstan wrote: > Also, why have the CPPFLAGS made their way into the LDFLAGS? That seems > wrong. Not only CPPFLAGS. I pass down some custom CFLAGS to the meson command as well, and these find their way to LDFLAGS on top of CFLAGS for the

Re: [RFC] building postgres with meson - v12

2022-11-14 Thread Justin Pryzby
On Sun, Aug 28, 2022 at 01:37:41PM -0700, Andres Freund wrote: > > You're running tap tests via a python script. There's no problem with > > that, but it's different from what's done by the existing makefiles. > > I was able to remove the python indirection - maybe that's better to > > talk about

Re: Suppressing useless wakeups in walreceiver

2022-11-14 Thread Nathan Bossart
On Tue, Nov 15, 2022 at 09:42:26AM +1300, Thomas Munro wrote: > That works for 020_pg_receivewal. I wonder if there are also tests > that stream a bit of WAL first and then do wait_for_catchup that were > previously benefiting from the 100ms-after-startup message by > scheduling luck (as in, that

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Peter Geoghegan
On Mon, Nov 14, 2022 at 2:58 PM Andres Freund wrote: > On 2022-11-14 14:42:16 -0800, Peter Geoghegan wrote: > > What does this have to tell us, if anything, about the implications > > for code on HEAD? > > Nothing really test I sent (*) - I wanted to advance the discussion about the > patch being

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 14:42:16 -0800, Peter Geoghegan wrote: > What does this have to tell us, if anything, about the implications > for code on HEAD? Nothing really test I sent (*) - I wanted to advance the discussion about the patch being wrong as-is in a concrete way. This logic was one of my

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Peter Geoghegan
On Mon, Nov 14, 2022 at 2:33 PM Andres Freund wrote: > > Why don't you think that there is corruption? > > I looked at the state after the test and the complaint is bogus. It's caused > by the patch ignoring the cur->xmax == next->xmin condition if next->xmin is > FrozenTransactionId. The

meson oddities

2022-11-14 Thread Andrew Dunstan
Here's a couple of things I've noticed. andrew@ub22:HEAD $ inst.meson/bin/pg_config --libdir --ldflags /home/andrew/pgl/pg_head/root/HEAD/inst.meson/lib/x86_64-linux-gnu -fuse-ld=lld -DCOPY_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST -DWRITE_READ_PARSE_PLAN_TREES Are we really intending

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 14:13:10 -0800, Peter Geoghegan wrote: > > I think the problem partially is that the proposed verify_heapam() code is > > too > > "aggressive" considering things to be part of the same hot chain - which > > then > > means we have to be very careful about erroring out. > > > >

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Peter Geoghegan
On Mon, Nov 14, 2022 at 12:58 PM Andres Freund wrote: > I wonder if we ought to add an error check to heap_prepare_freeze_tuple() > against this scenario. We're working towards being more aggressive around > freezing, which will make it more likely to hit corner cases around this. In theory my

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-14 Thread Andres Freund
Hi, On 2022-11-08 14:57:35 +1300, David Rowley wrote: > On Tue, 8 Nov 2022 at 05:24, Andres Freund wrote: > > Should we handle the case where we get a suitably aligned pointer from > > MemoryContextAllocExtended() differently? > > Maybe it would be worth the extra check. I'm trying to imagine

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 14:27:54 -0500, Robert Haas wrote: > On Wed, Nov 9, 2022 at 5:08 PM Andres Freund wrote: > > I don't really understand this logic - why can't we populate the predecessor > > array, if we can construct a successor entry? > > This whole thing was my idea, so let me try to

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Peter Geoghegan
On Mon, Nov 14, 2022 at 11:28 AM Robert Haas wrote: > Part of the motivation here is also driven by trying to figure out how > to word the complaints. We have a dedicated field in the amcheck that > can hold one tuple offset or the other, but if we're checking the > relationships between tuples,

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 13:43:41 -0500, Robert Haas wrote: > On Mon, Nov 14, 2022 at 12:47 PM Andres Freund wrote: > > I'd go the other way. It's pretty unimportant whether it overflowed, it's > > important how many subtxns there are. The cases where overflowing causes > > real > > problems are when

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 14, 2022 at 12:47 PM Andres Freund wrote: >> I'd go the other way. It's pretty unimportant whether it overflowed, it's >> important how many subtxns there are. The cases where overflowing causes real >> problems are when there's many thousand subtxns - which one

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 09:50:48 -0800, Peter Geoghegan wrote: > On Mon, Nov 14, 2022 at 9:38 AM Andres Freund wrote: > > Anyway, I played a bit around with this. It's hard to hit, not because we > > somehow won't choose such a horizon, but because we'll commonly prune the > > earlier tuple version

Re: Suppressing useless wakeups in walreceiver

2022-11-14 Thread Thomas Munro
On Tue, Nov 15, 2022 at 8:01 AM Nathan Bossart wrote: > Is there any reason we should wait for 100ms before sending the initial > reply? ISTM the previous behavior essentially caused the first reply (and > feedback message) to be sent at the first opportunity after streaming > begins. My

Change error to warning and increase thresholds of tsearch

2022-11-14 Thread Ankit Kumar Pandey
Hi, I was looking at Todo item:/Consider changing error to warning for strings larger than one megabyte/ and after going through existing mails and suggestions. I would like to propose a patch for tsearch to change error into warning for string larger than one mb and also increase word and

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-14 Thread David Christensen
Enclosed is v8, which uses the replication slot method to retain WAL as well as fsync'ing the output directory when everything is done. v8-0001-Teach-pg_waldump-to-extract-FPIs-from-the-WAL-str.patch Description: Binary data

Re: Allow single table VACUUM in transaction block

2022-11-14 Thread Simon Riggs
On Tue, 8 Nov 2022 at 03:10, Simon Riggs wrote: > > On Mon, 7 Nov 2022 at 08:20, Simon Riggs wrote: > > > Temp tables are actually easier, since we don't need any of the > > concurrency features we get with lazy vacuum. > Thoughts? New patch, which does this, when in a xact block 1. For temp

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Robert Haas
On Mon, Nov 14, 2022 at 2:17 PM David G. Johnston wrote: > Assuming getting an actual count value to print is fairly cheap, or even a > sunk cost if you are going to report overflow, I don't see why we wouldn't > want to provide the more detailed data. > > My concern, through ignorance, with

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Robert Haas
On Wed, Nov 9, 2022 at 5:08 PM Andres Freund wrote: > I don't really understand this logic - why can't we populate the predecessor > array, if we can construct a successor entry? This whole thing was my idea, so let me try to explain. I think the naming and comments need work, but I believe the

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread David G. Johnston
On Mon, Nov 14, 2022 at 11:43 AM Robert Haas wrote: > On Mon, Nov 14, 2022 at 12:47 PM Andres Freund wrote: > > I'd go the other way. It's pretty unimportant whether it overflowed, it's > > important how many subtxns there are. The cases where overflowing causes > real > > problems are when

Re: Suppressing useless wakeups in walreceiver

2022-11-14 Thread Nathan Bossart
On Mon, Nov 14, 2022 at 02:47:14PM +1300, Thomas Munro wrote: > Ahh, I think I might have it. In the old coding, sendTime starts out > as 0, which I think caused it to send its first reply message after > the first 100ms sleep, and only after that fall into a cadence of >

Re: [PoC] Let libpq reject unexpected authentication requests

2022-11-14 Thread Jacob Champion
On 11/11/22 22:57, Aleksander Alekseev wrote: > I did a little more research and I think you are right. What happens > according to the C standard: Thanks for confirming! (I personally prefer -1 to a *MAX macro, because it works regardless of the length of the type.) --Jacob

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Robert Haas
On Mon, Nov 14, 2022 at 12:47 PM Andres Freund wrote: > I'd go the other way. It's pretty unimportant whether it overflowed, it's > important how many subtxns there are. The cases where overflowing causes real > problems are when there's many thousand subtxns - which one can't judge just > from

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-11-14 Thread David Christensen
On Fri, Nov 11, 2022 at 4:57 AM Bharath Rupireddy wrote: > > Well if it's not the same output then I guess you're right and there's > > not a use for the `--fixup` mode. By the same token, I'd say > > calculating/setting the checksum also wouldn't need to be done, we > > should just include the

Re: Documentation for building with meson

2022-11-14 Thread samay sharma
Hi, On Thu, Nov 10, 2022 at 4:46 AM Nazir Bilal Yavuz wrote: > Hi, > > I did some tests on windows. I used 'ninja' as a backend. > On 11/8/2022 9:23 PM, samay sharma wrote: > > Hi, > > On Sat, Nov 5, 2022 at 2:39 PM Andres Freund wrote: > >> Hi, >> >> On 2022-10-30 20:51:59 -0700, samay sharma

Re: libpq support for NegotiateProtocolVersion

2022-11-14 Thread Jacob Champion
On 11/13/22 01:21, Peter Eisentraut wrote: > On 11.11.22 23:28, Jacob Champion wrote: >> Put another way, why do we loop around and poll for more data when we >> hit the end of the connection buffer, if we've already checked at this >> point that we should have the entire message buffered locally?

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-14 Thread Aleksander Alekseev
Hi hackers, > Thanks, done! Dilip Kumar asked a good question in the thread about the 0001..0003 subset [1]. I would like to duplicate it here to make sure it was not missed by mistake: """ Have we measured the WAL overhead because of this patch set? maybe these particular patches will not

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Peter Geoghegan
On Mon, Nov 14, 2022 at 9:38 AM Andres Freund wrote: > Anyway, I played a bit around with this. It's hard to hit, not because we > somehow won't choose such a horizon, but because we'll commonly prune the > earlier tuple version away due to xmax being old enough. That must be a bug, then. Since,

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

2022-11-14 Thread Andres Freund
On 2022-11-14 16:06:27 +0530, Amit Kapila wrote: > Pushed. Thanks.

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Andres Freund
Hi, On 2022-11-14 12:29:58 -0500, Tom Lane wrote: > I'd vote for just overflowed true/false. Why do people need to know > the exact number of subtransactions? (If there is a use-case, that > would definitely be material for an auxiliary function instead of a > view column.) I'd go the other

Re: HOT chain validation in verify_heapam()

2022-11-14 Thread Andres Freund
Hi, On 2022-11-09 18:35:12 -0800, Peter Geoghegan wrote: > On Wed, Nov 9, 2022 at 6:10 PM Andres Freund wrote: > > And thinking about it, it'd be quite bad if the horizon worked that way. > > You can easily construct a workload where every single xid would "skewer" > > some chain, never

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Tom Lane
"David G. Johnston" writes: > On Mon, Nov 14, 2022 at 9:41 AM Robert Haas wrote: >> The overhead of fetching the information is not large, but Justin is >> concerned about the effect on the display width. I feel that's kind of >> a lost cause because it's so wide already anyway, but I don't see

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread David G. Johnston
On Mon, Nov 14, 2022 at 9:41 AM Robert Haas wrote: > On Mon, Nov 14, 2022 at 11:35 AM Amit Singh > wrote: > > Making the information available in pg_stat_activity makes it a lot > easier to identify the pid which has caused the subtran overflow. Debugging > through the app code can be an

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Robert Haas
On Mon, Nov 14, 2022 at 11:35 AM Amit Singh wrote: > Making the information available in pg_stat_activity makes it a lot easier to > identify the pid which has caused the subtran overflow. Debugging through the > app code can be an endless exercise and logging every statement in postgresql >

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Amit Singh
Making the information available in pg_stat_activity makes it a lot easier to identify the pid which has caused the subtran overflow. Debugging through the app code can be an endless exercise and logging every statement in postgresql logs is not practical either. If the overhead of fetching the

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Robert Haas
On Mon, Nov 14, 2022 at 11:18 AM David G. Johnston wrote: >> I guess that's OK. I don't particularly favor that approach here but I >> can live with it. I agree that too-wide views are annoying, but as far >> as pg_stat_activity goes, that ship has pretty much sailed already, >> and the same is

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread David G. Johnston
On Mon, Nov 14, 2022 at 9:04 AM Robert Haas wrote: > On Mon, Nov 14, 2022 at 10:57 AM Justin Pryzby > wrote: > > > First, we're just talking about an extra couple of columns in > > > pg_stat_activity here, which does not seem like a heavy price to pay. > > > > The most recent patch adds a

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Robert Haas
On Mon, Nov 14, 2022 at 10:57 AM Justin Pryzby wrote: > > First, we're just talking about an extra couple of columns in > > pg_stat_activity here, which does not seem like a heavy price to pay. > > The most recent patch adds a separate function rather than adding more > columns to

Re: when the startup process doesn't (logging startup delays)

2022-11-14 Thread Robert Haas
On Mon, Nov 14, 2022 at 7:37 AM Simon Riggs wrote: > > Whilte at it, I noticed that we report redo progress for PITR, but we > > don't report when standby enters archive recovery mode, say due to a > > failure in the connection to primary or after the promote signal is > > found. Isn't it useful

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Justin Pryzby
On Mon, Nov 14, 2022 at 10:09:57AM -0500, Robert Haas wrote: > On Mon, Mar 21, 2022 at 7:45 PM Andres Freund wrote: > > > It feels to me like far too much effort is being invested in fundamentally > > > the wrong direction here. If the subxact overflow business is causing > > > real-world

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Robert Haas
On Mon, Nov 14, 2022 at 10:41 AM Tom Lane wrote: > Maybe the original patch took an hour to write, but it's sure been > bikeshedded to death :-(. I was complaining about the total amount > of attention spent more than the patch itself. Unfortunately, that problem is not unique to this patch,

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Tom Lane
Robert Haas writes: > In short, I think this is a good idea, and if somebody thinks that we > should solve the underlying problem instead, I'd like to hear what > people think a realistic solution might be. Because to me, it looks > like we're refusing to commit a patch that probably took an hour

Re: Small TAP improvements

2022-11-14 Thread Andrew Dunstan
On 2022-11-09 We 05:35, Andrew Dunstan wrote: > On 2022-11-06 Su 08:51, Álvaro Herrera wrote: >> On 2022-Jun-14, Andrew Dunstan wrote: >> >>> OK, here's a more principled couple of patches. For config_data, if you >>> give multiple options it gives you back the list of values. If you don't >>>

Re: Add sub-transaction overflow status in pg_stat_activity

2022-11-14 Thread Robert Haas
On Mon, Mar 21, 2022 at 7:45 PM Andres Freund wrote: > > It feels to me like far too much effort is being invested in fundamentally > > the wrong direction here. If the subxact overflow business is causing > > real-world performance problems, let's find a way to fix that, not put > > effort into

List of Bitmapset (was Re: ExecRTCheckPerms() and many prunable partitions)

2022-11-14 Thread Tom Lane
[ I'm intentionally forking this off as a new thread, so as to not confuse the cfbot about what's the live patchset on the ExecRTCheckPerms thread. ] Amit Langote writes: > On Sat, Nov 12, 2022 at 1:46 AM Tom Lane wrote: >> The main thing I was wondering about in connection with that >> was

Re: pg_basebackup's --gzip switch misbehaves

2022-11-14 Thread Daniel Gustafsson
> On 14 Nov 2022, at 15:23, Tom Lane wrote: > > Daniel Gustafsson writes: >> How about this version? > > This isn't correct shell syntax is it? > > +PG_TEST_NOCLEAN make -C src/bin/pg_dump check > > I think you meant > > +PG_TEST_NOCLEAN=1 make -C src/bin/pg_dump check > > or the like.

Re: pg_basebackup's --gzip switch misbehaves

2022-11-14 Thread Tom Lane
Daniel Gustafsson writes: > How about this version? This isn't correct shell syntax is it? +PG_TEST_NOCLEAN make -C src/bin/pg_dump check I think you meant +PG_TEST_NOCLEAN=1 make -C src/bin/pg_dump check or the like. regards, tom lane

Re: psql: Add command to use extended query protocol

2022-11-14 Thread Daniel Verite
Peter Eisentraut wrote: > > I assume that we may sometimes want to use the > > extended protocol on all queries of a script, like > > pgbench does with --protocol=extended. > > But is there an actual use case for this in psql? In pgbench, there are > scenarios where you want to test

Re: Commit fest 2022-11

2022-11-14 Thread Ian Lawrence Barwick
2022年11月14日(月) 22:23 James Coleman : > > On Mon, Nov 14, 2022 at 7:08 AM Ian Lawrence Barwick > wrote: > > > > 2022年11月9日(水) 8:12 Justin Pryzby : > > > > If my script is not wrong, these patches add TAP tests, but don't update > > > the requisite ./meson.build file. It seems like it'd be

Re: Commit fest 2022-11

2022-11-14 Thread James Coleman
On Mon, Nov 14, 2022 at 7:08 AM Ian Lawrence Barwick wrote: > > 2022年11月9日(水) 8:12 Justin Pryzby : > > If my script is not wrong, these patches add TAP tests, but don't update > > the requisite ./meson.build file. It seems like it'd be reasonable to > > set them all as WOA until that's

Error on missing Python module in Meson setup

2022-11-14 Thread Daniel Gustafsson
When setting up a postgres tree with Meson on an almost empty Debian 11 VM I hit an error on "meson setup -Ddebug=true build ." like this: Program python3 found: YES (/usr/bin/python3) meson.build:987:2: ERROR: Unknown method "dependency" in object. The error in itself isn't terribly

RE: Time delayed LR (WAS Re: logical replication restrictions)

2022-11-14 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > > > It seems to me Kuroda-San has proposed this change [1] to fix the test > > > but it is not clear to me why such a change is required. Why can't > > > CHECK_FOR_INTERRUPTS() after waiting, followed by the existing below > > > code [2] in LogicalRepApplyLoop() sufficient to handle

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-14 Thread John Naylor
On Mon, Nov 14, 2022 at 3:44 PM Masahiko Sawada wrote: > > 0004 patch is a new patch supporting a pointer tagging of the node > kind. Also, it introduces rt_node_ptr we discussed so that internal > functions use it rather than having two arguments for encoded and > decoded pointers. With this

Re: when the startup process doesn't (logging startup delays)

2022-11-14 Thread Simon Riggs
On Tue, 8 Nov 2022 at 12:33, Bharath Rupireddy wrote: > > On Tue, Nov 8, 2022 at 4:35 PM Thomas Munro wrote: > > > > On Sat, Oct 30, 2021 at 7:44 AM Robert Haas wrote: > > > Committed. > > > > Is it expected that an otherwise idle standby's recovery process > > receives SIGALRM every N seconds,

Re: pg_basebackup's --gzip switch misbehaves

2022-11-14 Thread Daniel Gustafsson
> On 3 Nov 2022, at 12:49, Michael Paquier wrote: > > On Wed, Nov 02, 2022 at 09:42:12PM +0100, Daniel Gustafsson wrote: >> I think that's a good idea, not everyone running tests will read the >> internals >> documentation (or even know abou it even). How about the attached? > > Thanks for

Re: Commit fest 2022-11

2022-11-14 Thread Ian Lawrence Barwick
2022年11月9日(水) 8:12 Justin Pryzby : > > On Thu, Nov 03, 2022 at 07:43:03PM -0500, Justin Pryzby wrote: > > On Thu, Nov 03, 2022 at 11:33:57AM +0900, Ian Lawrence Barwick wrote: > > > 2022年11月2日(水) 19:10 Greg Stark : > > > > On Tue, 1 Nov 2022 at 06:56, Michael Paquier > > > > wrote: > > > > > > >

Re: Allow logical replication to copy tables in binary format

2022-11-14 Thread Melih Mutlu
Hello, osumi.takami...@fujitsu.com , 12 Eki 2022 Çar, 04:36 tarihinde şunu yazdı: > > I agree with the direction to support binary copy for v16 and > later. > > > > IIUC, the binary format replication with different data types > fails even > > during apply phase on HEAD. > > I

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-14 Thread Thom Brown
On Thu, 3 Nov 2022 at 08:12, Maxim Orlov wrote: > > Hi! > >> I attach an additional V48-0009 patch as they are just comments, apply it if >> you want to. > > Big thank you for your review. I've applied your addition in the recent patch > set below. > > Besides, mentioned above, next changes are

Re: Unit tests for SLRU

2022-11-14 Thread Michael Paquier
On Fri, Nov 11, 2022 at 02:11:08PM +0900, Michael Paquier wrote: > Is there a reason why you need a TAP test here? It is by design more > expensive than pg_regress and it does not require --enable-tap-tests. > See for example what we do for snapshot_too_old, commit_ts, > worker_spi, etc., where

Re: Privileges on PUBLICATION

2022-11-14 Thread Antonin Houska
Peter Eisentraut wrote: > On 04.11.22 08:28, Antonin Houska wrote: > > I thought about the whole concept a bit more and I doubt if the PUBLICATION > > privilege is the best approach. In particular, the user specified in CREATE > > SUBSCRIPTION ... CONNECTION ... (say "subscription user") needs

Re: Fix order of checking ICU options in initdb and create database

2022-11-14 Thread Marina Polyakova
On 2022-11-12 22:43, Jose Arthur Benetasso Villanova wrote: Hello Marina. I just reviewed your patch. It applied cleanly at my current master (commit d6a3dbe14f98d867b2fc3faeb99d2d3c2a48ca67). Also, it worked as described in email. Since it's a clarification in an error message, I think the

Re: Split index and table statistics into different types of stats

2022-11-14 Thread Drouvot, Bertrand
Hi, On 11/4/22 9:51 PM, Melanie Plageman wrote: Hi Bertrand, I'm glad you are working on this. I had a few thoughts/ideas Thanks for looking at it! It seems better to have all of the counts in the various stats structs not be prefixed with n_, i_, t_ typedef struct PgStat_StatDBEntry {

Re: hash_xlog_split_allocate_page: failed to acquire cleanup lock

2022-11-14 Thread Amit Kapila
On Tue, Nov 8, 2022 at 3:07 PM Amit Kapila wrote: > > On Mon, Nov 7, 2022 at 11:12 PM Robert Haas wrote: > > > > On Mon, Oct 31, 2022 at 11:49 PM Amit Kapila > > wrote: > > > I am fine with any of those. Would you like to commit or do you prefer > > > me to take care of this? > > > > Sorry for

Re: Time delayed LR (WAS Re: logical replication restrictions)

2022-11-14 Thread Amit Kapila
On Mon, Nov 14, 2022 at 2:28 PM Hayato Kuroda (Fujitsu) wrote: > > > I don't understand the reason for the below change in the patch: > > > > + /* > > + * If this subscription has been disabled and it has an apply > > + * delay set, wake up the logical replication worker to finish > > + * it as

  1   2   >