Re: simplehash.h: "SH_SCOPE static" causes warnings

2024-04-09 Thread Andres Freund
a point in having a simplehash instantiation. Might make it slightly easier to notice obsoleted uses of simplehash. Greetings, Andres Freund

Re: Annoying build warnings from latest Apple toolchain

2024-04-09 Thread Andres Freund
Hi, Looks like I forgot to update the thread to note that I finally committed the remaining warning fixes. I had already fixed a bunch of others in upstream meson. commit a3da95deee38ee067b0bead639c830eacbe894d5 Author: Andres Freund Date: 2024-03-13 01:40:53 -0700 meson: macos: Avoid

Re: macOS Ventura won't generate core dumps

2024-04-09 Thread Andres Freund
tion without a problem. Perhaps it's worth picking that up again? https://github.com/anarazel/postgres/tree/meson-rpath https://github.com/anarazel/postgres/commit/46f1963fee7525c3cc3837ef8423cbf6cb08d10a Greetings, Andres Freund

Re: Synchronizing slots from primary to standby

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 16:01:41 +0530, Amit Kapila wrote: > Pushed. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder&dt=2024-04-08%2012%3A04%3A27 This unfortunately is a commit after commit 6f3d8d5e7cc Author: Amit Kapila Date: 2024-04-08 13:21:55 +0530 Fix the intermittent buil

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
On 2024-04-08 08:37:44 -0700, Andres Freund wrote: > On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > > I was under the impression there are not so many out-of-core table > > > AMs, which have non-dummy analysis

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
ble API. To me the new API doesn't look well polished either, so it's not a question of a smoother transition or something like that. I don't think redesigning extension APIs at this stage of the release cycle makes sense. Greetings, Andres Freund

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 09:26:09 -0400, Robert Haas wrote: > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > And maybe we need to think of a way to further mitigate this crush of > last minute commits. e.g. In the last week, you can't have more > feature commits, or more lines of insertions in

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 16:18:21 +1200, David Rowley wrote: > On Mon, 8 Apr 2024 at 16:08, Andres Freund wrote: > > I think visible would be ok, the serialization checks are IMO about > > visibility too. But if you'd prefer I'd also be ok with something like > > page_c

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 15:43:12 +1200, David Rowley wrote: > On Mon, 8 Apr 2024 at 15:13, Andres Freund wrote: > > Off-list Melanie suggested heap_page_collect_visible_tuples(). Given the > > separate callsites (making long names annoying) and the fact that it's > > really

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 14:43:21 +1200, David Rowley wrote: > On Sun, 7 Apr 2024 at 19:30, Andres Freund wrote: > > Good call. Added and pushed. > > I understand you're already aware of the reference in the comment to > heapgetpage(), which no longer exists as of 44086b097. Y

Re: Coverity complains about simplehash.h's SH_STAT()

2024-04-07 Thread Andres Freund
On 2024-04-07 21:41:23 -0400, Tom Lane wrote: > Andres Freund writes: > > I can't see a way it could hurt in the back branches, so I'm inclined to > > backpatch the pfree? > > +1 Done

Re: Coverity complains about simplehash.h's SH_STAT()

2024-04-07 Thread Andres Freund
plain about that just now. If coverity had started to complain after da41d71070d, I'd understand, but that was ~5 years ago. I can't see a way it could hurt in the back branches, so I'm inclined to backpatch the pfree? Greetings, Andres Freund

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Andres Freund
aph uses "it is useful to be able to evict arbitrary blocks" that seems to describe things at least as well as "invalidate"? Greetings, Andres Freund

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 02:25:17 +0300, Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 an

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-04-07 Thread Andres Freund
n = cpp.get_supported_arguments(common_builtin_flags) > +endif This seems like a fair amount of extra configure tests. Particularly because /W* isn't ever interesting for Makefile.global - they're msvc flags - because you can't use that with msvc. I'm also doubtful that it's worth supporting warning_level=3/everything, you end up with a completely flood of warnings that way. Greetings, Andres Freund

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Andres Freund
total_allocated == > context->mem_allocated"), File: "bump.c", Line: 808, PID: 7809 > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=turaco&dt=2024-04-07%2022%3A42%3A54 What makes you think that's unrelated? To me that looks like the same issue? Greetings, Andres Freund

Re: Streaming read-ready sequential scan code

2024-04-07 Thread Andres Freund
fer(), so it'd be pretty much free to call another callback there. Greetings, Andres Freund

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Andres Freund
Remove BlockSampler_HasMore(). > > Author: Melanie Plageman, Nazir Bilal Yavuz > Discussion: > https://postgr.es/m/flat/CAN55FZ0UhXqk9v3y-zW_fp4-WCp43V8y0A72xPmLkOM%2B6M%2BmJg%40mail.gmail.com The justification here seems somewhat odd. Sure, the previous commit stopped using BlockSampler_HasMore in acquire_sample_rows - but only because it was moved to block_sampling_streaming_read_next()? Greetings, Andres Freund

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
interface basically requires reimplementing all of analyze.c. What am I missing here? Greetings, Andres Freund

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Andres Freund
t(total_allocated == context->mem_allocated); > (gdb) p total_allocated > $1 = 8120 > (gdb) p context->mem_allocated > $2 = 8192 Greetings, Andres Freund

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-07 Thread Andres Freund
'char' doesn't enforce any alignment, but you're storing a BlocktableEntry in a char[]. You can't just do that. Look at how we do that for e.g. PGAlignedblock. With the attached minimal fix, the tests pass again. Greetings, Andres Freund diff --git i/src/backend/access/

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andres Freund
On 2024-04-07 18:51:40 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2024, at 18:28, Andres Freund wrote: > > > > On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: > >>> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > >>> On 2024-04-06 Sa 20:49

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andres Freund
On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > > On 2024-04-06 Sa 20:49, Andres Freund wrote: > > >> That's probably unnecessary optimization, but it seems a tad silly to read > >> an > >>

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 12:07:22 +0700, John Naylor wrote: > Just in time ;-) The commit message should also have "reviewed by > Zhang Mingli" and "tested by Quan Zongliang", who shared results in a > thread for a withrawn CF entry with a similar idea but covering fewer > cases: Good call. Added and pu

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-06 Thread Andres Freund
Hi, On 2024-01-22 13:01:31 +0700, John Naylor wrote: > On Mon, Jul 17, 2023 at 9:58 PM Andres Freund wrote: > > And 397->320ms > > for something as core as this, is imo worth considering on its own. > > Hi Andres, this interesting work seems to have fallen off the

Re: Flushing large data immediately in pqcomm

2024-04-06 Thread Andres Freund
Hi, On 2024-04-07 00:45:31 +0200, Jelte Fennema-Nio wrote: > On Sat, 6 Apr 2024 at 22:21, Andres Freund wrote: > > The small regression for small results is still kinda visible, I haven't yet > > tested the patch downthread. > > Thanks a lot for the faster test scr

Re: Cluster::restart dumping logs when stop fails

2024-04-06 Thread Andres Freund
Hi, On 2024-04-07 00:19:35 +0200, Daniel Gustafsson wrote: > > On 6 Apr 2024, at 23:44, Andres Freund wrote: > > > It might be useful to print a few lines, but the whole log files can be > > several megabytes worth of output. > > The non-context aware fix would be

Cluster::restart dumping logs when stop fails

2024-04-06 Thread Andres Freund
the "normal" contents of regress_log* continue. Greetings, Andres Freund

Re: Synchronizing slots from primary to standby

2024-04-06 Thread Andres Freund
ntical across all of them, I think. I've also seen a bunch of failures of this test locally. Greetings, Andres Freund

Re: Flushing large data immediately in pqcomm

2024-04-06 Thread Andres Freund
small regression for small results is still kinda visible, I haven't yet tested the patch downthread. Greetings, Andres Freund #!/bin/bash set -e dbname=postgres port=5440 host=/tmp host=localhost test_cases=( "100 100 100" # only 100 bytes "1024 10240 10

Re: Speed up clean meson builds by ~25%

2024-04-05 Thread Andres Freund
o something about this from our side as well, I suspect. The times aren't great with gcc either, even if not as bad as with clang. Greetings, Andres Freund

Re: Recent 027_streaming_regress.pl hangs

2024-04-04 Thread Andres Freund
Hi, On 2024-04-04 19:00:00 +0300, Alexander Lakhin wrote: > 26.03.2024 10:59, Andres Freund wrote: > > Late, will try to look more in the next few days. > > > > AFAICS, last 027_streaming_regress.pl failures on calliphoridae, > culicidae, tamandua occurred be

Re: Streaming read-ready sequential scan code

2024-04-04 Thread Andres Freund
to dynamically size the default at some point - but we probably should overhaul the infrastructure first... Greetings, Andres Freund

Re: Security lessons from liblzma - libsystemd

2024-04-03 Thread Andres Freund
Hi, On 2024-04-03 17:58:55 -0400, Tom Lane wrote: > Magnus Hagander writes: > > On Wed, Apr 3, 2024 at 7:57 PM Andres Freund wrote: > >> Openssh has now integrated [1] a patch to remove the dependency on > >> libsystemd > >> for triggering service manager

Re: Streaming read-ready sequential scan code

2024-04-03 Thread Andres Freund
kend before you crash into dirty data; as you discovered, > prefetching accidentally makes that effect worse in.a BAS_VACUUM strategy, > by taking away space that is effectively deferring WAL flushes, so I'd at > least like to double the size for if we use "/ 2" above. I think for VACUUM we should probably go a bit further. There's no comparable L1/L2 issue, because the per-buffer processing + WAL insertion is a lot more expensive, compared to a seqscan. I'd go or at lest 4x-8x. Greetings, Andres Freund

Re: Parent/child context relation in pg_get_backend_memory_contexts()

2024-04-03 Thread Andres Freund
this. Greetings, Andres Freund

Re: Security lessons from liblzma - libsystemd

2024-04-03 Thread Andres Freund
service dependencies that can rely on the server actually having started up. Greetings, Andres Freund [1] https://bugzilla.mindrot.org/show_bug.cgi?id=2641 [2] https://www.freedesktop.org/software/systemd/man/devel/systemd.html#Readiness%20Protocol [3] https://www.freedesktop.org/softwa

Re: Security lessons from liblzma

2024-03-31 Thread Andres Freund
when set to an empty string but can be overridden? Greetings, Andres Freund

Re: Security lessons from liblzma

2024-03-30 Thread Andres Freund
generated docs into a separately downloadable archive. We already stopped including other generated files recently. > We shouldn't make the mistake of assuming that bad things can't happen to > us. +1 Greetings, Andres Freund

Re: Security lessons from liblzma

2024-03-29 Thread Andres Freund
f the test changes in a large commit. It's not too hard to hide something sneaky. I comparison to that hiding something in configure.ac seems less likely to succeed IMO, that imo tends to be more scrutinized. And hiding just in configure directly wouldn't get you far, it'd just get removed when the committer or some other committer at a later time, regenerates configure. Greetings, Andres Freund

Re: Recent 027_streaming_regress.pl hangs

2024-03-26 Thread Andres Freund
ll try to look more in the next few days. Greetings, Andres Freund

Re: make dist using git archive

2024-03-26 Thread Andres Freund
Hi, On 2024-03-26 08:36:58 +0100, Peter Eisentraut wrote: > On 26.03.24 01:23, Andres Freund wrote: > > On 2024-03-25 06:44:33 +0100, Peter Eisentraut wrote: > > > Done and committed. > > > > This triggered a new warning for me: > > > > ../../../../../h

Re: Recent 027_streaming_regress.pl hangs

2024-03-25 Thread Andres Freund
Hi, On 2024-03-26 00:00:38 -0400, Tom Lane wrote: > Andres Freund writes: > > I think there must be some actual regression involved. The frequency of > > failures on HEAD vs failures on 16 - both of which run the tests > > concurrently > > via meson - is just vastly

Re: Recent 027_streaming_regress.pl hangs

2024-03-25 Thread Andres Freund
Hi, On 2024-03-20 17:41:45 -0700, Andres Freund wrote: > On 2024-03-14 16:56:39 -0400, Tom Lane wrote: > > Also, this is probably not > > helping anything: > > > >

Re: Sync scan & regression tests

2024-03-25 Thread Andres Freund
Hi, On 2024-03-24 11:28:12 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 19/09/2023 01:57, Andres Freund wrote: > >> On 2023-09-18 13:49:24 +0300, Heikki Linnakangas wrote: > >>> d) Copy fewer rows to the table in the test. If we copy only 6 rows, for

Re: RFC: Logging plan of the running query

2024-03-25 Thread Andres Freund
ould > avoid the need to do this? No. I think it's acceptable though. However it might be worth inventing an executor tree walker in a preliminary step. We have already quite a few switches over all plan nodes, which we could largely replace with a helper. Greetings, Andres Freund

Re: make dist using git archive

2024-03-25 Thread Andres Freund
Hi, On 2024-03-25 06:44:33 +0100, Peter Eisentraut wrote: > Done and committed. This triggered a new warning for me: ../../../../../home/andres/src/postgresql/meson.build:3422: WARNING: Project targets '>=0.54' but uses feature introduced in '0.55.0': Passing executable/found program object to

Re: pgsql: Clean up role created in new subscription test.

2024-03-25 Thread Andres Freund
I haven't checked what your patch is doing, but perhaps the issue is that it's seeing global objects concurrently created by another test? Greetings, Andres Freund

Re: Trying to build x86 version on windows using meson

2024-03-22 Thread Andres Freund
ike the following (put it in a file, point meson to it wity --cross-file) might give you a start: [properties] needs_exe_wrapper = false [binaries] c = 'cl' cpp = 'cl' ar = 'lib' windres = 'rc' [host_machine] system = 'windows' cpu_family = 'x86_64' cpu = 'x86_64' endian = 'little' Greetings, Andres Freund

Re: Cannot find a working 64-bit integer type on Illumos

2024-03-22 Thread Andres Freund
Hi, On 2024-03-22 15:02:45 -0400, Robert Haas wrote: > On Fri, Mar 22, 2024 at 2:38 PM Andres Freund wrote: > > I wonder if we ought to make configure warn if it sees -Werror in CFLAGS - > > this is far from the first time somebody stumbling with -Werror. Including a > > few

Re: Cannot find a working 64-bit integer type on Illumos

2024-03-22 Thread Andres Freund
he best way is to use Makefile.custom. Greetings, Andres Freund

Re: Cannot find a working 64-bit integer type on Illumos

2024-03-22 Thread Andres Freund
;s an unrelated warning triggering the configure test to fail. We'd need to see config.log to see what that is. Greetings, Andres Freund

Re: An improved README experience for PostgreSQL

2024-03-21 Thread Andres Freund
On 2024-02-28 14:59:16 -0600, Nathan Bossart wrote: > On Wed, Feb 28, 2024 at 02:21:49PM -0600, Nathan Bossart wrote: > > On Wed, Feb 28, 2024 at 02:07:34PM -0600, Andrew Atkinson wrote: > >> I agree that starting with the direct conversion is reasonable. Markdown > >> "modernizes" the file using a

Re: Trying to build x86 version on windows using meson

2024-03-21 Thread Andres Freund
e these warnings Use --disable-stdcall-fixup to disable these fixups /usr/bin/i686-w64-mingw32-ld: warning: resolving _DllUnregisterServer by linking to _DllUnregisterServer@0 Greetings, Andres Freund

Re: Recent 027_streaming_regress.pl hangs

2024-03-20 Thread Andres Freund
Hi, On 2024-03-20 17:41:45 -0700, Andres Freund wrote: > 2024-03-20 22:14:01.904 UTC [56343][client backend][6/1925:0] LOG: > connection authorized: user=bf database=postgres > application_name=027_stream_regress.pl > 2024-03-20 22:14:01.930 UTC [56343][client backend][6

Re: Recent 027_streaming_regress.pl hangs

2024-03-20 Thread Andres Freund
Hi, On 2024-03-20 17:41:47 -0700, Andres Freund wrote: > There's a lot of other animals on the same machine, however it's rarely fuly > loaded (with either CPU or IO). > > I don't think the test just being slow is the issue here, e.g. in the last > failing iterat

Re: Recent 027_streaming_regress.pl hangs

2024-03-20 Thread Andres Freund
we had practically no test coverage of fsync, so I made my animals use fsync. I think we still have little coverage. I probably could reduce the number of animals using it though. Greetings, Andres Freund

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
Hi, On 2024-03-21 11:02:27 +1300, David Rowley wrote: > On Thu, 21 Mar 2024 at 11:00, Andres Freund wrote: > > > > On 2024-03-20 17:49:14 -0400, Dave Cramer wrote: > > > First off this is on an ARM64 machine > > > > Uh, that's a fairly crucial bit

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
Hi, On 2024-03-20 17:49:14 -0400, Dave Cramer wrote: > On Wed, 20 Mar 2024 at 17:11, Andres Freund wrote: > > On 2024-03-20 16:14:23 -0400, Dave Cramer wrote: > > > I am getting the following error > > > > > > meson.build:1479:17: ERROR: Can not

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
s86` This is not enough information to debug anything. At the very least we need the exact steps performed to set up the build and meson-logs/meson-log.txt > The docs say it is possible to build postgres for x86. Are there specific > instructions ? It should work. Greetings, Andres Freund

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-17 Thread Andres Freund
Hi, On 2024-03-16 21:25:18 +0100, Tomas Vondra wrote: > On 3/16/24 20:12, Andres Freund wrote: > > That would address some of the worst behaviour, but it doesn't really seem > > to > > address the underlying problem of the two iterators being modified > > independ

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-16 Thread Andres Freund
Hi, On 2024-03-15 18:42:29 -0400, Melanie Plageman wrote: > On Fri, Mar 15, 2024 at 5:14 PM Andres Freund wrote: > > On 2024-03-14 17:39:30 -0400, Melanie Plageman wrote: > > I spent a good amount of time looking into this with Melanie. After a bunch > > of > > wron

Re: Vectored I/O in bulk_write.c

2024-03-16 Thread Andres Freund
t's the benefit of combinding the two operations? I'm not dead-set against this, just not really convinced that it's a good idea to combine the operations. Greetings, Andres Freund

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-15 Thread Andres Freund
eems fairly substantial - we're moving the two iterators independently from each other, in multiple processes, without useful locking. I'm inclined to think this is a bug we ought to fix in the backbranches. Greetings, Andres Freund

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread Andres Freund
Hi, On March 6, 2024 9:06:50 AM GMT+01:00, John Naylor wrote: >On Wed, Mar 6, 2024 at 3:02 PM Masahiko Sawada wrote: >> >> On Wed, Mar 6, 2024 at 4:41 PM Andres Freund wrote: > >> > A few ARM buildfarm animals are complaining: >> > >> > https://bu

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-05 Thread Andres Freund
;dt=2024-03-06%2007%3A33%3A18 Greetings, Andres Freund

Re: Tidy fill hstv array (src/backend/access/heap/pruneheap.c)

2024-03-04 Thread Andres Freund
Hi, On 2024-03-04 08:47:11 -0300, Ranier Vilela wrote: > Does filling a memory area, one by one, with branches, need strong evidence > to prove that it is better than filling a memory area, all at once, without > branches? That's a bogus comparison: a) the memset version modifies the whole array

Re: Avoid stack frame setup in performance critical routines using tail calls

2024-03-04 Thread Andres Freund
set.c is > > just there for comparisons between slab and generation) > > > > The attached includes some additional tuning to generation.c: > > I've now pushed this. Thanks for working on all these, much appreciated! Greetings, Andres Freund

Re: Remove AIX Support (was: Re: Relation bulk write facility)

2024-02-29 Thread Andres Freund
Hi, On 2024-02-29 10:24:24 +0100, Michael Banck wrote: > On Thu, Feb 29, 2024 at 12:57:31AM -0800, Andres Freund wrote: > > On 2024-02-29 09:13:04 +0100, Michael Banck wrote: > > > The commit message says there is not a lot of user demand and that might > > > be r

Re: Remove AIX Support (was: Re: Relation bulk write facility)

2024-02-29 Thread Andres Freund
this. Then these users should have paid somebody to actually do maintenance work on the AIX support,o it doesn't regularly stand in the way of implementing various things. Greetings, Andres Freund

Re: Relation bulk write facility

2024-02-28 Thread Andres Freund
Hi, On 2024-02-27 12:59:14 -0800, Andres Freund wrote: > On 2024-02-27 15:45:45 -0500, Tom Lane wrote: > > Heikki Linnakangas writes: > > With AIX out of the picture, lapwing will be the only remaining > > animal testing MAXALIGN less than 8. That seems like a single

Re: Relation bulk write facility

2024-02-28 Thread Andres Freund
nditionally. Keep such columns before the first NameData column of > the > -- catalog, since packagers can override NAMEDATALEN to an odd number. > +-- (XXX: I'm not sure if any of the supported platforms have > MAXIMUM_ALIGNOF==8 and > +-- ALIGNOF_DOUBLE==4. Perhaps we should just require that > +-- ALIGNOF_DOUBLE==MAXIMUM_ALIGNOF) > -- > WITH check_columns AS ( > SELECT relname, attname, I agree, this should be an error, and we should then remove the test. Greetings, Andres Freund

Re: Relation bulk write facility

2024-02-27 Thread Andres Freund
animals? I had supposed that my faithful old PPC animal mamba > was helping to check this, but I see that under NetBSD it's > joined the ALIGNOF_DOUBLE==8 crowd. I can set up a i386 animal, albeit on an amd64 kernel. But I don't think the latter matters. Greetings, Andres Freund

Re: Relation bulk write facility

2024-02-24 Thread Andres Freund
estriction. Let's just drop AIX. This isn't the only alignment issue we've found and the solution for those isn't so much a fix as forcing everyone to carefully only look into one direction and not notice the cliffs to either side. Greetings, Andres Freund [1] https://www.ibm.com/docs/en/SSGH2K_13.1.2/com.ibm.compilers.aix.doc/proguide.pdf

Re: locked reads for atomics

2024-02-24 Thread Andres Freund
to be faster to just do a single atomic operation. And it's a bit simpler to analyze on which "side" of the read/write the barrier is needed. Greetings, Andres Freund

Re: locked reads for atomics

2024-02-24 Thread Andres Freund
ause the underlying variable isn't actually shared. In those cases using the other variants is a bug. The only use of pg_atomic_unlocked_write_u32() is temp-table buffers which share the data structure with the shared buffers case. Greetings, Andres Freund

Re: Avoid stack frame setup in performance critical routines using tail calls

2024-02-22 Thread Andres Freund
o review or test it, please do > so soon. Makes sense! > @@ -1061,6 +1072,16 @@ MemoryContextAlloc(MemoryContext context, Size size) > > context->isReset = false; > For a moment this made me wonder if we could move the isReset handling into the allocator slow paths as well - it's annoying to write that bit (and thus dirty the cacheline) over and ove. But it'd be somewhat awkward due to pre-allocated blocks. So that'd be a larger change better done separately. Greetings, Andres Freund

Re: Why is pq_begintypsend so slow?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 10:02:52 +0900, Sutou Kouhei wrote: > In <20240218200906.zvihkrs46yl2j...@awork3.anarazel.de> > "Re: Why is pq_begintypsend so slow?" on Sun, 18 Feb 2024 12:09:06 -0800, > Andres Freund wrote: > > >> [1] > >&

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 13:54:01 -0500, Joe Conway wrote: > On 2/19/24 13:13, Andres Freund wrote: > > On 2024-02-19 09:19:16 -0500, Joe Conway wrote: > > > Couldn't we scale the rounding, e.g. allow small allocations as we do now, > > > but above some number always r

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 09:19:16 -0500, Joe Conway wrote: > On 2/18/24 15:35, Andres Freund wrote: > > On 2024-02-18 17:06:09 +0530, Robert Haas wrote: > > > How many people set shared_buffers to something that's not a whole > > > number of GB these days? > > >

Re: PGC_SIGHUP shared_buffers?

2024-02-18 Thread Andres Freund
Hi, On 2024-02-18 17:06:09 +0530, Robert Haas wrote: > On Sat, Feb 17, 2024 at 12:38 AM Andres Freund wrote: > > IMO the ability to *shrink* shared_buffers dynamically and cheaply is more > > important than growing it in a way, except that they are related of > > cours

Re: Why is pq_begintypsend so slow?

2024-02-18 Thread Andres Freund
Hi, On 2024-02-18 17:38:09 +0900, Sutou Kouhei wrote: > In <20240218015955.rmw5mcmobt5hb...@awork3.anarazel.de> > "Re: Why is pq_begintypsend so slow?" on Sat, 17 Feb 2024 17:59:55 -0800, > Andres Freund wrote: > > > v3-0008-wip-make-in-out-send-recv-cal

Re: Why is pq_begintypsend so slow?

2024-02-17 Thread Andres Freund
Hi, I was reminded of the patchset I had posted in this thread by https://postgr.es/m/679d5455cbbb0af667ccb753da51a475bae1eaed.camel%40cybertec.at On 2020-07-31 13:35:43 -0400, Robert Haas wrote: > On Fri, Jul 31, 2020 at 1:00 PM Andres Freund wrote: > > > Perhaps we could add a c

Re: PGC_SIGHUP shared_buffers?

2024-02-17 Thread Andres Freund
recent CPUs that has grown to 57bits [1], that's a lot of space. And if you do run out of VM space, wouldn't that also affect lots of other things, like mmap() for malloc? Greetings, Andres Freund [1] https://en.wikipedia.org/wiki/Intel_5-level_paging

Re: Speeding up COPY TO for uuids and arrays

2024-02-17 Thread Andres Freund
e dashes. > - Patch 0003 speeds up array_out a bit by avoiding some zero > byte writes. The measured speed gain is under 2%. Makes sense. Greetings, Andres Freund

Re: Add bump memory context type and use it for tuplesorts

2024-02-17 Thread Andres Freund
ould have been done > some other way than by consuming a context type ID.) Possibly, I just don't quite know how. Greetings, Andres Freund

Re: System username in pg_stat_activity

2024-02-16 Thread Andres Freund
On 2024-02-16 21:56:25 +0100, Magnus Hagander wrote: > On Fri, Feb 16, 2024 at 9:51 PM Andres Freund wrote: > > I only skimmed the patch, but it sure looks to me that we could end up with > > none of the branches setting 31,32, so I think you'd have to make sure to > > h

Re: System username in pg_stat_activity

2024-02-16 Thread Andres Freund
>backendStatus.st_auth_identity); > + } > + } Mostly, yes. I only skimmed the patch, but it sure looks to me that we could end up with none of the branches setting 31,32, so I think you'd have to make sure to handle that case. Greetings, Andres Freund

Re: System username in pg_stat_activity

2024-02-16 Thread Andres Freund
ity. We even already have a patch to add pg_stat_session https://commitfest.postgresql.org/47/3405/ Greetings, Andres Freund

Re: System username in pg_stat_activity

2024-02-16 Thread Andres Freund
Hi, On 2024-02-16 20:57:59 +0100, Magnus Hagander wrote: > On Fri, Feb 16, 2024 at 8:41 PM Andres Freund wrote: > > On 2024-01-10 12:46:34 +0100, Magnus Hagander wrote: > > > The attached patch adds a column "authuser" to pg_stat_activity which > > > co

Re: System username in pg_stat_activity

2024-02-16 Thread Andres Freund
ady is quite complicated, particularly for pgstat, we shouldn't make it more complicated. Especially not when the initialization is moved quite a bit away from all the other calls. Besides just that, I also don't think delaying visibility of the worker in pg_stat_activity until parallel worker initialization has completed is good, that's not all cheap work. Maybe I am missing something, but why aren't we just getting the value from the leader's entry, instead of copying it? Greetings, Andres Freund

Re: System username in pg_stat_activity

2024-02-16 Thread Andres Freund
urther down that road. It's not just a usability issue, it also makes it more expensive to query pg_stat_activity. This is of course more pronounced with textual columns than with integer ones. Greetings, Andres Freund

Re: SQL Property Graph Queries (SQL/PGQ)

2024-02-16 Thread Andres Freund
that benefit from the graph query formulation. But more importantly, we IMO should reduce usage of the rewrite system, not increase it. Greetings, Andres Freund

Re: PGC_SIGHUP shared_buffers?

2024-02-16 Thread Andres Freund
d to be easier to resize and maybe > simpler in some other ways that sits between shared_buffers and the OS > cache. This doesn't seem super-appealing because it requires a lot of > data copying, but maybe it's worth considering as a last resort. Yea, that seems quite unappealing. Needing buffer replacement to be able to pin a buffer would be ... unattractive. Greetings, Andres Freund

Re: Transaction timeout

2024-02-15 Thread Andres Freund
to test that quick query does not restart transaction_timeout > +step s8_select_1 { SELECT 1; } > +step s8_sleep{ SELECT pg_sleep(0.6); } > + > +session checker > +step checker_sleep { SELECT pg_sleep(0.3); } Isn't this test going to be very fragile on busy / slow machines? What if the pg_sleep() takes one second, because there were other tasks to schedule? I'd be surprised if this didn't fail under valgrind, for example. Greetings, Andres Freund

Re: [PATCH] Avoid mixing custom and OpenSSL BIO functions

2024-02-15 Thread Andres Freund
should rename this. It's pretty odd that we still call it's not really related to s_socket anymore, and doesn't even really implement the same interface (e.g. get_fd doesn't work anymore). Similarly, my_SSL_set_fd() doesn't actually call set_fd() anymore, which sure seems odd. Greetings, Andres Freund [1] https://www.postgresql.org/message-id/20210715021747.h2hih7mw56ivyntt%40alap3.anarazel.de

Re: index prefetching

2024-02-15 Thread Andres Freund
d - I suspect we'll trigger regressions without that in place. One way to *sometimes* process more than a single leaf page, without having to redesign kill_prior_tuple, would be to use the visibilitymap to check if the target pages are all-visible. If all the table pages on a leaf page are all-visible, we know that we don't need to kill index entries, and thus can move on to the next leaf page Greetings, Andres Freund

Re: RFC: Logging plan of the running query

2024-02-15 Thread Andres Freund
me. A large sort, for example, could spend a fair amount of time inside tuplesort, similarly, a gather node might need to wait for a worker for a while etc. > It's really hard for me to accept that the heavyweight lock problem > for which the patch contains a workaround is the only one that exists. > I can't see any reason why that should be true. I suspect you're right. Greetings, Andres Freund

Re: index prefetching

2024-02-14 Thread Andres Freund
nd (2) we have some mechanism for limiting the > number of extra pins that we're holding to a reasonable number given the > size of shared_buffers. FWIW, there's already some logic for (2) in LimitAdditionalPins(). Currently used to limit how many buffers a backend may pin for bulk re

Re: index prefetching

2024-02-14 Thread Andres Freund
th partitioned queries it often is. Depending on the form of "control" that's harder if IAMs are in control, because control flow needs to return to the executor to be able to switch to a different node, so we can't wait for IO inside the AM. There probably are ways IAMs could be in "control" that would be compatible with such constraints however. Greetings, Andres Freund

<    1   2   3   4   5   6   7   8   9   10   >