Re: Remove pthread_is_threaded_np() checks in postmaster

2024-01-23 Thread Andres Freund
sn't limited to a single thread. You'd definitely need to use pthread_sigprocmask etc to address that aspect alone. Greetings, Andres Freund

Re: the s_lock_stuck on perform_spin_delay

2024-01-23 Thread Andres Freund
init_local_spin_delay(); > > + START_SPIN_LOCK(); > while (true) > { > /* set BM_LOCKED flag */ Seems pretty odd that we now need init_local_spin_delay() and START_SPIN_LOCK(). Note that init_local_spin_delay() also wraps handling of __FILE__, __LINE__ etc, so it seems we're duplicating state here. Greetings, Andres Freund

Re: the s_lock_stuck on perform_spin_delay

2024-01-23 Thread Andres Freund
on this isn't using a plain spinlock is that this way we can modify more state with a single atomic operation. But all the dangers of using spinlocks apply just as well. Greetings, Andres Freund

Re: Refactoring backend fork+exec code

2024-01-23 Thread Andres Freund
Hi, On 2024-01-23 21:07:08 +0200, Heikki Linnakangas wrote: > On 22/01/2024 23:07, Andres Freund wrote: > > > diff --git a/src/backend/utils/activity/backend_status.c > > > b/src/backend/utils/activity/backend_status.c > > > index 1a1050c8da1..92f24db4e18 100644

Re: Remove pthread_is_threaded_np() checks in postmaster

2024-01-23 Thread Andres Freund
es which use libcurl (like Neon). Under the hood, libcurl calls > SCDynamicStoreCopyProxies[1], which apparently causes the check to fail. Maybe I'm missing something, but isn't that indicating the exact opposite, namely that the check is precisely doing what it's intended? Greetings, Andres Freund

Re: FEATURE REQUEST: Role vCPU limit/priority

2024-01-23 Thread Andres Freund
Hi, On 2024-01-23 13:09:22 +0100, Thomas Kellerer wrote: > Yoni Sade schrieb am 21.01.2024 um 19:07: > > It would be useful to have the ability to define for a role default > > vCPU affinity limits/thread priority settings so that more active > > sessions could coexist similar to MySQL resource

Re: core dumps in auto_prewarm, tests succeed

2024-01-23 Thread Andres Freund
Hi, On 2024-01-23 08:00:00 +0300, Alexander Lakhin wrote: > 22.01.2024 23:41, Andres Freund wrote: > > ISTM that we shouldn't basically silently overlook shutdowns due to crashes > > in > > the tests. How to not do so is unfortunately not immediately obvious to > &

Re: core dumps in auto_prewarm, tests succeed

2024-01-23 Thread Andres Freund
case where needed, than to add a new function doing checking and converting most things to that. Greetings, Andres Freund

Re: Network failure may prevent promotion

2024-01-22 Thread Andres Freund
re a point in keeping 728f86fec6 as well on HEAD? That does not > strike me as wise to keep that in the tree for now. If it needs to be > reworked, looking at this problem from scratch would be a safer > approach. IDK, I think we'll introduce this type of bug over and over if we don't fix it

Re: core dumps in auto_prewarm, tests succeed

2024-01-22 Thread Andres Freund
Hi, On 2024-01-22 15:19:36 -0600, Nathan Bossart wrote: > On Mon, Jan 22, 2024 at 02:44:57PM -0600, Nathan Bossart wrote: > > On Mon, Jan 22, 2024 at 12:41:17PM -0800, Andres Freund wrote: > >> I noticed that I was getting core dumps while executing the tests, without > >

Re: Refactoring backend fork+exec code

2024-01-22 Thread Andres Freund
favor of new Am*Process() macros that > use MyBackendType. For consistency with the existing Am*Process() > macros. The Am*Process() macros aren't realy new, they are just implemented differently, right? I guess there are a few more of them now. Given that we are probably going to have more process types in the future, it seems like a better direction would be a AmProcessType(proctype) style macro/inline function. That we we don't have to mirror the list of process types in the enum and a set of macros. Greetings, Andres Freund

Re: Evaluate arguments of correlated SubPlans in the referencing ExprState

2024-01-22 Thread Andres Freund
independent of this patch - by coincidence I just sent an email about the issue https://www.postgresql.org/message-id/20240122204117.swton324xcoodnyi%40awork3.anarazel.de a few minutes ago. Greetings, Andres Freund

core dumps in auto_prewarm, tests succeed

2024-01-22 Thread Andres Freund
e system is ready to accept connections ISTM that we shouldn't basically silently overlook shutdowns due to crashes in the tests. How to not do so is unfortunately not immediately obvious to me... Greetings, Andres Freund

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2024-01-22 Thread Andres Freund
_BLCKSZ)) && > + phdr->xlp_tli == tli)) > + break; I still think that anything that requires such checks shouldn't be merged. It's completely bogus to check page contents for validity when we should have metadata telling us which range of the buffers is valid and which not. Greetings, Andres Freund

Re: Does redundant extension exist During faster COPY in PG16

2024-01-22 Thread Andres Freund
immediately acquire it, search the fsm, and retry if we couldn't find any free space, which would substantially increase contention. The FSM is the source of substantial contention, disabling it actually results in substantial throughput increases. Vastly increasing the number of lookups in the

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-09 Thread Andres Freund
Hi, On January 9, 2024 11:33:29 AM PST, Robert Haas wrote: >On Tue, Jan 9, 2024 at 2:23 PM Melanie Plageman > wrote: >> Yes, I agree. I thought about it more, and I prefer updating the FSM >> and setting nonempty_pages into lazy_scan_[no]prune(). Originally, I >> had ordered the patch set with

Re: Build versionless .so for Android

2024-01-05 Thread Andres Freund
an you > provide a minimal example? And so on. I looked around for a bit, and couldn't really find good documentation around this. The best I found is https://developer.android.com/ndk/guides/abis#native-code-in-app-packages which does strongly imply that the library names aren't versioned. Greetings, Andres Freund

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-05 Thread Andres Freund
Hi, On 2024-01-05 15:23:12 -0500, Robert Haas wrote: > On Fri, Jan 5, 2024 at 3:05 PM Andres Freund wrote: > > An aside: > > > > As I think we chatted about before, I eventually would like the option to > > remove index entries for a tuple during on-access pruning,

Re: Build versionless .so for Android

2024-01-05 Thread Andres Freund
x would be to just map android to linux, similar to how dragonflybsd is mapped to netbsd. Blind attempt attached. It looks like that might be all that's required, as it looks like meson knows that android doesn't properly deal with soversion. Greetings, Andres Freund diff --git i/meson.build w/m

Re: Stack overflow issue

2024-01-05 Thread Andres Freund
existing MemoryContextStatsInternal() when that's the case. Greetings, Andres Freund

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-05 Thread Andres Freund
ve huge indexes. The reason I mention this is that then we'd have another source of marking items unused during pruning. Greetings, Andres Freund

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-05 Thread Andres Freund
Hi, On 2024-01-04 17:37:27 -0500, Melanie Plageman wrote: > On Thu, Jan 4, 2024 at 3:03 PM Andres Freund wrote: > > > > On 2023-11-17 18:12:08 -0500, Melanie Plageman wrote: > > > Assert(ItemIdIsNormal(lp)); > > > htup = (He

Re: the s_lock_stuck on perform_spin_delay

2024-01-05 Thread Andres Freund
Hi, On 2024-01-05 14:19:23 -0500, Robert Haas wrote: > On Fri, Jan 5, 2024 at 2:11 PM Andres Freund wrote: > > I see it fairly regularly. Including finding several related bugs that lead > > to > > stuck systems last year (signal handlers are a menace). > > In that

Re: the s_lock_stuck on perform_spin_delay

2024-01-05 Thread Andres Freund
Hi, On 2024-01-05 10:20:39 +0800, Andy Fan wrote: > Andres Freund writes: > > On 2024-01-04 14:59:06 +0800, Andy Fan wrote: > >> My question is if someone doesn't obey the rule by mistake (everyone > >> can make mistake), shall we PANIC on a production environme

Re: the s_lock_stuck on perform_spin_delay

2024-01-05 Thread Andres Freund
Hi, On 2024-01-05 08:51:53 -0500, Robert Haas wrote: > On Thu, Jan 4, 2024 at 6:06 PM Andres Freund wrote: > > I think we should add infrastructure to detect bugs like this during > > development, but not PANICing when this happens in production seems > > completely > &

Re: the s_lock_stuck on perform_spin_delay

2024-01-04 Thread Andres Freund
is. I don't think that's particularly promising. Lackey is *slow*. And it requires actually reaching problematic states. Consider e.g. the case that was reported upthread, an lwlock acquired within a spinlock protected section - most of the time that's not going to result in a lot of cycles, because the lwlock is free. Greetings, Andres Freund

Re: the s_lock_stuck on perform_spin_delay

2024-01-04 Thread Andres Freund
hard. I think we should add infrastructure to detect bugs like this during development, but not PANICing when this happens in production seems completely non-viable. Greetings, Andres Freund

Re: the s_lock_stuck on perform_spin_delay

2024-01-04 Thread Andres Freund
g scalability. FWIW, these days spinlocks do report a wait event when in perform_spin_delay() - albeit without detail which lock is being held. Greetings, Andres Freund

Re: Confine vacuum skip logic to lazy_scan_skip

2024-01-04 Thread Andres Freund
atural change on its own. VacSkipState is per > block and the LVRelState is referenced for the whole relation vacuum. I'd do it the other way round, i.e. either embed VacSkipState ino LVRelState or point to it from VacSkipState. LVRelState is already tied to the iteration state, so I don't think there's a reason not to do so. Greetings, Andres Freund

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-04 Thread Andres Freund
loop body after the call to lazy_scan_prune() is specific to the lazy_scan_prune() path, it doesn't make sense that it's at the same level as the the calls to lazy_scan_noprune(), lazy_scan_new_or_empty() or lazy_scan_prune(). Either it should be in lazy_scan_prune() or a new wrapper function. 3) It's imo wrong that we have UnlockReleaseBuffer() (there are 6 different places unlocking if I didn't miscount!) and RecordPageWithFreeSpace() calls in this many places. I think this is largely a consequence of the previous points. Once those are addressed, we can have one common place. But I digress. Greetings, Andres Freund

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-04 Thread Andres Freund
think "reap" has a precise, universally-understood meaning. Less concerned about that. > You could call this "mark_unused_now" or "immediately_mark_unused" or > something and it would be far more self-documenting, IMHO. How about 'no_indexes' or such? Greetings, Andres Freund

Re: index prefetching

2023-12-21 Thread Andres Freund
changed it. > > So don't blame me. :-) But I'd like you to feel guilty (no, not really) and fix it (yes, really) :) Greetings, Andres Freund

Re: ci: Build standalone INSTALL file

2023-12-21 Thread Andres Freund
Hi, On 2023-12-21 10:46:02 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-12-21 10:22:49 -0500, Tom Lane wrote: > >> We could make it version-specific, > >> https://www.postgresql.org/docs/17/installation.html > >> and task src/tools/version_st

Re: index prefetching

2023-12-21 Thread Andres Freund
Hi, On 2023-12-21 16:20:45 +0100, Tomas Vondra wrote: > On 12/21/23 14:43, Andres Freund wrote: > >> AFAICS this seems similar to some of the AIO patch, I wonder what that > >> plans to do. I need to check. > > > > Yes, most of this exists there. The diffe

Re: ci: Build standalone INSTALL file

2023-12-21 Thread Andres Freund
the branch is made? I think it runs a few times a day - breaking the link for a few hours wouldn't be optimal, but also not the end of the world. But redirecting $vnext -> to devel would probably be a more reliable approach. Greetings, Andres Freund

Re: Track in pg_replication_slots the reason why slots conflict?

2023-12-21 Thread Andres Freund
On 2023-12-21 19:55:51 +0530, Amit Kapila wrote: > On Thu, Dec 21, 2023 at 5:05 PM Andres Freund wrote: > > We clearly can't just expose the numerical value for a C enum. So it has to > > be > > converted to something SQL representable. > > > > We can re

Re: index prefetching

2023-12-21 Thread Andres Freund
Yes, most of this exists there. The difference that with the AIO you don't need to prefetch, as you can just initiate the IO for real, and wait for it to complete. Greetings, Andres Freund

Re: index prefetching

2023-12-21 Thread Andres Freund
Similar for sequential scan with the infrastructure in https://postgr.es/m/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com Greetings, Andres Freund

Re: Building PosgresSQL with LLVM fails on Solaris 11.4

2023-12-21 Thread Andres Freund
; restart a server process. > We had to back to the previous installation. > > It seems there is definitievly something wrong with the result gcc created. I suspect that the LLVM version you used does something wrong on sparc. Which version of LLVM is it? Greetings, Andres Freund

Re: Detecting some cases of missing backup_label

2023-12-21 Thread Andres Freund
P was replayed - I think we already have the infrastructure to queue file deletions that we can hook into b) when replaying a shutdown checkpoint / after creation of a shutdown checkpoint Greetings, Andres Freund

Re: Track in pg_replication_slots the reason why slots conflict?

2023-12-21 Thread Andres Freund
Hi, On 2023-12-21 16:08:48 +0530, shveta malik wrote: > On Thu, Dec 21, 2023 at 3:10 PM Andres Freund wrote: > > > > Hi, > > > > On 2023-12-21 09:21:04 +0900, Michael Paquier wrote: > > > While listening at Bertrand's talk about logical decoding on stand

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-21 Thread Andres Freund
nts in the file just say "FIXME" in places that apparently are affected by design changes. This makes it hard to catch up here. Greetings, Andres Freund

Re: Track in pg_replication_slots the reason why slots conflict?

2023-12-21 Thread Andres Freund
lumns aren't free from a usability perspective. IFF we do something, I think it should be a single column with a cause. Greetings, Andres Freund

Re: ci: Build standalone INSTALL file

2023-12-21 Thread Andres Freund
Hi, On 2023-12-21 08:39:26 +0900, Michael Paquier wrote: > On Wed, Dec 20, 2023 at 11:36:28AM -0500, Tom Lane wrote: > > Andres Freund writes: > >> We fairly regularly have commits breaking the generation of INSTALL. IIRC > >> we > >> recently discussed

Re: ci: Build standalone INSTALL file

2023-12-21 Thread Andres Freund
Hi, On 2023-12-20 15:28:56 +0100, Daniel Gustafsson wrote: > + time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL > unrelated pet peeve: "make -C doc/src/sgml all" doesn't build all docs > targets.. Well, building the PDF takes a *long* time and is rarely required. I think there's an

Re: ci: Build standalone INSTALL file

2023-12-21 Thread Andres Freund
On 2023-12-21 08:44:33 +0900, Michael Paquier wrote: > On Wed, Dec 20, 2023 at 03:28:56PM +0100, Daniel Gustafsson wrote: > > + time make -s -j${BUILD_JOBS} -C doc/src/sgml all INSTALL > > unrelated pet peeve: "make -C doc/src/sgml all" doesn't build all docs > > targets.. > > That seems

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
rc/tools/win32tzlist.pl on !windows? It's obviously windows specific code, and it's special case enough that there doesn't seem like a need to develop it on !windows. Greetings, Andres Freund

Re: ci: Build standalone INSTALL file

2023-12-20 Thread Andres Freund
Hi, The attached patch had a slight bug. Also turned out that the CI environment didn't have pandoc installed. Fixed that. Greetings, Andres Freund >From 6a8157b2f14327351798c805a873e9e910f5cd67 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 20 Dec 2023 03:44:44 -0800 Subject: [PA

ci: Build standalone INSTALL file

2023-12-20 Thread Andres Freund
Hi, We fairly regularly have commits breaking the generation of INSTALL. IIRC we recently discussed building it locally unconditionally, but I couldn't immediately find that discussion. Until then, I think we should at least build it in CI so that cfbot can warn. Greetings, Andres Freund >F

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
ike a problem. > Another observation is that we could include the executable name in > this case something like: > ERROR: Tried to use not-found external program "sed" in "command" It's a meson generated message, so you'd need to open a bug report / feature request for it. Greetings, Andres Freund

Re: backtrace_on_internal_error

2023-12-20 Thread Andres Freund
it this way on a green field, but given the current state Tom's approach seems more sensible... Greetings, Andres Freund

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
On 2023-12-20 09:48:37 +0900, Michael Paquier wrote: > On Mon, Nov 20, 2023 at 05:03:28PM +0900, Michael Paquier wrote: > > Your suggestion to create a new sect2 for "Windows" as much as Andres' > > suggestion are OK by as an intermediate step, and I suspect that the > > end result will likely not

Re: Clang optimiser vs preproc.c

2023-12-16 Thread Andres Freund
concrete > idea about what to do specifically, and the thread slid off the radar > screen. One interest aspect might be that preproc.c ends up with ~33% more states than gram.c gram.c: #define YYLAST 116587 preproc.c: #define YYLAST 155495 Greetings, Andres Freund

Re: [meson] expose buildtype debug/optimization info to pg_config

2023-12-15 Thread Andres Freund
ption is enabled, which > is different from old building system. I wanted to skip the C++ tests when we don't need C++, because it makes configure take longer. But I could be convinced that we should always at least determine the C++ compiler for Makefile.global. Greetings, Andres Freund

Re: optimize atomic exchanges

2023-12-15 Thread Andres Freund
e asm. Otherwise looks good. Greetings, Andres Freund

Re: backtrace_on_internal_error

2023-12-09 Thread Andres Freund
Hi, On 2023-12-09 12:41:30 -0500, Tom Lane wrote: > I wrote: > > Andres Freund writes: > >> I was wondering about that too. But if we do so, why not also do it for > >> writes? > > > Writes don't act that way, do they? EOF on a pipe gives you an error,

Re: backtrace_on_internal_error

2023-12-09 Thread Andres Freund
Hi, On 2023-12-08 19:39:20 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-12-08 17:29:45 -0500, Tom Lane wrote: > >> Agreed. I think we want to do that after the initial handshake, > >> too, so maybe as attached. > > > I was wondering about that t

Re: backtrace_on_internal_error

2023-12-08 Thread Andres Freund
Hi, On 2023-12-08 17:35:26 -0500, Tom Lane wrote: > Andres Freund writes: > > I thought it'd be nice to have a test for this, particularly because it's > > not > > clear that the behaviour is consistent across openssl versions. > > Perhaps, but ... > > > T

Re: backtrace_on_internal_error

2023-12-08 Thread Andres Freund
On 2023-12-08 17:29:45 -0500, Tom Lane wrote: > Andres Freund writes: > >> I think I figured it it out. Looks like we need to translate a closed > >> socket > >> (recvfrom() returning 0) to ECONNRESET or such. > > > Seems like we should just treat

Re: backtrace_on_internal_error

2023-12-08 Thread Andres Freund
Hi, On 2023-12-08 11:33:16 -0800, Andres Freund wrote: > On 2023-12-08 10:51:01 -0800, Andres Freund wrote: > > On 2023-12-08 13:46:07 -0500, Tom Lane wrote: > > > Andres Freund writes: > > > > On 2023-12-08 13:23:50 -0500, Tom Lane wrote: > > > >> H

Re: backtrace_on_internal_error

2023-12-08 Thread Andres Freund
Hi, On 2023-12-08 10:51:01 -0800, Andres Freund wrote: > On 2023-12-08 13:46:07 -0500, Tom Lane wrote: > > Andres Freund writes: > > > On 2023-12-08 13:23:50 -0500, Tom Lane wrote: > > >> Hmm, don't suppose you have a way to reproduce that? > > > > >

Re: backtrace_on_internal_error

2023-12-08 Thread Andres Freund
Hi, On 2023-12-08 13:46:07 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-12-08 13:23:50 -0500, Tom Lane wrote: > >> Hmm, don't suppose you have a way to reproduce that? > > > After a bit of trying, yes. I put an abort() into pgtls_open_client(), &

Re: backtrace_on_internal_error

2023-12-08 Thread Andres Freund
Hi, On 2023-12-08 13:23:50 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-12-08 10:05:09 -0500, Tom Lane wrote: > >> ... there was already opinion upthread that this should be on by > >> default, which I agree with. You shouldn't be hitting cases lik

Re: Streaming I/O, vectored I/O (WIP)

2023-12-08 Thread Andres Freund
is level, can you? > Unless you decide that fd.c should get into the business of raising > errors for I/O failures, which would be a bit of a departure. > > That's why I did the retry higher up in md.c. I think that's the right call. I think for AIO we can't do retry handling purely in fd.c, or at least it'd be quite awkward. It doesn't seem like it'd buy us that much in md.c anyway, we still need to handle the cross segment case and such, from what I can tell? Greetings, Andres Freund

Re: backtrace_on_internal_error

2023-12-08 Thread Andres Freund
were a lot more in older PG versions, I tried to filter those out. I'm a bit confused about the huge number of "could not accept SSL connection: %m" with a zero errno. I guess we must be clearing errno somehow, but I don't immediately see where. Or perhaps we need to actually look at what SSL_get_error() returns? Greetings, Andres Freund

Re: remaining sql/json patches

2023-12-08 Thread Andres Freund
ng JSON data using > jsonpath expressions. The functions are: > > JSON_EXISTS() > JSON_QUERY() > JSON_VALUE() > > JSON_EXISTS() tests if the jsonpath expression applied to the jsonb > value yields any values. > > JSON_VALUE() must return a single value, and an err

Re: errors building on windows using meson

2023-12-07 Thread Andres Freund
Hi, On 2023-12-07 14:16:52 -0500, Dave Cramer wrote: > On Thu, 7 Dec 2023 at 13:53, Andres Freund wrote: > > > Hi, > > > > On 2023-12-07 12:54:27 -0500, Dave Cramer wrote: > > > state-exec: run failed: cannot create new executor meta: cannot get > > >

Re: Remove MSVC scripts from the tree

2023-12-07 Thread Andres Freund
Hi, On 2023-12-07 13:49:44 -0500, Andrew Dunstan wrote: > On 2023-12-07 Th 12:34, Andres Freund wrote: > > However: I used godbolt to compile the test code on msvc, and it seems to > > build with 19.15 (which is the version Andrew referenced upthread), with a > > war

Re: errors building on windows using meson

2023-12-07 Thread Andres Freund
; state-exec: Not user serviceable; Please contact support for assistance. > > anyone seen this or have a fix ? I've not seen that before. Please provide a bit more detail. Compiler, building with ninja or msbuild/visual studio, when exactly you're encountering the issue, ... Greetings, Andres Freund

Re: Remove MSVC scripts from the tree

2023-12-07 Thread Andres Freund
to compile the test code on msvc, and it seems to build with 19.15 (which is the version Andrew referenced upthread), with a warning that's triggered independent of the structfunc bit. https://godbolt.org/z/j99E9MeEK Andrew, could you attach meson.log from the failed build? Greetings, Andres Freund

Re: Remove MSVC scripts from the tree

2023-12-07 Thread Andres Freund
Hi, On 2023-12-04 15:11:47 -0500, Andrew Dunstan wrote: > I just had a look at shifting bowerbird to use meson, and it got stymied at > the c99 test, which apparently doesn't compile with anything less than > VS2019. What error or warning is being raised by msvc? Andres

Re: Building PosgresSQL with LLVM fails on Solaris 11.4

2023-12-07 Thread Andres Freund
ecurse] Error 2 I don't know where the -KPIC is coming from. And TBH, I don't see much point trying to fix a scenario involving matching sun studio C with g++. > With ggc it fails at the same step as before. > I have attached the log files of the SunStudio and gcc runs to the email. I don't se

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

2023-12-06 Thread Andres Freund
Hi, This looks good to me! Greetings, Andres Freund

Re: Building PosgresSQL with LLVM fails on Solaris 11.4

2023-12-06 Thread Andres Freund
Could you attach config.log and gmake for the gcc based build? Because so far I have no idea what causes the linker issue there. Greetings, Andres Freund diff --git i/src/backend/port/Makefile w/src/backend/port/Makefile index 47338d99229..4b3bd9af229 100644 --- i/src/backend/port/Makefile +++ w/sr

Re: Use PGAlignedBlock instead of "char buf[BLCKSZ]" in more places

2023-12-04 Thread Andres Freund
began using BLCKSZ because that was just a handy thing to > > do, and because we know they would never get beyond that. > > Yeah, it's not clear why these need to be block-sized. We shouldn't > perpetuate this without more clarity about this. If we change something, we should consider making buffers like these aligned to page sizes, rather than just MAXALIGNED. Greetings, Andres Freund

Re: [PATCH] plpython function causes server panic

2023-12-01 Thread Andres Freund
.g. doesn't immediately look like plperl's usage would be affected in a similar way? Greetings, Andres Freund

Re: pgsql: meson: docs: Add {html,man} targets, rename install-doc-*

2023-12-01 Thread Andres Freund
Hi, On 2023-12-01 15:55:29 -0600, Tristan Partin wrote: > Commits look fine to me, but I hate the new target names... You shouldn't ever need to use them anywhere - that's what the alias is for... Happy to go another route if you have a suggestion. > > +for name, v in

Re: Building PosgresSQL with LLVM fails on Solaris 11.4

2023-12-01 Thread Andres Freund
nly case where .s files matter today is building with sun studio, and that's a compiler we're planning to remove support for. So I'm not sure it's worth fixing, if it adds complexity. Greetings, Andres Freund diff --git i/src/backend/port/Makefile w/src/backend/port/Makefile index 47338d99229..005921

Re: meson: Stop using deprecated way getting path of files

2023-11-30 Thread Andres Freund
On 2023-11-29 10:50:53 -0800, Andres Freund wrote: > I plan to apply this soon, unless I hear some opposition / better ideas / Pushed.

Re: Annoying build warnings from latest Apple toolchain

2023-11-30 Thread Andres Freund
Hi, On 2023-11-30 21:24:21 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-11-28 10:48:04 -0500, Robert Haas wrote: > >> What a stupid, annoying decision on Apple's part. It seems like > >> -Wl,-undefined,error is the default behavior, so they could have

Re: optimize atomic exchanges

2023-11-30 Thread Andres Freund
0% speedup with this patch on x86_64. On bigger machines, with contention, the wins are likely much higher. I see two orders of magnitude higher throughput in a test program that I had around, on a two socket cascade lake machine. Of course it's also much less powerfull... Greetings, Andres Freund

Re: Annoying build warnings from latest Apple toolchain

2023-11-30 Thread Andres Freund
is, I just noticed that there is -no_warn_duplicate_libraries, which would at least get rid of the ld: warning: ignoring duplicate libraries: '-lxml2' style warnings. Greetings, Andres Freund

Re: Refactoring backend fork+exec code

2023-11-30 Thread Andres Freund
Hi, On 2023-12-01 01:36:13 +0200, Heikki Linnakangas wrote: > On 30/11/2023 22:26, Andres Freund wrote: > > On 2023-11-30 01:36:25 +0200, Heikki Linnakangas wrote: > > > From a96b6e92fdeaa947bf32774c425419b8f987b8e2 Mon Sep 17 00:00:00 2001 > > > From: Heikki Linnakang

Re: PostgreSql: Canceled on conflict out to old pivot

2023-11-30 Thread Andres Freund
t quite often we lack the necessary data, because various backends have stored the "global" xmin in the procarray. Greetings, Andres Freund

Re: Something seems weird inside tts_virtual_copyslot()

2023-11-30 Thread Andres Freund
Hi, On 2023-11-06 11:16:26 +1300, David Rowley wrote: > On Sat, 4 Nov 2023 at 15:15, Andres Freund wrote: > > > > On 2023-11-01 11:35:50 +1300, David Rowley wrote: > > > I changed the Assert in tts_virtual_copyslot() to check the natts > > > match in each of th

Re: Add missing error codes to PANIC/FATAL error reports in xlogrecovery

2023-11-30 Thread Andres Freund
Hi, On 2023-11-30 16:02:55 -0500, Tom Lane wrote: > Andres Freund writes: > > Wondering if we should add a ERRCODE_CLUSTER_CORRUPTED for cases like this. > > We > > have ERRCODE_DATA_CORRUPTED and ERRCODE_INDEX_CORRUPTED, which make > > ERRCODE_DATA_CORRUP

Detecting some cases of missing backup_label

2023-11-30 Thread Andres Freund
the start of recovery. That'd probably make the logic for when we can remove the backup state files a bit more complicated, but I think we could deal with that. Comments? Swear words? Greetings, Andres Freund

Re: Refactoring backend fork+exec code

2023-11-30 Thread Andres Freund
a separate stack, which meant that the set_stack_base() we did in postmaster would yield a completely bogus stack depth estimation. So this comment should likely have been removed. Thomas? Greetings, Andres Freund

Re: Refactoring backend fork+exec code

2023-11-30 Thread Andres Freund
StartChildProcess(WalWriterProcess) > -#define StartWalReceiver() StartChildProcess(WalReceiverProcess) > +#define StartupDataBase()StartChildProcess(PMC_STARTUP) > +#define StartArchiver() StartChildProcess(PMC_ARCHIVER) > +#define StartBackgroundWriter() StartChildProcess(PMC_BGWRITER) > +#define StartCheckpointer() StartChildProcess(PMC_CHECKPOINTER) > +#define StartWalWriter() StartChildProcess(PMC_WAL_WRITER) > +#define StartWalReceiver() StartChildProcess(PMC_WAL_RECEIVER) > + > +#define StartAutoVacLauncher() StartChildProcess(PMC_AV_LAUNCHER); > +#define StartAutoVacWorker() StartChildProcess(PMC_AV_WORKER); Obviously not your fault, but these macros are so pointless... Making it harder to find where we start child processes, all to save a a few characters in one place, while adding considerably more in others. > +void > +BackendMain(char *startup_data, size_t startup_data_len) > +{ Is there any remaining reason for this to live in postmaster.c? Given that other backend types don't, that seems oddly assymmetrical. Greetings, Andres Freund

Re: Add missing error codes to PANIC/FATAL error reports in xlogrecovery

2023-11-30 Thread Andres Freund
recoveryTargetTLI), >errdetail("Latest checkpoint is at %X/%X on > timeline %u, but in the history of the requested timeline, the server forked > off from that timeline at %X/%X.", > > LSN_FORMAT_ARGS(ControlFile->checkPoint), Hm, this one arguably is not corruption, but we still cannot continue. ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE or maybe a new error code? Greetings, Andres Freund

Re: meson: Stop using deprecated way getting path of files

2023-11-29 Thread Andres Freund
it would be great if you could take a > look at this thread[0], where I am trying to compile Postgres with > -fsanitize=address,undefined (-Db_sanitize=address,undefined). Done. Not sure it helps you much though :) Greetings, Andres Freund

Re: remaining sql/json patches

2023-11-29 Thread Andres Freund
Hi, On 2023-11-29 14:21:59 -0500, Andrew Dunstan wrote: > On 2023-11-29 We 12:42, Andres Freund wrote: > > > I do have a CFLAGS setting, but for meson I used '-Ddebug=true' and no > > > buildtype  or optimization setting. However, I see that in meson.build > &g

Re: Fix some memory leaks in ecpg.addons

2023-11-29 Thread Andres Freund
rying to do something around this is the basically non-existing documentation around all of this. I haven't even found documentation referencing the fact that there are headers like sanitizer/asan_interface.h, you just have to figure that out yourself. Compare that to something like valgrind, which has documented this at least somewhat. Greetings, Andres Freund

Re: Fix some memory leaks in ecpg.addons

2023-11-29 Thread Andres Freund
ams. > > Are people using some suppression file or setting ASAN_OPTIONS to something? You pretty much have to. Locally I use this: export ASAN_OPTIONS='debug=1:print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0:detect_stack_use_after_return=0' UBSAN_OPTIONS='print_stacktrace=1:disable_coredump=0:abort_on_error=1' CI uses something similar. Greetings, Andres Freund

Re: Fix assertion in autovacuum worker

2023-11-29 Thread Andres Freund
Hi, On 2023-11-29 11:52:01 -0600, Nathan Bossart wrote: > On Tue, Nov 28, 2023 at 06:48:59PM -0800, Andres Freund wrote: > > On 2023-11-28 20:42:47 -0600, Nathan Bossart wrote: > >> Right. Perhaps we could add a LWLockReleaseAll() to > >> pgstat_shutdown_hook() ins

meson: Stop using deprecated way getting path of files

2023-11-29 Thread Andres Freund
some opposition / better ideas / Greetings, Andres Freund >From d296c088ceadeac281a54d6f7f748112c5efb33a Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 29 Nov 2023 10:47:20 -0800 Subject: [PATCH v1] meson: Stop using deprecated way getting path of files The just released meson

Re: pgsql: meson: docs: Add {html,man} targets, rename install-doc-*

2023-11-29 Thread Andres Freund
Hi, This started at https://www.postgresql.org/message-id/746ba786-85bb-d1f7-b613-57bec35c642a%40dunslane.net but seems worth discussing on -hackers. On 2023-11-29 07:20:59 -0500, Andrew Dunstan wrote: > On 2023-11-28 Tu 21:28, Andres Freund wrote: > > On 2023-11-23 08:32:21 -050

Re: remaining sql/json patches

2023-11-29 Thread Andres Freund
Hi, On 2023-11-29 07:37:53 -0500, Andrew Dunstan wrote: > On 2023-11-28 Tu 21:10, Andres Freund wrote: > > Hi, > > > > On 2023-11-28 20:58:41 -0500, Andrew Dunstan wrote: > > > On 2023-11-28 Tu 19:32, Tom Lane wrote: > > > > Andrew Dunstan writes:

Re: Whose Cirrus CI credits are used when making a PR to the GitHub mirror?

2023-11-29 Thread Andres Freund
I am currently not too worried about this threat. I do wish github would allow disabling PRs... Greetings, Andres Freund

Re: Fix assertion in autovacuum worker

2023-11-28 Thread Andres Freund
Hi, On 2023-11-28 20:42:47 -0600, Nathan Bossart wrote: > On Tue, Nov 28, 2023 at 04:03:49PM -0800, Andres Freund wrote: > > On 2023-11-28 16:05:16 -0600, Nathan Bossart wrote: > >> From a glance, it looks to me like the problem is that pgstat_shutdown_hook

<    1   2   3   4   5   6   7   8   9   10   >