Hi,
GlobalVisTestNonRemovableHorizon()/GlobalVisTestNonRemovableFullHorizon() only
existed for snapshot_too_old - but that was removed in f691f5b80a8. I'm
inclined to think we should remove those functions for 17. No new code should
use them.
Greetings,
Andres Freund
age/buffer/bufmgr.c.gcov.html#L5978
...
Greetings,
Andres Freund
#2333
Greetings,
Andres Freund
nfortunately the logs aren't accessible anymore, so I can't check the precise
patch level of the compiler and/or the precise invocation used.
Greetings,
Andres Freund
Thoughts?
On the code level, I thought if it'd be good to have a common prefix for all
the automatically set variables. Right now that's CI_, but I'm not at all
wedded to that.
Greetings,
Andres Freund
>From 5d26ecfedbcbc83b4cb6e41a34c1af9a3ab24cdb Mon Sep 17 00:00:00 2001
From: Andres Freund
he automated image generation
for CI:
https://github.com/anarazel/pg-vm-images/blob/main/scripts/windows_install_mingw64.ps1#L21-L22
I wonder if we should maintain something like that list somewhere in the
postgres repo instead...
Greetings,
Andres Freund
Hi,
On 2024-04-12 11:33:17 -0700, Andres Freund wrote:
> I wonder if, for easy backpatching, the easiest solution is to just reset
> errno before calling pg_usleep(), and only increment status->delays if
> errno != EINTR. Given our pg_usleep() implementations, that'd preven the stuc
Hi,
Given that I found several ways to spuriously trigger the stuck spinlock
logic, I think we need to backpatch a fix.
On 2024-04-11 21:41:39 -0700, Andres Freund wrote:
> Tracking the total amount of time spent sleeping doesn't require any
> additional syscalls, due to the nanosleep()
lots of creaky old code
- libpam - undermaintained, lots of creaky old code
- the rest
Greetings,
Andres Freund
Hi,
On 2024-04-12 09:43:46 -0400, Tom Lane wrote:
> Andres Freund writes:
> > Oh my. There's a workload that completely trivially hits this, without even
> > trying hard. LISTEN/NOTIFY.
>
> Hm. Bug in the NOTIFY logic perhaps?
I don't think it's a bug, async.c:SignalB
Hi,
On 2024-04-11 21:41:39 -0700, Andres Freund wrote:
> FWIW, I just reproduced the scenario with signals. I added tracking of the
> total time actually slept and lost to SpinDelayStatus, and added a function to
> trigger a wait on a spinlock.
>
> To wait less, I set max_standby_
Hi,
On 2024-04-11 20:47:37 -0700, Andres Freund wrote:
> > So there's that. But that's not an argument that we need to be in a
> > hurry to timeout; if the built-in reaction time is less than perhaps
> > 10 minutes you're still miles ahead of the manual solution.
>
e
released on error. Lwlocks also take e.g. care to hold interrupts so code
doesn't just jump out of a section with lwlocks held.
Greetings,
Andres Freund
Hi,
On 2024-04-11 15:07:11 -0700, Andres Freund wrote:
> On 2024-04-11 16:35:58 -0400, Tom Lane wrote:
> > Indeed. I recall reading, not long ago, some Linux kernel docs to the
> > effect that automatic stack growth is triggered by a reference into
> > the page just be
Hi,
On 2024-04-11 16:35:58 -0400, Tom Lane wrote:
> Andres Freund writes:
> > d8f5acbdb9b made me wonder if we should add a compiler option to warn when
> > stack frames are large. gcc compatible compilers have -Wstack-usage=limit,
> > so
> > that's not ha
Hi,
On 2024-04-11 16:46:23 -0400, Robert Haas wrote:
> On Thu, Apr 11, 2024 at 3:52 PM Andres Freund wrote:
> > My suspicion is that most of the false positives are caused by lots of
> > signals
> > interrupting the pg_usleep()s. Because we measure the number of delays, no
Hi,
On 2024-04-11 16:46:10 -0400, Tom Lane wrote:
> Andres Freund writes:
> > On 2024-04-11 16:11:40 -0400, Tom Lane wrote:
> >> We wouldn't need to fix it, if we simply removed the NUM_DELAYS
> >> limit. Whatever kicked us off the sleep doesn't matter, we
right that this post feature-freeze works on
> designing a new API? Yes, 27bc1772fc masked the problem. But it was
> committed on Mar 30.
Note that there were versions of the patch that were targeting the
pre-27bc1772fc interface.
Greetings,
Andres Freund
Hi,
On 2024-04-11 16:11:40 -0400, Tom Lane wrote:
> Andres Freund writes:
> > On 2024-04-11 15:24:28 -0400, Robert Haas wrote:
> >> Or, rip out the whole, whole mechanism and just don't PANIC.
>
> > I continue believe that that'd be a quite bad idea.
>
> I
Hi,
On 2024-04-11 15:16:57 -0400, Andrew Dunstan wrote:
> On 2024-04-11 Th 15:01, Andres Freund wrote:
> > [1345/1940 42 69%] Compiling C object
> > src/bin/pg_verifybackup/pg_verifybackup.p/pg_verifybackup.c.o
> > ../../../../../home/andres/src/postgresql/sr
have come up with a way
to do away with that spinlock.
OTOH, there are plenty other lwlocks where we pay the price of lwlocks being
an rwlock, but just use the exclusive mode. So perhaps we should just add a
exclusive-only lwlock variant.
Greetings,
Andres Freund
r than one that's just
down.
Greetings,
Andres Freund
4400 bytes [-Wstack-usage=]
742 | GetFileBackupMethod(IncrementalBackupInfo *ib, const char *path,
| ^~~
I don't really have an opinion about the concrete warning limit to use.
Greetings,
Andres Freund
.
Could you expand a bit on the concrete scenario you're worried about here?
PG_TRY/CATCH aren't free, so adding something like this to a quite common
path, in the back branches, without a concrete analysis as to why it's needed,
seems a bit scary.
Greetings,
Andres Freund
Hi,
On 2024-04-10 16:50:44 -0400, Melanie Plageman wrote:
> This brings up a question about the prefetching. We never had to have
> this discussion for sequential scan streaming read because it didn't
> (and still doesn't) do prefetching. But, if we push the streaming read
> code down into the
reaming read) to review? Is it
> v1-0001-revert-Generalize-relation-analyze-in-table-AM-in.patch?
I think so. It's at least what I've been looking at.
Greetings,
Andres Freund
Hi,
On 2024-04-10 16:05:21 -0400, Tom Lane wrote:
> Andres Freund writes:
> > I think it could exascerbate the issue. Parag reported ~7k connections on a
> > 128 core machine. The buffer replacement logic in < 16 tries to lock the old
> > and new lock partitions at once
ich is probably not needed. At the same time I can't get this error
> from the verify pass with LLVM-12 or LLVM-15 (I have those at hand by
> accident). To make it even more confusing I've found a few similar
> examples in other projects, where this was really triggered by an issue
> in LLVM [1].
I'm afraid that it actually has changed over time, I'm fairly sure that it was
required at some point.
Greetings,
Andres Freund
o it made sense for 041b96802e
to create the stream in acquire_sample_rows() and have
block_sampling_read_stream_next() be in analyze.c. But eventually that should
be in access/heap/. Compared to 16, the state post the revert does tie
analyze.c a bit closer to the internals of the AM than before, but I'm not
sure the increase matters.
Greetings,
Andres Freund
greatly exacerbated by the issue fixed in a4adc31f690, once the
waitqueue is long, the spinlock will be held for an extended amount of time.
Greetings,
Andres Freund
own, without coordination.
That obviously isn't a reason to not fix the delay logic in lwlock.c.
Looks like the wrong logic was introduced by me in
commit 008608b9d51061b1f598c197477b3dc7be9c4a64
Author: Andres Freund
Date: 2016-04-10 20:12:32 -0700
Avoid the use of a separate spinloc
ock usage pattern that is
> causing this spot to become so heavily contended.)
My suspicion is that it's a4adc31f690 which we only recently backpatched to
< 16.
Greetings,
Andres Freund
h to manager our ratelimiting counters.
What version of PG was this? I think it's much more likely that you're
hitting a bug that caused a lot more contention inside lwlocks. That was fixed
for 16+ in a4adc31f690 on 2022-11-20, but only backpatched to 12-15 on
2024-01-18.
Greetings,
Andres Freund
he commit message it's hard
to evaluate how this actually intended to be used.
Greetings,
Andres Freund
m that RNG unfairness is a real part of the problem here.
Greetings,
Andres Freund
Hi,
On 2024-04-09 20:12:48 -0400, Tom Lane wrote:
> Andres Freund writes:
> > FWIW, just redefining mm_strdup() that way doesn't help much here either,
> > even with an affected compiler. The gain increases substantially after
> > simplifying unreserved_keyword etc to just u
clang-19 -O2
"plain"0m24.354s
mm_strdup redefined0m23.741s
+use default action 0m14.218s
Greetings,
Andres Freund
Hi,
On 2024-04-09 19:00:41 -0400, Tom Lane wrote:
> Andres Freund writes:
> > I think we need to do something about the compile time of this file, even
> > with
> > gcc. Our main grammar already is an issue and stacking all the ecpg stuff on
> > top makes it co
Hi,
On 2024-04-09 15:33:10 -0700, Andres Freund wrote:
> Which leads to a control flow graph with *many* incoming edges to the basic
> block containing the function call to mm_strdup(), triggering a normally
> harmless O(N^2) or such.
With clang-16 -O2 there is a basic block with 3904
ng-16 -O2 from 345.188 to 158.084s
clang-19 -O2 from 26.018s to 15.200s
I suspect what is happening is that clang tries to optimize the number of
calls to mm_strdup(), by separating the argument setup from the function
call. Which leads to a control flow graph with *many* incoming edges to the
basic block containing the function call to mm_strdup(), triggering a normally
harmless O(N^2) or such.
Greetings,
Andres Freund
, I did some
> statistical analysis on the RNG generator that PG uses to create the
> backoff for the spin locks.
ISTM that the fix here is to not use a spinlock for whatever the contention is
on, rather than improve the RNG.
Greetings,
Andres Freund
tiation. Might
make it slightly easier to notice obsoleted uses of simplehash.
Greetings,
Andres Freund
Hi,
Looks like I forgot to update the thread to note that I finally committed the
remaining warning fixes. I had already fixed a bunch of others in upstream
meson.
commit a3da95deee38ee067b0bead639c830eacbe894d5
Author: Andres Freund
Date: 2024-03-13 01:40:53 -0700
meson: macos: Avoid
n without a problem.
Perhaps it's worth picking that up again?
https://github.com/anarazel/postgres/tree/meson-rpath
https://github.com/anarazel/postgres/commit/46f1963fee7525c3cc3837ef8423cbf6cb08d10a
Greetings,
Andres Freund
Hi,
On 2024-04-08 16:01:41 +0530, Amit Kapila wrote:
> Pushed.
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder=2024-04-08%2012%3A04%3A27
This unfortunately is a commit after
commit 6f3d8d5e7cc
Author: Amit Kapila
Date: 2024-04-08 13:21:55 +0530
Fix the intermittent
On 2024-04-08 08:37:44 -0700, Andres Freund wrote:
> On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote:
> > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov
> > > I was under the impression there are not so many out-of-core table
> > > AMs, which have non-dummy analysis
the new API doesn't
look well polished either, so it's not a question of a smoother transition or
something like that.
I don't think redesigning extension APIs at this stage of the release cycle
makes sense.
Greetings,
Andres Freund
Hi,
On 2024-04-08 09:26:09 -0400, Robert Haas wrote:
> On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote:
> And maybe we need to think of a way to further mitigate this crush of
> last minute commits. e.g. In the last week, you can't have more
> feature commits, or more lines of insertions in
Hi,
On 2024-04-08 16:18:21 +1200, David Rowley wrote:
> On Mon, 8 Apr 2024 at 16:08, Andres Freund wrote:
> > I think visible would be ok, the serialization checks are IMO about
> > visibility too. But if you'd prefer I'd also be ok with something like
> > page_collect_tup
Hi,
On 2024-04-08 15:43:12 +1200, David Rowley wrote:
> On Mon, 8 Apr 2024 at 15:13, Andres Freund wrote:
> > Off-list Melanie suggested heap_page_collect_visible_tuples(). Given the
> > separate callsites (making long names annoying) and the fact that it's
> > really
>
Hi,
On 2024-04-08 14:43:21 +1200, David Rowley wrote:
> On Sun, 7 Apr 2024 at 19:30, Andres Freund wrote:
> > Good call. Added and pushed.
>
> I understand you're already aware of the reference in the comment to
> heapgetpage(), which no longer exists as of 44086b097.
Yea, h
On 2024-04-07 21:41:23 -0400, Tom Lane wrote:
> Andres Freund writes:
> > I can't see a way it could hurt in the back branches, so I'm inclined to
> > backpatch the pfree?
>
> +1
Done
t now. If coverity had started to complain
after da41d71070d, I'd understand, but that was ~5 years ago.
I can't see a way it could hurt in the back branches, so I'm inclined to
backpatch the pfree?
Greetings,
Andres Freund
useful to be able to evict arbitrary blocks" that seems to describe
things at least as well as "invalidate"?
Greetings,
Andres Freund
Hi,
On 2024-04-08 02:25:17 +0300, Alexander Korotkov wrote:
> On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote:
> > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote:
> > > I've pushed 0001, 0002 and 0006.
> >
> > I briefly looked at 27bc1772fc81 an
nteresting for Makefile.global - they're msvc flags - because
you can't use that with msvc.
I'm also doubtful that it's worth supporting warning_level=3/everything, you
end up with a completely flood of warnings that way.
Greetings,
Andres Freund
total_allocated ==
> context->mem_allocated"), File: "bump.c", Line: 808, PID: 7809
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=turaco=2024-04-07%2022%3A42%3A54
What makes you think that's unrelated? To me that looks like the same issue?
Greetings,
Andres Freund
etty much
free to call another callback there.
Greetings,
Andres Freund
gt;
> Author: Melanie Plageman, Nazir Bilal Yavuz
> Discussion:
> https://postgr.es/m/flat/CAN55FZ0UhXqk9v3y-zW_fp4-WCp43V8y0A72xPmLkOM%2B6M%2BmJg%40mail.gmail.com
The justification here seems somewhat odd. Sure, the previous commit stopped
using BlockSampler_HasMore in acquire_sample_rows - but only because it was
moved to block_sampling_streaming_read_next()?
Greetings,
Andres Freund
res
reimplementing all of analyze.c.
What am I missing here?
Greetings,
Andres Freund
ext->mem_allocated);
> (gdb) p total_allocated
> $1 = 8120
> (gdb) p context->mem_allocated
> $2 = 8192
Greetings,
Andres Freund
ut you're storing a BlocktableEntry in
a char[]. You can't just do that. Look at how we do that for
e.g. PGAlignedblock.
With the attached minimal fix, the tests pass again.
Greetings,
Andres Freund
diff --git i/src/backend/access/common/tidstore.c w/src/backend/access/common/tidstore.c
index e1
On 2024-04-07 18:51:40 +0200, Daniel Gustafsson wrote:
> > On 7 Apr 2024, at 18:28, Andres Freund wrote:
> >
> > On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote:
> >>> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote:
> >>> On 2024-04-06 Sa 20:49
On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote:
> > On 7 Apr 2024, at 14:51, Andrew Dunstan wrote:
> > On 2024-04-06 Sa 20:49, Andres Freund wrote:
>
> >> That's probably unnecessary optimization, but it seems a tad silly to read
> >> an
> >> ent
Hi,
On 2024-04-07 12:07:22 +0700, John Naylor wrote:
> Just in time ;-) The commit message should also have "reviewed by
> Zhang Mingli" and "tested by Quan Zongliang", who shared results in a
> thread for a withrawn CF entry with a similar idea but covering fewer
> cases:
Good call. Added and
Hi,
On 2024-01-22 13:01:31 +0700, John Naylor wrote:
> On Mon, Jul 17, 2023 at 9:58 PM Andres Freund wrote:
> > And 397->320ms
> > for something as core as this, is imo worth considering on its own.
>
> Hi Andres, this interesting work seems to have fallen off the
Hi,
On 2024-04-07 00:45:31 +0200, Jelte Fennema-Nio wrote:
> On Sat, 6 Apr 2024 at 22:21, Andres Freund wrote:
> > The small regression for small results is still kinda visible, I haven't yet
> > tested the patch downthread.
>
> Thanks a lot for the faster test script,
Hi,
On 2024-04-07 00:19:35 +0200, Daniel Gustafsson wrote:
> > On 6 Apr 2024, at 23:44, Andres Freund wrote:
>
> > It might be useful to print a few lines, but the whole log files can be
> > several megabytes worth of output.
>
> The non-context aware fix would be
he "normal" contents of regress_log* continue.
Greetings,
Andres Freund
n a bunch of failures of this test locally.
Greetings,
Andres Freund
l results is still kinda visible, I haven't yet
tested the patch downthread.
Greetings,
Andres Freund
#!/bin/bash
set -e
dbname=postgres
port=5440
host=/tmp
host=localhost
test_cases=(
"100 100 100" # only 100 bytes
"1024 10240 10"# 1Kb and 10Kb
&
g about this from our side as well, I
suspect. The times aren't great with gcc either, even if not as bad as with
clang.
Greetings,
Andres Freund
Hi,
On 2024-04-04 19:00:00 +0300, Alexander Lakhin wrote:
> 26.03.2024 10:59, Andres Freund wrote:
> > Late, will try to look more in the next few days.
> >
>
> AFAICS, last 027_streaming_regress.pl failures on calliphoridae,
> culicidae, tamandua occurred be
efault at
some point - but we probably should overhaul the infrastructure first...
Greetings,
Andres Freund
Hi,
On 2024-04-03 17:58:55 -0400, Tom Lane wrote:
> Magnus Hagander writes:
> > On Wed, Apr 3, 2024 at 7:57 PM Andres Freund wrote:
> >> Openssh has now integrated [1] a patch to remove the dependency on
> >> libsystemd
> >> for triggering service manager
kes that effect worse in.a BAS_VACUUM strategy,
> by taking away space that is effectively deferring WAL flushes, so I'd at
> least like to double the size for if we use "/ 2" above.
I think for VACUUM we should probably go a bit further. There's no comparable
L1/L2 issue, because the per-buffer processing + WAL insertion is a lot more
expensive, compared to a seqscan. I'd go or at lest 4x-8x.
Greetings,
Andres Freund
etings,
Andres Freund
that can rely on the server
actually having started up.
Greetings,
Andres Freund
[1] https://bugzilla.mindrot.org/show_bug.cgi?id=2641
[2]
https://www.freedesktop.org/software/systemd/man/devel/systemd.html#Readiness%20Protocol
[3] https://www.freedesktop.org/software/systemd/man/devel
an empty string but can
be overridden?
Greetings,
Andres Freund
parately
downloadable archive. We already stopped including other generated files
recently.
> We shouldn't make the mistake of assuming that bad things can't happen to
> us.
+1
Greetings,
Andres Freund
hard to hide
something sneaky. I comparison to that hiding something in configure.ac seems
less likely to succeed IMO, that imo tends to be more scrutinized. And hiding
just in configure directly wouldn't get you far, it'd just get removed when
the committer or some other committer at a later time, regenerates configure.
Greetings,
Andres Freund
look more in the next few days.
Greetings,
Andres Freund
Hi,
On 2024-03-26 08:36:58 +0100, Peter Eisentraut wrote:
> On 26.03.24 01:23, Andres Freund wrote:
> > On 2024-03-25 06:44:33 +0100, Peter Eisentraut wrote:
> > > Done and committed.
> >
> > This triggered a new warning for me:
> >
> > ../../../../../h
Hi,
On 2024-03-26 00:00:38 -0400, Tom Lane wrote:
> Andres Freund writes:
> > I think there must be some actual regression involved. The frequency of
> > failures on HEAD vs failures on 16 - both of which run the tests
> > concurrently
> > via meson - is just vastly
Hi,
On 2024-03-20 17:41:45 -0700, Andres Freund wrote:
> On 2024-03-14 16:56:39 -0400, Tom Lane wrote:
> > Also, this is probably not
> > helping anything:
> >
> >
Hi,
On 2024-03-24 11:28:12 -0400, Tom Lane wrote:
> Heikki Linnakangas writes:
> > On 19/09/2023 01:57, Andres Freund wrote:
> >> On 2023-09-18 13:49:24 +0300, Heikki Linnakangas wrote:
> >>> d) Copy fewer rows to the table in the test. If we copy only 6 rows
to do this?
No. I think it's acceptable though.
However it might be worth inventing an executor tree walker in a preliminary
step. We have already quite a few switches over all plan nodes, which we could
largely replace with a helper.
Greetings,
Andres Freund
Hi,
On 2024-03-25 06:44:33 +0100, Peter Eisentraut wrote:
> Done and committed.
This triggered a new warning for me:
../../../../../home/andres/src/postgresql/meson.build:3422: WARNING: Project
targets '>=0.54' but uses feature introduced in '0.55.0': Passing
executable/found program object
d what your patch is doing, but perhaps
the issue is that it's seeing global objects concurrently created by another
test?
Greetings,
Andres Freund
--cross-file) might give you a start:
[properties]
needs_exe_wrapper = false
[binaries]
c = 'cl'
cpp = 'cl'
ar = 'lib'
windres = 'rc'
[host_machine]
system = 'windows'
cpu_family = 'x86_64'
cpu = 'x86_64'
endian = 'little'
Greetings,
Andres Freund
Hi,
On 2024-03-22 15:02:45 -0400, Robert Haas wrote:
> On Fri, Mar 22, 2024 at 2:38 PM Andres Freund wrote:
> > I wonder if we ought to make configure warn if it sees -Werror in CFLAGS -
> > this is far from the first time somebody stumbling with -Werror. Including a
> > few
to use Makefile.custom.
Greetings,
Andres Freund
s an unrelated warning triggering the configure test to
fail. We'd need to see config.log to see what that is.
Greetings,
Andres Freund
On 2024-02-28 14:59:16 -0600, Nathan Bossart wrote:
> On Wed, Feb 28, 2024 at 02:21:49PM -0600, Nathan Bossart wrote:
> > On Wed, Feb 28, 2024 at 02:07:34PM -0600, Andrew Atkinson wrote:
> >> I agree that starting with the direct conversion is reasonable. Markdown
> >> "modernizes" the file using
s
Use --disable-stdcall-fixup to disable these fixups
/usr/bin/i686-w64-mingw32-ld: warning: resolving _DllUnregisterServer by
linking to _DllUnregisterServer@0
Greetings,
Andres Freund
Hi,
On 2024-03-20 17:41:45 -0700, Andres Freund wrote:
> 2024-03-20 22:14:01.904 UTC [56343][client backend][6/1925:0] LOG:
> connection authorized: user=bf database=postgres
> application_name=027_stream_regress.pl
> 2024-03-20 22:14:01.930 UTC [56343][client backend][6
Hi,
On 2024-03-20 17:41:47 -0700, Andres Freund wrote:
> There's a lot of other animals on the same machine, however it's rarely fuly
> loaded (with either CPU or IO).
>
> I don't think the test just being slow is the issue here, e.g. in the last
> failing iteration
>
> [
le coverage. I probably could
reduce the number of animals using it though.
Greetings,
Andres Freund
Hi,
On 2024-03-21 11:02:27 +1300, David Rowley wrote:
> On Thu, 21 Mar 2024 at 11:00, Andres Freund wrote:
> >
> > On 2024-03-20 17:49:14 -0400, Dave Cramer wrote:
> > > First off this is on an ARM64 machine
> >
> > Uh, that's a fairly crucial bit - you
Hi,
On 2024-03-20 17:49:14 -0400, Dave Cramer wrote:
> On Wed, 20 Mar 2024 at 17:11, Andres Freund wrote:
> > On 2024-03-20 16:14:23 -0400, Dave Cramer wrote:
> > > I am getting the following error
> > >
> > > meson.build:1479:17: ERROR: Can not
101 - 200 of 8855 matches
Mail list logo
