rity finds
suspicious. It's possible that both of those scan keys actually did
have arrays, but _bt_compare_scankey_args just treats that as a case
of being unable to prove which scan key was redundant/contradictory
due to a lack of suitable cross-type support -- so the assertion won't
be reached.
Would Coverity stop complaining if I just removed the assertion? I
could just do that, I suppose, but that seems backwards to me.
--
Peter Geoghegan
SEARCHARRAY scan keys mean that _bt_preprocess_array_keys
* failed to eliminate redundant arrays through array merging.
* _bt_compare_scankey_args just returns false when it sees
* this; it won't even try to examine either array.
*/
Do you think it needs more work?
--
Peter Geoghegan
rray->scan_key);
> Assert(!(cur->sk_flags & SK_SEARCHARRAY));
> }
>
> those first two Asserts are redundant with the "if" as well.
I'll get rid of those other two assertions as well, then.
--
Peter Geoghegan
On Sun, Apr 7, 2024 at 9:57 PM Peter Geoghegan wrote:
> On Sun, Apr 7, 2024 at 9:50 PM Tom Lane wrote:
> > those first two Asserts are redundant with the "if" as well.
>
> I'll get rid of those other two assertions as well, then.
Done that way.
--
Peter Geoghegan
e nothing
to say about the importance of these sorts of cases. Most of these
cases will only have 2 or 3 constants, just because that's what's most
common in general.
--
Peter Geoghegan
ignificant advantage in its own right.
ISTM that the planner should always prefer index quals over expression
evaluation, on general principle, even when there's no reason to think
it'll work out. At worst the executor has essentially the same
physical access patterns as the expression evaluation case. On the
other hand, providing nbtree with that context might end up being a
great deal faster.
--
Peter Geoghegan
ng early is a good idea. Of course it also
wouldn't be a bad idea to have a BF animal for that, especially
because we already have BF animals that test things far more niche
than this.
--
Peter Geoghegan
years. I think that he deserves some
consideration here. Say a week or two, to work through some of the
more complicated issues -- and to take a breather. I just don't see
any upside to rushing through this process, given where we are now.
--
Peter Geoghegan
ight
prevent _bt_killitems() from setting LP_DEAD bits. It's totally
unsurprising that breaking kill_prior_tuple in some way could be
missed. Andres wrote the MVCC test in question precisely because
certain aspects of kill_prior_tuple were broken for months without
anybody noticing.
[1] https://www.postgresql.org/docs/devel/index-locking.html
--
Peter Geoghegan
ncurrent TID recycling). We
conservatively do nothing (don't mark any index tuples LP_DEAD),
unless the LSN is exactly the same as it was back when the page was
scanned/read by _bt_readpage().
So some accidental detail with LSNs (like using or not using an
unlogged index) could cause bugs in this area to "accidentally fail to
fail". Since the nbtree index AM has its own optimizations here, which
probably has a tendency to mask problems/bugs. (I sometimes use
unlogged indexes for some of my nbtree related test cases, just to
reduce certain kinds of variability, including variability in this
area.)
[1]
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/nbtree/README;h=52e646c7f759a5d9cfdc32b86f6aff8460891e12;hb=3e8235ba4f9cc3375b061fb5d3f3575434539b5f#l443
--
Peter Geoghegan
ct that we'll need to share some limited information across
different layers of abstraction, because that's just fundamentally
what's required by the constraints we're operating under. Can't really
prove it, though.
--
Peter Geoghegan
the
> page would obviously be better.
Quite possibly, yes. But it's hard to say for sure without far more
detailed analysis. Plus you have problems with things like unlogged
indexes not having an LSN to use as a canary condition, which makes it
a bit messy (it's already kind of weird that we treat unlogged indexes
differently here IMV).
--
Peter Geoghegan
On Wed, Feb 14, 2024 at 7:28 PM Andres Freund wrote:
> On 2024-02-13 14:54:14 -0500, Peter Geoghegan wrote:
> > This property of index scans is fundamental to how index scans work.
> > Pinning an index page as an interlock against concurrently TID
> > recycling by VACUUM is
On Thu, Feb 15, 2024 at 9:36 AM Tomas Vondra
wrote:
> On 2/15/24 00:06, Peter Geoghegan wrote:
> > I suppose that it might be much more important than I imagine it is
> > right now, but it'd be nice to have something a bit more concrete to
> > go on.
> >
>
>
ly simple one.
> Yeah. The basic idea was that by moving this above index AM it will work
> for all indexes automatically - but given the current discussion about
> kill_prior_tuple, locking etc. I'm not sure that's really feasible.
>
> The index AM clearly needs to have more control over this.
Cool. I think that that makes the layering question a lot clearer, then.
--
Peter Geoghegan
isible, we know that we don't need to kill index entries, and thus can
> move on to the next leaf page
It's possible that we'll need a variety of different strategies.
nbtree already has two such strategies in _bt_killitems(), in a way.
Though its "Modified while not pinned means hinting is not safe" path
(LSN doesn't match canary value path) seems pretty naive. The
prefetching stuff might present us with a good opportunity to replace
that with something fundamentally better.
--
Peter Geoghegan
it's in no way equivalent to the debug #ifdef USE_ASSERT_CHECKING
case). It's also just really hard to understand what's going on here.
If I was going to do this kind of thing, I'd use two completely
separate loops, that were obviously completely separate (maybe even
two functions). I'd then memcmp() each array at the end.
--
Peter Geoghegan
aints that the tooling makes are
false positives. At least in some deeper sense.
--
Peter Geoghegan
ressions, that more or
less worked as a reference of agreed upon best practices. Can we do
that part first, rather than starting out with a blanket assumption
that everything that happened before now must have been perfect?
--
Peter Geoghegan
explaining the _bt_killitems IOS issue, that actually seemed to make
sense.
What you really want to do here is to balance costs and benefits.
That's just what's required. The fact that those costs and benefits
span multiple levels of abstractions makes it a bit awkward, but
doesn't (and can't) change the basic shape of the problem.
--
Peter Geoghegan
On Wed, Mar 6, 2024 at 4:46 PM Matthias van de Meent
wrote:
> On Wed, 6 Mar 2024 at 01:50, Peter Geoghegan wrote:
> > I think that there is supposed to be a closing parenthesis here? So
> > "... (such as those described in ") might
> > perform...".
>
&
ade
sense (a bit of sense) back when _bt_preprocess_keys was subordinate
to _bt_preprocess_array_keys, but it's kinda the other way around now.
We could probably even get rid of this remaining limited form of
arrayKeyData, but that doesn't seem like it would add much.
--
Peter Geoghegan
ing the case where we just have the
VM's all-frozen bit set for a given block (not the all-visible bit
set) -- which is always wrong. There was good reason to be concerned
about that possibility when 980ae17310 went in.
--
Peter Geoghegan
at's now located at the end of lazy_scan_prune. Perhaps the two
distinct blocks that call visibilitymap_set() could be combined into
one.
--
Peter Geoghegan
On Fri, Mar 8, 2024 at 11:00 AM Peter Geoghegan wrote:
> Seems like it might be possible to simplify/consolidate the VM-setting
> code that's now located at the end of lazy_scan_prune. Perhaps the two
> distinct blocks that call visibilitymap_set() could be combined into
> one.
of _bt_search() (obviously not doable for
non-_bt_first callers, which need to call _bt_binsrch_insert instead).
This whole approach will have been made easier by the refactoring I
did late last year, in commit c9c0589fda.
--
Peter Geoghegan
On Fri, Mar 8, 2024 at 2:14 PM Peter Geoghegan wrote:
> What benchmarking have you done here?
I think that the memcmp() test is subtly wrong:
> + if (PointerIsValid(rightsep))
> + {
> + /*
> +* Note: we're not in the rightmost page (see
apparent in L&Y itself.
But the Lanin & Shasha paper has a far more optimistic approach. They
make rather bold claims about how many locks they can get away with
holding at any one time. That makes it significantly different to L&Y
as well as nbtree (nbtree is far closer to L&Y than it is to Lanin &
Shasha).
--
Peter Geoghegan
t it would be particularly
compelling, since L&Y/nbtree don't need to do lock coupling for the
vast majority of individual inserts or searches.
I don't think that the L&Y paper is particularly clear, or
particularly well written. It needs to be interpreted in its original
context, which is quite far removed from the current concerns of
nbtree. It's a 41 year old paper.
--
Peter Geoghegan
My problem
isn't with that behavior in general. It's with the fact that we use it
even when it's clearly inappropriate -- wildly inappropriate. We have
plenty of information that strongly hints at whether or not laziness
is a good idea. It's a good idea whenever laziness has a decent chance
of avoiding completely unnecessary work altogether, provided we can
afford to be wrong about that without having to pay too high a cost
later on, when we have to course correct. What this mostly boils down
to is this: lazy freezing is generally a good idea in small tables
only.
--
Peter Geoghegan
we also need to reason
about which kinds of mispredictions we cannot afford to make, and
which kinds are okay. Some mistakes hurt a lot more than others.
--
Peter Geoghegan
On Tue, Dec 13, 2022 at 9:16 AM Peter Geoghegan wrote:
> That's not the only thing we care about, though. And to the extent we
> care about it, we mostly care about the consequences of either
> freezing or not freezing eagerly. Concentration of unfrozen pages in
> one particul
bination of the two, not any one factor on its own [1].
[1]
https://wiki.postgresql.org/wiki/Freezing/skipping_strategies_patch:_motivating_examples#Opportunistically_advancing_relfrozenxid_with_bursty.2C_real-world_workloads
--
Peter Geoghegan
> builds on related added
Fixed.
Thanks
--
Peter Geoghegan
address that in the next version. I'll consider using this
scheme for v10.
[1] https://commitfest.postgresql.org/41/4027/
[2]
https://postgr.es/m/CAH2-Wz=MGFwJEpEjVzXwEjY5yx=uunpza6bt4dsmasrgluq...@mail.gmail.com
[3]
https://postgr.es/m/cah2-wznrzc-ohkb+qzqs65o+8_jtj6rxadjh+8ebqjrd1f8...@mail.gmail.com
[4]
https://towardsdatascience.com/the-inspection-paradox-is-everywhere-2ef1c2e9d709
[5]
https://wiki.postgresql.org/wiki/Freezing/skipping_strategies_patch:_motivating_examples#Scanned_pages.2C_visibility_map_snapshot
--
Peter Geoghegan
ressive AV takes as long as (say) 5 regular autovacuums would
have taken, and if you really needed those 5 separate autovacuums to
run, just to deal with the bloat, then that's a real problem. The
aggressive AV effectively causes bloat with such a workload.
--
Peter Geoghegan
es) along with periodic verify_heapam(). No problems there.
>
> Did you already describe the testing you've done for 0001+0002
> specfiically? It's not radically new logic, but it would be good to try
> to catch minor state-handling errors.
Lots of stuff with contrib/amcheck, which, as you must already know,
will notice when an XID/MXID is contained in a table whose
relfrozenxid and/or relminmxid indicates that it shouldn't be there.
(Though VACUUM itself does the same thing, albeit not as effectively.)
Obviously the invariants haven't changed here. In many ways it's a
very small set of changes. But in one or two ways it's a significant
shift. It depends on how you think about it.
--
Peter Geoghegan
On Tue, Dec 20, 2022 at 7:15 PM Peter Geoghegan wrote:
> On Tue, Dec 20, 2022 at 5:44 PM Jeff Davis wrote:
> > Next, the 'freeze_required' field suggests that it's more involved in
> > the control flow that causes freezing than it actually is. All it does
> >
need to take action here. Perhaps rename the
> variable.
Andres was the one that suggested this name, actually. I initially
just called it "freeze", but I think that Andres had it right.
> I think 0001+0002 are about ready.
Great. I plan on committing 0001 in the next few days. Committing 0002
might take a bit longer.
Thanks
--
Peter Geoghegan
ast in performance sensitive code.
--
Peter Geoghegan
ion is wrong, and simply needs to be removed.
Thanks for the report
--
Peter Geoghegan
ntation) shows that this patch
eliminates 100% of all relevant calls to TransactionIdDidCommit(), for
both the freeze_xmin and the freeze_xmax callsites.
--
Peter Geoghegan
v1-0001-Avoid-unnecessary-clog-lookups-when-freezing.patch
Description: Binary data
ge_is_all_visible() just because hint bits couldn't
be set earlier on, back when lazy_scan_prune() processed the same page
during VACUUM's initial heap pass.
--
Peter Geoghegan
ink of it as a general sanity check.
The important call to avoid with page-level freezing is the xmin call to
TransactionIdDidCommit(), not the xmax call. The xmax call only occurs
when VACUUM prepares to freeze a tuple that was updated by an updater
(not a locker) that aborted. While the xmin calls will now take place with most
unfrozen tuples.
--
Peter Geoghegan
agine that the one-element cache works alright in some scenarios,
but then suddenly doesn't work so well, even though not very much has
changed. Behavior like that makes the problems difficult to analyze,
and easy to miss. I'm suspicious of that.
--
Peter Geoghegan
eze based on
the same observation about the cost. (It already does a certain amount
of this kind of thing, in fact.)
--
Peter Geoghegan
v2-0001-Check-xmin-xmax-commit-status-when-freezing-execu.patch
Description: Binary data
On Thu, Dec 29, 2022 at 12:20 PM Peter Geoghegan wrote:
> > It seems somewhat wrong that we discard all the work that
> > heap_prepare_freeze_tuple() did. Yes, we force freezing to actually happen
> > in
> > a bunch of important cases (e.g. creating a new multixac
On Thu, Dec 29, 2022 at 12:50 PM Peter Geoghegan wrote:
> On Thu, Dec 29, 2022 at 12:20 PM Peter Geoghegan wrote:
> > > It seems somewhat wrong that we discard all the work that
> > > heap_prepare_freeze_tuple() did. Yes, we force freezing to actually
> > > happen
On Fri, Nov 25, 2022 at 2:47 PM Peter Geoghegan wrote:
> Attached WIP patch invents the idea of a regular autovacuum that is
> tasked with advancing relfrozenxid -- which is really just another
> trigger criteria, reported on in the server log in its autovacuum
> reports.
Attached
On Thu, Dec 29, 2022 at 12:20 PM Peter Geoghegan wrote:
> On Thu, Dec 29, 2022 at 12:00 PM Andres Freund wrote:
> > > I could just move the same tests from heap_prepare_freeze_tuple() to
> > > heap_freeze_execute_prepared(), without changing any of the details.
> &g
often the case that "freezing the page" will perform
maximally aggressive freezing, in the sense that it does precisely
what a VACUUM FREEZE would do given the same page (in any Postgres
version).
--
Peter Geoghegan
On Fri, Dec 30, 2022 at 1:12 PM Peter Geoghegan wrote:
> > "Nominal freezing" is happening when there are no freeze plans at all.
> > I get that it's to manage control flow so that the right thing happens
> > later. But I think it should be defined in terms of
On Sat, Dec 31, 2022 at 11:46 AM Jeff Davis wrote:
> On Fri, 2022-12-30 at 16:58 -0800, Peter Geoghegan wrote:
> > Following the path of freezing a page is *always* valid, by
> > definition. Including when there are zero freeze plans to execute, or
> > even zero tuples to
etween
the first and second heap pass, which seems like a clear
maintainability win -- everybody can pass the
already-pinned/already-setup vmbuffer by value.
--
Peter Geoghegan
v1-0001-Avoid-special-XIDs-in-snapshotConflictHorizon-val.patch
Description: Binary data
v1-0002-Never-just-set-the-all-frozen-bit-in-VM.patch
Description: Binary data
On Sat, Dec 31, 2022 at 4:53 PM Peter Geoghegan wrote:
> The first patch makes sure that the snapshotConflictHorizon cutoff
> (XID cutoff for recovery conflicts) is never a special XID, unless
> that XID is InvalidTransactionId, which is interpreted as a
> snapshotConflictHorizon val
On Sat, Dec 31, 2022 at 12:45 PM Peter Geoghegan wrote:
> On Sat, Dec 31, 2022 at 11:46 AM Jeff Davis wrote:
> > "We have no freeze plans to execute, so there's no cost to following
> > the freeze path. This is important in the case where the page is
> > entirely f
(Pruning -committers from the list, since cross-posting to -hackers
resulted in this being held up for moderation.)
On Tue, Jan 3, 2023 at 5:15 PM Peter Geoghegan wrote:
>
> On Tue, Jan 3, 2023 at 4:54 PM Andres Freund wrote:
> > There's some changes from Transactio
ing about similar changes in any
future commit messages, in the unlikely event that I ever end up
moving MarkBufferDirty() around in some existing critical section in
the future.
--
Peter Geoghegan
On Tue, Jan 3, 2023 at 8:29 PM Peter Geoghegan wrote:
> I find this astonishing. Why isn't there a prominent comment that
> advertises that TransactionIdDidAbort() just doesn't work reliably?
I pushed a fix for this now.
We should add a comment about this issue to TransactionIdD
explicit aborts.
That's quite a stretch. There are numerous comments that pretty much
imply that TransactionIdDidCommit/TransactionIdDidAbort are very
similar, for example any discussion of how you need to call
TransactionIdIsInProgress first before calling either of the other
two.
--
Peter Geoghegan
ind reorganizing these
other comments, or making the comment over TransactionIdDidAbort()
mostly just point to the other comments.
--
Peter Geoghegan
e
> thing to change at the same time as something unrelated, particularly without
> even mentioning it?
I changed it because the new order is idiomatic. I didn't think that
this was particularly worth mentioning, or even subtle. The logic from
heap_execute_freeze_tuple() only performs simple in-place
modifications.
--
Peter Geoghegan
ould be doing that don't seem to lead to these situations.
[1]
https://postgr.es/m/cah2-wznungszf8v6osgjac5aysb3cz6hw6mlm30x0d65cms...@mail.gmail.com
--
Peter Geoghegan
y -- a good outcome
for everybody. If you and Robert can find a way to accommodate that,
then in all likelihood we won't need to have any more heated and
protracted arguments like the one from early in 2022. I will be quite
happy to get back to working on B-Tree, likely the skip scan work.
[1]
https://wiki.postgresql.org/wiki/Freezing/skipping_strategies_patch:_motivating_examples
--
Peter Geoghegan
On Wed, Jan 4, 2023 at 10:59 PM Amit Kapila wrote:
> You are an extremely valuable person for this project and I wish that
> you continue working with the same enthusiasm.
Thank you, Amit. Knowing that my efforts are appreciated by colleagues
does make it easier to persevere.
--
Peter Geoghegan
is particular error message is from the hardening added to Postgres
15 in commit e7428a99. So it's not surprising that Michail didn't see
the same error on 14.
--
Peter Geoghegan
On Thu, Jan 5, 2023 at 3:27 PM Peter Geoghegan wrote:
> This particular error message is from the hardening added to Postgres
> 15 in commit e7428a99. So it's not surprising that Michail didn't see
> the same error on 14.
Reproduced this on HEAD locally (no docker), without an
On Tue, Jan 3, 2023 at 10:52 PM Peter Geoghegan wrote:
> > And it'd make sense to have
> > the explanation of why TransactionIdDidAbort() isn't the same as
> > !TransactionIdDidCommit(), even for !TransactionIdIsInProgress() xacts, near
> > the explanation f
ong the wait will be -- all bets are off. Could be
a day, a week, a month -- who knows? The application itself is the
crucial factor here, and in general the application can do whatever it
wants to do -- that is the reality. So we should be willing to kick
the can down the road in almost all cases -- that is actually the
responsible thing to do under the circumstances. We need to get on
with freezing every other page in the table!
There just cannot be very many pages that can't be cleanup locked at
any given time, so waiting indefinitely is a very drastic measure in
response to a problem that is quite likely to go away on its own. A
problem that waiting doesn't really solve anyway. Maybe the only thing
that will work is waiting for a very long time, but we have nothing to
lose (and everything to gain) by waiting to wait.
--
Peter Geoghegan
automatically make the user aware of the issues that surface around
> XID wraparound. Retaining the explainer for XID wraparound in the docs
> seems like a decent idea - it may be moved, but please don't delete
> it.
We do need to stop telling users to enter single user mode. It's quite
simply obsolete, bad advice, and has been since Postgres 14. It's the
worst thing that you could do, in fact.
--
Peter Geoghegan
rules?
FWIW, I found an existing comment about this rule in the call to
TransactionIdAbortTree() from RecordTransactionAbort() -- which took
me quite a while to find. So you might have been remembering that
comment before.
--
Peter Geoghegan
On Mon, Jan 2, 2023 at 10:31 AM Peter Geoghegan wrote:
> Would be helpful if I could get a +1 on
> v1-0002-Never-just-set-the-all-frozen-bit-in-VM.patch, which is
> somewhat more substantial than the others.
There has been no response on this thread for over a full week at this
point. I
e. It just makes it hard to review the
> patch.
I didn't think that it was that big of a deal to tweak the style of
one or two details in and around lazy_vacuum_heap_rel() in passing,
for consistency with lazy_scan_heap(), since the patch already needs
to do some of that. I do take your point, though.
--
Peter Geoghegan
On Thu, Dec 29, 2022 at 7:01 PM Peter Geoghegan wrote:
> Attached is v2, which is just to fix bitrot.
Attached is v3. We no longer apply vacuum_failsafe_age when
determining the cutoff for antiwraparound autovacuuming -- the new
approach is a bit simpler.
This is a fairly small change over
On Sun, Jan 8, 2023 at 4:27 PM Peter Geoghegan wrote:
> We're vulnerable to allowing "all-frozen but not all-visible"
> inconsistencies because of two factors: this business with not passing
> VISIBILITYMAP_ALL_VISIBLE along with VISIBILITYMAP_ALL_FROZEN to
> visibility
On Sun, Jan 8, 2023 at 6:43 PM Peter Geoghegan wrote:
> On further reflection even v2 won't repair the page-level
> PD_ALL_VISIBLE flag in passing in this scenario. ISTM that on HEAD we
> might actually leave the all-frozen bit set in the VM, while both the
> all-visible bit a
erm
approach. We will very likely need to keep all_visible_according_to_vm
as a cache for performance reasons for as long as we have
SKIP_PAGES_THRESHOLD.
Can we just update all_visible_according_to_vm using
PageIsAllVisible(), without making all_visible_according_to_vm
significantly less useful a
cope because
it's part of the same overall problem of updating the visibility map
based on potentially stale information. It makes zero sense to check
with the visibility map before updating it when we already know that
the page is all-visible. I mean, are we trying to avoid the work of
needlessly updating the visibility map in cases where its state was
corrupt, but then became uncorrupt (relative to the heap page) by
mistake?
--
Peter Geoghegan
eak?
They're not too weak. I'm not sure why the memcpy() was used. I see
your point; it makes you wonder if it must be necessary, which then
seems to call into question why it's okay to access the main array as
an array. I can change this detail, too.
I'll try to get back to it this week.
--
Peter Geoghegan
onId arguments
before now, which was 100% guaranteed to be a waste of cycles. I saw
no need to wait more than a few days for a +1, given that this
particular issue was so completely clear cut.
--
Peter Geoghegan
ard"
autovacuum is launched on a table whose relfrozenxid age is 1.5
billion, it'll just be a regular dead tuples/inserted tuples
autovacuum, with the same old familiar locking characteristics as
today.
--
Peter Geoghegan
hink that it
easily could be a bit annoying. How hard would it be to invent a
general mechanism to control the verbosity of what we'll show for each
WAL record?
--
Peter Geoghegan
On Mon, Jan 9, 2023 at 12:58 PM Peter Geoghegan wrote:
> I didn't realize that affected visibilitymap_set() calls could
> generate useless set-VM WAL records until you pointed it out. That's
> far more likely to happen than the race condition that I described --
> it has not
due to these very fine details. (Assuming I haven't
missed another path to the problem with aborted Multis or something,
but looks like I haven't.)
--
Peter Geoghegan
On Tue, Jan 10, 2023 at 11:47 AM Peter Geoghegan wrote:
> In summary, I think that there is currently no way that we can have
> the VM (or the PD_ALL_VISIBLE flag) concurrently unset, while leaving
> the page all_frozen. It can happen and leave the page all_visible, but
> not all_fr
ge being
all-visible, as if it might take a dissenting view that needs to be
taken into consideration (obviously we know what's going on with the
page because we just scanned it ourselves, and determined that it was
at least all-visible).
--
Peter Geoghegan
On Tue, Jan 10, 2023 at 12:08 PM Peter Geoghegan wrote:
> Actually, FreezeMultiXactId() can fully remove an xmax that has some
> member XIDs >= OldestXmin, provided FRM_NOOP processing isn't
> possible, at least when no individual member is still running. Doesn't
> ha
On Tue, Jan 10, 2023 at 4:39 PM Peter Geoghegan wrote:
> * Run VACUUM FREEZE. We need FREEZE in order to be able to hit the
> relevant visibilitymap_set() call site (the one that passes
> VISIBILITYMAP_ALL_FROZEN as its flags, without also passing
> VISIBILITYMAP_ALL_VISIB
g the failsafe mode,
> similar to [auto]vacuum cost delays getting disabled
> (c.f. lazy_check_wraparound_failsafe()). If things are bad enough that we're
> soon going to shut down, we want to be aggressive.
+1
--
Peter Geoghegan
on quite a bit where available.
--
Peter Geoghegan
s a fair point.
My vote goes to "REUSE_BUFFERS", then.
--
Peter Geoghegan
get_xid_status() need a reference to these rules?
>
> Don't think so? Whad made you ask?
Just the fact that it seems to more or less follow the protocol
described at the top of heapam_visibility.c. Not very important,
though.
--
Peter Geoghegan
v2-0001-Improve-TransactionIdDidAbort
ebugging it seems to have been a
heapam thing, just because there's a lot more that can go wrong with
pruning, which is spread across many different places.
--
Peter Geoghegan
al of
diversity that we need to be considered. For example, the WAL records
used by each individual index access method are all very similar. In
fact the most important index AM WAL records used by each index AM
(e.g. insert, delete, vacuum) have virtually the same format as each
other already.
--
Peter Geoghegan
d/crashed through process of elimination instead.
>
> s/by now//?
Did it that way in the commit I pushed just now.
Thanks
--
Peter Geoghegan
On Mon, Jan 9, 2023 at 2:18 PM Peter Geoghegan wrote:
> I'll try to get back to it this week.
Attached patch fixes up these issues. It's almost totally mechanical.
(Ended up using "git diff --color-moved=dimmed-zebra
--color-moved-ws=ignore-all-space" with this, per you
On Wed, Jan 11, 2023 at 4:44 PM Andres Freund wrote:
> > Attached patch fixes up these issues. It's almost totally mechanical.
>
> Looks better, thanks!
Pushed that just now.
Thanks
--
Peter Geoghegan
On Mon, Jan 9, 2023 at 12:58 PM Peter Geoghegan wrote:
> On Mon, Jan 9, 2023 at 11:57 AM Andres Freund wrote:
> > Afaict we'll need to backpatch this all the way?
>
> I thought that we probably wouldn't need to, at first. But I now think
> that we really have to.
At
.
The similar outages I was called in to help with personally had either
an automated TRUNCATE or an automated CREATE INDEX. Had autovacuum
only been willing to yield once or twice, then it probably would have
been fine -- the situation probably would have worked itself out
naturally. That's the best outcome you can hope for.
--
Peter Geoghegan
e way. The point
is to make decisions dynamically, based on the observed conditions in
the table. And to delay committing to things until there really is no
alternative, to maximize our opportunities to avoid disaster. In
short: loose, springy behavior.
Imposing absolute obligations on VACUUM has the potential to create
lots of problems. It is sometimes necessary, but can easily be
overused, making a bad situation much worse.
--
Peter Geoghegan
1201 - 1300 of 3218 matches
Mail list logo
