On Fri, Jan 27, 2023 at 12:52 AM Andres Freund wrote:
> I agree with bringing high-level context into the decision about whether to
> freeze agressively - my problem with the eager freezing strategy patch isn't
> that it did that too much, it's that it didn't do it enough.
>
>
> But I also don't
Hi,
On 2023-01-27 12:53:58 -0500, Robert Haas wrote:
> On Fri, Jan 27, 2023 at 12:58 AM Andres Freund wrote:
> > Essentially the "any fpi" logic is a very coarse grained way of using the
> > page
> > LSN as a measurement. As I said, I don't think "has a checkpoint occurred
> > since the last
On Fri, Jan 27, 2023 at 12:58 AM Andres Freund wrote:
> Essentially the "any fpi" logic is a very coarse grained way of using the page
> LSN as a measurement. As I said, I don't think "has a checkpoint occurred
> since the last write" is a good metric to avoid unnecessary freezing - it's
> too
On Fri, Jan 27, 2023 at 6:48 AM Robert Haas wrote:
> > One of the key strengths of systems like Postgres is the ability to
> > inexpensively store a relatively large amount of data that has just
> > about zero chance of being read, let alone modified. While at the same
> > time having decent OLTP
On Thu, Jan 26, 2023 at 6:37 PM Peter Geoghegan wrote:
> > I don't see what your reference to checkpoint timeout is about here?
> >
> > Also, as I mentioned before, the problem isn't specific to
> > checkpoint_timeout
> > = 1min. It just makes it cheaper to reproduce.
>
> That's flagrantly
On Thu, Jan 26, 2023 at 4:51 PM Peter Geoghegan wrote:
> This is the kind of remark that makes me think that you don't get it.
>
> The most influential OLTP benchmark of all time is TPC-C, which has
> exactly this problem. In spades -- it's enormously disruptive. Which
> is one reason why I used
Hi,
On 2023-01-27 00:51:59 -0800, Andres Freund wrote:
> One use-case where the eager strategy is particularly useful is
> [nearly-]append-only tables - and it's also the one workload that's reasonably
> easy to detect using stats. Maybe something like
> (dead_tuples_since_last_vacuum /
Hi,
On 2023-01-26 23:11:41 -0800, Peter Geoghegan wrote:
> > Essentially the "any fpi" logic is a very coarse grained way of using the
> > page
> > LSN as a measurement. As I said, I don't think "has a checkpoint occurred
> > since the last write" is a good metric to avoid unnecessary freezing -
On Thu, Jan 26, 2023 at 9:58 PM Andres Freund wrote:
> It doesn't seem like a great proxy to me. ISTM that this means that how
> aggressive vacuum is about opportunistically freezing pages depends on config
> variables like checkpoint_timeout & max_wal_size (less common opportunistic
> freezing),
Hi,
On 2023-01-26 19:01:03 -0800, Peter Geoghegan wrote:
> On Thu, Jan 26, 2023 at 6:37 PM Andres Freund wrote:
> > I also don't really see how that is responsive to anything else in my
> > email. That's just as true for the current gating condition (the issuance of
> > an FPI during
On Thu, Jan 26, 2023 at 6:37 PM Andres Freund wrote:
> I also don't really see how that is responsive to anything else in my
> email. That's just as true for the current gating condition (the issuance of
> an FPI during heap_page_prune() / HTSV()).
>
> What I was wondering about is whether we
Hi,
On 2023-01-26 15:36:52 -0800, Peter Geoghegan wrote:
> On Thu, Jan 26, 2023 at 12:45 PM Andres Freund wrote:
> > > Most of the overhead of FREEZE WAL records (with freeze plan
> > > deduplication and page-level freezing in) is generic WAL record header
> > > overhead. Your recent adversarial
Hi,
On 2023-01-26 14:27:53 -0500, Robert Haas wrote:
> One idea that I've had about how to solve this problem is to try to
> make vacuum try to aggressively freeze some portion of the table on
> each pass, and to behave less aggressively on the rest of the table so
> that, hopefully, no single
On Thu, Jan 26, 2023 at 12:45 PM Andres Freund wrote:
> > Most of the overhead of FREEZE WAL records (with freeze plan
> > deduplication and page-level freezing in) is generic WAL record header
> > overhead. Your recent adversarial test case is going to choke on that,
> > too. At least if you set
On Thu, Jan 26, 2023 at 1:22 PM Robert Haas wrote:
> On Thu, Jan 26, 2023 at 4:06 PM Peter Geoghegan wrote:
> > There is very good reason to believe that the large majority of all
> > data that people store in a system like Postgres is extremely cold
> > data:
>
> The systems where I end up
On Thu, Jan 26, 2023 at 4:06 PM Peter Geoghegan wrote:
> There is very good reason to believe that the large majority of all
> data that people store in a system like Postgres is extremely cold
> data:
The systems where I end up troubleshooting problems seem to be, most
typically, busy OLTP
On Thu, Jan 26, 2023 at 12:54 PM Robert Haas wrote:
> > The overwhelming cost is usually FPIs in any case. If you're not
> > mostly focussing on that, you're focussing on the wrong thing. At
> > least with larger tables. You just have to focus on the picture over
> > time, across multiple VACUUM
Hi,
On 2023-01-26 20:26:00 +0100, Matthias van de Meent wrote:
> Could someone explain to me why we don't currently (optionally)
> include the functionality of page freezing in the PRUNE records?
I think we definitely should (and have argued for it a couple times). It's not
just about reducing
On Thu, Jan 26, 2023 at 2:57 PM Peter Geoghegan wrote:
> Relatively difficult for Andres, or for somebody else? What are the
> real parameters here? Obviously there are no clear answers available.
Andres is certainly smarter than the average guy, but practically any
scenario that someone can
Hi,
On 2023-01-26 10:44:45 -0800, Peter Geoghegan wrote:
> On Thu, Jan 26, 2023 at 9:53 AM Andres Freund wrote:
> > > That's going to be very significantly more aggressive. For example
> > > it'll impact small tables very differently.
> >
> > Maybe it would be too aggressive, not sure. The cost
On Thu, Jan 26, 2023 at 11:26 AM Matthias van de Meent
wrote:
> Could someone explain to me why we don't currently (optionally)
> include the functionality of page freezing in the PRUNE records? I
> think they're quite closely related (in that they both execute in
> VACUUM and are required for
On Thu, Jan 26, 2023 at 11:28 AM Robert Haas wrote:
> I think it's pretty much impossible to freeze more aggressively
> without losing in some scenario or other. If waiting longer to freeze
> would have resulted in the data getting updated again or deleted
> before we froze it, then waiting
On Thu, Jan 26, 2023 at 11:35 AM Peter Geoghegan wrote:
> You complained about the descriptions being theoretical. But there's
> nothing theoretical about the fact that we more or less do *all*
> freezing in an eventual aggressive VACUUM in many important cases,
> including very simple cases like
On Thu, 26 Jan 2023 at 19:45, Peter Geoghegan wrote:
>
> On Thu, Jan 26, 2023 at 9:53 AM Andres Freund wrote:
> > I assume the case you're thinking of is that pruning did *not* do any
> > changes,
> > but in the process of figuring out that nothing needed to be pruned, we did
> > a
> >
On Thu, Jan 26, 2023 at 9:53 AM Andres Freund wrote:
> I assume the case you're thinking of is that pruning did *not* do any changes,
> but in the process of figuring out that nothing needed to be pruned, we did a
> MarkBufferDirtyHint(), and as part of that emitted an FPI?
Yes.
> > That's
Hi,
On 2023-01-26 08:54:55 -0800, Peter Geoghegan wrote:
> On Thu, Jan 26, 2023 at 8:35 AM Andres Freund wrote:
> > I think it's probably ok, but perhaps deserves a bit more thought about when
> > to "opportunistically" freeze. Perhaps to make it *more* aggressive than
> > it's
> > now.
> >
> >
On Thu, Jan 26, 2023 at 8:35 AM Andres Freund wrote:
> I think it's probably ok, but perhaps deserves a bit more thought about when
> to "opportunistically" freeze. Perhaps to make it *more* aggressive than it's
> now.
>
> With "opportunistic freezing" I mean freezing the page, even though we
Hi,
On 2023-01-26 09:20:57 -0500, Robert Haas wrote:
> On Wed, Jan 25, 2023 at 10:56 PM Andres Freund wrote:
> > but that's only true because page level freezing neutered
> > vacuum_freeze_min_age. Compared to <16, it's a *huge* change.
>
> Do you think that page-level freezing
>
On Thu, Jan 26, 2023 at 5:41 AM Robert Haas wrote:
> On Wed, Jan 25, 2023 at 11:25 PM Peter Geoghegan wrote:
> > On Wed, Jan 25, 2023 at 7:41 PM Robert Haas wrote:
> > > Both Andres and I have repeatedly expressed concern about how much is
> > > being changed in the behavior of vacuum, and how
On Wed, Jan 25, 2023 at 7:56 PM Andres Freund wrote:
> > https://wiki.postgresql.org/wiki/Freezing/skipping_strategies_patch:_motivating_examples#Patch_2
> >
> > The difference between this and VACUUM FREEZE is described here:
> >
> > "Note how we freeze most pages, but still leave a significant
On Wed, Jan 25, 2023 at 10:56 PM Andres Freund wrote:
> but that's only true because page level freezing neutered
> vacuum_freeze_min_age. Compared to <16, it's a *huge* change.
Do you think that page-level freezing
(1de58df4fec7325d91f5a8345757314be7ac05da) was improvidently
committed?
I have
On Wed, Jan 25, 2023 at 11:25 PM Peter Geoghegan wrote:
> On Wed, Jan 25, 2023 at 7:41 PM Robert Haas wrote:
> > Both Andres and I have repeatedly expressed concern about how much is
> > being changed in the behavior of vacuum, and how quickly, and IMHO on
> > the basis of very limited evidence
On Wed, Jan 25, 2023 at 8:24 PM Peter Geoghegan wrote:
> > I think we're on a very dangerous path here. I want VACUUM to be
> > better as the next person, but I really don't believe that's the
> > direction we're headed. I think if we release like this, we're going
> > to experience more VACUUM
On Wed, Jan 25, 2023 at 8:12 PM John Naylor
wrote:
> That was followed by several paragraphs that never got around to explaining
> why table size should drive freezing strategy.
You were talking about the system level view of freeze debt, and how
the table view might not be a sufficient proxy
On Wed, Jan 25, 2023 at 7:41 PM Robert Haas wrote:
> Both Andres and I have repeatedly expressed concern about how much is
> being changed in the behavior of vacuum, and how quickly, and IMHO on
> the basis of very limited evidence that the changes are improvements.
> The fact that Andres was
On Thu, Jan 26, 2023 at 10:11 AM Andres Freund wrote:
> I am. Just not every tradeoff. I just don't see any useful tradeoffs
purely
> based on the relation size.
I expressed reservations about relation size six weeks ago:
On Wed, Dec 14, 2022 at 12:16 AM Peter Geoghegan wrote:
>
> On Tue, Dec
Hi,
On 2023-01-25 18:43:10 -0800, Peter Geoghegan wrote:
> On Wed, Jan 25, 2023 at 6:33 PM Andres Freund wrote:
> > As far as I can tell, with the eager strategy, the only thing
> > vacuum_freeze_min_age really influences is whether we'll block waiting for a
> > cleanup lock. IOW, VACUUM on a
On Wed, Jan 25, 2023 at 7:11 PM Andres Freund wrote:
> > > I switched between vacuum_freeze_strategy_threshold=0 and
> > > vacuum_freeze_strategy_threshold=too-high, because it's quicker/takes less
> > > warmup to set up something with smaller tables.
> >
> > This makes no sense to me, at all.
>
On Wed, Jan 25, 2023 at 8:49 PM Andres Freund wrote:
> The concrete setting of vacuum_freeze_strategy_threshold doesn't matter.
> Table size simply isn't a usable proxy for whether eager freezing is a good
> idea or not.
I strongly agree. I can't imagine how a size-based threshold can make
any
Hk,
On 2023-01-25 18:31:16 -0800, Peter Geoghegan wrote:
> On Wed, Jan 25, 2023 at 5:49 PM Andres Freund wrote:
> > Sure. But significantly regressing plausible if not common workloads is
> > different than knowing that there'll be some edge case where we'll do
> > something worse.
>
> That's
On Wed, Jan 25, 2023 at 6:33 PM Andres Freund wrote:
> My point was the other way round. That vacuum_freeze_min_age *prevented* us
> from freezing rows "too soon" - obviously a very blunt instrument.
Yes, not freezing at all until aggressive vacuum is definitely good
when you don't really need
Hi,
On 2023-01-25 17:28:48 -0800, Peter Geoghegan wrote:
> On Wed, Jan 25, 2023 at 5:15 PM Andres Freund wrote:
> > However, it significantly increases the overall work when rows have a
> > somewhat
> > limited lifetime. The documented reason why vacuum_freeze_min_age exist -
> > although I
On Wed, Jan 25, 2023 at 5:49 PM Andres Freund wrote:
> Sure. But significantly regressing plausible if not common workloads is
> different than knowing that there'll be some edge case where we'll do
> something worse.
That's very vague. Significant to whom, for what purpose?
> prep:
> CREATE
Hi,
On 2023-01-25 17:37:17 -0800, Peter Geoghegan wrote:
> On Wed, Jan 25, 2023 at 5:26 PM Andres Freund wrote:
> > Another bad scenario: Some longrunning / hung transaction caused us to get
> > close to the xid wraparound. Problem was resolved, autovacuum runs.
> > Previously
> > we wouldn't
Hi,
On 2023-01-25 17:22:32 -0800, Peter Geoghegan wrote:
> On Wed, Jan 25, 2023 at 4:43 PM Andres Freund wrote:
> > I unfortunately haven't been able to keep up with the thread and saw this
> > just
> > now. But I've expressed the concern below several times before, so it
> > shouldn't come as
On Wed, Jan 25, 2023 at 5:26 PM Andres Freund wrote:
> Another bad scenario: Some longrunning / hung transaction caused us to get
> close to the xid wraparound. Problem was resolved, autovacuum runs. Previously
> we wouldn't have frozen the portion of the table that was actively changing,
> now
On Wed, Jan 25, 2023 at 5:15 PM Andres Freund wrote:
> However, it significantly increases the overall work when rows have a somewhat
> limited lifetime. The documented reason why vacuum_freeze_min_age exist -
> although I think it doesn't really achieve its documented goal anymore, after
> the
Hi,
On 2023-01-25 16:43:47 -0800, Andres Freund wrote:
> I think, as committed, this will cause serious issues for some reasonably
> common workloads, due to substantially increased WAL traffic.
>
>
> The most common problematic scenario I see are tables full of rows with
> limited lifetime.
On Wed, Jan 25, 2023 at 4:43 PM Andres Freund wrote:
> I unfortunately haven't been able to keep up with the thread and saw this just
> now. But I've expressed the concern below several times before, so it
> shouldn't come as a surprise.
You missed the announcement 9 days ago, and the similar
Hi,
On 2023-01-24 14:49:38 -0800, Peter Geoghegan wrote:
> From e41d3f45fcd6f639b768c22139006ad11422575f Mon Sep 17 00:00:00 2001
> From: Peter Geoghegan
> Date: Thu, 24 Nov 2022 18:20:36 -0800
> Subject: [PATCH v17 1/3] Add eager and lazy freezing strategies to VACUUM.
>
> Eager freezing
Hi,
On 2023-01-24 14:49:38 -0800, Peter Geoghegan wrote:
> On Mon, Jan 16, 2023 at 5:55 PM Peter Geoghegan wrote:
> > 0001 (the freezing strategies patch) is now committable IMV. Or at
> > least will be once I polish the docs a bit more. I plan on committing
> > 0001 some time next week, barring
On Tue, 24 Jan 2023 at 23:50, Peter Geoghegan wrote:
>
> On Mon, Jan 16, 2023 at 5:55 PM Peter Geoghegan wrote:
> > 0001 (the freezing strategies patch) is now committable IMV. Or at
> > least will be once I polish the docs a bit more. I plan on committing
> > 0001 some time next week, barring
On Mon, Jan 23, 2023 at 3:17 AM Dilip Kumar wrote:
> My final set of comments for 0002
Thanks for the review!
> I do not understand much use of maintaining these two
> 'scanned_pages_lazy' and 'scanned_pages_eager' variables. I think
> just maintaining 'scanned_pages' should be sufficient. I
On Wed, Jan 18, 2023 at 1:47 PM Dilip Kumar wrote:
>
> On Tue, Jan 17, 2023 at 10:05 AM Peter Geoghegan wrote:
My final set of comments for 0002
1.
+struct vmsnapshot
+{
+/* Target heap rel */
+Relationrel;
+/* Scanning strategy used by VACUUM operation */
+vmstrategy
On Tue, Jan 17, 2023 at 10:05 AM Peter Geoghegan wrote:
>
> On Mon, Jan 16, 2023 at 8:13 PM Dilip Kumar wrote:
> > I think that it makes sense to keep 'vacuum_freeze_strategy_threshold'
> > strictly for freezing. But the point is that the eager scanning
> > strategy is driven by table freezing
On Mon, Jan 16, 2023 at 8:13 PM Dilip Kumar wrote:
> I think that it makes sense to keep 'vacuum_freeze_strategy_threshold'
> strictly for freezing. But the point is that the eager scanning
> strategy is driven by table freezing needs of the table (tableagefrac)
> that make sense, but if we have
On Mon, Jan 16, 2023 at 11:31 PM Peter Geoghegan wrote:
>
> > I think '(nextXID - cutoffs->relfrozenxid) / freeze_table_age' should
> > be the actual fraction right? What is the point of adding 0.5 to the
> > divisor? If there is a logical reason, maybe we can explain in the
> > comments.
>
>
On Mon, Jan 16, 2023 at 10:00 AM Peter Geoghegan wrote:
> Now we treat the scanning and freezing strategies as two independent
> choices. Of course they're not independent in any practical sense, but
> I think it's slightly simpler and more elegant that way -- it makes
> the GUC
On Mon, Jan 16, 2023 at 10:10 AM Peter Geoghegan wrote:
> Attached is v16, which incorporates some of Matthias' feedback.
0001 (the freezing strategies patch) is now committable IMV. Or at
least will be once I polish the docs a bit more. I plan on committing
0001 some time next week, barring any
On Sun, Jan 15, 2023 at 9:13 PM Dilip Kumar wrote:
> I have looked into the patch set, I think 0001 looks good to me about
> 0002 I have a few questions, 0003 I haven't yet looked at
Thanks for taking a look.
> I think '(nextXID - cutoffs->relfrozenxid) / freeze_table_age' should
> be the
On Mon, Jan 9, 2023 at 7:16 AM Peter Geoghegan wrote:
>
> On Tue, Jan 3, 2023 at 12:30 PM Peter Geoghegan wrote:
> > Attached is v14.
>
> This has stopped applying due to conflicts with nearby work on VACUUM
> from Tom. So I attached a new revision, v15, just to make CFTester
> green again.
>
>
On Thu, Jan 5, 2023 at 10:19 AM Matthias van de Meent
wrote:
> Could this use something like autovacuum_cost_delay? I don't quite
> like the use of arbitrary hardcoded millisecond delays
It's not unlike (say) the way that there can sometimes be hardcoded
waits inside GetMultiXactIdMembers(),
On Wed, Jan 4, 2023 at 5:21 PM Matthias van de Meent
wrote:
> Some reviews (untested; only code review so far) on these versions of
> the patches:
Thanks for the review!
> > [PATCH v14 1/3] Add eager and lazy freezing strategies to VACUUM.
> I don't think the mention of 'cutoff point' is
On Thu, 5 Jan 2023 at 02:21, I wrote:
>
> On Tue, 3 Jan 2023 at 21:30, Peter Geoghegan wrote:
> >
> > Attached is v14.
> > [PATCH v14 3/3] Finish removing aggressive mode VACUUM.
>
> I've not completed a review for this patch - I'll continue on that
> tomorrow:
This is that.
> @@ -2152,10
On Tue, 3 Jan 2023 at 21:30, Peter Geoghegan wrote:
>
> Attached is v14.
Some reviews (untested; only code review so far) on these versions of
the patches:
> [PATCH v14 1/3] Add eager and lazy freezing strategies to VACUUM.
> +/*
> + * Threshold cutoff point (expressed in # of physical
On Mon, 2023-01-02 at 11:45 -0800, Peter Geoghegan wrote:
> What do you think of the wording adjustments in the attached patch?
> It's based on your suggested wording.
Great, thank you.
--
Jeff Davis
PostgreSQL Contributor Team - AWS
On Sat, Dec 31, 2022 at 12:45 PM Peter Geoghegan wrote:
> On Sat, Dec 31, 2022 at 11:46 AM Jeff Davis wrote:
> > "We have no freeze plans to execute, so there's no cost to following
> > the freeze path. This is important in the case where the page is
> > entirely frozen already, so that the page
On Sat, Dec 31, 2022 at 11:46 AM Jeff Davis wrote:
> On Fri, 2022-12-30 at 16:58 -0800, Peter Geoghegan wrote:
> > Following the path of freezing a page is *always* valid, by
> > definition. Including when there are zero freeze plans to execute, or
> > even zero tuples to examine in the first
On Fri, 2022-12-30 at 16:58 -0800, Peter Geoghegan wrote:
> Following the path of freezing a page is *always* valid, by
> definition. Including when there are zero freeze plans to execute, or
> even zero tuples to examine in the first place -- we'll at least be
> able to perform nominal freezing,
On Fri, Dec 30, 2022 at 1:12 PM Peter Geoghegan wrote:
> > "Nominal freezing" is happening when there are no freeze plans at all.
> > I get that it's to manage control flow so that the right thing happens
> > later. But I think it should be defined in terms of what state the page
> > is in so
On Fri, Dec 30, 2022 at 12:43 PM Jeff Davis wrote:
> I always understood "freezing" to mean that a concrete action was
> taken, and associated WAL generated.
"When I use a word… it means just what I choose it to mean -- neither
more nor less".
I have also always understood freezing that way
On Mon, 2022-12-26 at 12:53 -0800, Peter Geoghegan wrote:
> * v12 merges together the code for the "freeze the page"
> lazy_scan_prune path with the block that actually calls
> heap_freeze_execute_prepared().
>
> This should make it clear that pagefrz.freeze_required really does
> mean that
Dear Peter,
> Anyway, this assertion is wrong, and simply needs to be removed.
> Thanks for the report
Thanks for modifying for quickly! I found your commit in the remote repository.
I will watch and report again if there are another issue.
Best Regards,
Hayato Kuroda
FUJITSU LIMITED
On Mon, Dec 26, 2022 at 10:57 PM Hayato Kuroda (Fujitsu)
wrote:
> I guessed that this assertion failure seemed to be caused by the commit
> 4ce3af[2],
> because the Assert() seemed to be added by the commit.
I agree that the problem is with this assertion, which is on the
master branch (not in
Dear Peter, Jeff,
While reviewing other patches, I found that cfbot raised ERROR during the
VACUUM FREEZE [1] on FreeBSD instance.
It seemed that same error has been occurred in other threads.
```
2022-12-23 08:50:20.175 UTC [34653][postmaster] LOG: server process (PID
37171) was terminated
On Wed, Dec 21, 2022 at 4:30 PM Jeff Davis wrote:
> The confusing thing to me is perhaps just the name -- to me,
> "freeze_required" suggests that if it were set to true, it would cause
> freezing to happen. But as far as I can tell, it does not cause
> freezing to happen, it causes some other
On Tue, 2022-12-20 at 21:26 -0800, Peter Geoghegan wrote:
> When freeze_required is set to true, that means that lazy_scan_prune
> literally has no choice -- it simply must freeze the page as
> instructed by heap_prepare_freeze_tuple/FreezeMultiXactId. It's not
> just a strong suggestion -- it's
On Tue, Dec 20, 2022 at 7:15 PM Peter Geoghegan wrote:
> On Tue, Dec 20, 2022 at 5:44 PM Jeff Davis wrote:
> > Next, the 'freeze_required' field suggests that it's more involved in
> > the control flow that causes freezing than it actually is. All it does
> > is communicate how the trackers need
On Tue, Dec 20, 2022 at 5:44 PM Jeff Davis wrote:
> Comments on 0002:
>
> Can you explain the following portion of the diff:
>
>
> - else if (MultiXactIdPrecedes(multi, cutoffs->MultiXactCutoff))
> + else if (MultiXactIdPrecedes(multi, cutoffs->OldestMxact))
>
> ...
>
> + /* Can't violate
On Sun, 2022-12-18 at 14:20 -0800, Peter Geoghegan wrote:
> On Thu, Dec 15, 2022 at 10:53 AM Peter Geoghegan wrote:
> > I agree that the burden of catch-up freezing is excessive here (in
> > fact I already wrote something to that effect on the wiki page).
> > The
> > likely solution can be simple
Hi!
I'll try to apply this patch onto my branch with Pluggable TOAST to test
these mechanics with new TOAST. Would reply on the result. It could
be difficult though, because both have a lot of changes that affect
the same code.
>I'm not sure how much this would help with bloat. I suspect that it
On Sun, 2022-12-18 at 14:20 -0800, Peter Geoghegan wrote:
> Attached is v10, which fixes this issue, but using a different
> approach to the one I sketched here.
In 0001, it's fairly straightforward rearrangement and looks like an
improvement to me. I have a few complaints, but they are about
On Thu, Dec 15, 2022 at 11:59 PM Nikita Malakhov wrote:
> I've found this discussion very interesting, in view of vacuuming
> TOAST tables is always a problem because these tables tend to
> bloat very quickly with dead data - just to remind, all TOAST-able
> columns of the relation use the same
On Thu, Dec 15, 2022 at 11:48 PM John Naylor
wrote:
> Thanks for this. This is the kind of concrete, data-based evidence that I
> find much more convincing, or at least easy to reason about.
I'm glad to hear that it helped. It's always difficult to judge where
other people are coming from,
Hi!
I've found this discussion very interesting, in view of vacuuming
TOAST tables is always a problem because these tables tend to
bloat very quickly with dead data - just to remind, all TOAST-able
columns of the relation use the same TOAST table which is one
for the relation, and TOASTed data
On Wed, Dec 14, 2022 at 6:07 AM Peter Geoghegan wrote:
>
> At the suggestion of Jeff, I wrote a Wiki page that shows motivating
> examples for the patch series:
>
>
https://wiki.postgresql.org/wiki/Freezing/skipping_strategies_patch:_motivating_examples
>
> These are all cases where VACUUM
On Thu, Dec 15, 2022 at 11:11 AM Justin Pryzby wrote:
> The patches (003 and 005) are missing a word
> should use to decide whether to its eager freezing strategy.
I mangled this during rebasing for v9, which reordered the commits.
Will be fixed in v10.
> On the wiki, missing a word:
> builds
The patches (003 and 005) are missing a word
should use to decide whether to its eager freezing strategy.
On the wiki, missing a word:
builds on related added
On Thu, Dec 15, 2022 at 6:50 AM Matthias van de Meent
wrote:
> This first run of (auto)vacuum after the 8GB threshold seems
> to appear as a significant IO event (both in WAL and relation
> read/write traffic) with 50% of the table updated and WAL-logged. I
> think this should be limited to some
On Wed, 14 Dec 2022 at 00:07, Peter Geoghegan wrote:
>
> On Tue, Dec 13, 2022 at 9:16 AM Peter Geoghegan wrote:
> > That's not the only thing we care about, though. And to the extent we
> > care about it, we mostly care about the consequences of either
> > freezing or not freezing eagerly.
On Tue, Dec 13, 2022 at 9:16 AM Peter Geoghegan wrote:
> That's not the only thing we care about, though. And to the extent we
> care about it, we mostly care about the consequences of either
> freezing or not freezing eagerly. Concentration of unfrozen pages in
> one particular table is a lot
On Tue, Dec 13, 2022 at 12:29 AM John Naylor
wrote:
> If the number of unfrozen heap pages is the thing we care about, perhaps
> that, and not the total size of the table, should be the parameter that
> drives freezing strategy?
That's not the only thing we care about, though. And to the
On Tue, Dec 13, 2022 at 8:00 AM Peter Geoghegan wrote:
>
> On Mon, Dec 12, 2022 at 3:47 PM Jeff Davis wrote:
> > But the heuristic also seems off to me. What if you have lots of
partitions
> > in an append-only range-partitioned table? That would tend to use the
> > lazy freezing strategy
On Mon, Dec 12, 2022 at 3:47 PM Jeff Davis wrote:
> Freezing is driven by a need to keep the age of the oldest
> transaction ID in a table to less than ~2B; and also the need to
> truncate the clog (and reduce lookups of really old xids). It's fine to
> give a brief explanation about why we can't
On Sat, 2022-12-10 at 18:11 -0800, Peter Geoghegan wrote:
> On Tue, Dec 6, 2022 at 1:45 PM Peter Geoghegan wrote:
> > v9 will also address some of the concerns you raised in your review
> > that weren't covered by v8, especially about the VM snapshotting
> > infrastructure. But also your concerns
On Tue, Dec 6, 2022 at 10:42 AM Andres Freund wrote:
> The docs don't build:
> https://cirrus-ci.com/task/5456939761532928
> [20:00:58.203] postgres.sgml:52: element link: validity error : IDREF
> attribute linkend references an unknown ID "vacuum-for-wraparound"
Thanks for pointing this out.
Hi,
On 2022-11-23 15:06:52 -0800, Peter Geoghegan wrote:
> Attached is v8.
The docs don't build:
https://cirrus-ci.com/task/5456939761532928
[20:00:58.203] postgres.sgml:52: element link: validity error : IDREF attribute
linkend references an unknown ID "vacuum-for-wraparound"
Greetings,
Hi,
On 2022-11-15 19:02:12 -0800, Peter Geoghegan wrote:
> From 352867c5027fae6194ab1c6480cd326963e201b1 Mon Sep 17 00:00:00 2001
> From: Peter Geoghegan
> Date: Sun, 12 Jun 2022 15:46:08 -0700
> Subject: [PATCH v6 1/6] Add page-level freezing to VACUUM.
>
> Teach VACUUM to decide on whether or
Note that this fails under -fsanitize=align
Subject: [PATCH v5 2/6] Teach VACUUM to use visibility map snapshot.
performing post-bootstrap initialization ...
../src/backend/access/heap/visibilitymap.c:482:38: runtime error: load of
misaligned address 0x5559e1352424 for type 'uint64', which
On Tue, Oct 4, 2022 at 7:59 PM Jeff Davis wrote:
> I am fine with that, but I'd like us all to understand what the
> downsides are.
Although I'm sure that there must be one case that loses measurably,
it's not particularly obvious where to start looking for one. I mean
it's easy to imagine
1 - 100 of 124 matches
Mail list logo
