On Mon, Mar 25, 2024 at 9:04 AM Heikki Linnakangas wrote:
> I included changes like this in commit f83d709760 ("Merge prune, freeze
> and vacuum WAL record formats"). Marking this as Committed in the
> commitfest.
Thanks for making sure that that happened. I suspect that the amount
of pruning
On 09/12/2023 23:48, Peter Geoghegan wrote:
On Tue, Mar 21, 2023 at 3:37 PM Peter Geoghegan wrote:
I think that we should do something like the attached, to completely
avoid this ambiguity. This patch adds a new XLOG_HEAP2 bit that's
similar to XLOG_HEAP_INIT_PAGE -- XLOG_HEAP2_BYVACUUM. This
On Tue, Mar 21, 2023 at 3:37 PM Peter Geoghegan wrote:
> I think that we should do something like the attached, to completely
> avoid this ambiguity. This patch adds a new XLOG_HEAP2 bit that's
> similar to XLOG_HEAP_INIT_PAGE -- XLOG_HEAP2_BYVACUUM. This allows all
> XLOG_HEAP2 record types to
On 04/09/2023 23:02, Melanie Plageman wrote:
I might phrase the last bit as "neither the description functions nor
the output format should be considered part of a stable API"
+Guidelines for rmgrdesc output format
+=
I noticed you used === for both headings
On Mon, Jul 10, 2023 at 3:44 AM Heikki Linnakangas wrote:
> I'm late to the party, but regarding commit c03c2eae0a, which added the
> guidelines for writing formatting desc functions:
>
> You moved the comment from rmgrdesc_utils.c into rmgrdesc_utils.h, but I
> don't think that was a good idea.
On Mon, Jul 10, 2023 at 10:29 PM Peter Geoghegan wrote:
> > Let's add a src/backend/access/rmgrdesc/README file. We don't currently
> > have any explanation anywhere why the rmgr desc functions are in a
> > separate directory. The README would be a good place to explain that,
> > and to have the
On Mon, Jul 10, 2023 at 12:44 AM Heikki Linnakangas wrote:
> This is still listed in the July commitfest; is there some work remaining?
I don't think so; not in the scope of the original patch series from
Melanie, at least.
> You moved the comment from rmgrdesc_utils.c into rmgrdesc_utils.h,
On 12/04/2023 01:29, Peter Geoghegan wrote:
Thanks for your help with the follow-up work. Seems like we're done
with this now.
This is still listed in the July commitfest; is there some work remaining?
I'm late to the party, but regarding commit c03c2eae0a, which added the
guidelines for
On Tue, Apr 11, 2023 at 2:29 PM Melanie Plageman
wrote:
> > That doesn't seem great to me either. I don't like this ambiguity,
> > because it seems like it makes the description hard to parse in a way
> > that flies in the face of what we're trying to do here, in general.
> > So it seems like it
On Tue, Apr 11, 2023 at 1:35 PM Peter Geoghegan wrote:
>
> On Tue, Apr 11, 2023 at 7:40 AM Melanie Plageman
> wrote:
> > Not the fault of this patch, but I also noticed that heap UPDATE and
> > HOT_UPDATE records have xmax twice and don't differentiate between new
> > and old. I think that was
On Tue, Apr 11, 2023 at 11:48 AM Peter Geoghegan wrote:
> Attached revision deals with this by spelling out the names in full
> (e.g., "old_xmax" and "new_xmax"). It also reorders the output fields
> to match the order from the physical UPDATE, HOT_UPDATE, and LOCK WAL
> record types, on the
On Tue, Apr 11, 2023 at 10:34 AM Peter Geoghegan wrote:
> > description | off: 119, xmax: 1105, flags: 0x00, old_infobits:
> > [], new off: 100, xmax 0
>
> That doesn't seem great to me either. I don't like this ambiguity,
> because it seems like it makes the description hard to parse in a
On Tue, Apr 11, 2023 at 7:40 AM Melanie Plageman
wrote:
> static void
> infobits_desc(StringInfo buf, uint8 infobits, const char *keyname)
> {
> appendStringInfo(buf, "%s: [", keyname);
>
> Why can we assume that there will be no space at the end here?
I don't think that anybody is going to
Hi,
static void
infobits_desc(StringInfo buf, uint8 infobits, const char *keyname)
{
appendStringInfo(buf, "%s: [", keyname);
Why can we assume that there will be no space at the end here?
I know we need to be able to avoid doing the comma overwriting if no
flags were set. In general, we
On Mon, Apr 10, 2023 at 5:23 PM Melanie Plageman
wrote:
> If you keep the name, I'd explain it briefly in a comment above the code
> then -- for those of us who spend less time with btrees. It is a tool
> that will be often used by developers, so it is not unreasonable to
> assume they may read
On Mon, Apr 10, 2023 at 04:31:44PM -0700, Peter Geoghegan wrote:
> On Mon, Apr 10, 2023 at 3:04 PM Melanie Plageman
> wrote:
> >
> > I will say that the prefix of p in "ptid" makes it sound like pointer to
> > a tid, which I don't believe is what you meant.
>
> I was thinking of the symbol
On Mon, Apr 10, 2023 at 3:04 PM Melanie Plageman
wrote:
> I took a look at the first patch even though you've pushed the bugfix
> part. Any reason you didn't use array_desc() for the inner array (of
> "ptids")? I find that following the pattern of using array_desc (when it
> is correct, of
On Sun, Apr 9, 2023 at 8:12 PM Peter Geoghegan wrote:
>
> On Fri, Apr 7, 2023 at 4:46 PM Peter Geoghegan wrote:
> > Pushed that one too.
>
> I noticed that the nbtree VACUUM and DELETE record types have their
> update/xl_btree_update arrays output incorrectly. We cannot use the
> generic
On Sun, Apr 9, 2023 at 5:12 PM Peter Geoghegan wrote:
> I noticed that the nbtree VACUUM and DELETE record types have their
> update/xl_btree_update arrays output incorrectly. We cannot use the
> generic array_desc() approach with xl_btree_update elements, because
> they're variable-width
On Fri, Apr 7, 2023 at 4:46 PM Peter Geoghegan wrote:
> Pushed that one too.
I noticed that the nbtree VACUUM and DELETE record types have their
update/xl_btree_update arrays output incorrectly. We cannot use the
generic array_desc() approach with xl_btree_update elements, because
they're
On Fri, Apr 7, 2023 at 4:21 PM Melanie Plageman
wrote:
> It's come to my attention that I forgot to include the btree patch earlier.
Pushed that one too.
Also removed the use of the "restrict" keyword here.
Thanks
--
Peter Geoghegan
On Fri, Apr 7, 2023 at 7:09 PM Peter Geoghegan wrote:
>
> On Fri, Apr 7, 2023 at 4:01 PM Melanie Plageman
> wrote:
> > LGTM
>
> Pushed, thanks.
It's come to my attention that I forgot to include the btree patch earlier.
PFA
From 4f502b2513ba79d738e7ed87aaf7d18ed2a2e30f Mon Sep 17 00:00:00 2001
On Fri, Apr 7, 2023 at 4:01 PM Melanie Plageman
wrote:
> LGTM
Pushed, thanks.
--
Peter Geoghegan
On Fri, Apr 7, 2023 at 5:43 PM Peter Geoghegan wrote:
>
> On Fri, Apr 7, 2023 at 1:33 PM Melanie Plageman
> wrote:
> > Attached v3 is cleaned up and includes a pg_walinspect docs update as
> > well as some edited comments in rmgr_utils.c
>
> Attached v4 has some small tweaks on your v3. Mostly
On Fri, Apr 7, 2023 at 1:33 PM Melanie Plageman
wrote:
> Attached v3 is cleaned up and includes a pg_walinspect docs update as
> well as some edited comments in rmgr_utils.c
Attached v4 has some small tweaks on your v3. Mostly just whitespace
tweaks. Two slightly notable tweaks:
* I changed the
Attached v3 is cleaned up and includes a pg_walinspect docs update as
well as some edited comments in rmgr_utils.c
On Mon, Mar 27, 2023 at 6:27 PM Peter Geoghegan wrote:
>
> On Mon, Mar 27, 2023 at 2:29 PM Melanie Plageman
> wrote:
> > I went to add dedup records and noticed that since the
On Mon, Mar 27, 2023 at 2:29 PM Melanie Plageman
wrote:
> I went to add dedup records and noticed that since the actual
> BTDedupInterval struct is what is put in the xlog, I would need access
> to that type from nbtdesc.c, however, including nbtree.h doesn't seem to
> work because it includes
On Mon, Mar 13, 2023 at 9:41 PM Peter Geoghegan wrote:
> On Mon, Mar 13, 2023 at 4:01 PM Melanie Plageman
> wrote:
>
> > I have added detail to xl_btree_delete and xl_btree_vacuum. I have added
> > the updated/deleted target offset numbers and the updated tuples
> > metadata.
> >
> > I wondered
On Tue, Mar 21, 2023 at 3:37 PM Peter Geoghegan wrote:
> One problem that I often run into when performing analysis of VACUUM
> using pg_walinspect is the issue of *who* pruned which heap page, for
> any given PRUNE record. Was it VACUUM/autovacuum, or was it
> opportunistic pruning? There is no
On Mon, Mar 13, 2023 at 6:41 PM Peter Geoghegan wrote:
> There are several different things that seem important to me
> personally. These are in tension with each other, to a degree. These
> are:
>
> 1. Like Andres, I'd really like to have some way of inspecting things
> like heapam PRUNE,
On Mon, Mar 13, 2023 at 4:01 PM Melanie Plageman
wrote:
> On Fri, Jan 27, 2023 at 3:02 PM Robert Haas wrote:
> > I'm not sure what's best in terms of formatting details but I
> > definitely like the idea of making pg_waldump show more details.
> If I'm not mistaken, this would be quite
Thanks for the various perspectives and feedback.
Attached v2 has additional info for xl_btree_vacuum and xl_btree_delete.
I've quoted various emails by various senders below and replied.
On Fri, Jan 27, 2023 at 3:02 PM Robert Haas wrote:
>
> On Fri, Jan 27, 2023 at 12:24 PM Melanie Plageman
>
On 01.03.23 17:11, Melanie Plageman wrote:
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0.sql
b/contrib/pg_walinspect/pg_walinspect--1.0.sql
index 08b3dd5556..eb8ff82dd8 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0.sql
@@ -17,7
On Tue, Jan 31, 2023 at 5:48 PM Peter Geoghegan wrote:
>
> On Tue, Jan 31, 2023 at 1:52 PM Peter Geoghegan wrote:
> > > I would also like to see functions like XLogRecGetBlockRefInfo() pass
> > > something more useful than a stringinfo buffer so that we could easily
> > > extract out the
On Wed, Feb 1, 2023 at 12:47 PM Peter Geoghegan wrote:
> On Wed, Feb 1, 2023 at 5:20 AM Robert Haas wrote:
> > If we're dumping a lot of details out of each WAL record, we might
> > want to switch to a multi-line format of some kind. No one enjoys a
> > 460-character wide line, let alone 46000.
On Wed, Feb 1, 2023 at 5:20 AM Robert Haas wrote:
> If we're dumping a lot of details out of each WAL record, we might
> want to switch to a multi-line format of some kind. No one enjoys a
> 460-character wide line, let alone 46000.
I generally prefer it when I can use psql without using
On Tue, Jan 31, 2023 at 6:20 PM Peter Geoghegan wrote:
> Actually the really wide output comes from COMMIT records. After I run
> the regression tests, and execute some of my own custom pg_walinspect
> queries, I see that some individual COMMIT records have a
> length(description) of over 10,000
On Tue, Jan 31, 2023 at 1:52 PM Peter Geoghegan wrote:
> Obviously what you're doing here will lead to a significant increase
> in the verbosity of the output for affected WAL records. I don't feel
> too bad about that, though. It's really an existing problem, and one
> that should be fixed
On Tue, Jan 31, 2023 at 1:52 PM Peter Geoghegan wrote:
> > I would also like to see functions like XLogRecGetBlockRefInfo() pass
> > something more useful than a stringinfo buffer so that we could easily
> > extract out the relfilenode in pgwalinspect.
>
> That does seem particularly important.
On Fri, Jan 27, 2023 at 9:24 AM Melanie Plageman
wrote:
> I have taken a stab at doing some of the tasks listed in this email.
Cool.
> I have made the new files rmgr_utils.c/h.
>
> I have come up with a standard format that I like for the output and
> used it in all the heap record types.
>
>
On Fri, Jan 27, 2023 at 12:24 PM Melanie Plageman
wrote:
> I believe I have addressed this in the attached patch.
I'm not sure what's best in terms of formatting details but I
definitely like the idea of making pg_waldump show more details. I'd
even like to have a way to extract the tuple data,
Hi,
I have taken a stab at doing some of the tasks listed in this email.
I have made the new files rmgr_utils.c/h.
I have come up with a standard format that I like for the output and
used it in all the heap record types.
Examples below:
snapshotConflictHorizon: 2184, nplans: 2, plans [ {
On Wed, Jan 11, 2023 at 3:11 PM Peter Geoghegan wrote:
> On Wed, Jan 11, 2023 at 3:00 PM Andres Freund wrote:
> > What are your thoughts about the place for the helper functions? You're ok
> > with rmgrdesc_utils.[ch]?
>
> Yeah, that seems okay.
BTW, while playing around with this patch today,
On Wed, Jan 11, 2023 at 3:00 PM Andres Freund wrote:
> What are your thoughts about the place for the helper functions? You're ok
> with rmgrdesc_utils.[ch]?
Yeah, that seems okay.
We may well need to put more stuff in that file. We're overdue a big
overhaul of the rmgr output, so that
Hi,
On 2023-01-11 14:53:54 -0800, Peter Geoghegan wrote:
> On Tue, Jan 10, 2023 at 11:35 AM Andres Freund wrote:
> > Nontrivial, I'm afraid. We don't pass any relevant parameters to rm_desc:
> > void(*rm_desc) (StringInfo buf, XLogReaderState
> > *record);
> >
> > so we'd
On Tue, Jan 10, 2023 at 11:35 AM Andres Freund wrote:
> Nontrivial, I'm afraid. We don't pass any relevant parameters to rm_desc:
> void(*rm_desc) (StringInfo buf, XLogReaderState *record);
>
> so we'd need to patch all of them. That might be worth doing at some point,
> but I
Hi,
On 2023-01-09 19:59:42 -0800, Peter Geoghegan wrote:
> On Mon, Jan 9, 2023 at 1:58 PM Andres Freund wrote:
> > A couple times when investigating data corruption issues, the last time just
> > yesterday in [1], I needed to see the offsets affected by PRUNE and VACUUM
> > records. As that's
On Mon, Jan 9, 2023 at 1:58 PM Andres Freund wrote:
> A couple times when investigating data corruption issues, the last time just
> yesterday in [1], I needed to see the offsets affected by PRUNE and VACUUM
> records. As that's probably not just me, I think we should make that change
> in-tree.
Hi,
A couple times when investigating data corruption issues, the last time just
yesterday in [1], I needed to see the offsets affected by PRUNE and VACUUM
records. As that's probably not just me, I think we should make that change
in-tree.
The attached patch adds details to XLOG_HEAP2_PRUNE,
49 matches
Mail list logo
