Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-23 Thread Magnus Hagander
On Fri, Dec 23, 2022 at 2:06 AM Michael Paquier wrote: > On Thu, Dec 22, 2022 at 05:03:35PM +0530, Bharath Rupireddy wrote: > > On Thu, Dec 22, 2022 at 4:57 PM Michael Paquier > wrote: > >> As in using "sequence number" removing "file" from the docs and > >> changing the OUT parameter name to

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-22 Thread Michael Paquier
On Thu, Dec 22, 2022 at 05:03:35PM +0530, Bharath Rupireddy wrote: > On Thu, Dec 22, 2022 at 4:57 PM Michael Paquier wrote: >> As in using "sequence number" removing "file" from the docs and >> changing the OUT parameter name to segment_number rather than segno? >> Fine by me. > > +1. Okay,

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-22 Thread Bharath Rupireddy
On Thu, Dec 22, 2022 at 4:57 PM Michael Paquier wrote: > > On Thu, Dec 22, 2022 at 05:19:24PM +0900, Kyotaro Horiguchi wrote: > > In the first place "file sequence number" and "segno" can hardly be > > associated by appearance by readers, I think. (Yeah, we can identify > > that since the

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-22 Thread Michael Paquier
On Thu, Dec 22, 2022 at 05:19:24PM +0900, Kyotaro Horiguchi wrote: > In the first place "file sequence number" and "segno" can hardly be > associated by appearance by readers, I think. (Yeah, we can identify > that since the another parameter is identifiable alone.) Why don't we > spell out the

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-22 Thread Kyotaro Horiguchi
At Thu, 22 Dec 2022 10:09:23 +0530, Bharath Rupireddy wrote in > On Thu, Dec 22, 2022 at 7:57 AM Michael Paquier wrote: > > > > On Wed, Dec 21, 2022 at 10:22:02PM +0100, Magnus Hagander wrote: > > > Basically, we take one thing and turn it into 3. That very naturally rings > > > with "split"

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-21 Thread Bharath Rupireddy
On Thu, Dec 22, 2022 at 7:57 AM Michael Paquier wrote: > > On Wed, Dec 21, 2022 at 10:22:02PM +0100, Magnus Hagander wrote: > > Basically, we take one thing and turn it into 3. That very naturally rings > > with "split" to me. > > > > Parse might work as well, certainly better than dissect. I'd

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-21 Thread Michael Paquier
On Wed, Dec 21, 2022 at 10:22:02PM +0100, Magnus Hagander wrote: > Basically, we take one thing and turn it into 3. That very naturally rings > with "split" to me. > > Parse might work as well, certainly better than dissect. I'd still prefer > split though. Honestly, I don't have any

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-21 Thread Magnus Hagander
On Wed, Dec 21, 2022 at 1:09 AM Michael Paquier wrote: > On Tue, Dec 20, 2022 at 06:04:40PM +0530, Bharath Rupireddy wrote: > > On Tue, Dec 20, 2022 at 1:27 PM Magnus Hagander > wrote: > >> Caught this thread late. To me, pg_dissect_walfile_name() is a > >> really strange name for a function.

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-20 Thread Bharath Rupireddy
On Wed, Dec 21, 2022 at 5:39 AM Michael Paquier wrote: > > On Tue, Dec 20, 2022 at 06:04:40PM +0530, Bharath Rupireddy wrote: > > On Tue, Dec 20, 2022 at 1:27 PM Magnus Hagander wrote: > >> Caught this thread late. To me, pg_dissect_walfile_name() is a > >> really strange name for a function.

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-20 Thread Michael Paquier
On Tue, Dec 20, 2022 at 06:04:40PM +0530, Bharath Rupireddy wrote: > On Tue, Dec 20, 2022 at 1:27 PM Magnus Hagander wrote: >> Caught this thread late. To me, pg_dissect_walfile_name() is a >> really strange name for a function. Grepping our I code I see the >> term dissect s used somewhere

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-20 Thread Ian Lawrence Barwick
2022年12月20日(火) 21:35 Bharath Rupireddy : > > On Tue, Dec 20, 2022 at 1:27 PM Magnus Hagander wrote: > > > > On Tue, Dec 20, 2022 at 5:40 AM Michael Paquier wrote: > >> > >> On Tue, Dec 20, 2022 at 09:01:02AM +0900, Michael Paquier wrote: > >> > Yeah, my mind was considering as well yesterday the

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-20 Thread Bharath Rupireddy
On Tue, Dec 20, 2022 at 1:27 PM Magnus Hagander wrote: > > On Tue, Dec 20, 2022 at 5:40 AM Michael Paquier wrote: >> >> On Tue, Dec 20, 2022 at 09:01:02AM +0900, Michael Paquier wrote: >> > Yeah, my mind was considering as well yesterday the addition of a note >> > in the docs about something

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-19 Thread Magnus Hagander
On Tue, Dec 20, 2022 at 5:40 AM Michael Paquier wrote: > On Tue, Dec 20, 2022 at 09:01:02AM +0900, Michael Paquier wrote: > > Yeah, my mind was considering as well yesterday the addition of a note > > in the docs about something among these lines, so fine by me. > > And applied that, after

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-19 Thread Michael Paquier
On Tue, Dec 20, 2022 at 09:01:02AM +0900, Michael Paquier wrote: > Yeah, my mind was considering as well yesterday the addition of a note > in the docs about something among these lines, so fine by me. And applied that, after tweaking a few tiny things on a last lookup with a catversion bump.

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-19 Thread Michael Paquier
On Mon, Dec 19, 2022 at 05:51:19PM +0530, Bharath Rupireddy wrote: > A nitpick - can we also specify a use case for the function > pg_dissect_walfile_name(), that is, computing LSN from offset and WAL > file name, something like [1]? Yeah, my mind was considering as well yesterday the addition of

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-19 Thread Bharath Rupireddy
On Mon, Dec 19, 2022 at 5:22 PM Bharath Rupireddy wrote: > > On Mon, Dec 19, 2022 at 1:37 PM Michael Paquier wrote: > > > > On Tue, Dec 13, 2022 at 09:32:19PM +0530, Bharath Rupireddy wrote: > > > Okay, here's the v5 patch that I could come up with. It basically adds > > > functions for

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-19 Thread Bharath Rupireddy
On Mon, Dec 19, 2022 at 1:37 PM Michael Paquier wrote: > > On Tue, Dec 13, 2022 at 09:32:19PM +0530, Bharath Rupireddy wrote: > > Okay, here's the v5 patch that I could come up with. It basically adds > > functions for dissecting WAL file names and computing offset from lsn. > > Thoughts? > > I

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-19 Thread Michael Paquier
On Tue, Dec 13, 2022 at 09:32:19PM +0530, Bharath Rupireddy wrote: > Okay, here's the v5 patch that I could come up with. It basically adds > functions for dissecting WAL file names and computing offset from lsn. > Thoughts? I had a second look at that, and I still have mixed feelings about the

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-18 Thread Michael Paquier
On Tue, Dec 06, 2022 at 04:27:50PM +0900, Kyotaro Horiguchi wrote: > At Mon, 5 Dec 2022 10:03:55 +0900, Michael Paquier > wrote in >> Hence I would tend to let XLogFromFileName do the job, while having a >> SQL function that is just a thin wrapper around it that returns the >> segment TLI and

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-13 Thread Bharath Rupireddy
On Tue, Dec 6, 2022 at 4:46 PM Bharath Rupireddy wrote: > > That said, I think we can have a single function > pg_dissect_walfile_name(wal_file_name, offset optional) returning > segno (XLogSegNo - physical log file sequence number), tli, lsn (if > offset is given). This way there is no need for

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-06 Thread Bharath Rupireddy
On Tue, Dec 6, 2022 at 12:57 PM Kyotaro Horiguchi wrote: > > At Mon, 5 Dec 2022 13:13:23 +0900, Michael Paquier > wrote in > > On Mon, Dec 05, 2022 at 08:48:25AM +0530, Bharath Rupireddy wrote: > > > So, a SQL function pg_dissect_walfile_name (or some other better name) > > > given a WAL file

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-05 Thread Kyotaro Horiguchi
At Mon, 5 Dec 2022 13:13:23 +0900, Michael Paquier wrote in > On Mon, Dec 05, 2022 at 08:48:25AM +0530, Bharath Rupireddy wrote: > > So, a SQL function pg_dissect_walfile_name (or some other better name) > > given a WAL file name returns the tli and seg number. Then the > >

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-04 Thread Michael Paquier
On Mon, Dec 05, 2022 at 08:48:25AM +0530, Bharath Rupireddy wrote: > So, a SQL function pg_dissect_walfile_name (or some other better name) > given a WAL file name returns the tli and seg number. Then the > pg_walfile_offset_lsn can just be a SQL-defined function (in > system_functions.sql) using

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-04 Thread Bharath Rupireddy
On Mon, Dec 5, 2022 at 6:34 AM Michael Paquier wrote: > > Regarding pg_walfile_offset_lsn(), I am not sure that this is the best > move we can do as it is possible to compile a LSN from 0/0 with just a > segment number, say: > select '0/0'::pg_lsn + :segno * setting::int + :offset > from

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-04 Thread Michael Paquier
On Sat, Dec 03, 2022 at 09:07:38AM +0530, Bharath Rupireddy wrote: > Yes, I removed those changes. Even if someone sees an offset of a > record within a WAL file elsewhere, they have the new utility function > (0002) pg_walfile_offset_lsn(). > > I'm attaching the v4 patch set for further review.

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-02 Thread Bharath Rupireddy
On Fri, Dec 2, 2022 at 12:50 PM Michael Paquier wrote: > > On Thu, Nov 17, 2022 at 11:53:23AM +0530, Bharath Rupireddy wrote: > > Please do, if you feel so. Thanks for your review. > > I don't really mind the addition of the LSN when operating on a given > record where we are reading a location,

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-01 Thread Michael Paquier
On Thu, Nov 17, 2022 at 11:53:23AM +0530, Bharath Rupireddy wrote: > Please do, if you feel so. Thanks for your review. I don't really mind the addition of the LSN when operating on a given record where we are reading a location, like in the five error paths for the header validation or the three

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-11-16 Thread Bharath Rupireddy
On Wed, Nov 16, 2022 at 6:55 PM Maxim Orlov wrote: > >> These functions are marked as 'STRICT', meaning a null is returned, >> without even calling the function, if any of the input parameters is >> null. I think we can keep the behaviour the same as its friends. > > Thanks for the explanations.

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-11-16 Thread Maxim Orlov
On Tue, 15 Nov 2022 at 13:02, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Fri, Nov 11, 2022 at 5:52 PM Maxim Orlov wrote: > > > But, in my view, some improvements may be proposed. We should be more, > let's say, cautious (or a paranoid if you wish), > > in

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-11-15 Thread Bharath Rupireddy
On Fri, Nov 11, 2022 at 5:52 PM Maxim Orlov wrote: > > Hi! > > I've watched over the patch and consider it useful. Applies without > conflicts. The functionality of the patch itself is > meets declared functionality. Thanks for reviewing. > But, in my view, some improvements may be proposed.

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-11-11 Thread Alvaro Herrera
Hmm, in 0002, why not return the timeline from the LSN too? It seems a waste not to have it. + ereport(ERROR, + (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE), +errmsg("\"offset\" must not be negative or greater than or "

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-11-11 Thread Maxim Orlov
Hi! I've watched over the patch and consider it useful. Applies without conflicts. The functionality of the patch itself is meets declared functionality. But, in my view, some improvements may be proposed. We should be more, let's say, cautious (or a paranoid if you wish), in

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-10-27 Thread Bharath Rupireddy
On Tue, Oct 4, 2022 at 2:58 PM Bharath Rupireddy wrote: > > On Thu, Sep 29, 2022 at 7:43 PM Bharath Rupireddy > wrote: > > > > Please see the attached v1 patch. > > FWIW, I'm attaching Nathan's patch that introduced the new function > pg_walfile_offset_lsn as 0002 in the v1 patch set. Here's the

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-10-04 Thread Bharath Rupireddy
On Thu, Sep 29, 2022 at 7:43 PM Bharath Rupireddy wrote: > > Please see the attached v1 patch. FWIW, I'm attaching Nathan's patch that introduced the new function pg_walfile_offset_lsn as 0002 in the v1 patch set. Here's the CF entry - https://commitfest.postgresql.org/40/3909/. Please consider

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-29 Thread Bharath Rupireddy
On Tue, Sep 27, 2022 at 8:31 AM Kyotaro Horiguchi wrote: > > If all error-emitting site knows the LSN, we don't need the context > message. But *I* would like that the additional message looks like > "while reading record at LSN %X/%X" or slightly shorter version of > it. Because the targetRecPtr

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-26 Thread Kyotaro Horiguchi
At Tue, 20 Sep 2022 17:40:36 +0530, Bharath Rupireddy wrote in > On Tue, Sep 20, 2022 at 12:57 PM Alvaro Herrera > wrote: > > > > On 2022-Sep-19, Bharath Rupireddy wrote: > > > > > We have a bunch of messages [1] that have an offset, but not LSN in > > > the error message. Firstly, is there

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-20 Thread Bharath Rupireddy
On Tue, Sep 20, 2022 at 12:57 PM Alvaro Herrera wrote: > > On 2022-Sep-19, Bharath Rupireddy wrote: > > > We have a bunch of messages [1] that have an offset, but not LSN in > > the error message. Firstly, is there an easiest way to figure out LSN > > from offset reported in the error messages?

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-20 Thread Bharath Rupireddy
On Tue, Sep 20, 2022 at 9:02 AM Nathan Bossart wrote: > > On Mon, Sep 19, 2022 at 03:16:42PM -0700, Nathan Bossart wrote: > > It seems like you want the opposite of pg_walfile_name_offset(). Perhaps > > we could add a function like pg_walfile_offset_lsn() that accepts a WAL > > file name and

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-20 Thread Alvaro Herrera
On 2022-Sep-19, Bharath Rupireddy wrote: > We have a bunch of messages [1] that have an offset, but not LSN in > the error message. Firstly, is there an easiest way to figure out LSN > from offset reported in the error messages? If not, is adding LSN to > these messages along with offset a good

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-19 Thread Nathan Bossart
On Mon, Sep 19, 2022 at 03:16:42PM -0700, Nathan Bossart wrote: > It seems like you want the opposite of pg_walfile_name_offset(). Perhaps > we could add a function like pg_walfile_offset_lsn() that accepts a WAL > file name and byte offset and returns the LSN. Like so... -- Nathan Bossart

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-19 Thread Nathan Bossart
On Mon, Sep 19, 2022 at 07:26:57PM +0530, Bharath Rupireddy wrote: > We have a bunch of messages [1] that have an offset, but not LSN in > the error message. Firstly, is there an easiest way to figure out LSN > from offset reported in the error messages? If not, is adding LSN to > these messages

Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-09-19 Thread Bharath Rupireddy
Hi, I was recently asked about converting an offset reported in WAL read error messages[1] to an LSN with which pg_waldump can be used to verify the records or WAL file around that LSN (basically one can filter out the output based on LSN). AFAICS, there's no function that takes offset as an