回复: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi orlo...@gmail.com That's good news, I think we can continue discuss for (https://commitfest.postgresql.org/43/3594/) Best regards 发件人: John Naylor 发送时间: 2023年12月4日 18:22 收件人: Pavel Borisov 抄送: Alexander Korotkov ; Tom Lane ; Alexander Lakhin ;

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, Dec 4, 2023 at 3:12 PM Pavel Borisov wrote: > I think this is complete and could be closed. Done.

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, 4 Dec 2023 at 10:34, John Naylor wrote: > On Thu, Nov 30, 2023 at 4:37 PM Alexander Korotkov > wrote: > > > > On Thu, Nov 30, 2023 at 10:29 AM Pavel Borisov > wrote: > > > Agree. The fix is attached. > > > > What an oversight. > > Thank you, pushed! > > With that, is there any more

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Thu, Nov 30, 2023 at 4:37 PM Alexander Korotkov wrote: > > On Thu, Nov 30, 2023 at 10:29 AM Pavel Borisov wrote: > > Agree. The fix is attached. > > What an oversight. > Thank you, pushed! With that, is there any more work pending, or can we close the CF entry?

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Thu, Nov 30, 2023 at 10:29 AM Pavel Borisov wrote: > Agree. The fix is attached. What an oversight. Thank you, pushed! -- Regards, Alexander Korotkov

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Thu, 30 Nov 2023 at 08:03, Tom Lane wrote: > Alexander Lakhin writes: > > And a warning: > > $ CC=gcc-12 CFLAGS="-Wall -Wextra -Wno-unused-parameter > -Wno-sign-compare -Wno-clobbered > > -Wno-missing-field-initializers" ./configure -q && make -s > > slru.c:63:1: warning: ‘inline’ is not at

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Alexander Lakhin writes: > And a warning: > $ CC=gcc-12 CFLAGS="-Wall -Wextra -Wno-unused-parameter -Wno-sign-compare > -Wno-clobbered > -Wno-missing-field-initializers" ./configure -q && make -s > slru.c:63:1: warning: ‘inline’ is not at beginning of declaration > [-Wold-style-declaration] >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Андрей, привет! Текущее положение у меня такое. . *pg_stats and range statisticsTracking statements entry timestamp in pg_stat_statements* Уже закоммичены. *XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)* Если всё будет и замечаний не возникнет ок, завтра утром

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hello Alexander, 27.11.2023 02:43, Alexander Korotkov wrote: v61 looks good to me. I'm going to push it as long as there are no objections. I've looked at the patch set and found a typo: occured And a warning: $ CC=gcc-12 CFLAGS="-Wall -Wextra -Wno-unused-parameter -Wno-sign-compare

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Alexander, Maxim, Thank you for revisions. On Thu, Nov 9, 2023 at 6:22 PM Maxim Orlov wrote: > Aleksander Alekseev, > > > Maxim, > > I see both of us accounted for Alexanders feedback and submitted v59. > > Your newer version seems to have issues on cfbot, so resubmitting the > > previous

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Aleksander Alekseev, > Maxim, > I see both of us accounted for Alexanders feedback and submitted v59. > Your newer version seems to have issues on cfbot, so resubmitting the > previous patchset that passes the tests. Please feel free to add > changes. For unknown reasons, I do not receive any of

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Maxim, I see both of us accounted for Alexanders feedback and submitted v59. Your newer version seems to have issues on cfbot, so resubmitting the previous patchset that passes the tests. Please feel free to add changes. > See SlruCorrectSegmentFilenameLength(): > > ``` > if

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, 6 Nov 2023 at 16:07, Alexander Korotkov wrote: > Hi! > > On Wed, Jul 5, 2023 at 4:46 PM Aleksander Alekseev < > aleksan...@timescale.com> wrote: > > PFE the corrected patchset v58. > > I'd like to revive this thread. > Hi! Great news! > BTW, there is a typo in a word "exceeed". >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi again, > PFA the corrected patchset v59. On second thought, I believe this Assert is incorrect: ``` +else +{ +Assert(segno >= 0 && segno <= 0x); +return snprintf(path, MAXPGPATH, "%s/%04X", (ctl)->Dir, +(unsigned int) segno); +} ```

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi Alexander, > -#define TransactionIdToCTsPage(xid) \ > - ((xid) / (TransactionId) COMMIT_TS_XACTS_PER_PAGE) > + > +/* > + * Although we return an int64 the actual value can't currently exceeed > 2**32. > + */ > +static inline int64 > +TransactionIdToCTsPage(TransactionId xid) > +{ > +

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, Nov 6, 2023 at 4:38 PM Aleksander Alekseev wrote: > > > PFE the corrected patchset v58. > > > > I'd like to revive this thread. > > Many thanks for your comments and suggestions. > > > I think it worth adding asserts here to verify there is no overflow making us mapping different segments

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Alexander, > > PFE the corrected patchset v58. > > I'd like to revive this thread. Many thanks for your comments and suggestions. > I think it worth adding asserts here to verify there is no overflow making us > mapping different segments into the same files. Sorry, I didn't understand this

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > > > If I remember right, the compiler will make equivalent code from > > > inline functions and macros, and functions has an additional benefit: > > > the compiler will report type mismatch if any. That was the only > > > reason. > > > > Then it's OK to leave it as an inline function. +1

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, 6 Nov 2023 at 18:01, Alexander Korotkov wrote: > > On Mon, Nov 6, 2023 at 3:42 PM Pavel Borisov wrote: > > > > On Mon, 6 Nov 2023 at 17:07, Alexander Korotkov > > wrote: > > > On Wed, Jul 5, 2023 at 4:46 PM Aleksander Alekseev > > > wrote: > > > > PFE the corrected patchset v58. > >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, Nov 6, 2023 at 3:42 PM Pavel Borisov wrote: > > On Mon, 6 Nov 2023 at 17:07, Alexander Korotkov wrote: > > On Wed, Jul 5, 2023 at 4:46 PM Aleksander Alekseev > > wrote: > > > PFE the corrected patchset v58. > > > > I'd like to revive this thread. > > > > This patchset is extracted from

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, 6 Nov 2023 at 17:07, Alexander Korotkov wrote: > > Hi! > > On Wed, Jul 5, 2023 at 4:46 PM Aleksander Alekseev > wrote: > > PFE the corrected patchset v58. > > I'd like to revive this thread. > > This patchset is extracted from a larger patchset implementing 64-bit xids. > It converts

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi! On Wed, Jul 5, 2023 at 4:46 PM Aleksander Alekseev wrote: > PFE the corrected patchset v58. I'd like to revive this thread. This patchset is extracted from a larger patchset implementing 64-bit xids. It converts page numbers in SLRUs into 64 bits. The most SLRUs save the same file naming

回复: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> > This patch makes the filenames always 12 characters wide (for SLRUs that > > opt-in to the new naming). That's actually not enough for the full range > > that a 64 bit page number can represent. Should we make it 16 characters > > now, to avoid repeating the same mistake we made earlier? Or

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > > Reviewing this now. I think it's almost ready to be committed. > > > > There's another big effort going on to move SLRUs to the regular buffer > > cache (https://commitfest.postgresql.org/43/3514/). I wonder how moving > > to 64 bit page numbers will affect that. BlockNumber is still 32

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > Reviewing this now. I think it's almost ready to be committed. > > There's another big effort going on to move SLRUs to the regular buffer > cache (https://commitfest.postgresql.org/43/3514/). I wonder how moving > to 64 bit page numbers will affect that. BlockNumber is still 32 bits, >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 09/03/2023 17:21, Aleksander Alekseev wrote: v57-0001-Index-SLRUs-by-64-bit-integers-rather-than-by-32.patch Reviewing this now. I think it's almost ready to be committed. There's another big effort going on to move SLRUs to the regular buffer cache

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > As suggested before by Heikki Linnakangas, I've added a patch for making 2PC > transaction state 64-bit. > At first, my intention was to rebuild all twophase interface to use > FullTransactionId. But doing this in a proper > manner would lead to switching from TransactionId to

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi! As suggested before by Heikki Linnakangas, I've added a patch for making 2PC transaction state 64-bit. At first, my intention was to rebuild all twophase interface to use FullTransactionId. But doing this in a proper manner would lead to switching from TransactionId to FullTransactionId in

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > 1. Use internal 64 bit page numbers in SLRUs without changing segments naming. > 2. Use the larger segment file names in async.c, to lift the current 8 > GB limit on the max number of pending notifications. > [...] > > Where our main focus for PG16 is going to be 1 and 2, and we may try >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Tue, 7 Mar 2023 at 15:38, Heikki Linnakangas wrote: > > That is true for pg_multixact/offsets. We will indeed need to add an > epoch and introduce the concept of FullMultiXactIds for that. However, > we can change pg_multixact/members independently of that. We can extend > MultiXactOffset

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > Here's how I see the development path for this [...] > So, in my view, the plan should be [...] > Thoughts? The plan looks great! I would also explicitly include this: > As we start to refactor these things, I also think it would be good to > have more explicit tracking of the valid

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 07/03/2023 13:38, Maxim Orlov wrote: As for making pg_multixact 64 bit, I spend the last couple of days to make proper pg_upgrade for pg_multixact's and for pg_xact's with wraparound and I've understood, that it is not a simple task compare to pg_xact's. The problem is, we do not have epoch

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Tue, 17 Jan 2023 at 16:33, Aleksander Alekseev wrote: > Hi hackers, > > Maxim, perhaps you could share with us what your reasoning was here? > > I'm really sorry for late response, but better late than never. Yes, we can not access shared memory without lock. In this particular case, we use

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 01/03/2023 12:21, Aleksander Alekseev wrote: Hi, I'm surprised that these patches extend the page numbering to 64 bits, but never actually uses the high bits. The XID "epoch" is not used, and pg_xact still wraps around and the segment names are still reused. I thought we could stop doing

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > I'm surprised that these patches extend the page numbering to 64 bits, > but never actually uses the high bits. The XID "epoch" is not used, and > pg_xact still wraps around and the segment names are still reused. I > thought we could stop doing that. To clarify, the idea is to let CLOG

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 28/02/2023 16:17, Maxim Orlov wrote: Either I do not understand something, or the files from pg_commit_ts directory are not copied. Huh, yeah you're right, pg_upgrade doesn't copy pg_commit_ts. - Heikki

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, 27 Feb 2023 at 12:10, Heikki Linnakangas wrote: > (CC list trimmed, gmail wouldn't let me send otherwise) > > That sounds right for pg_serial, pg_notify, and pg_subtrans. But not for > pg_commit_ts and the pg_multixacts. > > This needs tests for pg_upgrading those SLRUs, after 0, 1 and N

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

(CC list trimmed, gmail wouldn't let me send otherwise) On 22/02/2023 16:29, Maxim Orlov wrote: On Tue, 21 Feb 2023 at 16:59, Aleksander Alekseev mailto:aleksan...@timescale.com>> wrote: One thing that still bothers me is that during the upgrade we only migrate the CLOG segments (i.e.

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Tue, 21 Feb 2023 at 16:59, Aleksander Alekseev wrote: > Hi, > > One thing that still bothers me is that during the upgrade we only > migrate the CLOG segments (i.e. pg_xact / pg_clog) and completely > ignore all the rest of SLRUs: > > * pg_commit_ts > * pg_multixact/offsets > *

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > The convert_pg_xact_segments() function is still obviously > overengineered. As I understand, all it has to do is simply renaming > pg_xact/ to pg_xact/. Unfortunately I used up all the > mana for today and have to take a long rest in order to continue. PFA the corrected

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, > OK, here is the patchset v53 where I mostly modified the commit > messages. It is explicitly said that 0001 modifies the WAL records and > why we decided to do it in this patch. Additionally any mention of > 64-bit XIDs is removed since it is not guaranteed that the rest of the > patches

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi hackers, > Yeah, pg_upgrade will briefly start and stop the old server to make > sure all the WAL is replayed, and won't transfer any of the files > over. AFAIK, major-version WAL changes are fine; it was the previous > claim that we could do it in a minor version that I was unsure about. OK,

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Wed, Jan 11, 2023 at 1:48 AM Aleksander Alekseev wrote: > After reading [1] carefully it looks like we shouldn't worry about > this. The upgrade procedure explicitly requires to run `pg_ctl stop` > during step 8 of the upgrade procedure, i.e. not in the immediate mode > [2]. Yeah, pg_upgrade

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi Maxim, > Secondly, shouldn't we introduce a new WAL record type in order to > make the code backward compatible with previous PG versions? I'm not > 100% sure how the upgrade procedure works in all the details. If it > requires the DBMS to be gracefully shut down before the upgrade then > we

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi Maxim, > Here is a new patch set. > I've added comments and make use GetClogDirName call in copy_subdir_files. Jacob Champion pointed out (offlist, cc:'ed) that we may be wrong on this one: > 0001 patch changes the SLRU internals without affecting the callers. > 0001 - should make SLRU

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi! Here is a new patch set. I've added comments and make use GetClogDirName call in copy_subdir_files. -- Best regards, Maxim Orlov. v52-0003-Make-pg_upgrade-from-32-bit-to-64-bit-SLRU.patch Description: Binary data v52-0002-Use-64-bit-pages-representation-in-SLRU-callers.patch

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Fri, 6 Jan 2023 at 09:51, Japin Li wrote: > > For v51-0003. We can use GetClogDirName instead of GET_MAJOR_VERSION in > copy_subdir_files(). > Of course! Tanks! I'll address this in the next iteration, v52. -- Best regards, Maxim Orlov.

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi Andrey, > Hi! I think that 64-bit xids are a very important feature and I want > to help advance it. That's why I want to try to understand a patch > better. Thanks for your interest to the patchset! > Do I get it right that the proposed v51 patchset only changes the SLRU > filenames and

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> > Do I get it right that the proposed v51 patchset only changes the SLRU > filenames and type of pageno representation? Is SLRU wraparound still > exactly there after 0x byte? > After applying the whole patch set, SLRU will become 64–bit without a wraparound. Thus, no wraparound should

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, 19 Dec 2022 at 22:40, Maxim Orlov wrote: > Hi! > > As a result of discussion in the thread [0], Robert Haas proposed to focus > on making SLRU 64 bit, as a first step towards 64 bit XIDs. > Here is the patch set. > > In overall, code of this patch set is based on the existing code from

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, Dec 19, 2022 at 6:41 AM Maxim Orlov wrote: > > As always, reviews and opinions are very welcome. Hi! I think that 64-bit xids are a very important feature and I want to help advance it. That's why I want to try to understand a patch better. Do I get it right that the proposed v51

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi! As a result of discussion in the thread [0], Robert Haas proposed to focus on making SLRU 64 bit, as a first step towards 64 bit XIDs. Here is the patch set. In overall, code of this patch set is based on the existing code from [0] and may be simplified, due to the fact, that

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Wed, Dec 7, 2022 at 9:50 AM Andres Freund wrote: > performing post-bootstrap initialization ... > ../src/backend/access/transam/slru.c:1520:9: runtime error: load of > misaligned address 0x7fff6362db8c for type 'int64', which requires 8 byte > alignment > 0x7fff6362db8c: note: pointer

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, On 2022-12-07 11:40:08 +0300, Aleksander Alekseev wrote: > Hi Michael, > > > The CF bot is showing some failures here. You may want to > > double-check. > > Thanks! PFA v48. This causes a lot of failures with ubsan: https://cirrus-ci.com/task/6035600772431872 performing post-bootstrap

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Mon, Nov 21, 2022 at 12:21:09PM +0300, Aleksander Alekseev wrote: > After merging 1489b1ce [1] the patchset needed a rebase. PFA v47. > > [1]: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=1489b1ce The CF bot is showing some failures here. You may want to double-check. --

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022年11月3日(木) 17:15 Maxim Orlov : > > Hi! > >> >> This entry was marked "Ready for committer" in the CommitFest app but cfbot >> reports the patch no longer applies. > > Thanks for the reminder. I think, this should work. Thanks for the quick update, cfbot reports no issues. I've set the CF

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Thu, Nov 3, 2022 at 1:46 PM Maxim Orlov wrote: > > Hi! > >> >> This entry was marked "Ready for committer" in the CommitFest app but cfbot >> reports the patch no longer applies. > > Thanks for the reminder. I think, this should work. Have we measured the WAL overhead because of this patch

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022年10月10日(月) 18:16 Aleksander Alekseev : > > Hi hackers, > > > Sorry about that. > > No problem. > > > Thanks, Simon. Not 100% sure who exactly is invited to vote, but just > > in case here is +1 from me. These patches were "Ready for Committer" > > for several months now and are unlikely to get

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> > This is the wrong thread / CF entry. Please see > Yep, my fault. Sorry about that. -- Best regards, Maxim Orlov.

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Maxim, > Here is a rebased version of the patch set. This is the wrong thread / CF entry. Please see http://cfbot.cputube.org/ and https://commitfest.postgresql.org/ and the first email in the thread. -- Best regards, Aleksander Alekseev

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Tue, 26 Jul 2022 at 21:35, Simon Riggs wrote: > On Fri, 15 Jul 2022 at 12:29, Aleksander Alekseev > wrote: > > Personally I didn't expect that > > merging patches in this thread would take that long. They are in > > "Ready for Committer" state for a long time now and there are no known > >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Fri, 15 Jul 2022 at 12:29, Aleksander Alekseev wrote: > Personally I didn't expect that > merging patches in this thread would take that long. They are in > "Ready for Committer" state for a long time now and there are no known > issues with them other than unit tests for SLRU [1] should be

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Pavel, Justin, >> Is there any reason to continue with two separate threads and CF entries ? >> The original reason was to have a smaller patch for considerate late in v15. > I see the main goal of this split is to make discussion of this (easier) > thread separate to the discussion of a whole

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> > Is there any reason to continue with two separate threads and CF entries ? > The original reason was to have a smaller patch for considerate late in > v15. > > But right now, it just seems to cause every update to imply two email > messages > rather than one. > > Since the patch is split into

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Fri, May 13, 2022 at 04:21:29PM +0300, Maxim Orlov wrote: > We have posted an updated version v34 of the whole patchset in [1]. > Changes of patches 0001-0003 there are identical to v33. So, no update is > needed in this thread. Is there any reason to continue with two separate threads and CF

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> here is the rebased v32 version of the patch. > > The patchset rotted a bit. Here is a rebased version. > We have posted an updated version v34 of the whole patchset in [1]. Changes of patches 0001-0003 there are identical to v33. So, no update is needed in this thread. [1]

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi Peter, > That isn't really what I meant. When I said > > > At this point, I'm more concerned that code review is still finding > > bugs, and that we have no test coverage for any of this > > this meant especially the SLRU refactoring patch. Got it, and sorry for the confusion. I decided to

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 29.03.22 15:09, Aleksander Alekseev wrote: Hi hackers, I took another look at the patchset. Personally I don't think it will get much better than it is now. I'm inclined to change the status of the CF entry to "Ready for Committer" unless anyone disagrees. About

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi hackers, > I took another look at the patchset. Personally I don't think it will get much > better than it is now. I'm inclined to change the status of the CF entry to > "Ready for Committer" unless anyone disagrees. > > About v25-0002-Use-64-bit-format-to-output-XIDs.patch: > > I don't see

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 28.03.22 12:46, Aleksander Alekseev wrote: Personally I don't have a strong opinion here. Merging the patch sooner will allow us to move toward 64-bit XIDs faster (e.g. to gather the feedback from the early adopters, allow the translators to do their thing earlier, etc). Merging it later will

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi hackers, > Here is v30. I took another look at the patchset. Personally I don't think it will get much better than it is now. I'm inclined to change the status of the CF entry to "Ready for Committer" unless anyone disagrees. cfbot reports a problem with t/013_partition.pl but the test seems

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Just forgot to mention that answers in a previous message are describing the changes that are in v26. -- Best regards, Pavel Borisov Postgres Professional: http://postgrespro.com

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, Peter! Thanks for your review! About v25-0001-Use-unsigned-64-bit-numbering-of-SLRU-pages.patch: > > -static bool CLOGPagePrecedes(int page1, int page2); > +static bool CLOGPagePrecedes(uint64 page1, uint64 page2); > > You are changing the API from signed to unsigned. Is this intentional? >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 23.03.22 10:51, Maxim Orlov wrote: Thanks for updating the patchs. I have a little comment on 0002. +                                errmsg_internal("found xmax %llu" " (infomask 0x%04x) not frozen, not multi, not normal", +

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Wed, 23 Mar 2022 at 01:22, Maxim Orlov wrote: > Hi! > > Here is v24. Changes are: > - correct commit messages for 0001 and 0002 > - use uint64 for SLRU page numbering instead of int64 in v23 > - fix code formatting and indent > - and minor fixes in slru.c > > Reviews are very welcome! >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi hackers, > Here is v23. As was suggested by Alexander above, I've changed the order of > the patches and improved the commit message. Now, SLRU patch is the first. Many thanks! > There were proposals to make use of SLRU pages numbers that are in fact > unsigned and change from int to

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> > > I think there needs to be a bit more soul searching here on how to > > handle that in the future and how to transition it. I don't think > > targeting this patch for PG15 is useful at this point. > > The patches can be reordered so that we are still able to deliver SLRU > refactoring in

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi Peter, > I think there needs to be a bit more soul searching here on how to > handle that in the future and how to transition it. I don't think > targeting this patch for PG15 is useful at this point. The patches can be reordered so that we are still able to deliver SLRU refactoring in PG15.

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 18.03.22 16:14, Maxim Orlov wrote: Here is v22. It addresses things mentioned by Tom and Kyotaro. Also added Aleksander's changes from v21. The v22-0002-Update-XID-formatting-in-the-.po-files.patch is not necessary. That is done by a separate procedure. I'm wondering about things like

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> On 2022-03-18 18:14:52 +0300, Maxim Orlov wrote: > > Subject: [PATCH v22 3/6] Use 64-bit pages in SLRU > > > > This is one step toward 64-bit XIDs. > > I think this should explain in more detail why this move is done. Neither > the > commit message nor the rest of the thread does so afaics. It's

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, On 2022-03-18 18:14:52 +0300, Maxim Orlov wrote: > Subject: [PATCH v22 3/6] Use 64-bit pages in SLRU > > This is one step toward 64-bit XIDs. I think this should explain in more detail why this move is done. Neither the commit message nor the rest of the thread does so afaics. It's not too

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> > > To me personally v21 looks almost OK. > > For some reason I didn't receive the recent e-mails from Tom and Kyotaro. > I've just discovered them accidentally by reading the thread through the > web interface. These comments were not addressed in v21. > Aleksander, as of now we're preparing a

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi hackers, > Here is an new version with the following changes compared to v20: > [...] > To me personally v21 looks almost OK. For some reason I didn't receive the recent e-mails from Tom and Kyotaro. I've just discovered them accidentally by reading the thread through the web interface. These

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

> > On Thu, 17 Mar 2022 at 21:31, Alexander Korotkov > wrote: > > On Thu, Mar 17, 2022 at 4:23 PM Peter Eisentraut > > wrote: > >> On 17.03.22 14:12, Aleksander Alekseev wrote: > >> > v19-0001 changes the format string for XIDs from %u to XID_FMT. This > >> > refactoring allows us to switch to

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Japin Li writes: > Maybe, we should format it to string before for localization messages, > like the following code snippet comes from pg_backup_tar.c. > However, I do not think it is a good way. > snprintf(buf1, sizeof(buf1), INT64_FORMAT, (int64) len); > snprintf(buf2,

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Thu, 17 Mar 2022 at 21:31, Alexander Korotkov wrote: > On Thu, Mar 17, 2022 at 4:23 PM Peter Eisentraut > wrote: >> On 17.03.22 14:12, Aleksander Alekseev wrote: >> > v19-0001 changes the format string for XIDs from %u to XID_FMT. This >> > refactoring allows us to switch to UINT64_FORMAT

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On Thu, Mar 17, 2022 at 4:23 PM Peter Eisentraut wrote: > On 17.03.22 14:12, Aleksander Alekseev wrote: > > v19-0001 changes the format string for XIDs from %u to XID_FMT. This > > refactoring allows us to switch to UINT64_FORMAT by changing one > > #define in the future patches. > > > > Kyotaro

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

On 17.03.22 14:12, Aleksander Alekseev wrote: v19-0001 changes the format string for XIDs from %u to XID_FMT. This refactoring allows us to switch to UINT64_FORMAT by changing one #define in the future patches. Kyotaro suggested using `errmsg("blah blah %lld ..", (long long) xid)` instead in

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

Hi, Aleksander! On Thu, Mar 17, 2022 at 4:12 PM Aleksander Alekseev wrote: > This thread is a fork of [1], created per request by several people in > the discussion. It includes two patches from the patchset that we > believe can be delivered in PG15. The rest of the patches are > targeting PG