Re: POC: make mxidoff 64 bits

2024-10-24 Thread wenhui qiu
HI Maxim Orlov > After a bit of thought, I've realized that to be conservative here is the way to go. >We can reuse a maximum of existing logic. I mean, we can remove offset wraparound "error logic" and reuse "warning logic". But set the threshold for "warning >logic" to a much higher value. For no

Re: POC: make mxidoff 64 bits

2024-10-23 Thread Maxim Orlov
After a bit of thought, I've realized that to be conservative here is the way to go. We can reuse a maximum of existing logic. I mean, we can remove offset wraparound "error logic" and reuse "warning logic". But set the threshold for "warning logic" to a much higher value. For now, I choose 2^32-1

Re: POC: make mxidoff 64 bits

2024-10-22 Thread Maxim Orlov
On Tue, 22 Oct 2024 at 12:43, Heikki Linnakangas wrote: > MultiXactMemberFreezeThreshold looks quite bogus now. Now that > MaxMultiXactOffset==2^64-1, you cannot get anywhere near the > MULTIXACT_MEMBER_SAFE_THRESHOLD and MULTIXACT_MEMBER_DANGER_THRESHOLD > values anymore. Can we just get rid of

Re: POC: make mxidoff 64 bits

2024-10-22 Thread Heikki Linnakangas
On 07/09/2024 07:36, Maxim Orlov wrote: Here is v3. MultiXactMemberFreezeThreshold looks quite bogus now. Now that MaxMultiXactOffset==2^64-1, you cannot get anywhere near the MULTIXACT_MEMBER_SAFE_THRESHOLD and MULTIXACT_MEMBER_DANGER_THRESHOLD values anymore. Can we just get rid of MultiXa

Re: POC: make mxidoff 64 bits

2024-09-12 Thread Alvaro Herrera
On 2024-Sep-12, Pavel Borisov wrote: > Hi, Maxim! > > Previously we accessed offsets in shared MultiXactState without locks as > 32-bit read is always atomic. But I'm not sure it's so when offset become > 64-bit. > E.g. GetNewMultiXactId(): > > nextOffset = MultiXactState->nextOffset; > is outsi

Re: POC: make mxidoff 64 bits

2024-09-12 Thread Pavel Borisov
On Thu, 12 Sept 2024 at 16:09, Pavel Borisov wrote: > Hi, Maxim! > > Previously we accessed offsets in shared MultiXactState without locks as > 32-bit read is always atomic. But I'm not sure it's so when offset become > 64-bit. > E.g. GetNewMultiXactId(): > > nextOffset = MultiXactState->nextOffs

Re: POC: make mxidoff 64 bits

2024-09-12 Thread Pavel Borisov
Hi, Maxim! Previously we accessed offsets in shared MultiXactState without locks as 32-bit read is always atomic. But I'm not sure it's so when offset become 64-bit. E.g. GetNewMultiXactId(): nextOffset = MultiXactState->nextOffset; is outside lock. There might be other places we do the same as

Re: POC: make mxidoff 64 bits

2024-09-06 Thread Maxim Orlov
Here is v3. I removed CATALOG_VERSION_NO change, so this should be done by the actual commiter. -- Best regards, Maxim Orlov. v3-0002-Use-64-bit-multixact-offsets.patch Description: Binary data v3-0001-Use-64-bit-format-output-for-multixact-offsets.patch Description: Binary data v3-0003-Mak

Re: POC: make mxidoff 64 bits

2024-09-04 Thread Maxim Orlov
On Tue, 3 Sept 2024 at 16:32, Alexander Korotkov wrote: > I don't think you need to maintain CATALOG_VERSION_NO change in your > patch for the exact reason you have mentioned: patch will get conflict > each time CATALOG_VERSION_NO is advanced. It's responsibility of > committer to advance CATALO

Re: POC: make mxidoff 64 bits

2024-09-03 Thread Alexander Korotkov
On Tue, Sep 3, 2024 at 4:30 PM Maxim Orlov wrote: > Here is rebase. Apparently I'll have to do it often, since the > CATALOG_VERSION_NO changed in the patch. I don't think you need to maintain CATALOG_VERSION_NO change in your patch for the exact reason you have mentioned: patch will get confli

Re: POC: make mxidoff 64 bits

2024-09-03 Thread Maxim Orlov
Here is rebase. Apparently I'll have to do it often, since the CATALOG_VERSION_NO changed in the patch. -- Best regards, Maxim Orlov. v2-0001-Use-64-bit-format-output-for-multixact-offsets.patch Description: Binary data v2-0003-Make-pg_upgrade-convert-multixact-offsets.patch Description: Bin

Re: POC: make mxidoff 64 bits

2024-08-14 Thread Maxim Orlov
Hi! Sorry for delay. I was a bit busy last month. Anyway, here is my proposal for making multioffsets 64 bit. The patch set consists of three parts: 0001 - making user output of offsets 64-bit ready; 0002 - making offsets 64-bit; 0003 - provide 32 to 64 bit conversion in pg_upgarde. I'm pretty su

Re: POC: make mxidoff 64 bits

2024-04-25 Thread Maxim Orlov
On Tue, 23 Apr 2024 at 12:37, Heikki Linnakangas wrote: > This is really a bug fix. It didn't matter when TransactionId and > MultiXactOffset were both typedefs of uint32, but it was always wrong. > The argument name 'xid' is also misleading. > > I think there are some more like that, MXOffsetToF

Re: POC: make mxidoff 64 bits

2024-04-23 Thread wenhui qiu
Hi Maxim Orlov Thank you so much for your tireless work on this. Increasing the WAL size by a few bytes should have very little impact with today's disk performance(Logical replication of this feature wal log is also increased a lot, logical replication is a milestone new feature, and the commun

Re: POC: make mxidoff 64 bits

2024-04-23 Thread Andrey M. Borodin
> On 23 Apr 2024, at 11:23, Maxim Orlov wrote: > > Make multixact offsets 64 bit. - ereport(ERROR, - (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), -errmsg("multixact \"members\" limit exceeded"), Personally, I'd be happy

Re: POC: make mxidoff 64 bits

2024-04-23 Thread Heikki Linnakangas
On 23/04/2024 11:23, Maxim Orlov wrote: PROPOSAL Make multixact offsets 64 bit. +1, this is a good next step and useful regardless of 64-bit XIDs. @@ -156,7 +148,7 @@ ((uint32) ((0x % MULTIXACT_MEMBERS_PER_PAGE) + 1)) /* page in which a member is to be found */ -#d

POC: make mxidoff 64 bits

2024-04-23 Thread Maxim Orlov
Hi! I've been trying to introduce 64-bit transaction identifications to Postgres for quite a while [0]. All this implies, of course, an enormous amount of change that will have to take place in various modules. Consider this, the patch set become too big to be committed “at once”. The obvious sol