Re: Is monotonous xid8 is a right way to do?

[Pavel Borisov]

>
> A different and more important issue (IMO) is that the xlog record
>
header currently only supports 32-bit xids -- long-running
> transactions can reasonably see a xid4 wraparound in their lifetime.

You're completely right. This is a first of making xid's 64-bit proposed
[1] i.e, making SLRU 64-bit [2]

[1]
https://www.postgresql.org/message-id/flat/CACG%3DezZe1NQSCnfHOr78AtAZxJZeCvxrts0ygrxYwe%3DpyyjVWA%40mail.gmail.com

[2]
https://www.postgresql.org/message-id/flat/CALT9ZEEf1uywYN%2BVaRuSwNMGE5%3DeFOy7ZTwtP2g%2BW9oJDszqQw%40mail.gmail.com#bd4f64b73cb3b969e119da7e5a7b1f30

-- 
Best regards,
Pavel Borisov

Postgres Professional: http://postgrespro.com 

Re: Is monotonous xid8 is a right way to do?

[Matthias van de Meent]

On Fri, 1 Apr 2022 at 14:13, Pavel Borisov  wrote:
>
> Hi hackers!
>
> Now we have two data types xid and xid8. The first one (xid) makes a numeric 
> ring, and xid8 are monotonous.
>
> As per [1] "Unlike xid values, xid8 values increase strictly monotonically 
> and cannot be reused in the lifetime of a database cluster."
>
> As a consequence of [1] xid8 can have min/max functions (committed in [2]), 
> which xid can not have.
>
> When working on 64xid patch [3] we assume that even 64xid's technically can 
> be wraparound-ed, although it's very much unlikely. I wonder what is expected 
> to be with xid8 values at this (unlikely) 64xid wraparound?
>
> What do you think about this? Wouldn't it be better to change xid8 to form a 
> numeric ring like xid? I think it is necessary for any 64-wraparound-enabled 
> implementation of 64xids.
>
> Please feel free to share your thoughts.

Assuming that each Xid is WAL-logged (or at least one in 8) we won't
see xid8 wraparound, as our WAL is byte-addressable with only 64 bits
used as the identifier. As such, we can only fit a maximum of 2^61
xid8s in our WAL; which is less than what would be needed to wrap
around.

Addressed another way: If we'd have a system that consumed one xid
every CPU clock; then the best available x86 hardware right now would
currently consume ~ 5.5B xids every second. This would still leave
around 100 years of this system running non-stop before we'd be
hitting xid8 wraparound (= 2^64 / 5.5e9 (xid8 /sec) / 3600 (min /hour)
/ 730 (hour / month)/ 12 (month /year)).

I don't think we'll have to consider that an issue for now. Maybe,
eventually, if we start doing distributed transactions where
transaction IDs are reasonably consumed at a rate higher than 5B /sec
(and not logged at that rate) we can start considering this to be a
problem.

A different and more important issue (IMO) is that the xlog record
header currently only supports 32-bit xids -- long-running
transactions can reasonably see a xid4 wraparound in their lifetime.

Enjoy,

Matthias van de Meent

Re: Is monotonous xid8 is a right way to do?

[Dagfinn Ilmari Mannsåker]

Dagfinn Ilmari Mannsåker  writes:

> Even if a cluster was consuming a million XIDs per second, it would take
> over half a million years to wrap around the 64bit range. Is that really
> something we should worry about?
>
> ilmari@[local]:5432 ~=# select 2::numeric^64/10^9/3600/24/365;

Oops, that should be 10^6, not 10^9. I was dithering over whether to do
it as a million or a billion per second.  For a billion XIDs per second
it would last a mere half millennium.

> ┌──┐
> │ ?column? │
> ├──┤
> │ 584942.417355072 │
> └──┘
>
> - ilmari

Re: Is monotonous xid8 is a right way to do?

[Maxim Orlov]

Hi!

In my view, FullTransactionId type was implemented without considering 64
bit wraparound. Which seems to be unlikely to happen. Then on that basis
xid8 type was created. Details of that particular implementation
infiltrated into documentation and became sort of normal. In my opinion,
semantically, both of these types should be treated as similar
types although with different sizes. Thus, again, xid and xid8 types should
be a ring and have no min and max functions. At least, in a sort of
"conventional" way when minimal value is minimal in a mathematical way and
so for maximum.

For example, max may be implemented as max(0, 42, 18446744073709551615) =
42, which is a bit weird.

-- 
Best regards,
Maxim Orlov.

Re: Is monotonous xid8 is a right way to do?

[Dagfinn Ilmari Mannsåker]

Pavel Borisov  writes:

> Hi hackers!
>
> Now we have two data types xid and xid8. The first one (xid) makes a
> numeric ring, and xid8 are monotonous.
>
> As per [1] "Unlike xid values, xid8 values increase strictly monotonically
> and cannot be reused in the lifetime of a database cluster."
>
> As a consequence of [1] xid8 can have min/max functions (committed in [2]),
> which xid can not have.
>
> When working on 64xid patch [3] we assume that even 64xid's technically can
> be wraparound-ed, although it's very much unlikely. I wonder what is
> expected to be with xid8 values at this (unlikely) 64xid wraparound?

Even if a cluster was consuming a million XIDs per second, it would take
over half a million years to wrap around the 64bit range. Is that really
something we should worry about?

ilmari@[local]:5432 ~=# select 2::numeric^64/10^9/3600/24/365;
┌──┐
│ ?column? │
├──┤
│ 584942.417355072 │
└──┘

- ilmari