Re: Usage of epoch in txid_current

On Sun, Jun 30, 2019 at 9:07 AM Robert Haas wrote: > On Mon, Jun 24, 2019 at 1:43 PM Alvaro Herrera > wrote: > > I think enlarging multixacts to 64 bits is a terrible idea. I would > > rather get rid of multixacts completely; zheap is proposing not to use > > multixacts at all, for example. >

Re: Usage of epoch in txid_current

On Sat, Jun 22, 2019 at 11:01 AM Alexander Korotkov wrote: > 3. Change SLRU on-disk format to handle 64-bit xids and multixacts. > In particular make SLRU page numbers 64-bit too. Add SLRU upgrade > procedure to pg_upgrade. +1 for doing this for the xid-indexed SLRUs so the segment file names

Re: Usage of epoch in txid_current

On Mon, Jun 24, 2019 at 1:43 PM Alvaro Herrera wrote: > I think enlarging multixacts to 64 bits is a terrible idea. I would > rather get rid of multixacts completely; zheap is proposing not to use > multixacts at all, for example. But zedstore, at least as of the Saturday after PGCon, is

Re: Usage of epoch in txid_current

On Mon, Jun 24, 2019 at 8:43 PM Alvaro Herrera wrote: > > On 2019-Jun-22, Alexander Korotkov wrote: > > > 2. Also introduce FullMultixactId, and apply to MultixactId the > > similar change as #1. > > 3. Change SLRU on-disk format to handle 64-bit xids and multixacts. > > In particular make SLRU

Re: Usage of epoch in txid_current

Hi! On Mon, Jun 24, 2019 at 6:27 PM Andres Freund wrote: > On June 24, 2019 8:19:13 AM PDT, Robert Haas wrote: > >On Fri, Jun 21, 2019 at 7:01 PM Alexander Korotkov > > wrote: > >> On Thu, Mar 28, 2019 at 8:30 AM Thomas Munro > >wrote: > >> > Thanks for the reviews! Pushed. > >> > >> Any

Re: Usage of epoch in txid_current

On 2019-Jun-22, Alexander Korotkov wrote: > 2. Also introduce FullMultixactId, and apply to MultixactId the > similar change as #1. > 3. Change SLRU on-disk format to handle 64-bit xids and multixacts. > In particular make SLRU page numbers 64-bit too. Add SLRU upgrade > procedure to pg_upgrade.

Re: Usage of epoch in txid_current

Hi, On June 24, 2019 8:19:13 AM PDT, Robert Haas wrote: >On Fri, Jun 21, 2019 at 7:01 PM Alexander Korotkov > wrote: >> On Thu, Mar 28, 2019 at 8:30 AM Thomas Munro >wrote: >> > Thanks for the reviews! Pushed. >> >> Any ideas we should move towards 64-bit xids in more places? That >has >>

Re: Usage of epoch in txid_current

On Fri, Jun 21, 2019 at 7:01 PM Alexander Korotkov wrote: > On Thu, Mar 28, 2019 at 8:30 AM Thomas Munro wrote: > > Thanks for the reviews! Pushed. > > Any ideas we should move towards 64-bit xids in more places? That has > been discussed several times already. I think last time it was >

Re: Usage of epoch in txid_current

On Thu, Mar 28, 2019 at 8:30 AM Thomas Munro wrote: > Thanks for the reviews! Pushed. Any ideas we should move towards 64-bit xids in more places? That has been discussed several times already. I think last time it was discussed in person during FOSDEM PGDay 2018 Developer Meeting [1]. There

Re: Usage of epoch in txid_current

Round pegs fit into square holes when the peg is smaller.,.. Hey why did the chicken follow Simon off the ledge? Just because he told him to?!! Was he really trying to get home, did he find out the hard way things no one else knows but what he had learnt on the other side, just to let you

Re: Usage of epoch in txid_current

On Sat, May 4, 2019 at 1:34 PM Jeff Janes wrote: > On Thu, Mar 28, 2019 at 1:30 AM Thomas Munro > wrote: > >> On Thu, Mar 28, 2019 at 1:48 AM Heikki Linnakangas >> wrote: >> > Once we have the FullTransactionId type and basic macros in place, I'm >> > sure we could tidy up a bunch of code by

Re: Usage of epoch in txid_current

On Thu, Mar 28, 2019 at 1:30 AM Thomas Munro wrote: > On Thu, Mar 28, 2019 at 1:48 AM Heikki Linnakangas > wrote: > > Once we have the FullTransactionId type and basic macros in place, I'm > > sure we could tidy up a bunch of code by using them. Thanks for the reviews! Pushed. > I think

Re: Usage of epoch in txid_current

On Thu, Mar 28, 2019 at 1:48 AM Heikki Linnakangas wrote: > Once we have the FullTransactionId type and basic macros in place, I'm > sure we could tidy up a bunch of code by using them. For example, > TransactionIdInRecentPast() in walsender.c would be simpler, if the > caller dealt with

Re: Usage of epoch in txid_current

On 27/03/2019 13:44, Thomas Munro wrote: * I tidied up the code that serialises transaction state. It was already hammering round pegs into square holes, and the previous patch made that even worse, so I added a new struct SerializedTransactionState to do this properly. Ah, good, it used to

Re: Usage of epoch in txid_current

On Tue, Mar 26, 2019 at 12:58 PM Thomas Munro wrote: > On Tue, Mar 26, 2019 at 3:23 AM Heikki Linnakangas wrote: > > Looks good. I did some testing and proof-reading and made a few minor changes: * I tidied up the code that serialises transaction state. It was already hammering round pegs

Re: Usage of epoch in txid_current

On Tue, Mar 26, 2019 at 12:58 PM Thomas Munro wrote: > ... I think you could probably reclaim that space by > using a more compact representation of vacuumFlags, overflowed, > delayChkpt, nxids (it's funny, the comment says "as tightly as > possible", which clearly isn't the case). Woops, I

Re: Usage of epoch in txid_current

On Tue, Mar 26, 2019 at 3:23 AM Heikki Linnakangas wrote: > Looks good. > > I started to write a patch to use XID & epoch in dealing with GiST page > deletions [1], and I really could've used an epoch to go with > RecentGlobalXmin. I presume that would be pretty straightforward to have > with

Re: Usage of epoch in txid_current

On 25/03/2019 12:49, Thomas Munro wrote: On Mon, Mar 25, 2019 at 5:01 PM Thomas Munro wrote: New version attached. I'd like to commit this for PG12. Here is a follow-up sketch patch that shows FullTransactionId being used in the transaction stack, so you can call eg

Re: Usage of epoch in txid_current

On Mon, Mar 25, 2019 at 5:01 PM Thomas Munro wrote: > New version attached. I'd like to commit this for PG12. Here is a follow-up sketch patch that shows FullTransactionId being used in the transaction stack, so you can call eg GetCurrentFullTransactionId(). A table access method could use

Re: Usage of epoch in txid_current

On Mon, Feb 4, 2019 at 8:41 PM Andres Freund wrote: > On 2018-09-19 13:58:36 +1200, Thomas Munro wrote: > > > +/* > > + * Advance nextFullXid to the value after a given xid. The epoch is > > inferred. > > + * If lock_free_check is true, then the caller must be sure that it's safe > > to > > +

Re: Usage of epoch in txid_current

Hi, On 2018-09-19 13:58:36 +1200, Thomas Munro wrote: > +/* > + * Advance nextFullXid to the value after a given xid. The epoch is > inferred. > + * If lock_free_check is true, then the caller must be sure that it's safe to > + * read nextFullXid without holding XidGenLock (ie during

Re: Usage of epoch in txid_current

On February 4, 2019 6:43:44 AM GMT+01:00, Michael Paquier wrote: >On Sun, Feb 03, 2019 at 10:58:02PM +1100, Thomas Munro wrote: >> If there are no objections, I'm planning to do a round of testing and >> commit this shortly. > >Hm. That looks sane to me at quick glance. I am a bit on the

Re: Usage of epoch in txid_current

Michael Paquier writes: > Hm. That looks sane to me at quick glance. I am a bit on the edge > regaring the naming "FullTransactionId", which is actually a 64-bit > value with a 32-bit XID and a 32-bit epoch. Something like > TransactionIdWithEpoch or EpochTransactionId sounds a bit better to >

Re: Usage of epoch in txid_current

On Sun, Feb 03, 2019 at 10:58:02PM +1100, Thomas Munro wrote: > If there are no objections, I'm planning to do a round of testing and > commit this shortly. Hm. That looks sane to me at quick glance. I am a bit on the edge regaring the naming "FullTransactionId", which is actually a 64-bit

Re: Usage of epoch in txid_current

On Fri, Nov 30, 2018 at 12:12 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Tue, Jul 24, 2018 at 7:25 AM Thomas Munro > > wrote: > > Here's a new version. I did some cosmetic clean-up, and I dropped the > > changes to pg_controldata output, replication epoch/xid processing > > code and

Re: Usage of epoch in txid_current

> On Tue, Jul 24, 2018 at 7:25 AM Thomas Munro > wrote: > > Here's a new version. I did some cosmetic clean-up, and I dropped the > changes to pg_controldata output, replication epoch/xid processing > code and various similar non-essential changes. For this patch I want > just the new type +

Re: Usage of epoch in txid_current

On Tue, Jul 24, 2018 at 5:24 PM Thomas Munro wrote: > Here's a new version. Bitrot, rebased. -- Thomas Munro http://www.enterprisedb.com 0001-Track-the-next-xid-using-64-bits-v6.patch Description: Binary data

Re: Usage of epoch in txid_current

On Tue, Jul 17, 2018 at 1:55 AM, Tom Lane wrote: > Andres Freund writes: >> On 2018-07-15 16:41:35 -0400, Tom Lane wrote: >>> Andres Freund writes: On 2018-07-09 19:56:25 -0400, Tom Lane wrote: > Or, perhaps, use a struct in assert builds and int64 otherwise? > You could hide the

Re: Usage of epoch in txid_current

Andres Freund writes: > On 2018-07-15 16:41:35 -0400, Tom Lane wrote: >> Andres Freund writes: >>> On 2018-07-09 19:56:25 -0400, Tom Lane wrote: Or, perhaps, use a struct in assert builds and int64 otherwise? You could hide the ensuing notational differences in macros. >> [ bunch of

Re: Usage of epoch in txid_current

Hi, On 2018-07-15 16:41:35 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-07-09 19:56:25 -0400, Tom Lane wrote: > >> Or, perhaps, use a struct in assert builds and int64 otherwise? > >> You could hide the ensuing notational differences in macros. > > > That should be doable. But I'd

Re: Usage of epoch in txid_current

Andres Freund writes: > On 2018-07-09 19:56:25 -0400, Tom Lane wrote: >> Or, perhaps, use a struct in assert builds and int64 otherwise? >> You could hide the ensuing notational differences in macros. > That should be doable. But I'd like to check if it's necessary > first. Optimizing passing an

Re: Usage of epoch in txid_current

On Tue, Jul 10, 2018 at 2:15 PM, Thomas Munro wrote: > On Tue, Jul 10, 2018 at 1:30 PM, Andres Freund wrote:, >>> (errmsg_internal("next transaction ID: %u:%u; next >>> OID: %u", >>> - >>> checkPoint.nextXidEpoch,

Re: Usage of epoch in txid_current

On 10 July 2018 at 10:40, Andres Freund wrote: > On 2018-07-10 10:32:44 +0800, Craig Ringer wrote: > > On 10 July 2018 at 07:35, Thomas Munro > > wrote: > > > > > > > > I played around with this idea yesterday. Experiment-grade patch > > > attached. Approach: > > > > > > 1. Introduce a new

Re: Usage of epoch in txid_current

On 2018-07-10 10:32:44 +0800, Craig Ringer wrote: > On 10 July 2018 at 07:35, Thomas Munro > wrote: > > > > > I played around with this idea yesterday. Experiment-grade patch > > attached. Approach: > > > > 1. Introduce a new type BigTransactionId (better names welcome). > > > >

Re: Usage of epoch in txid_current

On 10 July 2018 at 10:32, Craig Ringer wrote: > > >> I think it's probably a good idea to make it very explicit when moving >> between big and small transaction IDs, hence the including of the word >> 'big' in variable and function names and the use of a function-like >> macro (rather than

Re: Usage of epoch in txid_current

On 10 July 2018 at 07:35, Thomas Munro wrote: > > I played around with this idea yesterday. Experiment-grade patch > attached. Approach: > > 1. Introduce a new type BigTransactionId (better names welcome). > txid_current() should be changed to BigTransactionId too. It's currently bigint.

Re: Usage of epoch in txid_current

On Tue, Jul 10, 2018 at 1:30 PM, Andres Freund wrote: > On 2018-07-10 13:20:52 +1200, Thomas Munro wrote: >> I don't know what to think about the encoding or meaning of non-normal >> xids in this thing. > > I'm not following what you mean by this? While defining FullTransactionIdPrecedes() in

Re: Usage of epoch in txid_current

On 2018-07-10 13:20:52 +1200, Thomas Munro wrote: > defined in transam.h because c.h didn't feel right. Yea, that looks better. > Client code lost the ability to use operator < directly. I needed to > use a static inline function as a constructor. I lost the > interchangeability with the wide

Re: Usage of epoch in txid_current

On Tue, Jul 10, 2018 at 12:08 PM, Andres Freund wrote: > On 2018-07-09 19:56:25 -0400, Tom Lane wrote: >> Andres Freund writes: >> > On 2018-07-10 11:35:59 +1200, Thomas Munro wrote: >> >> I think it's probably a good idea to make it very explicit when moving >> >> between big and small

Re: Usage of epoch in txid_current

On 2018-07-09 17:08:34 -0700, Andres Freund wrote: > Hi, > > On 2018-07-09 19:56:25 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2018-07-10 11:35:59 +1200, Thomas Munro wrote: > > >> I think it's probably a good idea to make it very explicit when moving > > >> between big and small

Re: Usage of epoch in txid_current

Hi, On 2018-07-09 19:56:25 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-07-10 11:35:59 +1200, Thomas Munro wrote: > >> I think it's probably a good idea to make it very explicit when moving > >> between big and small transaction IDs, hence the including of the word > >> 'big' in

Re: Usage of epoch in txid_current

Andres Freund writes: > On 2018-07-10 11:35:59 +1200, Thomas Munro wrote: >> I think it's probably a good idea to make it very explicit when moving >> between big and small transaction IDs, hence the including of the word >> 'big' in variable and function names and the use of a function-like >>

Re: Usage of epoch in txid_current

On 2018-07-10 11:35:59 +1200, Thomas Munro wrote: > I played around with this idea yesterday. Experiment-grade patch > attached. Cool! > I think it's probably a good idea to make it very explicit when moving > between big and small transaction IDs, hence the including of the word > 'big' in

Re: Usage of epoch in txid_current

On Fri, Dec 8, 2017 at 3:36 PM, Amit Kapila wrote: > On Wed, Dec 6, 2017 at 11:26 PM, Andres Freund wrote: >>> Either way, it is not clear to me how we will keep it >>> updated after recovery. Right now, the mechanism is quite simple, at >>> the beginning of a recovery we take the value of

Re: Usage of epoch in txid_current

On Wed, Dec 6, 2017 at 11:26 PM, Andres Freund wrote: > >> Either way, it is not clear to me how we will keep it >> updated after recovery. Right now, the mechanism is quite simple, at >> the beginning of a recovery we take the value of nextXid from >> checkpoint record and

Re: Usage of epoch in txid_current

Hi, On 2017-12-06 17:39:09 +0530, Amit Kapila wrote: > We are using ShmemVariableCache->nextXid at many places so always > converting/truncating it to 32-bit number before using seems slightly > awkward, so we can think of using a separate nextBigXid 64bit number > as well. -many It's not

Re: Usage of epoch in txid_current

On Tue, Dec 5, 2017 at 2:49 PM, Alexander Korotkov wrote: > On Tue, Dec 5, 2017 at 6:19 AM, Amit Kapila wrote: >> >> Currently, txid_current and friends export a 64-bit format of >> transaction id that is extended with an “epoch” counter so

Re: Usage of epoch in txid_current

Hi! On Tue, Dec 5, 2017 at 6:19 AM, Amit Kapila wrote: > Currently, txid_current and friends export a 64-bit format of > transaction id that is extended with an “epoch” counter so that it > will not wrap around during the life of an installation. The epoch > value it

Usage of epoch in txid_current

Hi, Currently, txid_current and friends export a 64-bit format of transaction id that is extended with an “epoch” counter so that it will not wrap around during the life of an installation. The epoch value it uses is based on the epoch that is maintained by checkpoint (aka only checkpoint