Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
On 2014-01-02 14:44:34 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2014-01-02 12:46:34 -0500, Tom Lane wrote: > >> For real > >> forensics work, you need to be able to see all tuples, which makes me > >> think that something akin to pgstattuple is the right API; that is "return > >> a se

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > Well, that's fair enough. I don't mind having two functions. Should > the whole-table function also include invisible tuples? Certainly, that's exactly why I was proposing it. You can do a join if you want to suppress them. regards, tom lane --

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 2:56 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jan 2, 2014 at 2:44 PM, Tom Lane wrote: >>> As I commented to Robert, the page-at-a-time behavior of pageinspect >>> is not an API detail we'd want to copy for this. I envision something >>> like >>> >>> select hdr

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 2, 2014 at 2:44 PM, Tom Lane wrote: >> As I commented to Robert, the page-at-a-time behavior of pageinspect >> is not an API detail we'd want to copy for this. I envision something >> like >> >> select hdr.*, foo.* >> from tuple_header_details('foo'::regclass)

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > We could certainly add a function that returns SETOF record, taking > e.g. regclass as an argument, but it doesn't seem a stretch to me to > think that you might want to get tuple header information for some but > not all tuples in the relation, and I don't see any real good

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 2:44 PM, Tom Lane wrote: > As I commented to Robert, the page-at-a-time behavior of pageinspect > is not an API detail we'd want to copy for this. I envision something > like > >select hdr.*, foo.* > from tuple_header_details('foo'::regclass) as hdr >

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Andres Freund writes: > On 2014-01-02 12:46:34 -0500, Tom Lane wrote: >> For real >> forensics work, you need to be able to see all tuples, which makes me >> think that something akin to pgstattuple is the right API; that is "return >> a set of the header info for all tuples on such-and-such pages

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 2:03 PM, Tom Lane wrote: >> I both agree and disagree with this. I think that pgstattuple is >> useful, and I further agree that adding a stat to it about how much of >> the heap is frozen would be worthwhile. However, an aggregate number >> isn't always what you want, and

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
On 2014-01-02 12:46:34 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2014-01-02 09:40:54 -0500, Tom Lane wrote: > >> Actually, I thought the function approach was a good proposal. You are > >> right that func(tab.*) isn't going to work, because it's going to get a > >> Datum-ified tuple n

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 2, 2014 at 12:46 PM, Tom Lane wrote: >> In any case, upon further reflection I'm not convinced that doing this >> with a SELECT-based query is the right thing, no matter whether the query >> looks at a function or a system column; because by definition, you'll on

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 12:46 PM, Tom Lane wrote: > Andres Freund writes: >> On 2014-01-02 09:40:54 -0500, Tom Lane wrote: >>> Actually, I thought the function approach was a good proposal. You are >>> right that func(tab.*) isn't going to work, because it's going to get a >>> Datum-ified tuple n

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Andres Freund writes: > On 2014-01-02 09:40:54 -0500, Tom Lane wrote: >> Actually, I thought the function approach was a good proposal. You are >> right that func(tab.*) isn't going to work, because it's going to get a >> Datum-ified tuple not a pointer to raw on-disk storage. But an inspection

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
On 2014-01-02 09:40:54 -0500, Tom Lane wrote: > Robert Haas writes: > > On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: > >> 1) it's a bit weird to go to this effort to eliminate system columns by > >> using a scheme that depends on having a system column -- ctid > > > At any rate, my goal is

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: >> 1) it's a bit weird to go to this effort to eliminate system columns by >> using a scheme that depends on having a system column -- ctid > At any rate, my goal isn't really to get rid of system columns, but to > address

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 7:40 AM, Andres Freund wrote: > On 2014-01-02 07:35:11 -0500, Robert Haas wrote: >> On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: >> >> I fail to see why. What is so ugly about this: >> > >> >> select x.* from pgbench_accounts a, pg_tuple_header(a.tableoid, a.ctid) x;

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
On 2014-01-02 07:35:11 -0500, Robert Haas wrote: > On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: > >> I fail to see why. What is so ugly about this: > > > >> select x.* from pgbench_accounts a, pg_tuple_header(a.tableoid, a.ctid) x; > > > > Two points: > > > > 1) it's a bit weird to go to th

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: >> I fail to see why. What is so ugly about this: > >> select x.* from pgbench_accounts a, pg_tuple_header(a.tableoid, a.ctid) x; > > Two points: > > 1) it's a bit weird to go to this effort to eliminate system columns by > using a scheme that de

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
Hi, On 2014-01-02 05:26:26 +, Greg Stark wrote: > 2) refetching a row could conceivably end up retrieving different data than > was present when the row was originally read. (In some cases that might > actually be the intended behaviour) That's possible with system columns as well. In the nor

Re: [HACKERS] preserving forensic information when we freeze

2014-01-01 Thread Greg Stark
> I fail to see why. What is so ugly about this: > select x.* from pgbench_accounts a, pg_tuple_header(a.tableoid, a.ctid) x; Two points: 1) it's a bit weird to go to this effort to eliminate system columns by using a scheme that depends on having a system column -- ctid 2) refetching a row co

Re: [HACKERS] preserving forensic information when we freeze

2014-01-01 Thread Robert Haas
On Fri, Dec 27, 2013 at 9:24 PM, Stephen Frost wrote: >> I'm not sure what you mean by "doesn't work", because it clearly does >> work. I've already posted a patch. You may find it ugly, but that's >> not the same as not working. > > I meant *practically*, it doesn't work. By which, I mean, "it

Re: [HACKERS] preserving forensic information when we freeze

2013-12-27 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Dec 27, 2013 at 6:15 AM, Stephen Frost wrote: > > For my 2c, I tend to agree w/ Andres on this one. I like the *idea* of > > having a function instead, but, practically, it doesn't really work. > > Perhaps if the 'function' approach

Re: [HACKERS] preserving forensic information when we freeze

2013-12-27 Thread Robert Haas
On Fri, Dec 27, 2013 at 6:15 AM, Stephen Frost wrote: > * Andres Freund (and...@2ndquadrant.com) wrote: >> On 2013-12-26 15:25:41 -0800, Robert Haas wrote: >> > I dunno, I just have an uneasy feeling about it. I guess if >> > everyone's happy to add pg_infomask and pg_infomask2 as new system >> >

Re: [HACKERS] preserving forensic information when we freeze

2013-12-27 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2013-12-26 15:25:41 -0800, Robert Haas wrote: > > I dunno, I just have an uneasy feeling about it. I guess if > > everyone's happy to add pg_infomask and pg_infomask2 as new system > > columns, we could go that route. For my part, I think that

Re: [HACKERS] preserving forensic information when we freeze

2013-12-27 Thread Andres Freund
On 2013-12-26 15:25:41 -0800, Robert Haas wrote: > On Tue, Dec 24, 2013 at 6:22 AM, Andres Freund wrote: > > If system columns don't have an overhead anymore, I fail to see the > > advantage that functions have over simply accessing parts of the row in > > the normal way parts of rows are accessed

Re: [HACKERS] preserving forensic information when we freeze

2013-12-26 Thread Robert Haas
On Tue, Dec 24, 2013 at 6:22 AM, Andres Freund wrote: > On 2013-12-23 14:15:29 -0500, Robert Haas wrote: >> On Mon, Dec 23, 2013 at 1:57 PM, Andres Freund >> wrote: >> > Well, all of the fundamental changes (combocids, the initial multixact >> > introduction) have been quite some time ago. I thi

Re: [HACKERS] preserving forensic information when we freeze

2013-12-24 Thread Andres Freund
On 2013-12-23 14:15:29 -0500, Robert Haas wrote: > On Mon, Dec 23, 2013 at 1:57 PM, Andres Freund wrote: > > Well, all of the fundamental changes (combocids, the initial multixact > > introduction) have been quite some time ago. I think backward compat has > > a much higher value these days (I als

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Robert Haas
On Mon, Dec 23, 2013 at 1:57 PM, Andres Freund wrote: > Well, all of the fundamental changes (combocids, the initial multixact > introduction) have been quite some time ago. I think backward compat has > a much higher value these days (I also don't see much point in looking > at cmin/cmax for anyt

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Andres Freund
On 2013-12-23 10:56:07 -0500, Robert Haas wrote: > > Well, it really depends on the usecase. Reading > > SELECT header.xmin, header.xmax > > FROM sometable tbl, > > pg_tuple_header(tbl.tableoid, tbl.ctid) header; > > is surely harder to understand than > > SELECT tbl.xmin, tbl.xmax > > FROM som

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Kevin Grittner
Robert Haas wrote: > Meh.  Who are all of these people who are fetching xmin, xmax, cmin, > and cmax in complex queries, and why are they doing that?  If those > columns are just for forensic purposes, then whatever performance > disadvantages the function-based approach has don't really matter. 

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Robert Haas
On Mon, Dec 23, 2013 at 10:42 AM, Andres Freund wrote: >> But TBH, I'm kind of enamored of the function-based approach at the >> moment. It just seems a lot more flexible. Adding a function is >> nearly free and we can add as many as we need doing whatever we want, >> and they can even go into c

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Andres Freund
On 2013-12-23 10:26:49 -0500, Robert Haas wrote: > On Mon, Dec 23, 2013 at 7:45 AM, Andres Freund wrote: > > I've spent some time yesterday hacking up a prototype for this. The > > amount of effort seems reasonable, although the attached patch certainly > > doesn't do all the neccessary things. Th

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Robert Haas
On Mon, Dec 23, 2013 at 7:45 AM, Andres Freund wrote: > On 2013-12-20 13:22:07 +0100, Andres Freund wrote: >> On 2013-12-20 07:12:01 -0500, Robert Haas wrote: >> > I think the root of the problem is that nobody's very eager to add >> > more hidden system catalog columns because each one bloats >>

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Andres Freund
On 2013-12-20 21:56:42 -0500, Robert Haas wrote: > On Fri, Dec 20, 2013 at 7:51 AM, Andres Freund wrote: > >> I wondered that, too, but it's not well-defined for all tuples. What > >> happens if you pass in constructed tuple rather than an on-disk tuple? > > > > Those should be discernible I thin

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Andres Freund
On 2013-12-20 13:22:07 +0100, Andres Freund wrote: > On 2013-12-20 07:12:01 -0500, Robert Haas wrote: > > I think the root of the problem is that nobody's very eager to add > > more hidden system catalog columns because each one bloats > > pg_attribute significantly. > > I think that part is actua

Re: [HACKERS] preserving forensic information when we freeze

2013-12-23 Thread Robert Haas
On Fri, Dec 20, 2013 at 9:56 PM, Robert Haas wrote: > On Fri, Dec 20, 2013 at 7:51 AM, Andres Freund wrote: >>> I wondered that, too, but it's not well-defined for all tuples. What >>> happens if you pass in constructed tuple rather than an on-disk tuple? >> >> Those should be discernible I thin

Re: [HACKERS] preserving forensic information when we freeze

2013-12-22 Thread Robert Haas
On Fri, Dec 20, 2013 at 8:01 AM, Andres Freund wrote: > On 2013-12-20 07:58:46 -0500, Robert Haas wrote: >> I think the immediate problem is to decide whether this patch ought to >> make the xmin column display the result of GetXmin() or GetRawXmin(). >> Thoughts on that? > > I slightly favor GetR

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Robert Haas
On Fri, Dec 20, 2013 at 7:51 AM, Andres Freund wrote: >> I wondered that, too, but it's not well-defined for all tuples. What >> happens if you pass in constructed tuple rather than an on-disk tuple? > > Those should be discernible I think, t_self/t_tableOid won't be set for > generated tuples.

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Robert Haas
On Fri, Dec 20, 2013 at 2:17 PM, Alvaro Herrera wrote: > Robert Haas escribió: >> On Fri, Dec 20, 2013 at 1:41 PM, Alvaro Herrera >> wrote: > >> > I assume without checking that passing reloid/ctid would allow this to >> > work for tuples in a RETURNING clause; and if we ever have an OLD >> > ref

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Alvaro Herrera
Robert Haas escribió: > On Fri, Dec 20, 2013 at 1:41 PM, Alvaro Herrera > wrote: > > I assume without checking that passing reloid/ctid would allow this to > > work for tuples in a RETURNING clause; and if we ever have an OLD > > reference for the RETURNING clause of an UPDATE, that it would work

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Robert Haas
On Fri, Dec 20, 2013 at 1:41 PM, Alvaro Herrera wrote: > Robert Haas escribió: >> On Fri, Dec 20, 2013 at 7:22 AM, Andres Freund >> wrote: >> >> Maybe what we should do is add a function something like >> >> pg_tuple_header(tableoid, ctid) that returns a record, maybe something >> >> like (rawxm

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Alvaro Herrera
Robert Haas escribió: > On Fri, Dec 20, 2013 at 7:22 AM, Andres Freund wrote: > >> Maybe what we should do is add a function something like > >> pg_tuple_header(tableoid, ctid) that returns a record, maybe something > >> like (rawxmin xid, rawxmax xid, rawcid cid, infomask int, infomask2 > >> int,

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Alvaro Herrera
Robert Haas escribió: > Maybe what we should do is add a function something like > pg_tuple_header(tableoid, ctid) that returns a record, maybe something > like (rawxmin xid, rawxmax xid, rawcid cid, infomask int, infomask2 > int, hoff int). Or perhaps some slightly more cooked version of that >

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Andres Freund
On 2013-12-20 07:58:46 -0500, Robert Haas wrote: > I think the immediate problem is to decide whether this patch ought to > make the xmin column display the result of GetXmin() or GetRawXmin(). > Thoughts on that? I slightly favor GetRawXmin(). Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Robert Haas
On Fri, Dec 20, 2013 at 7:51 AM, Andres Freund wrote: > On 2013-12-20 07:47:17 -0500, Robert Haas wrote: >> On Fri, Dec 20, 2013 at 7:22 AM, Andres Freund >> wrote: >> >> Maybe what we should do is add a function something like >> >> pg_tuple_header(tableoid, ctid) that returns a record, maybe s

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Andres Freund
On 2013-12-20 07:47:17 -0500, Robert Haas wrote: > On Fri, Dec 20, 2013 at 7:22 AM, Andres Freund wrote: > >> Maybe what we should do is add a function something like > >> pg_tuple_header(tableoid, ctid) that returns a record, maybe something > >> like (rawxmin xid, rawxmax xid, rawcid cid, infoma

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Robert Haas
On Fri, Dec 20, 2013 at 7:22 AM, Andres Freund wrote: >> Maybe what we should do is add a function something like >> pg_tuple_header(tableoid, ctid) that returns a record, maybe something >> like (rawxmin xid, rawxmax xid, rawcid cid, infomask int, infomask2 >> int, hoff int). Or perhaps some sli

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Andres Freund
On 2013-12-20 07:12:01 -0500, Robert Haas wrote: > I think the root of the problem is that nobody's very eager to add > more hidden system catalog columns because each one bloats > pg_attribute significantly. I think that part is actually solveable - there's no real need for them to be real column

Re: [HACKERS] preserving forensic information when we freeze

2013-12-20 Thread Robert Haas
On Thu, Dec 19, 2013 at 9:22 PM, Alvaro Herrera wrote: > Jim Nasby escribió: >> One thing users will lose in this patch is the ability to reliably see if a >> tuple is frozen via SQL. Today you can do that just by selecting xmin from >> the table. >> >> Obviously people don't generally need to d

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Alvaro Herrera
Jim Nasby escribió: > One thing users will lose in this patch is the ability to reliably see if a > tuple is frozen via SQL. Today you can do that just by selecting xmin from > the table. > > Obviously people don't generally need to do that... but it's one of those > things that when you do nee

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Jim Nasby
One thing users will lose in this patch is the ability to reliably see if a tuple is frozen via SQL. Today you can do that just by selecting xmin from the table. Obviously people don't generally need to do that... but it's one of those things that when you do need it it's incredibly handy to h

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Robert Haas
On Thu, Dec 19, 2013 at 3:36 PM, Alvaro Herrera wrote: > Robert Haas escribió: >> I think it may have been a mistake to divide responsibility between >> the prepare and execute functions the way we did in this case, because >> it doesn't appear to be a clean separation of concerns. But it's not >

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Alvaro Herrera
Robert Haas escribió: > I think it may have been a mistake to divide responsibility between > the prepare and execute functions the way we did in this case, because > it doesn't appear to be a clean separation of concerns. But it's not > this patch's job to kibitz that decision, so this version j

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Robert Haas
On Thu, Dec 19, 2013 at 9:37 AM, Robert Haas wrote: > On Thu, Dec 19, 2013 at 9:19 AM, Andres Freund wrote: >> On 2013-12-19 07:40:40 -0500, Robert Haas wrote: >>> On Thu, Dec 19, 2013 at 5:44 AM, Andres Freund >>> wrote: >>> > On 2013-12-18 21:42:25 -0500, Robert Haas wrote: >>> >>> > I dislik

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Robert Haas
On Thu, Dec 19, 2013 at 9:19 AM, Andres Freund wrote: > On 2013-12-19 07:40:40 -0500, Robert Haas wrote: >> On Thu, Dec 19, 2013 at 5:44 AM, Andres Freund >> wrote: >> > On 2013-12-18 21:42:25 -0500, Robert Haas wrote: >> >> > I dislike transporting the >> > infomask in the wal record and then c

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Andres Freund
On 2013-12-19 07:40:40 -0500, Robert Haas wrote: > On Thu, Dec 19, 2013 at 5:44 AM, Andres Freund wrote: > > On 2013-12-18 21:42:25 -0500, Robert Haas wrote: > > > I dislike transporting the > > infomask in the wal record and then changing it away from that again > > afterwards. > > I don't rea

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Robert Haas
On Thu, Dec 19, 2013 at 5:44 AM, Andres Freund wrote: > On 2013-12-18 21:42:25 -0500, Robert Haas wrote: >> On Wed, Dec 18, 2013 at 5:54 PM, Andres Freund >> wrote: >> >> if (frz->frzflags & XLH_FREEZE_XVAC) >> >> + { >> >> HeapTupleHeaderSetXvac(tuple, FrozenTransactionI

Re: [HACKERS] preserving forensic information when we freeze

2013-12-19 Thread Andres Freund
On 2013-12-18 21:42:25 -0500, Robert Haas wrote: > On Wed, Dec 18, 2013 at 5:54 PM, Andres Freund wrote: > >> if (frz->frzflags & XLH_FREEZE_XVAC) > >> + { > >> HeapTupleHeaderSetXvac(tuple, FrozenTransactionId); > >> + /* If we somehow haven't hinted the tuple

Re: [HACKERS] preserving forensic information when we freeze

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 5:54 PM, Andres Freund wrote: >> if (frz->frzflags & XLH_FREEZE_XVAC) >> + { >> HeapTupleHeaderSetXvac(tuple, FrozenTransactionId); >> + /* If we somehow haven't hinted the tuple previously, do it >> now. */ >> + HeapTupleHea

Re: [HACKERS] preserving forensic information when we freeze

2013-12-18 Thread Andres Freund
Hi, On 2013-12-17 16:00:14 -0500, Robert Haas wrote: > @@ -5874,19 +5858,27 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple, > TransactionId cutoff_xid, > void > heap_execute_freeze_tuple(HeapTupleHeader tuple, xl_heap_freeze_tuple *frz) > { > + tuple->t_infomask = frz->t_infomask; > +

Re: [HACKERS] preserving forensic information when we freeze

2013-12-17 Thread Robert Haas
On Wed, Dec 11, 2013 at 5:25 PM, Alvaro Herrera wrote: > Andres Freund escribió: >> What's your plan to commit this? I'd prefer to wait till Alvaro's >> freezing changes get in, so his patch will look the same in HEAD and >> 9.3. But I think he plans to commit soon. > > Yes, I do. I'm waiting on

Re: [HACKERS] preserving forensic information when we freeze

2013-12-11 Thread Alvaro Herrera
Andres Freund escribió: > What's your plan to commit this? I'd prefer to wait till Alvaro's > freezing changes get in, so his patch will look the same in HEAD and > 9.3. But I think he plans to commit soon. Yes, I do. I'm waiting on feedback on the patch I posted this afternoon, so if there's no

Re: [HACKERS] preserving forensic information when we freeze

2013-12-11 Thread Andres Freund
On 2013-12-11 14:17:27 -0500, Robert Haas wrote: > > But in which cases would that actually be slower? There'll be no > > additional code executed if the hint bits for frozen are set, and in > > case not it will usually safe us an external function call to > > TransactionIdPrecedes(). > > Dunno.

Re: [HACKERS] preserving forensic information when we freeze

2013-12-11 Thread Heikki Linnakangas
On 12/11/2013 09:17 PM, Robert Haas wrote: On Thu, Nov 21, 2013 at 4:51 PM, Andres Freund wrote: On 2013-11-21 15:59:35 -0500, Robert Haas wrote: Separate patch, but yeah, something like that. If we have to mark the page all-visible, we might as well freeze it while we're there. We should th

Re: [HACKERS] preserving forensic information when we freeze

2013-12-11 Thread Robert Haas
On Thu, Nov 21, 2013 at 4:51 PM, Andres Freund wrote: > On 2013-11-21 15:59:35 -0500, Robert Haas wrote: >> > * Should HeapTupleHeaderXminFrozen also check for FrozenTransactionId? >> > It seems quite possible that people think they've delt with frozen >> > xmin entirely after checking, but th

Re: [HACKERS] preserving forensic information when we freeze

2013-11-24 Thread Simon Riggs
On 21 November 2013 16:51, Andres Freund wrote: > Definitely separate yes. And I agree, it's partially moot if Heikki's > patch gets in, but I am not sure it will make it into 9.4. There seems > to be quite some work left. I'd prefer to do all 3 patches in one release. Don't mind which one. --

Re: [HACKERS] preserving forensic information when we freeze

2013-11-21 Thread Andres Freund
On 2013-11-21 15:59:35 -0500, Robert Haas wrote: > > * Should HeapTupleHeaderXminFrozen also check for FrozenTransactionId? > > It seems quite possible that people think they've delt with frozen > > xmin entirely after checking, but they still might get > > FrozenTransactionId back in a pg_up

Re: [HACKERS] preserving forensic information when we freeze

2013-11-21 Thread Andres Freund
Hi, As promised I'm currently updating the patch. Some questions arose during that: * Should HeapTupleHeaderXminFrozen also check for FrozenTransactionId? It seems quite possible that people think they've delt with frozen xmin entirely after checking, but they still might get FrozenTransact

Re: [HACKERS] preserving forensic information when we freeze

2013-11-21 Thread Robert Haas
On Thu, Nov 21, 2013 at 9:47 AM, Andres Freund wrote: > As promised I'm currently updating the patch. Some questions arose > during that: > > * Should HeapTupleHeaderXminFrozen also check for FrozenTransactionId? > It seems quite possible that people think they've delt with frozen > xmin entir

Re: [HACKERS] preserving forensic information when we freeze

2013-07-08 Thread Robert Haas
On Jul 8, 2013, at 1:34 PM, Josh Berkus wrote: > On 07/03/2013 11:59 AM, Robert Haas wrote: >> Yeah. I think the system columns that we have today are pretty much >> crap. I wonder if we couldn't throw them out and replace them with >> some kind of functions that you can pass a row to. That wou

Re: [HACKERS] preserving forensic information when we freeze

2013-07-08 Thread Josh Berkus
On 07/03/2013 11:59 AM, Robert Haas wrote: > Yeah. I think the system columns that we have today are pretty much > crap. I wonder if we couldn't throw them out and replace them with > some kind of functions that you can pass a row to. That would allow > us to expose a lot more detail without add

Re: [HACKERS] preserving forensic information when we freeze

2013-07-03 Thread Marko Kreen
On Wed, Jul 3, 2013 at 8:07 PM, Andres Freund wrote: > On 2013-07-02 16:29:56 -0400, Robert Haas wrote: >> There's no possibility of getting confused here; if X is still around >> at all, it's xmax is of the same generation as Y's xmin. Otherwise, >> we've had an undetected XID wraparound. > > An

Re: [HACKERS] preserving forensic information when we freeze

2013-07-03 Thread Robert Haas
On Wed, Jul 3, 2013 at 1:07 PM, Andres Freund wrote: > Well, nothing would prevent using the HeapTupleHeaderGetRawXmin() in > those places. Exactly the number of callsites is what makes me think > that somebody will get this wrong in the future. Well, I guess I could go rework the whole patch tha

Re: [HACKERS] preserving forensic information when we freeze

2013-07-03 Thread Andres Freund
On 2013-07-02 16:29:56 -0400, Robert Haas wrote: > On Mon, Jun 24, 2013 at 8:12 AM, Andres Freund wrote: > > Agreed. And it also improves the status quo when debugging. I wish this > > would have been the representation since the beginning. > > > > Some remarks: > > * I don't really like that Heap

Re: [HACKERS] preserving forensic information when we freeze

2013-07-02 Thread Robert Haas
On Mon, Jun 24, 2013 at 8:12 AM, Andres Freund wrote: > Agreed. And it also improves the status quo when debugging. I wish this > would have been the representation since the beginning. > > Some remarks: > * I don't really like that HeapTupleHeaderXminFrozen() now is frequently > performed witho

Re: [HACKERS] preserving forensic information when we freeze

2013-06-24 Thread Andres Freund
On 2013-05-28 09:21:27 -0400, Robert Haas wrote: > As a general statement, I view this work as something that is likely > needed no matter which one of the "remove freezing" approaches that > have been proposed we choose to adopt. It does not fix anything in > and of itself, but it (hopefully) rem

Re: [HACKERS] preserving forensic information when we freeze

2013-05-29 Thread Alvaro Herrera
Robert Haas escribió: > On Tue, May 28, 2013 at 10:08 PM, Andres Freund > wrote: > > On 2013-05-28 21:26:49 -0400, Robert Haas wrote: > >> > I am all for adding a comment why this is safe though. We thought about > >> > it for some while before we were convinced... > >> > >> I'm fine with that,

Re: [HACKERS] preserving forensic information when we freeze

2013-05-29 Thread Peter Eisentraut
On 5/28/13 8:00 PM, Andres Freund wrote: >> - Various procedural languages use the combination of TID and XMIN to >> > determine whether a function needs to be recompiled. > Hm. As previously said, I am less than convinced of those adhoc > mechanisms and I think this should get properly integrated

Re: [HACKERS] preserving forensic information when we freeze

2013-05-29 Thread Robert Haas
On Tue, May 28, 2013 at 10:08 PM, Andres Freund wrote: > On 2013-05-28 21:26:49 -0400, Robert Haas wrote: >> On Tue, May 28, 2013 at 8:00 PM, Andres Freund >> wrote: >> > I only suggested MOVED_IN/OFF because I didn't remember >> > HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID was still free ;). So, unl

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Andres Freund
On 2013-05-28 21:26:49 -0400, Robert Haas wrote: > On Tue, May 28, 2013 at 8:00 PM, Andres Freund wrote: > > I only suggested MOVED_IN/OFF because I didn't remember > > HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID was still free ;). So, unless that > > combination could have been produced in the past in

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 8:00 PM, Andres Freund wrote: > I only suggested MOVED_IN/OFF because I didn't remember > HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID was still free ;). So, unless that > combination could have been produced in the past in a way I couldn't > find so far, I am all for it. I don't

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Andres Freund
On 2013-05-28 09:21:27 -0400, Robert Haas wrote: > I have attempted to implement this. Trouble is, we're out of infomask > bits. Using an infomask2 bit might work but we don't have many of > them left either, so it's worth casting about a bit for a better > solution. Andres proposed using HEAP_

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 7:27 PM, Andres Freund wrote: > On 2013-05-28 09:39:13 -0700, Josh Berkus wrote: >> On 05/28/2013 06:21 AM, Robert Haas wrote: >> > As a general statement, I view this work as something that is likely >> > needed no matter which one of the "remove freezing" approaches that

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Andres Freund
On 2013-05-28 09:39:13 -0700, Josh Berkus wrote: > On 05/28/2013 06:21 AM, Robert Haas wrote: > > As a general statement, I view this work as something that is likely > > needed no matter which one of the "remove freezing" approaches that > > have been proposed we choose to adopt. It does not fix

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Josh Berkus
On 05/28/2013 06:21 AM, Robert Haas wrote: > As a general statement, I view this work as something that is likely > needed no matter which one of the "remove freezing" approaches that > have been proposed we choose to adopt. It does not fix anything in > and of itself, but it (hopefully) removes a

[HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Robert Haas
Various people, including at least Heikki, Andres, and myself, have proposed various schemes for avoiding freezing that amount to doing it sooner, when we're already writing WAL anyway, or at least when the buffer is already dirty anyway, or at least while the buffer is already in shared_buffers an