Re: [HACKERS] Vacuum/visibility is busted

2013-06-20 Thread Andres Freund
On 2013-06-19 15:01:44 -0700, Jeff Janes wrote: On Thu, Feb 7, 2013 at 12:01 PM, Andres Freund and...@2ndquadrant.comwrote: On 2013-02-07 11:15:46 -0800, Jeff Janes wrote: Does anyone have suggestions on how to hack the system to make it fast-forward the current transaction id? It

Re: [HACKERS] Vacuum/visibility is busted

2013-06-19 Thread Jeff Janes
On Thu, Feb 7, 2013 at 12:01 PM, Andres Freund and...@2ndquadrant.comwrote: On 2013-02-07 11:15:46 -0800, Jeff Janes wrote: Does anyone have suggestions on how to hack the system to make it fast-forward the current transaction id? It would certainly make testing this kind of thing faster

Re: [HACKERS] Vacuum/visibility is busted

2013-02-08 Thread Pavan Deolasee
On Fri, Feb 8, 2013 at 10:08 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera escribió: Alvaro Herrera escribió: Hm, if the foreign key patch is to blame, this sounds like these tuples had a different set of XMAX hint bits and a different Xmax, and they were clobbered by

Re: [HACKERS] Vacuum/visibility is busted

2013-02-08 Thread Alvaro Herrera
Pavan Deolasee escribió: I'm trying to reason how this bug explains what we saw. In the test, we'd left with duplicate tuples. If I just take index 219 in the table as an example, that tuple had three duplicates. The tuple with CTID (150, 126) had the index pointer and the rest two were

Re: [HACKERS] Vacuum/visibility is busted

2013-02-08 Thread Alvaro Herrera
Tom Lane escribió: Alvaro Herrera alvhe...@2ndquadrant.com writes: xid = HeapTupleHeaderGetRawXmax(tuple); ! if (((tuple-t_infomask HEAP_XMAX_IS_MULTI) !MultiXactIdIsValid(xid) !MultiXactIdPrecedes(xid, cutoff_multi)) || ! ((!(tuple-t_infomask

Re: [HACKERS] Vacuum/visibility is busted

2013-02-08 Thread Jeff Janes
On Thu, Feb 7, 2013 at 8:38 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera escribió: Alvaro Herrera escribió: Hm, if the foreign key patch is to blame, this sounds like these tuples had a different set of XMAX hint bits and a different Xmax, and they were clobbered by

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Simon Riggs
On 7 February 2013 05:39, Jeff Janes jeff.ja...@gmail.com wrote: While stress testing Pavan's 2nd pass vacuum visibility patch, I realized that vacuum/visibility was busted. But it wasn't his patch that busted it. As far as I can tell, the bad commit was in the range

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 11:09 AM, Jeff Janes jeff.ja...@gmail.com wrote: While stress testing Pavan's 2nd pass vacuum visibility patch, I realized that vacuum/visibility was busted. But it wasn't his patch that busted it. As far as I can tell, the bad commit was in the range

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Will look more into it, but thought this might be useful for others to spot the problem. And here is some more forensic info about one of the pages having duplicate tuples. jjanes=# select *, xmin, xmax, ctid

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Pavan Deolasee escribió: On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Will look more into it, but thought this might be useful for others to spot the problem. And here is some more forensic info about one of the pages having duplicate tuples.

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Alvaro Herrera escribió: Hm, if the foreign key patch is to blame, this sounds like these tuples had a different set of XMAX hint bits and a different Xmax, and they were clobbered by something like vacuum or page pruning. Hm, I think heap_freeze_tuple is busted, yes. -- Álvaro Herrera

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 1:44 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: Will look more into it, but thought this might be useful for others to spot the problem. And here is some more forensic info about

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Thu, Feb 7, 2013 at 11:09 AM, Jeff Janes jeff.ja...@gmail.com wrote: While stress testing Pavan's 2nd pass vacuum visibility patch, I realized that vacuum/visibility was busted. But it wasn't his patch that

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 9:32 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Index scans do not return any duplicates and you need to force a seq scan to see them. Assuming that the page level VM bit might be

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Jeff Janes escribió: On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: I don't see the assertion failure myself. If I do REINDEX INDEX it gives a duplicate key violation, and if I do REINDEX TABLE or REINDEX DATABASE I get claimed success. This is using

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 10:48 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, Feb 7, 2013 at 1:44 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: jjanes=# select *, xmin, xmax, ctid from foo where index IN (select index from foo group by index having count(*) 1 ORDER by index) ORDER

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Right. I don't have the database handy at this moment, but earlier in the day I ran some queries against it and found that most of the duplicates which are not accessible via indexes have xmin very close to

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Andres Freund
On 2013-02-07 11:15:46 -0800, Jeff Janes wrote: On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Right. I don't have the database handy at this moment, but earlier in the day I ran some queries against it and found that most of the duplicates which are not

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: Does anyone have suggestions on how to hack the system to make it fast-forward the current transaction id? What I've generally done is to stop the server then use pg_resetxlog to put the XID counter where I want it. I believe you'll need to manually

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Simon Riggs
On 7 February 2013 19:15, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Right. I don't have the database handy at this moment, but earlier in the day I ran some queries against it and found that most of the duplicates

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Jeff Janes escribió: On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Right. I don't have the database handy at this moment, but earlier in the day I ran some queries against it and found that most of the duplicates which are not accessible via indexes have

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Alvaro Herrera escribió: Alvaro Herrera escribió: Hm, if the foreign key patch is to blame, this sounds like these tuples had a different set of XMAX hint bits and a different Xmax, and they were clobbered by something like vacuum or page pruning. Hm, I think heap_freeze_tuple is

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: xid = HeapTupleHeaderGetRawXmax(tuple); ! if (((tuple-t_infomask HEAP_XMAX_IS_MULTI) ! MultiXactIdIsValid(xid) ! MultiXactIdPrecedes(xid, cutoff_multi)) || ! ((!(tuple-t_infomask

[HACKERS] Vacuum/visibility is busted

2013-02-06 Thread Jeff Janes
While stress testing Pavan's 2nd pass vacuum visibility patch, I realized that vacuum/visibility was busted. But it wasn't his patch that busted it. As far as I can tell, the bad commit was in the range 692079e5dcb331..168d3157032879 Since a run takes 12 to 24 hours, it will take a while to

Re: [HACKERS] Vacuum/visibility is busted

2013-02-06 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: While stress testing Pavan's 2nd pass vacuum visibility patch, I realized that vacuum/visibility was busted. But it wasn't his patch that busted it. As far as I can tell, the bad commit was in the range 692079e5dcb331..168d3157032879 Since a run