Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-14 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > If that assumption is true (and it must be true for us to move > the chain in pieces), doesn't that mean we don't really need to > move the RECENTLY_DEAD tuples preceding a DEAD tuple ? As I've already said several times: they are dead, but at least f

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-13 Thread Pavan Deolasee
Tom Lane wrote: > > I've developed the attached patch against HEAD, and no longer see any > funny behavior. Would appreciate it if you'd test some more, though. > The patch works for me. With the patch applied, I don't see the weird errors in the pgbench and other customized tests that I used to

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-13 Thread Tom Lane
> "Pavan Deolasee" <[EMAIL PROTECTED]> writes: >> The problem mentioned before is hard to reproduce with the >> suggested change, but its not completely gone away. I have >> seen that again on CVS HEAD with the patch applied. >> I am facing another issue with VACUUM FULL. This >> problem gets repro

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-10 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > [ raised eyebrow... ] You sure about that? If you replace an XID > before OldestXmin with one after, or vice versa, ISTM you *could* > be creating a problem. "Committed" is not good enough. So it looks > to me like you can't remove a DEAD tuple whose pr

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-10 Thread Pavan Deolasee
On 3/10/07, Pavan Deolasee <[EMAIL PROTECTED]> wrote: scan_heap() would usually have collected the DEAD tuple in offsets_free list. How do you plan to check if the tuple is in middle on a chain which has RECENTLY_DEAD tuple before the tuple under check ? Don't we need to collect the TID of the

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-10 Thread Pavan Deolasee
On 3/10/07, Tom Lane <[EMAIL PROTECTED]> wrote: Although this shouldn't happen anymore after fixing the chaining conditions, I'm inclined to introduce an additional test to verify that the starting tuple is actually MOVED_OFF after we finish the chain move. If not, give up on repair_frag the sa

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-10 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > In general, I believe that the most likely cause for earlier reported > errors is that we are failing to clean up one or more index entries > in VACUUM FULL, thus causing all sorts of errors. I had a hard > time fixing this case for HOT. Yeah, the ca

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-10 Thread Pavan Deolasee
On 3/10/07, Tom Lane <[EMAIL PROTECTED]> wrote: Also, we know this case works because it already is working: in the situation where VACUUM happens to visit and remove the DEAD tuple(s) before reaching the RECENTLY_DEAD tuples that link forward to them, it treats the RECENTLY_DEAD tuples as a di

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-10 Thread Pavan Deolasee
On 3/10/07, Tom Lane <[EMAIL PROTECTED]> wrote: I've been banging away on this since yesterday, and I think I've achieved a full understanding of what's going on. There are three or four different-looking pathologies but they all seem to arise from the same problem: the update-chain-moving cod

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-10 Thread Simon Riggs
On Fri, 2007-03-09 at 18:00 -0500, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > On Fri, 2007-03-09 at 16:40 -0500, Tom Lane wrote: > >> I wonder whether this has any implications for HOT ... > > > My general feeling, expressed in a number of recent posts was that the > > VACUUM

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-09 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > Breaking the chain up into pieces seems weird. It seems like it's obviously > bogus and only works because we're sure the tuples are dead anyways so it > doesn't really matter what we do with them. Yup, exactly. If we wanted to be tense about this we'd

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-09 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: >> Hm, I wonder if you could just notice that xmin is younger than OldestXmin. > > You can see that at the newer tuple, but the problem is to propagate the > knowledge back to the older tuple(s). Or were you suggesting that we > treat the newer tuple as REC

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-09 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > So txn 4's xmin is txn 3, leaving the global OldestXmin = txn 3 which lies > between txn 1 and txn 2. > And the tuple chain consists of two tuples. The original which has xmax > younger than OldestXmin and so is RECENTLY_DEAD. And the updated tuple which

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-09 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > It's not surprising that tuples could have xmax less than xmin, since > transactions can commit in orders different than they start; when using > READ COMMITTED updates it's not at all surprising that a transaction > might update rows after a later-numbered

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-09 Thread Joshua D. Drake
> Put it another way: if anybody submitted a patch that does what VACUUM > FULL does, coded in the way it is, it would never be applied, now. Have an opinion do we? How about we just alias VACUUM FULL to cluster and add the reporting stuff from VERBOSE? Joshua D. Drake -- === The Pos

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-09 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > On Fri, 2007-03-09 at 16:40 -0500, Tom Lane wrote: >> I wonder whether this has any implications for HOT ... > My general feeling, expressed in a number of recent posts was that the > VACUUM FULL code really isn't worth the trouble it causes. Especially

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-09 Thread Simon Riggs
On Fri, 2007-03-09 at 16:40 -0500, Tom Lane wrote: > I wonder whether this has any implications for HOT ... My general feeling, expressed in a number of recent posts was that the VACUUM FULL code really isn't worth the trouble it causes. Especially when CLUSTER does a better job anyway? I've prop

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-09 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > The problem mentioned before is hard to reproduce with the > suggested change, but its not completely gone away. I have > seen that again on CVS HEAD with the patch applied. > I am facing another issue with VACUUM FULL. This > problem gets reproduced w

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-08 Thread Pavan Deolasee
Pavan Deolasee wrote: > > Thanks a lot, Tom. It seems to work fine for me. I will do some > more tests and report if I see any issue. > The problem mentioned before is hard to reproduce with the suggested change, but its not completely gone away. I have seen that again on CVS HEAD with the patch

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-07 Thread Pavan Deolasee
Tom Lane wrote: > > Please check if this makes it go away for you --- I'm a bit busy > at the moment. > Thanks a lot, Tom. It seems to work fine for me. I will do some more tests and report if I see any issue. Btw, the patch as per your suggestion is attached. Thanks, Pavan *** src/backend/comm

Re: [HACKERS] Bug in VACUUM FULL ?

2007-03-07 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > I am right now working on to get HOT and VACUUM FULL work > together. I hit upon a bug which I initially thought is > something that HOT has introduced. But I can reproduce > it with CVS HEAD as well. I think we broke this in 8.2: vac_update_relstats

[HACKERS] Bug in VACUUM FULL ?

2007-03-07 Thread Pavan Deolasee
Hi, I am right now working on to get HOT and VACUUM FULL work together. I hit upon a bug which I initially thought is something that HOT has introduced. But I can reproduce it with CVS HEAD as well. Here is what I do: Create a table a simple table with three columns and one index. Insert a sing