Re: [BUGS] Duplicate values found when reindexing unique index

2008-01-04 Thread Michael Orozco
Someone tell me now to get off this list -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Alvaro Herrera Sent: Thursday, January 03, 2008 8:30 PM To: Tom Lane Cc: Mason Hale; pgsql-bugs@postgresql.org Subject: Re: [BUGS] Duplicate values found when

Re: [BUGS] Duplicate values found when reindexing unique index

2008-01-04 Thread Kevin Grittner
>>> On Fri, Jan 4, 2008 at 8:00 AM, in message <[EMAIL PROTECTED]>, "Michael Orozco" <[EMAIL PROTECTED]> wrote: > Someone tell me now to get off this list From the main web page at: http://www.postgresql.org/ There is a link to "Mailing Lists". If you take that, you will see, up at the to

Re: [BUGS] Duplicate values found when reindexing unique index

2008-01-03 Thread Alvaro Herrera
Tom Lane escribió: > Given this, and the index corruption you showed before (the wrong > sibling link, which would represent index breakage quite independent of > what was in the heap), and the curious contents of your WAL files > (likewise not explainable by anything going wrong within a table),

Re: [BUGS] Duplicate values found when reindexing unique index

2008-01-03 Thread Mason Hale
Hi everyone -- Sorry to revisit a dead horse, but I wanted to clear up some misinformation -- On Dec 31, 2007 5:35 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Mason Hale" <[EMAIL PROTECTED]> writes: > >> This could be the kernel's fault, but I'm wondering whether the > >> RAID controller is going

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Simon Riggs
On Mon, 2007-12-31 at 18:35 -0500, Tom Lane wrote: > "Mason Hale" <[EMAIL PROTECTED]> writes: > >> This could be the kernel's fault, but I'm wondering whether the > >> RAID controller is going south. > > > To clarify a bit further -- on the production server, the data is written to > > a 10-disk R

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: >> This could be the kernel's fault, but I'm wondering whether the >> RAID controller is going south. > To clarify a bit further -- on the production server, the data is written to > a 10-disk RAID 1+0, but the pg_xlog directory is symlinked to a separate,

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
Hello Tom -- a thousand thanks for taking the time to walk through those files. This could be the kernel's fault, but I'm wondering whether the > RAID controller is going south. Offhand it seems like the controller > would be the only place that would be likely to explain both the > bogus-data an

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: >> If you'd send them to me privately, I'd be interested. > I will send these to you right away. Okay, the 00058 file looks pretty sane, except for what we already knew about the first page being overwritten with a much later shutdown checkpoint. All the

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
On Dec 31, 2007 12:51 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Mason Hale" <[EMAIL PROTECTED]> writes: > > If it is some mysterious hardware or kernel error -- any suggestions for > how > > I should go about narrowing that down? > > The first thing to ask is whether you're running an up-to-date

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: > If it is some mysterious hardware or kernel error -- any suggestions for how > I should go about narrowing that down? The first thing to ask is whether you're running an up-to-date kernel (and whose is it). > Finally, if it would help to see the full wal

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
> > Something else interesting about that: the apparent interloper page > contains just a single WAL record, which appears to be a shutdown > checkpoint bearing the timestamp "Thu Dec 27 2007, 16:55:01 EST". > Not sure if Mason can correlate that with any recent activity... > Nothing jumps to mind

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Simon Riggs
On Mon, 2007-12-31 at 13:14 -0500, Tom Lane wrote: > "Mason Hale" <[EMAIL PROTECTED]> writes: > > [EMAIL PROTECTED] wal_archive]$ od -x 000104220058 | head -n15 > > 000 d05e 0002 0001 0423 c100 > > 020 f7df 472e e701 4728 0100 2000 > > 040 a1db 81e6

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: > [EMAIL PROTECTED] wal_archive]$ od -x 000104220058 | head -n15 > 000 d05e 0002 0001 0423 c100 > 020 f7df 472e e701 4728 0100 2000 > 040 a1db 81e6 0423 0068 c000 > 060 0048 002c 0

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2007-12-31 at 12:33 -0500, Tom Lane wrote: >> Actually, the other problem with that theory is that the slave swallowed >> the file without complaint. > No, it barfed. Mason showed us a recovery script, so it came from the > slave. No, it barfed

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Simon Riggs
On Mon, 2007-12-31 at 12:33 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Mon, 2007-12-31 at 11:53 -0500, Tom Lane wrote: > >> The state of the ...0058 file might be explained by the theory that > >> you'd archived it a bit too late (after the first page had been > >> over

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2007-12-31 at 11:53 -0500, Tom Lane wrote: >> The state of the ...0058 file might be explained by the theory that >> you'd archived it a bit too late (after the first page had been >> overwritten with newer WAL data), > The interlock with .ready a

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: > Tom, I'll send these to you privately. Thanks. I don't see anything particularly surprising there though. What I was wondering about was whether your application was in the habit of doing repeated no-op updates on the same "entry" row. The pg_filedump o

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Simon Riggs
On Mon, 2007-12-31 at 11:53 -0500, Tom Lane wrote: > It might be worth trawling through both files to check the page headers > (every 8K) and see which ones agree with expectation and which don't. > The state of the ...0058 file might be explained by the theory that > you'd archived it a bit too l

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: > On Dec 31, 2007 9:48 AM, Tom Lane <[EMAIL PROTECTED]> wrote: >> Do you by any chance still have 000104220058 and >> 000104220059 archived? If so it would be useful to >> look at the first dozen lines of "od -x" dump of each of them

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
Tom, I'll send these to you privately. Mason On Dec 31, 2007 10:22 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Mason Hale" <[EMAIL PROTECTED]> writes: > >> Can you show us all the triggers on this table? > > > Here they are: > > > Triggers: > > entry_correct_published_at_trigger BEFORE INSERT

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: >> Can you show us all the triggers on this table? > Here they are: > Triggers: > entry_correct_published_at_trigger BEFORE INSERT OR UPDATE ON entry FOR > EACH ROW EXECUTE PROCEDURE correct_published_at() > entry_feed_page_trigger BEFORE INSERT OR

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
On Dec 31, 2007 9:48 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Mason Hale" <[EMAIL PROTECTED]> writes: > >> I think you misread Mason's post of 20:23 GMT-6 where he says the > >> created_at values are the *same*, not different. Mason's previous bug > >> report 3724 also had duplicate rows with ma

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: >> I think you misread Mason's post of 20:23 GMT-6 where he says the >> created_at values are the *same*, not different. Mason's previous bug >> report 3724 also had duplicate rows with matching created_at values. > Yes, to confirm, the created_at values ar

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
> > > I think you misread Mason's post of 20:23 GMT-6 where he says the > created_at values are the *same*, not different. Mason's previous bug > report 3724 also had duplicate rows with matching created_at values. > Yes, to confirm, the created_at values are the same. The *only* values that are d

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Simon Riggs
On Mon, 2007-12-31 at 01:27 -0500, Tom Lane wrote: > Gregory Stark <[EMAIL PROTECTED]> writes: > > "Tom Lane" <[EMAIL PROTECTED]> writes: > >> I wonder whether it's just a coincidence that these have the same offset > >> number... > > > I can't imagine any Postgres bug which would depend on the of

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
> > > Can you show us all the triggers on this table? Here they are: Triggers: entry_correct_published_at_trigger BEFORE INSERT OR UPDATE ON entry FOR EACH ROW EXECUTE PROCEDURE correct_published_at() entry_feed_page_trigger BEFORE INSERT OR UPDATE ON entry FOR EACH ROW EXECUTE PROCEDURE

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
Trolling through my server log I found this error: 2007-12-30 20:02:08 CST (10.11.199.136) PANIC: right sibling's left-link doesn't match 2007-12-30 20:02:08 CST (10.11.199.136) STATEMENT: update bdu.entry set title=$1, author=$2, description_type=$3, description_length=$4, description=$5, publis

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-31 Thread Mason Hale
On Dec 30, 2007 12:09 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Mason Hale" <[EMAIL PROTECTED]> writes: > > Given my experience, the reliability of unique indexes is becoming > somewhat > > suspect. Please help. ;-) > > Well, as in the previous report, there is not enough information here to > of

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-30 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes: >> I wonder whether it's just a coincidence that these have the same offset >> number... > I can't imagine any Postgres bug which would depend on the offsets > being the same. But what I could imagine is filesystem

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-30 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: >> Here's the system column data you requested. > >> id | ctid | xmin | xmax | cmin | cmax >> ---+--+--+--+--+-- >> 151341072 | (1508573,11) |2 |0 | 19 |0 >> 151341072 | (1818219,11) |2

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-30 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: > I have downloaded, compiled and installed pg_filedump -- but I am not sure > how to determine which file I should have it dump. I am not very familiar > with the postgres file structure. Can you please provide some guidance? How > do I determine the correc

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-30 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: > I found a single pair of rows that were duplicated. Interestingly it was not > just the guid and feed_id that were duplicated but all columns were > indentical, including the primary key, except an update_at column which is > automatically populated via a

Re: [BUGS] Duplicate values found when reindexing unique index

2007-12-30 Thread Tom Lane
"Mason Hale" <[EMAIL PROTECTED]> writes: > Given my experience, the reliability of unique indexes is becoming somewhat > suspect. Please help. ;-) Well, as in the previous report, there is not enough information here to offer much chance of understanding what's going wrong. Have you tried reindex

[BUGS] Duplicate values found when reindexing unique index

2007-12-30 Thread Mason Hale
Hello -- I noticed recently some errors in my postgres log like the following: ERROR: could not open segment 9 of relation 1663/16830/1384385 (target block 776929292): No such file or directory These are occurring at a rate of 1 every 2-3 days. But that rate has been increasing. After Googling