Re: [HACKERS] pg_index updates and SI invalidation
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > Please see the attached updated patch, based on Tom's comments. > Attempt to reload index information for system indexes such as > pg_class_oid_index can cause infinite recursion. But I realized that > we don't need to reload system index information because we > neither allow CREATE INDEX or CIC on system relations. Only > REINDEX is allowed which does not need any reload. So we skip > index information reload for system relations. Applied with revisions --- mostly, trying to keep the comments in sync with the code. I also added a forced relcache inval on the index's parent table at the end of CREATE INDEX CONCURRENTLY; this is to flush cached plans and allow the newly valid index to be considered in replanning. (The relcache inval on the index won't do it, since by definition the index is not mentioned in any such plan...) regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] pg_index updates and SI invalidation
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. --- Pavan Deolasee wrote: > On 4/3/07, Bruce Momjian <[EMAIL PROTECTED]> wrote: > > > > > > Where are we on this? > > > > > > --- > > > > Tom Lane wrote: > > > > > > [squint...] How can that fail during a reload if it worked the first > > > time? Needs a closer look at what's happening. > > > > > > > Please see the attached updated patch, based on Tom's comments. > > Attempt to reload index information for system indexes such as > pg_class_oid_index can cause infinite recursion. But I realized that > we don't need to reload system index information because we > neither allow CREATE INDEX or CIC on system relations. Only > REINDEX is allowed which does not need any reload. So we skip > index information reload for system relations. > > Thanks, > Pavan > > -- > > EnterpriseDB http://www.enterprisedb.com [ Attachment, skipping... ] -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] pg_index updates and SI invalidation
On 4/3/07, Bruce Momjian <[EMAIL PROTECTED]> wrote: Where are we on this? --- Tom Lane wrote: > > [squint...] How can that fail during a reload if it worked the first > time? Needs a closer look at what's happening. > Please see the attached updated patch, based on Tom's comments. Attempt to reload index information for system indexes such as pg_class_oid_index can cause infinite recursion. But I realized that we don't need to reload system index information because we neither allow CREATE INDEX or CIC on system relations. Only REINDEX is allowed which does not need any reload. So we skip index information reload for system relations. Thanks, Pavan -- EnterpriseDB http://www.enterprisedb.com pg_index_SI_inval_v2.patch Description: Binary data ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_index updates and SI invalidation
Where are we on this? --- Tom Lane wrote: > "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > > On 3/28/07, Tom Lane <[EMAIL PROTECTED]> wrote: > >> It seems a bit brute-force. Why didn't you use SearchSysCache(INDEXRELID) > >> the same as RelationInitIndexAccessInfo does? > > > I tried that initially, but it gets into infinite recursion during initdb. > > [squint...] How can that fail during a reload if it worked the first > time? Needs a closer look at what's happening. > > regards, tom lane > > ---(end of broadcast)--- > TIP 3: Have you checked our extensive FAQ? > >http://www.postgresql.org/docs/faq -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] pg_index updates and SI invalidation
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > On 3/28/07, Tom Lane <[EMAIL PROTECTED]> wrote: >> It seems a bit brute-force. Why didn't you use SearchSysCache(INDEXRELID) >> the same as RelationInitIndexAccessInfo does? > I tried that initially, but it gets into infinite recursion during initdb. [squint...] How can that fail during a reload if it worked the first time? Needs a closer look at what's happening. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] pg_index updates and SI invalidation
On 3/28/07, Tom Lane <[EMAIL PROTECTED]> wrote: It seems a bit brute-force. Why didn't you use SearchSysCache(INDEXRELID) the same as RelationInitIndexAccessInfo does? I tried that initially, but it gets into infinite recursion during initdb. And what's the point of the extra tuple copy step, instead of assigning the values into the cache entry immediately? Oops, sorry. Thats a copy-paste error. We certainly don't need to copy the tuple. Thanks, Pavan -- EnterpriseDB http://www.enterprisedb.com
Re: [HACKERS] pg_index updates and SI invalidation
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > Here is a patch which fixes this. We re-read part of the pg_index > row and update rd_index with the new data. I tested REINDEX and CIC > and both seems to work fine with the patch applied. > Tom, does this look good ? It seems a bit brute-force. Why didn't you use SearchSysCache(INDEXRELID) the same as RelationInitIndexAccessInfo does? And what's the point of the extra tuple copy step, instead of assigning the values into the cache entry immediately? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] pg_index updates and SI invalidation
On 3/26/07, Tom Lane <[EMAIL PROTECTED]> wrote: It might be feasible to have RelationReloadClassinfo re-read the pg_index row and apply only the updates for specific known-changeable columns. The stuff it's worried about is the subsidiary data such as support function fmgr lookup records, but we don't need those to change on the fly. Here is a patch which fixes this. We re-read part of the pg_index row and update rd_index with the new data. I tested REINDEX and CIC and both seems to work fine with the patch applied. Tom, does this look good ? Thanks, Pavan -- EnterpriseDB http://www.enterprisedb.com pg_index_SI_inval.patch Description: Binary data ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] pg_index updates and SI invalidation
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > On 3/26/07, Tom Lane <[EMAIL PROTECTED]> wrote: >> Hmm ... actually, CREATE INDEX CONCURRENTLY gets this wrong already, no? > Yes, C.I.C gets it wrong. I confirmed that new index is seen as invalid > for existing sessions. Is it something we should fix ? Certainly. It might be feasible to have RelationReloadClassinfo re-read the pg_index row and apply only the updates for specific known-changeable columns. The stuff it's worried about is the subsidiary data such as support function fmgr lookup records, but we don't need those to change on the fly. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_index updates and SI invalidation
On 3/26/07, Tom Lane <[EMAIL PROTECTED]> wrote: Hmm ... actually, CREATE INDEX CONCURRENTLY gets this wrong already, no? I suspect that sessions existing at the time C.I.C is done will never see the new index as valid, unless something else happens to make them drop and rebuild their relcache entries for it. Yes, C.I.C gets it wrong. I confirmed that new index is seen as invalid for existing sessions. Is it something we should fix ? Thanks, Pavan -- EnterpriseDB http://www.enterprisedb.com
Re: [HACKERS] pg_index updates and SI invalidation
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > While experimenting with the proposed CREATE INDEX support with > HOT, I realized that SI invalidation are not sent properly for pg_index > updates. Hmm ... actually, CREATE INDEX CONCURRENTLY gets this wrong already, no? I suspect that sessions existing at the time C.I.C is done will never see the new index as valid, unless something else happens to make them drop and rebuild their relcache entries for it. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
