On 09/15/2015 12:02 PM, Jan Wieck wrote:
On 09/15/2015 11:54 AM, Tom Lane wrote:
Jan Wieck <j...@wi3ck.info> writes:
Would it be appropriate to use (Un)RegisterXactCallback() in case of
detecting excessive sequential scanning of that cache and remove all
invalid entries from it at that time?
Another idea is that it's not the size of the cache that's the problem,
it's the cost of finding the entries that need to be invalidated.
So we could also consider adding list links that chain only the entries
that are currently marked valid. If the list gets too long, we could mark
them all invalid and thereby reset the list to nil. This doesn't do
anything for the cache's space consumption, but that wasn't what you were
worried about anyway was it?
That sounds like a workable solution to this edge case. I'll see how
that works.
Attached is a complete rework of the fix from scratch, based on Tom's
suggestion.
The code now maintains a double linked list as suggested, but only uses
it to mark all currently valid entries as invalid when hashvalue == 0.
If a specific entry is invalidated it performs a hash lookup for maximum
efficiency in that case.
This code does pass the regression tests with CLOBBER_CACHE_ALWAYS, does
have the desired performance impact on restore of hundreds of thousands
of foreign key constraints and does not show any negative impact on bulk
loading of data with foreign keys.
Regards, Jan
--
Jan Wieck
Senior Software Engineer
http://slony.info
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 61edde9..eaec737 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -125,6 +125,8 @@ typedef struct RI_ConstraintInfo
* PK) */
Oid ff_eq_oprs[RI_MAX_NUMKEYS]; /* equality operators (FK =
* FK) */
+ struct RI_ConstraintInfo *prev_valid; /* Previous valid entry */
+ struct RI_ConstraintInfo *next_valid; /* Next valid entry */
} RI_ConstraintInfo;
@@ -185,6 +187,7 @@ typedef struct RI_CompareHashEntry
static HTAB *ri_constraint_cache = NULL;
static HTAB *ri_query_cache = NULL;
static HTAB *ri_compare_cache = NULL;
+static RI_ConstraintInfo *ri_constraint_cache_valid_list = NULL;
/* ----------
@@ -2924,6 +2927,19 @@ ri_LoadConstraintInfo(Oid constraintOid)
ReleaseSysCache(tup);
+ /*
+ * At the time a cache invalidation message is processed there may be
+ * active references to the cache. Because of this we never remove entries
+ * from the cache, but only mark them invalid. For efficient processing
+ * of invalidation messages below we keep a double linked list of
+ * currently valid entries.
+ */
+ if (ri_constraint_cache_valid_list != NULL)
+ ri_constraint_cache_valid_list->prev_valid = riinfo;
+ riinfo->prev_valid = NULL;
+ riinfo->next_valid = ri_constraint_cache_valid_list;
+ ri_constraint_cache_valid_list = riinfo;
+
riinfo->valid = true;
return riinfo;
@@ -2940,17 +2956,55 @@ ri_LoadConstraintInfo(Oid constraintOid)
static void
InvalidateConstraintCacheCallBack(Datum arg, int cacheid, uint32 hashvalue)
{
- HASH_SEQ_STATUS status;
- RI_ConstraintInfo *hentry;
+ RI_ConstraintInfo *hentry;
+ RI_ConstraintInfo *hnext;
+ bool found;
Assert(ri_constraint_cache != NULL);
- hash_seq_init(&status, ri_constraint_cache);
- while ((hentry = (RI_ConstraintInfo *) hash_seq_search(&status)) != NULL)
+ if (hashvalue == 0)
{
- if (hentry->valid &&
- (hashvalue == 0 || hentry->oidHashValue == hashvalue))
+ /*
+ * Set all valid entries in the cache to invalid and clear the
+ * list of valid entries.
+ */
+ hnext = ri_constraint_cache_valid_list;
+ while (hnext)
+ {
+ hentry = hnext;
+ hnext = hnext->next_valid;
+
+ Assert(hentry->valid);
+
+ hentry->prev_valid = NULL;
+ hentry->next_valid = NULL;
hentry->valid = false;
+ }
+ ri_constraint_cache_valid_list = NULL;
+ }
+ else
+ {
+ /*
+ * Search for the specified entry and set that one invalid
+ * and remove it from the list of valid entries.
+ */
+ hentry = (RI_ConstraintInfo *) hash_search(ri_constraint_cache,
+ (void *) &hashvalue,
+ HASH_FIND, &found);
+ if (found && hentry->valid)
+ {
+ Assert(hentry->oidHashValue == hashvalue);
+
+ if (hentry->prev_valid != NULL)
+ hentry->prev_valid->next_valid = hentry->next_valid;
+ else
+ ri_constraint_cache_valid_list = hentry->next_valid;
+ if (hentry->next_valid != NULL)
+ hentry->next_valid->prev_valid = hentry->prev_valid;
+ hentry->prev_valid = NULL;
+ hentry->next_valid = NULL;
+ hentry->valid = false;
+ }
}
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
