Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
On Fri, Mar 11, 2011 at 4:46 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Is this an open item for 9.1? Simon fixed it, commit b9075a6d2f9b07a00262a670dd60272904c79dce. Oh, thanks! Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
On Thu, Dec 9, 2010 at 4:55 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote: On 09.12.2010 00:10, Heikki Linnakangas wrote: On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. Looking to commit in next few hours barring objections/suggestions, to both HEAD and 9_0_STABLE, in time for next minor release. Please note that it was Noah Misch that raised this a while ago: http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php On closer look, that's not actually the same issue, sorry for the noise.. Heikki, this one *is* important. Will fix. Thanks for the analysis Noah. Is this an open item for 9.1? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
On 11.03.2011 06:21, Fujii Masao wrote: On Thu, Dec 9, 2010 at 4:55 PM, Simon Riggssi...@2ndquadrant.com wrote: On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote: On 09.12.2010 00:10, Heikki Linnakangas wrote: On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. Looking to commit in next few hours barring objections/suggestions, to both HEAD and 9_0_STABLE, in time for next minor release. Please note that it was Noah Misch that raised this a while ago: http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php On closer look, that's not actually the same issue, sorry for the noise.. Heikki, this one *is* important. Will fix. Thanks for the analysis Noah. Is this an open item for 9.1? Simon fixed it, commit b9075a6d2f9b07a00262a670dd60272904c79dce. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
Heikki pointed out to me that the btree delete record processing does
not respect vacuum_defer_cleanup_age. It should.
Attached patch to implement that.
Looking to commit in next few hours barring objections/suggestions, to
both HEAD and 9_0_STABLE, in time for next minor release.
--
Simon Riggs http://www.2ndQuadrant.com/books/
PostgreSQL Development, 24x7 Support, Training and Services
diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c
index 0822f5c..5b2d58a 100644
--- a/src/backend/access/nbtree/nbtxlog.c
+++ b/src/backend/access/nbtree/nbtxlog.c
@@ -683,6 +683,12 @@ btree_xlog_delete_get_latestRemovedXid(XLogRecord *record)
UnlockReleaseBuffer(ibuffer);
/*
+ * Apply vacuum_defer_cleanup_age, if we have a valid xid.
+ */
+ if (TransactionIdIsValid(latestRemovedXid))
+ TransactionIdRetreatMany(latestRemovedXid, vacuum_defer_cleanup_age);
+
+ /*
* Note that if all heap tuples were LP_DEAD then we will be returning
* InvalidTransactionId here. That can happen if we are re-replaying this
* record type, though that will be before the consistency point and will
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 6e7a6db..c16a287 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -1129,8 +1129,7 @@ GetOldestXmin(bool allDbs, bool ignoreVacuum)
LWLockRelease(ProcArrayLock);
/*
- * Compute the cutoff XID, being careful not to generate a permanent
- * XID.
+ * Compute the cutoff XID, being careful not to generate a reserved XID.
*
* vacuum_defer_cleanup_age provides some additional slop for the
* benefit of hot standby queries on slave servers. This is quick and
@@ -1140,9 +1139,7 @@ GetOldestXmin(bool allDbs, bool ignoreVacuum)
* wraparound --- so guc.c should limit it to no more than the
* xidStopLimit threshold in varsup.c.
*/
- result -= vacuum_defer_cleanup_age;
- if (!TransactionIdIsNormal(result))
- result = FirstNormalTransactionId;
+ TransactionIdRetreatMany(result, vacuum_defer_cleanup_age);
return result;
}
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index a7ae752..2f7070e 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -58,6 +58,19 @@
(dest)--; \
} while ((dest) FirstNormalTransactionId)
+#define TransactionIdRetreatMany(dest, many) \
+{ \
+ if ((dest) = (many)) \
+ { \
+ (dest) -= (many); \
+ while ((dest) FirstNormalTransactionId) \
+ { \
+ (dest)--; \
+ } \
+ } \
+ else \
+ (dest) = MaxTransactionId - (many) + (dest); \
+}
/* --
* Object ID (OID) zero is InvalidOid.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. Looking to commit in next few hours barring objections/suggestions, to both HEAD and 9_0_STABLE, in time for next minor release. Please note that it was Noah Misch that raised this a while ago: http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
On 09.12.2010 00:10, Heikki Linnakangas wrote: On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. Looking to commit in next few hours barring objections/suggestions, to both HEAD and 9_0_STABLE, in time for next minor release. Please note that it was Noah Misch that raised this a while ago: http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php On closer look, that's not actually the same issue, sorry for the noise.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. This doesn't look right to me. btree_xlog_delete_get_latestRemovedXid() function calculates the latest XID present on the tuples that we're removing b-tree pointers for. btree_xlog_delete_get_latestRemovedXid() is used during recovery. vacuum_defer_cleanup_age should take effect in the master, not during recovery. With the patch, btree_xlog_delete_get_latestRemovedXid() returns a value that's much smaller than it should. That's just wrong, it means that recovery in the standby will incorrectly think that all the removed tuples are old and not visible to any running read-only queries anymore, and will go ahead and remove the index tuples for them. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
On Thu, 2010-12-09 at 00:39 +0100, Heikki Linnakangas wrote: vacuum_defer_cleanup_age should take effect in the master, not during recovery. Hmmm, more to the point, it does take effect on the master and so there is no need for this at all. What were you thinking? What was I? Doh. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age
On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote: On 09.12.2010 00:10, Heikki Linnakangas wrote: On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. Looking to commit in next few hours barring objections/suggestions, to both HEAD and 9_0_STABLE, in time for next minor release. Please note that it was Noah Misch that raised this a while ago: http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php On closer look, that's not actually the same issue, sorry for the noise.. Heikki, this one *is* important. Will fix. Thanks for the analysis Noah. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
