On Sat, Jan 17, 2015 at 2:44 AM, Andres Freund <[email protected]> wrote:
> On 2015-01-05 17:16:43 +0900, Michael Paquier wrote:
>> + <varlistentry id="restore-command-retry-interval"
>> xreflabel="restore_command_retry_interval">
>> + <term><varname>restore_command_retry_interval</varname>
>> (<type>integer</type>)
>> + <indexterm>
>> + <primary><varname>restore_command_retry_interval</> recovery
>> parameter</primary>
>> + </indexterm>
>> + </term>
>> + <listitem>
>> + <para>
>> + If <varname>restore_command</> returns nonzero exit status code,
>> retry
>> + command after the interval of time specified by this parameter,
>> + measured in milliseconds if no unit is specified. Default value is
>> + <literal>5s</>.
>> + </para>
>> + <para>
>> + This command is particularly useful for warm standby configurations
>> + to leverage the amount of retries done to restore a WAL segment file
>> + depending on the activity of a master node.
>> + </para>
>
> Don't really like this paragraph. What's "leveraging the amount of
> retries done" supposed to mean?
Actually I have reworked the whole paragraph with the renaming of the
parameter. Hopefully that's more clear.
>> +# restore_command_retry_interval
>> +#
>> +# specifies an optional retry interval for restore_command command, if
>> +# previous command returned nonzero exit status code. This can be useful
>> +# to increase or decrease the number of calls of restore_command.
> It's not really the number but frequency, right?
Sure.
>> + else if (strcmp(item->name, "restore_command_retry_interval")
>> == 0)
>> + {
>> + const char *hintmsg;
>> +
>> + if (!parse_int(item->value,
>> &restore_command_retry_interval, GUC_UNIT_MS,
>> + &hintmsg))
>> + ereport(ERROR,
>> +
>> (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
>> + errmsg("parameter \"%s\"
>> requires a temporal value",
>> +
>> "restore_command_retry_interval"),
>> + hintmsg ? errhint("%s",
>> _(hintmsg)) : 0));
>
> "temporal value" sounds odd to my ears. I'd rather keep in line with
> what guc.c outputs in such a case.
Yeah, I have been pondering about that a bit and I think that
wal_availability_check_interval is the closest thing as we want to
check if WAL is available for a walreceiver at record level and at
segment level for a restore_command.
> Am I missing something or does this handle/affect streaming failures
> just the same? That might not be a bad idea, because the current
> interval is far too long for e.g. a sync replication setup. But it
> certainly makes the name a complete misnomer.
Hm.
> Not this patch's fault, but I'm getting a bit tired seeing the above open
> coded. How about adding a function that does the sleeping based on a
> timestamptz and a ms interval?
You mean in plugins, right? I don't recall seeing similar patterns in
other code paths of backend. But I think that we can use something
like that in timestamp.c then because we need to leverage that between
two timestamps, the last failure and now():
TimestampSleepDifference(start_time, stop_time, internal_ms);
Perhaps you have something else in mind?
Attached is an updated patch.
--
Michael
From 0206c417f5b28eadb5f9d67ee0643320d7c0b621 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Mon, 19 Jan 2015 16:08:48 +0900
Subject: [PATCH] Add wal_availability_check_interval to control WAL fetching
on failure
Default value is 5s.
---
doc/src/sgml/recovery-config.sgml | 21 +++++++++++++
src/backend/access/transam/recovery.conf.sample | 10 +++++++
src/backend/access/transam/xlog.c | 39 ++++++++++++++++++++-----
src/backend/utils/adt/timestamp.c | 19 ++++++++++++
src/include/utils/timestamp.h | 2 ++
5 files changed, 83 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml
index 0c64ff2..7fd51ce 100644
--- a/doc/src/sgml/recovery-config.sgml
+++ b/doc/src/sgml/recovery-config.sgml
@@ -145,6 +145,27 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
</listitem>
</varlistentry>
+ <varlistentry id="wal-availability-check-interval" xreflabel="wal_availability_check_interval">
+ <term><varname>wal_availability_check_interval</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>wal_availability_check_interval</> recovery parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the amount of time to wait when
+ WAL is not available for a node in recovery. Default value is
+ <literal>5s</>.
+ </para>
+ <para>
+ A node in recovery will wait for this amount of time if
+ <varname>restore_command</> returns nonzero exit status code when
+ fetching new WAL segment files from archive or when a WAL receiver
+ is not able to fetch a WAL record when using streaming replication.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/src/backend/access/transam/recovery.conf.sample b/src/backend/access/transam/recovery.conf.sample
index b777400..70d3946 100644
--- a/src/backend/access/transam/recovery.conf.sample
+++ b/src/backend/access/transam/recovery.conf.sample
@@ -58,6 +58,16 @@
#
#recovery_end_command = ''
#
+#
+# wal_availability_check_interval
+#
+# specifies an optional interval to check for availability of WAL when
+# recovering a node. This interval of time represents the frequency of
+# retries if a previous command of restore_command returned nonzero exit
+# status code or if a walreceiver did not stream completely a WAL record.
+#
+#wal_availability_check_interval = '5s'
+#
#---------------------------------------------------------------------------
# RECOVERY TARGET PARAMETERS
#---------------------------------------------------------------------------
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 629a457..0da9f5c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -235,6 +235,7 @@ static TimestampTz recoveryTargetTime;
static char *recoveryTargetName;
static int recovery_min_apply_delay = 0;
static TimestampTz recoveryDelayUntilTime;
+static int wal_availability_check_interval = 5000;
/* options taken from recovery.conf for XLOG streaming */
static bool StandbyModeRequested = false;
@@ -4881,6 +4882,26 @@ readRecoveryCommandFile(void)
(errmsg_internal("trigger_file = '%s'",
TriggerFile)));
}
+ else if (strcmp(item->name, "wal_availability_check_interval") == 0)
+ {
+ const char *hintmsg;
+
+ if (!parse_int(item->value, &wal_availability_check_interval, GUC_UNIT_MS,
+ &hintmsg))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("parameter \"%s\" requires a temporal value",
+ "wal_availability_check_interval"),
+ hintmsg ? errhint("%s", _(hintmsg)) : 0));
+ ereport(DEBUG2,
+ (errmsg_internal("wal_availability_check_interval = '%s'", item->value)));
+
+ if (wal_availability_check_interval <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" must have a value strictly positive",
+ "wal_availability_check_interval")));
+ }
else if (strcmp(item->name, "recovery_min_apply_delay") == 0)
{
const char *hintmsg;
@@ -10340,8 +10361,8 @@ static bool
WaitForWALToBecomeAvailable(XLogRecPtr RecPtr, bool randAccess,
bool fetching_ckpt, XLogRecPtr tliRecPtr)
{
- static pg_time_t last_fail_time = 0;
- pg_time_t now;
+ TimestampTz now = GetCurrentTimestamp();
+ TimestampTz last_fail_time = now;
/*-------
* Standby mode is implemented by a state machine:
@@ -10490,14 +10511,16 @@ WaitForWALToBecomeAvailable(XLogRecPtr RecPtr, bool randAccess,
* machine, so we've exhausted all the options for
* obtaining the requested WAL. We're going to loop back
* and retry from the archive, but if it hasn't been long
- * since last attempt, sleep 5 seconds to avoid
- * busy-waiting.
+ * since last attempt, sleep the amount of time specified
+ * by wal_availability_check_interval to avoid busy-waiting.
*/
- now = (pg_time_t) time(NULL);
- if ((now - last_fail_time) < 5)
+ now = GetCurrentTimestamp();
+ if (!TimestampDifferenceExceeds(last_fail_time, now,
+ wal_availability_check_interval))
{
- pg_usleep(1000000L * (5 - (now - last_fail_time)));
- now = (pg_time_t) time(NULL);
+ TimestampSleepDifference(last_fail_time, now,
+ wal_availability_check_interval);
+ now = GetCurrentTimestamp();
}
last_fail_time = now;
currentSource = XLOG_FROM_ARCHIVE;
diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c
index 67e0cf9..1098fc9 100644
--- a/src/backend/utils/adt/timestamp.c
+++ b/src/backend/utils/adt/timestamp.c
@@ -1674,6 +1674,25 @@ TimestampDifferenceExceeds(TimestampTz start_time,
}
/*
+ * TimestampSleepDifference -- sleep for the amout of interval time
+ * specified, reduced by the difference between two timestamps.
+ *
+ * Both inputs must be ordinary finite timestamps (in current usage,
+ * they'll be results from GetCurrentTimestamp()).
+ */
+void
+TimestampSleepDifference(TimestampTz start_time,
+ TimestampTz stop_time,
+ int interval_msec)
+{
+ long secs;
+ int microsecs;
+
+ TimestampDifference(start_time, stop_time, &secs, µsecs);
+ pg_usleep(interval_msec * 1000L - (1000000L * secs + 1L * microsecs));
+}
+
+/*
* Convert a time_t to TimestampTz.
*
* We do not use time_t internally in Postgres, but this is provided for use
diff --git a/src/include/utils/timestamp.h b/src/include/utils/timestamp.h
index 70118f5..e271d0b 100644
--- a/src/include/utils/timestamp.h
+++ b/src/include/utils/timestamp.h
@@ -217,6 +217,8 @@ extern void TimestampDifference(TimestampTz start_time, TimestampTz stop_time,
extern bool TimestampDifferenceExceeds(TimestampTz start_time,
TimestampTz stop_time,
int msec);
+extern void TimestampSleepDifference(TimestampTz start_time,
+ TimestampTz stop_time, int interval_msec);
/*
* Prototypes for functions to deal with integer timestamps, when the native
--
2.2.2
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
