Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Fujii Masao
On Mon, Dec 26, 2011 at 5:08 AM, Alexander Björnhagen
alex.bjornha...@gmail.com wrote:
 I’m new here so maybe someone else already has this in the works ?

No, as far as I know.

 And so on ... any comments are welcome :)

Basically I like this whole idea, but I'd like to know why do you
think this functionality is required?

When is the replication mode switched from standalone to sync?
That happens as soon as
sync standby appears? or it has caught up with the master? The former
might block the
transactions for a long time until the standby has caught up with the
master even though
synchronous_standalone_master is enabled and a user wants to avoid
such a downtime.

When standalone master is enabled, you might lose some committed
transactions at failover
as follows:

1. While synchronous replication is running normally, replication
connection is closed because of
network outage.
2. The master works standalone because of
synchronous_standalone_master=on and some
new transactions are committed though their WAL records are not
replicated to the standby.
3. The master crashes for some reasons, the clusterware detects it and
triggers a failover.
4. The standby which doesn't have recent committed transactions
becomes the master at a failover...

Is this scenario acceptable?

To avoid such a loss of transactions, I'm thinking to introduce new
GUC parameter specifying
the shell command which is executed when replication mode is switched
from sync to standalone.
If we set it to something like STONITH command, we can shut down
forcibly the standby before
the master resumes the transactions, and avoid the failover to the
obsolete standby when the
master crashes.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Alexander Björnhagen
Hello and thank you for your feedback I appreciate it.

Updated patch : sync-standalone-v2.patch

I am sorry to be spamming the list but I just cleaned it up a little
bit, wrote better comments and tried to move most of the logic into
syncrep.c since that's where it belongs anyway and also fixed a small
bug where standalone mode was disabled/enabled runtime via SIGHUP.

 Basically I like this whole idea, but I'd like to know why do you think this 
 functionality is required?

How should a synchronous master handle the situation where all
standbys have failed ?

Well, I think this is one of those cases where you could argue either
way. Someone caring more about high availability of the system will
want to let the master continue and just raise an alert to the
operators. Someone looking for an absolute guarantee of data
replication will say otherwise.

I don’t like introducing config variables just for the fun of it, but
I think in this case there is no right and wrong.

Oracle dataguard replication has three different configurable modes
called “performance/availability/protection” which for postgres
corresponds exactly with “async/sync+standalone/sync”.

 When is the replication mode switched from standalone to sync?

Good question. Currently that happens when a standby server has
connected and also been deemed suitable for synchronous commit by the
master ( meaning that its name matches the config variable
synchronous_standby_names ). So in a setup with both synchronous and
asynchronous standbys, the master only considers the synchronous ones
when deciding on standalone mode. The asynchronous standbys are
“useless” to a synchronous master anyway.

 The former might block the transactions for a long time until the standby has 
 caught up with the master even though synchronous_standalone_master is 
 enabled and a user wants to avoid such a downtime.

If we a talking about a network “glitch”, than the standby would take
a few seconds/minutes to catch up (not hours!) which is acceptable if
you ask me.

If we are talking about say a node failure, I suppose the workaround
even on current code is to bring up the new standby first as
asynchronous and then simply switch it to synchronous by editing
synchronous_standby_names on the master. Voila ! :)

So in effect this is a non-issue since there is a possible workaround, agree ?

 1. While synchronous replication is running normally, replication
 connection is closed because of
network outage.
 2. The master works standalone because of
 synchronous_standalone_master=on and some
new transactions are committed though their WAL records are not
 replicated to the standby.
 3. The master crashes for some reasons, the clusterware detects it and
 triggers a failover.
 4. The standby which doesn't have recent committed transactions
becomes the master at a failover...

 Is this scenario acceptable?

So you have two separate failures in less time than an admin would
have time to react and manually bring up a new standby.

I’d argue that your system in not designed to be redundant enough if
that kind of scenario worries you. But the point where it all goes
wrong is where the ”clusterware” decides to fail over automatically.
In that kind of setup synchronous_standalone_master must likely be off
but again if the “clusterware” is not smart enough to take the right
decision then it should not act at all. Better to just log critical
alerts, send sms to people etc.

Makes sense ? :)

Cheers,

/A


sync-standalone-v2.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Magnus Hagander
On Mon, Dec 26, 2011 at 13:51, Alexander Björnhagen
alex.bjornha...@gmail.com wrote:
 Hello and thank you for your feedback I appreciate it.

 Updated patch : sync-standalone-v2.patch

 I am sorry to be spamming the list but I just cleaned it up a little
 bit, wrote better comments and tried to move most of the logic into
 syncrep.c since that's where it belongs anyway and also fixed a small
 bug where standalone mode was disabled/enabled runtime via SIGHUP.

It's not spam when it's an updated patch ;)


 Basically I like this whole idea, but I'd like to know why do you think this 
 functionality is required?

 How should a synchronous master handle the situation where all
 standbys have failed ?

 Well, I think this is one of those cases where you could argue either
 way. Someone caring more about high availability of the system will
 want to let the master continue and just raise an alert to the
 operators. Someone looking for an absolute guarantee of data
 replication will say otherwise.

If you don't care about the absolute guarantee of data, why not just
use async replication? It's still going to replicate the data over to
the client as quickly as it can - which in the end is the same level
of guarantee that you get with this switch set, isn't it?


 When is the replication mode switched from standalone to sync?

 Good question. Currently that happens when a standby server has
 connected and also been deemed suitable for synchronous commit by the
 master ( meaning that its name matches the config variable
 synchronous_standby_names ). So in a setup with both synchronous and
 asynchronous standbys, the master only considers the synchronous ones
 when deciding on standalone mode. The asynchronous standbys are
 “useless” to a synchronous master anyway.

But wouldn't an async standby still be a lot better than no standby at
all (standalone)?


 The former might block the transactions for a long time until the standby 
 has caught up with the master even though synchronous_standalone_master is 
 enabled and a user wants to avoid such a downtime.

 If we a talking about a network “glitch”, than the standby would take
 a few seconds/minutes to catch up (not hours!) which is acceptable if
 you ask me.

So it's not Ok to block the master when the standby goes away, but it
is ok to block it when it comes back and catches up? The goes away
might be the same amount of time - or even shorter, depending on
exactly how the network works..


 1. While synchronous replication is running normally, replication
 connection is closed because of
    network outage.
 2. The master works standalone because of
 synchronous_standalone_master=on and some
    new transactions are committed though their WAL records are not
 replicated to the standby.
 3. The master crashes for some reasons, the clusterware detects it and
 triggers a failover.
 4. The standby which doesn't have recent committed transactions
 becomes the master at a failover...

 Is this scenario acceptable?

 So you have two separate failures in less time than an admin would
 have time to react and manually bring up a new standby.

Given that one is a network failure, and one is a node failure, I
don't see that being strange at all. For example, a HA network
environment might cause a short glitch when it's failing over to a
redundant node - enough to bring down the replication connection and
require it to reconnect (during which the master would be ahead of the
slave).

In fact, both might well be network failures - one just making the
master completely inaccessble, and thus triggering the need for a
failover.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Review: Non-inheritable check constraints

2011-12-26 Thread Nikhil Sontakke
  I don't think this is a given ...  In fact, IMO if we're only two or
  three fixes away from having it all nice and consistent, I think
  reverting is not necessary.

 Sure.  It's the if part of that sentence that I'm not too sure about.


Any specific area of the code that you think is/has become fragile (apart
from the non-support for CREATE TABLE based ONLY constraints)? The second
bug is a variant of the first. And I have provided a patch for it.

Regards,
Nikhils


Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Alexander Björnhagen
Interesting discussion!

 Basically I like this whole idea, but I'd like to know why do you think 
 this functionality is required?

 How should a synchronous master handle the situation where all
 standbys have failed ?

 Well, I think this is one of those cases where you could argue either
 way. Someone caring more about high availability of the system will
 want to let the master continue and just raise an alert to the
 operators. Someone looking for an absolute guarantee of data
 replication will say otherwise.

If you don't care about the absolute guarantee of data, why not just
use async replication? It's still going to replicate the data over to
the client as quickly as it can - which in the end is the same level
of guarantee that you get with this switch set, isn't it?

This setup does still guarantee that if the master fails, then you can
still fail over to the standby without any possible data loss because
all data is synchronously replicated.

I want to replicate data with synchronous guarantee to a disaster site
*when possible*. If there is any chance that commits can be
replicated, then I’d like to wait for that.

If however the disaster node/site/link just plain fails and
replication goes down for an *indefinite* amount of time, then I want
the primary node to continue operating, raise an alert and deal with
that. Rather than have the whole system grind to a halt just because a
standby node failed.

It’s not so much that I don’t “care” about replication guarantee, then
I’d just use asynchronous and be done with it. My point is that it is
not always black and white and for some system setups you have to
balance a few things against each other.

If we were just talking about network glitches then I would be fine
with the current behavior because I do not believe they are
long-lasting anyway and they are also *quantifiable* which is a huge
bonus.

My primary focus is system availability but I also care about all that
other stuff too.

I want to have the cake and eat it at the same time as we say in Sweden ;)

 When is the replication mode switched from standalone to sync?

 Good question. Currently that happens when a standby server has
 connected and also been deemed suitable for synchronous commit by the
 master ( meaning that its name matches the config variable
 synchronous_standby_names ). So in a setup with both synchronous and
 asynchronous standbys, the master only considers the synchronous ones
 when deciding on standalone mode. The asynchronous standbys are
 “useless” to a synchronous master anyway.

But wouldn't an async standby still be a lot better than no standby at
all (standalone)?

As soon as the standby comes back online, I want to wait for it to sync.

 The former might block the transactions for a long time until the standby 
 has caught up with the master even though synchronous_standalone_master is 
 enabled and a user wants to avoid such a downtime.

 If we a talking about a network “glitch”, than the standby would take
 a few seconds/minutes to catch up (not hours!) which is acceptable if
 you ask me.

So it's not Ok to block the master when the standby goes away, but it
is ok to block it when it comes back and catches up? The goes away
might be the same amount of time - or even shorter, depending on
exactly how the network works..

To be honest I don’t have a very strong opinion here, we could go
either way, I just wanted to keep this patch as small as possible to
begin with. But again network glitches aren’t my primary concern in a
HA system because the amount of data that the standby lags behind is
possible to estimate and plan for.

Typically switch convergence takes in the order of 15-30 seconds and I
can thus typically assume that the restarted standby can recover that
gap in less than a minute. So once upon a blue moon when something
like that happens, commits would take up to say 1 minute longer. No
big deal IMHO.

 1. While synchronous replication is running normally, replication
 connection is closed because of
network outage.
 2. The master works standalone because of
 synchronous_standalone_master=on and some
new transactions are committed though their WAL records are not
 replicated to the standby.
 3. The master crashes for some reasons, the clusterware detects it and
 triggers a failover.
 4. The standby which doesn't have recent committed transactions
 becomes the master at a failover...

 Is this scenario acceptable?

 So you have two separate failures in less time than an admin would
 have time to react and manually bring up a new standby.

Given that one is a network failure, and one is a node failure, I
don't see that being strange at all. For example, a HA network
environment might cause a short glitch when it's failing over to a
redundant node - enough to bring down the replication connection and
require it to reconnect (during which the master would be ahead of the
slave).

In fact, both might well be network failures - one just making the

Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes:
 If you don't care about the absolute guarantee of data, why not just
 use async replication? It's still going to replicate the data over to
 the client as quickly as it can - which in the end is the same level
 of guarantee that you get with this switch set, isn't it?

Isn't that equivalent to setting synchronous_standby_names to '' and
reloading the server?

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Magnus Hagander
On Mon, Dec 26, 2011 at 15:59, Alexander Björnhagen
alex.bjornha...@gmail.com wrote:
 Basically I like this whole idea, but I'd like to know why do you think 
 this functionality is required?

 How should a synchronous master handle the situation where all
 standbys have failed ?

 Well, I think this is one of those cases where you could argue either
 way. Someone caring more about high availability of the system will
 want to let the master continue and just raise an alert to the
 operators. Someone looking for an absolute guarantee of data
 replication will say otherwise.

If you don't care about the absolute guarantee of data, why not just
use async replication? It's still going to replicate the data over to
the client as quickly as it can - which in the end is the same level
of guarantee that you get with this switch set, isn't it?

 This setup does still guarantee that if the master fails, then you can
 still fail over to the standby without any possible data loss because
 all data is synchronously replicated.

Only if you didn't have a network hitch, or if your slave was down.

Which basically means it doesn't *guarantee* it.



 I want to replicate data with synchronous guarantee to a disaster site
 *when possible*. If there is any chance that commits can be
 replicated, then I’d like to wait for that.

There's always a chance, it's just about how long you're willing to wait ;)

Another thought could be to have something like a sync_wait_timeout,
saying i'm willing to wait n seconds for the syncrep to be caught
up. If nobody is cauth up within that time,then I can back down to
async mode/standalone mode. That way, data availaibility wouldn't
be affected by short-time network glitches.

 If however the disaster node/site/link just plain fails and
 replication goes down for an *indefinite* amount of time, then I want
 the primary node to continue operating, raise an alert and deal with
 that. Rather than have the whole system grind to a halt just because a
 standby node failed.

If the standby node failed and can be determined to actually be failed
(by say a cluster manager), you can always have your cluster software
(or DBA, of course) turn it off by editing the config setting and
reloading. Doing it that way you can actually *verify* that the site
is gone for an indefinite amount of time.


 It’s not so much that I don’t “care” about replication guarantee, then
 I’d just use asynchronous and be done with it. My point is that it is
 not always black and white and for some system setups you have to
 balance a few things against each other.

Agreed in principle :-)


 If we were just talking about network glitches then I would be fine
 with the current behavior because I do not believe they are
 long-lasting anyway and they are also *quantifiable* which is a huge
 bonus.

But the proposed switches doesn't actually make it possible to
differentiate between these non-long-lasting issues and long-lasting
ones, does it? We might want an interface that actually does...


 My primary focus is system availability but I also care about all that
 other stuff too.

 I want to have the cake and eat it at the same time as we say in Sweden ;)

Of course - we all do :D


 When is the replication mode switched from standalone to sync?

 Good question. Currently that happens when a standby server has
 connected and also been deemed suitable for synchronous commit by the
 master ( meaning that its name matches the config variable
 synchronous_standby_names ). So in a setup with both synchronous and
 asynchronous standbys, the master only considers the synchronous ones
 when deciding on standalone mode. The asynchronous standbys are
 “useless” to a synchronous master anyway.

But wouldn't an async standby still be a lot better than no standby at
all (standalone)?

 As soon as the standby comes back online, I want to wait for it to sync.

I guess I just find this very inconsistent. You're willing to wait,
but only sometimes. You're not willing to wait when it goes down, but
you are willing to wait when it comes back. I don't see why this
should be different, and I don't see how you can reliably
differentiate between these two.


 The former might block the transactions for a long time until the standby 
 has caught up with the master even though synchronous_standalone_master is 
 enabled and a user wants to avoid such a downtime.

 If we a talking about a network “glitch”, than the standby would take
 a few seconds/minutes to catch up (not hours!) which is acceptable if
 you ask me.

So it's not Ok to block the master when the standby goes away, but it
is ok to block it when it comes back and catches up? The goes away
might be the same amount of time - or even shorter, depending on
exactly how the network works..

 To be honest I don’t have a very strong opinion here, we could go
 either way, I just wanted to keep this patch as small as possible to
 begin with. But again network glitches aren’t my primary concern in a
 

Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Guillaume Lelarge
On Mon, 2011-12-26 at 16:23 +0100, Magnus Hagander wrote:
 On Mon, Dec 26, 2011 at 15:59, Alexander Björnhagen
 alex.bjornha...@gmail.com wrote:
  Basically I like this whole idea, but I'd like to know why do you think 
  this functionality is required?
 
  How should a synchronous master handle the situation where all
  standbys have failed ?
 
  Well, I think this is one of those cases where you could argue either
  way. Someone caring more about high availability of the system will
  want to let the master continue and just raise an alert to the
  operators. Someone looking for an absolute guarantee of data
  replication will say otherwise.
 
 If you don't care about the absolute guarantee of data, why not just
 use async replication? It's still going to replicate the data over to
 the client as quickly as it can - which in the end is the same level
 of guarantee that you get with this switch set, isn't it?
 
  This setup does still guarantee that if the master fails, then you can
  still fail over to the standby without any possible data loss because
  all data is synchronously replicated.
 
 Only if you didn't have a network hitch, or if your slave was down.
 
 Which basically means it doesn't *guarantee* it.
 

It doesn't guarantee it, but it increases the master availability.
That's the kind of customization some users would like to have. Though I
find it weird to introduce another GUC there. Why not add a new enum
value to synchronous_commit, such as local_only_if_slaves_unavailable
(yeah, the enum value is completely stupid, but you get my point).


-- 
Guillaume
  http://blog.guillaume.lelarge.info
  http://www.dalibo.com
  PostgreSQL Sessions #3: http://www.postgresql-sessions.org


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Alexander Björnhagen
Hmm,

I suppose this conversation would lend itself better to a whiteboard
or a maybe over a few beers instead of via e-mail  ...

 Basically I like this whole idea, but I'd like to know why do you think 
 this functionality is required?

 How should a synchronous master handle the situation where all
 standbys have failed ?

 Well, I think this is one of those cases where you could argue either
 way. Someone caring more about high availability of the system will
 want to let the master continue and just raise an alert to the
 operators. Someone looking for an absolute guarantee of data
 replication will say otherwise.

If you don't care about the absolute guarantee of data, why not just
use async replication? It's still going to replicate the data over to
the client as quickly as it can - which in the end is the same level
of guarantee that you get with this switch set, isn't it?

 This setup does still guarantee that if the master fails, then you can
 still fail over to the standby without any possible data loss because
 all data is synchronously replicated.

Only if you didn't have a network hitch, or if your slave was down.

Which basically means it doesn't *guarantee* it.

True. In my two-node system, I’m willing to take that risk when my
only standby has failed.

Most likely (compared to any other scenario), we can re-gain
redundancy before another failure occurs.

Say each one of your nodes can fail once a year. Most people have much
better track record than with their production machines/network/etc
but just as an example. Then on any given day there is a 0,27% chance
that at given node will fail (1/365*100=0,27), right ?

Then the probability of both failing on the same day is (0,27%)^2 =
0,74 % or about 1 in 13500. And given that it would take only a
few hours tops to restore redundancy, it is even less of a chance than
that because you would not be exposed for the entire day.

So, to be a bit blunt about it and I hope I don’t come off a rude
here, this dual-failure or creeping-doom type scenario on a two-node
system is probably not relevant but more an academical question.

 I want to replicate data with synchronous guarantee to a disaster site
 *when possible*. If there is any chance that commits can be
 replicated, then I’d like to wait for that.

There's always a chance, it's just about how long you're willing to wait ;)

Yes, exactly. When I can estimate it I’m willing to wait.

Another thought could be to have something like a sync_wait_timeout,
saying i'm willing to wait n seconds for the syncrep to be caught
up. If nobody is cauth up within that time,then I can back down to
async mode/standalone mode. That way, data availaibility wouldn't
be affected by short-time network glitches.

This was also mentioned in the previous thread I linked to,
“replication_timeout“ :

http://archives.postgresql.org/pgsql-hackers/2010-10/msg01009.php

In a HA environment you have redundant networking and bonded
interfaces on each node. The only “glitch” would really be if a switch
failed over and that’s a pretty big “if” right there.

 If however the disaster node/site/link just plain fails and
 replication goes down for an *indefinite* amount of time, then I want
 the primary node to continue operating, raise an alert and deal with
 that. Rather than have the whole system grind to a halt just because a
 standby node failed.

If the standby node failed and can be determined to actually be failed
(by say a cluster manager), you can always have your cluster software
(or DBA, of course) turn it off by editing the config setting and
reloading. Doing it that way you can actually *verify* that the site
is gone for an indefinite amount of time.

The system might as well do this for me when the standby gets
disconnected instead of halting the master.

 If we were just talking about network glitches then I would be fine
 with the current behavior because I do not believe they are
 long-lasting anyway and they are also *quantifiable* which is a huge
 bonus.

But the proposed switches doesn't actually make it possible to
differentiate between these non-long-lasting issues and long-lasting
ones, does it? We might want an interface that actually does...

“replication_timeout” where the primary disconnects the WAL sender
after a timeout together with “synchronous_standalone_master” which
tells the primary it can continue anyway when that happens allows
exactly that. This would then be first part towards that but I wanted
to start out small and I personally think it is sufficient to draw the
line at TCP disconnect of the standby.

 When is the replication mode switched from standalone to sync?

 Good question. Currently that happens when a standby server has
 connected and also been deemed suitable for synchronous commit by the
 master ( meaning that its name matches the config variable
 synchronous_standby_names ). So in a setup with both synchronous and
 asynchronous standbys, the master only considers the 

Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Alexander Björnhagen
On Mon, Dec 26, 2011 at 5:18 PM, Guillaume Lelarge
guilla...@lelarge.info wrote:
 On Mon, 2011-12-26 at 16:23 +0100, Magnus Hagander wrote:
 On Mon, Dec 26, 2011 at 15:59, Alexander Björnhagen
 alex.bjornha...@gmail.com wrote:
  Basically I like this whole idea, but I'd like to know why do you think 
  this functionality is required?
 
  How should a synchronous master handle the situation where all
  standbys have failed ?
 
  Well, I think this is one of those cases where you could argue either
  way. Someone caring more about high availability of the system will
  want to let the master continue and just raise an alert to the
  operators. Someone looking for an absolute guarantee of data
  replication will say otherwise.
 
 If you don't care about the absolute guarantee of data, why not just
 use async replication? It's still going to replicate the data over to
 the client as quickly as it can - which in the end is the same level
 of guarantee that you get with this switch set, isn't it?
 
  This setup does still guarantee that if the master fails, then you can
  still fail over to the standby without any possible data loss because
  all data is synchronously replicated.

 Only if you didn't have a network hitch, or if your slave was down.

 Which basically means it doesn't *guarantee* it.


 It doesn't guarantee it, but it increases the master availability.

Yes exactly.

 That's the kind of customization some users would like to have. Though I
 find it weird to introduce another GUC there. Why not add a new enum
 value to synchronous_commit, such as local_only_if_slaves_unavailable
 (yeah, the enum value is completely stupid, but you get my point).

You are right an enum makes much more sense, and the patch would be
much smaller as well so I’ll rework that bit.

/A

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Standalone synchronous master

2011-12-26 Thread Magnus Hagander
On Mon, Dec 26, 2011 at 18:01, Alexander Björnhagen
alex.bjornha...@gmail.com wrote:
 Hmm,

 I suppose this conversation would lend itself better to a whiteboard
 or a maybe over a few beers instead of via e-mail  ...

mmm. beer... :-)


 Well, I think this is one of those cases where you could argue either
 way. Someone caring more about high availability of the system will
 want to let the master continue and just raise an alert to the
 operators. Someone looking for an absolute guarantee of data
 replication will say otherwise.

If you don't care about the absolute guarantee of data, why not just
use async replication? It's still going to replicate the data over to
the client as quickly as it can - which in the end is the same level
of guarantee that you get with this switch set, isn't it?

 This setup does still guarantee that if the master fails, then you can
 still fail over to the standby without any possible data loss because
 all data is synchronously replicated.

Only if you didn't have a network hitch, or if your slave was down.

Which basically means it doesn't *guarantee* it.

 True. In my two-node system, I’m willing to take that risk when my
 only standby has failed.

 Most likely (compared to any other scenario), we can re-gain
 redundancy before another failure occurs.

 Say each one of your nodes can fail once a year. Most people have much
 better track record than with their production machines/network/etc
 but just as an example. Then on any given day there is a 0,27% chance
 that at given node will fail (1/365*100=0,27), right ?

 Then the probability of both failing on the same day is (0,27%)^2 =
 0,74 % or about 1 in 13500. And given that it would take only a
 few hours tops to restore redundancy, it is even less of a chance than
 that because you would not be exposed for the entire day.

That is assuming the failures are actually independent. In my
experience, they're usually not.

But that's diverging into math, which really isn't my strong side here :D


 So, to be a bit blunt about it and I hope I don’t come off a rude
 here, this dual-failure or creeping-doom type scenario on a two-node
 system is probably not relevant but more an academical question.

Given how many times I've seen it, I'm going to respectfully disagree
with that ;)

That said, I agree it's not necessarily reasonable to try to defend
against that in a two node cluster. You can always make it three-node
if you need to do that. I'm worried that the interface seems a bit
fragile and that it's hard to be sure. Predictable interfaces are
good.. :-)



 I want to replicate data with synchronous guarantee to a disaster site
 *when possible*. If there is any chance that commits can be
 replicated, then I’d like to wait for that.

There's always a chance, it's just about how long you're willing to wait ;)

 Yes, exactly. When I can estimate it I’m willing to wait.

Another thought could be to have something like a sync_wait_timeout,
saying i'm willing to wait n seconds for the syncrep to be caught
up. If nobody is cauth up within that time,then I can back down to
async mode/standalone mode. That way, data availaibility wouldn't
be affected by short-time network glitches.

 This was also mentioned in the previous thread I linked to,
 “replication_timeout“ :

 http://archives.postgresql.org/pgsql-hackers/2010-10/msg01009.php

Hmm. That link was gone from the thread when I read it - I missed it
completely. Sorry about that.

So reading that thread, it really only takes care of one of the cases
- the replication_timeout only fires if the slave goes dead. It
could be useful if a similar timeout would apply if I did a pg_ctl
restart on the slave - making the master wait n seconds before
going into standalone mode. The way I read the proposal now, the
master would immediately go into standalone mode if the standby
actually *closes* the connection instead of timing it out?


 In a HA environment you have redundant networking and bonded
 interfaces on each node. The only “glitch” would really be if a switch
 failed over and that’s a pretty big “if” right there.

Switches fail a lot. And there are a lot more things in between that
can fail. I don't think it's such a big if - network issues are by far
the most common cases of a HA environment failing I've seen lately.


 If however the disaster node/site/link just plain fails and
 replication goes down for an *indefinite* amount of time, then I want
 the primary node to continue operating, raise an alert and deal with
 that. Rather than have the whole system grind to a halt just because a
 standby node failed.

If the standby node failed and can be determined to actually be failed
(by say a cluster manager), you can always have your cluster software
(or DBA, of course) turn it off by editing the config setting and
reloading. Doing it that way you can actually *verify* that the site
is gone for an indefinite amount of time.

 The system might as well do this for me when the 

[HACKERS] contrib/README

2011-12-26 Thread Alvaro Herrera

Apparently we forgot to update the README file in contrib/.  I wonder if
it's necessary to explain that within each directory you find one or
more .control file that determines what can be run ... or maybe just
mention the pg_extensions views?

What about this?

diff --git a/contrib/README b/contrib/README
index a1d42a1..8900693 100644
--- a/contrib/README
+++ b/contrib/README
@@ -11,11 +11,11 @@ User documentation for each module appears in the main SGML 
documentation.
 Most items can be built with `gmake all' and installed with
 `gmake install' in the usual fashion, after you have run the `configure'
 script in the top-level directory.  Some directories supply new
-user-defined functions, operators, or types.  In these cases, after you have
+user-defined Postgres extensions.  In these cases, after you have
 installed the files you need to register the new entities in the database
-system by running the commands in the supplied .sql file.  For example,
+system by running an appropriate CREATE EXTENSION command.  For example,
 
-   $ psql -d dbname -f module.sql
+   $ psql -d dbname -c CREATE EXTENSION pg_trgm
 
 See the PostgreSQL documentation for more information about this
 procedure.


-- 
Álvaro Herrera alvhe...@alvh.no-ip.org

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Adding XSLT support to PostgreSQL core?

2011-12-26 Thread Alvaro Herrera

Excerpts from Volker Grabsch's message of mar dic 06 06:34:37 -0300 2011:
 Dear PostgreSQL hackers,

 
 While all xpath_*() functions seem to have been successfully
 collapsed into a generic xpath() function, and xml_is_well_formed()
 has been moved into the type check for the XML type, I wonder
 what happened to the XSLT support.

Nothing happened because there wasn't manpower enough to do the job,
IIRC.  I remember there being discussions about the exact API we need to
provide, but that also requires manpower to research.

(XPath wasn't all that successfully collapsed; I think there have been
some complaints about the exact API we provide, but that, again,
requires more research.  And I might be misremembering.)

 Do you need some volunteer to put work into this?

Absolutely.  Feel free to study and submit a proposal.  Expect this to
take 10x more time than you initially think, just in discussions.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] contrib/README

2011-12-26 Thread Tom Lane
Alvaro Herrera alvhe...@alvh.no-ip.org writes:
 Apparently we forgot to update the README file in contrib/.

I wonder whether it's time to drop that file altogether ... it served a
purpose back before we integrated contrib into the SGML docs, but now
I'm not quite sure why we should bother with it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers