Re: [389-devel] Thought excersize: A different take on replication

2010-12-03 Thread Soeren Malchow (MCon)
Hi Gerrard,

Since you are already mentioning OSPF, how about grouping the Masterservers 
into zones and have only one of the hosts taking care of the communication to 
other zones.

Changes could always only be replicated via the communication master.

The main problem that arises here is the potential outage of one of those 
communication masters, this could be solved by either an election process or 
by an explicit order.

I think this would be a good way to go because it is very close to real life 
scenarios form my point of view, I most likely have a view masters in 
geographically distributed locations, but not tens of servers in one location ( 
maybe already counting e.g. different buildings on a campus as locations )

Not sure if that is a way to go, but maybe it helps a little

Best Regards,

Soeren Malchow
CIO
Phone: +49 40 806008 120
Mobile: +49 171 5525102
Email: soeren.malc...@mcon.net
Web: www.mcon.net
MCon Germany GmbH
Neuer Pferdemarkt 1
20359 Hamburg
Germany
Berlin . Hamburg . Karlsruhe . Koeln . Muenchen
Traderegister HRB 110600 Local Court Mannheim | VAT-Id. No.: DE235236333 | Tax 
No.: 35 007 055033
Managing Partner: Dirk Meissner, Guenther Kreuzpaintner, Soeren Malchow

Member of MCon Group
Austria . China . Czech Republic . France . Germany . Japan . Malaysia . Russia 
. South Korea . Switzerland . UK . USA

This e-mail may contain confidential and/ or privileged information. If you are 
not the intended recipient (or have received 
this e-mail in error) please notify the sender immediately and destroy this 
e-mail. Any unauthorized copying, disclosure 
or distribution of the material contained in this e-mail is strictly prohibited.
Please consider the environment before printing this e-mail


-Original Message-
From: 389-devel-boun...@lists.fedoraproject.org 
[mailto:389-devel-boun...@lists.fedoraproject.org] On Behalf Of Gerrard 
Geldenhuis
Sent: Friday, December 03, 2010 5:25 PM
To: '389-de...@lists.fedoraproject.org'
Subject: [389-devel] Thought excersize: A different take on replication

Hi
We were discussing different setups of replication agreements (multi master) 
between a large number of hosts and ways to minimize contention during updates 
with interconnected hosts. For example the same change might arrive on a host 
from two other hosts via different paths at the same time causing errors in 
the log because of exponential back off. If you have to many connections you 
get a replication storm, to little connections and replication takes to long.

The problem to us sounds very much like a network problem or maybe the 
effectiveness of the underlying database to lock the data more effectively. 

We dreamt up a couple of solutions/ideas and I am writing this email to illicit 
some more discussions and/or comments. One solution would be to change the 
underlying database to one that supports improved granular locking (firebird 
comes to mind ) .

Another idea we discussed was based on the following question:
What if you could only define the list of master servers and let the master 
servers figure out the details with regards to doing multi mastering and 
distributing the data and taking care of broken paths? There is similarities 
with OSPF...

Do you have any thoughts on this? Have you had similar ideas? Are we missing 
the point?

Regards


In order to protect our email recipients, Betfair Group use SkyScan from 
MessageLabs to scan all Incoming and Outgoing mail for viruses.


--
389-devel mailing list
389-de...@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel
--
389-devel mailing list
389-de...@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel


Re: [389-devel] Thought excersize: A different take on replication

2010-12-03 Thread Soeren Malchow (MCon)
OSPF was only the example for me, in OSPF the daemon itself has this election 
logic built in.
So either the service is running and can take part in the election or not, this 
means, the logic needs to be implemented within the ldap daemon.

It also implies, that there is some kind of heartbeat between the nodes.


-Original Message-
From: 389-devel-boun...@lists.fedoraproject.org 
[mailto:389-devel-boun...@lists.fedoraproject.org] On Behalf Of Gerrard 
Geldenhuis
Sent: Friday, December 03, 2010 5:52 PM
To: '389 Directory server developer discussion.'
Subject: Re: [389-devel] Thought excersize: A different take on replication

 -Original Message-
 From: 389-devel-boun...@lists.fedoraproject.org [mailto:389-devel- 
 boun...@lists.fedoraproject.org] On Behalf Of Soeren Malchow (MCon)
 Sent: 03 December 2010 16:42
 To: 389 Directory server developer discussion.
 Subject: Re: [389-devel] Thought excersize: A different take on 
 replication
 
 Hi Gerrard,
 
 Since you are already mentioning OSPF, how about grouping the 
 Masterservers into zones and have only one of the hosts taking care 
 of the communication to other zones.
 
 Changes could always only be replicated via the communication master.
 
 The main problem that arises here is the potential outage of one of 
 those communication masters, this could be solved by either an 
 election process or by an explicit order.

Thanks for the reply Soeren, would this election process be on a OSPF 
basis/level? How do you differentiate between the host not available from a 
network point of view and from a service point of view. My network skills are 
limited and to be honest; OSPF was mentioned during our discussions but I had 
to read up on it afterwards and it seemed similar to the solutions we were 
discussing.

 
 I think this would be a good way to go because it is very close to 
 real life scenarios form my point of view, I most likely have a view 
 masters in geographically distributed locations, but not tens of 
 servers in one location ( maybe already counting e.g. different 
 buildings on a campus as locations )
 
 Not sure if that is a way to go, but maybe it helps a little
 
Regards


In order to protect our email recipients, Betfair Group use SkyScan from 
MessageLabs to scan all Incoming and Outgoing mail for viruses.


--
389-devel mailing list
389-de...@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel
--
389-devel mailing list
389-de...@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-devel