Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Dave Kempe]

Crossfire wrote:
I want to be able to set it up so /home (and maybe other filesystems) 
are replicated from one to the other, in both directions, in real time 
so they can run in an all-hot redundant cluster.



I think I found something that might help you:
http://gluster.org/docs/index.php/GlusterFS

http://gluster.org/docs/index.php/GlusterFS_Translators_v1.3#Automatic_File_Replication_Translator_.28AFR.29

I haven't tried it yet, but it looks good, so I might.
sorry to revive an old thread :)

dave
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Matthew Hannigan]

I don't know whether it would suit you at all, but
I'll mention

http://allmydata.org/trac/tahoe

for the simple reason it looks interesting and
it only just announced version 1.0

RC's mention of Ceph jogged my memory on this.

Matt


-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Matt Moor]

Crossfire wrote:

I've just spent some time quickly researching this to no real satisfaction.

What I'm looking for is a way to do real-time hot-replication of a whole 
filesystem or filesystem tree over 2 nodes (and strictly 2 nodes) 
without STOMITH[1].


The scenario is I have two identical systems with local (software) 
RAID1.  They will be tethered onto their internet feed via ethernet, and 
can optionally be tethered to each other via Gig.


I want to be able to set it up so /home (and maybe other filesystems) 
are replicated from one to the other, in both directions, in real time 
so they can run in an all-hot redundant cluster.


The environment should be mostly read-oriented, so I can live with 
write-latent solutions as long as they handle the race/collision 
gracefully (preferably by actually detecting and reporting it if they 
can't avoid it).




I've had some success with Software iSCSI targets on Linux to date. I'm 
currently using software iSCSI over Gigabit Ethernet to back a VMware 
ESX cluster[0].


Software iSCSI targets (I have experience only with tgtd - the only one 
that seemed current) present a Linux block device as an iSCSI target 
over the network. I present an LVM logical volume.


One could conceive of an eventuality where you made both machines iSCSI 
targets and initiators and ran RAID1 over the local and remote iSCSI 
targets[1]. I have no idea what sort of (terrible) performance you might 
get out of this sort of setup, but it would meet your requirements, and 
with enough RAM for read-caching in each node, it might not be too bad. 
You would need that Gigabit cross-connect.


There are large warnings in the scsi_tgt code about using it in 
production, however.


I suspect this problem space isn't addressed terribly often because, 
well, (1) it's Hard, (2) most people who care about this stuff buy 
shared storage (check ebay), (3) It's even Harder once you start talking 
file systems that do this[2].


Cheers,

Matt

0. I can post my recipe for the target bits, if anyone cares.

1. With a global filesystem, of course.

2. Ceph, which Robert Collins suggested above, is a really good example 
of a brilliantly designed distributed file system (much better than 
MogileFS, which is more an on-disk hash table with quirks), but I have 
my doubts about it's suitability for production systems (though I hope 
it gets there).

--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Robert Collins]

On Sat, 2008-04-05 at 09:52 +1100, Crossfire wrote:
> I've just spent some time quickly researching this to no real satisfaction.
> 
> What I'm looking for is a way to do real-time hot-replication of a whole 
> filesystem or filesystem tree over 2 nodes (and strictly 2 nodes) 
> without STOMITH[1].
> 
> The scenario is I have two identical systems with local (software) 
> RAID1.  They will be tethered onto their internet feed via ethernet, and 
> can optionally be tethered to each other via Gig.
> 
> I want to be able to set it up so /home (and maybe other filesystems) 
> are replicated from one to the other, in both directions, in real time 
> so they can run in an all-hot redundant cluster.
> 
> The environment should be mostly read-oriented, so I can live with 
> write-latent solutions as long as they handle the race/collision 
> gracefully (preferably by actually detecting and reporting it if they 
> can't avoid it).
> 
> The options I've investigated so far:
> 
> * Lustre (MDS requirements[2] make this not an option)
> * GlobalFS (STOMITH requirements make this not an option.  Oriented
>towards shared media too, which I am not using)
> * tsync (Naive concurrent operation model, but otherwise viable)
> * MogileFS (not quite what I was looking for, but none the less useful).
> * OpenAFS (read-only replication only, loss of the node hosting the
>write volume still renders the volume unwritable).
> 
> Is anybody aware of any other options that I've missed?

http://sourceforge.net/projects/ceph/

-Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Chris Collins]

Adrian Chadd wrote:

I looked into it about a year ago and I couldn't find any simple way of
doing this using free software. There's CODA/AFS as possible solutions but
they still push the notion of master/slave rather than equal peers, which
Chris mentions he needs (ie, constant synchronisation between each member
rather than periodic pushback..)

Chris, try looking at CODA/AFS support?


OpenAFS was already considered.  R/O replication is a pain, as is the 
whole volume host death problem. (ie: write volume goes away if the host 
holding the volume dies).


I haven't looked at Coda recently.  They still seem to be active (I 
thought they'd all abandoned ship for Intermezzo - seems I was wrong). 
I'll check it out sometime soon.


C.
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Jamie Wilkinson]

This one time, at band camp, Amos Shapira wrote:
>1. You CAN'T mount a non-cluster-aware file system even read-only on the
>secondary node since the primary will change FS-structs under the feet of
>the read-only node and cause it to crash (because non-cluster-aware
>filesystems assume that they are the only ones who touch that partition).
>2. You CAN mount read-write on multiple nodes if you use one of the
>cluster-aware filesystems (GFS and OCFS are regularly mentioned, but if you
>find any other cluster-aware file system then it sounds like it will work
>too).

You're right, the example I was thinking of does not mount the filesystem on
the secondary nodes until the primary goes down; once the FS is not mounted
one of the secondaries takes over and mounts it read/write.
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Amos Shapira]

On Sun, Apr 6, 2008 at 9:25 PM, Adrian Chadd <[EMAIL PROTECTED]> wrote:

> On Sun, Apr 06, 2008, Amos Shapira wrote:
>
> > I've been using DRBD for a few months now (just in stand-by mode, but
> been
> > following the forums and docs during that time) and all indications are
> > that:
> >
> > 1. You CAN'T mount a non-cluster-aware file system even read-only on the
> > secondary node since the primary will change FS-structs under the feet
> of
> > the read-only node and cause it to crash (because non-cluster-aware
> > filesystems assume that they are the only ones who touch that
> partition).
>
> > 2. You CAN mount read-write on multiple nodes if you use one of the
> > cluster-aware filesystems (GFS and OCFS are regularly mentioned, but if
> you
> > find any other cluster-aware file system then it sounds like it will
> work
> > too).
>
> IIRC they assume a single back-end device. Does DRBD give you a journaling
> block device which will stall updates until they've been pushed? How will
> the FSes tolerate the device IO being possibly milliseconds later than the
> master?


Again - I haven't got around to actually use it (as much as I'd like to just
sit down and try it) but you can see in the link that I sent with my
previous reply that they clearly claim that it is supported.

> Have you tried this suggestions? From all I read about DRBD this will
> cause
> > all secondary nodes to crash.
>
> I looked into it about a year ago and I couldn't find any simple way of


Could it be that you looked at 0.7? I always used 0.8+ and got the
impression that there were major improvements introduced in it over 0.7.

doing this using free software. There's CODA/AFS as possible solutions but
> they still push the notion of master/slave rather than equal peers, which
> Chris mentions he needs (ie, constant synchronisation between each member
> rather than periodic pushback..)


That's what DRBD 0.8+GFS/OCFS is promoted as .

Cheers,

--Amos
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Adrian Chadd]

On Sun, Apr 06, 2008, Amos Shapira wrote:

> I've been using DRBD for a few months now (just in stand-by mode, but been
> following the forums and docs during that time) and all indications are
> that:
> 
> 1. You CAN'T mount a non-cluster-aware file system even read-only on the
> secondary node since the primary will change FS-structs under the feet of
> the read-only node and cause it to crash (because non-cluster-aware
> filesystems assume that they are the only ones who touch that partition).

> 2. You CAN mount read-write on multiple nodes if you use one of the
> cluster-aware filesystems (GFS and OCFS are regularly mentioned, but if you
> find any other cluster-aware file system then it sounds like it will work
> too).

IIRC they assume a single back-end device. Does DRBD give you a journaling
block device which will stall updates until they've been pushed? How will
the FSes tolerate the device IO being possibly milliseconds later than the
master?

> Have you tried this suggestions? From all I read about DRBD this will cause
> all secondary nodes to crash.

I looked into it about a year ago and I couldn't find any simple way of
doing this using free software. There's CODA/AFS as possible solutions but
they still push the notion of master/slave rather than equal peers, which
Chris mentions he needs (ie, constant synchronisation between each member
rather than periodic pushback..)

Chris, try looking at CODA/AFS support?



Adrian

-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Amos Shapira]

On Sun, Apr 6, 2008 at 2:47 PM, Jamie Wilkinson <[EMAIL PROTECTED]> wrote:

> This one time, at band camp, Crossfire wrote:
> > Dave Kempe wrote:
> >> Crossfire wrote:
> >>> I want to be able to set it up so /home (and maybe other filesystems)
> >>> are replicated from one to the other, in both directions, in real
> >>> time so they can run in an all-hot redundant cluster.
> >>>
> >>> The environment should be mostly read-oriented, so I can live with
> >>> write-latent solutions as long as they handle the race/collision
> >>> gracefully (preferably by actually detecting and reporting it if they
> >>> can't avoid it).
> >>>
> >> isn't this just a description of a network filesytem... say NFS?
> >
> > No.  Network Filesystems still have a distinct single storage location.
> > If that storage is taken offline, clients can only error or hang.
> >
> > With a hot real-time replicated filesystem, all involved nodes would
> > have a full local copy at all times and would be able to continue
> > operation.
>
> I agreed with your earlier decision about not using drbd because you
> wouldn't be able to write from multiple nodes to the filesystem; all the
> slaves would have to be mounted read-only.  However if you wanted to get


Can you provide links which support this?

I've been using DRBD for a few months now (just in stand-by mode, but been
following the forums and docs during that time) and all indications are
that:

1. You CAN'T mount a non-cluster-aware file system even read-only on the
secondary node since the primary will change FS-structs under the feet of
the read-only node and cause it to crash (because non-cluster-aware
filesystems assume that they are the only ones who touch that partition).
2. You CAN mount read-write on multiple nodes if you use one of the
cluster-aware filesystems (GFS and OCFS are regularly mentioned, but if you
find any other cluster-aware file system then it sounds like it will work
too).

Ref:
http://www.linux-ha.org/DRBD/FAQ#head-2cad8caa095cfb6e2935261cb595390c742ebd86


> fancy you could still use drbd (which is a great fit for all your other
> requirements) on a multi-node fileserver, and do some nifty failover using
> IP takeover.
>
> Or if you're trying to share the local disk of a lot of nodes, then what
> if
> you used DRBD on them all to replicate the block device, and run a NFS
> server on the nodes thremselves?  Yes you'd get a lot of network traffic
> between them, but it'd work, no? :)


Have you tried this suggestions? From all I read about DRBD this will cause
all secondary nodes to crash.

Cheers,

--Amos
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Jamie Wilkinson]

This one time, at band camp, Crossfire wrote:
> Dave Kempe wrote:
>> Crossfire wrote:
>>> I want to be able to set it up so /home (and maybe other filesystems) 
>>> are replicated from one to the other, in both directions, in real 
>>> time so they can run in an all-hot redundant cluster.
>>>
>>> The environment should be mostly read-oriented, so I can live with  
>>> write-latent solutions as long as they handle the race/collision  
>>> gracefully (preferably by actually detecting and reporting it if they 
>>> can't avoid it).
>>>
>> isn't this just a description of a network filesytem... say NFS?
>
> No.  Network Filesystems still have a distinct single storage location.  
> If that storage is taken offline, clients can only error or hang.
>
> With a hot real-time replicated filesystem, all involved nodes would  
> have a full local copy at all times and would be able to continue 
> operation.

I agreed with your earlier decision about not using drbd because you
wouldn't be able to write from multiple nodes to the filesystem; all the
slaves would have to be mounted read-only.  However if you wanted to get
fancy you could still use drbd (which is a great fit for all your other
requirements) on a multi-node fileserver, and do some nifty failover using
IP takeover.

Or if you're trying to share the local disk of a lot of nodes, then what if
you used DRBD on them all to replicate the block device, and run a NFS
server on the nodes thremselves?  Yes you'd get a lot of network traffic
between them, but it'd work, no? :)
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Alex Samad]

On Sat, Apr 05, 2008 at 10:11:00AM +1100, John Ferlito wrote:
> Keeping in mind I've never done this so no idea how well it works. I'd
> say a combination of
> 
> Global File System - http://sources.redhat.com/cluster/gfs/
I think the requirements where for no STOMITH and GFS uses that in both
incarnations, so does ocfs, psfs



> and
> Global Network Block Device - http://sourceware.org/cluster/gnbd/
> 
> should do the trick, this document explains how
> http://sources.redhat.com/cluster/wiki/DRBD_Cookbook?highlight=(CategoryHowTo)
> 
> On Sat, Apr 05, 2008 at 09:52:55AM +1100, Crossfire wrote:
> > I've just spent some time quickly researching this to no real satisfaction.
> >
> > What I'm looking for is a way to do real-time hot-replication of a whole  
> > filesystem or filesystem tree over 2 nodes (and strictly 2 nodes) without 
> > STOMITH[1].
> >
> > The scenario is I have two identical systems with local (software) RAID1.  
> > They will be tethered onto their internet feed via ethernet, and can 
> > optionally be tethered to each other via Gig.
> >
> > I want to be able to set it up so /home (and maybe other filesystems) are 
> > replicated from one to the other, in both directions, in real time so they 
> > can run in an all-hot redundant cluster.
> >
> > The environment should be mostly read-oriented, so I can live with  
> > write-latent solutions as long as they handle the race/collision  
> > gracefully (preferably by actually detecting and reporting it if they  
> > can't avoid it).
> >
> > The options I've investigated so far:
> >
> > * Lustre (MDS requirements[2] make this not an option)
> > * GlobalFS (STOMITH requirements make this not an option.  Oriented
> >   towards shared media too, which I am not using)
> > * tsync (Naive concurrent operation model, but otherwise viable)
> > * MogileFS (not quite what I was looking for, but none the less useful).
> > * OpenAFS (read-only replication only, loss of the node hosting the
> >   write volume still renders the volume unwritable).
> >
> > Is anybody aware of any other options that I've missed?
> >
> > C.
> >
> > [1] "Shoot The Other Machine In The Head" - the ability for any node to
> > forcibly powerdown any other node believed to be malfunctioning.
> > [2] Single instance MDS only, only clusterable through shared storage.
> > d'oh.
> > [3] People suggesting rsync will be taken out back and shot for not
> > reading the requirements.
> > -- 
> > SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
> > Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
> >
> 
> -- 
> John
> http://www.inodes.org/
> -- 
> SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
> Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
> 

-- 
"It is clear our nation is reliant upon big foreign oil. More and more of our 
imports come from overseas."

- George W. Bush
09/25/2000
Beaverton, OR


signature.asc
Description: Digital signature
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[John Ferlito]

Keeping in mind I've never done this so no idea how well it works. I'd
say a combination of

Global File System - http://sources.redhat.com/cluster/gfs/
and
Global Network Block Device - http://sourceware.org/cluster/gnbd/

should do the trick, this document explains how
http://sources.redhat.com/cluster/wiki/DRBD_Cookbook?highlight=(CategoryHowTo)

On Sat, Apr 05, 2008 at 09:52:55AM +1100, Crossfire wrote:
> I've just spent some time quickly researching this to no real satisfaction.
>
> What I'm looking for is a way to do real-time hot-replication of a whole  
> filesystem or filesystem tree over 2 nodes (and strictly 2 nodes) without 
> STOMITH[1].
>
> The scenario is I have two identical systems with local (software) RAID1.  
> They will be tethered onto their internet feed via ethernet, and can 
> optionally be tethered to each other via Gig.
>
> I want to be able to set it up so /home (and maybe other filesystems) are 
> replicated from one to the other, in both directions, in real time so they 
> can run in an all-hot redundant cluster.
>
> The environment should be mostly read-oriented, so I can live with  
> write-latent solutions as long as they handle the race/collision  
> gracefully (preferably by actually detecting and reporting it if they  
> can't avoid it).
>
> The options I've investigated so far:
>
> * Lustre (MDS requirements[2] make this not an option)
> * GlobalFS (STOMITH requirements make this not an option.  Oriented
>   towards shared media too, which I am not using)
> * tsync (Naive concurrent operation model, but otherwise viable)
> * MogileFS (not quite what I was looking for, but none the less useful).
> * OpenAFS (read-only replication only, loss of the node hosting the
>   write volume still renders the volume unwritable).
>
> Is anybody aware of any other options that I've missed?
>
> C.
>
> [1] "Shoot The Other Machine In The Head" - the ability for any node to
> forcibly powerdown any other node believed to be malfunctioning.
> [2] Single instance MDS only, only clusterable through shared storage.
> d'oh.
> [3] People suggesting rsync will be taken out back and shot for not
> reading the requirements.
> -- 
> SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
> Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
>

-- 
John
http://www.inodes.org/
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Alex Samad]

On Sat, Apr 05, 2008 at 09:52:55AM +1100, Crossfire wrote:
> I've just spent some time quickly researching this to no real satisfaction.
>
> What I'm looking for is a way to do real-time hot-replication of a whole  
> filesystem or filesystem tree over 2 nodes (and strictly 2 nodes)  
> without STOMITH[1].
>
> The scenario is I have two identical systems with local (software)  
> RAID1.  They will be tethered onto their internet feed via ethernet, and  
> can optionally be tethered to each other via Gig.
>
> I want to be able to set it up so /home (and maybe other filesystems)  
> are replicated from one to the other, in both directions, in real time  
> so they can run in an all-hot redundant cluster.
>
> The environment should be mostly read-oriented, so I can live with  
> write-latent solutions as long as they handle the race/collision  
> gracefully (preferably by actually detecting and reporting it if they  
> can't avoid it).
>
> The options I've investigated so far:
>
> * Lustre (MDS requirements[2] make this not an option)
> * GlobalFS (STOMITH requirements make this not an option.  Oriented
>   towards shared media too, which I am not using)
> * tsync (Naive concurrent operation model, but otherwise viable)
> * MogileFS (not quite what I was looking for, but none the less useful).
> * OpenAFS (read-only replication only, loss of the node hosting the
>   write volume still renders the volume unwritable).
>
> Is anybody aware of any other options that I've missed?
I think once you ask for no STOMITH and also read/write access from more
than one location, you have made it really hard for yourself. Unless you
go to something NFS.

Lustre doesn't really met you requirements. its more for file striping
and speed then for HA

if 1 can be readonly then you could setup a raid 1 with one device being
a network block device 
>
> C.
>
> [1] "Shoot The Other Machine In The Head" - the ability for any node to
> forcibly powerdown any other node believed to be malfunctioning.
> [2] Single instance MDS only, only clusterable through shared storage.
> d'oh.
> [3] People suggesting rsync will be taken out back and shot for not
> reading the requirements.
> -- 
> SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
> Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
>

-- 
"I also understand how tender the free enterprise system can be."

- George W. Bush
07/08/2002
White House press conference, Washington, D.C.


signature.asc
Description: Digital signature
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Crossfire]

Dave Kempe wrote:

Crossfire wrote:
I want to be able to set it up so /home (and maybe other filesystems) 
are replicated from one to the other, in both directions, in real time 
so they can run in an all-hot redundant cluster.


The environment should be mostly read-oriented, so I can live with 
write-latent solutions as long as they handle the race/collision 
gracefully (preferably by actually detecting and reporting it if they 
can't avoid it).



isn't this just a description of a network filesytem... say NFS?


No.  Network Filesystems still have a distinct single storage location. 
 If that storage is taken offline, clients can only error or hang.


With a hot real-time replicated filesystem, all involved nodes would 
have a full local copy at all times and would be able to continue operation.


C.
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Dave Kempe]

Dave Kempe wrote:
I was thinking just yesterday some sort of fuse filesystem is what we 
need :)


dave



haven't tried it, but this is fuse
http://www.furquim.org/chironfs/

dave

--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Crossfire]

Mick Pollard wrote:

On Sat, 05 Apr 2008 09:52:55 +1100
Crossfire <[EMAIL PROTECTED]> wrote:


I've just spent some time quickly researching this to no real satisfaction.

What I'm looking for is a way to do real-time hot-replication of a whole 
filesystem or filesystem tree over 2 nodes (and strictly 2 nodes) 
without STOMITH[1].


The scenario is I have two identical systems with local (software) 
RAID1.  They will be tethered onto their internet feed via ethernet, and 
can optionally be tethered to each other via Gig.



Have you had a look at http://www.drbd.org/ ?
It basically mirrors a blockdevice over ethernet. 
A raid1 of sorts.


DRBD doesn't actually solve the problem - it either provides a warm 
replication of a normal filesystem, or provides an pseudo-shared storage 
device.


Warm replication is a no-go since I will need effectively local access 
to the filesystem on both nodes so there isn't a song-and-dance routine 
to perform w.r.t mounts, etc, during a failure.


As a psuedo-shared storage device, I doubt it's of any particular use 
due to (drastically-higher) latency incurred by having to replicate the 
changes between the storage pools rather than the storage pool being 
shared.  I'd also be concerned about split-brain recovery with cluster 
filesystems (split-brain is not actually possible with a real shared 
drive, but completely possible with DRBD). Initial comments I've seen 
from people trying to use it with OCFS2 also seem poor.


C.
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Dave Kempe]

Crossfire wrote:
I want to be able to set it up so /home (and maybe other filesystems) 
are replicated from one to the other, in both directions, in real time 
so they can run in an all-hot redundant cluster.


The environment should be mostly read-oriented, so I can live with 
write-latent solutions as long as they handle the race/collision 
gracefully (preferably by actually detecting and reporting it if they 
can't avoid it).



isn't this just a description of a network filesytem... say NFS?
I am also interested in what you come up with, but haven't seen anything 
that matchs. DRBD is not RW from both nodes.
I have also used RAID1 over AoE and iSCSI, but not sure if this would 
help you at all either with only two nodes.
I was thinking just yesterday some sort of fuse filesystem is what we 
need :)


dave

--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Network Real-Time Hot Filesystem Replication?

[Mick Pollard]

On Sat, 05 Apr 2008 09:52:55 +1100
Crossfire <[EMAIL PROTECTED]> wrote:

> I've just spent some time quickly researching this to no real satisfaction.
> 
> What I'm looking for is a way to do real-time hot-replication of a whole 
> filesystem or filesystem tree over 2 nodes (and strictly 2 nodes) 
> without STOMITH[1].
> 
> The scenario is I have two identical systems with local (software) 
> RAID1.  They will be tethered onto their internet feed via ethernet, and 
> can optionally be tethered to each other via Gig.
> 
Have you had a look at http://www.drbd.org/ ?
It basically mirrors a blockdevice over ethernet. 
A raid1 of sorts.

-- 
Regards
Mick Pollard ( lunix )

BOFH Excuse of the day:
Intermittant Checksum Invalidation Error


pgpPPkdV5wNbV.pgp
Description: PGP signature
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html