Re: NetBSD disk backup over network
Ashley Moran wrote: I just saw this slashdotted article: http://ezine.daemonnews.org/200603/dermouse.html Just to satisfy my curiosity, is it the sort of thing that can be implemented as a GEOM layer? The idea is bloody clever but sounds like a bit of a hack right now. You can already do this with GEOM. On your server node, create a sparse file using dd, that is the same size or bigger than the partition you would like to back up. Then set it up with ggated. Now, on your machine with the partition that needs mirroring (backing up), use ggatec to connect to the backup node's shared sparse file you previously created. Then, use gmirror to mirror your partition to the now local ggatec'ed device. Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: NetBSD disk backup over network
Le Tuesday 7 March 2006 15:46, Dag-Erling Smørgrav a écrit : > Ashley Moran <[EMAIL PROTECTED]> writes: > > I just saw this slashdotted article: > > http://ezine.daemonnews.org/200603/dermouse.html > > > > Just to satisfy my curiosity, is it the sort of thing that can be > > implemented as a GEOM layer? The idea is bloody clever but sounds > > like a bit of a hack right now. It's just bad that the FreeBSD champion is (was ?) Yahoo and not Google, as Google seems to be soon offering a large online storage (see http://today.reuters.co.uk/news/NewsArticle.aspx?type=internetNews&storyID=2006-03-07T080717Z_01_N07296137_RTRIDST_0_OUKIN-UK-GOOGLE-STORAGE.XML) An enterprising hacker could create a new geom class to build a mirror with the local disk and the new google remote disk. TfH ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: NetBSD disk backup over network
Matthew D. Fuller wrote: You can easily save the stream of updates as a redo log (well, that's the idea I've been running around with). Isn't that what the gjournal SoC thing was about? No, not exactly. The idea was to make a journal of a GEOM device I/O requests on a separate device in the attempt to solve filesystem journaling, but that didn't work out (actually, gjournal works more or less fine, but as far as I understand it, there's a problem similar as with SoftUpdates - even if you journal writes on device level with filesystem mounted "sync", UFS keeps references to inodes in not-entirely-consistent way, so a fsck is always needed after unclean shutdown; in other words, UFS journaling must be done on UFS/VFS level, not GEOM in order to keep track of metadata semantics). Something like a filesystem that doesn't do any read caching, and keeps ALL data ALWAYS in sync with on-disk state could work with gjournal and also with gmirror+geom_gate combination. Such filesystem would probably be similar to GFS, AFAIK. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Re: NetBSD disk backup over network
On Tue, Mar 07, 2006 at 08:33:37AM -0600 I heard the voice of Sergey Babkin, and lo! it spake thus: > > You can easily save the stream of updates as a redo log (well, > that's the idea I've been running around with). Isn't that what the gjournal SoC thing was about? -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: NetBSD disk backup over network
From: Sergey Babkin <[EMAIL PROTECTED]> Subject: Re: Re: NetBSD disk backup over network Date: Tue, 07 Mar 2006 08:33:37 -0600 (CST) > >From: Bernd Walter <[EMAIL PROTECTED]> > >> >From: Ashley Moran <[EMAIL PROTECTED]> > >> > >> >I just saw this slashdotted article: > >> >http://ezine.daemonnews.org/200603/dermouse.html > >> > >> Well, I've been running around with this kind of idea for > >> around 10 years now. Never actually implemented it though. > >> I can't quite believe that encryption at full disk speeds > >> makes no noticeable CPU overhead. > > > >This sounds as nothing more than a mirror with one disk beeing a remote > >file. > >And this is not really a new idea - remote mirror has a long standing > >tradition. > >You can already configure these things with GEOM right now. > >But this is in no way a backup, this just saves you from disk failures > >which is the purpose of a mirror. > >What is missing is history in the remote image so that one can access > >older contents. > > You can easily save the stream of updates as a redo > log (well, that's the idea I've been running around with). > Then you can roll forward from the full backup points using > this log, and also use it for online backups while > the operations are still running. Of course, it would > probably require an fsck to get things actually mounted. > My impression from the article was that he had this thing > as well. Years ago, I did something called Datastar. We did remote geographic mirroring between different sites at the block level. This worked only so so. The biggest problem was that on solaris (and FreeBSD too, I'd think) the buffer cache would persist in core a long time before being shipped out to disk in some cases for busy blocks. This made the remote system unusable most of the time. We had no way of knowing how good/bad the actual data was due to these buffering issues. The problem also was that fsck would break the mirror if you did it in write mode, forcing a complete resync (optimized, of course, but still expensive). As would mounting the file system r/w. There were also major issues with solaris mounting it read-only when the mirroring daemon was writing to the disk. These issues aren't insurmountable, but can be difficult to cope with. The missing data issue is likely the biggest, most difficult problem to face. Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: NetBSD disk backup over network
Ashley Moran <[EMAIL PROTECTED]> writes: > I just saw this slashdotted article: > http://ezine.daemonnews.org/200603/dermouse.html > > Just to satisfy my curiosity, is it the sort of thing that can be > implemented as a GEOM layer? The idea is bloody clever but sounds > like a bit of a hack right now. Set up ggated on the "backup server": # truncate -s16G /backup/foo.img (assuming the size of the disk you want to mirror is 16 GB) # echo "foo RW /backup/foo.img" >/etc/gg.exports where foo is the name or IP address of the client. start ggated: # ggated /etc/gg.exports start ggatec on the client: # ggatec create bar /backup/foo.img where bar is the name or IP address of the server. now you can create a mirror on the client: # gmirror load # gmirror label -b prefer baz /dev/ggate0 # gmirror insert -p 1000 baz /dev/whatever # newfs -U /dev/mirror/baz # mount -t ufs /dev/mirror/baz /mnt (baz can be any name you want to give your mirror) if /dev/whatever on the client dies, you can simply mdconfig /backup/foo.img on the server and mount it to extract data. If you take care not to modify it, you can easily restore the volume on the client: # ggatec create bar /backup/foo.img # gmirror load # gmirror forget baz # gmirror insert -p 1000 baz /dev/whatever gmirror will immediately start resynchronizing the mirror; you can follow its progress with 'gmirror status'. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Re: NetBSD disk backup over network
>From: Bernd Walter <[EMAIL PROTECTED]> >> >From: Ashley Moran <[EMAIL PROTECTED]> >> >> >I just saw this slashdotted article: >> >http://ezine.daemonnews.org/200603/dermouse.html >> >> Well, I've been running around with this kind of idea for >> around 10 years now. Never actually implemented it though. >> I can't quite believe that encryption at full disk speeds >> makes no noticeable CPU overhead. > >This sounds as nothing more than a mirror with one disk beeing a remote >file. >And this is not really a new idea - remote mirror has a long standing >tradition. >You can already configure these things with GEOM right now. >But this is in no way a backup, this just saves you from disk failures >which is the purpose of a mirror. >What is missing is history in the remote image so that one can access >older contents. You can easily save the stream of updates as a redo log (well, that's the idea I've been running around with). Then you can roll forward from the full backup points using this log, and also use it for online backups while the operations are still running. Of course, it would probably require an fsck to get things actually mounted. My impression from the article was that he had this thing as well. -SB ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: NetBSD disk backup over network
On Tue, 2006-03-07 at 14:43 +0100, Bernd Walter wrote: > On Tue, Mar 07, 2006 at 07:17:20AM -0600, Sergey Babkin wrote: > > >From: Ashley Moran <[EMAIL PROTECTED]> > > > > >I just saw this slashdotted article: > > >http://ezine.daemonnews.org/200603/dermouse.html > > > > > >Just to satisfy my curiosity, is it the sort of thing that can be > > >implemented > > >as a GEOM layer? The idea is bloody clever but sounds like a bit of a > > >hack > > >right now. > > > > Well, I've been running around with this kind of idea for > > around 10 years now. Never actually implemented it though. > > I can't quite believe that encryption at full disk speeds > > makes no noticeable CPU overhead. > > This sounds as nothing more than a mirror with one disk beeing a remote > file. > And this is not really a new idea - remote mirror has a long standing > tradition. > You can already configure these things with GEOM right now. That's geom_gate (for the record). -- Florent Thoumie [EMAIL PROTECTED] FreeBSD Committer ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: NetBSD disk backup over network
On Tue, Mar 07, 2006 at 07:17:20AM -0600, Sergey Babkin wrote: > >From: Ashley Moran <[EMAIL PROTECTED]> > > >I just saw this slashdotted article: > >http://ezine.daemonnews.org/200603/dermouse.html > > > >Just to satisfy my curiosity, is it the sort of thing that can be > >implemented > >as a GEOM layer? The idea is bloody clever but sounds like a bit of a hack > >right now. > > Well, I've been running around with this kind of idea for > around 10 years now. Never actually implemented it though. > I can't quite believe that encryption at full disk speeds > makes no noticeable CPU overhead. This sounds as nothing more than a mirror with one disk beeing a remote file. And this is not really a new idea - remote mirror has a long standing tradition. You can already configure these things with GEOM right now. But this is in no way a backup, this just saves you from disk failures which is the purpose of a mirror. What is missing is history in the remote image so that one can access older contents. -- B.Walterhttp://www.bwct.de http://www.fizon.de [EMAIL PROTECTED] [EMAIL PROTECTED][EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: NetBSD disk backup over network
>From: Ashley Moran <[EMAIL PROTECTED]> >I just saw this slashdotted article: >http://ezine.daemonnews.org/200603/dermouse.html > >Just to satisfy my curiosity, is it the sort of thing that can be implemented >as a GEOM layer? The idea is bloody clever but sounds like a bit of a hack >right now. Well, I've been running around with this kind of idea for around 10 years now. Never actually implemented it though. I can't quite believe that encryption at full disk speeds makes no noticeable CPU overhead. -SB ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
NetBSD disk backup over network
I just saw this slashdotted article: http://ezine.daemonnews.org/200603/dermouse.html Just to satisfy my curiosity, is it the sort of thing that can be implemented as a GEOM layer? The idea is bloody clever but sounds like a bit of a hack right now. Ashley ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
