Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-29 Thread Nicolas Williams
On Sat, Dec 27, 2008 at 02:29:58PM -0800, Ross wrote: > All of which sound like good reasons to use send/receive and a 2nd zfs > pool instead of mirroring. Yes. > Send/receive has the advantage that the receiving filesystem is > guaranteed to be in a stable state. How would you go about recoveri

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-28 Thread Tim
On Sat, Dec 27, 2008 at 3:24 PM, Miles Nordin wrote: > > "t" == Tim writes: > > t> couldn't you simply do a detach before removing the disk, and > t> do a re-attach everytime you wanted to re-mirror? > > no, for two reasons. First, when you detach a disk, ZFS writes > something to

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-28 Thread Kees Nuyt
On Sun, 28 Dec 2008 15:27:00 +0100, dick hoogendijk wrote: >On Sat, 27 Dec 2008 14:29:58 PST >Ross wrote: > >> All of which sound like good reasons to use send/receive and a 2nd >> zfs pool instead of mirroring. >> >> Send/receive has the advantage that the receiving filesystem is >> guaranteed

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-28 Thread Volker A. Brandt
> > Send/receive has the advantage that the receiving filesystem is > > guaranteed to be in a stable state. > > Can send/receive be used on a multiuser running server system? Yes. > Will > this slowdown the services on the server much? "Depends". On a modern box with good disk layout it should

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-28 Thread dick hoogendijk
On Sat, 27 Dec 2008 14:29:58 PST Ross wrote: > All of which sound like good reasons to use send/receive and a 2nd > zfs pool instead of mirroring. > > Send/receive has the advantage that the receiving filesystem is > guaranteed to be in a stable state. Can send/receive be used on a multiuser ru

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-27 Thread Ross
All of which sound like good reasons to use send/receive and a 2nd zfs pool instead of mirroring. Send/receive has the advantage that the receiving filesystem is guaranteed to be in a stable state. How would you go about recovering the system in the event of a drive failure though? Would you

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-27 Thread Miles Nordin
> "t" == Tim writes: t> couldn't you simply do a detach before removing the disk, and t> do a re-attach everytime you wanted to re-mirror? no, for two reasons. First, when you detach a disk, ZFS writes something to the disk that makes it unrecoverable. The simple-UI wallpaper bl

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-22 Thread Ross Smith
On Fri, Dec 19, 2008 at 6:47 PM, Richard Elling wrote: > Ross wrote: >> >> Well, I really like the idea of an automatic service to manage >> send/receives to backup devices, so if you guys don't mind, I'm going to >> share some other ideas for features I think would be useful. >> > > cool. > >> On

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-19 Thread Richard Elling
Ross wrote: > Well, I really like the idea of an automatic service to manage send/receives > to backup devices, so if you guys don't mind, I'm going to share some other > ideas for features I think would be useful. > cool. > One of the first is that you need some kind of capacity management

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross
Well, I really like the idea of an automatic service to manage send/receives to backup devices, so if you guys don't mind, I'm going to share some other ideas for features I think would be useful. One of the first is that you need some kind of capacity management and snapshot deletion. Eventua

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Nicolas Williams
On Thu, Dec 18, 2008 at 12:57:54PM -0800, Richard Elling wrote: > Nicolas Williams wrote: > >Device names are, but there's no harm in showing them if there's > >something else that's less variable. Pool names are not very variable > >at all. > > I was thinking of something a little different. Do

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross Smith
I was thinking more something like: - find all disk devices and slices that have ZFS pools on them - show users the devices and pool names (and UUIDs and device paths in case of conflicts).. >>> >>> I was thinking that device & pool names are too variable, you need

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross Smith
>> Of course, you'll need some settings for this so it's not annoying if >> people don't want to use it. A simple tick box on that pop up dialog >> allowing people to say "don't ask me again" would probably do. > > I would like something better than that. "Don't ask me again" sucks > when much, m

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Richard Elling
Nicolas Williams wrote: > On Thu, Dec 18, 2008 at 07:55:14PM +, Ross Smith wrote: > >> On Thu, Dec 18, 2008 at 7:11 PM, Nicolas Williams >> wrote: >> >>> I was thinking more something like: >>> >>> - find all disk devices and slices that have ZFS pools on them >>> - show users the de

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Nicolas Williams
On Thu, Dec 18, 2008 at 07:55:14PM +, Ross Smith wrote: > On Thu, Dec 18, 2008 at 7:11 PM, Nicolas Williams > wrote: > > I was thinking more something like: > > > > - find all disk devices and slices that have ZFS pools on them > > - show users the devices and pool names (and UUIDs and devic

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross Smith
On Thu, Dec 18, 2008 at 7:11 PM, Nicolas Williams wrote: > On Thu, Dec 18, 2008 at 07:05:44PM +, Ross Smith wrote: >> > Absolutely. >> > >> > The tool shouldn't need to know that the backup disk is accessed via >> > USB, or whatever. The GUI should, however, present devices >> > intelligently

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Nicolas Williams
On Thu, Dec 18, 2008 at 07:05:44PM +, Ross Smith wrote: > > Absolutely. > > > > The tool shouldn't need to know that the backup disk is accessed via > > USB, or whatever. The GUI should, however, present devices > > intelligently, not as cXtYdZ! > > Yup, and that's easily achieved by simply p

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Ross Smith
> Absolutely. > > The tool shouldn't need to know that the backup disk is accessed via > USB, or whatever. The GUI should, however, present devices > intelligently, not as cXtYdZ! Yup, and that's easily achieved by simply prompting for a user friendly name as devices are attached. Now you could

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-18 Thread Nicolas Williams
On Wed, Dec 17, 2008 at 10:02:18AM -0800, Ross wrote: > In fact, thinking about it, could this be more generic than just a USB > backup service? Absolutely. The tool shouldn't need to know that the backup disk is accessed via USB, or whatever. The GUI should, however, present devices intelligent

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Ross
In fact, thinking about it, could this be more generic than just a USB backup service? If this were a scheduled backup system, regularly sending snapshots to another device, with a nice end user GUI, there's nothing stopping it working with any device the user points it at. So you could use US

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Nicolas Williams
On Wed, Dec 17, 2008 at 08:51:54AM -0800, Niall Power wrote: > > What serious compat issues ? There has been one and > > only one > > incompatible change in the stream format and that > > only impacted really > > really early (before S10 FCS IIRC) adopters. > > Here are the issues that I am awa

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Ross
Hey Niall, > Here are the issues that I am aware: > - Running "zfs upgrade" on a zfs filesystem will > cause the "zfs send" stream output format to be > incompatible with older versions of the software. > This is according to the zfs man page. > - Again from the zfs man page: > "The format of th

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Niall Power
> What serious compat issues ? There has been one and > only one > incompatible change in the stream format and that > only impacted really > really early (before S10 FCS IIRC) adopters. Here are the issues that I am aware: - Running "zfs upgrade" on a zfs filesystem will cause the "zfs send"

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Nicolas Williams
On Wed, Dec 17, 2008 at 12:05:50AM -0800, Ross wrote: > Thinking about it, I think Darren is right. An automatic send/receive to the > external drive may be preferable, and it sounds like it has many advantages: You forgot *the* most important advantage of using send/recv instead of mirroring as

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Niall Power
> In the long run some USB stick problems may surface > because the wear > leveling is done in 16MB sections, and you could blow > your stick if > you have a 16MB region which is ``hot''. >  I wonder if parts of a zpool > are hotter than others?  With AVS the dirty > bitmap might be hot. > > I gue

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-17 Thread Ross
Thinking about it, I think Darren is right. An automatic send/receive to the external drive may be preferable, and it sounds like it has many advantages: 1. It's directional, your backups will always go from your live drive to the backup, never the other way unless you actually force it with -

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Darren J Moffat
Niall Power wrote: > Hi all, > > A while back, I posted here about the issues ZFS has with USB hotplugging > of ZFS formatted media when we were trying to plan an external media backup > solution for time-slider: > http://www.opensolaris.org/jive/thread.jspa?messageID=299501 > > As well as the U

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Tim
On Tue, Dec 16, 2008 at 1:53 PM, Miles Nordin wrote: > > "np" == Niall Power writes: > >np> So I'd like to ask if this is an appropriate use of ZFS mirror >np> functionality? > > I like it a lot. > > I tried to set up something like that ad-hoc using a firewire disk on > an Ultra10 a

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Miles Nordin
> "np" == Niall Power writes: np> So I'd like to ask if this is an appropriate use of ZFS mirror np> functionality? I like it a lot. I tried to set up something like that ad-hoc using a firewire disk on an Ultra10 at first, and then, just as you thought, tried using one firewire dis

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Niall Power
Andrew Gabriel wrote: > > Different USB memory sticks vary enormously in speed. > The speed is often not described on the packaging, so it's often not > possible to know how fast one is until after you've bought it and > tried it. > This was tested with an external laptop hardisk inside a USB enc

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Andrew Gabriel
Niall Power wrote: >> Yes to both I believe, while the USB device is >> attached your system will run slower, and it will run >> considerably slower while replicating data. >> Hopefully USB 3 or eSATA drives would address this >> to some extent. > > I think I've confirmed this is the case, at lea

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Niall Power
> > Yes to both I believe, while the USB device is > attached your system will run slower, and it will run > considerably slower while replicating data. > Hopefully USB 3 or eSATA drives would address this > to some extent. I think I've confirmed this is the case, at least in the configuration I

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Ross
> One other question I have about using mirrors is > potential performance implications. > In a common scenario the user might be using the main > S(ATA) attached disk and > a USB external disk as a mirror configuration. Could > the slower disk become a > bottleneck because of it's lower I/O read/

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Niall Power
Hi Volker, > Yes, by all means. I am doing something very similar > on my T1000, but > I have two separate one-disk pools and copy to the > backup pool using > rsync. I would very much like to replace this with > automatic resilvering. > > One prerequisite for wide adoption would be to fix > th

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Niall Power
> Does 1. really need to be fixed? > I'm not suggesting that it's currently broken I'm just asking if it would be reasonable to special case our usage a little bit in order to avoid unnecessary alarm to users. This will be seen as a fit and finish/polish issue. If it's easy to address that then

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Ross
PS. One thing that really would be a useful extension of ZFS for this would be the ability to mirror a raid-z volume. I know it's not in the spec, does anybody know if this is even vaguely possible or whether there's an RFE for this kind of functionality? -- This message posted from opensolar

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-16 Thread Ross
Does 1. really need to be fixed? I ask this since I imagine there will be some resistance from the ZFS team to essentially breaking the spec for the sake of not confusing some users. I would argue that anybody who knows enough to run "zpool status" is also capable of learning what a mirror is a

Re: [zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-15 Thread Volker A. Brandt
> A while back, I posted here about the issues ZFS has with USB hotplugging > of ZFS formatted media when we were trying to plan an external media backup > solution for time-slider: > http://www.opensolaris.org/jive/thread.jspa?messageID=299501 [...] > There are a few minor issues however which I

[zfs-discuss] Using zfs mirror as a simple backup mechanism for time-slider.

2008-12-15 Thread Niall Power
Hi all, A while back, I posted here about the issues ZFS has with USB hotplugging of ZFS formatted media when we were trying to plan an external media backup solution for time-slider: http://www.opensolaris.org/jive/thread.jspa?messageID=299501 As well as the USB issues in the subject we became