Hi Folks, Just thought I'd pipe up here with a few additions on Ian's post, as well as comments on a few others that have come in so far. Long post with a lot of ideas - sorry.
Disclaimer: my day job is testing zfs, so the zfs & install developers probably know more about features/roadmaps. http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide has more wisdom wrt. ZFS than me. Those out of the way, here goes: On Sun, 2007-06-24 at 15:47 -0400, Ian Murdock wrote: > One thing I don't see on the requirements list is ZFS as the default > file system. Yes, I agree. You might be interested in reading Eric's post at: http://blogs.sun.com/erickustarz/entry/zfs_on_a_laptop It has more things we might want to keep in mind: * set "copies" > 1 for the root filesystem (and anything the user cares about on a single-disk installation, obviously decided automatically as a function of total available disk-space, user-override allowed of course) * set compression = on A commenter to Eric's post asks about potential impacts to battery-life these may have - I've absolutely no idea, but if someone wants to send me a new laptop and a stopwatch, I'll check it out. > Of course, to take maximum advantage of ZFS, the entire system needs to > be on ZFS, which means ZFS boot, which I assume is an installer task? Is > this on the installer roadmap? What needs to be done here? (This also > necessarily affects SPARC support, another requirement that > doesn't seem to be listed, which I'll discuss in a separate thread..) Bit of both: the OS part of zfs boot is complete with some limitations. We can boot SMI labeled partitions, in mirrored or single-disk configurations. ZFS root on raid-Z, stripes and EFI boot isn't done yet. Sparc newboot hasn't gone back either, which ZFS boot on sparc would depend on. LiveUpgrade, and the Solaris install/upgrade tools don't yet understand ZFS, but as was mentioned, there's a prototype to do jumpstarts with via a tweaked pfinstall. (you can also manually setup a zfs boot from an existing ufs install if you want to experiment more) > Does putting the entire system on ZFS mean we can do away with > slice level partitioning? Partly, ZFS isn't yet supported as a dump device, and while you can swap to a zvol, I believe there's plans to change the way swap to ZFS works so we're not going through the zvol layer. For now, allocating a normal swap device would be a really sane thing to do (we *definitely* need to be able to savecore(1M) for Indiana!!) > One big win if we can do everything in a ZFS pool is that we don't have > to worry about partitioning. Big simplification there. 2 slices should be feasible eventually I think. There'll still be partitioning/fdisk concerns since we're looking to preserve dual-boot capability with Linux & Windows. > Assuming the entire system is on ZFS, what neat things can we do > with it? Let's brainstorm! ZFS on Zones is a natural fit ( http://blogs.sun.com/martin/entry/cloning_zones_using_zfs ) and I don't think we take enough advantage of Zones out of the box. I'd love to see: * Push-button Zone cloning (need a "fresh" Indiana install to test out a production server? "ping!" - here's one (or reboot your machine back to an earlier snapshot, but given the choice, I'd rather a zone) ) * a zone for GNU compatibility (or a zone for Solaris compatibility, depending on which way you swing - I'd prefer to stick GNU stuff in a zone, but what do I know) * a Linux Zone using BrandZ * VNC configured out of the box for seamless desktop sessions between Zones All of these might be a big ask though ... > We've already talked about using ZFS snapshots to implement > rollbacks after package/system upgrades. > > How might this work? Tim Haley did a BFU with a ZFS clone ([3]). > Automate this and replace bfu with apt-get install/upgrade > or equivalent, and do we has a ZFS-based replacement for LiveUpgrade > that doesn't require a whole lot of additional programming? > > [3] http://blogs.sun.com/timh/entry/friday_fun_with_bfu_and Don't know about LiveUpgrade, but I suspect it's on the roadmap. My zfs-bootadm.sh hack allows for multiple boot environments alá LiveUpgrade, but again, because the upgrade tools don't yet understand ZFS, it's awkward to upgrade them at the moment. Has anyone hacked this into working? > Tim Foster has done a really neat SMF-based automatic snapshot > service [4]. Thanks for the plug! > Tim Foster has also written a script to enable booting into ZFS > snapshots ([5]). Thanks again! Minor nit: snapshots are read-only, so we can only boot into clones of these snapshots, but I know what you mean. One thing I thought could be useful, is if the default install was able to provide "canned" root environments via snapshots: that is, we'd have snapshots of an OpenSolaris system setup as a webserver, another setup as a CIFS/NFS server, another setup as a simple nat/router/firewall, one with a GNU userland environment, etc. Being able to provide canned configurations on install might ease the path for new users looking to get easy access to (potentially) complex OpenSolaris configurations and experiment around with them without too much trouble. (but without burdening every user with having to run all of those services out of the box) The nice thing, would be that since we're using snapshots, not much additional space would be required: it would be a case of automatically setting up these environments during install, snapshotting each, then ensuring we boot back back into the default environment after the final post-install-reboot. [ this all would be similar to the vmware virtual appliances you see popping up all over the place - the difference would be, that you're running on a *real* computer, not a virtual machine, with all the good stuff that this would imply ] Would that be useful ? Teaching packaging about ZFS would be neat too, but I haven't quite wrapped my head around exactly how that could work (snapshots happen on a per-filesystem basis on the entire filesystem - so you couldn't do say: pkgadd A, snapshot, pkgadd B, snapshot, pkgadd C, then decide you don't like pkg B and rollback to A, without loosing pkg C... ) On Sun, 2007-06-24 at 21:19 +0100, Geoffrey Teale wrote: > DTrace is cool as well I love DTrace - I had another "DTrace moment" today, when I was trying to work out what application was triggering hits on my outgoing set of ipfilter rules: ipmon was only telling me that packets were being blocked, not who was trying to send them. 8 lines of D later, I had the answer - all hail DTrace! On Sun, 2007-06-24 at 22:49 +0200, Thomas Wagner wrote: > I would add the grub bootloader with ZFS-boot support to the > requirements list if it is not already there. It's part of the grub that's in ON now - so we should be getting that anyway. > Using the whole disk for ZFS is per default using EFI-Labels for the disks. > The nice thing here is, that the disks internal write-cache is enabled > (and flushed as needed). The old-style x86 fdisk partioning is completely > replaced by EFI and the disk is 100% used for ZFS-only. Afaik, you can still turn on the disk write-cache manually even if ZFS doesn't own the disk. I think we'd still issue the flush commands if caching is turned on regardless (if caching is off, those commands aren't sent) - feel free to correct me if I've got that arseways. > The BIOS of the machine and grub would need to be able booting of EFI > labeled Disks. Relocating grub to a old x86 fdisk disk might not be > enough to solve this. Yeah, no EFI boot just yet. > The only paritioning which might remain is, that we should consider > making "swap" a real Solaris Slice and the rest of the space the > zpool. Yep. > But the whole bunch of filesystems would be without partitioning inside > the zpool. I see only a little disadvantage from having one > slice swap, the rest as the zpool. Lori has some good questions about how to divide up filesystems on a single ZFS pool that would be worth considering: http://blogs.sun.com/lalt/entry/zfs_boot_issue_of_the > > Tim Foster has done a really neat SMF-based automatic snapshot > > service [4]. [......] > > I would say, create a snapshot not too often, but still automaticly > with a rotation schema. FWIW, on my desktop in work, I snapshot every 10 minutes, and keep 1 day's worth of those, snapshot every day, and keep 2 week's worth of those, snapshot every month (and auto-backup via zfs send/recv) and keep 6 of those. Snapshots are free, and I haven't noticed any slowdown despite having many many snapshots online. > I would strongly say, to do a snapshot at every install/upgrade/ > downgrade/remove operation of the package-tools. > This would enable rollbacks for failed operations. Yep, modulo what I mentioned above (again, snap A -> snap B -> snap C -> snap D, you can't rollback from D to A, but keep bits of C without manually backup/restoring) > But snapshotting such system-change-points should not affect > in-use userdata, since upgrades are often done during usage hours... > (emails arriving, ..., so keep user-data and spool-areas completely > separate) Filesystems are cheap in ZFS, so yes, you're absolutely right. > Another word to live-upgrade, there is one very important point Having something like LiveUpgrade grok ZFS would be excellent. On Sun, 2007-06-24 at 23:01 +0200, Roland Mainz wrote: > And ZFS currently lack important features like "user quotas" which > renders ZFS currently _unuseable_ for universities or other large sites Not sure this is a target market for Indiana. > the NFS performance on ZFS is currently not very good Roch has stuff to say about ZFS + NFS - please file a bug if you're seeing poor ZFS + NFS performance. http://blogs.sun.com/roch/entry/nfs_and_zfs_a_fine Right, bedtime - night all! cheers, tim -- Tim Foster, Sun Microsystems Inc, Solaris Engineering Ops http://blogs.sun.com/timf _______________________________________________ indiana-discuss mailing list [email protected] http://opensolaris.org/mailman/listinfo/indiana-discuss
