Re: [zfs-discuss] ZFS mirror with internal and external disks
I'm afraid I asked a very stupid question... This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
Nicolas Williams wrote: > On Fri, Oct 05, 2007 at 10:44:21AM -0700, John Plocher wrote: > >> Lori Alt wrote: >> >>> I'm not surprised that having /usr in a separate pool failed. >>> The design of zfs boot largely assumes that root, /usr, and >>> /var are all on the same pool, and it is unlikely that we would >>> do the work to support any other configuration any time soon. >>> >> This seems, uhm, undesirable. I could understand if the initial >> *implementation* chose to make these simplifying assumptions, but >> if the *architecture* or *design* of the feature requires this, >> then, IMO, this project needs a TCR to not be done that way. >> >> Certainly, many of us will be satisfied with all-in-one pool, >> just as we are today with all all-in-one filesystem, so this >> makes sense as a first step. But, there needs to be the >> presumption that the next steps towards multiple pool support >> are possible without having to re-architect or re-design the >> whole zfs boot system. >> > > I'm curious as to why you think this (note: I've nothing to do with ZFS > development). I understand the need for separate / and /usr in some > cases, but how does separate / and /usr add value in a ZFS bootroot > environment? Is it because one might like to have a very tiny pool > (e.g., on a USB flashdrive) to contain / and a larger one to contain > /usr? > Not a sufficient reason. Crypto policy is a data set (file system/zvol) policy not a storage pool policy. The question of why to have different storage pools has still not been satisfactorily addressed. Methinks people are still confusing pools and data sets. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
> But note that, for ZFS, the win with direct I/O will be somewhat > less. That's because you still need to read the page to compute > its checksum. So for direct I/O with ZFS (with checksums enabled), > the cost is W:LPS, R:2*LPS. Is saving one page of writes enough to > make a difference? Possibly not. It's more complicated than that. The kernel would be verifying checksums on buffers in a user's address space. For this to work, we have to map these buffers into the kernel and simultaneously arrange for these pages to be protected from other threads in the user's address space. We discussed some of the VM gymnastics required to properly implement this back in January: http://mail.opensolaris.org/pipermail/zfs-discuss/2007-January/thread.html#36890 -j ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
> Is there a specific reason why you need to do the caching at the DB > level instead of the file system? I'm really curious as i've got > conflicting data on why people do this. If i get more data on real > reasons on why we shouldn't cache at the file system, then this could > get bumped up in my priority queue. FWIW a MySQL database was recently moved to a FreeBSD system with ZFS. Performance ended up sucking because for some reason data did not make it into the cache in a predictable fashion (simple case of repeated queries were not cached; so for example a very common query, even when executed repeatedly on an idle system, would take more than 1 minute instead of 0.10 seconds or so when cached). Ended up convincing the person running the DB to switch from MyISAM (which does not seem to support DB level caching, other than of indexes) to InnoDB, thus allowing use of the InnoDB buffer cache. I don't know why it wasn't cached by ZFS/ARC to begin with (the size of the ARC cache was definitely large enough - ~ 800 MB, and I know the working set for this query was below 300 MB). Perhaps it has to do with ARC trying to be smart and avoiding flushing the cache with useless data? I am not read up on the details of the ARC. But in this particular case it was clear that a simple LRU had been much more useful - unless there was some other problem related to my setup or FreeBSD integration that somehow broke proper caching. -- / Peter Schuller PGP userID: 0xE9758B7D or 'Peter Schuller <[EMAIL PROTECTED]>' Key retrieval: Send an E-Mail to [EMAIL PROTECTED] E-Mail: [EMAIL PROTECTED] Web: http://www.scode.org pgpmGHbRPWivC.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Do we have a successful installation method for patch 120011-14?
>122660-10 does not have any issues that I am aware of. It is only obsolete, not withdrawn. Additionally, it appears that the circular patch dependency is by design if you read this BugID: So how to do you get it to install? I get... #patchadd 122660-10 Validating patches... Loading patches installed on the system... Done! Loading patches requested to install. Done! Checking patches that you specified for installation. Done! The following requested patches will not be installed because they have been made obsolete by other patches already installed on the system or by patches you have specified for installation. 0 All packages from patch 122660-10 are patched by higher revision patches. No patches to install. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS & array NVRAM cache?
So I went ahead and loaded 10u4 on a pair of V210 units. I am going to set this nocacheflush option and cross my fingers and see how it goes. I have my ZPool mirroring LUNs off 2 different arrays. I have single-controllers in each 3310. My belief is it's OK for me to do this even without dual controllers for NVRAM security. Worst case I lose an array and/or it's NVRAM contents but the other array should be doing it's job as part of the mirroring and I should be all good. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
On Fri, Oct 05, 2007 at 02:01:29PM -0500, Nicolas Williams wrote: > On Fri, Oct 05, 2007 at 06:54:21PM +, A Darren Dunham wrote: > > I wonder how much this would change if a functional "pivot-root" > > mechanism were available. It be handy nice to boot from flash, import a > > pool, then make that the running root. > > > > Does anyone know if that's a target of any OpenSolaris projects? > > That's pretty much what the ZFS mountroot effort was about, and it has > been replaced with ZFS bootroot instead. But I suspect you mean > something more general than a ZFS root scenario, something more > Linux-like (where even user-land processes can run prior to switching to > the real root and starting init). Perhaps. My thoughts were really wondering about something more general than ZFS, rather than any ZFS-specific solutions. -- Darren Dunham [EMAIL PROTECTED] Senior Technical Consultant TAOShttp://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
Nicolas Williams wrote: > I'm curious as to why you think this The characteristics of /, /usr and /var are quite different, from a usage and backup requirements perspective: / is read-mostly, but contains critical config data. /usr is read-only, and /var (/var/mail, /var/mysql, ...) can be high volume read/write. A scenerio with / on a pair of mirrored USB sticks, /usr on DVD media with with RAM-based cache and /var & /export/home on a large wide striped/mirrored/raided pool of its own isn't too far fetched an idea. -John ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
John Plocher wrote: > Lori Alt wrote: >> I'm not surprised that having /usr in a separate pool failed. >> The design of zfs boot largely assumes that root, /usr, and >> /var are all on the same pool, and it is unlikely that we would >> do the work to support any other configuration any time soon. > > > This seems, uhm, undesirable. I could understand if the initial > *implementation* chose to make these simplifying assumptions, but > if the *architecture* or *design* of the feature requires this, > then, IMO, this project needs a TCR to not be done that way. It's not inherent in the design of zfs boot. It's more a matter of the current implementation. For example, install and liveupgrade (and future boot-environment management tools) will get a fair amount of mileage from the fact that you can set a mount point at the top of a boot-environment dataset hierarchy and all of the subordinate file systems in the boot environment will inherit it. That kind of thing will only work if the file system composing the BE are in the same pool. In the initial release, the early SMF scripts that mount root, /usr, var, etc. assume that they are in a well-defined hierarchy: / //usr //var such that once the / is known, all of the other file systems can be found relative to it. This only works if they are in the same dataset. > > Certainly, many of us will be satisfied with all-in-one pool, > just as we are today with all all-in-one filesystem, so this > makes sense as a first step. But, there needs to be the > presumption that the next steps towards multiple pool support > are possible without having to re-architect or re-design the > whole zfs boot system. > No, re-architecting should not be necessary. Lori ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
On Fri, Oct 05, 2007 at 06:54:21PM +, A Darren Dunham wrote: > I wonder how much this would change if a functional "pivot-root" > mechanism were available. It be handy nice to boot from flash, import a > pool, then make that the running root. > > Does anyone know if that's a target of any OpenSolaris projects? That's pretty much what the ZFS mountroot effort was about, and it has been replaced with ZFS bootroot instead. But I suspect you mean something more general than a ZFS root scenario, something more Linux-like (where even user-land processes can run prior to switching to the real root and starting init). Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
On Fri, Oct 05, 2007 at 01:41:32PM -0500, Nicolas Williams wrote: > > Certainly, many of us will be satisfied with all-in-one pool, > > just as we are today with all all-in-one filesystem, so this > > makes sense as a first step. But, there needs to be the > > presumption that the next steps towards multiple pool support > > are possible without having to re-architect or re-design the > > whole zfs boot system. > > I'm curious as to why you think this (note: I've nothing to do with ZFS > development). I understand the need for separate / and /usr in some > cases, but how does separate / and /usr add value in a ZFS bootroot > environment? Is it because one might like to have a very tiny pool > (e.g., on a USB flashdrive) to contain / and a larger one to contain > /usr? > > Thinking of ZFS crypto, it might, since one might put / in cleartext on > a small capacity USB flashdrive, say and keep everything else encrypted. > But one should want ZFS crypto to protect / as well as everything else > (/usr and homedirs), and I would hope that when ZFS crypto gets around > to meeting ZFS bootroot then we'll able to do just that. I wonder how much this would change if a functional "pivot-root" mechanism were available. It be handy nice to boot from flash, import a pool, then make that the running root. Does anyone know if that's a target of any OpenSolaris projects? -- Darren Dunham [EMAIL PROTECTED] Senior Technical Consultant TAOShttp://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
On Fri, Oct 05, 2007 at 10:44:21AM -0700, John Plocher wrote: > Lori Alt wrote: > > I'm not surprised that having /usr in a separate pool failed. > > The design of zfs boot largely assumes that root, /usr, and > > /var are all on the same pool, and it is unlikely that we would > > do the work to support any other configuration any time soon. > > > This seems, uhm, undesirable. I could understand if the initial > *implementation* chose to make these simplifying assumptions, but > if the *architecture* or *design* of the feature requires this, > then, IMO, this project needs a TCR to not be done that way. > > Certainly, many of us will be satisfied with all-in-one pool, > just as we are today with all all-in-one filesystem, so this > makes sense as a first step. But, there needs to be the > presumption that the next steps towards multiple pool support > are possible without having to re-architect or re-design the > whole zfs boot system. I'm curious as to why you think this (note: I've nothing to do with ZFS development). I understand the need for separate / and /usr in some cases, but how does separate / and /usr add value in a ZFS bootroot environment? Is it because one might like to have a very tiny pool (e.g., on a USB flashdrive) to contain / and a larger one to contain /usr? Thinking of ZFS crypto, it might, since one might put / in cleartext on a small capacity USB flashdrive, say and keep everything else encrypted. But one should want ZFS crypto to protect / as well as everything else (/usr and homedirs), and I would hope that when ZFS crypto gets around to meeting ZFS bootroot then we'll able to do just that. So assuming that ZFS crypto and bootroot will eventually play very well together, what value is there to having / and /usr on separate pools, or even separate datasets, in a ZFS bootroot environment? Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Need Help Choosing a Rackmount Chassis
I have 2 of these with Tyan mobos (both EATX) very nice to work on. Sadly I'm not running Solaris on it at the moment, not that that really matters. http://rackmountmart.stores.yahoo.net/rm2uracchase.html Corey On Fri, 28 Sep 2007 12:57:01 -0400, Blake wrote: > I'm looking for a rackmount chassis for an x86 ZFS fileserver I wan to build > for my organization. > > Requirements: > > Hot-swap SATA disk support > Minimum of 4-disk SATA support (would prefer 6+) > Hot-swap power supply (redundant) > Some kind of availability for replacement parts > > I'll be putting in a board/proc/controller of my choice. Sadly, there is no > lower-end offering similar to the Thumper, which is way too costly for my > org at the moment. > > Any input or advice is greatly appreciated. > > Blake > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
Lori Alt wrote: > I'm not surprised that having /usr in a separate pool failed. > The design of zfs boot largely assumes that root, /usr, and > /var are all on the same pool, and it is unlikely that we would > do the work to support any other configuration any time soon. This seems, uhm, undesirable. I could understand if the initial *implementation* chose to make these simplifying assumptions, but if the *architecture* or *design* of the feature requires this, then, IMO, this project needs a TCR to not be done that way. Certainly, many of us will be satisfied with all-in-one pool, just as we are today with all all-in-one filesystem, so this makes sense as a first step. But, there needs to be the presumption that the next steps towards multiple pool support are possible without having to re-architect or re-design the whole zfs boot system. -John ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS booting with Solaris (2007-08)
Robert Milkowski wrote:
> Hello Richard,
>
> Friday, September 28, 2007, 7:45:47 PM, you wrote:
>
> RE> Kris Kasner wrote:
>
2. Back to Solaris Volume Manager (SVM), I guess. It's too bad too,
because I
don't like it with 2 SATA disks either. There isn't enough drives to put
the
State Database Replicas so that if either drive failed, the system would
reboot unattended. Unless there is a trick?
>>> There is a trick for this, not sure how long it's been around.
>>> Add to /etc/system:
>>> *Allow the system to boot if one of two rootdisks is missing
>>> set md:mirrored_root_flag=1
>>>
>
> RE> Before you do this, please read the fine manual:
> RE>
> http://docs.sun.com/app/docs/doc/819-2724/chapter2-161?l=en&a=view&q=mirrored_root_flag
>
> The description on docs.sun.com is somewhat misleading.
>
> http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/io/lvm/md/md_mddb.c#5659
>5659 if (mirrored_root_flag == 1 && setno == 0 &&
>5660 svm_bootpath[0] != 0) {
>5661 md_clr_setstatus(setno, MD_SET_STALE);
>
> Looks like it has to be diskset=0 bootpath has to reside on svm device
> and mirrored_root_flag has to be set to 1.
>
> So if you got other disks (>2) in a system just put them in a separate
> disk group.
>
>
>
>
If we have more than 2 disks, then we have space for a 3rd metadb copy.
-- richard
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
Nicolas Williams wrote: On Thu, Oct 04, 2007 at 10:26:24PM -0700, Jonathan Loran wrote: I can envision a highly optimized, pipelined system, where writes and reads pass through checksum, compression, encryption ASICs, that also locate data properly on disk. ... I've argued before that RAID-Z could be implemented in hardware. But I think that it's all about economics. Software is easier to develop and patch than hardware, so if we can put together systems with enough memory, general purpose CPU horsepower, and memory and I/O bandwidth, all cheaply enough, then that will be better than developing special purpose hardware for ZFS. Thumper is an example of such a system. Eventually we may find trends in system design once again favoring pushing special tasks to the edge. When that happens I'm sure we'll go there. But right now the trend is to put crypto co-processors and NICs on the same die as the CPU. Nico 1) We can put it on the same die also, or at least as a chip set on the MoBo. 2) Offload engines do have software, stored in firmware. Or maybe such an offload processor could run software out of a driver, could be booted in dynamically? 3) You all are aware of how many micro processors are involved in a normal file server right? There's one at almost every interface, disk to controller, controller to PCI bridge, PCI bridge to Hyperbus, etc. Imagine the burden if you did all that in the CPU only? I sometimes find it amazing computers are as stable as they are, but it's all in the maturity of the code running in every step of the way, and of course, good firmware coding practices. Your vanilla SCSI controllers and disk drives do a lot of very complex but useful processing. We trust these guys 100%, because the interface is stable, and the code and processors are mature, well used. I do agree, pushing ZFS to the edge will come down the road, when it becomes less dynamic (how boring) and we know more about the bottlenecks. Jon -- - _/ _/ / - Jonathan Loran - - -/ / /IT Manager - - _ / _ / / Space Sciences Laboratory, UC Berkeley -/ / / (510) 643-5146 [EMAIL PROTECTED] - __/__/__/ AST:7731^29u18e3 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
Same here, if zfs boot support raidz then my problems will be solved. On 05/10/2007, at 11:27 PM, Rob Logan wrote: >> I'm not surprised that having /usr in a separate pool failed. > > while this is discouraging, (I have several b62 machines with > root mirrored and /usr on raidz) if booting from raidz > is a pri, and comes soon, at least I'd be happy :-) > > Rob > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
> I'm not surprised that having /usr in a separate pool failed. while this is discouraging, (I have several b62 machines with root mirrored and /usr on raidz) if booting from raidz is a pri, and comes soon, at least I'd be happy :-) Rob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
/var has no problem being on a separate pool. Any reason why it assumes that root and /usr are on the same pool? You're forcing me to sacrifice one or two disk and SATA/IDE port to support "zfs boot" when a 1 gig flashdisk costs less than 10$. / would fit nicely on it, /usr doesn't. I guess I'll have to track down all these broken dependencies myself or wait until 8 gig flashdisk are stocked again. On 05/10/2007, at 10:52 PM, Lori Alt wrote: > Kugutsumen wrote: >> Thanks, this is really strange. >> In your particular case you have /usr on the same pool as your >> rootfs and I guess that's why it is working for you. >> >> Alll my attempts with b64, b70 and b73 failed if /usr is on a >> separate pool. >> > > I'm not surprised that having /usr in a separate pool failed. > The design of zfs boot largely assumes that root, /usr, and > /var are all on the same pool, and it is unlikely that we would > do the work to support any other configuration any time soon. > > Lori > >> >> >>> Hi Kugutsumen, >>> >>> Not sure abt the bugs, I follow instruction at http:// >>> www.opensolaris.org/os/community/zfs/boot/zfsboot-manual >>> and create separate /usr, /opt and /var filesystem. >>> >>> Here is the vfstab: >>> #device device mount FS fsck >>> mount mount >>> #to mount to fsck point typepass >>> at boot options >>> # >>> fd - /dev/fd fd - no - >>> /proc - /proc proc- no - >>> /dev/dsk/c0d0s1 - - swap- no - >>> /devices- /devicesdevfs - no - >>> sharefs - /etc/dfs/sharetab sharefs - no - >>> ctfs- /system/contractctfs- no - >>> objfs - /system/object objfs - no - >>> swap- /tmptmpfs - yes - >>> /dev/dsk/c0d0p0:1 /dev/rdsk/c0d0p0:1 /windows/C >>> pcfs2 yes >>> - >>> /dev/dsk/c0d0p0:2 /dev/rdsk/c0d0p0:2 /windows/D >>> pcfs2 yes >>> - >>> /dev/dsk/c0d0p0:3 /dev/rdsk/c0d0p0:3 /windows/E >>> pcfs2 yes >>> - >>> rootpool/rootfs - / zfs - no - >>> rootpool/rootfs/usr - /usr zfs - no - >>> rootpool/rootfs/var - /var zfs - no - >>> rootpool/rootfs/opt - /opt zfs - yes - >>> >>> The reason why I separate /usr, /opt, /var because I want to >>> compress them: >>> bash-3.00$ zfs get compressratio rootpool/rootfs/usr >>> NAME PROPERTY VALUE SOURCE >>> rootpool/rootfs/usr compressratio 1.65x - >>> bash-3.00$ zfs get compressratio rootpool/rootfs/var >>> NAME PROPERTY VALUE SOURCE >>> rootpool/rootfs/var compressratio 2.10x - >>> bash-3.00$ zfs get compressratio rootpool/rootfs/opt >>> NAME PROPERTY VALUE SOURCE >>> rootpool/rootfs/opt compressratio 1.66x >>> >>> My entire bootdisk only need 2.5GB (entire distribution): >>> bash-3.00$ zfs list rootpool/rootfs >>> NAME USED AVAIL REFER MOUNTPOINT >>> rootpool/rootfs 2.58G 1.85G 351M legacy >>> >>> To be able to rollback you need to create another boot >>> environment using snapshot and clone. I named the new zfs root >>> filesystem as rootpool/tx (planned to install Solaris trusted >>> extension on the new boot environment). >>> >>> bash-3.00$ zfs list -r rootpool/tx >>> NAME USED AVAIL REFER MOUNTPOINT >>> rootpool/tx 57.2M 1.85G 343M legacy >>> rootpool/tx/opt 30K 1.85G 230M legacy >>> rootpool/tx/usr 198K 1.85G 1.79G legacy >>> rootpool/tx/var 644K 1.85G 68.1M legacy >>> >>> If you want to rollback you need to boot to the clone BE then >>> rollback. >>> >>> Rgds, >>> Andre W. >>> >>> Kugutsumen wrote: >>> Please do share how you managed to have a separate ZFS /usr since b64; there are dependencies to /usr and they are not documented. - kv doesn't help too. I tried added /usr/lib/ libdisk* to a /usr/lib dir on the root partition and failed. Jurgen also pointed that there are two related bugs already filed: Bug ID 6570056 Synopsis / sbin/zpool should not link to files in /usr/lib http:// bugs.opensolaris.org/bugdatabase/ view_bug.do?bug_id=6570056 Bug ID 6494840 Synopsis libzfs should dlopen libiscsitgt rather than linking to it http:// bugs.opensolaris.org/bugdatabase/view_bug.do? bug_id=6494840 I can do a snapshot on bootroot too ... after I tried to do a rollback from failsafe I couldn't boot anymore, probably because there was no straightforward way to rebuild the boot archive. Regarding compression, if I am not mistaken, grub cannot access files that are compressed. Regards, K. On 05/10/2007, at 5:55 AM, Andre Wenas wrote: > Hi, Using bootroot I can do seperate /usr filesystem since b64. > I can also do snapshot, clone and compression. Rgds, Andre W. > Kugutsumen wrote: > >> Lori Alt told me that mountrount was a temporary hack until >>
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
> Regarding compression, if I am not mistaken, grub > cannot access files that are compressed. There was a bug where grub was unable to access files on zfs that contained holes: Bug ID 6541114 SynopsisGRUB/ZFS fails to load files from a default compressed (lzjb) root http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6541114 That has been fixed in snv_71. The description text is misleading, there was no issue with reading lzjb compressed files, the bug occurred when reading "hole" blocks from a zfs file. Grub is unable to read from gzip compressed zfs filesystems, though: Bug ID 6538017 SynopsisZFS boot to support gzip decompression http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6538017 This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
On Fri, Oct 05, 2007 at 08:56:26AM -0700, Tim Spriggs wrote: > Time for on board FPGAs! Heh! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
Nicolas Williams wrote: > On Thu, Oct 04, 2007 at 10:26:24PM -0700, Jonathan Loran wrote: > >> I can envision a highly optimized, pipelined system, where writes and >> reads pass through checksum, compression, encryption ASICs, that also >> locate data properly on disk. ... >> > > I've argued before that RAID-Z could be implemented in hardware. But I > think that it's all about economics. Software is easier to develop and > patch than hardware, so if we can put together systems with enough > memory, general purpose CPU horsepower, and memory and I/O bandwidth, > all cheaply enough, then that will be better than developing special > purpose hardware for ZFS. Thumper is an example of such a system. > > Eventually we may find trends in system design once again favoring > pushing special tasks to the edge. When that happens I'm sure we'll go > there. But right now the trend is to put crypto co-processors and NICs > on the same die as the CPU. > > Nico > Time for on board FPGAs! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
Kugutsumen wrote: > Thanks, this is really strange. > In your particular case you have /usr on the same pool as your rootfs > and I guess that's why it is working for you. > > Alll my attempts with b64, b70 and b73 failed if /usr is on a > separate pool. > I'm not surprised that having /usr in a separate pool failed. The design of zfs boot largely assumes that root, /usr, and /var are all on the same pool, and it is unlikely that we would do the work to support any other configuration any time soon. Lori > > >> Hi Kugutsumen, >> >> Not sure abt the bugs, I follow instruction at http:// >> www.opensolaris.org/os/community/zfs/boot/zfsboot-manual >> and create separate /usr, /opt and /var filesystem. >> >> Here is the vfstab: >> #device device mount FS fsck >> mount mount >> #to mount to fsck point typepassat >> boot options >> # >> fd - /dev/fd fd - no - >> /proc - /proc proc- no - >> /dev/dsk/c0d0s1 - - swap- no - >> /devices- /devicesdevfs - no - >> sharefs - /etc/dfs/sharetab sharefs - no - >> ctfs- /system/contractctfs- no - >> objfs - /system/object objfs - no - >> swap- /tmptmpfs - yes - >> /dev/dsk/c0d0p0:1 /dev/rdsk/c0d0p0:1 /windows/C >> pcfs2 yes >> - >> /dev/dsk/c0d0p0:2 /dev/rdsk/c0d0p0:2 /windows/D >> pcfs2 yes >> - >> /dev/dsk/c0d0p0:3 /dev/rdsk/c0d0p0:3 /windows/E >> pcfs2 yes >> - >> rootpool/rootfs - / zfs - no - >> rootpool/rootfs/usr - /usr zfs - no - >> rootpool/rootfs/var - /var zfs - no - >> rootpool/rootfs/opt - /opt zfs - yes - >> >> The reason why I separate /usr, /opt, /var because I want to >> compress them: >> bash-3.00$ zfs get compressratio rootpool/rootfs/usr >> NAME PROPERTY VALUE SOURCE >> rootpool/rootfs/usr compressratio 1.65x - >> bash-3.00$ zfs get compressratio rootpool/rootfs/var >> NAME PROPERTY VALUE SOURCE >> rootpool/rootfs/var compressratio 2.10x - >> bash-3.00$ zfs get compressratio rootpool/rootfs/opt >> NAME PROPERTY VALUE SOURCE >> rootpool/rootfs/opt compressratio 1.66x >> >> My entire bootdisk only need 2.5GB (entire distribution): >> bash-3.00$ zfs list rootpool/rootfs >> NAME USED AVAIL REFER MOUNTPOINT >> rootpool/rootfs 2.58G 1.85G 351M legacy >> >> To be able to rollback you need to create another boot environment >> using snapshot and clone. I named the new zfs root filesystem as >> rootpool/tx (planned to install Solaris trusted extension on the >> new boot environment). >> >> bash-3.00$ zfs list -r rootpool/tx >> NAME USED AVAIL REFER MOUNTPOINT >> rootpool/tx 57.2M 1.85G 343M legacy >> rootpool/tx/opt 30K 1.85G 230M legacy >> rootpool/tx/usr 198K 1.85G 1.79G legacy >> rootpool/tx/var 644K 1.85G 68.1M legacy >> >> If you want to rollback you need to boot to the clone BE then >> rollback. >> >> Rgds, >> Andre W. >> >> Kugutsumen wrote: >> >>> Please do share how you managed to have a separate ZFS /usr since >>> b64; there are dependencies to /usr and they are not documented. - >>> kv doesn't help too. I tried added /usr/lib/libdisk* to a /usr/lib >>> dir on the root partition and failed. Jurgen also pointed that >>> there are two related bugs already filed: Bug ID 6570056 Synopsis / >>> sbin/zpool should not link to files in /usr/lib http:// >>> bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6570056 Bug ID >>> 6494840 Synopsis libzfs should dlopen libiscsitgt rather than >>> linking to it http://bugs.opensolaris.org/bugdatabase/view_bug.do? >>> bug_id=6494840 I can do a snapshot on bootroot too ... after I >>> tried to do a rollback from failsafe I couldn't boot anymore, >>> probably because there was no straightforward way to rebuild the >>> boot archive. Regarding compression, if I am not mistaken, grub >>> cannot access files that are compressed. Regards, K. On >>> 05/10/2007, at 5:55 AM, Andre Wenas wrote: >>> Hi, Using bootroot I can do seperate /usr filesystem since b64. I can also do snapshot, clone and compression. Rgds, Andre W. Kugutsumen wrote: > Lori Alt told me that mountrount was a temporary hack until grub > could boot zfs natively. Since build 62, mountroot support was > dropped and I am not convinced that this is a mistake. Let's > compare the two: Mountroot: Pros: * can have root partition on > raid-z: YES * can have root partition on zfs stripping mirror: > YES * can have usr partition on separate ZFS partition with > build < 72 : YES * can snapshot and rollback root partition: YES > * can use copies on root partition on a single root disk (e.g. a > laptop ): YES * can use compression on root partition: YES Cons: > * grub n
Re: [zfs-discuss] Direct I/O ability with zfs?
On Thu, Oct 04, 2007 at 10:26:24PM -0700, Jonathan Loran wrote: > I can envision a highly optimized, pipelined system, where writes and > reads pass through checksum, compression, encryption ASICs, that also > locate data properly on disk. ... I've argued before that RAID-Z could be implemented in hardware. But I think that it's all about economics. Software is easier to develop and patch than hardware, so if we can put together systems with enough memory, general purpose CPU horsepower, and memory and I/O bandwidth, all cheaply enough, then that will be better than developing special purpose hardware for ZFS. Thumper is an example of such a system. Eventually we may find trends in system design once again favoring pushing special tasks to the edge. When that happens I'm sure we'll go there. But right now the trend is to put crypto co-processors and NICs on the same die as the CPU. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
On Fri, 2007-10-05 at 09:40 -0300, Toby Thain wrote: > How far would that compromise ZFS' #1 virtue (IMHO), end to end > integrity? Speed sells, and speed kills. If the offload were done on the HBA, it would extend the size of the "assumed correct" part of the hardware from just the CPU+memory to also include the offload device and all the I/O bridges, DMA widgets, etc., between it and memory. This is already the case for the TCP checksum offloads commonly performed in network cards, and, yes, people get bitten by it occasionally. An analysis of packets with bad tcp checksums i'm familiar with showed a fair number which showed patterns consistent with a DMA hiccup (repeated words or cache lines); those glitches on a system with checksum offload would turn into data corruption. See "When The CRC and TCP Checksum Disagree", http://www.sigcomm.org/sigcomm2000/conf/paper/sigcomm2000-9-1.pdf - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
ZFS boot is one of the best usage of ZFS for me. I can create more then 10 boot environment, rollback or destroy if necessary. Not afraid of bfu anymore or patching or any other software installation. If bfu breaks the OS, just rollback as simple as that. Rgds, Andre W. Kugutsumen wrote: > Thanks, this is really strange. > In your particular case you have /usr on the same pool as your rootfs > and I guess that's why it is working for you. > > Alll my attempts with b64, b70 and b73 failed if /usr is on a separate > pool. > > > On 05/10/2007, at 4:10 PM, Andre Wenas wrote: > >> Hi Kugutsumen, >> >> Not sure abt the bugs, I follow instruction at >> http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual >> and create separate /usr, /opt and /var filesystem. >> >> Here is the vfstab: >> #device device mount FS fsck >> mount mount >> #to mount to fsck point typepassat >> boot options >> # >> fd - /dev/fd fd - no - >> /proc - /proc proc- no - >> /dev/dsk/c0d0s1 - - swap- no - >> /devices- /devicesdevfs - no - >> sharefs - /etc/dfs/sharetab sharefs - no - >> ctfs- /system/contractctfs- no - >> objfs - /system/object objfs - no - >> swap- /tmptmpfs - yes - >> /dev/dsk/c0d0p0:1 /dev/rdsk/c0d0p0:1 /windows/C >> pcfs2 yes >> - >> /dev/dsk/c0d0p0:2 /dev/rdsk/c0d0p0:2 /windows/D >> pcfs2 yes >> - >> /dev/dsk/c0d0p0:3 /dev/rdsk/c0d0p0:3 /windows/E >> pcfs2 yes >> - >> rootpool/rootfs - / zfs - no - >> rootpool/rootfs/usr - /usr zfs - no - >> rootpool/rootfs/var - /var zfs - no - >> rootpool/rootfs/opt - /opt zfs - yes - >> >> The reason why I separate /usr, /opt, /var because I want to compress >> them: >> bash-3.00$ zfs get compressratio rootpool/rootfs/usr >> NAME PROPERTY VALUE SOURCE >> rootpool/rootfs/usr compressratio 1.65x - >> bash-3.00$ zfs get compressratio rootpool/rootfs/var >> NAME PROPERTY VALUE SOURCE >> rootpool/rootfs/var compressratio 2.10x - >> bash-3.00$ zfs get compressratio rootpool/rootfs/opt >> NAME PROPERTY VALUE SOURCE >> rootpool/rootfs/opt compressratio 1.66x >> >> My entire bootdisk only need 2.5GB (entire distribution): >> bash-3.00$ zfs list rootpool/rootfs >> NAME USED AVAIL REFER MOUNTPOINT >> rootpool/rootfs 2.58G 1.85G 351M legacy >> >> To be able to rollback you need to create another boot environment >> using snapshot and clone. I named the new zfs root filesystem as >> rootpool/tx (planned to install Solaris trusted extension on the new >> boot environment). >> >> bash-3.00$ zfs list -r rootpool/tx >> NAME USED AVAIL REFER MOUNTPOINT >> rootpool/tx 57.2M 1.85G 343M legacy >> rootpool/tx/opt 30K 1.85G 230M legacy >> rootpool/tx/usr 198K 1.85G 1.79G legacy >> rootpool/tx/var 644K 1.85G 68.1M legacy >> >> If you want to rollback you need to boot to the clone BE then rollback. >> >> Rgds, >> Andre W. >> >> Kugutsumen wrote: >>> Please do share how you managed to have a separate ZFS /usr since >>> b64; there are dependencies to /usr and they are not documented. -kv >>> doesn't help too. I tried added /usr/lib/libdisk* to a /usr/lib dir >>> on the root partition and failed. Jurgen also pointed that there are >>> two related bugs already filed: Bug ID 6570056 Synopsis /sbin/zpool >>> should not link to files in /usr/lib >>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6570056 >>> Bug ID 6494840 Synopsis libzfs should dlopen libiscsitgt rather than >>> linking to it >>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6494840 I >>> can do a snapshot on bootroot too ... after I tried to do a rollback >>> from failsafe I couldn't boot anymore, probably because there was no >>> straightforward way to rebuild the boot archive. Regarding >>> compression, if I am not mistaken, grub cannot access files that are >>> compressed. Regards, K. On 05/10/2007, at 5:55 AM, Andre Wenas wrote: Hi, Using bootroot I can do seperate /usr filesystem since b64. I can also do snapshot, clone and compression. Rgds, Andre W. Kugutsumen wrote: > Lori Alt told me that mountrount was a temporary hack until grub > could boot zfs natively. Since build 62, mountroot support was > dropped and I am not convinced that this is a mistake. Let's > compare the two: Mountroot: Pros: * can have root partition on > raid-z: YES * can have root partition on zfs stripping mirror: YES > * can have usr partition on separate ZFS partition with build < 72 > : YES * can snapshot and rollback root partition: YES * can use > copies on root partition on a single root disk (e.g. a laptop ): > YES * can use compression on root partition: YES Cons: * grub > native support
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
Thanks, this is really strange. In your particular case you have /usr on the same pool as your rootfs and I guess that's why it is working for you. Alll my attempts with b64, b70 and b73 failed if /usr is on a separate pool. On 05/10/2007, at 4:10 PM, Andre Wenas wrote: > Hi Kugutsumen, > > Not sure abt the bugs, I follow instruction at http:// > www.opensolaris.org/os/community/zfs/boot/zfsboot-manual > and create separate /usr, /opt and /var filesystem. > > Here is the vfstab: > #device device mount FS fsck > mount mount > #to mount to fsck point typepassat > boot options > # > fd - /dev/fd fd - no - > /proc - /proc proc- no - > /dev/dsk/c0d0s1 - - swap- no - > /devices- /devicesdevfs - no - > sharefs - /etc/dfs/sharetab sharefs - no - > ctfs- /system/contractctfs- no - > objfs - /system/object objfs - no - > swap- /tmptmpfs - yes - > /dev/dsk/c0d0p0:1 /dev/rdsk/c0d0p0:1 /windows/C > pcfs2 yes > - > /dev/dsk/c0d0p0:2 /dev/rdsk/c0d0p0:2 /windows/D > pcfs2 yes > - > /dev/dsk/c0d0p0:3 /dev/rdsk/c0d0p0:3 /windows/E > pcfs2 yes > - > rootpool/rootfs - / zfs - no - > rootpool/rootfs/usr - /usr zfs - no - > rootpool/rootfs/var - /var zfs - no - > rootpool/rootfs/opt - /opt zfs - yes - > > The reason why I separate /usr, /opt, /var because I want to > compress them: > bash-3.00$ zfs get compressratio rootpool/rootfs/usr > NAME PROPERTY VALUE SOURCE > rootpool/rootfs/usr compressratio 1.65x - > bash-3.00$ zfs get compressratio rootpool/rootfs/var > NAME PROPERTY VALUE SOURCE > rootpool/rootfs/var compressratio 2.10x - > bash-3.00$ zfs get compressratio rootpool/rootfs/opt > NAME PROPERTY VALUE SOURCE > rootpool/rootfs/opt compressratio 1.66x > > My entire bootdisk only need 2.5GB (entire distribution): > bash-3.00$ zfs list rootpool/rootfs > NAME USED AVAIL REFER MOUNTPOINT > rootpool/rootfs 2.58G 1.85G 351M legacy > > To be able to rollback you need to create another boot environment > using snapshot and clone. I named the new zfs root filesystem as > rootpool/tx (planned to install Solaris trusted extension on the > new boot environment). > > bash-3.00$ zfs list -r rootpool/tx > NAME USED AVAIL REFER MOUNTPOINT > rootpool/tx 57.2M 1.85G 343M legacy > rootpool/tx/opt 30K 1.85G 230M legacy > rootpool/tx/usr 198K 1.85G 1.79G legacy > rootpool/tx/var 644K 1.85G 68.1M legacy > > If you want to rollback you need to boot to the clone BE then > rollback. > > Rgds, > Andre W. > > Kugutsumen wrote: >> Please do share how you managed to have a separate ZFS /usr since >> b64; there are dependencies to /usr and they are not documented. - >> kv doesn't help too. I tried added /usr/lib/libdisk* to a /usr/lib >> dir on the root partition and failed. Jurgen also pointed that >> there are two related bugs already filed: Bug ID 6570056 Synopsis / >> sbin/zpool should not link to files in /usr/lib http:// >> bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6570056 Bug ID >> 6494840 Synopsis libzfs should dlopen libiscsitgt rather than >> linking to it http://bugs.opensolaris.org/bugdatabase/view_bug.do? >> bug_id=6494840 I can do a snapshot on bootroot too ... after I >> tried to do a rollback from failsafe I couldn't boot anymore, >> probably because there was no straightforward way to rebuild the >> boot archive. Regarding compression, if I am not mistaken, grub >> cannot access files that are compressed. Regards, K. On >> 05/10/2007, at 5:55 AM, Andre Wenas wrote: >>> Hi, Using bootroot I can do seperate /usr filesystem since b64. I >>> can also do snapshot, clone and compression. Rgds, Andre W. >>> Kugutsumen wrote: Lori Alt told me that mountrount was a temporary hack until grub could boot zfs natively. Since build 62, mountroot support was dropped and I am not convinced that this is a mistake. Let's compare the two: Mountroot: Pros: * can have root partition on raid-z: YES * can have root partition on zfs stripping mirror: YES * can have usr partition on separate ZFS partition with build < 72 : YES * can snapshot and rollback root partition: YES * can use copies on root partition on a single root disk (e.g. a laptop ): YES * can use compression on root partition: YES Cons: * grub native support: NO (if you use raid-z or stripping mirror, you will need to have a small UFS partition to bootstrap the system, but you can use a small usb stick for that purpose.) New and "improved" *sigh* bootroot scheme: Pros: * grub native support: YES Cons: * can have root partition on raid-z: NO * can have root partition on zfs stripping
Re: [zfs-discuss] Direct I/O ability with zfs?
Toby Thain wrote: > On 5-Oct-07, at 2:26 AM, Jonathan Loran wrote: > >> I've been thinking about this for awhile, but Anton's analysis >> makes me think about it even more: >> >> We all love ZFS, right. It's futuristic in a bold new way, which >> many virtues, I won't preach tot he choir. But to make it all >> glue together has some necessary CPU/Memory intensive operations >> around checksum generation/validation, compression, encryption, >> data placement/component load balancing, etc. Processors have >> gotten really powerful, much more so than the relative disk I/O >> gains, which in all honesty make ZFS possible. My question: Is >> anyone working on an offload engine for ZFS? > > How far would that compromise ZFS' #1 virtue (IMHO), end to end > integrity? It need not, in fact with ZFS Crypto you will already get the encryption and checksum offloaded if you have suitable hardware (eg a SCA-6000 card or a Niagara 2 processor). -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
On 10/5/07, Robert Milkowski <[EMAIL PROTECTED]> wrote: > RH> 2) Also, direct I/O is faster because it avoids double buffering. > > I doubt its buying you much... We don't know how much the performance gain is until we get a prototype and benchmark it - the behavior is different with different DBMSs, OSes, workloads (ie. I/O rate, hit ratio) etc. > However on UFS if you go with direct IO, you allow concurent writes to > the same file and you disable read-aheads - I guess it's buying you > much more in most cases than eliminating double buffering. I really hope that someone can sit down and look at the database interface provided in all the filesystems. So far, there are Direct I/O, Concurrent I/O (AIX JFS2), and Quick I/O (VxFS) http://eval.veritas.com/webfiles/docs/qiowp.pdf Then a prototype for ZFS will help us understand how much we can get... Rayson > > Now the question is - if application is usingi directio() call - what > happens if underlying fs is zfs? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
On 5-Oct-07, at 2:26 AM, Jonathan Loran wrote: > > I've been thinking about this for awhile, but Anton's analysis > makes me think about it even more: > > We all love ZFS, right. It's futuristic in a bold new way, which > many virtues, I won't preach tot he choir. But to make it all > glue together has some necessary CPU/Memory intensive operations > around checksum generation/validation, compression, encryption, > data placement/component load balancing, etc. Processors have > gotten really powerful, much more so than the relative disk I/O > gains, which in all honesty make ZFS possible. My question: Is > anyone working on an offload engine for ZFS? How far would that compromise ZFS' #1 virtue (IMHO), end to end integrity? --Toby > I can envision a highly optimized, pipelined system, where writes > and reads pass through checksum, compression, encryption ASICs, > that also locate data properly on disk. This could even be in the > form of a PCIe SATA/SAS card with many ports, or different options. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
From: "Anton B. Rang" <[EMAIL PROTECTED]> > For many databases, most of the I/O is writes (reads wind up > cached in memory). 2 words: table scan -=dave ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS booting with Solaris (2007-08)
Hello Richard,
Friday, September 28, 2007, 7:45:47 PM, you wrote:
RE> Kris Kasner wrote:
>>> 2. Back to Solaris Volume Manager (SVM), I guess. It's too bad too, because
>>> I
>>> don't like it with 2 SATA disks either. There isn't enough drives to put
>>> the
>>> State Database Replicas so that if either drive failed, the system would
>>> reboot unattended. Unless there is a trick?
>>
>> There is a trick for this, not sure how long it's been around.
>> Add to /etc/system:
>> *Allow the system to boot if one of two rootdisks is missing
>> set md:mirrored_root_flag=1
RE> Before you do this, please read the fine manual:
RE>
http://docs.sun.com/app/docs/doc/819-2724/chapter2-161?l=en&a=view&q=mirrored_root_flag
The description on docs.sun.com is somewhat misleading.
http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/io/lvm/md/md_mddb.c#5659
5659 if (mirrored_root_flag == 1 && setno == 0 &&
5660 svm_bootpath[0] != 0) {
5661 md_clr_setstatus(setno, MD_SET_STALE);
Looks like it has to be diskset=0 bootpath has to reside on svm device
and mirrored_root_flag has to be set to 1.
So if you got other disks (>2) in a system just put them in a separate
disk group.
--
Best regards,
Robert Milkowski mailto:[EMAIL PROTECTED]
http://milek.blogspot.com
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Direct I/O ability with zfs?
Hello Rayson, Tuesday, October 2, 2007, 8:56:09 PM, you wrote: RH> 1) Modern DBMSs cache database pages in their own buffer pool because RH> it is less expensive than to access data from the OS. (IIRC, MySQL's RH> MyISAM is the only one that relies on the FS cache, but a lot of MySQL RH> sites use INNODB which has its own buffer pool) RH> 2) Also, direct I/O is faster because it avoid double buffering. I doubt its buying you much... However on UFS if you go with direct IO, you allow concurent writes to the same file and you disable read-aheads - I guess it's buying you much more in most cases than eliminating double buffering. Now the question is - if application is usingi directio() call - what happens if underlying fs is zfs? -- Best regards, Robert Milkowski mailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Mountroot and Bootroot Comparison
Hi Kugutsumen, Not sure abt the bugs, I follow instruction at http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual and create separate /usr, /opt and /var filesystem. Here is the vfstab: #device device mount FS fsckmount mount #to mount to fsck point typepassat boot options # fd - /dev/fd fd - no - /proc - /proc proc- no - /dev/dsk/c0d0s1 - - swap- no - /devices- /devicesdevfs - no - sharefs - /etc/dfs/sharetab sharefs - no - ctfs- /system/contractctfs- no - objfs - /system/object objfs - no - swap- /tmptmpfs - yes - /dev/dsk/c0d0p0:1 /dev/rdsk/c0d0p0:1 /windows/C pcfs 2 yes - /dev/dsk/c0d0p0:2 /dev/rdsk/c0d0p0:2 /windows/D pcfs 2 yes - /dev/dsk/c0d0p0:3 /dev/rdsk/c0d0p0:3 /windows/E pcfs 2 yes - rootpool/rootfs - / zfs - no - rootpool/rootfs/usr - /usr zfs - no - rootpool/rootfs/var - /var zfs - no - rootpool/rootfs/opt - /opt zfs - yes - The reason why I separate /usr, /opt, /var because I want to compress them: bash-3.00$ zfs get compressratio rootpool/rootfs/usr NAME PROPERTY VALUE SOURCE rootpool/rootfs/usr compressratio 1.65x - bash-3.00$ zfs get compressratio rootpool/rootfs/var NAME PROPERTY VALUE SOURCE rootpool/rootfs/var compressratio 2.10x - bash-3.00$ zfs get compressratio rootpool/rootfs/opt NAME PROPERTY VALUE SOURCE rootpool/rootfs/opt compressratio 1.66x My entire bootdisk only need 2.5GB (entire distribution): bash-3.00$ zfs list rootpool/rootfs NAME USED AVAIL REFER MOUNTPOINT rootpool/rootfs 2.58G 1.85G 351M legacy To be able to rollback you need to create another boot environment using snapshot and clone. I named the new zfs root filesystem as rootpool/tx (planned to install Solaris trusted extension on the new boot environment). bash-3.00$ zfs list -r rootpool/tx NAME USED AVAIL REFER MOUNTPOINT rootpool/tx 57.2M 1.85G 343M legacy rootpool/tx/opt 30K 1.85G 230M legacy rootpool/tx/usr 198K 1.85G 1.79G legacy rootpool/tx/var 644K 1.85G 68.1M legacy If you want to rollback you need to boot to the clone BE then rollback. Rgds, Andre W. Kugutsumen wrote: Please do share how you managed to have a separate ZFS /usr since b64; there are dependencies to /usr and they are not documented. -kv doesn't help too. I tried added /usr/lib/libdisk* to a /usr/lib dir on the root partition and failed. Jurgen also pointed that there are two related bugs already filed: Bug ID 6570056 Synopsis/sbin/zpool should not link to files in /usr/lib http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6570056 Bug ID 6494840 Synopsislibzfs should dlopen libiscsitgt rather than linking to it http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6494840 I can do a snapshot on bootroot too ... after I tried to do a rollback from failsafe I couldn't boot anymore, probably because there was no straightforward way to rebuild the boot archive. Regarding compression, if I am not mistaken, grub cannot access files that are compressed. Regards, K. On 05/10/2007, at 5:55 AM, Andre Wenas wrote: Hi, Using bootroot I can do seperate /usr filesystem since b64. I can also do snapshot, clone and compression. Rgds, Andre W. Kugutsumen wrote: Lori Alt told me that mountrount was a temporary hack until grub could boot zfs natively. Since build 62, mountroot support was dropped and I am not convinced that this is a mistake. Let's compare the two: Mountroot: Pros: * can have root partition on raid-z: YES * can have root partition on zfs stripping mirror: YES * can have usr partition on separate ZFS partition with build < 72 : YES * can snapshot and rollback root partition: YES * can use copies on root partition on a single root disk (e.g. a laptop ): YES * can use compression on root partition: YES Cons: * grub native support: NO (if you use raid-z or stripping mirror, you will need to have a small UFS partition to bootstrap the system, but you can use a small usb stick for that purpose.) New and "improved" *sigh* bootroot scheme: Pros: * grub native support: YES Cons: * can have root partition on raid-z: NO * can have root partition on zfs stripping mirror: NO * can use copies on root partition on a single root disk (e.g. a laptop ): NO * can have usr partition on separate ZFS partition with build < 72 : NO * can snapshot and rollback root partition: NO * can use compression on root partition: NO * No backward compatibility with zfs mountroot. Why did we completely drop support for the old mountroot approach which is so much more flexible? Kugutsumen ___
Re: [zfs-discuss] About bug 6486493 (ZFS boot incompatible with
Hello Eric, Thursday, October 4, 2007, 5:54:06 PM, you wrote: ES> On Thu, Oct 04, 2007 at 05:22:58AM -0700, Ivan Wang wrote: >> > This bug was rendered moot via 6528732 in build >> > snv_68 (and s10_u5). We >> > now store physical devices paths with the vnodes, so >> > even though the >> > SATA framework doesn't correctly support open by >> > devid in early boot, we >> >> But if I read it right, there is still a problem in SATA framework (failing >> ldi_open_by_devid,) right? >> If this problem is framework-wide, it might just bite back some time in the >> future. >> ES> Yes, there is still a bug in the SATA framework, in that ES> ldi_open_by_devid() doesn't work early in boot. Opening by device path ES> works so long as you don't recable your boot devices. If we had open by ES> devid working in early boot, then this wouldn't be a problem. Even if someone re-cables sata disks couldn't we fallback to "read zfs label from all available disks and find our pool and import it"? -- Best regards, Robert Milkowski mailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
