Re: [zfs-discuss] Thoughts on CF/SSDs [was: ZFS - Use h/w raid or not?Thoughts.Considerations.]
On 6/1/07, Frank Cusack <[EMAIL PROTECTED]> wrote: On June 1, 2007 9:44:23 AM -0700 Richard Elling <[EMAIL PROTECTED]> wrote: [...] > Semiconductor memories are accessed in parallel. Spinning disks are > accessed > serially. Let's take a look at a few examples and see what this looks > like... > > Disk iops bw atime MTBF UER > endurance > - > - > SanDisk 32 GByte 2.5" SATA 7,450 67 0.11 2,000,000 10^-20 ? > SiliconSystems 8 GByte CF 500 8 2 4,000,000 10^-14 > >2,000,000 ... these are probably different technologies though? if cf cards aren't generally fast, then the sata device isn't a cf card just with a different form factor. or is the CF interface the limiting factor? also, isn't CF write very slow (relative to read)? if so, you should really show read vs write iops. Most vendors don't list this, for obvious reasons. SanDisk is honest enough to do so though, and the number is spectacularly bad: 15. Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Thoughts on CF/SSDs [was: ZFS - Use h/w raid or not?Thoughts.Considerations.]
On June 1, 2007 9:44:23 AM -0700 Richard Elling <[EMAIL PROTECTED]> wrote: Frank Cusack wrote: On May 31, 2007 1:59:04 PM -0700 Richard Elling <[EMAIL PROTECTED]> wrote: CF cards aren't generally very fast, so the solid state disk vendors are putting them into hard disk form factors with SAS/SATA interfaces. These If CF cards aren't fast, how will putting them into a different form factor make them faster? Semiconductor memories are accessed in parallel. Spinning disks are accessed serially. Let's take a look at a few examples and see what this looks like... Disk iops bw atime MTBF UER endurance - - SanDisk 32 GByte 2.5" SATA 7,450 67 0.11 2,000,000 10^-20 ? SiliconSystems 8 GByte CF 500 8 2 4,000,000 10^-14 >2,000,000 ... these are probably different technologies though? if cf cards aren't generally fast, then the sata device isn't a cf card just with a different form factor. or is the CF interface the limiting factor? also, isn't CF write very slow (relative to read)? if so, you should really show read vs write iops. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SMART
Eric Schrock <[EMAIL PROTECTED]> wrote on Friday, June 01, 2007 12:50:50: Only devices that use the SATA framework (Marvell, Silicon Image, and others - I don't remember the full list) use the SCSI emulation required to make this work. * Do I need any special SATA configuration to get the SCSI emulation? No, but your SATA HBA must be one of those supported by the SATA framework. Otherwise it will operate in PATA legacy mode and the information will not be available. Thanks. I don't know anything about SATA yet, so this helps. From a little research, it looks like the Solaris SATA framework is doing the SCSI emulation for the OS, and it requires a driver designed for that framework to talk to the SATA controller chip. I have an Asus M2NPV-VM, which uses the nVIDIA nForce 430 for SATA 3Gb/s. Unfortunately, this page suggests that it runs in legacy mode: http://blogs.sun.com/chrisg/entry/home_server_hardware_configuration#comment-1162973844000 Where can I find the full list of supported controllers for Solaris SATA HBA? If I buy SATA drives, I might buy another controller too (altho not if I lose ZFS boot). Cheers, 11011011 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: shareiscsi is cool, but what about sharefc or sharescsi?
On Jun 1, 2007, at 18:37, Richard L. Hamilton wrote: Can one use a spare SCSI or FC controller as if it were a target? we'd need an FC or SCSI target mode driver in Solaris .. let's just say we used to have one, and leave it mysteriously there. smart idea though! --- .je ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: shareiscsi is cool, but what about sharefc or sharescsi?
> I'd love to be able to server zvols out as SCSI or FC > targets. Are > there any plans to add this to ZFS? That would be > amazingly awesome. Can one use a spare SCSI or FC controller as if it were a target? Even if the hardware is capable, I don't see what you describe as a ZFS thing really; it isn't for iSCSI, except that ZFS supports a shareiscsi option (and property?) by "knowing" how to tell the iSCSI server to do the right thing. That is, there would have to be something like an iSCSI server except that it "listened" on an otherwise unused SCSI or FC interface. I think that would require not just the daemon but probably new driver facilities as well. Given that one can run IP over FC, it seems to me that in principle it ought to be possible, at least for FC. Not so sure about SCSI. Also not sure about performance. I suspect even high-end SAN controllers have a bit more latency than the underlying drives. And this is a general-purpose OS we're talking about doing this to; I don't know that it would be acceptably close, or as robust (depending on the hardware) as a high-end FC SAN, although it might be possible to be a good deal cheaper. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SMART
On 1-Jun-07, at 7:50 PM, Eric Schrock wrote: On Fri, Jun 01, 2007 at 12:33:29PM -1000, J. David Beutel wrote: Excellent! Thanks! I've gleaned the following from your blog. Is this correct? * A week ago you committed a change that will: ** get current SMART parameters and faults for SATA on x86 via a single function in a private library using SCSI emulation; ** decide whether they indicate any problem I need to be aware of in terms of over temperature, predictive failure, and self-test failure; ** periodically check for above problems and generate an FMA ereport and fault. Yep, that's about the gist of it. ... * Do I need any special SATA configuration to get the SCSI emulation? No, but your SATA HBA must be one of those supported by the SATA framework. Otherwise it will operate in PATA legacy mode and the information will not be available. Well that rules out Frank's and my X2100s :-/ --Toby * Does ZFS mirror boot work on SATA? Currently, there is a bug that prevents this from working. ... ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SMART
On Fri, Jun 01, 2007 at 12:33:29PM -1000, J. David Beutel wrote: > Excellent! Thanks! I've gleaned the following from your blog. Is this > correct? > > * A week ago you committed a change that will: > ** get current SMART parameters and faults for SATA on x86 via a single > function in a private library using SCSI emulation; > ** decide whether they indicate any problem I need to be aware of in > terms of over temperature, predictive failure, and self-test failure; > ** periodically check for above problems and generate an FMA ereport and > fault. Yep, that's about the gist of it. > I haven't used FMA yet, but this all sounds like just what I'm looking > for! Questions: > > * What Nevada build will this be in? This will be in build 65 of Nevada. > * No support for PATA? Nope. Only devices that use the SATA framework (Marvell, Silicon Image, and others - I don't remember the full list) use the SCSI emulation required to make this work. > * Do I need any special SATA configuration to get the SCSI emulation? No, but your SATA HBA must be one of those supported by the SATA framework. Otherwise it will operate in PATA legacy mode and the information will not be available. > * Does ZFS mirror boot work on SATA? Currently, there is a bug that prevents this from working. Basically, ZFS requires device IDs or /dev paths to open devices. For some unknown reason, ldi_open_by_devid() doesn't work for SATA devices early in boot, and /dev is obviously not available. I've fixed this in my upcoming FMA wad by also storing the /devices path with the vdev and falling back to that. Your mileage may vary, but at least for marvell SATA devices, ZFS boot doesn't work at the moment. > * Self-test failures are reported, but self-tests cannot be run? They are not explicitly scheduled by software. The drive firmware itself often runs tests independent of any software control. > * Is there a utility to output the raw disk_status_get() results? No. At one point I had an option to fmtopo to dump the output but I ripped it out during codereview because there had been plans to port the method to become a property. Now the fate is somewhat up in the air. In the meantime, a 20-line C program will do the trick (use nvlist_print to print the results). Sorry about that. > * Are there HDD-model specific configurations for SMART parameters? Not currently. Hope that helps, - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS Send/RECV
I'm trying to test an install of ZFS to see if I can backup data from one machine to another. I'm using Solaris 5.10 on two VMware installs. When I do the zfs send | ssh zfs recv part, the file system (folder) is getting created, but none of the data that I have in my snapshot is sent. I can browse on the source machine to view the snapshot data pool/.zfs/snapshot/snap-name and I see the data. Am I missing something to make it copy all of the data? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SMART
Excellent! Thanks! I've gleaned the following from your blog. Is this correct? * A week ago you committed a change that will: ** get current SMART parameters and faults for SATA on x86 via a single function in a private library using SCSI emulation; ** decide whether they indicate any problem I need to be aware of in terms of over temperature, predictive failure, and self-test failure; ** periodically check for above problems and generate an FMA ereport and fault. I haven't used FMA yet, but this all sounds like just what I'm looking for! Questions: * What Nevada build will this be in? * No support for PATA? * Do I need any special SATA configuration to get the SCSI emulation? * Does ZFS mirror boot work on SATA? * Self-test failures are reported, but self-tests cannot be run? * Is there a utility to output the raw disk_status_get() results? * Are there HDD-model specific configurations for SMART parameters? Cheers, 11011011 Eric Schrock <[EMAIL PROTECTED]> wrote on Friday, June 01, 2007 11:28:50: See: http://blogs.sun.com/eschrock/entry/solaris_platform_integration_generic_disk Prior to the above work, we only monitored disks on Thumper (x4500) platforms. With these changes we monitor basic SMART data for SATA drives. Monitoring for SCSI drives will be here soon. The next step will be tying this information into ZFS diagnosis to have a coherent diagnosis strategy (currently ZFS vdevs and Solaris devices are diagnosed as independent faults). Also in codereview right now is FMA work to proactively diagnose faulty drives based on I/O and checksum errors as seen by ZFS. Hope that helps, - Eric On Fri, Jun 01, 2007 at 11:20:02AM -1000, J. David Beutel wrote: On Solaris x86, does zpool (or anything) support PATA (or SATA) IDE SMART data? With the Predictive Self Healing feature, I assumed that Solaris would have at least some SMART support, but what I've googled so far has been discouraging. http://prefetch.net/blog/index.php/2006/10/29/solaris-needs-smart-support-please-help/ Bug ID: 4665068 SMART support in IDE driver http://bugs.opensolaris.org/view_bug.do?bug_id=4665068 Bug ID: 6280687 Collect SMART data from disks and deliver info to FMA http://bugs.opensolaris.org/view_bug.do?bug_id=6280687 If it's not automated into ZFS or FMA yet, then smartmontools would be enough for me. Did Sun's endowment of storage related code to OpenSolaris in April happen to include the dad driver (the SPARC IDE driver) that contains the missing ioctls for SMART that Matty thought could be ported to the cmdk driver (the x86 Solaris IDE driver)? http://www.itjungle.com/tug/tug042607-story08.html I'm a Solaris newbie trying Nevada b62 now that it supports ZFS mirror boot. The last time I installed Solaris, it was called SunOS. Cheers, 11011011 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Success: Re: [zfs-discuss] Re: I seem to have backed myself into a corner - how do I migrate filesyst
Mark J Musante wrote: Note that if you use the recursive snapshot and destroy, only one line is My "problem" (and it really is /not/ an important one) was that I had a cron job that every minute did min=`date "+%d"` snap="$pool/[EMAIL PROTECTED]" zfs destroy "$snap" zfs snapshot "$snap" and, after a couple of days (at 86 thousand minutes/day), the pool's history log seemed quite full (but not at capacity...) There were no clones to complicate things... -John ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SMART
See: http://blogs.sun.com/eschrock/entry/solaris_platform_integration_generic_disk Prior to the above work, we only monitored disks on Thumper (x4500) platforms. With these changes we monitor basic SMART data for SATA drives. Monitoring for SCSI drives will be here soon. The next step will be tying this information into ZFS diagnosis to have a coherent diagnosis strategy (currently ZFS vdevs and Solaris devices are diagnosed as independent faults). Also in codereview right now is FMA work to proactively diagnose faulty drives based on I/O and checksum errors as seen by ZFS. Hope that helps, - Eric On Fri, Jun 01, 2007 at 11:20:02AM -1000, J. David Beutel wrote: > On Solaris x86, does zpool (or anything) support PATA (or SATA) IDE > SMART data? With the Predictive Self Healing feature, I assumed that > Solaris would have at least some SMART support, but what I've googled so > far has been discouraging. > > http://prefetch.net/blog/index.php/2006/10/29/solaris-needs-smart-support-please-help/ > > Bug ID: 4665068 SMART support in IDE driver > http://bugs.opensolaris.org/view_bug.do?bug_id=4665068 > > Bug ID: 6280687 Collect SMART data from disks and deliver info to FMA > http://bugs.opensolaris.org/view_bug.do?bug_id=6280687 > > If it's not automated into ZFS or FMA yet, then smartmontools would be > enough for me. Did Sun's endowment of storage related code to > OpenSolaris in April happen to include the dad driver (the SPARC IDE > driver) that contains the missing ioctls for SMART that Matty thought > could be ported to the cmdk driver (the x86 Solaris IDE driver)? > http://www.itjungle.com/tug/tug042607-story08.html > > I'm a Solaris newbie trying Nevada b62 now that it supports ZFS mirror > boot. The last time I installed Solaris, it was called SunOS. > > Cheers, > 11011011 > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Success: Re: [zfs-discuss] Re: I seem to have backed myself into a corner - how do I migrate filesyst
On Fri, 1 Jun 2007, John Plocher wrote: > This seems especially true when there is closure on actions - the set of > zfs snapshot foo/[EMAIL PROTECTED] > zfs destroy foo/[EMAIL PROTECTED] > commands is (except for debugging zfs itself) a noop Note that if you use the recursive snapshot and destroy, only one line is entered into the history for all filesystems. Regards, markm ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] SMART
On Solaris x86, does zpool (or anything) support PATA (or SATA) IDE SMART data? With the Predictive Self Healing feature, I assumed that Solaris would have at least some SMART support, but what I've googled so far has been discouraging. http://prefetch.net/blog/index.php/2006/10/29/solaris-needs-smart-support-please-help/ Bug ID: 4665068 SMART support in IDE driver http://bugs.opensolaris.org/view_bug.do?bug_id=4665068 Bug ID: 6280687 Collect SMART data from disks and deliver info to FMA http://bugs.opensolaris.org/view_bug.do?bug_id=6280687 If it's not automated into ZFS or FMA yet, then smartmontools would be enough for me. Did Sun's endowment of storage related code to OpenSolaris in April happen to include the dad driver (the SPARC IDE driver) that contains the missing ioctls for SMART that Matty thought could be ported to the cmdk driver (the x86 Solaris IDE driver)? http://www.itjungle.com/tug/tug042607-story08.html I'm a Solaris newbie trying Nevada b62 now that it supports ZFS mirror boot. The last time I installed Solaris, it was called SunOS. Cheers, 11011011 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Success: Re: [zfs-discuss] Re: I seem to have backed myself into a corner - how do I migrate filesyst
On Fri, Jun 01, 2007 at 02:09:55PM -0700, John Plocher wrote: > eric kustarz wrote: > >We specifically didn't allow the admin the ability to truncate/prune the > >log as then it becomes unreliable - ooops i made a mistake, i better > >clear the log and file the bug against zfs > > I understand - auditing means never getting to blame someone else :-) > > There are things in the log that are (IMhO, and In My Particular Case) > more important than others. Snapshot creations & deletions are "noise" > compared with filesystem creations, property settings, etc. But clone creation == filesystem creation, and since you can only clone snapshots you'd want snapshotting included in the log, at least the ones referenced by live clones. Or if there was a pivot and the old fs and snapshot were destroyed you might still want to know about that. I think there has to be a way to truncate/filter the log, at least by date. > This seems especially true when there is closure on actions - the set of > zfs snapshot foo/[EMAIL PROTECTED] > zfs destroy foo/[EMAIL PROTECTED] > commands is (except for debugging zfs itself) a noop Yes, but it could be very complicated: zfs snapshot foo/[EMAIL PROTECTED] zfs clone foo/[EMAIL PROTECTED] foo/bar-then zfs clone foo/[EMAIL PROTECTED] foo/bar-then-again zfs snapshot foo/[EMAIL PROTECTED] zfs clone foo/[EMAIL PROTECTED] foo/bar-then-and-then zfs destroy -r foo/[EMAIL PROTECTED] ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Success: Re: [zfs-discuss] Re: I seem to have backed myself into a corner - how do I migrate filesyst
On Jun 1, 2007, at 2:09 PM, John Plocher wrote: eric kustarz wrote: We specifically didn't allow the admin the ability to truncate/ prune the log as then it becomes unreliable - ooops i made a mistake, i better clear the log and file the bug against zfs I understand - auditing means never getting to blame someone else :-) :) There are things in the log that are (IMhO, and In My Particular Case) more important than others. Snapshot creations & deletions are "noise" compared with filesystem creations, property settings, etc. This seems especially true when there is closure on actions - the set of zfs snapshot foo/[EMAIL PROTECTED] zfs destroy foo/[EMAIL PROTECTED] commands is (except for debugging zfs itself) a noop Looking at history.c, it doesn't look like there is an easy way to mark a set of messages as "unwanted" and compress the log without having to take the pool out of service first. Right, you'll have to do any post-processing yourself (something like a script + cron job). eric ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Success: Re: [zfs-discuss] Re: I seem to have backed myself into a corner - how do I migrate filesyst
eric kustarz wrote: We specifically didn't allow the admin the ability to truncate/prune the log as then it becomes unreliable - ooops i made a mistake, i better clear the log and file the bug against zfs I understand - auditing means never getting to blame someone else :-) There are things in the log that are (IMhO, and In My Particular Case) more important than others. Snapshot creations & deletions are "noise" compared with filesystem creations, property settings, etc. This seems especially true when there is closure on actions - the set of zfs snapshot foo/[EMAIL PROTECTED] zfs destroy foo/[EMAIL PROTECTED] commands is (except for debugging zfs itself) a noop Looking at history.c, it doesn't look like there is an easy way to mark a set of messages as "unwanted" and compress the log without having to take the pool out of service first. Oh well... -John ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Success: Re: [zfs-discuss] Re: I seem to have backed myself into a corner - how do I migrate filesyst
2) Following Chris's advice to do more with snapshots, I played with his cron-triggered snapshot routine: http://blogs.sun.com/chrisg/entry/snapping_every_minute Now, after a couple of days, zpool history shows almost 100,000 lines of output (from all the snapshots and deletions...) How can I purge or truncate this log (which has got to be taking up several Mb of space, not to mention the ever increasing sluggishness of the command...) You can check out the comment at the head of spa_history.c: http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/ common/fs/zfs/spa_history.c The history is implemented as a ring buffer (where the size is MIN (32MB, 1 %of your capacity)): http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/ common/fs/zfs/spa_history.c#105 We specifically didn't allow the admin the ability to truncate/prune the log as then it becomes unreliable - ooops i made a mistake, i better clear the log and file the bug against zfs eric ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Thoughts on CF/SSDs [was: ZFS - Use h/w raid or not?Thoughts.Considerations.]
Hello Richard, Thursday, May 31, 2007, 10:59:04 PM, you wrote: >> >> Having 2 cards would certainly make the "unlikely replacement" of a card >> a LOT more straight-forward than a single-card failure... Much of this >> would depend on the quality of these CF-cards and how they put up under >> load/stress/time RE> Disagree. With two cards, you have to implement software mirroring of RE> some sort. While ZFS is a step in the right direction (simplifying the RE> process) it is unproven for long term system administration. The costs RE> of implementing software mirroring occur in the complexity of managing RE> the software environment over time as upgrades and patches occur. RE> Reliability tends to trump availability for this reason. I don't know - I've been using SVM to mirror boot disks for years on several servers and I belive management is better than dealing with different PCI RAID cards, different BIOS'es (on-board RAID), different tools, different failure scenarios, etc. Or maybe you were thinking no-raid-at-all vs. mirror... -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re[2]: [zfs-discuss] Re: ZFS - Use h/w raid or not? Thoughts. Considerations.
Hello Richard, RE> But I am curious as to why you believe 2x CF are necessary? RE> I presume this is so that you can mirror. But the remaining memory RE> in such systems is not mirrored. Comments and experiences are welcome. I was thinking about mirroring - it's not clear from the comment above why it is not needed? -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Success: Re: [zfs-discuss] Re: I seem to have backed myself into a corner - how do I migrate filesyst
I managed to correct the problem by writing a script inspired by Chris Gerhard's blog that did a zfs send | zfs recv. Now that things are back up, I have a couple of lingering questions: 1) I noticed that the filesystem size information is not the same between the src and dst filesystem sets. Is this an expected behavior? [EMAIL PROTECTED]> zfs list -r tank/projects/sac NAMEUSED AVAIL REFER MOUNTPOINT tank/projects/sac 49.0G 218G 48.7G /export/sac tank/projects/[EMAIL PROTECTED] 104M - 48.7G - tank/projects/[EMAIL PROTECTED] 96.7M - 48.7G - tank/projects/[EMAIL PROTECTED] 74.3M - 48.7G - tank/projects/[EMAIL PROTECTED] 18.7M - 48.7G - [EMAIL PROTECTED]> zfs list -r tank2/projects/sac NAME USED AVAIL REFER MOUNTPOINT tank2/projects/sac 49.3G 110G 48.6G /export2/sac tank2/projects/[EMAIL PROTECTED] 99.7M - 48.6G - tank2/projects/[EMAIL PROTECTED] 92.3M - 48.6G - tank2/projects/[EMAIL PROTECTED] 70.1M - 48.6G - tank2/projects/[EMAIL PROTECTED] 70.7M - 48.6G - 2) Following Chris's advice to do more with snapshots, I played with his cron-triggered snapshot routine: http://blogs.sun.com/chrisg/entry/snapping_every_minute Now, after a couple of days, zpool history shows almost 100,000 lines of output (from all the snapshots and deletions...) How can I purge or truncate this log (which has got to be taking up several Mb of space, not to mention the ever increasing sluggishness of the command...) -John Oh, here's the script I used - it contains hardcoded zpool and zfs info, so it must be edited to match your specifics before it is used! It can be rerun safely; it only sends snapshots that haven't already been sent so that I could do the initial time-intensive copies while the system was still in use and only have to do a faster "resync" while down in single user mode. It isn't pretty (it /is/ a perl script) but it worked :-) -- #!/usr/bin/perl # John Plocher - May, 2007 # ZFS helper script to replicate the filesystems+snapshots in # SRCPOOL onto a new DSTPOOL that was a different size. # # Historical situation: # + zpool create tank raidz c1t1d0 c1t2d0 c1t3d0 # + zfs create tank/projects # + zfs set mountpoint=/export tank/projects # + zfs set sharenfs=on tank/projects # + zfs create tank/projects/... # ... fill up the above with data... # Drive c1t3d0 FAILED # + zpool offline tank c1t3d0 # ... find out that replacement drive is 10,000 sectors SMALLER # ... than the original, and zpool replace won't work with it. # # Usage Model: # Create a new (temp) pool large enough to hold all the data # currently on tank # + zpool create tank2 c2t2d0 c2t3d0 c2t4do # + zfs set mountpoint=/export2 tank2/projects # Set a baseline snapshot on tank # + zfs snapshot -r [EMAIL PROTECTED] # Edit and run this script to copy the data + filesystems from tank to # the new pool tank2 # + ./copyfs # Drop to single user mode, unshare the tank filesystems, # + init s # + zfs unshare tank # Shut down apache, cron and sendmail # + svcadm disable svc:/network/http:cswapache2 # + svcadm disable svc:/system/cron:default # + svcadm disable svc:/network/smtp:sendmail # Take another snapshot, # + zfs snapshot -r [EMAIL PROTECTED] # Rerun script to catch recent changes # + ./copyfs # Verify that the copies were successful, # + dircmp -s /export/projects /export2/projects # + zfs destroy tank # + zpool create tank raidz c1t1d0 c1t2d0 c1t3d0 # Modify script to reverse transfer and set properties, then # run script to recreate tank's filesystems, # + ./copyfs # Reverify that content is still correct # + dircmp -s /export/projects /export2/projects # Re-enable cron, http and mail # + svcadm enable svc:/network/http:cswapache2 # + svcadm enable svc:/system/cron:default # + svcadm enable svc:/network/smtp:sendmail # Go back to multiuser # + init 3 # Reshare filesystems. # + zfs share tank # Go home and get some sleep # $SRCPOOL="tank"; $DSTPOOL="tank2"; # Set various properties once the initial filesystem is recv'd... # (Uncomment these when copying the filesystems back to their original pool) # $props{"projects"} = (); # push( @{ $props{"projects"} }, ("zfs set mountpoint=/export tank/projects")); # push( @{ $props{"projects"} }, ("zfs set sharenfs=on tank/projects")); # $props{"projects/viper"} = (); # push( @{ $props{"projects/viper"} }, ("zfs set sharenfs=rw=bk-test:eressea:scuba:sac:viper:caboose,root=sac:viper:caboose,ro tank/projects/viper")); sub getsnapshots(@) { my (@filesystems) = @_; my @snaps; my @snapshots; foreach my $fs ( @filesystems ) { chomp($fs); next if ($fs eq $SRCPOOL); # print "Filesystem: $fs\n"; # Get a lis
Re: [zfs-discuss] Re: ZFS consistency guarantee
> If i put the database in hotbackup mode,then i will have to ensure > that the filesystem is consistent as well.So, you are saying that > taking a ZFS snapshot is the only method to guarantee consistency in > the filesystem since it flushes all the buffers to the filesystem , so > its consistent. The ZFS filesystem is always consistent on disk. By taking a snapshot, you simply make a consistent copy of the filesystem available. Flushing buffers would be a way of making sure all the writes have made it to storage. That's a different statement than consistency. > Just curious,is there any manual way of telling ZFS to flush the > buffers after queiscing the db other than taking a ZFS snapshot?. The usual methods of doing this on a filesystem are to run sync, or call fsync(), but that's not anything specific to ZFS. If you're not taking a snapshot, why would you want ZFS to flush the buffers? -- Darren Dunham [EMAIL PROTECTED] Senior Technical Consultant TAOShttp://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: ZFS consistency guarantee
Both levels, application and filesystem. If i put the database in hotbackup mode,then i will have to ensure that the filesystem is consistent as well.So, you are saying that taking a ZFS snapshot is the only method to guarantee consistency in the filesystem since it flushes all the buffers to the filesystem , so its consistent. Just curious,is there any manual way of telling ZFS to flush the buffers after queiscing the db other than taking a ZFS snapshot?. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: vxfs and zfs
Yes they can. -- Fred benita ulisano wrote: Hi, I would like to clarify one point for the forum experts on what I would like to do after it was brought to my attention that my posting might not describe a true picture of what I am trying to accomplish. All I want to do is setup a separate zfs file system running Oracle on the machine running our vxfs file systems running Oracle on the same server so that I can compare ease of use, and efficiency. The instance would be a clone so that the volume of data and queries would be similar in size. All I am asking is if zfs and vxfs file systems can run simultaneously on the same server. Hope that helps and a big thank you for all of the responses I have received so far at my email account! I appreciate it very much. Happyadm This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Fred Zlotnick Director, Solaris Data Technology Sun Microsystems, Inc. [EMAIL PROTECTED] x85006/+1 650 786 5006 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] [Fwd: zone mount points are busy following reboot of global zone 65505676]
Original Message Subject: zone mount points are busy following reboot of global zone 65505676 Date: Fri, 01 Jun 2007 12:33:57 -0400 From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] <[EMAIL PROTECTED]> I need help on this customers issue. I appreciate any help that you can provide. If a zone with ZFS partitions is rebooted by itself there seems to be no problem. If the global zone is rebooted and no steps are taken to shutdown the non-global first the non-global zones mount points are left in place. The existence of the mount points inhibits the correct mount. System sees the device as 'busy' on the reboot. Not sure if this is a bug or anomaly of ZFS.. As to the mount examples - Lisbon is a V490 running the global zone. Morocco is a whole-root non-global zone build to Lisbon. zonecfg -z morocco info zonepath: /zones/morocco autoboot: true pool: fs: dir: /mnt special: /cdrom raw not specified type: lofs options: [ro,nodevices] net: address: 216.83.115.46 physical: ce2 dataset: name: pool0/morocco-home dataset: name: pool1/morocco-usr2 Pool0 - Home is a mirrored ZFS partition cut from the dual internal 146 GB drives. Pool1 is 2 x 146GB drives in a 3120 JBOD array configured as RAID0. Pool0 and Poll1 are dedicated to morocco. If the system crashes or the non-global zone is rebooted on its own, the system does not come up correctly. It will bring itself to single user mode and just sits. We have to zlogin to morocco and delete the /home and /usr2 directories from the root and then reboot. The system comes up fine after the reboot. -- Thanks! Have a good day! Claire Grandalski - OS Technical Support Engineer [EMAIL PROTECTED] (800)USA-4SUN (Reference your Case Id #) Hours 8:00 - 3:00 EST Sun Support Services 4 Network Drive, UBUR04-105 Burlington MA 01803-0902 -- Thanks! Have a good day! Claire Grandalski - OS Technical Support Engineer [EMAIL PROTECTED] (800)USA-4SUN (Reference your Case Id #) Hours 8:00 - 3:00 EST Sun Support Services 4 Network Drive, UBUR04-105 Burlington MA 01803-0902 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Thoughts on CF/SSDs [was: ZFS - Use h/w raid or not?Thoughts.Considerations.]
Frank Cusack wrote: On May 31, 2007 1:59:04 PM -0700 Richard Elling <[EMAIL PROTECTED]> wrote: CF cards aren't generally very fast, so the solid state disk vendors are putting them into hard disk form factors with SAS/SATA interfaces. These If CF cards aren't fast, how will putting them into a different form factor make them faster? Semiconductor memories are accessed in parallel. Spinning disks are accessed serially. Let's take a look at a few examples and see what this looks like... Disk iops bw atime MTBF UER endurance -- SanDisk 32 GByte 2.5" SATA 7,450 67 0.11 2,000,000 10^-20 ? SiliconSystems 8 GByte CF 500 8 2 4,000,000 10^-14 >2,000,000 SanDisk 8 GByte CF ? 40 ? ? ?? Seagate 146 GByte 2.5" SATA141 41-63 4.1+ 1,400,000 10^-15 - Hitachi 500 GByte 3.5" SATA 79 31-65 8.5+ 1,000,000 10^-14 - iops = small, random read iops (I/O operations per second) [higher is better] bw = sustained media read bandwidth (MBytes/s) [higher is better] atime = access time (milliseconds) [lower is better] MTBF = mean time between failures (hours) [higher is better] UER = unrecoverable read error rate (errors/bits read) [lower is better] endurance = single block rewrite count [higher is better] http://www.sandisk.com/Assets/File/pdf/oem/SanDisk_SSD_SATA_5000_2.5_DS_P03_DS.pdf http://www.storagesearch.com/ssd-16.html http://www.hitachigst.com/portal/site/en/menuitem.eb9838d4792cb564c0f85074eac4f0a0 http://www.seagate.com/docs/pdf/datasheet/disc/ds_savvio_10k_2.pdf It is a little bit frustrating that all vendors have different amounts of data available on their product which is publically available :-(. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: vxfs and zfs
Hi, I would like to clarify one point for the forum experts on what I would like to do after it was brought to my attention that my posting might not describe a true picture of what I am trying to accomplish. All I want to do is setup a separate zfs file system running Oracle on the machine running our vxfs file systems running Oracle on the same server so that I can compare ease of use, and efficiency. The instance would be a clone so that the volume of data and queries would be similar in size. All I am asking is if zfs and vxfs file systems can run simultaneously on the same server. Hope that helps and a big thank you for all of the responses I have received so far at my email account! I appreciate it very much. Happyadm This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] vxfs and zfs
benita ulisano wrote: Hi, I have been given the task to research converting our vxfs/vm file systems and volumes to zfs. The volumes are attached to an EMC Clariion running raid-5, and raid 1_0. I have no test machine, just a migration machine that currently hosts other things. It is possible to setup a zfs file system while vxfs/vm are still running and controlling other file systems and volumes, or is it all or nothing. I searched many blogs and web docs and cannot find the answer to this question. I'm not quite sure what you're asking here: do you want to set up zpools/zfs on the same disks as vxvm/vxfs is running on *while* vxvm/vxfs is still running on them? that won't work. If you're asking "can I set up zfs on free disks while vxvm is still set up on others" I don't see why not. As long as there's no contention around actual disks, there shouldn't be an issue here. If you expand a bit on this, I'm sure our zfs experts can give you a more precise answer than this :-) HTH -- Michael Schuster Recursion, n.: see 'Recursion' ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Deterioration with zfs performance and recent zfs bits?
> Patching zfs_prefetch_disable = 1 has helped It's my belief this mainly aids scanning metadata. my testing with rsync and yours with find (and seen with du & ; zpool iostat -v 1 ) pans this out.. mainly tracked in bug 6437054 vdev_cache: wise up or die http://www.opensolaris.org/jive/thread.jspa?messageID=42212 so to link your code, it might help, but if one ran a clean down the tree, it would hurt compile times. Rob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: Deterioration with zfs performance and recent zfs bits?
I wrote > Has anyone else noticed a significant zfs performance > deterioration when running recent opensolaris bits? > > My 32-bit / 768 MB Toshiba Tecra S1 notebook was able > to do a full opensolaris release build in ~ 4 hours 45 > minutes (gcc shadow compilation disabled; using an lzjb > compressed zpool / zfs on a single notebook hdd p-ata drive). > > After upgrading to 2007-05-25 opensolaris release > bits (compiled from source), the same release build now > needs ~ 6 hours; that's ~ 25% slower. It might be Bug ID 6469558 "ZFS prefetch needs to be more aware of memory pressure": http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6469558 Instead of compiling opensolaris for 4-6 hours, I've now used the following find / grep test using on-2007-05-30 sources: 1st test using Nevada build 60: % cd /files/onnv-2007-05-30 % repeat 10 /bin/time find usr/src/ -name "*.[hc]" -exec grep FooBar {} + usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:22.5 user3.3 sys 5.8 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:28.4 user3.3 sys 4.8 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:18.0 user3.3 sys 4.7 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:17.3 user3.3 sys 4.8 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:15.0 user3.3 sys 4.7 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:12.0 user3.3 sys 4.7 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:21.9 user3.3 sys 4.7 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:18.7 user3.3 sys 4.7 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:19.5 user3.3 sys 4.7 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:17.2 user3.3 sys 4.7 Same test, but running onnv-2007-05-30 release bits (compiled from source). This is at least 25% slower than snv_60: (Note: zfs_prefetch_disable = 0 , the default value) % repeat 10 /bin/time find usr/src/ -name "*.[hc]" -exec grep FooBar {} + usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 8:04.3 user7.3 sys13.2 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 6:34.4 user7.3 sys11.2 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 6:33.8 user7.3 sys11.1 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 5:35.6 user7.3 sys10.6 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 5:39.8 user7.3 sys10.6 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 5:37.8 user7.3 sys11.1 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 5:53.5 user7.3 sys11.0 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 5:45.2 user7.3 sys11.1 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 5:44.8 user7.3 sys11.0 usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 5:49.1 user7.3 sys11.0 Then I patched zfs_prefetch_disable/W1, and now the find & grep test runs much faster on onnv-2007-05-30 bits: (Note: zfs_prefetch_disable = 1) % repeat 10 /bin/time find usr/src/ -name "*.[hc]" -exec grep FooBar {} + usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while real 4:01.3 user7.2 sys 9.9 usr/src/li
Re: [zfs-discuss] zfs migration
zpool replace == zpool attach + zpool detach It is not a good practice to detach and then attach as you are vulnerable after the detach and before the attach completes. It is a good practice to attach and then detach. There is no practical limit to the number of sides of a mirror in ZFS. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] vxfs and zfs
Hi, I have been given the task to research converting our vxfs/vm file systems and volumes to zfs. The volumes are attached to an EMC Clariion running raid-5, and raid 1_0. I have no test machine, just a migration machine that currently hosts other things. It is possible to setup a zfs file system while vxfs/vm are still running and controlling other file systems and volumes, or is it all or nothing. I searched many blogs and web docs and cannot find the answer to this question. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs migration
On Fri, 1 Jun 2007, Krzys wrote: > bash-3.00# zpool replace mypool c1t2d0 emcpower0a > bash-3.00# zpool status >pool: mypool > state: ONLINE > status: One or more devices is currently being resilvered. The pool will > continue to function, possibly in a degraded state. > action: Wait for the resilver to complete. > scrub: resilver in progress, 0.00% done, 17h50m to go > config: > > NAMESTATE READ WRITE CKSUM > mypool ONLINE 0 0 0 >replacing ONLINE 0 0 0 > c1t2d0 ONLINE 0 0 0 > emcpower0a ONLINE 0 0 0 I don't think this is what you want. Notice that it is in the process of replacing c1t2d0 with emcpower0a. Once the replacing operation is complete, c1t2d0 will be removed from the configuration. You've got two options. Let's say your current mirror is c1t2d0 and c1t3d0, and you want to replace c1t3d0 with emcpower0a. Option one: perform a direct replace: # zpool replace mypool c1t3d0 emcpower0a Option two: remove c1t3d0 and add in emcpower0a: # zpool detach mypool c1t3d0 # zpool attach mypool c1t2d0 emcpower0a Do not mix these two options, as you showed in your email. Do not perform a 'detach' followed by a 'replace'. This is mixing your options and you will end up with a config you were not expecting. Regards, markm ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: current state of play with ZFS boot and install?
Thankyou Lori, that's fantastic. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs migration
Ok, now its seems like its working what I wanted to do: bash-3.00# zpool status pool: mypool state: ONLINE scrub: resilver completed with 0 errors on Thu May 31 23:01:09 2007 config: NAMESTATE READ WRITE CKSUM mypool ONLINE 0 0 0 mirrorONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool detach mypool c1t3d0 bash-3.00# zpool status pool: mypool state: ONLINE scrub: resilver completed with 0 errors on Thu May 31 23:01:09 2007 config: NAMESTATE READ WRITE CKSUM mypool ONLINE 0 0 0 c1t2d0ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool replace mypool c1t2d0 emcpower0a bash-3.00# zpool status pool: mypool state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scrub: resilver in progress, 0.00% done, 17h50m to go config: NAMESTATE READ WRITE CKSUM mypool ONLINE 0 0 0 replacing ONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 emcpower0a ONLINE 0 0 0 errors: No known data errors bash-3.00# thank you everyone who helped me with this... Chris On Fri, 1 Jun 2007, Will Murnane wrote: On 6/1/07, Krzys <[EMAIL PROTECTED]> wrote: bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT mypool 68G 53.1G 14.9G78% ONLINE - mypool2 123M 83.5K123M 0% ONLINE - Are you sure you've allocated as large a LUN as you thought initially? Perhaps ZFS is doing something funky with it; does putting UFS on it show a large filesystem or a small one? Will !DSPAM:122,46601749220211363223461! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs migration
yeah it does something funky that I did not expect, zpool seems like its taking slice 0 of that emc lun rather than taking the whole device... so when I did create that lun, I formated disk and it looked like this: format> verify Primary label contents: Volume name = <> ascii name = pcyl= 51200 ncyl= 51198 acyl=2 nhead = 256 nsect = 16 Part TagFlag Cylinders SizeBlocks 0 rootwm 0 -63 128.00MB(64/0/0) 262144 1 swapwu 64 - 127 128.00MB(64/0/0) 262144 2 backupwu 0 - 51197 100.00GB(51198/0/0) 209707008 3 unassignedwm 00 (0/0/0) 0 4 unassignedwm 00 (0/0/0) 0 5 unassignedwm 00 (0/0/0) 0 6usrwm 128 - 51197 99.75GB(51070/0/0) 209182720 7 unassignedwm 00 (0/0/0) 0 that is the reason when I was trying to replace the other disk zpool did take slice 0 of that disk which was 128mb and treated it as pool rather than taking the whole disk or slice 2 or whatever it does with normal devices... I have that system connected to EMC clarion and I am using powerpath software from emc to do multipathing and stuff... ehh.. will try to replace that device old internal disk with this one and lets see how that will work. thanks so much for help. Chris On Fri, 1 Jun 2007, Will Murnane wrote: On 6/1/07, Krzys <[EMAIL PROTECTED]> wrote: bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT mypool 68G 53.1G 14.9G78% ONLINE - mypool2 123M 83.5K123M 0% ONLINE - Are you sure you've allocated as large a LUN as you thought initially? Perhaps ZFS is doing something funky with it; does putting UFS on it show a large filesystem or a small one? Will !DSPAM:122,46601749220211363223461! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs migration
ok, I think I did figure out what is the problem well what zpool does for that emc powerpath is it takes parition 0 from disk and is trying to attach it to my pool, so when I added emcpower0a I got the following: bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT mypool 68G 53.1G 14.9G78% ONLINE - mypool2 123M 83.5K123M 0% ONLINE - because my emcpower0a structure looked like this: format> verify Primary label contents: Volume name = <> ascii name = pcyl= 51200 ncyl= 51198 acyl=2 nhead = 256 nsect = 16 Part TagFlag Cylinders SizeBlocks 0 rootwm 0 -63 128.00MB(64/0/0) 262144 1 swapwu 64 - 127 128.00MB(64/0/0) 262144 2 backupwu 0 - 51197 100.00GB(51198/0/0) 209707008 3 unassignedwm 00 (0/0/0) 0 4 unassignedwm 00 (0/0/0) 0 5 unassignedwm 00 (0/0/0) 0 6usrwm 128 - 51197 99.75GB(51070/0/0) 209182720 7 unassignedwm 00 (0/0/0) 0 So what I did I changed my layout to look like this: Part TagFlag Cylinders SizeBlocks 0 rootwm 0 - 51197 100.00GB(51198/0/0) 209707008 1 swapwu 00 (0/0/0) 0 2 backupwu 0 - 51197 100.00GB(51198/0/0) 209707008 3 unassignedwm 00 (0/0/0) 0 4 unassignedwm 00 (0/0/0) 0 5 unassignedwm 00 (0/0/0) 0 6usrwm 00 (0/0/0) 0 7 unassignedwm 00 (0/0/0) 0 created new pool and I have the following: bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT mypool 68G 53.1G 14.9G78% ONLINE - mypool299.5G 80K 99.5G 0% ONLINE - so now I will try to replace it... I guess zpool does treat differently devices and in particular the ones that are under emc powerpath controll which is using the first slice of that disk to create pool and not the whole device... Anyway thanks to everyone for help, now that replace should work... I am going to try it now. Chris On Fri, 1 Jun 2007, Will Murnane wrote: On 5/31/07, Krzys <[EMAIL PROTECTED]> wrote: so I do run replace command and I get and error: bash-3.00# zpool replace mypool c1t2d0 emcpower0a cannot replace c1t2d0 with emcpower0a: device is too small Try "zpool attach mypool emcpower0a"; see http://docs.sun.com/app/docs/doc/819-5461/6n7ht6qrt?a=view . Will !DSPAM:122,465fa1d813332148481500! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs migration
On 6/1/07, Krzys <[EMAIL PROTECTED]> wrote: bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT mypool 68G 53.1G 14.9G78% ONLINE - mypool2 123M 83.5K123M 0% ONLINE - Are you sure you've allocated as large a LUN as you thought initially? Perhaps ZFS is doing something funky with it; does putting UFS on it show a large filesystem or a small one? Will ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs migration
Never the less I get the following error: bash-3.00# zpool attach mypool emcpower0a missing specification usage: attach [-f] bash-3.00# zpool status pool: mypool state: ONLINE scrub: resilver completed with 0 errors on Thu May 31 23:01:09 2007 config: NAMESTATE READ WRITE CKSUM mypool ONLINE 0 0 0 mirrorONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool attach mypool c1t2d0 emcpower0a cannot attach emcpower0a to c1t2d0: device is too small bash-3.00# Is there anyway to add that emc san to zfs at all? It seems like that emcpower0a cannot be added in any way... but check this out, I did try to add it in as a new pool and here is what I got: bash-3.00# zpool create mypool2 emcpower0a bash-3.00# zpool status pool: mypool state: ONLINE scrub: resilver completed with 0 errors on Thu May 31 23:01:09 2007 config: NAMESTATE READ WRITE CKSUM mypool ONLINE 0 0 0 mirrorONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 errors: No known data errors pool: mypool2 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM mypool2 ONLINE 0 0 0 emcpower0a ONLINE 0 0 0 errors: No known data errors bash-3.00# zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT mypool 68G 53.1G 14.9G78% ONLINE - mypool2 123M 83.5K123M 0% ONLINE - bash-3.00# On Fri, 1 Jun 2007, Will Murnane wrote: On 5/31/07, Krzys <[EMAIL PROTECTED]> wrote: so I do run replace command and I get and error: bash-3.00# zpool replace mypool c1t2d0 emcpower0a cannot replace c1t2d0 with emcpower0a: device is too small Try "zpool attach mypool emcpower0a"; see http://docs.sun.com/app/docs/doc/819-5461/6n7ht6qrt?a=view . Will !DSPAM:122,465fa1d813332148481500! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs migration
Yes by my goal is to replace exisiting disk which is internal disk 72gb with SAN storage disk which is 100GB in size... As long as I will be able to detach the old one then its going to be great... otherwise I will be stuck with one internal disk and oneSAN disk which I do not like that much to have. Regards, Chris On Fri, 1 Jun 2007, Will Murnane wrote: On 5/31/07, Krzys <[EMAIL PROTECTED]> wrote: so I do run replace command and I get and error: bash-3.00# zpool replace mypool c1t2d0 emcpower0a cannot replace c1t2d0 with emcpower0a: device is too small Try "zpool attach mypool emcpower0a"; see http://docs.sun.com/app/docs/doc/819-5461/6n7ht6qrt?a=view . Will !DSPAM:122,465fa1d813332148481500! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss