Re: [zfs-discuss] SAS/short stroking vs. SSDs for ZIL
You do seem to misunderstand ZIL. ZIL is quite simply write cache and using a short stroked rotating drive is never going to provide a performance increase that is worth talking about and more importantly ZIL was designed to be used with a RAM/Solid State Disk. We use sata2 *HyperDrive5* RAM disks in mirrors and they work well and are far cheaper than STEC or other enterprise SSD's and have non of the issue related to trim... Highly recommended... ;-) http://www.hyperossystems.co.uk/ Kevin On 29 December 2010 13:40, Edward Ned Harvey < opensolarisisdeadlongliveopensola...@nedharvey.com> wrote: > > From: Bob Friesenhahn [mailto:bfrie...@simple.dallas.tx.us] > > Sent: Tuesday, December 28, 2010 9:23 PM > > > > > The question of IOPS here is relevant to conversation because of ZIL > > > dedicated log. If you have advanced short-stroking to get the write > latency > > > of a log device down to zero, then it can compete against SSD for > purposes > > > of a log device, but nobody seems to believe such technology currently > > > exists, and it certainly couldn't compete against SSD for random reads. > > > (ZIL log is the only situation I know of, where write performance of a > drive > > > matters and read performance does not matter.) > > > > It seems that you may be confused. For the ZIL the drive's rotational > > latency (based on RPM) is the dominating factor and not the lateral > > head seek time on the media. In this case, the "short-stroking" you > > are talking about does not help any. The ZIL is already effectively > > "short-stroking" since it writes in order. > > Nope. I'm not confused at all. I'm making a distinction between "short > stroking" and "advanced short stroking." Where simple "short stroking" > does > as you said - eliminates the head seek time but still susceptible to > rotational latency. As you said, the ZIL already effectively accomplishes > that end result, provided a dedicated spindle disk for log device, but does > not do that if your ZIL is on the pool storage. And what I'm calling > "advanced short stroking" are techniques that effectively eliminate, or > minimize both seek & latency, to zero or near-zero. What I'm calling > "advanced short stroking" doesn't exist as far as I know, but is > theoretically possible through either special disk hardware or special > drivers. > > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Swapping disks in pool to facilitate pool growth
Hi Guys, We are a running a Solaris 10 production server being used for backup services within our DC. We have 8 500GB drives in a zpool and we wish to swap them out 1 by 1 for 1TB drives. I would like to know if it is viable to add larger disks to zfs pool to grow the pool size and then remove the smaller disks? I would assume this would degrade the pool and require it to resilver? Any advice would be gratefully received. Kind regards Kevin ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS development moving behind closed doors
To be fair, he did talk some sense about how everyone was claiming to have a product that was cloud computing, but I still don't like Oracle. With there current Java Patent war with Google and now this with OpenSolaris, it leaves a very bad taste in my mouth. Will this affect ZFS being used in FreeBSD? On 15 August 2010 15:13, David Magda wrote: > On Aug 14, 2010, at 19:39, Kevin Walker wrote: > > I once watched a video interview with Larry from Oracle, this ass rambled >> on >> about how he hates cloud computing and that everyone was getting into >> cloud >> computing and in his opinion no one understood cloud computing, apart from >> him... :-| >> > > If this is the video you're talking about, I think you misinterpreted what > he meant: > > Cloud computing is not only the future of computing, but it is the >> present, and the entire past of computing is all cloud. [...] All it is is a >> computer connected to a network. What do you think Google runs on? Do you >> think they run on water vapour? It's databases, and operating systems, and >> memory, and microprocessors, and the Internet. And all of a sudden it's none >> of that, it's "the cloud". [...] All "the cloud" is, is computers on a >> network, in terms of technology. In terms of business model, you can say >> it's rental. All SalesForce.com was, before they were cloud computing, was >> software-as-a-service, and then they became cloud computing. [...] Our >> industry is so bizarre: they change a term and think they invented >> technology. >> > > http://www.youtube.com/watch?v=rmrxN3GWHpM#t=45m > > I don't see any inaccurate in what said. > > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS development moving behind closed doors
I once watched a video interview with Larry from Oracle, this ass rambled on about how he hates cloud computing and that everyone was getting into cloud computing and in his opinion no one understood cloud computing, apart from him... :-| From that day on I felt enlightened about Oracle and how they want do business; they are run by a CEO who is narrow minded and clearly doesn't understand Open Source or cloud computing and Oracle are very, very greedy... I only hope that OpenSolaris can live on the Illumos project and assist great projects such as Nexentastor. http://www.illumos.org/ K On 15 August 2010 00:02, Mark Bennett wrote: > On 8/13/10 8:56 PM -0600 Eric D. Mudama wrote: > > On Fri, Aug 13 at 19:06, Frank Cusack wrote: > >> Interesting POV, and I agree. Most of the many "distributions" of > >> OpenSolaris had very little value-add. Nexenta was the most interesting > >> and why should Oracle enable them to build a business at their expense? > > > > These distributions are, in theory, the "gateway drug" where people > > can experiment inexpensively to try out new technologies (ZFS, dtrace, > > crossbow, comstar, etc.) and eventually step up to Oracle's "big iron" > > as their business grows. > > >I've never understood how OpenSolaris was supposed to get you to Solaris. > >OpenSolaris is for enthusiasts and great great folks like Nexenta. > >Solaris lags so far behind it's not really an upgrade path. > > Fedora is a great beta test arena for what eventually becomes a commercial > Enterprise offering. OpenSolaris was the Solaris equivalent. > > Losing the free bleeding edge testing community will no doubt impact on the > Solaris code quality. > > It is now even more likely Solaris will revert to it's niche on SPARC over > the next few years. > > Mark. > -- > This message posted from opensolaris.org > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] raidz2 drive failure zpool will not import
Thanks Richard, I tried removing the replacement drive and received the same error. Output of zdb -l /dev/rdsk/c5d1s0 results in: ke...@opensolaris:~# zdb -l /dev/rdsk/c5d1s0 cannot open '/dev/rdsk/c5d1s0': No such device or address All other drives have 4 readable labels 0-3 I even attempted the old trick of putting the failed drive in the freezer for an hour and it did spin up, but only for a minute and not long enough to be recognized by the system. Not sure what to try next. ~kevin -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] raidz2 drive failure zpool will not import
After attempting unsuccessfully to replace a failed drive in a 10 drive raidz2 array and reading as many forum entries as I could find I followed a suggestion to export and import the pool. In another attempt to import the pool I reinstalled the OS, but I have so far been unable to import the pool. Here is the output from format and zpool commands: ke...@opensolaris:~# zpool status pool: rpool state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM rpool ONLINE 0 0 0 c8d0s0ONLINE 0 0 0 errors: No known data errors ke...@opensolaris:~# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c4d0 /p...@0,0/pci8086,2...@1e/pci-...@1/i...@0/c...@0,0 1. c4d1 /p...@0,0/pci8086,2...@1e/pci-...@1/i...@0/c...@1,0 2. c5d0 /p...@0,0/pci8086,2...@1e/pci-...@1/i...@1/c...@0,0 3. c5d1 /p...@0,0/pci8086,2...@1e/pci-...@1/i...@1/c...@1,0 4. c6d0 /p...@0,0/pci8086,2...@1e/pci-...@2/i...@0/c...@0,0 5. c6d1 /p...@0,0/pci8086,2...@1e/pci-...@2/i...@0/c...@1,0 6. c7d0 /p...@0,0/pci8086,2...@1e/pci-...@2/i...@1/c...@0,0 7. c7d1 /p...@0,0/pci8086,2...@1e/pci-...@2/i...@1/c...@1,0 8. c8d0 /p...@0,0/pci-...@1f,1/i...@0/c...@0,0 9. c10d0 /p...@0,0/pci-...@1f,2/i...@0/c...@0,0 10. c11d0 /p...@0,0/pci-...@1f,2/i...@1/c...@0,0 Specify disk (enter its number): ^C ke...@opensolaris:~# zpool import pool: storage id: 18058787158441119951 state: UNAVAIL status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. see: http://www.sun.com/msg/ZFS-8000-EY config: storage UNAVAIL insufficient replicas raidz2-0 DEGRADED c4d0 ONLINE c4d1 ONLINE c5d0 ONLINE replacing-3 DEGRADED c5d1 ONLINE c5d1 FAULTED corrupted data c6d0 ONLINE c6d1 ONLINE c7d0 ONLINE c7d1 ONLINE c10d0ONLINE c11d0ONLINE ke...@opensolaris:~# zpool import -f pool: storage id: 18058787158441119951 state: UNAVAIL status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. see: http://www.sun.com/msg/ZFS-8000-EY config: storage UNAVAIL insufficient replicas raidz2-0 DEGRADED c4d0 ONLINE c4d1 ONLINE c5d0 ONLINE replacing-3 DEGRADED c5d1 ONLINE c5d1 FAULTED corrupted data c6d0 ONLINE c6d1 ONLINE c7d0 ONLINE c7d1 ONLINE c10d0ONLINE c11d0ONLINE ke...@opensolaris:~# zpool import -f storage cannot import 'storage': one or more devices is currently unavailable Destroy and re-create the pool from a backup source. Prior to exporting the pool I was able to offline the failed drive. Finally about a month ago I upgraded the zpool version to enable dedupe. The suggestions I have read include "playing with" the metadata and this is something I would need help with as I am just an "informed" user. I am hoping that as only one drive failed and this is a dual parity raid that there is someway to recover the pool. Thanks in advance, Kevin -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] vPool unavailable but RaidZ1 is online
I am trying to recover a raid set, there are only three drives that are part of the set. I attached a disk and discovered it was bad. It was never part of the raid set. The disk is now gone and when I try to import the pool I get the error listed below. Is there a chance to recover? TIA! Sun Microsystems Inc. SunOS 5.11 snv_112 November 2008 # zpool import pool: vpool id: 14231674658037629037 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://www.sun.com/msg/ZFS-8000-6X config: vpool UNAVAIL missing device raidz1ONLINE c0t0d0 ONLINE c0t1d0 ONLINE c0t2d0 ONLINE Additional devices are known to be part of this pool, though their exact configuration cannot be determined. # bash bash-3.2# zpool import -fF pool: vpool id: 14231674658037629037 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://www.sun.com/msg/ZFS-8000-6X config: vpool UNAVAIL missing device raidz1ONLINE c0t0d0 ONLINE c0t1d0 ONLINE c0t2d0 ONLINE Additional devices are known to be part of this pool, though their exact configuration cannot be determined. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + fsck
Hi all, Just subscribed to the list after a debate on our helpdesk lead me to the posting about ZFS corruption and the need for a fsck repair tool of some kind... Has there been any update on this? Kind regards, Kevin Walker Coreix Limited DDI: (+44) 0207 183 1725 ext 90 Mobile: (+44) 07960 967818 Fax: (+44) 0208 53 44 111 * This message is intended solely for the use of the individual or organisation to whom it is addressed. It may contain privileged or confidential information. If you are not the intended recipient, you should not use, copy, alter, or disclose the contents of this message * ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] strange performance drop of solaris 10/zfs
Hi
We have been using a Solaris 10 system (Sun-Fire-V245) for a while as
our primary file server. This is based on Solaris 10 06/06, plus
patches up to approx May 2007. It is a production machine, and until
about a week ago has had few problems.
Attached to the V245 is a SCSI RAID array, which presents one LUN to
the OS. On this lun is a zpool (tank), and within that 300+ zfs file
systems (one per user for automounted home directories). The system is
connected to our LAN via gigabit Ethernet,. most of our NFS clients
have just 100FD network connection.
In recent days performance of the file server seems to have gone off a
cliff. I don't know how to troubleshoot what might be wrong? Typical
"zpool iostat 120" output is shown below. If I run "truss -D df" I see
each call to statvfs64("/tank/bla) takes 2-3 seconds. The RAID itself
is healthy, and all disks are reporting as OK.
I have tried to establish if some client or clients are thrashing the
server via nfslogd, but without seeing anything obvious. Is there
some kind of per-zfs-filesystem iostat?
End users are reporting just saving small files can take 5-30 seconds?
prstat/top shows no process using significant CPU load. The system
has 8GB of RAM, vmstat shows nothing interesting.
I have another V245, with the same SCSI/RAID/zfs setup, and a similar
(though a bit less) load of data and users where this problem is NOT
apparent there?
Suggestions?
Kevin
Thu Jan 29 11:32:29 CET 2009
capacity operationsbandwidth
pool used avail read write read write
-- - - - - - -
tank2.09T 640G 10 66 825K 1.89M
tank2.09T 640G 39 5 4.80M 126K
tank2.09T 640G 38 8 4.73M 191K
tank2.09T 640G 40 5 4.79M 126K
tank2.09T 640G 39 5 4.73M 170K
tank2.09T 640G 40 3 4.88M 43.8K
tank2.09T 640G 40 3 4.87M 54.7K
tank2.09T 640G 39 4 4.81M 111K
tank2.09T 640G 39 9 4.78M 134K
tank2.09T 640G 37 5 4.61M 313K
tank2.09T 640G 39 3 4.89M 32.8K
tank2.09T 640G 35 7 4.31M 629K
tank2.09T 640G 28 13 3.47M 1.43M
tank2.09T 640G 5 51 433K 4.27M
tank2.09T 640G 6 51 450K 4.23M
tank2.09T 639G 5 52 543K 4.23M
tank2.09T 640G 26 57 3.00M 1.15M
tank2.09T 640G 39 6 4.82M 107K
tank2.09T 640G 39 3 4.80M 119K
tank2.09T 640G 38 8 4.64M 295K
tank2.09T 640G 40 7 4.82M 102K
tank2.09T 640G 43 5 4.79M 103K
tank2.09T 640G 39 4 4.73M 193K
tank2.09T 640G 39 5 4.87M 62.1K
tank2.09T 640G 40 3 4.88M 49.3K
tank2.09T 640G 40 3 4.80M 122K
tank2.09T 640G 42 4 4.83M 82.0K
tank2.09T 640G 40 3 4.89M 42.0K
...
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS iSCSI (For VirtualBox target) and SMB
Thanks Sanjeevb, By the way, this only seems to fail when I set up a volume instead of a file system. Should I be setting up a volume in this case, or will a file system suffice? If I turn off snapshots for this then it should work. I'll try this. Regards, Kevin -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS iSCSI (For VirtualBox target) and SMB
Hey all, I'm setting up a ZFS based fileserver to use both as a shared network drive and separately to have an iSCSI target to be used as the "Hard disk" of a windows based VM runninf on another machine. I've built the machine, installed the OS, created the RAIDZ pool and now have a couple of questions (I'm pretty much new to Solaris by the way but have been using Linux for some time). In my attempt to create the iSCSI target to be used and the VM disk I created (through the web frontend) a new dataset under the main pool of type "Volume" and gave it 30GB of space and called it iTunesVM. I then tried to run: zfs set shareiscsi=on tank/iTunesVM but got the error: cannot share 'tank/iTunesVM': iscsitgtd failed request to share cannot share 'tank/itune...@zfs-auto-snap:weekly-2009-01-02-15:02': iscsitgtd failed request to share I've checked and my iSCSI target service is on and running. With regards a network share accessible to both Windows, Linux and Mac OS machines on the network, what protocol would be best to use (NFS or SMB). I would then like to set up a locally hosted headless windows VM to run a Windows Media Player/iTunes share over the network for access to the music from my xbox/PS3. All help appreciated, Kevpatts -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] lofiadm -d keeps a lock on file in an nbmand-mounted zfs
Hi, Has anyone seen the following problem? After "lofiadm -d" removes an association, the file is still locked and cannot be moved or deleted if the file resides in a ZFS mounted with nbmand=on. There are two ways to remove the lock. (1) remount the zfs by the unmount+mount; the lock is removed even if nbmand=on option is given again, or (2) reboot the system. I don't have a system with UFS to test the nbmand mount-option to see if the problem exists for UFS as well. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Any commands to dump all zfs snapshots like NetApp "snapmirror"
Excuse me but could you please copy and paste the part of "zfs send -l " ? I couldn't find it in the link you send me: http://docs.sun.com/app/docs/doc/819-2240/zfs-1m?a=view What release is this "send -l " option available ? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Any commands to dump all zfs snapshots like NetApp "snapmirror"
The closest thing I can find is: http://bugs.opensolaris.org/view_bug.do?bug_id=6421958 But just like it says: " Incremental + recursive will be a bit tricker, because how do you specify the multiple source and dest snaps? " Let me clarify this more: Without "send -r" I need do something like this; Given a zfs file system "myzfs" in zpool "mypool", it has N snapshots: mypool/myzfs mypool/[EMAIL PROTECTED] mypool/[EMAIL PROTECTED] mypool/[EMAIL PROTECTED], Do following things: zfs snapshot mypool/[EMAIL PROTECTED] zfs send mypool/[EMAIL PROTECTED] | gzip - > /somewhere/myzfs-current.gz zfs send -i mypool/[EMAIL PROTECTED] mypool/[EMAIL PROTECTED] | gzip - > /somewhere/myzfs-1.gz zfs send -i mypool/[EMAIL PROTECTED] mypool/[EMAIL PROTECTED] | gzip - > /somewhere/myzfs-2.gz .. zfs send -i mypool/[EMAIL PROTECTED] mypool/[EMAIL PROTECTED] | gzip - > /somewhere/myzfs-N.gz As you can see, above commands are kind of a stupid solution, and it didn't reach maximum efficiency because those "myzfs-<1 ~ N>.gz" files contain a lot of common stuffs in them! I wonder how will "send -r" do in above situation? How does it choose multiple source and dest snaps? And can "-r" efficient enough to just dump the incremental changes? What is the corresponding receive command for "send -r"? (receive -r ? I guess? ) Thanks! -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Any commands to dump all zfs snapshots like NetApp "snapmirror"
Can you explain more about "zfs send -l " I know "zfs send -i" but didn't know there is a "-l" option? In which release is this option available? Thanks! -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Any commands to dump all zfs snapshots like NetApp "snapmirror"
I wonder if there are any equivalent commands in zfs to dump all its associated snapshots at maximum efficiency (only the changed data blocks among all snapshots)? I know you can just "zfs send" all snapshots but each one is like a full dump and if you use "zfs send -i" it is hard to maintain the relationship of the snapshots. In NetApp filer, I can do "snapmirror store /vol/vol01 ..." then everything in /vol/vol01 and all of its snapshots will be mirrored to destination, and it is block level which means only the changes in snapshots are sent out. So I wonder if there is an equivalent for ZFS to do similar things: For example: given one zfs: diskpool/myzfs, it has n snapshots: diskpool/[EMAIL PROTECTED], diskpool/[EMAIL PROTECTED],.. diskpool/[EMAIL PROTECTED], and if you just do "zfs < magic subcommands> diskpool/myzfs ..." and it will dump myzfs and all its n snapshots out at maximum efficiency (only changed data in snapshots) ? Any hints/helps are aprreciated! -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] raid card vs zfs
digg linked to an article related to the apple port of ZFS (http://www.dell.com/content/products/productdetails.aspx/print_1125?c=us&cs=19&l=en&s=dhss). I dont have a mac but was interested in ZFS. The article says that ZFS eliminates the need for a RAID card and is faster because the striping is running on the main cpu rather than an old chipset on a card. My question is, is this true? Can I install opensolaris with zfs and stripe and mirror a bunch of sata disc for a home NAS server? I sure would like to do that but the cost of the good raid cards has put me off; maybe this is the solution. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Disabling ZFS ACL
that is my thread and I'm still having issues even after applying that patch. It just came up again this week. [locahost] uname -a Linux dv-121-25.centtech.com 2.6.18-53.1.14.el5 #1 SMP Wed Mar 5 11:37:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux [localhost] cat /etc/issue CentOS release 5 (Final) Kernel \r on an \m [localhost: /n/scr20] touch test [localhost: /n/scr20] mv test /n/scr01/test/ ** this is a UFS mount on FreeBSD mv: preserving permissions for `/n/scr01/test/test': Operation not supported mv: preserving ACL for `/n/scr01/test/test': Operation not supported mv: preserving permissions for `/n/scr01/test/test': Operation not supported If I move it to the local /tmp, I get no errors. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ACL invalid argument from client
new problem. We have patched the system and it has fixed the error creating dirs/files on the ZFS filesystem. now I am getting permission errors with mv/cp from one of these ZFS areas to a regular FreeBSD server using UFS. thoughts? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ACL invalid argument from client
client CentOS 5.1 latest kernel mount option for zfs filesystem = rw,nosuid,nodev,remount,noatime,nfsvers=3,udp,intr,bg ,hard,rsize=8192,wsize=8192 directory and parent owned by user and users GID, 775 on client touch /tmp/dummy cd to zfs area mv /tmp/dummy . receive this error /bin/mv /tmp/dummy . /bin/mv: preserving permissions for `./dummy': Invalid argument /bin/mv: preserving ACL for `./dummy': Invalid argument /bin/mv: preserving permissions for `./dummy': Invalid argument the file does get copied. but it sends exit 0 when similarly used in a script zfs pool name = vol1 zfs get aclmode vol1 vol1 aclinherit discard local vol1 aclmode groupmasklocal vol1/scr20 is the mount point on the client vol1/scr20 aclinherit discard local vol1/scr20 aclmode groupmasklocal I've also tried using mount option noacl with no luck.. what do I need to do to ignore ACL's on my clients that support ACL's? and I've tried passthrough mode for aclinherit. thanks for any insight This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ?REFER in zfs list
What does the REFER column represent in zfs list. Thanks, kevin ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Degraded zpool won't online disk device, instead resilvers spare
5.79M 151K |c4t36d0- - 93 22 4.76M 151K |spare - - 0137 0 5.95M |--c5t29d0 - - 0 0 0 0 |--c5t21d0 - - 0136 0 5.95M |c4t37d0- - 74 23 3.86M 190K - - - - - - - So notice that there is 0 disk traffic for the disk we are trying to bring online (c5t29d0), but there is write disk traffic for the spare disk AND the other spare disk. So it looks like it's resilvering both mirror disks again? (why would it need to do that?) So I try using the replace command instead of the online command to tell it to bring itself online (and resilver only what has changed since it was brought online). But now it's complaining that the disk is already part of the same pool (since it's reading the old yet valid on-disk metadata for that disk which is still valid): ROOT $ zpool replace tank.2 c5t29d0 invalid vdev specification use '-f' to override the following errors: /dev/dsk/c5t29d0s0 is part of active ZFS pool tank.2. Please see zpool(1M). I could try the -f command to force it, but I want it to only resilver those parts that have changed. I tried detaching the mirror in hope that it would recognize that the c5t29d0 is online again: ROOT $ zpool detach tank.2 c5t21d0 However, running zpool status again shows that the spare has been removed, but no change other than that. When I reattach the spare device immediately, the resilver process begins again (it looks like again from zpool iostat or iostat -xn that it is resilvering both of the attached spares, not just the one that I'm attaching again). Also, this resilver process takes quite a long time (like it has to resilver everything all over again, as opposed to just changes). Does the resilver logic work differently if there is a spare involved? Any idea what is going wrong here? It seems that zfs should be able to online the disks since the OS can read/write perfectly fine to those devices. And it seems that if the online fails it shouldn't cause a resilver of both of the attached spares. You will notice that the pool was renamed by doing zpool export tank, zpool import tank tank.2. Could this be causing ZFS to get confused when the device is brought online? We are willing to try zpool replace -f on the disks that need to be brought online during the weekend to see what happens. Here is the system info: ROOT $ uname -a SunOS x.x.com 5.10 Generic_120012-14 i86pc i386 i86pc Will send showrev -p output if desired. Thanks, Kevin This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [xen-discuss] xVm blockers!
On Nov 28, 2007, at 5:38 AM, K wrote: > > 1/ Anchor vNic, the equivalent of linux dummy interfaces, we need more > flexibility in the way we setup xen networking. What is sad is that > the code is already available in the unreleased crossbow bits... but > it won't appear in nevada until Q1 2008 :( Indeed ... a frustration for many, including myself, who need specific pieces of functionality from larger projects that have larger schedules. We've been using a "dummy" NIC driver for the development of the Virtual Router project which should be coming online soon. I'm in the process of getting a legal approval for making the driver available sooner ... It's based on afe, which is already in the OpenSolaris repository, so I don't see any problems with making it available, but I need to check first. The intent of the driver, of course, is to bridge the gap until Crossbow Anchor VNICs appear in Nevada, so any long term dependency on the driver should be discouraged, but having to allocate hardware NICs for virtual interfaces in the mean time is certainly a more substantial discouragement. Kev > This is a real blocker for me as my ISP just started implementing port > security and locks my connection everytime it sees a foreign mac > address using one of the IP addresses that were originally assigned to > my dom0. On linux, I can setup a dummy interface and create a bridge > with it for a domU but on Solaris I need a physical NIC per bridge ! > $!! > @#$! > > For this particular feature, I am ready to give a few hundred dollars > as booty if anyone has a workaround. > > 2/ Pci passthru, this is really useful so you can let a domU access a > PCI card. It comes really handy if you want to virtualize a PBX that > is using cheap zaptel FXO cards. Again on linux, xen pci passthru has > been available for a while. Last time I mention this on the xen > solaris discussion, I received a very dry reply. > > 3/ Problem with DMA under Xen ... e.g. my areca raid cards works > perfect on a 8GB box without xen but because of the way xen allocates > memory... I am forced to allocate only 1 or 2 gig for the dom0 or the > areca drivers will fail miserably trying to do DMA above the first 4G > address space. This very same problem affected xen under linux over a > year ago and seems to have been addressed. Several persons on the ZFS > discuss list who complain about poor ZFS IO performance are affected > by this issue. > > 4/ Poor exploit mitigation under Solaris. In comparaison, OpenBSD, > grsec linux and Windows => XP SP2 have really good exploit > mitigation It is a shame because solaris offered a non-exec stack > before nearly everyone else... but it stopped there... no heap > protection, etc... > > The only thing that is preventing me from switching back to linux (no > zfs), freebsd (no xen) or openbsd (no xen and no zfs), right now is > ZFS and it is the same reason I switched to Solaris in the first > place. > > > > ___ > xen-discuss mailing list > [EMAIL PROTECTED] ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Mysterious corruption with raidz2 vdev
We'll try running all of the diagnostic tests to rule out any other issues. But my question is, wouldn't I need to see at least 3 checksum errors on the individual devices in order for there to be a visible error in the top level vdev? There doesn't appear to be enough raw checksum errors on the disks for there to have been 3 errors in the same vdev block. Or am I not understanding the checksum count correctly? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Mysterious corruption with raidz2 vdev (1 checksum err on disk, 2 on vd
Here's some additional output from the zpool and zfs tools: $ zpool list NAMESIZEUSED AVAILCAP HEALTH ALTROOT tank 10.2T 8.58T 1.64T83% ONLINE - $ zfs list NAME USED AVAIL REFER MOUNTPOINT tank 5.11T 901G 5.11T /tank Record size is 128K, checksums are on, compression is off, atime is off. This is the only zpool/filesystem in the system. Thanks. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Mysterious corruption with raidz2 vdev (1 checksum err on disk, 2 on vdev?)
After a scrub of a pool with 3 raidz2 vdevs (each with 5 disks in them) I see the following status output. Notice that the raidz2 vdev has 2 checksum errors, but only one disk inside the raidz2 vdev has a checksum error. How is this possible? I thought that you would have to have 3 errors in the same 'stripe' within a raidz2 vdev in order for the error to become unrecoverable. And I have not reset any errors with zpool clear ... Comments will be appreciated. Thanks. $ zpool status -v pool: tank state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: scrub completed with 1 errors on Mon Jul 23 19:59:07 2007 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 2 raidz2 ONLINE 0 0 2 c2t0d0 ONLINE 0 0 1 c2t1d0 ONLINE 0 0 0 c2t2d0 ONLINE 0 0 0 c2t3d0 ONLINE 0 0 0 c2t4d0 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c2t5d0 ONLINE 0 0 0 c2t6d0 ONLINE 0 0 0 c2t7d0 ONLINE 0 0 0 c2t8d0 ONLINE 0 0 0 c2t9d0 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c2t10d0 ONLINE 0 0 0 c2t11d0 ONLINE 0 0 0 c2t12d0 ONLINE 0 0 1 c2t13d0 ONLINE 0 0 0 c2t14d0 ONLINE 0 0 0 spares c2t15d0AVAIL errors: The following persistent errors have been detected: DATASET OBJECT RANGE 55fe9784 lvl=0 blkid=40299 This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zpool lost
Hi I'm using fairly stock S10, but this is really just a zfs/zpool question. # uname -a SunOS peach 5.10 Generic_118833-36 sun4u sparc SUNW,Sun-Blade-100 Misremembering the option to file for handling special files (-s), I executed the following: # file -m /dev/dsk/c*s2 My shell would have expanded this as follows: # ls /dev/dsk/c*s2 /dev/dsk/c0t0d0s2 /dev/dsk/c0t1d0s2 /dev/dsk/c0t2d0s2 This is a SB100 with 2 disks. c0t2d0 has the OS and other partitions, c0t2d0 is the CDROM. The other disk was one large, single disk, zpool. It's not mountable now. # zpool status -v pool: tank state: FAULTED status: One or more devices could not be opened. There are insufficient replicas for the pool to continue functioning. action: Attach the missing device and online it using 'zpool online'. see: http://www.sun.com/msg/ZFS-8000-D3 scrub: none requested config: NAMESTATE READ WRITE CKSUM tankUNAVAIL 0 0 0 insufficient replicas c0t0d0UNAVAIL 0 0 0 cannot open Not knowing what the "file -m ..." command might have done, I wonder if this situation is recoverable? Looking at the smf logs I see that my pool (tank) is at least recognized a bit: # fmdump -eV Feb 09 2007 16:41:46.082264521 ereport.fs.zfs.vdev.open_failed nvlist version: 0 class = ereport.fs.zfs.vdev.open_failed ena = 0x10163875741 detector = (embedded nvlist) nvlist version: 0 version = 0x0 scheme = zfs pool = 0x347b6d721b99340d vdev = 0xa7f43f56535e14a8 (end detector) pool = tank pool_guid = 0x347b6d721b99340d pool_context = 1 vdev_guid = 0xa7f43f56535e14a8 vdev_type = disk vdev_path = /dev/dsk/c0t0d0s0 vdev_devid = id1,[EMAIL PROTECTED]/a parent_guid = 0x347b6d721b99340d parent_type = root prev_state = 0x1 __ttl = 0x1 __tod = 0x45cc963a 0x4e741c9 Feb 09 2007 16:41:46.082265280 ereport.fs.zfs.zpool nvlist version: 0 class = ereport.fs.zfs.zpool ena = 0x10163875741 detector = (embedded nvlist) nvlist version: 0 version = 0x0 scheme = zfs pool = 0x347b6d721b99340d (end detector) pool = tank pool_guid = 0x347b6d721b99340d pool_context = 1 __ttl = 0x1 __tod = 0x45cc963a 0x4e744c0 Any ideas/suggestions? Kevin This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] NFS share problem with mac os x client
Hello, I test right now the beauty of zfs. I have installed opensolaris on a spare server to test nfs exports. After creating tank1 with zpool and a subfilesystem with zfs tank1/nfsshare, I have set the option sharenfs=on to tank1/nfsshare. With Mac OS X as client I can mount the filesystem in Finder.app with nfs://server/tank1/nfsshare, but if I copy a file an error ocours. Finder say "The operation cannot be completed because you do not have sufficient privileges for some of the items.". Until now I have shared the filesystems always with samba so I have almost no experience with nfs. Any ideas? Kevin This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] solaris - ata over ethernet - zfs - HPC
Hi, I'd like to consider using the coraid products with solaris and ZFS but I need them to work with x86_64 on on generic opteron/amd compatible hardware. Currently the AOE driver is beta for sparc only. I am planning to use the ZFS file system so the raid hardware in the coraid device will not be used as recommended for ZFS. Only the direct access over ethernet to the disks will be used. The installation will be part of new HPC cluster for computational chemistry and biology. Does this seem like a good idea? I am not a storage expert and am attempting to create a scalable distributed storage cluster for an HPC cluster. I expect have a limited budget and commodity hardware, except for the coraid box. Relevant links: http://en.wikipedia.org/wiki/Ata_over_ethernet http://www.coraid.com/ http://www.coraid.com/support/solaris/ All comments are welcome. Thank you, Kevin -- Kevin C. Abbey System Administrator Rutgers University - BioMaPS Institute Email: [EMAIL PROTECTED] Hill Center - Room 279 110 Frelinghuysen Road Piscataway, NJ 08854 Phone and Voice mail: 732-445-3288 Wright-Rieman Laboratories Room 201 610 Taylor Rd. Piscataway, NJ 08854-8087 Phone: 732-445-2069 Fax: 732-445-5958 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] commercial backup software and zfs
Hi Is the following an accurate sstatment of the current status with (for me) the 3 main commercial ackup software solutions out there (1) Legato Networker - support coming in 7.3.2 patch due soon. Until then backups of ZFS filesystems wont work at all as the acl(8) calls will fail. But will work if you use "legacy" mouning of zfs filesytsem (but even then wont include any ALCs). Or backup via NFS mounts. (2) IBM's TSM - no current or official support from IBM. Will/wont work? (3) Veritas NetBackup - client v5.1+ works out of the box, but without full/any ACL support. ? Kevin PS: I'm particularly interested in (2) This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
