Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images
Hmmm, that's odd. I have a number of VMs running on NFS (hosted on ESX, rather than Xen) with no problems at all. I did add a SLOG device to get performance up to a reasonable level, but it's been running flawlessly for a few months now. Previously I was using iSCSI for most of the connections, but with the addition of the SLOG device NFS has become feasible. All I'm using on the OSOL side is sharenfs=anon=0 and adding the server's addresses to /etc/hosts to permit access. Running osol2009.06. Cheers, Erik On 28 juil. 2010, at 21:11, sol wrote: Richard Elling wrote: Gregory Gee wrote: I am using OpenSolaris to host VM images over NFS for XenServer. I'm looking for tips on what parameters can be set to help optimize my ZFS pool that holds my VM images. There is nothing special about tuning for VMs, the normal NFS tuning applies. That's not been my experience. Out of the box VMware server would not work with the VMs stored on a zfs pool via NFS. I've not yet found out why but the analytics showed millions of getattr/access/lookup compared to read/write. A partial workaround was to turn off access time on the share and to mount with noatime,actimeo=60 But that's not perfect because when left along the VM got into a stuck state. I've never seen that state before when the VM was hosted on a local disk. Hosting VMs on NFS is not working well so far... ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images
On Jul 28, 2010, at 3:11 PM, sol wrote: A partial workaround was to turn off access time on the share and to mount with noatime,actimeo=60 But that's not perfect because when left along the VM got into a stuck state. I've never seen that state before when the VM was hosted on a local disk. Hosting VMs on NFS is not working well so far... We host a lot of VMs on NFS shares (from a 7000 series) on ESXi with no issues other than an occasional Ubuntu machine that would do something similar to what you describe. For us it was this: http://communities.vmware.com/thread/237699?tstart=30 when the timeout is set to 180 the issue has been completely eliminated. The current ESXi (and I think all versions of 4) VMWare tools does this properly. Also, EMC and NetApp have this description of using NFS shares for VMWare: http://virtualgeek.typepad.com/virtual_geek/2009/06/a-multivendor-post-to-help-our-mutual-nfs-customers-using-vmware.html which by and large applies to any NFS server rather than just their equipment. We found it helpful, but nothing is that surprising in it. Good luck, Ware ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images
-Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of sol Sent: Wednesday, July 28, 2010 3:12 PM To: Richard Elling; Gregory Gee Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images Richard Elling wrote: Gregory Gee wrote: I am using OpenSolaris to host VM images over NFS for XenServer. I'm looking for tips on what parameters can be set to help optimize my ZFS pool that holds my VM images. There is nothing special about tuning for VMs, the normal NFS tuning applies. That's not been my experience. Out of the box VMware server would not work with the VMs stored on a zfs pool via NFS. I've not yet found out why but the analytics showed millions of getattr/access/lookup compared to read/write. A partial workaround was to turn off access time on the share and to mount with noatime,actimeo=60 But that's not perfect because when left along the VM got into a stuck state. I've never seen that state before when the VM was hosted on a local disk. Hosting VMs on NFS is not working well so far... My guess is that it's a VMware Server + NFS client issue, not a VMs on NFS issue. I'm using an OpenSolaris b134 system as an 'experimental' NFS datastore for VMware vSphere/ESX and it works great. I've had as many as 30 mixed-IO VMs running on the system with no reported issues. -Will ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images
On Jul 28, 2010, at 12:11 PM, sol wrote: Richard Elling wrote: Gregory Gee wrote: I am using OpenSolaris to host VM images over NFS for XenServer. I'm looking for tips on what parameters can be set to help optimize my ZFS pool that holds my VM images. There is nothing special about tuning for VMs, the normal NFS tuning applies. That's not been my experience. Out of the box VMware server would not work with the VMs stored on a zfs pool via NFS. I do this regularly and know many people who run this way. I've not yet found out why but the analytics showed millions of getattr/access/lookup compared to read/write. These are requests from the client being serviced by the server. To find out why the client is sending such requests, you'll need to look at the client. A partial workaround was to turn off access time on the share and to mount with noatime,actimeo=60 Yes, these are common NFS tunables. -- richard But that's not perfect because when left along the VM got into a stuck state. I've never seen that state before when the VM was hosted on a local disk. Hosting VMs on NFS is not working well so far... -- Richard Elling rich...@nexenta.com +1-760-896-4422 Enterprise class storage for everyone www.nexenta.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images
On Jul 19, 2010, at 5:26 PM, Gregory Gee wrote: I am using OpenSolaris to host VM images over NFS for XenServer. I'm looking for tips on what parameters can be set to help optimize my ZFS pool that holds my VM images. I am using XenServer which is running the VMs from an NFS storage on my OpenSolaris server. Are there parameters that I should change to make the read and write to the VDH files perform better since these are large files. There is nothing special about tuning for VMs, the normal NFS tuning applies. Please consult the ZFS Best Practices Guide for details http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide -- richard -- Richard Elling rich...@nexenta.com +1-760-896-4422 Enterprise class storage for everyone www.nexenta.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images
To further this question, I have been searching for a while and can't find any reference to the difference and benefits between zfs sharenfs and nfs share. Currently I am using standard NFS I believe. share -F nfs -o anon=0,sec=sys,rw=xenserver0:xenserver1 /files/VM ad...@nas:/files$ zfs list NAME USED AVAIL REFER MOUNTPOINT pool2 40.8G 142G19K /pool2 pool2/VM 40.8G 142G 38.4G /files/VM Is one way better than the other? I've also been looking for a migration guide from nfs to zfs sharenfs. Any pointers? Thanks, Greg -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images
On Jul 20, 2010, at 6:14 PM, Gregory Gee wrote: To further this question, I have been searching for a while and can't find any reference to the difference and benefits between zfs sharenfs and nfs share. Currently I am using standard NFS I believe. share -F nfs -o anon=0,sec=sys,rw=xenserver0:xenserver1 /files/VM ad...@nas:/files$ zfs list NAME USED AVAIL REFER MOUNTPOINT pool2 40.8G 142G19K /pool2 pool2/VM 40.8G 142G 38.4G /files/VM Is one way better than the other? I've also been looking for a migration guide from nfs to zfs sharenfs. Any pointers? When imported, ZFS will send the contents of the sharenfs parameter to share(1m) along with the file share type (nfs) and file system mount point (eg /files/VM). Other than that, they are the same. For a standalone system, it probably is a wash. For clusters, it is very nice to be able to administer the shares once and have that administration travel along with the file system instead of being located elsewhere, like /etc/dfstab -- Richard Elling rich...@nexenta.com +1-760-896-4422 Enterprise class storage for everyone www.nexenta.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Tips for ZFS tuning for NFS store of VM images
Thanks. I guess I am in a 'If it ain't broken, don't fix it' for my NFS setup. Thanks, Greg -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
