If a low-RAM/over-provisioned host was doing a lot of swapping out to disk I could see there being a more significant advantage to that. The Virtual Disk Path contains the image being booted and run by each running VM, so the end-user benefits from fast random reads both in cases where an image would need to boot and in application startup time during a reservation.
On Mon, Jun 9, 2014 at 12:20 PM, Aaron Coburn <[email protected]> wrote: > Mike, > Were you finding that the "Virtual Disk Path" (RAID 1 with SSDs) was > seeing the biggest I/O latency while users had active VCL reservations? I > would have thought the "VM Working directory path" would be a better > candidate for SSDs. > > Aaron > > On Jun 9, 2014, at 10:06 AM, Mike Haudenschild <[email protected]> wrote: > > > Hi Dave, > > > > I think you're coming at this from a shared storage perspective, which > is totally understandable since you're currently using a SAN. In a local > storage setup, the ESXi host either a) pulls a copy of the image to local > storage from an NFS mount, or b) the management node has a copy of the > image (locally or is pointed at a remote mount) and pushes it to the ESXi > host. (I think the VCL docs recommend the former approach as being more > efficient since it can use ESXi's file management utilities.) > > > > The main benefit from using SSDs for local storage will be reduced load > times due to the image actually booting from those drives. > > > > I've attached screenshots of our host profile as well as the datastores > on the ESXi host. > > > > On the host profile (full docs on these settings at > http://vcl.apache.org/docs/vmwareconfiguration): > > > > - Repository path = NFS share from which the ESXi host will copy the > image the first time a management node tells this host to boot that image. > This is a RAID 50 on 10k SAS dives in this particular rig. See note [1]. > > - Virtual disk path = Local storage on the ESXi host from which end-user > VMs will boot the image. This is a RAID 1 of SSDs. > > - VM working directory path = Local storage on the ESXi host where VCL > will build the temporary directories used for each running VM. This is a > RAID 50 on 15k SAS drives. > > > > [1] NB... In the ESXi screenshot, disregard the datastore "Master VCL > Repository." The Linux system hosting the NFS share is actually a VM that > happens to be running on this ESXi host. > > > > The datastore screenshot shows the corresponding names of the > datastores. As long as you name the datastores the same across hosts, you > can get away with using a single VM host profile for your local-storage > hosts. If you have some hosts with dramatically different local storage > situations, you may need different profiles. > > > > Also, I haven't even touched the issue of vMotion here, which changes > the conversation about shared-vs-local storage a lot. If your use case for > vMotion is about planned ESXi host maintenance, you can always migrate VMs > away from a host-to-be-rebooted to other hosts via the VCL Web UI (any VMs > with active reservations will be placed in a pending state). vMotion for > HA with VCL local storage is a different story. > > > > Regards, > > Mike > > > > > > > > On Mon, Jun 9, 2014 at 9:44 AM, David DeMizio <[email protected]> wrote: > > Thanks Mike. > > > > I have attached of my current vanilla esxi local storage profile. > > > > I'm not to clear on why the Golden images would want to be on NFS share, > isn't that what is used to spin up the VMs? also "Create a directory on > your ESXi hosts' SSDs for the local copies of the images. The VMs will > boot from these copies and will benefit from the speed of the SSDs -- they > get written-to rarely and read a bunch" VCL automatically knows to use the > local copes or is it a setting the the profile. Just a bit confused on what > needs to go where in the profile that's why I have attached it. When you > say mount this with the same repository path on all your ESXI hosts are you > referring to actually ssh into esxi hosts and created and NFS share? Thank > you > > > > > > > > > > > > > > On Thu, Jun 5, 2014 at 4:57 PM, Mike Haudenschild <[email protected]> > wrote: > > Hi Dave, > > > > This is the configured I've used the most with VCL. You will have > multiple "virtual hosts," each with its own "computers" (VMs) assigned to > it. If VCL wants to spin up an image on VM#1, and VM#1 is assigned to VM > Host A, VCL's management node will open a connection with Host A and bring > up the VM. > > > > From what I've been reading, I think you can use the same VM host > profile for the VM hosts that will use local storage. > > > > - Have your golden images hosted via an NFS share available to all ESXi > hosts. > > - Mount this with the same repository path on all your ESXi hosts. > > - Create a directory on your ESXi hosts' SSDs for the local copies of > the images. The VMs will boot from these copies and will benefit from the > speed of the SSDs -- they get written-to rarely and read a bunch. > > - Create another directory (either on your SSD or spinning disk space on > the hosts, or even on the SAN) for the individual-VM files (the .vmx and > the virtual memory files). If you have enough SSD storage space, do it > here, else do it on spinning disks. Remember that these files get torn > down after every reservation, and that the virtual memory files created > correspond to the physical memory assigned to the VM -- so it's not > absolutely necessary or even advisable (from a wear perspective) to have > these on SSDs. > > > > If you use the same paths and network configs for both your > ESXi/SSD/local storage hosts, you can use a single VM host profile for > these hosts. You'll obviously still have a different host profile for your > SAN/shared storage hosts. > > > > Regards, > > Mike > > > > > > On Thu, Jun 5, 2014 at 4:48 PM, David DeMizio <[email protected]> wrote: > > Thanks Aaron, > > > > For your hosts with local storage do you use NFS share for your golden > images? So from what I have gathered so far, each esxi host will have it's > own VM Working Directory Path (local ssd storage). Also, each host will > always run the same virtual machines whereas with shared storage and using > vcenter host profile, it may put the virtual machine on any host in the > cluster. I'm just trying to picture how vcl will work when a user request > an image? How will VCL know which esxi hosts to connect to, to create the > virtual machine? > > > > > > > > > > On Thu, Jun 5, 2014 at 4:30 PM, Aaron Coburn <[email protected]> > wrote: > > Hi, David, > > > > > So am I correct that I need two vm host profiles for each esxi host? > > > > I believe that is correct. In our setup, we have some hosts with local > storage and some hosts with shared backend storage. For those with local > storage, the vcl connects directly to the esx host (not using vcenter -- > vcenter is only used for those hosts with a shared, SAN-based storage). It > may be the case that you could get this to work with vCenter by disabling > DRS (vMotion), but it is a lot easier to just connect directly to the > different hosts. > > > > Regards, > > Aaron Coburn > > > > > > On Jun 5, 2014, at 2:53 PM, David DeMizio <[email protected]> wrote: > > > > > So am I correct that I need two vm host profiles for each esxi host? > I'll have to have a better look at the VCL website. I'm just not clear on > how this is going to be setup. I've been using the vcenter profile since I > started experimenting with VCL. I guess each esxi host will have dedicated > virtual machines that will always run on that esxi host? > > > > > > > > > David DeMizio > > > Academic Systems Coordinator > > > Office of Information Technology > > > New College of Florida > > > Phone: 941-487-4222 | Fax: 941-487-4356 > > > www.ncf.edu > > > > > > > > > On Thu, Jun 5, 2014 at 2:06 PM, Dmitri Chebotarov <[email protected]> > wrote: > > > Hmm... I'm not sure if this can be done with vCenter profile. I think > vCenter profile assumes you have shared storage. > > > > > > > > > You could add ESXi hosts directly to VCL, and then create local > datastore in each host. You may need to remove ESXi hosts from vCenter, as > it won't allow you to have the same local datastore name on each host. > > > > > > From: David DeMizio <[email protected]> > > > Sent: Thursday, June 05, 2014 12:34 PM > > > To: user > > > Subject: Re: Local storage > > > > > > Thanks Andy > > > > > > I guess I just need clarification on what type of vm host profile I > will need to use with this type of setup and if I need a separate vm host > profile for each of my esxi host servers? > > > > > > > > > David DeMizio > > > Academic Systems Coordinator > > > Office of Information Technology > > > New College of Florida > > > Phone: 941-487-4222 | Fax: 941-487-4356 > > > www.ncf.edu > > > > > > > > > On Thu, Jun 5, 2014 at 12:09 PM, Andy Kurth <[email protected]> > wrote: > > > I haven't tried it, but apparently you can vMotion VMs without having > shared storage starting with 5.1: > > > > > > > http://blogs.vmware.com/vsphere/2012/09/vmotion-without-shared-storage-requirement-does-it-have-a-name.html > > > > > > -Andy > > > > > > > > > On Thu, Jun 5, 2014 at 11:22 AM, David DeMizio <[email protected]> > wrote: > > > Thank Dmitri, > > > > > > I'm getting a bit confused because the VM Working Directory Path > pointing to the local SSD which the other server knows nothing about. I'm > currently using the VMware Vcenter host profile will I need to change that > and use the esxi local storage profile? Seems like I'm going to need two > virtual hosts using the esxi local storage policy? > > > > > > > > > David DeMizio > > > Academic Systems Coordinator > > > Office of Information Technology > > > New College of Florida > > > Phone: 941-487-4222 | Fax: 941-487-4356 > > > www.ncf.edu > > > > > > > > > On Thu, Jun 5, 2014 at 10:22 AM, Dmitri Chebotarov <[email protected]> > wrote: > > > Hi David > > > > > > I think it should work fine with local SSD storage on ESXi hosts. You > still may want to use a shared NFS storage for your images (Virtual Disk > Path), but you can point VM Working Directory Path to local SSD storage. > > > > > > -- > > > Thank you, > > > > > > Dmitri Chebotarov > > > VCL Sys Eng, Engineering & Architectural Support, TSD - Ent Servers & > Messaging > > > 223 Aquia Building, Ffx, MSN: 1B5 > > > Phone: (703) 993-6175 | Fax: (703) 993-3404 > > > > > > > > > From: David DeMizio <[email protected]> > > > Sent: Thursday, June 05, 2014 10:08 AM > > > To: user > > > Subject: Local storage > > > > > > > > > Hello, > > > > > > We have two servers with local Solid state storage in both of them > that I would like to start testing with VCL. Currently, I am using vsphere > 5.5 with the vcl vmware module to provision my nodes by using a SAN. Of > course if I plan to test VCL just using the local SSD drives than I will > lose the ability of vmotion, at least I think I will because each server > will not have access to the other servers SSDs. Can I still use the vmware > module if I go this route? Any other suggestions are appreciated. Thank you. > > > > > > -Dave > > > > > > > > > > > > > > > > > > > > > > > > <host_profile.png><vmhost_datastores.png> > >
