Re: Virtual Machine Disk Configuration
The exchange 2k3 server has operating system - raid 1 , stores raid 5 and logs are raid 1. I have noticed that when email performance suffers the entire lun containg all the guest VM's suffers. I have started collecting stats from the SAN and I have noticed that even with a light load the lun in question appears to have a queue of 3 (limit should be 3 or less according to HP notes). Unfortunately load has been very light this afternoon. I will see what transpires first thing in the morning when load normally increases. On Tue, Nov 2, 2010 at 2:34 PM, Richard Stovall rich...@gmail.com wrote: On the VMs, are you running multiple disks in guest-OS-based RAID configurations underneath the EVA's vRAID5 protection, with all the VHDs on the same LUN? What is the exact config for the problematic Exchange server? On Mon, Nov 1, 2010 at 11:25 PM, Mark Milo markmilo2...@gmail.com wrote: I don't have stats on the eva performance as yet. What would you suggest the configuration should be for the main disk array? Just to clarify the disk pool is Vraid5 while the disks for individual guest VM's have typically been configured as raid 1 for operating system and log disks and raid 5 for data. On Tue, Nov 2, 2010 at 2:04 PM, Brian Desmond br...@briandesmond.comwrote: *Is there any sort of utilization stats on the EVA? Frankly I wouldn’t expect 21 spindles RAID5 to perform very well at all with a mixed workload like this. * * * *Thanks,* *Brian Desmond* *br...@briandesmond.com* * * *c – 312.731.3132* * * *From:* Mark Milo [mailto:markmilo2...@gmail.com] *Sent:* Monday, November 01, 2010 9:14 PM *To:* NT System Admin Issues *Subject:* Re: Virtual Machine Disk Configuration The san has 21 FC disks configured as a raid 5 array. There have two luns configured from this array. All 15 guest machines are located on one of these luns (including exchange). The performance hit appears mainly on exchange but other servers are suffering as well. The interesting thing is that IOPS for the entire lun holding the 15 VM's is only around 600 at the time that the disk busy is around 95%. The disk queue for that lun is around 4 - I will need to set up longer term monitoring to get the read/write times when the disk set gets busy (which is usually first thing in the morning) On Tue, Nov 2, 2010 at 12:35 PM, Sean Martin seanmarti...@gmail.com wrote: We'd need a lot more info. Start with your disk configuration. How many disks make up the storage pool or raid group the exchange lun is allocated from? Are there other luns allocated from the same pool? How many? Which exchange resources reside on the lun (stores, logs, etc.)? Disk busy warnings buy themselves don't hold much weight, unless of course there are noticeable performances issues, which you alluded to. You'll want to capture disk queue and disk latency stats ( avg disk queue length and avg disk sec/reads and writes). Avg queue depth should below X (where X equals the number of physical disks. Read/write times should avg below 20ms and not spike above 50ms. I'd also recommend running the exbpa and the troubleshooting assistant. If you can gather I/O stats you may be able to determine if the disks can handle the load. If it doesn't appear Exchange is stressing your resources, you may need your SAN monitoring tools to determine if total I/O (assuming multiple luns share the same disks) are more than the disks can handle. I'm not familiar with EVAs, but overloading disks in a SAN can lead to cache flushing which could affect performance of the array itself. - Sean On Nov 1, 2010, at 5:08 PM, Mark Milo markmilo2...@gmail.com wrote: The host is Windows 2008 R2 running Hyper V. The performance is monitored via UPTIME monitoring software but I see similar high reading using perfmon (%disk time). Don't see any error messages at high utilization but the exchange 2003 server is responding poorly when disk is busy. There are 15 guest machines on this particular host but most of them dont use too much in the way of resources. On Tue, Nov 2, 2010 at 11:51 AM, Richard Stovall rich...@gmail.com wrote: On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large number of VMs. If so what should the max utilization of the disk on that LUN be? I don't know of a hard and fast number/rule, but if you're using snapshot-based backup method (such as VCB
Re: Virtual Machine Disk Configuration
Are these software raid volumes created on and managed by the VM? If so, are all the VHDs on the same LUN? On Tue, Nov 2, 2010 at 5:00 AM, Mark Milo markmilo2...@gmail.com wrote: The exchange 2k3 server has operating system - raid 1 , stores raid 5 and logs are raid 1. I have noticed that when email performance suffers the entire lun containg all the guest VM's suffers. I have started collecting stats from the SAN and I have noticed that even with a light load the lun in question appears to have a queue of 3 (limit should be 3 or less according to HP notes). Unfortunately load has been very light this afternoon. I will see what transpires first thing in the morning when load normally increases. On Tue, Nov 2, 2010 at 2:34 PM, Richard Stovall rich...@gmail.com wrote: On the VMs, are you running multiple disks in guest-OS-based RAID configurations underneath the EVA's vRAID5 protection, with all the VHDs on the same LUN? What is the exact config for the problematic Exchange server? On Mon, Nov 1, 2010 at 11:25 PM, Mark Milo markmilo2...@gmail.comwrote: I don't have stats on the eva performance as yet. What would you suggest the configuration should be for the main disk array? Just to clarify the disk pool is Vraid5 while the disks for individual guest VM's have typically been configured as raid 1 for operating system and log disks and raid 5 for data. On Tue, Nov 2, 2010 at 2:04 PM, Brian Desmond br...@briandesmond.comwrote: *Is there any sort of utilization stats on the EVA? Frankly I wouldn’t expect 21 spindles RAID5 to perform very well at all with a mixed workload like this. * * * *Thanks,* *Brian Desmond* *br...@briandesmond.com* * * *c – 312.731.3132* * * *From:* Mark Milo [mailto:markmilo2...@gmail.com] *Sent:* Monday, November 01, 2010 9:14 PM *To:* NT System Admin Issues *Subject:* Re: Virtual Machine Disk Configuration The san has 21 FC disks configured as a raid 5 array. There have two luns configured from this array. All 15 guest machines are located on one of these luns (including exchange). The performance hit appears mainly on exchange but other servers are suffering as well. The interesting thing is that IOPS for the entire lun holding the 15 VM's is only around 600 at the time that the disk busy is around 95%. The disk queue for that lun is around 4 - I will need to set up longer term monitoring to get the read/write times when the disk set gets busy (which is usually first thing in the morning) On Tue, Nov 2, 2010 at 12:35 PM, Sean Martin seanmarti...@gmail.com wrote: We'd need a lot more info. Start with your disk configuration. How many disks make up the storage pool or raid group the exchange lun is allocated from? Are there other luns allocated from the same pool? How many? Which exchange resources reside on the lun (stores, logs, etc.)? Disk busy warnings buy themselves don't hold much weight, unless of course there are noticeable performances issues, which you alluded to. You'll want to capture disk queue and disk latency stats ( avg disk queue length and avg disk sec/reads and writes). Avg queue depth should below X (where X equals the number of physical disks. Read/write times should avg below 20ms and not spike above 50ms. I'd also recommend running the exbpa and the troubleshooting assistant. If you can gather I/O stats you may be able to determine if the disks can handle the load. If it doesn't appear Exchange is stressing your resources, you may need your SAN monitoring tools to determine if total I/O (assuming multiple luns share the same disks) are more than the disks can handle. I'm not familiar with EVAs, but overloading disks in a SAN can lead to cache flushing which could affect performance of the array itself. - Sean On Nov 1, 2010, at 5:08 PM, Mark Milo markmilo2...@gmail.com wrote: The host is Windows 2008 R2 running Hyper V. The performance is monitored via UPTIME monitoring software but I see similar high reading using perfmon (%disk time). Don't see any error messages at high utilization but the exchange 2003 server is responding poorly when disk is busy. There are 15 guest machines on this particular host but most of them dont use too much in the way of resources. On Tue, Nov 2, 2010 at 11:51 AM, Richard Stovall rich...@gmail.com wrote: On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large
Re: Virtual Machine Disk Configuration
On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large number of VMs. If so what should the max utilization of the disk on that LUN be? I don't know of a hard and fast number/rule, but if you're using snapshot-based backup method (such as VCB) you definitely need free space on the LUNs for the snapshots to grow until the backup is finished and the snapshots can be eliminated. What virtualization platform are you running? What version? ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
Re: Virtual Machine Disk Configuration
The host is Windows 2008 R2 running Hyper V. The performance is monitored via UPTIME monitoring software but I see similar high reading using perfmon (%disk time). Don't see any error messages at high utilization but the exchange 2003 server is responding poorly when disk is busy. There are 15 guest machines on this particular host but most of them dont use too much in the way of resources. On Tue, Nov 2, 2010 at 11:51 AM, Richard Stovall rich...@gmail.com wrote: On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large number of VMs. If so what should the max utilization of the disk on that LUN be? I don't know of a hard and fast number/rule, but if you're using snapshot-based backup method (such as VCB) you definitely need free space on the LUNs for the snapshots to grow until the backup is finished and the snapshots can be eliminated. What virtualization platform are you running? What version? ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
Re: Virtual Machine Disk Configuration
We'd need a lot more info. Start with your disk configuration. How many disks make up the storage pool or raid group the exchange lun is allocated from? Are there other luns allocated from the same pool? How many? Which exchange resources reside on the lun (stores, logs, etc.)? Disk busy warnings buy themselves don't hold much weight, unless of course there are noticeable performances issues, which you alluded to. You'll want to capture disk queue and disk latency stats ( avg disk queue length and avg disk sec/reads and writes). Avg queue depth should below X (where X equals the number of physical disks. Read/write times should avg below 20ms and not spike above 50ms. I'd also recommend running the exbpa and the troubleshooting assistant. If you can gather I/O stats you may be able to determine if the disks can handle the load. If it doesn't appear Exchange is stressing your resources, you may need your SAN monitoring tools to determine if total I/O (assuming multiple luns share the same disks) are more than the disks can handle. I'm not familiar with EVAs, but overloading disks in a SAN can lead to cache flushing which could affect performance of the array itself. - Sean On Nov 1, 2010, at 5:08 PM, Mark Milo markmilo2...@gmail.com wrote: The host is Windows 2008 R2 running Hyper V. The performance is monitored via UPTIME monitoring software but I see similar high reading using perfmon (%disk time). Don't see any error messages at high utilization but the exchange 2003 server is responding poorly when disk is busy. There are 15 guest machines on this particular host but most of them dont use too much in the way of resources. On Tue, Nov 2, 2010 at 11:51 AM, Richard Stovall rich...@gmail.com wrote: On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large number of VMs. If so what should the max utilization of the disk on that LUN be? I don't know of a hard and fast number/rule, but if you're using snapshot-based backup method (such as VCB) you definitely need free space on the LUNs for the snapshots to grow until the backup is finished and the snapshots can be eliminated. What virtualization platform are you running? What version? ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
Re: Virtual Machine Disk Configuration
The san has 21 FC disks configured as a raid 5 array. There have two luns configured from this array. All 15 guest machines are located on one of these luns (including exchange). The performance hit appears mainly on exchange but other servers are suffering as well. The interesting thing is that IOPS for the entire lun holding the 15 VM's is only around 600 at the time that the disk busy is around 95%. The disk queue for that lun is around 4 - I will need to set up longer term monitoring to get the read/write times when the disk set gets busy (which is usually first thing in the morning) On Tue, Nov 2, 2010 at 12:35 PM, Sean Martin seanmarti...@gmail.com wrote: We'd need a lot more info. Start with your disk configuration. How many disks make up the storage pool or raid group the exchange lun is allocated from? Are there other luns allocated from the same pool? How many? Which exchange resources reside on the lun (stores, logs, etc.)? Disk busy warnings buy themselves don't hold much weight, unless of course there are noticeable performances issues, which you alluded to. You'll want to capture disk queue and disk latency stats ( avg disk queue length and avg disk sec/reads and writes). Avg queue depth should below X (where X equals the number of physical disks. Read/write times should avg below 20ms and not spike above 50ms. I'd also recommend running the exbpa and the troubleshooting assistant. If you can gather I/O stats you may be able to determine if the disks can handle the load. If it doesn't appear Exchange is stressing your resources, you may need your SAN monitoring tools to determine if total I/O (assuming multiple luns share the same disks) are more than the disks can handle. I'm not familiar with EVAs, but overloading disks in a SAN can lead to cache flushing which could affect performance of the array itself. - Sean On Nov 1, 2010, at 5:08 PM, Mark Milo markmilo2...@gmail.com wrote: The host is Windows 2008 R2 running Hyper V. The performance is monitored via UPTIME monitoring software but I see similar high reading using perfmon (%disk time). Don't see any error messages at high utilization but the exchange 2003 server is responding poorly when disk is busy. There are 15 guest machines on this particular host but most of them dont use too much in the way of resources. On Tue, Nov 2, 2010 at 11:51 AM, Richard Stovall rich...@gmail.com rich...@gmail.com wrote: On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.com markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large number of VMs. If so what should the max utilization of the disk on that LUN be? I don't know of a hard and fast number/rule, but if you're using snapshot-based backup method (such as VCB) you definitely need free space on the LUNs for the snapshots to grow until the backup is finished and the snapshots can be eliminated. What virtualization platform are you running? What version? ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~
RE: Virtual Machine Disk Configuration
Is there any sort of utilization stats on the EVA? Frankly I wouldn't expect 21 spindles RAID5 to perform very well at all with a mixed workload like this. Thanks, Brian Desmond br...@briandesmond.com c - 312.731.3132 From: Mark Milo [mailto:markmilo2...@gmail.com] Sent: Monday, November 01, 2010 9:14 PM To: NT System Admin Issues Subject: Re: Virtual Machine Disk Configuration The san has 21 FC disks configured as a raid 5 array. There have two luns configured from this array. All 15 guest machines are located on one of these luns (including exchange). The performance hit appears mainly on exchange but other servers are suffering as well. The interesting thing is that IOPS for the entire lun holding the 15 VM's is only around 600 at the time that the disk busy is around 95%. The disk queue for that lun is around 4 - I will need to set up longer term monitoring to get the read/write times when the disk set gets busy (which is usually first thing in the morning) On Tue, Nov 2, 2010 at 12:35 PM, Sean Martin seanmarti...@gmail.commailto:seanmarti...@gmail.com wrote: We'd need a lot more info. Start with your disk configuration. How many disks make up the storage pool or raid group the exchange lun is allocated from? Are there other luns allocated from the same pool? How many? Which exchange resources reside on the lun (stores, logs, etc.)? Disk busy warnings buy themselves don't hold much weight, unless of course there are noticeable performances issues, which you alluded to. You'll want to capture disk queue and disk latency stats ( avg disk queue length and avg disk sec/reads and writes). Avg queue depth should below X (where X equals the number of physical disks. Read/write times should avg below 20ms and not spike above 50ms. I'd also recommend running the exbpa and the troubleshooting assistant. If you can gather I/O stats you may be able to determine if the disks can handle the load. If it doesn't appear Exchange is stressing your resources, you may need your SAN monitoring tools to determine if total I/O (assuming multiple luns share the same disks) are more than the disks can handle. I'm not familiar with EVAs, but overloading disks in a SAN can lead to cache flushing which could affect performance of the array itself. - Sean On Nov 1, 2010, at 5:08 PM, Mark Milo markmilo2...@gmail.commailto:markmilo2...@gmail.com wrote: The host is Windows 2008 R2 running Hyper V. The performance is monitored via UPTIME monitoring software but I see similar high reading using perfmon (%disk time). Don't see any error messages at high utilization but the exchange 2003 server is responding poorly when disk is busy. There are 15 guest machines on this particular host but most of them dont use too much in the way of resources. On Tue, Nov 2, 2010 at 11:51 AM, Richard Stovall rich...@gmail.commailto:rich...@gmail.com wrote: On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.commailto:markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large number of VMs. If so what should the max utilization of the disk on that LUN be? I don't know of a hard and fast number/rule, but if you're using snapshot-based backup method (such as VCB) you definitely need free space on the LUNs for the snapshots to grow until the backup is finished and the snapshots can be eliminated. What virtualization platform are you running? What version? ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.commailto:listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.commailto:listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.commailto:listmana...@lyris.sunbeltsoftware.com with the body
Re: Virtual Machine Disk Configuration
I don't have stats on the eva performance as yet. What would you suggest the configuration should be for the main disk array? Just to clarify the disk pool is Vraid5 while the disks for individual guest VM's have typically been configured as raid 1 for operating system and log disks and raid 5 for data. On Tue, Nov 2, 2010 at 2:04 PM, Brian Desmond br...@briandesmond.comwrote: *Is there any sort of utilization stats on the EVA? Frankly I wouldn’t expect 21 spindles RAID5 to perform very well at all with a mixed workload like this. * * * *Thanks,* *Brian Desmond* *br...@briandesmond.com* * * *c – 312.731.3132* * * *From:* Mark Milo [mailto:markmilo2...@gmail.com] *Sent:* Monday, November 01, 2010 9:14 PM *To:* NT System Admin Issues *Subject:* Re: Virtual Machine Disk Configuration The san has 21 FC disks configured as a raid 5 array. There have two luns configured from this array. All 15 guest machines are located on one of these luns (including exchange). The performance hit appears mainly on exchange but other servers are suffering as well. The interesting thing is that IOPS for the entire lun holding the 15 VM's is only around 600 at the time that the disk busy is around 95%. The disk queue for that lun is around 4 - I will need to set up longer term monitoring to get the read/write times when the disk set gets busy (which is usually first thing in the morning) On Tue, Nov 2, 2010 at 12:35 PM, Sean Martin seanmarti...@gmail.com wrote: We'd need a lot more info. Start with your disk configuration. How many disks make up the storage pool or raid group the exchange lun is allocated from? Are there other luns allocated from the same pool? How many? Which exchange resources reside on the lun (stores, logs, etc.)? Disk busy warnings buy themselves don't hold much weight, unless of course there are noticeable performances issues, which you alluded to. You'll want to capture disk queue and disk latency stats ( avg disk queue length and avg disk sec/reads and writes). Avg queue depth should below X (where X equals the number of physical disks. Read/write times should avg below 20ms and not spike above 50ms. I'd also recommend running the exbpa and the troubleshooting assistant. If you can gather I/O stats you may be able to determine if the disks can handle the load. If it doesn't appear Exchange is stressing your resources, you may need your SAN monitoring tools to determine if total I/O (assuming multiple luns share the same disks) are more than the disks can handle. I'm not familiar with EVAs, but overloading disks in a SAN can lead to cache flushing which could affect performance of the array itself. - Sean On Nov 1, 2010, at 5:08 PM, Mark Milo markmilo2...@gmail.com wrote: The host is Windows 2008 R2 running Hyper V. The performance is monitored via UPTIME monitoring software but I see similar high reading using perfmon (%disk time). Don't see any error messages at high utilization but the exchange 2003 server is responding poorly when disk is busy. There are 15 guest machines on this particular host but most of them dont use too much in the way of resources. On Tue, Nov 2, 2010 at 11:51 AM, Richard Stovall rich...@gmail.com wrote: On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large number of VMs. If so what should the max utilization of the disk on that LUN be? I don't know of a hard and fast number/rule, but if you're using snapshot-based backup method (such as VCB) you definitely need free space on the LUNs for the snapshots to grow until the backup is finished and the snapshots can be eliminated. What virtualization platform are you running? What version? ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security
Re: Virtual Machine Disk Configuration
On the VMs, are you running multiple disks in guest-OS-based RAID configurations underneath the EVA's vRAID5 protection, with all the VHDs on the same LUN? What is the exact config for the problematic Exchange server? On Mon, Nov 1, 2010 at 11:25 PM, Mark Milo markmilo2...@gmail.com wrote: I don't have stats on the eva performance as yet. What would you suggest the configuration should be for the main disk array? Just to clarify the disk pool is Vraid5 while the disks for individual guest VM's have typically been configured as raid 1 for operating system and log disks and raid 5 for data. On Tue, Nov 2, 2010 at 2:04 PM, Brian Desmond br...@briandesmond.comwrote: *Is there any sort of utilization stats on the EVA? Frankly I wouldn’t expect 21 spindles RAID5 to perform very well at all with a mixed workload like this. * * * *Thanks,* *Brian Desmond* *br...@briandesmond.com* * * *c – 312.731.3132* * * *From:* Mark Milo [mailto:markmilo2...@gmail.com] *Sent:* Monday, November 01, 2010 9:14 PM *To:* NT System Admin Issues *Subject:* Re: Virtual Machine Disk Configuration The san has 21 FC disks configured as a raid 5 array. There have two luns configured from this array. All 15 guest machines are located on one of these luns (including exchange). The performance hit appears mainly on exchange but other servers are suffering as well. The interesting thing is that IOPS for the entire lun holding the 15 VM's is only around 600 at the time that the disk busy is around 95%. The disk queue for that lun is around 4 - I will need to set up longer term monitoring to get the read/write times when the disk set gets busy (which is usually first thing in the morning) On Tue, Nov 2, 2010 at 12:35 PM, Sean Martin seanmarti...@gmail.com wrote: We'd need a lot more info. Start with your disk configuration. How many disks make up the storage pool or raid group the exchange lun is allocated from? Are there other luns allocated from the same pool? How many? Which exchange resources reside on the lun (stores, logs, etc.)? Disk busy warnings buy themselves don't hold much weight, unless of course there are noticeable performances issues, which you alluded to. You'll want to capture disk queue and disk latency stats ( avg disk queue length and avg disk sec/reads and writes). Avg queue depth should below X (where X equals the number of physical disks. Read/write times should avg below 20ms and not spike above 50ms. I'd also recommend running the exbpa and the troubleshooting assistant. If you can gather I/O stats you may be able to determine if the disks can handle the load. If it doesn't appear Exchange is stressing your resources, you may need your SAN monitoring tools to determine if total I/O (assuming multiple luns share the same disks) are more than the disks can handle. I'm not familiar with EVAs, but overloading disks in a SAN can lead to cache flushing which could affect performance of the array itself. - Sean On Nov 1, 2010, at 5:08 PM, Mark Milo markmilo2...@gmail.com wrote: The host is Windows 2008 R2 running Hyper V. The performance is monitored via UPTIME monitoring software but I see similar high reading using perfmon (%disk time). Don't see any error messages at high utilization but the exchange 2003 server is responding poorly when disk is busy. There are 15 guest machines on this particular host but most of them dont use too much in the way of resources. On Tue, Nov 2, 2010 at 11:51 AM, Richard Stovall rich...@gmail.com wrote: On Mon, Nov 1, 2010 at 8:33 PM, Mark Milo markmilo2...@gmail.com wrote: Hi - I am having issues with disk busy' on my virtual server host. The host is a DL 580 connected via fiber to an EVA 4400 SAN. Where do you see this message? On the VM? I understand that guest VM's should have disk layouts as per an equivalent physical machine... I don't really know what you mean by that. Can you elaborate a bit? ...but is it best practice to have a separate LUN per guest VM? Not that I've ever heard of. You'd have a management nightmare on your hands pretty quickly if you had a large number of VMs. If so what should the max utilization of the disk on that LUN be? I don't know of a hard and fast number/rule, but if you're using snapshot-based backup method (such as VCB) you definitely need free space on the LUNs for the snapshots to grow until the backup is finished and the snapshots can be eliminated. What virtualization platform are you running? What version? ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/ ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog
