Re: Exchange Legacy 2007 to VSS 2010 backup performance
Hi Ben, I wish TSM could perform both at the same time, but it is not doable. This is a Microsoft restriction, not a TSM implementation issue. The Microsoft interfaces to check the integrity are at the whole file-level, not at a buffer-level like we would need to perform the backup and integrity check at the same time. We have asked Microsoft for a better interface, but it has not raised high enough in their queue. And with their current emphasis on replication and also allowing customers to skip the integrity check in those DAG environments, I suspect they will not invest more in this and encourage customers to use Exchange 2010 DAG replication to handle their integrity checking. Thanks, Del "ADSM: Dist Stor Manager" wrote on 08/25/2011 04:41:08 PM: >> From: Ben Bullock >> To: ADSM-L@vm.marist.edu >> Date: 08/25/2011 04:57 PM >> Subject: Re: Exchange Legacy 2007 to VSS 2010 backup performance >> Sent by: "ADSM: Dist Stor Manager" >> >> Del, >> Thanks for piping in with the official word, since all my >> statements were "anecdotal" in nature. I wasn't too far off though. ;-) >> >> I actually like the idea of the integrity check being performed at >> the same time as the backup and then failing the backup if it finds >> anything. It just seems more efficient than reading the data twice. >> Our SAN admins I'm sure would prefer the lesser number of IOPS on >> the storage. >> >> Are there any options/flags I can use to make it behave that way? >> >> Thanks, >> Ben
Ang: Re: [ADSM-L] Exchange Legacy 2007 to VSS 2010 backup performance
Ben, not sure if you have counted it in, but have you turned off the mailbox history part? We have an Exchange environment running 4x2TB and the integrity check wasnt really the big issue, but with 19000 users, the mailbox history was. Daniel Sparrman Exist i Stockholm AB Växel: 08-754 98 00 Fax: 08-754 97 30 daniel.sparr...@exist.se http://www.existgruppen.se Posthusgatan 1 761 30 NORRTÄLJE -"ADSM: Dist Stor Manager" skrev: - Till: ADSM-L@VM.MARIST.EDU Från: Del Hoobler Sänt av: "ADSM: Dist Stor Manager" Datum: 08/25/2011 22:22 Ärende: Re: [ADSM-L] Exchange Legacy 2007 to VSS 2010 backup performance For the most part, your description is accurate... but I have a few comments: - Actually... an integrity check was performed for legacy backups, but the Exchange Server did it while it was reading the data, so the penalty was not as bad. If the Exchange server found an integrity problem while running the check, it would fail the backup. - The integrity check for VSS backups is done after the snapshot, and it must read all the pages of the .EDB and .LOG files using an Exchange interface to do it - Microsoft has relaxed their requirement for certain Exchange 2010 environments. They say: "Database mobility features, including Database Availability Groups (DAGs), provide more flexible and more reliable database replication. For databases in a DAG that have two or more healthy copies, the database consistency checking step can even be skipped." Source: http://msdn.microsoft.com/en-us/library/dd877010%28v=exchg.140%29.aspx - Since VSS operations use the BA client (DSMAGENT) to perform the backup of files, you can try increasing the RESOURCEUTILIZATION in the DSM.OPT file for the DSMAGENT (not the TDP) to allocate more threads to handle the backup load. - You can also create separate instances for backing up, but... if you do this, they should be snapping separate volumes. If all your databases or logs occupy the same volume, you cannot split them up. In addition, you will want to stagger your backup starts so that the snaps of the different volumes. - Perform backups from the DAG passive copies so that it offloads the impact to production databases. Thanks, Del "ADSM: Dist Stor Manager" wrote on 08/25/2011 12:24:42 PM: >> From: Ben Bullock >> To: ADSM-L@vm.marist.edu >> Date: 08/25/2011 03:37 PM >> Subject: Re: Exchange Legacy 2007 to VSS 2010 backup performance >> Sent by: "ADSM: Dist Stor Manager" >> >> We recently moved from Exchange 2007 to 2010 on our environment. >> Just for a reference, we have about 3.3 TB of exchange data. Our >> TSM server is v5.5.5.0 running on AIX 6.1. The exchange servers >> have v6.1.3.0 of the TDP client, v6.2.2.0 of the BA client, Windows >> 2008 R2 SP1 and Exchange 2010 SP1. >> >> We have found that the transfer of the data runs about the same >> once it starts moving data to TSM, but the VSS parts (which are no >> longer optional) caused our backups to take almost twice as long. >> >> We weren't able to find anything in the TDP documentation to tell >> us why backups were now taking twice as long, however by looking at >> the IO on the SAN and Exchange logs, we believe we were able to >> determine what was going on. Feel free to chime in if you think our >> assessment is incorrect. >> >> By default, the TDP causes the exchange software do its own an >> integrity check of the database every time it does a backup, >> (either full, incremental or differential). We found that >> essentially doubles the time the backup takes, because it seems to >> read/check what it wants to send for the integrity check and then >> read/send the data to TDP/TSM on another pass. So you are >> essentially reading all the bits twice for every backup. It seems >> like a rather inefficient way to run it, but perhaps that's the way >> it has to be done. >> >> There's a flag you can put in so that the integrity check is not >> called (/SKIPINTEGRITYCHECK), but there is an inherent risk in >> skipping it. Then again, it didn't do this check for the Legacy >> backups and we never had a corruption problem with the backups, but >> YMMV. Currently, a weekly full and daily differentials meet our >> SLA, and we are toying with the idea of leaving the integrity check >> on the fulls (and have it take twice as long as it used to) and >> turning it off for the differentials. Your choice may be different >> depending on your risk assessment. >> >> Test it out for yourself and let us know if your experiences are >> any different from ours. >
Re: Exchange Legacy 2007 to VSS 2010 backup performance
Del, Thanks for piping in with the official word, since all my statements were "anecdotal" in nature. I wasn't too far off though. ;-) I actually like the idea of the integrity check being performed at the same time as the backup and then failing the backup if it finds anything. It just seems more efficient than reading the data twice. Our SAN admins I'm sure would prefer the lesser number of IOPS on the storage. Are there any options/flags I can use to make it behave that way? Thanks, Ben -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Del Hoobler Sent: Thursday, August 25, 2011 2:23 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Exchange Legacy 2007 to VSS 2010 backup performance For the most part, your description is accurate... but I have a few comments: - Actually... an integrity check was performed for legacy backups, but the Exchange Server did it while it was reading the data, so the penalty was not as bad. If the Exchange server found an integrity problem while running the check, it would fail the backup. - The integrity check for VSS backups is done after the snapshot, and it must read all the pages of the .EDB and .LOG files using an Exchange interface to do it - Microsoft has relaxed their requirement for certain Exchange 2010 environments. They say: "Database mobility features, including Database Availability Groups (DAGs), provide more flexible and more reliable database replication. For databases in a DAG that have two or more healthy copies, the database consistency checking step can even be skipped." Source: http://msdn.microsoft.com/en-us/library/dd877010%28v=exchg.140%29.aspx - Since VSS operations use the BA client (DSMAGENT) to perform the backup of files, you can try increasing the RESOURCEUTILIZATION in the DSM.OPT file for the DSMAGENT (not the TDP) to allocate more threads to handle the backup load. - You can also create separate instances for backing up, but... if you do this, they should be snapping separate volumes. If all your databases or logs occupy the same volume, you cannot split them up. In addition, you will want to stagger your backup starts so that the snaps of the different volumes. - Perform backups from the DAG passive copies so that it offloads the impact to production databases. Thanks, Del "ADSM: Dist Stor Manager" wrote on 08/25/2011 12:24:42 PM: >> From: Ben Bullock >> To: ADSM-L@vm.marist.edu >> Date: 08/25/2011 03:37 PM >> Subject: Re: Exchange Legacy 2007 to VSS 2010 backup performance Sent >> by: "ADSM: Dist Stor Manager" >> >> We recently moved from Exchange 2007 to 2010 on our environment. >> Just for a reference, we have about 3.3 TB of exchange data. Our TSM >> server is v5.5.5.0 running on AIX 6.1. The exchange servers have >> v6.1.3.0 of the TDP client, v6.2.2.0 of the BA client, Windows >> 2008 R2 SP1 and Exchange 2010 SP1. >> >> We have found that the transfer of the data runs about the same once >> it starts moving data to TSM, but the VSS parts (which are no longer >> optional) caused our backups to take almost twice as long. >> >> We weren't able to find anything in the TDP documentation to tell us >> why backups were now taking twice as long, however by looking at the >> IO on the SAN and Exchange logs, we believe we were able to determine >> what was going on. Feel free to chime in if you think our assessment >> is incorrect. >> >> By default, the TDP causes the exchange software do its own an >> integrity check of the database every time it does a backup, (either >> full, incremental or differential). We found that essentially doubles >> the time the backup takes, because it seems to read/check what it >> wants to send for the integrity check and then read/send the data to >> TDP/TSM on another pass. So you are essentially reading all the bits >> twice for every backup. It seems like a rather inefficient way to run >> it, but perhaps that's the way it has to be done. >> >> There's a flag you can put in so that the integrity check is not >> called (/SKIPINTEGRITYCHECK), but there is an inherent risk in >> skipping it. Then again, it didn't do this check for the Legacy >> backups and we never had a corruption problem with the backups, but >> YMMV. Currently, a weekly full and daily differentials meet our SLA, >> and we are toying with the idea of leaving the integrity check on the >> fulls (and have it take twice as long as it used to) and turning it >> off for the differentials. Your choice may be different depending on >> y
Re: Exchange Legacy 2007 to VSS 2010 backup performance
For the most part, your description is accurate... but I have a few comments: - Actually... an integrity check was performed for legacy backups, but the Exchange Server did it while it was reading the data, so the penalty was not as bad. If the Exchange server found an integrity problem while running the check, it would fail the backup. - The integrity check for VSS backups is done after the snapshot, and it must read all the pages of the .EDB and .LOG files using an Exchange interface to do it - Microsoft has relaxed their requirement for certain Exchange 2010 environments. They say: "Database mobility features, including Database Availability Groups (DAGs), provide more flexible and more reliable database replication. For databases in a DAG that have two or more healthy copies, the database consistency checking step can even be skipped." Source: http://msdn.microsoft.com/en-us/library/dd877010%28v=exchg.140%29.aspx - Since VSS operations use the BA client (DSMAGENT) to perform the backup of files, you can try increasing the RESOURCEUTILIZATION in the DSM.OPT file for the DSMAGENT (not the TDP) to allocate more threads to handle the backup load. - You can also create separate instances for backing up, but... if you do this, they should be snapping separate volumes. If all your databases or logs occupy the same volume, you cannot split them up. In addition, you will want to stagger your backup starts so that the snaps of the different volumes. - Perform backups from the DAG passive copies so that it offloads the impact to production databases. Thanks, Del "ADSM: Dist Stor Manager" wrote on 08/25/2011 12:24:42 PM: >> From: Ben Bullock >> To: ADSM-L@vm.marist.edu >> Date: 08/25/2011 03:37 PM >> Subject: Re: Exchange Legacy 2007 to VSS 2010 backup performance >> Sent by: "ADSM: Dist Stor Manager" >> >> We recently moved from Exchange 2007 to 2010 on our environment. >> Just for a reference, we have about 3.3 TB of exchange data. Our >> TSM server is v5.5.5.0 running on AIX 6.1. The exchange servers >> have v6.1.3.0 of the TDP client, v6.2.2.0 of the BA client, Windows >> 2008 R2 SP1 and Exchange 2010 SP1. >> >> We have found that the transfer of the data runs about the same >> once it starts moving data to TSM, but the VSS parts (which are no >> longer optional) caused our backups to take almost twice as long. >> >> We weren't able to find anything in the TDP documentation to tell >> us why backups were now taking twice as long, however by looking at >> the IO on the SAN and Exchange logs, we believe we were able to >> determine what was going on. Feel free to chime in if you think our >> assessment is incorrect. >> >> By default, the TDP causes the exchange software do its own an >> integrity check of the database every time it does a backup, >> (either full, incremental or differential). We found that >> essentially doubles the time the backup takes, because it seems to >> read/check what it wants to send for the integrity check and then >> read/send the data to TDP/TSM on another pass. So you are >> essentially reading all the bits twice for every backup. It seems >> like a rather inefficient way to run it, but perhaps that's the way >> it has to be done. >> >> There's a flag you can put in so that the integrity check is not >> called (/SKIPINTEGRITYCHECK), but there is an inherent risk in >> skipping it. Then again, it didn't do this check for the Legacy >> backups and we never had a corruption problem with the backups, but >> YMMV. Currently, a weekly full and daily differentials meet our >> SLA, and we are toying with the idea of leaving the integrity check >> on the fulls (and have it take twice as long as it used to) and >> turning it off for the differentials. Your choice may be different >> depending on your risk assessment. >> >> Test it out for yourself and let us know if your experiences are >> any different from ours. >> >> Ben >> >> -Original Message- >> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On >> Behalf Of Ian Smith >> Sent: Wednesday, August 24, 2011 10:12 AM >> To: ADSM-L@VM.MARIST.EDU >> Subject: [ADSM-L] Exchange Legacy 2007 to VSS 2010 backup performance >> >> Our Mail/GroupWare service is migrating from Exchange 2007 to 2010 >> in the next few months. Currently we employ streaming (Legacy) >> backups across a private 2Gbit bonded private link direct to LTO5 >> tape and get around 50MByte/s for fulls and around 16-20Mbyte/s for >> incr
Re: Exchange Legacy 2007 to VSS 2010 backup performance
We recently moved from Exchange 2007 to 2010 on our environment. Just for a reference, we have about 3.3 TB of exchange data. Our TSM server is v5.5.5.0 running on AIX 6.1. The exchange servers have v6.1.3.0 of the TDP client, v6.2.2.0 of the BA client, Windows 2008 R2 SP1 and Exchange 2010 SP1. We have found that the transfer of the data runs about the same once it starts moving data to TSM, but the VSS parts (which are no longer optional) caused our backups to take almost twice as long. We weren't able to find anything in the TDP documentation to tell us why backups were now taking twice as long, however by looking at the IO on the SAN and Exchange logs, we believe we were able to determine what was going on. Feel free to chime in if you think our assessment is incorrect. By default, the TDP causes the exchange software do its own an integrity check of the database every time it does a backup, (either full, incremental or differential). We found that essentially doubles the time the backup takes, because it seems to read/check what it wants to send for the integrity check and then read/send the data to TDP/TSM on another pass. So you are essentially reading all the bits twice for every backup. It seems like a rather inefficient way to run it, but perhaps that's the way it has to be done. There's a flag you can put in so that the integrity check is not called (/SKIPINTEGRITYCHECK), but there is an inherent risk in skipping it. Then again, it didn't do this check for the Legacy backups and we never had a corruption problem with the backups, but YMMV. Currently, a weekly full and daily differentials meet our SLA, and we are toying with the idea of leaving the integrity check on the fulls (and have it take twice as long as it used to) and turning it off for the differentials. Your choice may be different depending on your risk assessment. Test it out for yourself and let us know if your experiences are any different from ours. Ben -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Ian Smith Sent: Wednesday, August 24, 2011 10:12 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Exchange Legacy 2007 to VSS 2010 backup performance Our Mail/GroupWare service is migrating from Exchange 2007 to 2010 in the next few months. Currently we employ streaming (Legacy) backups across a private 2Gbit bonded private link direct to LTO5 tape and get around 50MByte/s for fulls and around 16-20Mbyte/s for incrementals. In all we have about 20TB of mailstore. I've searched the threads and the user docs, developerworks etc etc but can't find any real figures on how legacy versus vss backups compare. I suspect they will be slower, because of the additional steps in the whole VSS backup protocol but I'm happy to be surprised. If anybody has any real world figures that address the questions below, I'd be truly grateful. 1. Legacy versus VSS-assisted (MS VSS provider) backups of Exchange 2007 to TSM server. 2. VSS-assisted backups of Exchange 2007 versus 2010. This is a big migration for all of us and I'd like to have an idea of what we should expect in testing. Thanks Ian Smith Oxford University/ England. The BCI Email Firewall made the following annotations - *Confidentiality Notice: This E-Mail is intended only for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you have received this communication in error, please do not distribute, and delete the original message. Thank you for your compliance. You may contact us at: Blue Cross of Idaho 3000 E. Pine Ave. Meridian, Idaho 83642 1.208.345.4550 -
Exchange Legacy 2007 to VSS 2010 backup performance
Our Mail/GroupWare service is migrating from Exchange 2007 to 2010 in the next few months. Currently we employ streaming (Legacy) backups across a private 2Gbit bonded private link direct to LTO5 tape and get around 50MByte/s for fulls and around 16-20Mbyte/s for incrementals. In all we have about 20TB of mailstore. I've searched the threads and the user docs, developerworks etc etc but can't find any real figures on how legacy versus vss backups compare. I suspect they will be slower, because of the additional steps in the whole VSS backup protocol but I'm happy to be surprised. If anybody has any real world figures that address the questions below, I'd be truly grateful. 1. Legacy versus VSS-assisted (MS VSS provider) backups of Exchange 2007 to TSM server. 2. VSS-assisted backups of Exchange 2007 versus 2010. This is a big migration for all of us and I'd like to have an idea of what we should expect in testing. Thanks Ian Smith Oxford University/ England.
