Re: hypothetical situation with dedup turned on
You hit the nail right on the head. Is TSM smart enough to trigger another backup and then fix any issues that might occur because of dedup? David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Alex Paschal Sent: Friday, November 16, 2012 1:09 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] hypothetical situation with dedup turned on Hi, David. You can still do as you're already doing: audit volume fix=yes to find the damaged blocks, then do a move data against the good data. That would leave the unreadable data on the volume. If the copypool volume is unavailable for a restore volume, then the only thing you could do is delete volume discarddata=yes and take the concomitant loss of data that refers to the bad blocks. TSM should then re-back up that data during the next full incremental backup. (Full incremental? Oxymoron! Also, maybe too much vodka. Stoli's Orange, tonight. ;-) Question for the IBMers: Is TSM smart enough to delete all of the file objects that refer to the deduped/damaged/discarded blocks? I would expect so, especially with the ~new DB2 referential integrity enforcement, but I think that's exactly what David's question is getting at. Could we get an authoritative answer on that? And a more egg-head question from me: if a few damaged blocks are inside an aggregate, my understanding is that the entire aggregate would be marked bad during the audit, which means TSM wouldn't be able to move data reconstruct=yes, which would cause a larger footprint of data loss. Is my hypothesis correct? Hmm. Now that I think about it, CRC would have to be enabled on the stgpool to detect those few bad blocks within an aggregate, otherwise the headers/magic numbers for the aggregate/blocks would still be readable/good and the aggregate would audit as intact. Thoughts? Another question: do file volumes get magic numbers? Haha (Sorry, I blame the vodka.) On 11/15/2012 12:58 PM, Tyree, David wrote: This a hypothetical situation. In this situation the needed tape from the copy pool is not available. I realize that the data would be lost but how what you do next? if we were still running v5 of TSM we would do a move data (MOVE VOL XXX) to save what we could then delete the volume (DEL VOL XXX). We would lose some data but the next backup cycle would rebackup any missing active data. Since we are now running v6 with dedup it seems like the process would be different. Each volume no longer contains a complete set of files. They now contain parts of files. David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Grigori Solonovitch Sent: Thursday, November 15, 2012 2:58 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] hypothetical situation with dedup turned on Have you tried to use standard copy pol to recover any problems in primary pool? Grigori G. Solonovitch Senior Technical Architect Ahli United Bank Kuwait www.ahliunited.com.kw Please consider the environment before printing this E-mail -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tyree, David Sent: Wednesday, November 14, 2012 10:36 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] hypothetical situation with dedup turned on I've had some sys admins ask me about a possible situation with using dedup on our primary storage pool. We are currently using dedup and I can't come up with a good answer. Ok, our primary storage pool is using dedup. Something (corruption, whatever) happens to one of the files in the primary pool and the data needed to recover the file in the primary pool is not available. I attempt to do a restore of the corrupt file and the needed tape is not available. How would I go about fixing that kind of a situation? Back before we started using dedup we could just do a move volume to save what we could and then do a delete volume and the next backup of the server would straighten everything out. We might lose inactive copies but the next backup cycle would catch the missing active files. With the way dedup works I'm not sure what we would do. Any suggestions? David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 Please consider the environment before printing this Email. CONFIDENTIALITY AND WAIVER: The information contained in this electronic mail message and any attachments hereto may be legally privileged and confidential. The information is intended only for the recipient(s) named in this message. If you are not the intended recipient you are notified that any use, disclosure
Re: hypothetical situation with dedup turned on
I can confirm that if you do audit volume fix=yes , then a MOVE DATA, that it LOOKS like it works. The issue you mention below hadn't occurred to me. Ick. Could turn ME into a vodka drinker.. But believe me, Ive got deduprequiresbackup yes in place, and back up to a non-deduped copy pool. (And, by the way, the term full incremental makes me twitch, even without the vodka!) -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Alex Paschal Sent: Friday, November 16, 2012 1:09 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] hypothetical situation with dedup turned on Hi, David. You can still do as you're already doing: audit volume fix=yes to find the damaged blocks, then do a move data against the good data. That would leave the unreadable data on the volume. If the copypool volume is unavailable for a restore volume, then the only thing you could do is delete volume discarddata=yes and take the concomitant loss of data that refers to the bad blocks. TSM should then re-back up that data during the next full incremental backup. (Full incremental? Oxymoron! Also, maybe too much vodka. Stoli's Orange, tonight. ;-) Question for the IBMers: Is TSM smart enough to delete all of the file objects that refer to the deduped/damaged/discarded blocks? I would expect so, especially with the ~new DB2 referential integrity enforcement, but I think that's exactly what David's question is getting at. Could we get an authoritative answer on that? And a more egg-head question from me: if a few damaged blocks are inside an aggregate, my understanding is that the entire aggregate would be marked bad during the audit, which means TSM wouldn't be able to move data reconstruct=yes, which would cause a larger footprint of data loss. Is my hypothesis correct? Hmm. Now that I think about it, CRC would have to be enabled on the stgpool to detect those few bad blocks within an aggregate, otherwise the headers/magic numbers for the aggregate/blocks would still be readable/good and the aggregate would audit as intact. Thoughts? Another question: do file volumes get magic numbers? Haha (Sorry, I blame the vodka.) On 11/15/2012 12:58 PM, Tyree, David wrote: This a hypothetical situation. In this situation the needed tape from the copy pool is not available. I realize that the data would be lost but how what you do next? if we were still running v5 of TSM we would do a move data (MOVE VOL XXX) to save what we could then delete the volume (DEL VOL XXX). We would lose some data but the next backup cycle would rebackup any missing active data. Since we are now running v6 with dedup it seems like the process would be different. Each volume no longer contains a complete set of files. They now contain parts of files. David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Grigori Solonovitch Sent: Thursday, November 15, 2012 2:58 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] hypothetical situation with dedup turned on Have you tried to use standard copy pol to recover any problems in primary pool? Grigori G. Solonovitch Senior Technical Architect Ahli United Bank Kuwait www.ahliunited.com.kw Please consider the environment before printing this E-mail -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tyree, David Sent: Wednesday, November 14, 2012 10:36 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] hypothetical situation with dedup turned on I've had some sys admins ask me about a possible situation with using dedup on our primary storage pool. We are currently using dedup and I can't come up with a good answer. Ok, our primary storage pool is using dedup. Something (corruption, whatever) happens to one of the files in the primary pool and the data needed to recover the file in the primary pool is not available. I attempt to do a restore of the corrupt file and the needed tape is not available. How would I go about fixing that kind of a situation? Back before we started using dedup we could just do a move volume to save what we could and then do a delete volume and the next backup of the server would straighten everything out. We might lose inactive copies but the next backup cycle would catch the missing active files. With the way dedup works I'm not sure what we would do. Any suggestions? David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 Please consider the environment before printing this Email. CONFIDENTIALITY AND WAIVER: The information contained in this electronic mail message and any attachments hereto may be legally privileged and confidential
Re: hypothetical situation with dedup turned on
FYI - We had all the normal settings and still lost our source dedupe data while running 6.3.0 to 6.3.1, and no TSM was not smart enough to figure out it was gone and back up a fresh copy. We are currently at 6.3.2.7. It was one of the 6.3.2.x updates, released around 10/1/2012, that finally fixed the problem, so that we don't appear to have lost any more source data since then. So make sure you are running one of the latest releases of TSM. Ray On Nov 16, 2012, at 9:51 AM, Prather, Wanda wanda.prat...@icfi.com wrote: I can confirm that if you do audit volume fix=yes , then a MOVE DATA, that it LOOKS like it works. The issue you mention below hadn't occurred to me. Ick. Could turn ME into a vodka drinker.. But believe me, Ive got deduprequiresbackup yes in place, and back up to a non-deduped copy pool. (And, by the way, the term full incremental makes me twitch, even without the vodka!) -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Alex Paschal Sent: Friday, November 16, 2012 1:09 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] hypothetical situation with dedup turned on Hi, David. You can still do as you're already doing: audit volume fix=yes to find the damaged blocks, then do a move data against the good data. That would leave the unreadable data on the volume. If the copypool volume is unavailable for a restore volume, then the only thing you could do is delete volume discarddata=yes and take the concomitant loss of data that refers to the bad blocks. TSM should then re-back up that data during the next full incremental backup. (Full incremental? Oxymoron! Also, maybe too much vodka. Stoli's Orange, tonight. ;-) Question for the IBMers: Is TSM smart enough to delete all of the file objects that refer to the deduped/damaged/discarded blocks? I would expect so, especially with the ~new DB2 referential integrity enforcement, but I think that's exactly what David's question is getting at. Could we get an authoritative answer on that? And a more egg-head question from me: if a few damaged blocks are inside an aggregate, my understanding is that the entire aggregate would be marked bad during the audit, which means TSM wouldn't be able to move data reconstruct=yes, which would cause a larger footprint of data loss. Is my hypothesis correct? Hmm. Now that I think about it, CRC would have to be enabled on the stgpool to detect those few bad blocks within an aggregate, otherwise the headers/magic numbers for the aggregate/blocks would still be readable/good and the aggregate would audit as intact. Thoughts? Another question: do file volumes get magic numbers? Haha (Sorry, I blame the vodka.) On 11/15/2012 12:58 PM, Tyree, David wrote: This a hypothetical situation. In this situation the needed tape from the copy pool is not available. I realize that the data would be lost but how what you do next? if we were still running v5 of TSM we would do a move data (MOVE VOL XXX) to save what we could then delete the volume (DEL VOL XXX). We would lose some data but the next backup cycle would rebackup any missing active data. Since we are now running v6 with dedup it seems like the process would be different. Each volume no longer contains a complete set of files. They now contain parts of files. David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Grigori Solonovitch Sent: Thursday, November 15, 2012 2:58 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] hypothetical situation with dedup turned on Have you tried to use standard copy pol to recover any problems in primary pool? Grigori G. Solonovitch Senior Technical Architect Ahli United Bank Kuwait www.ahliunited.com.kw Please consider the environment before printing this E-mail -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tyree, David Sent: Wednesday, November 14, 2012 10:36 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] hypothetical situation with dedup turned on I've had some sys admins ask me about a possible situation with using dedup on our primary storage pool. We are currently using dedup and I can't come up with a good answer. Ok, our primary storage pool is using dedup. Something (corruption, whatever) happens to one of the files in the primary pool and the data needed to recover the file in the primary pool is not available. I attempt to do a restore of the corrupt file and the needed tape is not available. How would I go about fixing that kind of a situation? Back before we started using dedup we could just do a move volume to save what we could
Re: hypothetical situation with dedup turned on
Have you tried to use standard copy pol to recover any problems in primary pool? Grigori G. Solonovitch Senior Technical Architect Ahli United Bank Kuwait www.ahliunited.com.kw Please consider the environment before printing this E-mail -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tyree, David Sent: Wednesday, November 14, 2012 10:36 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] hypothetical situation with dedup turned on I've had some sys admins ask me about a possible situation with using dedup on our primary storage pool. We are currently using dedup and I can't come up with a good answer. Ok, our primary storage pool is using dedup. Something (corruption, whatever) happens to one of the files in the primary pool and the data needed to recover the file in the primary pool is not available. I attempt to do a restore of the corrupt file and the needed tape is not available. How would I go about fixing that kind of a situation? Back before we started using dedup we could just do a move volume to save what we could and then do a delete volume and the next backup of the server would straighten everything out. We might lose inactive copies but the next backup cycle would catch the missing active files. With the way dedup works I'm not sure what we would do. Any suggestions? David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 Please consider the environment before printing this Email. CONFIDENTIALITY AND WAIVER: The information contained in this electronic mail message and any attachments hereto may be legally privileged and confidential. The information is intended only for the recipient(s) named in this message. If you are not the intended recipient you are notified that any use, disclosure, copying or distribution is prohibited. If you have received this in error please contact the sender and delete this message and any attachments from your computer system. We do not guarantee that this message or any attachment to it is secure or free from errors, computer viruses or other conditions that may damage or interfere with data, hardware or software.
Re: hypothetical situation with dedup turned on
This a hypothetical situation. In this situation the needed tape from the copy pool is not available. I realize that the data would be lost but how what you do next? if we were still running v5 of TSM we would do a move data (MOVE VOL XXX) to save what we could then delete the volume (DEL VOL XXX). We would lose some data but the next backup cycle would rebackup any missing active data. Since we are now running v6 with dedup it seems like the process would be different. Each volume no longer contains a complete set of files. They now contain parts of files. David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Grigori Solonovitch Sent: Thursday, November 15, 2012 2:58 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] hypothetical situation with dedup turned on Have you tried to use standard copy pol to recover any problems in primary pool? Grigori G. Solonovitch Senior Technical Architect Ahli United Bank Kuwait www.ahliunited.com.kw Please consider the environment before printing this E-mail -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tyree, David Sent: Wednesday, November 14, 2012 10:36 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] hypothetical situation with dedup turned on I've had some sys admins ask me about a possible situation with using dedup on our primary storage pool. We are currently using dedup and I can't come up with a good answer. Ok, our primary storage pool is using dedup. Something (corruption, whatever) happens to one of the files in the primary pool and the data needed to recover the file in the primary pool is not available. I attempt to do a restore of the corrupt file and the needed tape is not available. How would I go about fixing that kind of a situation? Back before we started using dedup we could just do a move volume to save what we could and then do a delete volume and the next backup of the server would straighten everything out. We might lose inactive copies but the next backup cycle would catch the missing active files. With the way dedup works I'm not sure what we would do. Any suggestions? David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 Please consider the environment before printing this Email. CONFIDENTIALITY AND WAIVER: The information contained in this electronic mail message and any attachments hereto may be legally privileged and confidential. The information is intended only for the recipient(s) named in this message. If you are not the intended recipient you are notified that any use, disclosure, copying or distribution is prohibited. If you have received this in error please contact the sender and delete this message and any attachments from your computer system. We do not guarantee that this message or any attachment to it is secure or free from errors, computer viruses or other conditions that may damage or interfere with data, hardware or software.
Re: hypothetical situation with dedup turned on
Hi, David. You can still do as you're already doing: audit volume fix=yes to find the damaged blocks, then do a move data against the good data. That would leave the unreadable data on the volume. If the copypool volume is unavailable for a restore volume, then the only thing you could do is delete volume discarddata=yes and take the concomitant loss of data that refers to the bad blocks. TSM should then re-back up that data during the next full incremental backup. (Full incremental? Oxymoron! Also, maybe too much vodka. Stoli's Orange, tonight. ;-) Question for the IBMers: Is TSM smart enough to delete all of the file objects that refer to the deduped/damaged/discarded blocks? I would expect so, especially with the ~new DB2 referential integrity enforcement, but I think that's exactly what David's question is getting at. Could we get an authoritative answer on that? And a more egg-head question from me: if a few damaged blocks are inside an aggregate, my understanding is that the entire aggregate would be marked bad during the audit, which means TSM wouldn't be able to move data reconstruct=yes, which would cause a larger footprint of data loss. Is my hypothesis correct? Hmm. Now that I think about it, CRC would have to be enabled on the stgpool to detect those few bad blocks within an aggregate, otherwise the headers/magic numbers for the aggregate/blocks would still be readable/good and the aggregate would audit as intact. Thoughts? Another question: do file volumes get magic numbers? Haha (Sorry, I blame the vodka.) On 11/15/2012 12:58 PM, Tyree, David wrote: This a hypothetical situation. In this situation the needed tape from the copy pool is not available. I realize that the data would be lost but how what you do next? if we were still running v5 of TSM we would do a move data (MOVE VOL XXX) to save what we could then delete the volume (DEL VOL XXX). We would lose some data but the next backup cycle would rebackup any missing active data. Since we are now running v6 with dedup it seems like the process would be different. Each volume no longer contains a complete set of files. They now contain parts of files. David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Grigori Solonovitch Sent: Thursday, November 15, 2012 2:58 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] hypothetical situation with dedup turned on Have you tried to use standard copy pol to recover any problems in primary pool? Grigori G. Solonovitch Senior Technical Architect Ahli United Bank Kuwait www.ahliunited.com.kw Please consider the environment before printing this E-mail -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Tyree, David Sent: Wednesday, November 14, 2012 10:36 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] hypothetical situation with dedup turned on I've had some sys admins ask me about a possible situation with using dedup on our primary storage pool. We are currently using dedup and I can't come up with a good answer. Ok, our primary storage pool is using dedup. Something (corruption, whatever) happens to one of the files in the primary pool and the data needed to recover the file in the primary pool is not available. I attempt to do a restore of the corrupt file and the needed tape is not available. How would I go about fixing that kind of a situation? Back before we started using dedup we could just do a move volume to save what we could and then do a delete volume and the next backup of the server would straighten everything out. We might lose inactive copies but the next backup cycle would catch the missing active files. With the way dedup works I'm not sure what we would do. Any suggestions? David Tyree Interface Analyst South Georgia Medical Center 229.333.1155 Please consider the environment before printing this Email. CONFIDENTIALITY AND WAIVER: The information contained in this electronic mail message and any attachments hereto may be legally privileged and confidential. The information is intended only for the recipient(s) named in this message. If you are not the intended recipient you are notified that any use, disclosure, copying or distribution is prohibited. If you have received this in error please contact the sender and delete this message and any attachments from your computer system. We do not guarantee that this message or any attachment to it is secure or free from errors, computer viruses or other conditions that may damage or interfere with data, hardware or software.
hypothetical situation with dedup turned on
I've had some sys admins ask me about a possible situation with using dedup on our primary storage pool. We are currently using dedup and I can't come up with a good answer. Ok, our primary storage pool is using dedup. Something (corruption, whatever) happens to one of the files in the primary pool and the data needed to recover the file in the primary pool is not available. I attempt to do a restore of the corrupt file and the needed tape is not available. How would I go about fixing that kind of a situation? Back before we started using dedup we could just do a move volume to save what we could and then do a delete volume and the next backup of the server would straighten everything out. We might lose inactive copies but the next backup cycle would catch the missing active files. With the way dedup works I'm not sure what we would do. Any suggestions? David Tyree Interface Analyst South Georgia Medical Center 229.333.1155
