Re: TSM 7.1 usage of volumes for dedupe
I agree. As a wise man once said, Stupid is as stupid does. In this case, the problem is really a lack of design. Running out of space on the LUN is inevitable, even a first year ComSci student could see that. So, the fact that TSM does not handle it properly is because they did not design a solution. The fact that the admin has to jump through hoops to clear it up once it does happen is a pretty good indication that something is broken. At least, I can remember a time when that is how it would have been viewed and IBM would have agreed. Perhaps I've just been around too long. I'm still willing to tilt at windmills once in awhile, so I'll give it a shot when I'm back from my trip and see what happens. Martha On 10/30/2014 5:25 PM, Remco Post wrote: Op 30 okt. 2014, om 22:04 heeft Colwell, William F. bcolw...@draper.com het volgende geschreven: Hi Martha, I am glad this was useful to you. I have not reported this as a bug; I expect they would say working-as-designed, try submitting an rfe. I never understood that working as designed is a reason to close a call. If the design is bodged, then it should be fixed. I’ve been told that even TSM is designed by people, and I’m sure sometimes those do make mistakes. - bill -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Thursday, October 30, 2014 10:09 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM 7.1 usage of volumes for dedupe Bill, I just wanted to let you know how much this information helped. I was able to clear out all the problem volumes and have removed the full LUNs from the devclass until there is enough space on them to be used again. This situation really seems strange to me. Why has TSM not been updated to handle the out of space condition better? If it has a command that shows how much space is left on the LUN, why can't TSM understand it is time to stop allocating volumes on it? Forcing admins to do manual clean up like this just to keep things healthy seems inconsistent with how the rest of TSM functions. Has anyone ever reported this as a bug? Martha On 10/22/2014 2:38 PM, Colwell, William F. wrote: Hi Martha, I see this situation occur when a filesystem gets almost completely full. Do 'q dirsp dev-class-name' to check for nearly full filesystems. The server doesn't fence off a filesystem like this, instead it keeps hammering on it, allocating new volumes. When it tries to write to a volume and gets an immediate out-of-space error, it marks the volume full so it won't try to use it again. I run this sql to find such volumes and delete them - select 'del v '||cast(volume_name as char(40)), cast(stgpool_name as char(30)), last_write_date - from volumes where upper(status) = 'FULL' and pct_utilized = 0 and pct_reclaim = 0 order by 2, 3 You should remove such filesystems from the devclass directory list until reclaim has emptied them a little bit. Hope his helps, Bill Colwell Draper Lab -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Wednesday, October 22, 2014 2:23 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM 7.1 usage of volumes for dedupe Interesting. Seems very similar, except the status of these volumes is FULL, not EMPTY. However, the %reclaimable space is 0.0. I think this is a bug. I would expect the volume to leave the pool once it is reclaimed. It would be OK with me if it did not. However, since the status is FULL, it will never be reused. That seems wrong. If it is going to remain attached to the dedupepool, the status should convert to EMPTY so the file can be reused. Or, go away altogether so the space can be reclaimed and reused. In looking at the filesystem on the Linux side (sorry I didn't mention this is running on RHEL), the file exists on /data0, but with no size: [urmm@tsmserver data0]$ ls -l *d57* -rw--- 1 tsminst1 tsmsrvrs 0 Oct 10 20:22 0d57.bfs /data0 is 100% utilized, so this file can never grow. Seems like it should get cleaned up rather than continue to exist. Martha On 10/22/2014 1:58 PM, Erwann SIMON wrote: hi Martha, See if this can apply : www-01.ibm.com/support/docview.wss?uid=swg21685554 Note that I had a situation where Q CONT returned that the volume was empty but it wasn't in reality since it was impossible to delete it (without discrading data). A select statement against the contents showed some files. Unforunately, I don't know how this story finished... -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 Notice: This email and any attachments may contain proprietary (Draper non-public) and/or export-controlled information of Draper Laboratory. If you are not the intended recipient of this email, please immediately notify the sender by replying
Re: TSM 7.1 usage of volumes for dedupe
Bill, I just wanted to let you know how much this information helped. I was able to clear out all the problem volumes and have removed the full LUNs from the devclass until there is enough space on them to be used again. This situation really seems strange to me. Why has TSM not been updated to handle the out of space condition better? If it has a command that shows how much space is left on the LUN, why can't TSM understand it is time to stop allocating volumes on it? Forcing admins to do manual clean up like this just to keep things healthy seems inconsistent with how the rest of TSM functions. Has anyone ever reported this as a bug? Martha On 10/22/2014 2:38 PM, Colwell, William F. wrote: Hi Martha, I see this situation occur when a filesystem gets almost completely full. Do 'q dirsp dev-class-name' to check for nearly full filesystems. The server doesn't fence off a filesystem like this, instead it keeps hammering on it, allocating new volumes. When it tries to write to a volume and gets an immediate out-of-space error, it marks the volume full so it won't try to use it again. I run this sql to find such volumes and delete them - select 'del v '||cast(volume_name as char(40)), cast(stgpool_name as char(30)), last_write_date - from volumes where upper(status) = 'FULL' and pct_utilized = 0 and pct_reclaim = 0 order by 2, 3 You should remove such filesystems from the devclass directory list until reclaim has emptied them a little bit. Hope his helps, Bill Colwell Draper Lab -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Wednesday, October 22, 2014 2:23 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM 7.1 usage of volumes for dedupe Interesting. Seems very similar, except the status of these volumes is FULL, not EMPTY. However, the %reclaimable space is 0.0. I think this is a bug. I would expect the volume to leave the pool once it is reclaimed. It would be OK with me if it did not. However, since the status is FULL, it will never be reused. That seems wrong. If it is going to remain attached to the dedupepool, the status should convert to EMPTY so the file can be reused. Or, go away altogether so the space can be reclaimed and reused. In looking at the filesystem on the Linux side (sorry I didn't mention this is running on RHEL), the file exists on /data0, but with no size: [urmm@tsmserver data0]$ ls -l *d57* -rw--- 1 tsminst1 tsmsrvrs 0 Oct 10 20:22 0d57.bfs /data0 is 100% utilized, so this file can never grow. Seems like it should get cleaned up rather than continue to exist. Martha On 10/22/2014 1:58 PM, Erwann SIMON wrote: hi Martha, See if this can apply : www-01.ibm.com/support/docview.wss?uid=swg21685554 Note that I had a situation where Q CONT returned that the volume was empty but it wasn't in reality since it was impossible to delete it (without discrading data). A select statement against the contents showed some files. Unforunately, I don't know how this story finished... -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 Notice: This email and any attachments may contain proprietary (Draper non-public) and/or export-controlled information of Draper Laboratory. If you are not the intended recipient of this email, please immediately notify the sender by replying to this email and immediately destroy all copies of this email. -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Hi Martha, I am glad this was useful to you. I have not reported this as a bug; I expect they would say working-as-designed, try submitting an rfe. - bill -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Thursday, October 30, 2014 10:09 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM 7.1 usage of volumes for dedupe Bill, I just wanted to let you know how much this information helped. I was able to clear out all the problem volumes and have removed the full LUNs from the devclass until there is enough space on them to be used again. This situation really seems strange to me. Why has TSM not been updated to handle the out of space condition better? If it has a command that shows how much space is left on the LUN, why can't TSM understand it is time to stop allocating volumes on it? Forcing admins to do manual clean up like this just to keep things healthy seems inconsistent with how the rest of TSM functions. Has anyone ever reported this as a bug? Martha On 10/22/2014 2:38 PM, Colwell, William F. wrote: Hi Martha, I see this situation occur when a filesystem gets almost completely full. Do 'q dirsp dev-class-name' to check for nearly full filesystems. The server doesn't fence off a filesystem like this, instead it keeps hammering on it, allocating new volumes. When it tries to write to a volume and gets an immediate out-of-space error, it marks the volume full so it won't try to use it again. I run this sql to find such volumes and delete them - select 'del v '||cast(volume_name as char(40)), cast(stgpool_name as char(30)), last_write_date - from volumes where upper(status) = 'FULL' and pct_utilized = 0 and pct_reclaim = 0 order by 2, 3 You should remove such filesystems from the devclass directory list until reclaim has emptied them a little bit. Hope his helps, Bill Colwell Draper Lab -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Wednesday, October 22, 2014 2:23 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM 7.1 usage of volumes for dedupe Interesting. Seems very similar, except the status of these volumes is FULL, not EMPTY. However, the %reclaimable space is 0.0. I think this is a bug. I would expect the volume to leave the pool once it is reclaimed. It would be OK with me if it did not. However, since the status is FULL, it will never be reused. That seems wrong. If it is going to remain attached to the dedupepool, the status should convert to EMPTY so the file can be reused. Or, go away altogether so the space can be reclaimed and reused. In looking at the filesystem on the Linux side (sorry I didn't mention this is running on RHEL), the file exists on /data0, but with no size: [urmm@tsmserver data0]$ ls -l *d57* -rw--- 1 tsminst1 tsmsrvrs 0 Oct 10 20:22 0d57.bfs /data0 is 100% utilized, so this file can never grow. Seems like it should get cleaned up rather than continue to exist. Martha On 10/22/2014 1:58 PM, Erwann SIMON wrote: hi Martha, See if this can apply : www-01.ibm.com/support/docview.wss?uid=swg21685554 Note that I had a situation where Q CONT returned that the volume was empty but it wasn't in reality since it was impossible to delete it (without discrading data). A select statement against the contents showed some files. Unforunately, I don't know how this story finished... -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 Notice: This email and any attachments may contain proprietary (Draper non-public) and/or export-controlled information of Draper Laboratory. If you are not the intended recipient of this email, please immediately notify the sender by replying to this email and immediately destroy all copies of this email. -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 Notice: This email and any attachments may contain proprietary (Draper non-public) and/or export-controlled information of Draper Laboratory. If you are not the intended recipient of this email, please immediately notify the sender by replying to this email and immediately destroy all copies of this email.
Re: TSM 7.1 usage of volumes for dedupe
Op 30 okt. 2014, om 22:04 heeft Colwell, William F. bcolw...@draper.com het volgende geschreven: Hi Martha, I am glad this was useful to you. I have not reported this as a bug; I expect they would say working-as-designed, try submitting an rfe. I never understood that working as designed is a reason to close a call. If the design is bodged, then it should be fixed. I’ve been told that even TSM is designed by people, and I’m sure sometimes those do make mistakes. - bill -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Thursday, October 30, 2014 10:09 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM 7.1 usage of volumes for dedupe Bill, I just wanted to let you know how much this information helped. I was able to clear out all the problem volumes and have removed the full LUNs from the devclass until there is enough space on them to be used again. This situation really seems strange to me. Why has TSM not been updated to handle the out of space condition better? If it has a command that shows how much space is left on the LUN, why can't TSM understand it is time to stop allocating volumes on it? Forcing admins to do manual clean up like this just to keep things healthy seems inconsistent with how the rest of TSM functions. Has anyone ever reported this as a bug? Martha On 10/22/2014 2:38 PM, Colwell, William F. wrote: Hi Martha, I see this situation occur when a filesystem gets almost completely full. Do 'q dirsp dev-class-name' to check for nearly full filesystems. The server doesn't fence off a filesystem like this, instead it keeps hammering on it, allocating new volumes. When it tries to write to a volume and gets an immediate out-of-space error, it marks the volume full so it won't try to use it again. I run this sql to find such volumes and delete them - select 'del v '||cast(volume_name as char(40)), cast(stgpool_name as char(30)), last_write_date - from volumes where upper(status) = 'FULL' and pct_utilized = 0 and pct_reclaim = 0 order by 2, 3 You should remove such filesystems from the devclass directory list until reclaim has emptied them a little bit. Hope his helps, Bill Colwell Draper Lab -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Wednesday, October 22, 2014 2:23 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM 7.1 usage of volumes for dedupe Interesting. Seems very similar, except the status of these volumes is FULL, not EMPTY. However, the %reclaimable space is 0.0. I think this is a bug. I would expect the volume to leave the pool once it is reclaimed. It would be OK with me if it did not. However, since the status is FULL, it will never be reused. That seems wrong. If it is going to remain attached to the dedupepool, the status should convert to EMPTY so the file can be reused. Or, go away altogether so the space can be reclaimed and reused. In looking at the filesystem on the Linux side (sorry I didn't mention this is running on RHEL), the file exists on /data0, but with no size: [urmm@tsmserver data0]$ ls -l *d57* -rw--- 1 tsminst1 tsmsrvrs 0 Oct 10 20:22 0d57.bfs /data0 is 100% utilized, so this file can never grow. Seems like it should get cleaned up rather than continue to exist. Martha On 10/22/2014 1:58 PM, Erwann SIMON wrote: hi Martha, See if this can apply : www-01.ibm.com/support/docview.wss?uid=swg21685554 Note that I had a situation where Q CONT returned that the volume was empty but it wasn't in reality since it was impossible to delete it (without discrading data). A select statement against the contents showed some files. Unforunately, I don't know how this story finished... -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 Notice: This email and any attachments may contain proprietary (Draper non-public) and/or export-controlled information of Draper Laboratory. If you are not the intended recipient of this email, please immediately notify the sender by replying to this email and immediately destroy all copies of this email. -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 Notice: This email and any attachments may contain proprietary (Draper non-public) and/or export-controlled information of Draper Laboratory. If you are not the intended recipient of this email, please immediately notify the sender by replying to this email and immediately destroy all copies of this email. -- Met vriendelijke groeten/Kind Regards, Remco
Re: TSM 7.1 usage of volumes for dedupe
Hi Martha, have you run query content on some of the volumes to see, if they are really empty? Michael On Wed, Oct 22, 2014 at 5:47 PM, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Hi Martha! Has the storage pool been backed up to a copy pool? If the DEDUPREQUIRESBACKUP option is set to YES (default), reclaim can't happen until the BACKUP STGPOOL is done. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Michael Roesch Sent: Wednesday, October 22, 2014 12:08 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM 7.1 usage of volumes for dedupe Hi Martha, have you run query content on some of the volumes to see, if they are really empty? Michael On Wed, Oct 22, 2014 at 5:47 PM, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Yup, they are really empty. /data0/0D57.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full tsm: TSMPRIMESERVERq content /data0/0D57.BFS Session established with server TSMPRIMESERVER: Linux/x86_64 Server Version 7, Release 1, Level 1.0 Server date/time: 10/22/2014 12:54:58 Last access: 10/22/2014 11:36:27 ANR2034E QUERY CONTENT: No match found using this criteria. ANS8001I Return code 11. Martha On 10/22/2014 12:08 PM, Michael Roesch wrote: Hi Martha, have you run query content on some of the volumes to see, if they are really empty? Michael On Wed, Oct 22, 2014 at 5:47 PM, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Yeah, I do it regularly, at least once a week. We also have a replication server, but I like the security of having them on tape too. I usually run the reclamation after the backup is done. Martha On 10/22/2014 12:29 PM, Prather, Wanda wrote: Hi Martha! Has the storage pool been backed up to a copy pool? If the DEDUPREQUIRESBACKUP option is set to YES (default), reclaim can't happen until the BACKUP STGPOOL is done. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Michael Roesch Sent: Wednesday, October 22, 2014 12:08 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM 7.1 usage of volumes for dedupe Hi Martha, have you run query content on some of the volumes to see, if they are really empty? Michael On Wed, Oct 22, 2014 at 5:47 PM, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
hi Martha, See if this can apply : www-01.ibm.com/support/docview.wss?uid=swg21685554 Note that I had a situation where Q CONT returned that the volume was empty but it wasn't in reality since it was impossible to delete it (without discrading data). A select statement against the contents showed some files. Unforunately, I don't know how this story finished... -- Best regards / Cordialement / مع تحياتي Erwann SIMON - Mail original - De: Martha M McConaghy martha.mccona...@marist.edu À: ADSM-L@VM.MARIST.EDU Envoyé: Mercredi 22 Octobre 2014 17:47:14 Objet: [ADSM-L] TSM 7.1 usage of volumes for dedupe We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Martha, !!Maybe your volumes are *not* really empty: retry with... Query CONtent [volname] FOLLOWLinks=Yes[especially for deduplicated content] See Help Query CONtent for details ... including default: FOLLOWLinks=No jim.o...@yale.edu (w#203.432.6693, c#203.494.9201, h#203.387.3030) On 10/22/2014 1:05 PM, Martha M McConaghy wrote: Yup, they are really empty. /data0/0D57.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full tsm: TSMPRIMESERVERq content /data0/0D57.BFS Session established with server TSMPRIMESERVER: Linux/x86_64 Server Version 7, Release 1, Level 1.0 Server date/time: 10/22/2014 12:54:58 Last access: 10/22/2014 11:36:27 ANR2034E QUERY CONTENT: No match found using this criteria. ANS8001I Return code 11. Martha On 10/22/2014 12:08 PM, Michael Roesch wrote: Hi Martha, have you run query content on some of the volumes to see, if they are really empty? Michael On Wed, Oct 22, 2014 at 5:47 PM, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Nice try, but no luck: tsm: TSMPRIMESERVERq content /data0/0D57.BFS followlinks=yes Session established with server TSMPRIMESERVER: Linux/x86_64 Server Version 7, Release 1, Level 1.0 Server date/time: 10/22/2014 14:10:44 Last access: 10/22/2014 12:54:58 ANR2034E QUERY CONTENT: No match found using this criteria. ANS8001I Return code 11. Martha On 10/22/2014 2:01 PM, James R Owen wrote: Martha, !!Maybe your volumes are *not* really empty: retry with... Query CONtent [volname] FOLLOWLinks=Yes[especially for deduplicated content] See Help Query CONtent for details ... including default: FOLLOWLinks=No jim.o...@yale.edu (w#203.432.6693, c#203.494.9201, h#203.387.3030) On 10/22/2014 1:05 PM, Martha M McConaghy wrote: Yup, they are really empty. /data0/0D57.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full tsm: TSMPRIMESERVERq content /data0/0D57.BFS Session established with server TSMPRIMESERVER: Linux/x86_64 Server Version 7, Release 1, Level 1.0 Server date/time: 10/22/2014 12:54:58 Last access: 10/22/2014 11:36:27 ANR2034E QUERY CONTENT: No match found using this criteria. ANS8001I Return code 11. Martha On 10/22/2014 12:08 PM, Michael Roesch wrote: Hi Martha, have you run query content on some of the volumes to see, if they are really empty? Michael On Wed, Oct 22, 2014 at 5:47 PM, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Interesting. Seems very similar, except the status of these volumes is FULL, not EMPTY. However, the %reclaimable space is 0.0. I think this is a bug. I would expect the volume to leave the pool once it is reclaimed. It would be OK with me if it did not. However, since the status is FULL, it will never be reused. That seems wrong. If it is going to remain attached to the dedupepool, the status should convert to EMPTY so the file can be reused. Or, go away altogether so the space can be reclaimed and reused. In looking at the filesystem on the Linux side (sorry I didn't mention this is running on RHEL), the file exists on /data0, but with no size: [urmm@tsmserver data0]$ ls -l *d57* -rw--- 1 tsminst1 tsmsrvrs 0 Oct 10 20:22 0d57.bfs /data0 is 100% utilized, so this file can never grow. Seems like it should get cleaned up rather than continue to exist. Martha On 10/22/2014 1:58 PM, Erwann SIMON wrote: hi Martha, See if this can apply : www-01.ibm.com/support/docview.wss?uid=swg21685554 Note that I had a situation where Q CONT returned that the volume was empty but it wasn't in reality since it was impossible to delete it (without discrading data). A select statement against the contents showed some files. Unforunately, I don't know how this story finished... -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Martha, I guess your FS is quite large. You can obtain some free space by reducing the percentage of reserved blocks. Default is 5%, you can safely decrease this value to 2%, see m option of the tune2fs command. -- Best regards / Cordialement / مع تحياتي Erwann SIMON - Mail original - De: Martha M McConaghy martha.mccona...@marist.edu À: ADSM-L@VM.MARIST.EDU Envoyé: Mercredi 22 Octobre 2014 20:22:31 Objet: Re: [ADSM-L] TSM 7.1 usage of volumes for dedupe Interesting. Seems very similar, except the status of these volumes is FULL, not EMPTY. However, the %reclaimable space is 0.0. I think this is a bug. I would expect the volume to leave the pool once it is reclaimed. It would be OK with me if it did not. However, since the status is FULL, it will never be reused. That seems wrong. If it is going to remain attached to the dedupepool, the status should convert to EMPTY so the file can be reused. Or, go away altogether so the space can be reclaimed and reused. In looking at the filesystem on the Linux side (sorry I didn't mention this is running on RHEL), the file exists on /data0, but with no size: [urmm@tsmserver data0]$ ls -l *d57* -rw--- 1 tsminst1 tsmsrvrs 0 Oct 10 20:22 0d57.bfs /data0 is 100% utilized, so this file can never grow. Seems like it should get cleaned up rather than continue to exist. Martha On 10/22/2014 1:58 PM, Erwann SIMON wrote: hi Martha, See if this can apply : www-01.ibm.com/support/docview.wss?uid=swg21685554 Note that I had a situation where Q CONT returned that the volume was empty but it wasn't in reality since it was impossible to delete it (without discrading data). A select statement against the contents showed some files. Unforunately, I don't know how this story finished... -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Hi Martha, I see this situation occur when a filesystem gets almost completely full. Do 'q dirsp dev-class-name' to check for nearly full filesystems. The server doesn't fence off a filesystem like this, instead it keeps hammering on it, allocating new volumes. When it tries to write to a volume and gets an immediate out-of-space error, it marks the volume full so it won't try to use it again. I run this sql to find such volumes and delete them - select 'del v '||cast(volume_name as char(40)), cast(stgpool_name as char(30)), last_write_date - from volumes where upper(status) = 'FULL' and pct_utilized = 0 and pct_reclaim = 0 order by 2, 3 You should remove such filesystems from the devclass directory list until reclaim has emptied them a little bit. Hope his helps, Bill Colwell Draper Lab -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Wednesday, October 22, 2014 2:23 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM 7.1 usage of volumes for dedupe Interesting. Seems very similar, except the status of these volumes is FULL, not EMPTY. However, the %reclaimable space is 0.0. I think this is a bug. I would expect the volume to leave the pool once it is reclaimed. It would be OK with me if it did not. However, since the status is FULL, it will never be reused. That seems wrong. If it is going to remain attached to the dedupepool, the status should convert to EMPTY so the file can be reused. Or, go away altogether so the space can be reclaimed and reused. In looking at the filesystem on the Linux side (sorry I didn't mention this is running on RHEL), the file exists on /data0, but with no size: [urmm@tsmserver data0]$ ls -l *d57* -rw--- 1 tsminst1 tsmsrvrs 0 Oct 10 20:22 0d57.bfs /data0 is 100% utilized, so this file can never grow. Seems like it should get cleaned up rather than continue to exist. Martha On 10/22/2014 1:58 PM, Erwann SIMON wrote: hi Martha, See if this can apply : www-01.ibm.com/support/docview.wss?uid=swg21685554 Note that I had a situation where Q CONT returned that the volume was empty but it wasn't in reality since it was impossible to delete it (without discrading data). A select statement against the contents showed some files. Unforunately, I don't know how this story finished... -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 Notice: This email and any attachments may contain proprietary (Draper non-public) and/or export-controlled information of Draper Laboratory. If you are not the intended recipient of this email, please immediately notify the sender by replying to this email and immediately destroy all copies of this email.
Re: TSM 7.1 usage of volumes for dedupe
What's your colocation set too for that stgpool? On 22 Oct 2014 16:50, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Don't believe that. I have seen it before - if you can do a MOVE DATA, it isn't really empty. I think it has to do with dedup. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Wednesday, October 22, 2014 1:06 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM 7.1 usage of volumes for dedupe Yup, they are really empty. /data0/0D57.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full tsm: TSMPRIMESERVERq content /data0/0D57.BFS Session established with server TSMPRIMESERVER: Linux/x86_64 Server Version 7, Release 1, Level 1.0 Server date/time: 10/22/2014 12:54:58 Last access: 10/22/2014 11:36:27 ANR2034E QUERY CONTENT: No match found using this criteria. ANS8001I Return code 11. Martha On 10/22/2014 12:08 PM, Michael Roesch wrote: Hi Martha, have you run query content on some of the volumes to see, if they are really empty? Michael On Wed, Oct 22, 2014 at 5:47 PM, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
Re: TSM 7.1 usage of volumes for dedupe
Try show dedupdeleteinfo - if there are chunks queued or threads actively working on deleting chunks, the volumes will remain. Regards, Joerg Pohlmann -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Prather, Wanda Sent: October 22, 2014 11:50 To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM 7.1 usage of volumes for dedupe Don't believe that. I have seen it before - if you can do a MOVE DATA, it isn't really empty. I think it has to do with dedup. -Original Message- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Martha M McConaghy Sent: Wednesday, October 22, 2014 1:06 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM 7.1 usage of volumes for dedupe Yup, they are really empty. /data0/0D57.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full tsm: TSMPRIMESERVERq content /data0/0D57.BFS Session established with server TSMPRIMESERVER: Linux/x86_64 Server Version 7, Release 1, Level 1.0 Server date/time: 10/22/2014 12:54:58 Last access: 10/22/2014 11:36:27 ANR2034E QUERY CONTENT: No match found using this criteria. ANS8001I Return code 11. Martha On 10/22/2014 12:08 PM, Michael Roesch wrote: Hi Martha, have you run query content on some of the volumes to see, if they are really empty? Michael On Wed, Oct 22, 2014 at 5:47 PM, Martha M McConaghy martha.mccona...@marist.edu wrote: We just installed TSM 7.1 during the summer and have been working on migrating our backups over from our old v5.5 system. We are using deduplication for our main storage pool and it seems to work great. However, I'm concerned about how it is using the volumes in the storage pool. Since we never ran v6, I don't know if this is normal or if we have stumbled upon a bug in 7.1. So, I figured I'd ask on the list and see if any of you have some insight. Our dedupe storage pool is dev class FILE, of course. It is set up to acquire new scratch volumes as it needs over time. Originally, I had the max scratch vols allowed at 999, which seemed reasonable. After about a month, though, we hit that max and I had to keep raising it. When I query the volumes belonging to this pool, I see many, many of them in full status, with pct util=0: Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /data0/0B55.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0B8F.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BCF.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0BD6.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C16.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C2A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C63.BFS DEDUPEPOOL FILE 49.9 G 100.0 Full /data0/0C72.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C79.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C84.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C8C.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C93.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0C9A.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CA1.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full /data0/0CB3.BFS DEDUPEPOOL FILE 50.0 G 0.0 Full Literally, hundreds of them. I run reclamations, but these volumes never get touched nor reclaimed. Seems to me that they should. I've gone over the admin guide several times, but have found nothing touching on this. We just applied the 7.1.1.0 updates, but that has not helped either. If I do a move data on each, they will disappear. However, more will return to take their place. Anyone seen this before, or have any suggestions? Martha -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601 -- Martha McConaghy Marist: System Architect/Technical Lead SHARE: Director of Operations Marist College IT Poughkeepsie, NY 12601
