Please do post results - expiration just ran for me, queue > 30M! 45 TB dedup pool
-----Original Message----- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of James R Owen Sent: Friday, December 20, 2013 11:19 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Deduplication "number of chunks waiting in queue" continues to rise? Sergio and Wanda, Thanks for your posts! I opened PMR 10702,L6Q,000 a couple weeks ago for slow performance [recently completely fell off the cliff!] with our SRV3 TSM v6.3.4.200 service that *was* successfully doing client+server deduplication for 72TB BackupDedup STGpool on NetApp FC [soon to be 3par] FC disks. I did not previously know about this command... SHow DEDUPDelinfo now shows >7M enqueued dedupdel chunks @ SRV3 TSM. I just requested escalation to consider whether TSMv6.3.4.207 efix will help us. Thanks again... hoping to re-post with better performance results soon! jim.o...@yale.edu (w#203.432.6693, c#203.494.9201, h#203.387.3030) On 12/20/2013 10:38 AM, Sergio O. Fuentes wrote: > Wanda, > > In trying to troubleshoot an unrelated performance PMR, IBM provided > me with an e-fix for the dedupdel bottleneck that it sounds like > you're experiencing. They obviously will want to do their > due-diligence on whether or not this efix will help solve your > problems, but it has proved very useful in my environment. They even > had to compile a solaris e-fix for me, cause it seems like I'm the > only one running TSM on Solaris. The e-fix was very simple to install. > > What you don't want to do is go to 6.3.4.2, unless they tell you to > because the e-fix is for that level (207). Don't run on 6.3.4.2 for > even a minute. Only install it to get to the e-fix level. > > Dedupdel gets populated by anything that deletes data from the > stgpool, I.e. move data, expire inv, delete filespace, move nodedata, etc. > > We run client-side dedupe (which works pretty well, except when you > run into performance issues on the server) and so our identifies don't > run very long, if at all. It might save you time to run client-side dedupe. > > BTW, when I finally got this efix and TSM was able to catch-up with > the deletes and reclaims as it needed to, I got some serious space > space back in my TDP Dedup pool. It went from 90% util to 60% util > (with about 10TB of total capacity). What finally really got me > before the fix was that I had to delete a bunch of old TDP MSSQL > filespaces and it just took forever for TSM to catch up. I have a few > deletes to do now, and I'm a bit wary because I don't want to hose my server > again. > > I would escalate with IBM support and have them supply you the e-fix. > 6.3.4.3 I don't think is slated for release any time within the next > few days, and you'll just be struggling to deal with the performance issue. > > HTH, > Sergio > > > > On 12/19/13 11:35 PM, "Prather, Wanda" <wanda.prat...@icfi.com> wrote: > >> TSM 6.3.4.00 on Win2K8 >> Perhaps some of you that have dealt with the dedup "chunking" problem >> can enlighten me. >> TSM/VE backs up to a dedup file pool, about 4 TB of changed blocks >> per day >> >> I currently have more than 2 TB (yep, terabytes) of volumes in that >> file pool that will not reclaim. >> We were told by support that when you do: >> >> SHOW DEDUPDELETEINFO >> That the "number of chunks waiting in queue" has to go to zero for >> those volumes to reclaim. >> >> (I know that there is a fix at 6.3.4.200 to improve the chunking >> process, but that has been APARed, and waiting on 6.3.4.300.) >> >> I have shut down IDENTIFY DUPLICATES and reclamation for this pool. >> There are no clients writing into the pool, we have redirected >> backups to a non-dedup pool for now to try and get this cleared up. >> There is no client-side dedup here, only server side. >> I've also set deduprequiresbackup to NO for now, although I hate >> doing that, to make sure that doesn't' interfere with the reclaim process. >> >> But SHOW DEDUPDELETEINFO shows that the "number of chunks waiting in >> queue" is *still* increasing. >> So, WHAT is putting stuff on that dedup delete queue? >> And how do I ever gain ground? >> >> W >> >> >> >> **Please note new office phone: >> Wanda Prather | Senior Technical Specialist | >> wanda.prat...@icfi.com >> | www.icfi.com >> ICF International | 443-718-4900 (o)
