Protectier volume shortage.
Hi Guys I'm looking for some help with managing a protectier VTL. A bit of background we have two sites TSM 7.1.7.300 and a protectier gateway at each with 1 PB of storage behind it. There are two main TSM instances at each site, one for VM snaps and one for BA Client backups. Offsite replication is by Global Mirror of the disk and Protectier replication. No TSM node replication. Each instance has its own Virtual Library So we went close to filling up one of the Protectiers a while back. Got down to 5% free space, when the limit is 10%. The situation has now improved by moving most of the BA client load on to a Data Domain, and unloading long term backups to real tape, so we are about 25% free on the protectier. Before the space issues, we were allocating Protectier carts at 100GB max. The protectier will allocate up to that limit but will truncate sooner if it runs out of space. When we got short on space we needed to allocate additional volumes and did so as we needed to. While we still specified 100GB most are smaller than that, some down as small as 30GB. What has happened now is that even though the protectier free space has improved, the size of volumes has not, and, as TSM reclaims volumes they are being truncated at a smaller size than they were. Thus we are running out of volumes again and reclaim is making things worse rather than better. A complicating factor is that if a volume is unavailable when needed by the VBS Datamover, which uses lan-free to the VTL, the VM backup fails, so running lean is not really an option. Questions: The space saving has come from a different virtual library to where the space is needed. Is this significant? Has anyone come across this and developed a management strategy? All I can think of is moving empty carts to the "shelf" and deleting them from there. If I do this from the same library with the space shortages that will make things worse in terms of number-of-volumes. Cheers Steve Steven Harris TSM Admin/Consultant Canberra Australia This message and any attachment is confidential and may be privileged or otherwise protected from disclosure. You should immediately delete the message if you are not the intended recipient. If you have received this email by mistake please delete it from your system; you should not copy the message or disclose its content to anyone. This electronic communication may contain general financial product advice but should not be relied upon or construed as a recommendation of any financial product. The information has been prepared without taking into account your objectives, financial situation or needs. You should consider the Product Disclosure Statement relating to the financial product and consult your financial adviser before making a decision about whether to acquire, hold or dispose of a financial product. For further details on the financial product please go to http://www.bt.com.au Past performance is not a reliable indicator of future performance.
Re: [EXT] [ADSM-L] VMFolder in VMware backups
VMware tagging was introduced as a better solution for managing backups (vs. VMFOLDER/subfolder). As for pagefile.sys ... one solution is to place the pagefile on its own volume and exclude that volume.You can also set this disk as Independent (VMware Edit Settings) and it will not participate in the snapshot, the vmprocessindependent=yes option will allow the backup of the VM skipping any Independent disk. Del "ADSM: Dist Stor Manager" wrote on 07/10/2018 09:17:00 AM: > From: Hans Christian Riksheim > To: ADSM-L@VM.MARIST.EDU > Date: 07/10/2018 09:17 AM > Subject: Re: [EXT] [ADSM-L] VMFolder in VMware backups > Sent by: "ADSM: Dist Stor Manager" > > Issued a PMR and from what I can gather it is theoretically impossible to > include subfolders due to VMware design. > > BTW , anyone know if there is any effort towards the ability to exclude > pagefile.sys? That one is killing my dedup ratio. > > Hans Chr. > > On Fri, Jul 6, 2018 at 11:12 AM Hans Christian Riksheim > wrote: > > > Hi Matthew. > > > > I do mostly cluster level backups but for some installations we have > > folders assigned to different customers and need a separate data mover for > > each. Without a subdir=yes type backup we get into the fun situation where > > someone has made a new sub folder without informing us. > > > > TSM has advanced and granular rules for file inclusion/exclusion but the > > TDP Vmware rules are too rudimentary to be useful in my opinion. > > > > As I understand IBM is abandoning the TSM way of doing things where the > > TSM admin controls the backup process with rules and policies and we will > > probably not see any improvement. If the responsibility of doing image > > backups is transferred to our VMware guys they will choose the backup > > software and TSM won't be on the short list. IBM should be aware of that. > > > > Hans Chr. > > > > On Thu, Jul 5, 2018 at 4:36 PM Matthew McGeary < > > matthew.mcge...@nutrien.com> wrote: > > > >> Good morning Hans, > >> > >> Subfolders have never worked with the -vmfolder option, which has driven > >> me crazy for years. I gave up and went to cluster-level backups. > >> > >> __ > >> Matthew McGeary > >> Senior Advisor, Datacenter > >> Information Technology > >> T: (306) 933-8921 > >> www.nutrien.com > >> > >> > >> -Original Message- > >> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of > >> Hans Christian Riksheim > >> Sent: Wednesday, July 4, 2018 12:21 AM > >> To: ADSM-L@VM.MARIST.EDU > >> Subject: [EXT] [ADSM-L] VMFolder in VMware backups > >> > >> WARNING: This email originated from outside of the organization. Exercise > >> caution when viewing attachments, clicking links, or responding > to requests. > >> > >> > >> Any tip to include any sub folder? I have always believed they were > >> included but now I see they are not. Not sure if this behavior has changed > >> with versions. Not helping that the documentation tells nothing about this > >> very important matter and turning yet another simple thing into a research > >> project. > >> > >> Hans C. Riksheim > >> For more information on Nutrien's email policy or to unsubscribe, click > >> here: INVALID URI REMOVED > u=https-3A__www.nutrien.com_important-2Dnotice&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=BXF- > LWtMuswSjpHce_iSqAIFmZMA3vrccCNmO8JaHNs&s=xspFi4Eewrfu2eBuPhfvJoXuBvQcvuVELPzkdsp0ZcQ&e= > >> Pour plus de renseignements sur la politique de courrier électronique > >> d’Nutrien ou pour vous désabonnez, cliquez ici : > >> INVALID URI REMOVED > u=https-3A__www.nutrien.com_avis-2Dimportant&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=BXF- > LWtMuswSjpHce_iSqAIFmZMA3vrccCNmO8JaHNs&s=tTPrBQGRIgYrlchkoLAJ7BXuluXwct- > IYK6ThdJ9pnA&e= > >> > > >
Re: [EXT] [ADSM-L] VMFolder in VMware backups
Issued a PMR and from what I can gather it is theoretically impossible to include subfolders due to VMware design. BTW , anyone know if there is any effort towards the ability to exclude pagefile.sys? That one is killing my dedup ratio. Hans Chr. On Fri, Jul 6, 2018 at 11:12 AM Hans Christian Riksheim wrote: > Hi Matthew. > > I do mostly cluster level backups but for some installations we have > folders assigned to different customers and need a separate data mover for > each. Without a subdir=yes type backup we get into the fun situation where > someone has made a new sub folder without informing us. > > TSM has advanced and granular rules for file inclusion/exclusion but the > TDP Vmware rules are too rudimentary to be useful in my opinion. > > As I understand IBM is abandoning the TSM way of doing things where the > TSM admin controls the backup process with rules and policies and we will > probably not see any improvement. If the responsibility of doing image > backups is transferred to our VMware guys they will choose the backup > software and TSM won't be on the short list. IBM should be aware of that. > > Hans Chr. > > On Thu, Jul 5, 2018 at 4:36 PM Matthew McGeary < > matthew.mcge...@nutrien.com> wrote: > >> Good morning Hans, >> >> Subfolders have never worked with the -vmfolder option, which has driven >> me crazy for years. I gave up and went to cluster-level backups. >> >> __ >> Matthew McGeary >> Senior Advisor, Datacenter >> Information Technology >> T: (306) 933-8921 >> www.nutrien.com >> >> >> -Original Message- >> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of >> Hans Christian Riksheim >> Sent: Wednesday, July 4, 2018 12:21 AM >> To: ADSM-L@VM.MARIST.EDU >> Subject: [EXT] [ADSM-L] VMFolder in VMware backups >> >> WARNING: This email originated from outside of the organization. Exercise >> caution when viewing attachments, clicking links, or responding to requests. >> >> >> Any tip to include any sub folder? I have always believed they were >> included but now I see they are not. Not sure if this behavior has changed >> with versions. Not helping that the documentation tells nothing about this >> very important matter and turning yet another simple thing into a research >> project. >> >> Hans C. Riksheim >> For more information on Nutrien's email policy or to unsubscribe, click >> here: https://www.nutrien.com/important-notice >> Pour plus de renseignements sur la politique de courrier électronique >> d’Nutrien ou pour vous désabonnez, cliquez ici : >> https://www.nutrien.com/avis-important >> >
Re: Looking for suggestions to deal with large backups not completing in 24-hours
It is possible to da a parallel backup of file system parts. https://www.gwdg.de/documents/20182/27257/GN_11-2016_www.pdf (german) have a look on page 10. --- Jonas Jansen IT Center Gruppe: Server & Storage Abteilung: Systeme & Betrieb RWTH Aachen University Seffenter Weg 23 52074 Aachen Tel: +49 241 80-28784 Fax: +49 241 80-22134 jan...@itc.rwth-aachen.de www.itc.rwth-aachen.de -Original Message- From: ADSM: Dist Stor Manager On Behalf Of Del Hoobler Sent: Monday, July 9, 2018 3:29 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Looking for suggestions to deal with large backups not completing in 24-hours They are a 3rd-party partner that offers an integrated Spectrum Protect solution for large filer backups. Del "ADSM: Dist Stor Manager" wrote on 07/09/2018 09:17:06 AM: > From: Zoltan Forray > To: ADSM-L@VM.MARIST.EDU > Date: 07/09/2018 09:17 AM > Subject: Re: Looking for suggestions to deal with large backups not > completing in 24-hours > Sent by: "ADSM: Dist Stor Manager" > > Thanks Del. Very interesting. Are they a VAR for IBM? > > Not sure if it would work in the current configuration we are using to back > up ISILON. I have passed the info on. > > BTW, FWIW, when I copied/pasted the info, Chrome spell-checker red-flagged > on "The easy way to incrementally backup billons of objects" (billions). > So if you know anybody at the company, please pass it on to them. > > On Mon, Jul 9, 2018 at 6:51 AM Del Hoobler wrote: > > > Another possible idea is to look at General Storage dsmISI MAGS: > > > > INVALID URI REMOVED > u=http-3A__www.general-2Dstorage.com_PRODUCTS_products.html&d=DwIBaQ&c=jf_ia SHvJObTbx- > siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=ofZM7gZ7p5GL1HFyHU75 lwUZLmc_kYAQxroVCZQUCSs&s=25_psxEcE0fvxruxybvMJZzSZv- > ach7r-VHXaLNVD_E&e= > > > > > > Del > > > > > > "ADSM: Dist Stor Manager" wrote on 07/05/2018 > > 02:52:27 PM: > > > > > From: Zoltan Forray > > > To: ADSM-L@VM.MARIST.EDU > > > Date: 07/05/2018 02:53 PM > > > Subject: Looking for suggestions to deal with large backups not > > > completing in 24-hours > > > Sent by: "ADSM: Dist Stor Manager" > > > > > > As I have mentioned in the past, we have gone through large migrations > > to > > > DFS based storage on EMC ISILON hardware. As you may recall, we backup > > > these DFS mounts (about 90 at last count) using multiple Windows servers > > > that run multiple ISP nodes (about 30-each) and they access each DFS > > > mount/filesystem via -object=\\rams.adp.vcu.edu\departmentname. > > > > > > This has lead to lots of performance issue with backups and some > > > departments are now complain that their backups are running into > > > multiple-days in some cases. > > > > > > One such case in a department with 2-nodes with over 30-million objects > > for > > > each node. In the past, their backups were able to finish quicker since > > > they were accessed via dedicated servers and were able to use Journaling > > to > > > reduce the scan times. Unless things have changed, I believe Journling > > is > > > not an option due to how the files are accessed. > > > > > > FWIW, average backups are usually <50k files and <200GB once it finished > > > scanning. > > > > > > Also, the idea of HSM/SPACEMANAGEMENT has reared its ugly head since > > many > > > of these objects haven't been accessed in many years old. But as I > > > understand it, that won't work either given our current configuration. > > > > > > Given the current DFS configuration (previously CIFS), what can we do to > > > improve backup performance? > > > > > > So, any-and-all ideas are up for discussion. There is even discussion > > on > > > replacing ISP/TSM due to these issues/limitations. > > > > > > -- > > > *Zoltan Forray* > > > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator > > > Xymon Monitor Administrator > > > VMware Administrator > > > Virginia Commonwealth University > > > UCC/Office of Technology Services > > > www.ucc.vcu.edu > > > zfor...@vcu.edu - 804-828-4807 > > > Don't be a phishing victim - VCU and other reputable organizations will > > > never use email to request that you reply with your password, social > > > security number or confidential personal information. For more details > > > visit INVALID URI REMOVED > > > u=http-3A__phishing.vcu.edu_&d=DwIBaQ&c=jf_iaSHvJObTbx- > > > siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=5bz_TktY3- > > > a432oKYronO-w1z- > > > ax8md3tzFqX9nGxoU&s=EudIhVvfUVx4-5UmfJHaRUzHCd7Agwk3Pog8wmEEpdA&e= > > > > > > > > -- > *Zoltan Forray* > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator > Xymon Monitor Administrator > VMware Administrator > Virginia Commonwealth University > UCC/Office of Technology Services > www.ucc.vcu.edu > zfor...@vcu.edu - 804-828-4807 > Don't be a phishing victim - VCU and other reputable organizations will > never use email to request that you reply with your passwor
