We have a document mgt application that stores gobs of files (documents) that are tracked by a bunch of databases. We were hitting the same kind of problems you are having in getting it backed up. And, even thought we had some kind of backup, it was clear that the server could never be restored in a DR situation. One of the first things we did to help was to SRDF the system (EMC storage) to the DR site. Over time it became clear that the overall backup/recovery design for this system was collapsing under it's own weight. We eventually had to completely change ithe way backup/recovery worked - we went to system of all snapshots and replication. Part of this was changing it's storage to NetApp, but EMC has similar tools.
Rick From: "Prather, Wanda" <wprat...@icfi.com> To: ADSM-L@VM.MARIST.EDU Date: 02/28/2011 07:30 PM Subject: Re: Ang: Windows servers with a kazillion files and Win2K8... Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> Good idea. No Netapp, it's on EMC DMX. But I'll check and see if they can snap that.... -----Original Message----- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Steven Langdale Sent: Saturday, February 26, 2011 6:25 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Ang: Windows servers with a kazillion files and Win2K8... Maybe a complete step change is needed here. It's worth thinking about a netapp/N series and snapvaulting off to a 2nd one. That works even with a huge environment. Steven On 26 Feb 2011 11:00, "Daniel Sparrman" <daniel.sparr...@exist.se> wrote: > Hi Wanda > > Two tiny questions: > > a) Do the users need instant access to the files or is long-term > archiving an option? > > b) Not sure if it would help the issue with long respons times when browsing, but HSM might be an option? > > Do you have an SLA for this system? Any decided restore times? Your biggest problem isnt now, it's when u need to restore those 70 million files. So reducing the amount of files either by HSM or archiving isnt even an option, it's something you need todo. Doing RAID, box mirroring and such wont protect the system from logical errors, so counting on HW todo the job just wont do it. The restore times needs to be acceptable from TSM. > > Upgrading to W2K8 wont solve the issue since you will still have > troubles browsing the fileserver, backing it up, and in the end, restoring it. > > Best Regards > > Daniel Sparrman > > -----"ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> skrev: ----- > > > Till: ADSM-L@VM.MARIST.EDU > Från: "Prather, Wanda" <wprat...@icfi.com> Sänt av: "ADSM: Dist Stor > Manager" <ADSM-L@VM.MARIST.EDU> > Datum: 02/25/2011 20:03 > Ärende: Windows servers with a kazillion files and Win2K8... > > I have a site with an application that generates kazillions of tiny > files that are stored forever. > I've already yelled about it, but it's a purchased, customer-facing black-box app that they really can't change. > (Naturally, when it was bought umpty years ago, nobody thought about > the problem reaching this size or what the ramifications would be.) Every day the app creates more files. > > They have multiple Win2K3 servers that already have multiple luns containing over 35M files each, one is over 75M files. > > We are using journaling to back them up successfully (most days). > But it's a struggle just to expand the file tree with Windows > explorer, and there are exposures on the days when the journal gets overrun (takes 72 hours for TSM to scan the filesystem and revalidate the journal). > > Looking for anything that might help save our bacon. > > Has anybody had experience with this issue and Win2K8? > Does Win2K8 do any better than Win2K3 at handling huge numbers of > files in 1 NTFS directory? > Upgrading the OS is something application-independent we might be able > to do. > > Thanks for any insight! > W > > > Wanda Prather | Senior Technical Specialist | wprat...@icfi.com<mailto: wprat...@icfi.com> | www.jasi.com<www.jasi.com%20> > ICF Jacob & Sundstrom | 401 E. Pratt St, Suite 2214, Baltimore, MD > 21202 | 410.539.1135 ----------------------------------------- The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.