Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
It's NTFS and you're creating and deleting a lot of files on the volume so of course it will fragment. Either defragment the volume or set the minimum threshold lower so that more files get deleted when the cleanup process runs to reduce the fragmentation. .../Ed -- Ed Wilts, Mounds View, MN, USA mailto:[EMAIL PROTECTED] Thanks for the info guys. It sounds like fragmentation is just a given when it comes to backing up to disk? I understand that, as seeing it explained does make sense. I have been looking for some of this well documented information and have come up empty. Searching for fragmentation on Symantecs site is like a journey through the looking glass. I will keep looking, but if anyone has any links to a white paper or something it would be much appreciated. Also, when you say set the minimum threshold lower so that more files get deleted... This confused me; I mean, isn't the fragmentation being caused by so many file creation/deletions? Wouldn't increasing the amount of files being deleted also increase the fragmentation? Or did I misread that? Thanks again for the info, T. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
Today's NTFS handles fragmentation alot better - in fact, FAT and FAT32 were really the main file systems that would always get fragmented. That is not to say NTFS is not immune to the fragmentation that people may experience, but there are ways around to minimise it even more. Depending on the volume itself, and its intention is the key to keeping fragmentation down. When you format a volume you get the option of a cluster size. But you must be aware of what the volume itself will be storing. (for example, large files, or millions of small files). By default, when formatting, Windows keeps a default setting in place. Choosing a smaller cluster variable will waste less disk space but likely to cause fragmentation. Likewise, a larger cluster variable will cause less fragmentation but waste space. further details can be found in the online help of Win2k3, XP, 2000, ect ! Not to put my foot in it, but I am sure other systems suffer, but maybe its a NTFS thing ;-) Simon. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tony T. Sent: Wednesday, February 20, 2008 4:07 PM To: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation It's NTFS and you're creating and deleting a lot of files on the volume so of course it will fragment. Either defragment the volume or set the minimum threshold lower so that more files get deleted when the cleanup process runs to reduce the fragmentation. .../Ed -- Ed Wilts, Mounds View, MN, USA mailto:[EMAIL PROTECTED] Thanks for the info guys. It sounds like fragmentation is just a given when it comes to backing up to disk? I understand that, as seeing it explained does make sense. I have been looking for some of this well documented information and have come up empty. Searching for fragmentation on Symantecs site is like a journey through the looking glass. I will keep looking, but if anyone has any links to a white paper or something it would be much appreciated. Also, when you say set the minimum threshold lower so that more files get deleted... This confused me; I mean, isn't the fragmentation being caused by so many file creation/deletions? Wouldn't increasing the amount of files being deleted also increase the fragmentation? Or did I misread that? Thanks again for the info, T. This email (including any attachments) may contain confidential and/or privileged information or information otherwise protected from disclosure. If you are not the intended recipient, please notify the sender immediately, do not copy this message or any attachments and do not use it for any purpose or disclose its content to any person, but delete this message and any attachments from your system. Astrium disclaims any and all liability if this email transmission was virus corrupted, altered or falsified. - Astrium Limited, Registered in England and Wales No. 2449259 REGISTERED OFFICE:- Gunnels Wood Road, Stevenage, Hertfordshire, SG1 2AS, England___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
I still think that MS's undelete feature plays a part in this. When data is deleted in Windows, NTFS marks blocks to be released without actually erasing them. Rather than reusing released blocks, NTFS prefers new, unused blocks, which leads to fragmentation. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of WEAVER, Simon (external) Sent: Wednesday, February 20, 2008 11:21 AM To: Tony T.; veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation Today's NTFS handles fragmentation alot better - in fact, FAT and FAT32 were really the main file systems that would always get fragmented. That is not to say NTFS is not immune to the fragmentation that people may experience, but there are ways around to minimise it even more. Depending on the volume itself, and its intention is the key to keeping fragmentation down. When you format a volume you get the option of a cluster size. But you must be aware of what the volume itself will be storing. (for example, large files, or millions of small files). By default, when formatting, Windows keeps a default setting in place. Choosing a smaller cluster variable will waste less disk space but likely to cause fragmentation. Likewise, a larger cluster variable will cause less fragmentation but waste space. further details can be found in the online help of Win2k3, XP, 2000, ect ! Not to put my foot in it, but I am sure other systems suffer, but maybe its a NTFS thing ;-) Simon. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tony T. Sent: Wednesday, February 20, 2008 4:07 PM To: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation It's NTFS and you're creating and deleting a lot of files on the volume so of course it will fragment. Either defragment the volume or set the minimum threshold lower so that more files get deleted when the cleanup process runs to reduce the fragmentation. .../Ed -- Ed Wilts, Mounds View, MN, USA mailto:[EMAIL PROTECTED] Thanks for the info guys. It sounds like fragmentation is just a given when it comes to backing up to disk? I understand that, as seeing it explained does make sense. I have been looking for some of this well documented information and have come up empty. Searching for fragmentation on Symantecs site is like a journey through the looking glass. I will keep looking, but if anyone has any links to a white paper or something it would be much appreciated. Also, when you say set the minimum threshold lower so that more files get deleted... This confused me; I mean, isn't the fragmentation being caused by so many file creation/deletions? Wouldn't increasing the amount of files being deleted also increase the fragmentation? Or did I misread that? Thanks again for the info, T. This email (including any attachments) may contain confidential and/or privileged information or information otherwise protected from disclosure. If you are not the intended recipient, please notify the sender immediately, do not copy this message or any attachments and do not use it for any purpose or disclose its content to any person, but delete this message and any attachments from your system. Astrium disclaims any and all liability if this email transmission was virus corrupted, altered or falsified. - Astrium Limited, Registered in England and Wales No. 2449259 REGISTERED OFFICE:- Gunnels Wood Road, Stevenage, Hertfordshire, SG1 2AS, England ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
H Not sure its common in NTFS though, however the older file systems FAT FAT32 definetely would play a part in this. Saying that, if users continually delete files from the same volume and they are restored from tape, would this increase fragmentation? From: Sesar, Steven L. [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 20, 2008 4:51 PM To: WEAVER, Simon (external); Tony T.; veritas-bu@mailman.eng.auburn.edu Subject: RE: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation I still think that MS's undelete feature plays a part in this. When data is deleted in Windows, NTFS marks blocks to be released without actually erasing them. Rather than reusing released blocks, NTFS prefers new, unused blocks, which leads to fragmentation. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of WEAVER, Simon (external) Sent: Wednesday, February 20, 2008 11:21 AM To: Tony T.; veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation Today's NTFS handles fragmentation alot better - in fact, FAT and FAT32 were really the main file systems that would always get fragmented. That is not to say NTFS is not immune to the fragmentation that people may experience, but there are ways around to minimise it even more. Depending on the volume itself, and its intention is the key to keeping fragmentation down. When you format a volume you get the option of a cluster size. But you must be aware of what the volume itself will be storing. (for example, large files, or millions of small files). By default, when formatting, Windows keeps a default setting in place. Choosing a smaller cluster variable will waste less disk space but likely to cause fragmentation. Likewise, a larger cluster variable will cause less fragmentation but waste space. further details can be found in the online help of Win2k3, XP, 2000, ect ! Not to put my foot in it, but I am sure other systems suffer, but maybe its a NTFS thing ;-) Simon. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tony T. Sent: Wednesday, February 20, 2008 4:07 PM To: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation It's NTFS and you're creating and deleting a lot of files on the volume so of course it will fragment. Either defragment the volume or set the minimum threshold lower so that more files get deleted when the cleanup process runs to reduce the fragmentation. .../Ed -- Ed Wilts, Mounds View, MN, USA mailto:[EMAIL PROTECTED] Thanks for the info guys. It sounds like fragmentation is just a given when it comes to backing up to disk? I understand that, as seeing it explained does make sense. I have been looking for some of this well documented information and have come up empty. Searching for fragmentation on Symantecs site is like a journey through the looking glass. I will keep looking, but if anyone has any links to a white paper or something it would be much appreciated. Also, when you say set the minimum threshold lower so that more files get deleted... This confused me; I mean, isn't the fragmentation being caused by so many file creation/deletions? Wouldn't increasing the amount of files being deleted also increase the fragmentation? Or did I misread that? Thanks again for the info, T. This email (including any attachments) may contain confidential and/or privileged information or information otherwise protected from disclosure. If you are not the intended recipient, please notify the sender immediately, do not copy this message or any attachments and do not use it for any purpose or disclose its content to any person, but delete this message and any attachments from your system. Astrium disclaims any and all liability if this email transmission was virus corrupted, altered or falsified. - Astrium Limited, Registered in England and Wales No. 2449259 REGISTERED OFFICE:- Gunnels Wood Road, Stevenage, Hertfordshire, SG1 2AS, England This email (including any attachments) may contain confidential and/or privileged information or information otherwise protected from disclosure. If you are not the intended recipient, please notify the sender immediately, do not copy this message or any attachments and do not use it for any purpose or disclose its content to any person, but delete this message and any attachments from your system. Astrium disclaims any and all liability if this email transmission was virus corrupted, altered or falsified. - Astrium Limited, Registered in England and Wales No. 2449259 REGISTERED OFFICE:- Gunnels Wood Road, Stevenage, Hertfordshire, SG1 2AS
Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
I'm curious if the image fragment size has any impact on file system fragmentation? I have used 2gb for disk staging on vxfs/solaris successfully, but I never got the DSU volume much past 80% full. -Jon Today's NTFS handles fragmentation alot better - in fact, FAT and FAT32 were really the main file systems that would always get fragmented. That is not to say NTFS is not immune to the fragmentation that people may experience, but there are ways around to minimise it even more. Depending on the volume itself, and its intention is the key to keeping fragmentation down. When you format a volume you get the option of a cluster size. But you must be aware of what the volume itself will be storing. (for example, large files, or millions of small files). By default, when formatting, Windows keeps a default setting in place. Choosing a smaller cluster variable will waste less disk space but likely to cause fragmentation. Likewise, a larger cluster variable will cause less fragmentation but waste space. further details can be found in the online help of Win2k3, XP, 2000, ect ! Not to put my foot in it, but I am sure other systems suffer, but maybe its a NTFS thing ;-) Simon. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
On Feb 20, 2008 11:10 AM, Martin, Jonathan [EMAIL PROTECTED] wrote: I have found through testing when we first added some 16TB of storage for Disk-2-Disk that your fragmentation is going to be directly related to how many simultaneous streams you write to a DSU/DSSU at a time. In my testing, I got a minimum amount of fragmentation with 1-2 Streams, a bit worse with 4 streams, and at my current 20 max streams we're completely 100% fragmented. Sysinternals has a took that allows you to look at your disks sector by sector to look for fragmentation, but its a bit difficult to you use on disks that are 1TB or better. I created this slide for a Netbackup Training presentation I did 6 months ago... Wow, this is excellent! Thanks much for the insight. T. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
Has anyone heard of this issue? I have an environment where NBU 5.1 (plans to get to 6.5.1 are in the works) is backing up to disk then staging to tape. The admins tell me that the area where the backups are going gets very fragmented after each backup. It is a Windows box, using NTFS. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
It is pretty well documented that any area you use for disk staging should be defragmented regularly. Reneé Carlisle ServerWare Corporation _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tony T. Sent: Tuesday, February 19, 2008 1:37 PM To: veritas-bu@mailman.eng.auburn.edu Subject: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation Has anyone heard of this issue? I have an environment where NBU 5.1 (plans to get to 6.5.1 are in the works) is backing up to disk then staging to tape. The admins tell me that the area where the backups are going gets very fragmented after each backup. It is a Windows box, using NTFS. This Email has been scanned for all viruses by PAETEC Email Scanning Services, utilizing MessageLabs proprietary SkyScan infrastructure. For more information on a proactive anti-virus service working around the clock, around the globe, visit http://www.paetec.com. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] NBU 5.1: Disk staging causing heavy fragmentation
On Feb 19, 2008 12:37 PM, Tony T. [EMAIL PROTECTED] wrote: Has anyone heard of this issue? I have an environment where NBU 5.1 (plans to get to 6.5.1 are in the works) is backing up to disk then staging to tape. The admins tell me that the area where the backups are going gets very fragmented after each backup. It is a Windows box, using NTFS. It's NTFS and you're creating and deleting a lot of files on the volume so of course it will fragment. Either defragment the volume or set the minimum threshold lower so that more files get deleted when the cleanup process runs to reduce the fragmentation. .../Ed -- Ed Wilts, Mounds View, MN, USA mailto:[EMAIL PROTECTED] ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
