Re: AFS on XFS or ext3?
Michael Mansour when are you going to stop the FUD?? Red Hat doesn't support it because it is developed and maintain by its competition. Dependability, reliability, robustness is NOT a factor, it is pure politics and nothing more. There have been plenty of people complaining that XFS isn't supported by RH & SL as much as it should be. I do NOT want a `flame war', but if you are going to continue to lie about XFS, you leave me no alternative but to support it. If your posts had said positive things about ext3 instead of lying about XFS, I would have ignored you. However, YOU started the lying, so YOU prompted the XFS support replies. I do not mind someone supporting their favorite what ever, but when they start lying about something else just to support their position, then THAT is something entirely different. Stop the lies and I'll stop the replies. It is entierly up to you.
Re: AFS on XFS or ext3?
On Tue, 17 Feb 2009, Bob Barton wrote: Hi All I am setting up a 2 TB file system to use for AFS volumes on an AFS file server and I am wondering which file system I should use - XFS or ext3. I plan to use Scientific Linux 5.1 or 5.2 x86_64 as the operating system on the file server machine. Suggestions, comments and recommendations are very welcome. Several people have said how wonderful each of XFS and ext3 are, how long they have used them and never had problems. I have no experiece of XFS but the only multi-terabyte ext3 filesystem that I use (as user not sysadmin) had some disk/controller problems and when we tried to restore the backups verification failed repeatedly 9somewhere between 3 and 4 terabytes I believe). I don't mean that fsck (I think) reported that the filesystem was corrupt, but that it ceased to make progress checking for problems. The sysadmin who actually handled this is on the list and may wish to give more details, but I understand that he is now uncomfortable using ext3 for filesystems larger than 2-3 TB. As I say, we have no way of knowing whether XFS would have been any better. -- Dr. Andrew C. Aitchison Computer Officer, DPMMS, Cambridge a.c.aitchi...@dpmms.cam.ac.uk http://www.dpmms.cam.ac.uk/~werdna
Re: AFS on XFS or ext3?
OK, Enough of this religious file system bashing. If this doesn't stop, people are going to be taken off the list. Michael, you said something that is completely false, and I need to correct what you said. Red Hat = Scientific Linux, so if it's not supported by TUV then it's not supported by SL. Red Hat does *NOT* equal Scientific Linux. Scientific Linux adds more stuff into it's release that RedHat does. Scientific Linux supports those extra products as best we can, which is often a "best effort" support. Troy -- __ Troy Dawson daw...@fnal.gov (630)840-6468 Fermilab ComputingDivision/LCSI/CSI DSS Group __
Re: AFS on XFS or ext3?
Hi Bob, > I certainly don't want to start any flame wars about choosing > between ext3 of XFS. One reason I was thinking of using XFS was > because recently when I set up an ext3 system, during the setup a > note popped up that an fsck would be forced on the file system after > 180 days. Having to take down a crucial resource for a long period > to do an fsck on 2TB of ext3 file system every 1/2 year is certainly > unattractive! I know there are ways to change this default using > tune2fs but I am uncertain what the implications of doing so are. My There is nothing wrong with extending the mount and disk check times using tune2fs, especially in production. Consider though, that as with any journaling filesystem, just because it's journaled doesn't mean it's consistent. I personally extend the lengths of checks using tune2fs on production servers but I always allow an fsck to run at some stage (either manually when I'm organising a boot or organising downtime at some stage with the customer). fsck checks many aspects of the filesystem and should be run.. eventually. > previous experience with AFS file servers has been with AIX3.x - > AIX4.x and Solaris 9 and I essentially turned the systems on and > left them alone for years (literally - they went down whenever the > building power failed for some reason or other). I am hoping that > XFS would have similar characteristics. Some advice, do a web search of people that have had problems with XFS in large environments (maybe do the same with ext3), and then make your decision. But remember one important note, when using a Red Hat based system, Red Hat themselves don't recommend XFS, they don't test it, they don't run it, so getting any type of support if you had problems with XFS on SL is just that much more difficult. Regards, Michael. > -- > Bob Barton > Local Area Administrator (780) 492-5160 > 7-095 ECERF > Chemical & Materials Engineering > University of Alberta, > Edmonton Alberta, T6G 2V4 --- End of Original Message ---
RE: AFS on XFS or ext3?
Hi Brunner, > -Original Message- > From: owner-scientific-linux-us...@listserv.fnal.gov > [mailto:owner-scientific-linux-us...@listserv.fnal.gov] On Behalf Of > Bob Barton Sent: Thursday, February 19, 2009 3:16 PM To: scientific- > linux-us...@fnal.gov Subject: Re: AFS on XFS or ext3? > > I certainly don't want to start any flame wars about choosing between > ext3 of XFS. > One reason I was thinking of using XFS was because recently when I > set up an ext3 system, during the setup a note popped up that an > fsck would be forced on the file system after 180 days. Having to > take down a crucial resource for a long period to do an fsck on 2TB > of ext3 file system every 1/2 year is certainly unattractive! I know > there are ways to change this default using tune2fs but I am > uncertain what the implications of doing so are. My previous > experience with AFS file servers has been with AIX3.x - AIX4.x and > Solaris 9 and I essentially turned the systems on and left them > alone for years (literally - they went down whenever the building > power failed for some reason or other). I am hoping that XFS would have > similar characteristics. > > == > > >From my experience with ext3, the fsck takes place upon the first reboot > that takes place 6+months after that pop-up. It doesn't watch it's > wristwatch, say it's been 6 months, and do an fsck (THAT would be > annoying). So, it waits for a power-fail or shutdown, when disk service > is normally not expected. If I'm going to boot a server and I don't want it to check the disks after the reboot, I make sure before I reboot to use tune2fs to make sure the disks won't check after n mounts etc. Get into the habit of doing that and no surprises on reboots :) Regards, Michael. > If an ext3 fs is mounted read-only (like my /usr partition) fsck > never runs on it at reboot, no matter what. > > My systems, also, are left running for years; and they're located in > charming get-aways like Kazakhstan: irate users are to be avoided at > all costs. *** > This email and any files transmitted with it are confidential and > intended solely for the use of the individual or entity to whom > they are addressed. If you have received this email in error please > notify the system manager. This footnote also confirms that this > email message has been swept for the presence of computer viruses. > www.Hubbell.com - Hubbell Incorporated** --- End of Original Message ---
Re: AFS on XFS or ext3?
Hi Brent, > Michael Mansour, cut the CRAP/FUD out! I would NOT depend on Hmm.. > ext3 if I CARED about what was stored on my disks. I ONLY use ext3 > if the data stored is NOT of "very high importance". I use XFS when > I DO CARE, so I use it all the time. XFS is the most reliable, If XFS was that reliable then Red Hat would support it commercially. They do not specifically because ext3 is more reliable and robust. Red Hat = Scientific Linux, so if it's not supported by TUV then it's not supported by SL. Don't believe me, raise a case with Red Hat and see. > dependable, and robust file system out there and independent tests > have consistently shown it to be much faster than ext3. It has far Please read my first email, ext3 can perform just as fast with various features turned off. > more YEARS and Pentabytes of service under it's belt than ext3, a > LOT more! I've had XFS do a much better job of surviving system > crashes and disk failures than ext3. Different people will give the same arguments as you do. The fact is ext3 is slower than XFS because it has more redundancy built in, turn off the redundancy features and you get the same speeds as XFS. Regards, Michael.
RE: AFS on XFS or ext3?
-Original Message- From: owner-scientific-linux-us...@listserv.fnal.gov [mailto:owner-scientific-linux-us...@listserv.fnal.gov] On Behalf Of Bob Barton Sent: Thursday, February 19, 2009 3:16 PM To: scientific-linux-us...@fnal.gov Subject: Re: AFS on XFS or ext3? I certainly don't want to start any flame wars about choosing between ext3 of XFS. One reason I was thinking of using XFS was because recently when I set up an ext3 system, during the setup a note popped up that an fsck would be forced on the file system after 180 days. Having to take down a crucial resource for a long period to do an fsck on 2TB of ext3 file system every 1/2 year is certainly unattractive! I know there are ways to change this default using tune2fs but I am uncertain what the implications of doing so are. My previous experience with AFS file servers has been with AIX3.x - AIX4.x and Solaris 9 and I essentially turned the systems on and left them alone for years (literally - they went down whenever the building power failed for some reason or other). I am hoping that XFS would have similar characteristics. == >From my experience with ext3, the fsck takes place upon the first reboot that takes place 6+months after that pop-up. It doesn't watch it's wristwatch, say it's been 6 months, and do an fsck (THAT would be annoying). So, it waits for a power-fail or shutdown, when disk service is normally not expected. If an ext3 fs is mounted read-only (like my /usr partition) fsck never runs on it at reboot, no matter what. My systems, also, are left running for years; and they're located in charming get-aways like Kazakhstan: irate users are to be avoided at all costs. *** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com - Hubbell Incorporated**
Re: AFS on XFS or ext3?
I certainly don't want to start any flame wars about choosing between ext3 of XFS. One reason I was thinking of using XFS was because recently when I set up an ext3 system, during the setup a note popped up that an fsck would be forced on the file system after 180 days. Having to take down a crucial resource for a long period to do an fsck on 2TB of ext3 file system every 1/2 year is certainly unattractive! I know there are ways to change this default using tune2fs but I am uncertain what the implications of doing so are. My previous experience with AFS file servers has been with AIX3.x - AIX4.x and Solaris 9 and I essentially turned the systems on and left them alone for years (literally - they went down whenever the building power failed for some reason or other). I am hoping that XFS would have similar characteristics. -- Bob Barton Local Area Administrator (780) 492-5160 7-095 ECERF Chemical & Materials Engineering University of Alberta, Edmonton Alberta, T6G 2V4
Re: AFS on XFS or ext3?
Brent L. Bates said... | | Michael Mansour, cut the CRAP/FUD out! I would NOT depend on ext3 if I Let's not get nasty. |CARED about what was stored on my disks. I ONLY use ext3 if the data stored |is NOT of "very high importance". I use XFS when I DO CARE, so I use it all |the time. XFS is the most reliable, dependable, and robust file system out |there and independent tests have consistently shown it to be much faster than |ext3. It has far more YEARS and Pentabytes of service under it's belt than |ext3, a LOT more! I've had XFS do a much better job of surviving system |crashes and disk failures than ext3. We use ext3 in production on desktops, compute servers, and crucial, fast storage, and have never had a problem with it. We have stuck with it because (a) it just works, (b) we're familiar with it, and (c) it installs by default. We use xfs on systems where we need something ext3 doesn't provide (such as many, many millions of inodes, etc). Both have performed flawlessly for us, ext3 since RH9 or EL3, whenever we installed it. -- Miles O'Neal Intrinsity, Inc. |m...@intrinsity.com 11612 Bee Caves Rd.|512-421-2242 (v) Bldg II / Suite 200|512-577-3133 (c) <- best bet Austin, Texas 78738|512-263-0795 (f)
Re: AFS on XFS or ext3?
Hi, we currently have a problem on SL46 with kernel 2.6.9-78.0.13.ELsmp to run a Legato backup on a 4.8 TB AFS partition on Xfs, composed of 4 physical drives by means of LVM. Contemporary there are about 1.9 TB of data to be read, but the backup job always breaks after 1.6 TB. # kernel-2.6.9-78.0.13.ELsmp.x86_64 # kernel-module-xfs-2.6.9-78.0.13.ELsmp-0.4-1.x86_64 With the older kernel and corresponding modules it worked fine. # kernel-2.6.9-78.0.8.ELsmp.x86_64 # kernel-module-xfs-2.6.9-78.0.8.ELsmp-0.4-1.x86_64 However, as you are planning to use SL5 this kind of problem will not affect you. Generally we observe much better read/write performance on xfs than on ext3. Cheers Marc Bob Barton wrote: Hi All I am setting up a 2 TB file system to use for AFS volumes on an AFS file server and I am wondering which file system I should use - XFS or ext3. I plan to use Scientific Linux 5.1 or 5.2 x86_64 as the operating system on the file server machine. Suggestions, comments and recommendations are very welcome.
Re: AFS on XFS or ext3?
Hi, On Tue, 17 Feb 2009, Bob Barton wrote: Hi All I am setting up a 2 TB file system to use for AFS volumes on an AFS file server and I am wondering which file system I should use - XFS or ext3. I plan to use Scientific Linux 5.1 or 5.2 x86_64 as the operating system on the file server machine. Suggestions, comments and recommendations are very welcome. we're running a not-so-small AFS cell on SL5.2 x86_64 servers. All vice partitions are ext3, we've never had to regret this choice, and have no plans to use xfs instead. We also run a not-so-small amount of non-AFS storage - using xfs, because for that application it does have significant advantages over ext3. Whatever filesystem you choose, make sure you get the rest of your setup right, it will have much more impact on your fileserver performance. In particular: - Choose the right set of fileserver parameters. Start out from the "large" set, but increase the number of threads (>= # of clients if possible, max is 128-n where n~=7). - Use the right I/O scheduler. Depending on your hardware, either "deadline" or "noop" should work best. The default "cfq" probably won't. - Depending on access patterns, increasing readahead may help as well. Try values from 1024 to 16384. The latter two can even be changed on the fly, which makes it really simple to find the optimum while your clients are doing real world work. Notice that 1.4.7 still has a 2 TB size limit for the vice partitions, so you want to stay a little bit below unless you go for 1.4.8. Hope this helps. Let us know what you chose and how well it works and performs. Regards, Stephan -- Stephan Wiesand DESY - DV - Platanenallee 6 15738 Zeuthen, Germany
Re: AFS on XFS or ext3?
Michael Mansour, cut the CRAP/FUD out! I would NOT depend on ext3 if I CARED about what was stored on my disks. I ONLY use ext3 if the data stored is NOT of "very high importance". I use XFS when I DO CARE, so I use it all the time. XFS is the most reliable, dependable, and robust file system out there and independent tests have consistently shown it to be much faster than ext3. It has far more YEARS and Pentabytes of service under it's belt than ext3, a LOT more! I've had XFS do a much better job of surviving system crashes and disk failures than ext3.
Re: AFS on XFS or ext3?
Hi, > Hi Bob! > > On Tue, 17 Feb 2009 09:55:41 -0700 > Bob Barton wrote: > > > I am setting up a 2 TB file system to use for AFS volumes > > on an AFS file server and I am wondering which file > > system I should use - XFS or ext3. I plan to use > > Scientific Linux 5.1 or 5.2 x86_64 as the operating > > system on the file server machine. > > Suggestions, comments and recommendations are very > > welcome. > Assuming that you use LVM anyway, I would recommend xfs. > It is said to be slightly faster than ext3 (I have not ext3 can be just as fast by disabling it's "more redundant" features (atime, etc). XFS isn't as redundant as ext3 so is typically not recommended to be used to store data of "very high importance". > tested this by myself), it can take more directories, > which in the case of AFS is not important and - the main > point - you can modify the size of the filesystem > with # xfs_growfs while xfs is mounted! You can do exactly the same thing with ext3 using resize2fs, from the man page: The resize2fs program will resize ext2 or ext3 file systems. It can be used to enlarge or shrink an unmounted file system located on device. If the filesystem is mounted, it can be used to expand the size of the mounted filesystem, assuming the kernel supports on-line resizing. (As of this writing, the Linux 2.6 kernel supports on-line resize for filesystems mounted using ext3 only.). I've used it many times before and it works fine. Regards, Michael. > E.g. if you want to add disks to your RAID ... > I have very good experience extending LVM and xfs > whithout stopping the service. > Please keep in mind that you cannot extend an xfs > while it is 100.00% full. At least some blocks > must be free :-) > > Cheers > > Anton J. Gamel > > HPC und GRID-Computing > Physikalisches Institut > Abteilung Professor Herten > > c/o Rechenzentrum der Universität Freiburg > Arbeitsgruppe Dr. Winterer > Hermann-Herder-Straße 10 > 79104 Freiburg > > Tel.: ++49 (0)761 203 -4672 > > -- > Es bleibt immer ein Rest - und ein Rest vom Rest. --- End of Original Message ---
Re: AFS on XFS or ext3?
Hi Bob! On Tue, 17 Feb 2009 09:55:41 -0700 Bob Barton wrote: > I am setting up a 2 TB file system to use for AFS volumes > on an AFS file server and I am wondering which file > system I should use - XFS or ext3. I plan to use > Scientific Linux 5.1 or 5.2 x86_64 as the operating > system on the file server machine. > Suggestions, comments and recommendations are very > welcome. Assuming that you use LVM anyway, I would recommend xfs. It is said to be slightly faster than ext3 (I have not tested this by myself), it can take more directories, which in the case of AFS is not important and - the main point - you can modify the size of the filesystem with # xfs_growfs while xfs is mounted! E.g. if you want to add disks to your RAID ... I have very good experience extending LVM and xfs whithout stopping the service. Please keep in mind that you cannot extend an xfs while it is 100.00% full. At least some blocks must be free :-) Cheers Anton J. Gamel HPC und GRID-Computing Physikalisches Institut Abteilung Professor Herten c/o Rechenzentrum der Universität Freiburg Arbeitsgruppe Dr. Winterer Hermann-Herder-Straße 10 79104 Freiburg Tel.: ++49 (0)761 203 -4672 -- Es bleibt immer ein Rest - und ein Rest vom Rest.
Re: AFS on XFS or ext3?
Bob Barton wrote: Hi All I am setting up a 2 TB file system to use for AFS volumes on an AFS file server and I am wondering which file system I should use - XFS or ext3. I plan to use Scientific Linux 5.1 or 5.2 x86_64 as the operating system on the file server machine. Suggestions, comments and recommendations are very welcome. We have many 2Tb+ file systems on ext3 and they work great. V.stable. -- Please sign my petition: http://petitions.number10.gov.uk/alcohol-buying/ - Faye Gibbins, Computing Officer (Infrastructure Services) GeoS KB; Linux, Unix, Security and Networks. Beekeeper - The Apiary Project, KB - www.bees.ed.ac.uk - I grabbed at spannungsbogen before I knew I wanted it. (x(x_(X_x(O_o)x_x)_X)x) The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
Re: AFS on XFS or ext3?
* Bob Barton (bar...@ualberta.ca) [20090217 09:55]: Bob, > I am setting up a 2 TB file system to use for AFS volumes on an > AFS file server and I am wondering which file system I should > use - XFS or ext3. I plan to use Scientific Linux 5.1 or 5.2 > x86_64 as the operating system on the file server machine. We're running a few terabytes of AFS scratch space on XFS, no major issues so far. 1. Make sure that you optimize your XFS layout for small files (and underlying RAID layout if you use something else than mirrors). This is what we use for 500G partitions: mkfs.xfs -d agcount=8,unwritten=1 -i size=256,align=1 -n size=16384 -l version=2,size=128m 2. Make sure your /vicepX partitions have maxed out log mount options (inode64 for 64-bit systems): noatime,logbsize=256k,logbufs=8,inode64 3. If you can handle the operational burden of keeping track of external logs for your filesystems, I can only recommend it, makes AFS fly. 4. Finally, you should be prepared to hold the pieces together if it falls apart. XFS in SL is not exactly supported by any commercial entity nor the XFS developer community. HTH, Peter -- .+'''+. .+'''+. .+'''+. .+'''+. .+'' Kelemen Péter / \ / \ peter.kele...@cern.ch .+' `+...+' `+...+' `+...+' `+...+'
AFS on XFS or ext3?
Hi All I am setting up a 2 TB file system to use for AFS volumes on an AFS file server and I am wondering which file system I should use - XFS or ext3. I plan to use Scientific Linux 5.1 or 5.2 x86_64 as the operating system on the file server machine. Suggestions, comments and recommendations are very welcome. -- Bob Barton Local Area Administrator (780) 492-5160 7-095 ECERF Chemical & Materials Engineering University of Alberta, Edmonton Alberta, T6G 2V4
