Re: Large files 2GB+ RAID?
Marc Mutz [EMAIL PROTECTED] writes, in response to a request by Jason Titus about getting 2GB file support and raid-0.90: 2.4 will surely have an up-to-date raid implementation. Features are supposed to get into the "stable" branch like 2.4 by first being implemented in an "experimental" 2.3.x or 2.5.x branch. As far as I can tell, nobody has taken this step, so you should not be on raid 0.90 being available in 2.4. Adam J. Richter __ __ 4880 Stevens Creek Blvd, Suite 104 [EMAIL PROTECTED] \ / San Jose, California 95129-1034 +1 408 261-6630 | g g d r a s i l United States of America fax +1 408 261-6631 "Free Software For The Rest Of Us."
Re: Large files 2GB+ RAID?
I wrote: | [...] so you should not be on raid 0.90 being available in 2.4. That should read, "[...] so you should not _bet_ on raid 0.90 being available in 2.4." Sorry for the typo. Adam J. Richter __ __ 4880 Stevens Creek Blvd, Suite 104 [EMAIL PROTECTED] \ / San Jose, California 95129-1034 +1 408 261-6630 | g g d r a s i l United States of America fax +1 408 261-6631 "Free Software For The Rest Of Us."
Re: Large files 2GB+ RAID?
On Wed, Dec 29, 1999 at 04:55:08AM -0800, Adam J. Richter wrote: Marc Mutz [EMAIL PROTECTED] writes, in response to a request by Jason Titus about getting 2GB file support and raid-0.90: 2.4 will surely have an up-to-date raid implementation. Features are supposed to get into the "stable" branch like 2.4 by first being implemented in an "experimental" 2.3.x or 2.5.x branch. As far as I can tell, nobody has taken this step, so you should not be on raid 0.90 being available in 2.4. [ These are fuzzy recollections from around Sept 1999 ] The RAID 0.90 code interacts badly with the new page-cache system, and its problems are of same type (I recall) as RAMDISK has/had. I think there are/were also some lock recursion problems when going thru FS-BDEV-RAID-BDEV-DEVICE layers. (After all, RAID looks like a DEVICE for the generic BDEV layer which FSes use..) Ask Ingo for more info. Adam J. Richter __ __ 4880 Stevens Creek Blvd, Suite 104 [EMAIL PROTECTED] \ / San Jose, California 95129-1034 /Matti Aarnio
RE: Large files 2GB+ RAID?
I've been following this thread a little bit and would like to ask a question. I installed Redhat 6.1 and used the built in raid support. After attempting to upgrade the kernel to a 2.3.x series, I soon discovered what Redhat had done. Is there any hope of a patch to the 2.3.x / 2.4.x series of the kernel? The speed improvements (Raid1 vs. single partition) were great, but I need to run the 2.3.x series for various reasons. I had to reinstall without using raid. This question may be a repeat; sorry if it is. Thanks, Craig -Original Message- From: Adam J. Richter [SMTP:[EMAIL PROTECTED]] Sent: Wednesday, December 29, 1999 8:08 AM To: [EMAIL PROTECTED] Subject: Re: Large files 2GB+ RAID? I wrote: | [...] so you should not be on raid 0.90 being available in 2.4. That should read, "[...] so you should not _bet_ on raid 0.90 being available in 2.4." Sorry for the typo. Adam J. Richter __ __ 4880 Stevens Creek Blvd, Suite 104 [EMAIL PROTECTED] \ / San Jose, California 95129-1034 +1 408 261-6630 | g g d r a s i l United States of America fax +1 408 261-6631 "Free Software For The Rest Of Us."
Re: Large files 2GB+ RAID?
Seth Vidal wrote: snip My understanding is that the bigmem patches are FS patches not memory patches - they are inappropriately named perhaps. snip Bigmem is support for 1G _RAM_. The reson it is in 2.2 and large file support is not is that the latter breaks libc (and posix?) and the first one is a pure kernel issue. Now back to Jason's question: I meant that you have two options if you insist on having RAID and large file support in Linux _right now_: 1.) get 64bit hardware and use 2.2.13+raid0145 2.) get a hw-RAID controller and use 2.3.x, which supports 2G files, you said. If you don't want to go with any of that, you have the third option 3.) use another OS If you want to stay true to Linux, you can invest some months of waiting to gain access to options 4.) wait for Ingo et al. to make sw-RAID 0.90 stable on 2.3/2.4 5.) wait for 2G file support to be backported to 2.2, both of which is more a matter of believe than of fact, as others have pointed out. I personally think that raid-0.90 will make it into 2.4 and do not think that 2G files will ever become part of 2.2. Marc -- Marc Mutz [EMAIL PROTECTED]http://marc.mutz.com/Encryption-HOWTO/ University of Bielefeld, Dep. of Mathematics / Dep. of Physics PGP-keyID's: 0xd46ce9ab (RSA), 0x7ae55b9e (DSS/DH)
Large files 2GB+ RAID?
We've been working on getting a x86 Linux system together that would support both RAID and larger then 2 GB file sizes - so far with little luck. RAID works fine on 2.2 kernels, and 2GB files works on 2.3 kernels but RAID doesn't seem to like 2.3 (or at least the 2.3.34 w/ the included RAID). When we attempt to make the ext2 filesystem on the RAID 1 drive, it kernel panics. Seems like this is pretty key for Linux to be used in large data server environments. Has anyone else gotten RAID working with large files on Intel? Thanks for any help / information, Jason. [EMAIL PROTECTED]
Re: Large files 2GB+ RAID?
Unfortunately the hardware RAID still doesn't solve the 2GB+ problem. I also have a hard time with the 'if you want big files, buy a 64 bit machine' argument. What percentage of Linux users are on 64 bit platforms? How many other x86 OS's support 64 bit filesystems (NT, FreeBSD, BeOS, Solaris, etc)? This is a serious impediment to Linux being a server OS, and is most likely going to make us switch to FreeBSD for our projects - even though I much prefer Linux (and know it better). Sure would be nicer to stay with the penguin then turn to the devil Jason. [EMAIL PROTECTED] -- From: Marc Mutz [EMAIL PROTECTED] To: Stephen Waters [EMAIL PROTECTED] Cc: Jason Titus [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: Large files 2GB+ RAID? Date: Tue, Dec 28, 1999, 5:37 PM Stephen Waters wrote: snip 2.3.x are _not_ the kernels one wants to use in production environment. 2.4 will surely have an up-to-date raid implementation. if you need 2GB filesize support and RAID _now_, go buy a non-intel system (alpha, sparc64), or a hardware raid controller that is supported under 2.2.13. that's nice in theory, but a lot of commercial giant database people (oracle, sybase, etc.) only support Linux on x86. So what's wrong with buying a harware raid controller or waiting for 2.4? Marc -- Marc Mutz [EMAIL PROTECTED]http://marc.mutz.com/Encryption-HOWTO/ University of Bielefeld, Dep. of Mathematics / Dep. of Physics PGP-keyID's: 0xd46ce9ab (RSA), 0x7ae55b9e (DSS/DH)
Re: Large files 2GB+ RAID?
Unfortunately the hardware RAID still doesn't solve the 2GB+ problem. I also have a hard time with the 'if you want big files, buy a 64 bit machine' argument. What percentage of Linux users are on 64 bit platforms? How many other x86 OS's support 64 bit filesystems (NT, FreeBSD, BeOS, Solaris, etc)? This is a serious impediment to Linux being a server OS, and is most likely going to make us switch to FreeBSD for our projects - even though I much prefer Linux (and know it better). Sure would be nicer to stay with the penguin then turn to the devil frightenlingly bad puns should be left out of this mailinglist. :) I thought there were motions to backport the BIGMEM patches to 2.2 (unofficial patch of course) any clarification would be useful. -sv
Re: Large files 2GB+ RAID?
Ah, sorry for the puns and any confusion. I am talking about 2GB+ file sizes, not memory. The also proves my point - we now have 4GB memory on 32 bit systems - which is only applicable for a VERY small percentage of Linux users, but not 2GB files on 32 bit systems (once again - even though many other 32 bit OSes have them)... Jason My understanding is that the bigmem patches are FS patches not memory patches - they are inappropriately named perhaps. maybe I'm off. -sv
Re: Large files 2GB+ RAID?
Ah, sorry for the puns and any confusion. I am talking about 2GB+ file sizes, not memory. The also proves my point - we now have 4GB memory on 32 bit systems - which is only applicable for a VERY small percentage of Linux users, but not 2GB files on 32 bit systems (once again - even though many other 32 bit OSes have them)... Jason -- From: Seth Vidal [EMAIL PROTECTED] To: Jason Titus [EMAIL PROTECTED] Cc: Marc Mutz [EMAIL PROTECTED], Stephen Waters [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: Large files 2GB+ RAID? Date: Tue, Dec 28, 1999, 6:09 PM Unfortunately the hardware RAID still doesn't solve the 2GB+ problem. I also have a hard time with the 'if you want big files, buy a 64 bit machine' argument. What percentage of Linux users are on 64 bit platforms? How many other x86 OS's support 64 bit filesystems (NT, FreeBSD, BeOS, Solaris, etc)? This is a serious impediment to Linux being a server OS, and is most likely going to make us switch to FreeBSD for our projects - even though I much prefer Linux (and know it better). Sure would be nicer to stay with the penguin then turn to the devil frightenlingly bad puns should be left out of this mailinglist. :) I thought there were motions to backport the BIGMEM patches to 2.2 (unofficial patch of course) any clarification would be useful. -sv
Re: Large files 2GB+ RAID?
Nope. Bigmem was for 4 GB RAM and such, and has been pretty much replaced by highmem (all culled from the Linux Memory Management mailing list). All of the 2GB file stuff is refereed to mostly as Large File Summit (LFS) not to be confused with Log File System (LFS - no idea what it does. Some sort of journal type thing). Once again, any information about large files under RAID would be much appreciated. The pull of FreeBSD is almost inescapable. ahh but freebsd smp is in sore shape in comparison to linux smp. so there is that point. -sv
Re: Large files 2GB+ RAID?
On Tue, Dec 28, 1999 at 06:42:15PM -0500, Jason Titus wrote: by highmem (all culled from the Linux Memory Management mailing list). All of the 2GB file stuff is refereed to mostly as Large File Summit (LFS) not to be confused with Log File System (LFS - no idea what it does. Some sort of journal type thing). Pardon my (mis)understanding of things, but I just had to butt in here. Isn't the question you're asking 'What's the deal with large files under Linux'? I'm not exactly sure how all the RAID stuff works (which is why I'm subscribed to this list..), but AFAIK RAID is just a block device like any other, including a hard disk - so you don't need any specific 'Large files for RAID' patches. It's only the filesystem (and I guess the C library) which must support it. I'm quite sure you can access 2GB on block devices 8). *Maybe* it's possible that you don't have to run ext2 on Linux/RAID? If, for instance, FFS on Linux supported files 2GB (I have no idea if it does), I guess you could run that as the primary filesystem. You could port over the fsck etc. programs from BSD. Might be slower and possibly less reliable, but it might work. I know there are also Large File Summit patches around, but I haven't investigated them (I think the largest file I've got on my system is like 50meg :). I have no clue about this - maybe a post to linux-kernel would help. Otherwise, if you can guarantee that you'll only have a few large files, you can always partition your hard drive and get the apps to write directly to /dev/sda5 instead of a file on a filesystem. But that's probably not very feasible. Once again, any information about large files under RAID would be much appreciated. The pull of FreeBSD is almost inescapable. You're making FreeBSD sound like a bad thing ;). -- : Andre Pang [EMAIL PROTECTED] - Purruna Pty Ltd - ph# 0411.882299 : : #ozone - http://www.vjolnir.org/ozone/:
Re: Large files 2GB+ RAID?
[I left off CC'ing the universe on this one...] On Wed, 29 Dec 1999, Andre Pang wrote: On Tue, Dec 28, 1999 at 06:42:15PM -0500, Jason Titus wrote: by highmem (all culled from the Linux Memory Management mailing list). All of the 2GB file stuff is refereed to mostly as Large File Summit (LFS) not to be confused with Log File System (LFS - no idea what it does. Some sort of journal type thing). Logging file systems are a (hard simplification) kind of journaling. Basically, nobody is going to use a true logging filesystem these days - the databases themselves do rollback. The other advantage to logging is the part that is also a part of journaling, and journaling is thus more general purpose. [Pleading for mercy, I'm not a heavy DB guy] *Maybe* it's possible that you don't have to run ext2 on Linux/RAID? If, for instance, FFS on Linux supported files 2GB (I have no idea if it does), I guess you could run that as the primary filesystem. You could port over the fsck etc. programs from BSD. Might be slower and possibly less reliable, but it might work. Large files have to be supported in 3 places: kernel, libc, _and_ the FS. I know there are also Large File Summit patches around, but I haven't investigated them (I think the largest file I've got on my system is like 50meg :). I have no clue about this - maybe a post to linux-kernel would help. My take from the kernel list is that the new style raid stuff shipping on redhat and suse is rock solid in 2.2, but 2.3 is still shaky. There is not that I know of a firm commitment to have new raid in the 2.4 kernel. :( So, look for LFS back-ported to 2.2. Don't run production systems with 2.3 right now. Dark Evil awaits. If anybody will have this soon, it'll probably be Suse due to their LVM work. (Got enough TLA's yet?). Otherwise, if you can guarantee that you'll only have a few large files, you can always partition your hard drive and get the apps to write directly to /dev/sda5 instead of a file on a filesystem. But that's probably not very feasible. Flee in terror. Once again, any information about large files under RAID would be much appreciated. The pull of FreeBSD is almost inescapable. You're making FreeBSD sound like a bad thing ;). Which they are not - and I'm a linux person. :) -- Hunter Matthews Unix / Network Administrator Office: BioSci 222/244 Bioscience Never take candy from strangers. Especially on the Internet.
