What Filesystem?
Dear Sirs, I am a visual artist and musician. As well as a range of traditional media I use Linux Mandrake 8.0 on an old Athlon-based PC. I wrote to Mr. Hans Reiser, after following a mailto link on the Namesys website, with a slightly lengthier version of the question written below, and was advised that This is really a better question for our mailing list ([EMAIL PROTECTED]) than for me, as I am obviously biased. I am therefore sending this email to you in the hope that somebody may be able to solve my puzzle - or at least direct me towards another knowledgeable source of information! Thank you. The question: ... I have done much research in the field of computer /hardware/ suitable for commercial Digital Content Creation (P4 Xeon; Wildcat graphics; ART's PURE raytracing PCI render board, etc. ...) and now look to a better understanding of the choices I might make for a (presumably) Linux-based OS running on (presumably) Intel 32-bit hardware SUCH AS these two elements:- 1. Kernel Patches - pre-empt and low latency; 2. File System type - EXT3, ReiserFS, SGi's XFS, HFS, JFS; Would you be able to advsie me on any issues I might need to be aware of and perhaps any firm decisions you think would be good for me to make, regarding OS choice and configuration? I realise that all pre-built workstations supplied by Dell, IBM and HP-Compaq come with RedHat, but, just for the record, I like Mandrake and I am getting into the WindowMaker desktop :). I also develop my website locally using Apache (but hosted on Freeserve :( ), and I understand pre-empt /or low-latency patches are counter-effective for servers. However, as long as no real harm can be expected, my priority is for graphics and sound creation, editing, compositing, publishing, etc., while fast response for just developing html pages isn't. I would like to believe that, somewhere, I can get a 'standard' Linux patched for optimal DCC, with suitable FS type available to choose from during installation. If not, I will have to find and apply patches, but I know that with FS type any change demands a reformat, which can really disrupt a working week! [End question] Yours faithfully, James Thompson, Visual Artist Musician H.E. Student of Fine Arts
Re: What Filesystem?
On Wed, Jan 29, 2003 at 03:20:26PM -0500, James Thompson wrote: I am a visual artist and musician. Check out the document at http://myweb.cableone.net/eviltwin69/ALSA_JACK_ARDOUR.html. There a section that benchmarks various filesystems for their latency. The short story is that Reiserfs wins handily over Ext2, Ext3, and FAT32 (duh! why would someone test that...). Unfortunately, there's no comparison between Reiser/JFS/XFS. I think that would be more of a fair match. Anyhow though, for general low-latency multimedia work, ReiserFS looks like it's a good choice. -- Ross Vandegrift [EMAIL PROTECTED] A Pope has a Water Cannon. It is a Water Cannon. He fires Holy-Water from it.It is a Holy-Water Cannon. He Blesses it. It is a Holy Holy-Water Cannon. He Blesses the Hell out of it. It is a Wholly Holy Holy-Water Cannon. He has it pierced.It is a Holey Wholly Holy Holy-Water Cannon. He makes it official. It is a Canon Holey Wholly Holy Holy-Water Cannon. Batman and Robin arrive. He shoots them.
Re: What Filesystem?
On Wed, Jan 29, 2003 at 03:20:26PM -0500, James Thompson wrote: I am a visual artist and musician. Check out the document at http://myweb.cableone.net/eviltwin69/ALSA_JACK_ARDOUR.html. There a section that benchmarks various filesystems for their latency. The short story is that Reiserfs wins handily over Ext2, Ext3, and FAT32 (duh! why would someone test that...). Simply because FAT filesystem is very simple and has little overhead. If dealing with little amount of files and folders then it can be rather quick, especially if you need CPU resources to other things. Unfortunately, there's no comparison between Reiser/JFS/XFS. I think that would be more of a fair match. Anyhow though, for general low-latency multimedia work, ReiserFS looks like it's a good choice.
Re: What Filesystem?
Reiserfs is probably not what you want for doing lots of high volume data. Reiserfs is good with many small files and general purpose use. It is actually the slowest filesystem for bulk data. I'm sure some smartass will probably post some benchmark to prove me wrong, but SGI have a long heritage of making filesystems for exactly what you want and so if XFS works I'd say use that. It's pumpin'. Of course more quantitative comparisons can easily be found on the linux-kernel mailing list. However, the practical difference between the high performance filesystems you mentioned, or the difference between running pre-empt or not should be considered marginal compared to other factors - such as, the load that your hard disk controller places on the system and the number of physical disks you have (and how many RPM they run at). Keep in mind that you can often nearly double the data throughput of a system by doubling the number of physical disks in it, and using RAID. Even with the latest UDMA-133, I haven't seen any IDE system actually perform without bothering the CPU non-trivially (of course YMMV). Using SCSI disks and controllers will give you a smoother system ride; which is why 95% of high-end workstations come equipped with SCSI. You can get 15,000 RPM U320 SCSI disks, which are f*** fast (though loud). This is largely because the SCSI protocol was designed properly, IDE/ATA is a hack. Serial-ATA promises to offer SCSI like host efficiency, but I'll only believe it when I see it. And at the moment the costs are as bad as SCSI anyway. Of course, being able to move around large chunks of data quickly extends to other parts of the system, too. The bigger and faster the system BUS, the better. Having `researched hardware platforms' you should know this, of course. As far as Athlon Motherboards go, Tyan are a reputable vendor who used to produce Sun clone motherboards - they have a really nice dual capable system with dual U160 SCSI controllers and 66MHz PCI slots - which must mean that the SCSI controllers run at that speed. It's definitely worth the ~$500 price tag. They all pale in comparison to R1+ based SGI or Sparc 9+ platforms of course. From a graphic artist's perspective, you're probably better off buying a new G4 based system and running MacOS X, you know :-). Linux isn't exactly `The Platform' for digital content creation. The Mac's CPU and hardware platform are a lot better at moving data around, and if you've got a Mac then you can run Mac OS X, or Linux + Mac OS inside an emulator (which runs FAST!). As a final note, keep in mind that a filesystem reformat does not mean a re-install; keep your partitions as small as practical and you can change them over individually. Sam. On Thu, 30 Jan 2003 09:20, James Thompson wrote: Dear Sirs, I am a visual artist and musician. As well as a range of traditional media I use Linux Mandrake 8.0 on an old Athlon-based PC. I wrote to Mr. Hans Reiser, after following a mailto link on the Namesys website, with a slightly lengthier version of the question written below, and was advised that This is really a better question for our mailing list ([EMAIL PROTECTED]) than for me, as I am obviously biased. I am therefore sending this email to you in the hope that somebody may be able to solve my puzzle - or at least direct me towards another knowledgeable source of information! Thank you. The question: ... I have done much research in the field of computer /hardware/ suitable for commercial Digital Content Creation (P4 Xeon; Wildcat graphics; ART's PURE raytracing PCI render board, etc. ...) and now look to a better understanding of the choices I might make for a (presumably) Linux-based OS running on (presumably) Intel 32-bit hardware SUCH AS these two elements:- 1. Kernel Patches - pre-empt and low latency; 2. File System type - EXT3, ReiserFS, SGi's XFS, HFS, JFS; Would you be able to advsie me on any issues I might need to be aware of and perhaps any firm decisions you think would be good for me to make, regarding OS choice and configuration? I realise that all pre-built workstations supplied by Dell, IBM and HP-Compaq come with RedHat, but, just for the record, I like Mandrake and I am getting into the WindowMaker desktop :). I also develop my website locally using Apache (but hosted on Freeserve :( ), and I understand pre-empt /or low-latency patches are counter-effective for servers. However, as long as no real harm can be expected, my priority is for graphics and sound creation, editing, compositing, publishing, etc., while fast response for just developing html pages isn't. I would like to believe that, somewhere, I can get a 'standard' Linux patched for optimal DCC, with suitable FS type available to choose from during installation. If not, I will have to find and apply patches, but I know that with FS type any change demands a
Re: What Filesystem?
On Wed, 29 Jan 2003 15:20:26 EST, James Thompson [EMAIL PROTECTED] said: ... I have done much research in the field of computer /hardware/ suitable for commercial Digital Content Creation (P4 Xeon; Wildcat graphics; ART's PURE raytracing PCI render board, etc. ...) and now Hmm.. so we're looking at high-end rendering, which is usually a CPU hog. 1. Kernel Patches - pre-empt and low latency; so I'm not clear on why you're worried about low latency? Remember that it *does* come with an overhead - the low-latency stuff is good if you're more concerned about fast response than total system load (for instance, on my laptop I'm willing to give up 5% of the CPU if it makes the X server run perceivably faster. If I was doing a lot of rendering, I'd want that 5% for user cycles. Could you be more specific regarding what sort of content you are making? (i.e. single frame images suitable for monitor display (1600x1200 and smaller), or large-format for high-resolution printing (posters, etc), or video, etc..) The resources needed to produce a 10-minute video clip are different from the things you'll need to produce a 4 foot x 5 foot poster at 600DPI. -- Valdis Kletnieks Computer Systems Senior Engineer Virginia Tech msg07439/pgp0.pgp Description: PGP signature
Re: What Filesystem?
Am Mittwoch, 29. Januar 2003 16:28 schrieb Ross Vandegrift: On Wed, Jan 29, 2003 at 03:20:26PM -0500, James Thompson wrote: I am a visual artist and musician. Check out the document at http://myweb.cableone.net/eviltwin69/ALSA_JACK_ARDOUR.html. There a section that benchmarks various filesystems for their latency. The short story is that Reiserfs wins handily over Ext2, Ext3, and FAT32 (duh! why would someone test that...). Unfortunately, there's no comparison between Reiser/JFS/XFS. I think that would be more of a fair match. XFS could win _today_ for real time (granted video bandwidth, for which it was designed in the first place), but this could change (compensate) with the latest ReiserFS 3.x patches (data logging) and finally Reiser4 (see the Reiser Homepage). Anyhow though, for general low-latency multimedia work, ReiserFS looks like it's a good choice. Yes. Go with low-latency _and_ preemption patches (try Gentoo, it has it all). I did beta testing for Robert Love (MontaVista) for ages and it is _the way to go_ for multi media machines. Greetings, Dieter -- Dieter Nützel Graduate Student, Computer Science University of Hamburg Department of Computer Science @home: Dieter.Nuetzel at hamburg.de (replace at with @)
Re: What Filesystem?
Am Mittwoch, 29. Januar 2003 19:16 schrieb [EMAIL PROTECTED]: On Wed, 29 Jan 2003 15:20:26 EST, James Thompson [EMAIL PROTECTED] said: ... I have done much research in the field of computer /hardware/ suitable for commercial Digital Content Creation (P4 Xeon; Wildcat graphics; ART's PURE raytracing PCI render board, etc. ...) and now Hmm.. so we're looking at high-end rendering, which is usually a CPU hog. 1. Kernel Patches - pre-empt and low latency; so I'm not clear on why you're worried about low latency? Remember that it *does* come with an overhead What? --- Sorry, quantify it. I can't. All low latency and pre-emption tests have showed _improved_ throughput (Yes) and much better multi media (video/audio) experience ;-) No measurable overhead an single and SMP systems (both Athlon). That's why it is in 2.5/2.6. - the low-latency stuff is good if you're more concerned about fast response than total system load (for instance, on my laptop I'm willing to give up 5% of the CPU if it makes the X server run perceivably faster. If I was doing a lot of rendering, I'd want that 5% for user cycles. But you want to have the much better task switching behavior together with the brand new O(1) scheduler. Could you be more specific regarding what sort of content you are making? (i.e. single frame images suitable for monitor display (1600x1200 and smaller), or large-format for high-resolution printing (posters, etc), or video, etc..) The resources needed to produce a 10-minute video clip are different from the things you'll need to produce a 4 foot x 5 foot poster at 600DPI. You mean single vs 2-/4-/8-/etc. (NUMA) SMP systems or even clusters? ;-) Greetings, Dieter -- Dieter Nützel Graduate Student, Computer Science University of Hamburg Department of Computer Science @home: Dieter.Nuetzel at hamburg.de (replace at with @)
Re: What Filesystem?
On Wed, 29 Jan 2003 20:43:38 +0100, Dieter =?iso-8859-1?q?N=FCtzel?= said: All low latency and pre-emption tests have showed _improved_ throughput (Yes) and much better multi media (video/audio) experience ;-) No measurable overhead an single and SMP systems (both Athlon). That's why it is in 2.5/2.6. Yes, but my *point* was that if the box is busy doing a 5-hour rendering job, the low-latency *wont make a difference*. If you're doing audio work or video captures, yes the low-latency/preempt stuff will help. If you're in a single thread that's CPU-bound, there's no need to go installing 2.5 to get it. But you want to have the much better task switching behavior together with the brand new O(1) scheduler. You want that *if* the actual load will benefit from it. See my comment above about rendering... You mean single vs 2-/4-/8-/etc. (NUMA) SMP systems or even clusters? ;-) Some things are doable with a single CPU, some things need the NUMA stuff, other things can use a Beowulf. Thus the need to quantify what the actual requirements are. Video editing will bottleneck on different things than photorealistic rendering of graphics. -- Valdis Kletnieks Computer Systems Senior Engineer Virginia Tech msg07446/pgp0.pgp Description: PGP signature
