Re: Filesystem Tests
Mike Fedyk [EMAIL PROTECTED] wrote: On Wed, Aug 06, 2003 at 06:34:10PM +0200, Diego Calleja Garc?a wrote: El Wed, 06 Aug 2003 18:06:37 +0400 Hans Reiser [EMAIL PROTECTED] escribi?: I don't think ext2 is a serious option for servers of the sort that Linux specializes in, which is probably why he didn't measure it. Why? Because if you have a power outage, or a crash, you have to run the filesystem check tools on it or risk damaging it further. Journaled filesystems have a much smaller chance of having problems after a crash. Journalled filesytems have a runtime cost, and you're paying that all the time. If you're going 200 days between crashes on a disk-intensive box then using a journalling fs to save 30 minutes at reboot time just doesn't stack up: you've lost much, much more time than that across the 200 days. It all depends on what the machine is doing and what your max downtime requirements are.
Re: Filesystem Tests
I've never wrote I made my guesses from the CPU percentage alone, you explained correctly why. I encourage you too to calculate yourself how much more CPU time reiser4 needs. Ok, fair enough :) -- Jamie
Re: Filesystem Tests
Hans Reiser wrote: reiser4 cpu consumption is still dropping rapidly as others and I find kruft in the code and remove it. Major kruft remains still. If a file system is getting greater throughput, that means the relevant code is being run more, which means more CPU will be used for the purpose of setting up DMA, etc. That is, if a FS gets twice the throughput, it would not be unreasonable to expect it to use 2x the CPU time. Furthermore, in order to achieve greater throughput, one has to write more intelligent code. More intelligent code is probably going to require more computation time. That is to say, if your FS is twice as fast, saying it has a problem purely on the basis that it's using more CPU ignores certain facts and basic logic. Now, if you can manage to make it twice as fast while NOT increasing the CPU usage, well, then that's brilliant, but the fact that ReiserFS uses more CPU doesn't bother me in the least.
Re: Filesystem Tests
On Wed, Aug 06, 2003 at 08:45:14PM +0200, Diego Calleja Garc?a wrote: El Wed, 6 Aug 2003 11:04:27 -0700 Mike Fedyk [EMAIL PROTECTED] escribi?: Journaled filesystems have a much smaller chance of having problems after a crash. I've had (several) filesystem corruption in a desktop system with (several) journaled filesystems on several disks. (They seem pretty stable these days, though) However I've not had any fs corrution in ext2; ext2 it's (from my experience) rock stable. Personally I'd consider twice the really serious option for a serious server. I've had corruption caused by hardware, and nothing else. I haven't run into any serious bugs. But with servers, the larger your filesystem, the longer it will take to fsck. And that is bad for uptime. Period. I would be running ext2 also if I wasn't running so many test kernels (and they do oops on you), and I've been glad that I didn't have to fsck every time I oopsed (though I do every once in a while, just to make sure).
Re: Filesystem Tests
On Sat, 9 Aug 2003, Jamie Lokier wrote: reiser4 is using approximately twice the CPU percentage, but completes in approximately half the time, therefore it uses about the same amount of CPU time at the others. Therefore on a loaded system, with a load carefully chosen to make the test CPU bound rather than I/O bound, one could expect reiser4 to complete in approximately the same time as the others, _not_ slowest. Depends how you define approximation, margins. I dropped them and calculated reiser4 needs the most CPU time. Hans wrote it's worked on. However guessing performance on a whatever carefully chosen loaded system from results on an unloaded system is exactly that, guess, not fact. That's why it's misleading to draw conclusions from the CPU percentage alone. I've never wrote I made my guesses from the CPU percentage alone, you explained correctly why. I encourage you too to calculate yourself how much more CPU time reiser4 needs. Szaka
Re: Filesystem Tests
On Tue, 5 Aug 2003, Andrew Morton wrote: Solutions to this inaccuracy are to make the test so long-running (ten minutes or more) that the difference is minor, or to include the `sync' in the time measurement. And/or reduce RAM at kernel boot, etc. Anyway, I also asked for 'sync' yesterday and Grant included some but not after every each tests. I run the results through some scripts to make it more readable. It indeed has some interesting things ... reiser4 reiserfs ext3XFSJFS copy 33.39,34% 39.55,32% 39.42,25% 43.50,32% 48.15,20% sync 1.54, 0% 3.15, 1% 9.05, 0% 2.08, 1% 3.05, 1% recopy1 31.09,34% 75.15,13% 79.96, 9% 102.37,12% 108.39, 5% recopy2 33.15,33% 77.62,13% 98.84, 7% 108.00,12% 114.96, 5% sync 2.89, 3% 3.84, 1% 8.15, 0% 2.40, 2% 3.86, 0% du2.05,42% 2.46,21% 3.31,11% 3.73,32% 2.42,17% delete7.41,52% 5.22,58% 3.71,39% 8.75,56% 15.33, 7% tar 52.25,25% 90.83,12% 74.93,13% 157.61, 7% 135.86, 6% sync 6.77, 2% 4.19, 3% 1.67, 1% 0.95, 1% 38.18, 0% overall 171.28,30% 302.53,16% 319.71,11% 429.79,13% 470.88, 6% BTW, zsh has a built-in 'time' so measuring a full operation can be easily done as 'sync; time ( my_test; sync )' Szaka
Re: Filesystem Tests
On Wed, Aug 06, 2003 at 07:37:42PM -0400, Timothy Miller wrote: Hans Reiser wrote: reiser4 cpu consumption is still dropping rapidly as others and I find kruft in the code and remove it. Major kruft remains still. Now, if you can manage to make it twice as fast while NOT increasing the CPU usage, well, then that's brilliant, but the fact that ReiserFS uses more CPU doesn't bother me in the least. Basically he's saying it's faster and still not at its peak effeciency yet too.
Re: Filesystem Tests
El Wed, 06 Aug 2003 18:06:37 +0400 Hans Reiser [EMAIL PROTECTED] escribió: I don't think ext2 is a serious option for servers of the sort that Linux specializes in, which is probably why he didn't measure it. Why? reiser4 cpu consumption is still dropping rapidly as others and I find kruft in the code and remove it. Major kruft remains still. Cool.
Re: Filesystem Tests
On Wed, Aug 06, 2003 at 06:34:10PM +0200, Diego Calleja Garc?a wrote: El Wed, 06 Aug 2003 18:06:37 +0400 Hans Reiser [EMAIL PROTECTED] escribi?: I don't think ext2 is a serious option for servers of the sort that Linux specializes in, which is probably why he didn't measure it. Why? Because if you have a power outage, or a crash, you have to run the filesystem check tools on it or risk damaging it further. Journaled filesystems have a much smaller chance of having problems after a crash.
Re: Filesystem Tests
On Wed, 2003-08-06 at 04:30, Grant Miner wrote: I tested the performace of various filesystems with a mozilla build tree of 295MB, with primarily writing and copying operations. The test system is Linux 2.6.0-test2, 512MB memory, 11531.85MB partition for tests. Sync is run a few times throughout the test (for full script see bottom of this email). I ran mkfs on the partition before every test. Running the tests again tends to produces similar times, about +/- 3 seconds. The first item number is time, in seconds, to complete the test (lower is better). The second number is CPU use percentage (lower is better). reiser4 171.28s, 30%CPU (1.x time; 1.0x CPU) reiserfs 302.53s, 16%CPU (1.7663x time; 0.53x CPU) ext3 319.71s, 11%CPU (1.8666x time; 0.36x CPU) xfs 429.79s, 13%CPU (2.5093x time; 0.43x CPU) jfs 470.88s, 6%CPU (2.7492x time 0.02x CPU) What about ext2? :-) I think it could be interesting to compare the overhead of a journaled fs against a non-one, simply for reference.
Re: Filesystem Tests
El Wed, 6 Aug 2003 12:08:50 -0700 Mike Fedyk [EMAIL PROTECTED] escribió: But with servers, the larger your filesystem, the longer it will take to fsck. And that is bad for uptime. Period. Sure. But Han's don't benchmark ext2 because it's not an option isn't a valid stament, at least to me. I'm not saying ext2 is the best fs on earth, but i *really* think it's a real option, and as such, it must be benchmarked.
Re: Filesystem Tests
Grant Miner [EMAIL PROTECTED] wrote: I tested the performace of various filesystems with a mozilla build tree of 295MB, with primarily writing and copying operations. The test system is Linux 2.6.0-test2, 512MB memory, 11531.85MB partition for tests. Sync is run a few times throughout the test (for full script see bottom of this email). I ran mkfs on the partition before every test. Running the tests again tends to produces similar times, about +/- 3 seconds. The first item number is time, in seconds, to complete the test (lower is better). The second number is CPU use percentage (lower is better). reiser4 171.28s, 30%CPU (1.x time; 1.0x CPU) reiserfs 302.53s, 16%CPU (1.7663x time; 0.53x CPU) ext3 319.71s, 11%CPU (1.8666x time; 0.36x CPU) xfs 429.79s, 13%CPU (2.5093x time; 0.43x CPU) jfs 470.88s, 6%CPU (2.7492x time 0.02x CPU) But different filesystems will leave different amounts of dirty, unwritten data in memory at the end of the test. On your machine, up to 200MB of dirty data could be sitting there in memory at the end of the timing interval. You need to decide how to account for that unwritten data in the measurement. Simply ignoring it as you have done is certainly valid, but is only realistic in a couple of scenarios: a) the files are about the be deleted again b) the application which your benchmark simulates is about to spend more than 30 seconds not touching the disk. This discrepancy is especially significant with ext3 which, in ordered data mode, will commit all that data every five seconds. If the test takes more than five seconds then ext3 can appear to take a _lot_ longer. But it is somewhat artificial: that data has to be written out sometime. Solutions to this inaccuracy are to make the test so long-running (ten minutes or more) that the difference is minor, or to include the `sync' in the time measurement. And when benching things, please include ext2. It is the reference filesystem, as it were. It tends to be the fastest, too.
