Re: [Jfs-discussion] benchmark results

2010-01-15 Thread Edward Shishkin
When things didn't match up that was a clue that either - the benchmark was broken - the code was broken [...] I would carry out an object-oriented dualism here. [1] methods (kernel module) [2] objects (formatted partition) || |

Re: [Jfs-discussion] benchmark results

2010-01-10 Thread Larry McVoy
On Sun, Jan 10, 2010 at 08:03:04PM -0500, Casey Allen Shobe wrote: > On Dec 25, 2009, at 11:22 AM, Larry McVoy wrote: >> Dudes, sync() doesn't flush the fs cache, you have to unmount for >> that. >> Once upon a time Linux had an ioctl() to flush the fs buffers, I used >> it in lmbench. > > > You

Re: [Jfs-discussion] benchmark results

2010-01-10 Thread Casey Allen Shobe
On Dec 25, 2009, at 11:22 AM, Larry McVoy wrote: Dudes, sync() doesn't flush the fs cache, you have to unmount for that. Once upon a time Linux had an ioctl() to flush the fs buffers, I used it in lmbench. You do not need to unmount - 2.6.16+ have a mechanism in /proc to flush caches. See

Re: [Jfs-discussion] benchmark results

2010-01-05 Thread Steven Pratt
Dave Chinner wrote: On Mon, Jan 04, 2010 at 11:27:48AM -0500, Chris Mason wrote: On Fri, Dec 25, 2009 at 11:11:46AM -0500, ty...@mit.edu wrote: On Fri, Dec 25, 2009 at 02:46:31AM +0300, Evgeniy Polyakov wrote: [1] http://samba.org/ftp/tridge/dbench/README Was not abl

Re: [Jfs-discussion] benchmark results

2010-01-04 Thread Dave Chinner
On Mon, Jan 04, 2010 at 11:27:48AM -0500, Chris Mason wrote: > On Fri, Dec 25, 2009 at 11:11:46AM -0500, ty...@mit.edu wrote: > > On Fri, Dec 25, 2009 at 02:46:31AM +0300, Evgeniy Polyakov wrote: > > > > [1] http://samba.org/ftp/tridge/dbench/README > > > > > > Was not able to resist to write a sm

Re: [Jfs-discussion] benchmark results

2010-01-04 Thread Michael Rubin
Google is currently in the middle of upgrading from ext2 to a more up to date file system. We ended up choosing ext4. This thread touches upon many of the issues we wrestled with, so I thought it would be interesting to share. We should be sending out more details soon. The driving performance rea

Re: [Jfs-discussion] benchmark results

2010-01-04 Thread Chris Mason
On Fri, Dec 25, 2009 at 11:11:46AM -0500, ty...@mit.edu wrote: > On Fri, Dec 25, 2009 at 02:46:31AM +0300, Evgeniy Polyakov wrote: > > > [1] http://samba.org/ftp/tridge/dbench/README > > > > Was not able to resist to write a small notice, what no matter what, but > > whatever benchmark is running,

Re: [Jfs-discussion] benchmark results

2009-12-28 Thread Larry McVoy
> The bottom line is that it's very hard to do good comparisons that are > useful in the general case. It has always amazed me watching people go about benchmarking. I should have a blog called "you're doing it wrong" or something. Personally, I use benchmarks to validate what I already believe

Re: [Jfs-discussion] benchmark results

2009-12-27 Thread Christian Kujau
On Sun, 27 Dec 2009 at 17:33, ty...@mit.edu wrote: > Yes, but given many of the file systems have almost *exactly* the same "Almost" indeed - but curiously enough some filesystem are *not* the same, although they should. Again: we have 8GB RAM, I'm copying ~3GB of data, so why _are_ there differ

Re: [Jfs-discussion] benchmark results

2009-12-27 Thread tytso
On Sun, Dec 27, 2009 at 01:55:26PM -0800, Christian Kujau wrote: > On Sun, 27 Dec 2009 at 14:50, jim owens wrote: > > And I don't even care about comparing 2 filesystems, I only care about > > timing 2 versions of code in the single filesystem I am working on, > > and forgetting about hardware cach

Re: [Jfs-discussion] benchmark results

2009-12-27 Thread Christian Kujau
On Sun, 27 Dec 2009 at 14:50, jim owens wrote: > And I don't even care about comparing 2 filesystems, I only care about > timing 2 versions of code in the single filesystem I am working on, > and forgetting about hardware cache effects has screwed me there. Not me, I'm comparing filesystems - an

Re: [Jfs-discussion] benchmark results

2009-12-27 Thread jim owens
Christian Kujau wrote: > On 26.12.09 08:00, jim owens wrote: >>> I was using "sync" to make sure that the data "should" be on the disks >> Good, but not good enough for many tests... info sync > [...] >>On Linux, sync is only guaranteed to schedule the dirty blocks >> for >>w

Re: [Jfs-discussion] benchmark results

2009-12-26 Thread tytso
On Sat, Dec 26, 2009 at 11:00:59AM -0500, jim owens wrote: > Christian Kujau wrote: > > > I was using "sync" to make sure that the data "should" be on the disks > > Good, but not good enough for many tests... info sync > > CONFORMING TO >POSIX.2 > > NOTES >On Linux, sync is on

Re: [Jfs-discussion] benchmark results

2009-12-26 Thread Christian Kujau
On 26.12.09 08:00, jim owens wrote: >> I was using "sync" to make sure that the data "should" be on the disks > > Good, but not good enough for many tests... info sync [...] >On Linux, sync is only guaranteed to schedule the dirty blocks for >writing; it can actually take

Re: [Jfs-discussion] benchmark results

2009-12-26 Thread jim owens
Christian Kujau wrote: > I was using "sync" to make sure that the data "should" be on the disks Good, but not good enough for many tests... info sync CONFORMING TO POSIX.2 NOTES On Linux, sync is only guaranteed to schedule the dirty blocks for writing; it can act

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread Christian Kujau
On Fri, 25 Dec 2009 at 10:56, Christian Kujau wrote: > Thanks for the hint, I could find sys/vm/drop-caches documented in --^ not, was what I meant to say, but it's all there, as "drop_caches" in Documentation/sysctl/vm.txt Christian. > Documentation/ but it's good to

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread Christian Kujau
On Fri, 25 Dec 2009 at 11:33, ty...@mit.edu wrote: > caches, though; if you are going to measure read as well as writes, > then you'll probably want to do something like "echo 3 > > /proc/sys/vm/drop-caches". Thanks for the hint, I could find sys/vm/drop-caches documented in Documentation/ but it

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread Christian Kujau
On Fri, 25 Dec 2009 at 08:22, Larry McVoy wrote: > Dudes, sync() doesn't flush the fs cache, you have to unmount for that. Thanks Larry, that was exactly my point[0] too, I should add that to the results page to avoid further confusion or misassumptions: > Well, I do "sync" after each operati

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread Christian Kujau
On Fri, 25 Dec 2009 at 11:14, ty...@mit.edu wrote: > Did you include the "sync" in part of what you timed? In my "generic" tests[0] I do "sync" after each of the cp/tar/rm operations. > Peter was quite > right --- the fact that the measured bandwidth in your "cp" test is > five times faster than

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread Larry McVoy
On Fri, Dec 25, 2009 at 11:14:53AM -0500, ty...@mit.edu wrote: > On Thu, Dec 24, 2009 at 05:52:34PM -0800, Christian Kujau wrote: > > > > Well, I do "sync" after each operation, so the data should be on disk, but > > that doesn't mean it'll clear the filesystem buffers - but this doesn't > > hap

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread tytso
On Fri, Dec 25, 2009 at 08:22:38AM -0800, Larry McVoy wrote: > > Dudes, sync() doesn't flush the fs cache, you have to unmount for that. > Once upon a time Linux had an ioctl() to flush the fs buffers, I used > it in lmbench. > > ioctl(fd, BLKFLSBUF, 0); > > No idea if that is still supp

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread tytso
On Thu, Dec 24, 2009 at 05:52:34PM -0800, Christian Kujau wrote: > > Well, I do "sync" after each operation, so the data should be on disk, but > that doesn't mean it'll clear the filesystem buffers - but this doesn't > happen that often in the real world too. Also, all filesystem were tested >

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread tytso
On Fri, Dec 25, 2009 at 02:46:31AM +0300, Evgeniy Polyakov wrote: > > [1] http://samba.org/ftp/tridge/dbench/README > > Was not able to resist to write a small notice, what no matter what, but > whatever benchmark is running, it _does_ show system behaviour in one > or another condition. And when

Re: [Jfs-discussion] benchmark results

2009-12-25 Thread lakshmi pathi
I'm a file system testing newbie, I have a question/doubt,please let me know if i'm wrong. Do you think a tool, which uses output from "hdparm" command,to get hard drives maximum performance and compares it specific file system (say for example,"ext4 provides xx throughput against max. device th

Re: [Jfs-discussion] benchmark results

2009-12-24 Thread Christian Kujau
On Thu, 24 Dec 2009 at 16:27, ty...@mit.edu wrote: > If you don't do a "sync" after the tar, then in most cases you will be > measuring the memory bandwidth, because data won't have been written Well, I do "sync" after each operation, so the data should be on disk, but that doesn't mean it'll cle

Re: [Jfs-discussion] benchmark results

2009-12-24 Thread Evgeniy Polyakov
Hi Ted. On Thu, Dec 24, 2009 at 04:27:56PM -0500, ty...@mit.edu (ty...@mit.edu) wrote: > > Unfortunately there seems to be an overproduction of rather > > meaningless file system "benchmarks"... > > One of the problems is that very few people are interested in writing > or maintaining file system

Re: [Jfs-discussion] benchmark results

2009-12-24 Thread tytso
On Thu, Dec 24, 2009 at 01:05:39PM +, Peter Grandi wrote: > > I've had the chance to use a testsystem here and couldn't > > resist > > Unfortunately there seems to be an overproduction of rather > meaningless file system "benchmarks"... One of the problems is that very few people are interest

Re: [Jfs-discussion] benchmark results

2009-12-24 Thread Peter Grandi
> I've had the chance to use a testsystem here and couldn't > resist Unfortunately there seems to be an overproduction of rather meaningless file system "benchmarks"... > running a few benchmark programs on them: bonnie++, tiobench, > dbench and a few generic ones (cp/rm/tar/etc...) on ext{234},