FileBench Users,
I have modified FileBench so that it now features a couple of "run to
completion" modes of operation. This is useful for benchmarks that do a
specified amount of work and then quit rather than running for a
specified time. Currently with the finishonbytes and finishoncount
flowops a workload will cleanly shut down when the finish condition is
met, or when the time expires if that happens first. But the workloads:
bringover.f copyfiles.f createfiles.f and deletefiles.f will stop when
they have processed all the files in their filesets. If the number of
entries in the filesets is too small, or the runtime is too long, they
will generate errors when the usable files are exhausted, which is
confusing.
With the FileBench code covered by this webrev you can specify:
set mode quit timeout
to quit when the timer expires just as now (the default setting).
or:
set mode quit firstdone
to quit when the first thread runs out of files to use.
or:
set mode quit alldone
to quit when the last thread runs out of files to use.
These modes can be set in the workload file, or entered interactively,
however the intention is to have them set in the workload files of those
workloads for which firstdone or alldone behavior is appropriate. When
these modes are in effect, FileBench will gracefully exit just as though
it had run out of runtime. If timeout mode is in effect, it will still
report running out of files as an error, but do so less cryptically than
it does now.
The webrev can be found at:
http://cr.opensolaris.org/~dreww/run2completion/
Drew
P.S. three examples:
First, the alldone mode.
FileBench Version 1.1.1
filebench> load copyfiles
104594: 3.533: CopyFiles Version 2.2 personality successfully loaded
104594: 3.535: Usage: set $dir=<dir>
104594: 3.535: set $filesize=<size> defaults to 16384
104594: 3.535: set $nfiles=<value> defaults to 5000
104594: 3.535: set $iosize=<size> defaults to 1048576
104594: 3.536: set $dirwidth=<value> defaults to 20
104594: 3.536: set $nthreads=<value> defaults to 16
104594: 3.536:
104594: 3.536: run 0
filebench> run 0
104594: 5.372: Creating/pre-allocating files and filesets
104594: 5.581: Fileset destfiles: 5000 files, avg dir = 20.0, avg depth
= 2.8, mbytes=78
104594: 6.079: Removed any existing fileset destfiles in 1 seconds
104594: 6.274: Creating fileset destfiles...
104594: 6.274: Preallocated 0 of 5000 of fileset destfiles in 1 seconds
104594: 6.644: Fileset bigfileset: 5000 files, avg dir = 20.0, avg depth
= 2.8, mbytes=78
104594: 7.107: Removed any existing fileset bigfileset in 1 seconds
104594: 7.284: Creating fileset bigfileset...
104594: 87.671: Preallocated 5000 of 5000 of fileset bigfileset in 81
seconds
104594: 87.671: waiting for fileset pre-allocation to finish
104594: 87.672: Starting 1 filereader instances
104599: 88.685: Starting 16 filereaderthread threads
104594: 91.694: Running...
104594: 100.764: Run took 9 seconds...
104594: 100.768: Per-Operation Breakdown
closefile2 551ops/s 0.0mb/s 0.1ms/op
25us/op-cpu
closefile1 551ops/s 0.0mb/s 0.0ms/op
29us/op-cpu
writefile2 551ops/s 8.6mb/s 0.6ms/op
347us/op-cpu
createfile2 551ops/s 0.0mb/s 1.4ms/op
639us/op-cpu
readfile1 551ops/s 8.6mb/s 0.4ms/op
110us/op-cpu
openfile1 551ops/s 0.0mb/s 0.7ms/op
219us/op-cpu
104594: 100.769:
IO Summary: 30000 ops 3308.1 ops/s, (551/551 r/w) 17.2mb/s,
1613us cpu/op, 1.6ms latency
104594: 100.769: Shutting down processes
filebench> quit
Then, what happens with firstdone mode. Note that the total number of
ops is slightly less because a few threads are prevented from finishing
their last cycle through their flowop list.
FileBench Version 1.1.1
filebench> load copyfiles
104600: 2.581: CopyFiles Version 2.2 personality successfully loaded
104600: 2.582: Usage: set $dir=<dir>
104600: 2.583: set $filesize=<size> defaults to 16384
104600: 2.583: set $nfiles=<value> defaults to 5000
104600: 2.583: set $iosize=<size> defaults to 1048576
104600: 2.583: set $dirwidth=<value> defaults to 20
104600: 2.583: set $nthreads=<value> defaults to 16
104600: 2.584:
104600: 2.584: run 0
filebench> set mode quitfirstdone
filebench> run 0
104600: 15.614: Creating/pre-allocating files and filesets
104600: 15.823: Fileset destfiles: 5000 files, avg dir = 20.0, avg depth
= 2.8, mbytes=78
104600: 18.178: Removed any existing fileset destfiles in 3 seconds
104600: 18.371: Creating fileset destfiles...
104600: 18.371: Preallocated 0 of 5000 of fileset destfiles in 1 seconds
104600: 18.723: Fileset bigfileset: 5000 files, avg dir = 20.0, avg
depth = 2.8, mbytes=78
104600: 21.177: Removed any existing fileset bigfileset in 3 seconds
104600: 21.354: Creating fileset bigfileset...
104600: 109.906: Preallocated 5000 of 5000 of fileset bigfileset in 89
seconds
104600: 109.906: waiting for fileset pre-allocation to finish
104600: 109.908: Starting 1 filereader instances
104605: 110.913: Starting 16 filereaderthread threads
104600: 113.922: Running...
104600: 121.972: Run took 8 seconds...
104600: 121.976: Per-Operation Breakdown
closefile2 621ops/s 0.0mb/s 0.0ms/op
26us/op-cpu
closefile1 621ops/s 0.0mb/s 0.0ms/op
30us/op-cpu
writefile2 621ops/s 9.7mb/s 1.1ms/op
355us/op-cpu
createfile2 621ops/s 0.0mb/s 2.1ms/op
659us/op-cpu
readfile1 620ops/s 9.7mb/s 0.4ms/op
113us/op-cpu
openfile1 621ops/s 0.0mb/s 1.0ms/op
227us/op-cpu
104600: 121.977:
IO Summary: 29990 ops 3726.1 ops/s, (620/621 r/w) 19.4mb/s,
1425us cpu/op, 2.3ms latency
104600: 121.977: Shutting down processes
filebench> quit
And finally, what happens if you use timeout mode and stop too soon:
FileBench Version 1.1.1
filebench> load copyfiles
104606: 3.277: CopyFiles Version 2.2 personality successfully loaded
104606: 3.278: Usage: set $dir=<dir>
104606: 3.279: set $filesize=<size> defaults to 16384
104606: 3.279: set $nfiles=<value> defaults to 5000
104606: 3.279: set $iosize=<size> defaults to 1048576
104606: 3.279: set $dirwidth=<value> defaults to 20
104606: 3.279: set $nthreads=<value> defaults to 16
104606: 3.280:
104606: 3.280: run 0
filebench> set mode quittimeout
filebench> run 60
104606: 15.982: Creating/pre-allocating files and filesets
104606: 16.191: Fileset destfiles: 5000 files, avg dir = 20.0, avg depth
= 2.8, mbytes=78
104606: 18.799: Removed any existing fileset destfiles in 3 seconds
104606: 18.991: Creating fileset destfiles...
104606: 18.991: Preallocated 0 of 5000 of fileset destfiles in 1 seconds
104606: 19.341: Fileset bigfileset: 5000 files, avg dir = 20.0, avg
depth = 2.8, mbytes=78
104606: 23.356: Removed any existing fileset bigfileset in 5 seconds
104606: 23.538: Creating fileset bigfileset...
104606: 106.955: Preallocated 5000 of 5000 of fileset bigfileset in 84
seconds
104606: 106.956: waiting for fileset pre-allocation to finish
104606: 106.957: Starting 1 filereader instances
104611: 107.965: Starting 16 filereaderthread threads
104606: 110.975: Running...
104611: 118.296: Run stopped early:
flowop createfile2-1 could not obtain a file. Please
reduce runtime, increase fileset entries, or switch modes
104606: 119.035: Run took 8 seconds...
104606: 119.039: Per-Operation Breakdown
closefile2 620ops/s 0.0mb/s 0.0ms/op
26us/op-cpu
closefile1 620ops/s 0.0mb/s 0.0ms/op
30us/op-cpu
writefile2 620ops/s 9.7mb/s 1.5ms/op
355us/op-cpu
createfile2 620ops/s 0.0mb/s 2.6ms/op
663us/op-cpu
readfile1 619ops/s 9.7mb/s 0.6ms/op
113us/op-cpu
openfile1 620ops/s 0.0mb/s 0.7ms/op
226us/op-cpu
104606: 119.039:
IO Summary: 29990 ops 3721.5 ops/s, (619/620 r/w) 19.4mb/s,
1463us cpu/op, 2.7ms latency
104606: 119.039: Shutting down processes
filebench> quit
_______________________________________________
perf-discuss mailing list
[email protected]
