Hi,
as the patch description says this is a *crude hack* that only works on
Linux. Figuring ways to make this work on other systems would be great.
============== ============ ============== ===========
darcs darcs
============== ============ ============== ===========
get (full) 0.7s,1.0M get (full) 0.0s,1.0M
get (lazy) 0.4s,1.0M get (lazy) 0.0s,1.0M
pull 100 0.9s,3.0M pull 100 0.2s,3.0M
wh 223.5ms,0.0M wh 7.2ms,0.0M
wh mod 254.5ms,0.0M wh mod 7.3ms,0.0M
wh -l 335.8ms,1.0M wh -l 7.6ms,1.0M
record mod 499.5ms,2.0M record mod 36.2ms,2.0M
revert mod 397.6ms,2.0M revert mod 17.9ms,2.0M
(un)revert mod 410.6ms,2.0M (un)revert mod 49.9ms,2.0M
============== ============ ============== ===========
These are with --cold and without --cold respectively. I am posting this mostly
because I am running out of time and if anyone else finds a bit of time to make
this less crude, they could. Feel free to not use this patch at all and take a
fresh start. Tis' a proof-of-concept really.
Yours,
Petr.
1 patch for repository http://code.haskell.org/darcs/darcs-benchmark:
Sat Feb 20 17:05:56 CET 2010 Petr Rockai <[email protected]>
* A very crude hack to allow cold-cache measurement (on Linux). Use --cold.
New patches:
[A very crude hack to allow cold-cache measurement (on Linux). Use --cold.
Petr Rockai <[email protected]>**20100220160556
Ignore-this: 8f04a655cb10d1ea39387922aaeb071d
] hunk ./Benchmark.hs 28
import System.Console.CmdArgs (isLoud)
import System.Process( runInteractiveProcess, runInteractiveCommand,
waitForProcess )
+import Data.IORef
+import System.IO.Unsafe
import Text.JSON
precision :: Int
hunk ./Benchmark.hs 36
precision = 1
combine :: Ord a => [a] -> a
combine = minimum
+maybeVMFlush = unsafePerformIO $ newIORef $ \x -> return ()
type Darcs = [String] -> Command String
type BenchmarkCmd a = Darcs -> TestRepo -> Command a
hunk ./Benchmark.hs 218
Just p -> return p
times <- adaptive 2 10 100 . sub $ do
prepareIfDifferent (trPath tr)
+ liftIO (readIORef maybeVMFlush >>= \go -> go darcs_path)
timed (exec benchmark darcs_path tr)
let time = combine [ t | MemTime _ t <- times ]
mem = combine [ m | MemTime m _ <- times ]
hunk ./Config.hs 9
data Config = Get { repos :: [String] }
| Run { fast :: Bool
+ , cold :: Bool
, only :: [String]
, dump :: Bool
, extra :: [String] }
hunk ./Config.hs 19
defaultConfig =
[ mode $ Get { repos = [] &= args & typ "REPONAME" }
, mode $ Run { fast = False &= text "Exclude the most time-consuming benchmarks"
+ , cold = False &= text "Try to flush VM caches between iterations"
, dump = False &= text "Produce machine-readable output on stdout"
, only = [] &= text "Only run benchmarks with one of these substrings in their names" & typ "BENCHMARK"
, extra = [] &= args & typ "BINARY"
hunk ./darcs-benchmark.cabal 35
json == 0.4.*,
containers, bytestring, network,
HTTP >= 4000.0.8 && < 4000.1,
- tar, zlib
+ tar, zlib, haskell98
main-is: main.hs
other-modules: Shellish
hunk ./main.hs 19
import Data.Version ( showVersion )
import Paths_darcs_benchmark
import Text.JSON
+import Data.IORef
+import System( system )
help :: String
help = unlines [ "darcs-benchmark " ++ showVersion version ++ ": run standard darcs benchmarks"
hunk ./main.hs 47
download_repos :: [String] -> IO ()
download_repos r = forM_ (if null r then known_repos else r) download
+doVMFlush path = do system "sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'"
+ system $ path ++ " --version"
+ return ()
+
config :: [TestRepo] -> C.Config -> IO ([TestRepo], [TestBinary], [Benchmark ()])
config allrepos cfg = do
case cfg of
hunk ./main.hs 70
usetests = case C.only cfg of
[] -> usetests'
os -> filter (\b -> any (\o -> o `isInfixOf` show b) os) usetests'
+ when (C.cold cfg) $ writeIORef maybeVMFlush doVMFlush
when (null usebins) $ do
hPutStrLn stderr "Please specify at least one darcs binary to benchmark"
exitWith (ExitFailure 1)
Context:
[Fix think-o in adaptive.
Eric Kow <[email protected]>**20100220153323
Ignore-this: 4eadb5319247474c42e5c8a8bca53cf3
]
[Add config file for tabular and tahoe-lafs.
Eric Kow <[email protected]>**20100220152616
Ignore-this: dbe07040ce5e1c13836e6fe3c77e16fa
]
[Remove iterations constant.
Eric Kow <[email protected]>**20100220152158
Ignore-this: 633e8c0451cd60096c674f0100d4ea6e
]
[Remove now superfluous benchmark iterations code.
Eric Kow <[email protected]>**20100220151404
Ignore-this: 6060396601105cb515504c5ae8c806e6
In its place, we introduce a notion of 'minimum iterations'.
]
[Leverage the adaptive support in record benchmark as well.
Petr Rockai <[email protected]>**20100220125833
Ignore-this: 41ebb9b6aa3f6c9ada5c315c1ca66c9
]
[Restore shell-friendliness of the verbose output.
Petr Rockai <[email protected]>**20100220124512
Ignore-this: 356dd104524f851aa3a921caebbd245
]
[Add at least default variant for the repos without config.
Petr Rockai <[email protected]>**20100220124333
Ignore-this: f7352ba8f8656d67922383f0a59150a
]
[Remove config.example that confuses the JSON-config thing.
Petr Rockai <[email protected]>**20100220124314
Ignore-this: 2c3cd4a30c1b67c08bfb760e2adaae80
]
[Simplify benchmarks to account for adaptive testing.
Eric Kow <[email protected]>**20100220111523
Ignore-this: e29f6262eacf2014da9ed6100c28193
]
[Allow for benchmarks in different units.
Eric Kow <[email protected]>**20100220105406
Ignore-this: 54146314f78d5059a4f532b8a8212bec
]
[Back off from trying to implement Read on Benchmark.
Eric Kow <[email protected]>**20100220100445
Ignore-this: a3a5336ef5586c2f516ea072ed63b89a
Keep life simple.
]
[Adaptive benchmarking to avoid ad-hoc (xN multipliers)
Eric Kow <[email protected]>**20100220095202
Ignore-this: c850278f6c737794475ba69eff32faf4
]
[Use authorspellings file.
Eric Kow <[email protected]>**20100220090957
Ignore-this: 7a869a8ee495b4d5f15e44f074eff760
]
[Warn that we're setting up a variant.
Eric Kow <[email protected]>**20100220002538
Ignore-this: e07f5b9e4df44d48928f51c851fa4b33
]
[Kill some trailing whitespace.
Eric Kow <[email protected]>**20100220002527
Ignore-this: d51675a507684d7401ce474286d1767c
]
[Give variants names that won't get picked up during darcs-benchmark run
Eric Kow <[email protected]>**20100220002445
Ignore-this: e56446790e00f7e2125074a766124733
]
[Resolve #3eb.
Eric Kow <[email protected]>**20100220000958
Ignore-this: e5a8a4a46cee9c4b0e40e03bf199cfab
]
[Bump to 0.1.6.
Eric Kow <[email protected]>**20100220000834
Ignore-this: 29017313a3cb8c9502d8b55abf7c72f
]
[Keep track of repo config files.
Eric Kow <[email protected]>**20100219234834
Ignore-this: 82bd1195307bbbcea1e45b18b8268e63
]
[Wibble.
Eric Kow <[email protected]>**20100219234200
Ignore-this: 17f4aa956bbb8bab3027f00674c65bf0
]
[Consolidate tables for repo variants.
Eric Kow <[email protected]>**20100219234133
Ignore-this: 27ed470d50192764977c9528be515070
]
[Simplify handling of variants.
Eric Kow <[email protected]>**20100219234111
Ignore-this: f3921ce9dc1306baabd37e0ad97853e9
]
[Do not use a dash in the variant suffixes.
Eric Kow <[email protected]>**20100219231020
Ignore-this: 193fc83323a2b313beb5d550e7d94af7
We want to abuse them for something else.
]
[Add notion of "default" variant in case we ever add old-fashioned variant.
Eric Kow <[email protected]>**20100219230312
Ignore-this: ae61928ec4935611309bba948c301782
]
[Add a notion of "coreName" to TestRepo for tracking variants.
Eric Kow <[email protected]>**20100219223350
Ignore-this: 520d7330cb6d616b7b6d8e78f8d514dd
I'm getting a bit worried about how complicated this is starting to become.
]
[Fix missing trailing blank line in RST table rendering.
Eric Kow <[email protected]>**20100219222949
Ignore-this: 2d12ec4fbe2740b1b39105b4cd2a1f66
]
[Run benchmarks on repository variants.
Eric Kow <[email protected]>**20100219222508
Ignore-this: 59482af8df081b90789a207f7613c974
]
[Create variants of repositories on demand.
Eric Kow <[email protected]>**20100219222506
Ignore-this: 69a34812e91434e4c0a6aa54bbaf8767
]
[Introduce notion of variants to config files.
Eric Kow <[email protected]>**20100219221237
Ignore-this: 4c31a5c9792d1ee54cd46e188c24809e
Examples of variants we may want to introduce
- an optimize --pristine version of the repository
- an old-fashioned one
- in the future, a packed one
The idea is to eventually support benchmarking of different variants to
demonstrate the effect of eg, optimizing pristine.
]
[TAG 0.1.5.1
Eric Kow <[email protected]>**20100216222241
Ignore-this: 779d942891cee166e0119b09eba5abe9
]
Patch bundle hash:
09e76952c8b47ca15d8075b24aed0f02c2a7afe6
_______________________________________________
darcs-users mailing list
[email protected]
http://lists.osuosl.org/mailman/listinfo/darcs-users
