Hi, one small thing to make the reports more human-friendly. I am sending this, as opposed to pushing (yes, I can push now, finally) because this makes the tabulated output really ugly. Basically, this could make use of rowspans which I have no idea how to express (neither in RST nor in the tabular internal format). Something along the lines of this:
+---------------------------+ | | darcs-2.3 | +---------+-------+---------+ | wh | 6.6ms | d=0.6ms | +---------+-------+---------+ (it'd be even better without the extra line between time and deviation, but it really helps to have them both right-aligned separately) Yours, Petr. 1 patch for repository code.haskell.org:/srv/code/darcs/darcs-benchmark: Wed Feb 24 00:15:55 CET 2010 Petr Rockai <[email protected]> * Adaptively choose display units. PS: With the patch, the output looks like this: hashed-storage -------------- ============== ================= ================= darcs-2.3 darcs-2.3.99.2 ============== ================= ================= (un)revert mod 99.8ms d=0.8ms 94.7ms d=1.6ms annotate 162.7ms d=1.8ms 161.8ms d=1.9ms check 183.7ms d=2.0ms 276.4ms d=4.6ms get (full) 126.4ms d=4.0ms 125.4ms d=1.0ms get (lazy) 32.1ms d=8.3ms 31.2ms d=0.3ms pull 100 482.7ms d=10.4ms 440.7ms d=4.5ms pull 1000 1106.7ms d=17.4ms 1030.6ms d=13.4ms record mod 110.4ms d=11.3ms 88.6ms d=4.1ms repair 184.3ms d=2.2ms 270.9ms d=3.8ms revert mod 32.4ms d=0.8ms 30.8ms d=0.5ms wh 6.6ms d=0.6ms 7.0ms d=0.1ms wh -l 10.9ms d=0.4ms 9.1ms d=0.5ms wh mod 11.9ms d=0.3ms 12.3ms d=0.2ms ============== ================= ================= GHC --- ============== =============== darcs-2.3 ============== =============== (un)revert mod 2.2s d=9.2ms get (full) 13.3s d=843.0ms get (lazy) 5.1s d=900.0ms pull 100 6.8s d=229.6ms record mod 4.3s d=14.2ms revert mod 320.9ms d=6.3ms wh 16.0ms d=1.0ms wh -l 223.1ms d=2.5ms wh mod 79.1ms d=8.2ms ============== =============== (the second table is incomplete, from "darcs-benchmark report" of an interrupted benchmark run)
New patches: [Adaptively choose display units. Petr Rockai <[email protected]>**20100223231555 Ignore-this: cc15af240ff5243b51a8c9551709626d ] hunk ./Benchmark.hs 49 data MemTimeOutput = MemTimeOutput { mtTimeMean :: Double , mtTimeDev :: Double , mtSampleSize :: Int - , mtUnit :: TimeUnit , mtMemMean :: Rational } deriving (Show) hunk ./Benchmark.hs 53 -mkMemTimeOutput :: [MemTime] -> TimeUnit -> MemTimeOutput -mkMemTimeOutput xs u = +mkMemTimeOutput :: [MemTime] -> MemTimeOutput +mkMemTimeOutput xs = MemTimeOutput { mtTimeMean = mean time_v , mtTimeDev = stdDev time_v , mtMemMean = toRational (mean mem_v) hunk ./Benchmark.hs 59 , mtSampleSize = lengthU time_v - , mtUnit = u } where time_v = toU [ t | MemTime _ t <- xs ] hunk ./Benchmark.hs 95 data TestBinary = TestBinary String deriving (Show, Read) -data TimeUnit = MilliSeconds | Seconds - -instance Show TimeUnit where - show MilliSeconds = "ms" - show Seconds = "s" - -multiplier :: TimeUnit -> Double -multiplier MilliSeconds = 1000 -multiplier Seconds = 1 - -data Benchmark a = Idempotent TimeUnit String (BenchmarkCmd a) - | Destructive TimeUnit String (BenchmarkCmd a) +data Benchmark a = Idempotent String (BenchmarkCmd a) + | Destructive String (BenchmarkCmd a) | Description String instance Show (Benchmark a) where hunk ./Benchmark.hs 140 when is_file $ copyFile (from </> item) (to </> item) description :: Benchmark a -> String -description (Idempotent _ d _) = d -description (Destructive _ d _) = d +description (Idempotent d _) = d +description (Destructive d _) = d description (Description d) = d hunk ./Benchmark.hs 144 -timeUnit :: Benchmark a -> TimeUnit -timeUnit (Idempotent u _ _) = u -timeUnit (Destructive u _ _) = u -timeUnit (Description _) = Seconds - reset :: Command () reset = do resetMemoryUsed hunk ./Benchmark.hs 150 resetTimeUsed exec :: Benchmark a -> FilePath -> TestRepo -> Command a -exec (Idempotent _ _ cmd) darcs_path tr = do +exec (Idempotent _ cmd) darcs_path tr = do cd "_playground" verbose "cd _playground" cmd (darcs darcs_path) tr hunk ./Benchmark.hs 154 -exec (Destructive _ _ cmd) darcs_path tr = do +exec (Destructive _ cmd) darcs_path tr = do cd "_playground" let cleanup = verbose "cd .. ; rm -rf _playground" >> cd ".." >> rm_rf "_playground" res <- cmd (darcs darcs_path) tr `catchError` \e -> (cleanup >> throw e) hunk ./Benchmark.hs 227 liftIO (readIORef maybeVMFlush >>= \go -> go darcs_path) m <- timed (exec benchmark darcs_path tr) return m - let result = mkMemTimeOutput times (timeUnit benchmark) + let result = mkMemTimeOutput times spaces = 45 - (length bin + length (description benchmark) + length (trName tr)) result_str = unwords [ formatTimeResult result, formatMemoryResult result, formatSampleSize result ] liftIO $ appendResult () test times hunk ./Benchmark.hs 359 formatTimeResult :: MemTimeOutput -> String formatTimeResult mt = - formatNumber (adjust (mtTimeMean mt)) ++ show (mtUnit mt) ++ " d=" ++ formatNumber (mtTimeDev mt) + time (mtTimeMean mt) ++ " d=" ++ time (mtTimeDev mt) where hunk ./Benchmark.hs 361 - adjust = (*) (multiplier (mtUnit mt)) + secs :: Double -> Maybe Double + secs x | x > 2 = Just $ (fromInteger (floor x `mod` 60)) + (x - fromInteger (floor x)) + | otherwise = Nothing + mins x | x > 60 = Just (floor x `div` 60) + | otherwise = Nothing + time raw = case (mins raw, secs raw) of + (Nothing, Nothing) -> formatNumber (raw * 1000) ++ "ms" + (Nothing, Just s) -> formatNumber s ++ "s" + (Just m, Just s) -> show m ++ "m" ++ formatNumber s ++ "s" formatMemoryResult :: MemTimeOutput -> String formatMemoryResult mt = hunk ./Report.hs 24 process ((_, repo, dbin, bm), times) = (key, val) where key = Test (Description bm) (mkTr repo) (TestBinary dbin) - val = Just . flip mkMemTimeOutput MilliSeconds $ times -- TODO: infer units from benchmarks + val = Just $ mkMemTimeOutput times mkTr n = TestRepo n n n Nothing [] -- TODO: infer variants from names parseLine l = hunk ./Standard.hs 103 return () fast :: [ Benchmark () ] -fast = [ Destructive Seconds "get (full)" $ get [] - , Destructive Seconds "get (lazy)" $ get ["--lazy"] - , Idempotent Seconds "pull 100" $ pull 100 - , Idempotent MilliSeconds "wh" wh - , Idempotent MilliSeconds "wh mod" wh_mod - , Idempotent MilliSeconds "wh -l" wh_l - , Idempotent MilliSeconds "record mod" $ record_mod - , Idempotent MilliSeconds "revert mod" revert_mod - , Idempotent MilliSeconds "(un)revert mod" revert_unrevert +fast = [ Destructive "get (full)" $ get [] + , Destructive "get (lazy)" $ get ["--lazy"] + , Idempotent "pull 100" $ pull 100 + , Idempotent "wh" wh + , Idempotent "wh mod" wh_mod + , Idempotent "wh -l" wh_l + , Idempotent "record mod" $ record_mod + , Idempotent "revert mod" revert_mod + , Idempotent "(un)revert mod" revert_unrevert ] standard :: [ Benchmark () ] hunk ./Standard.hs 116 standard = fast ++ - [ Idempotent Seconds "check" check - , Idempotent Seconds "repair" repair - , Idempotent Seconds "annotate" annotate - , Idempotent Seconds "pull 1000" $ pull 1000 ] + [ Idempotent "check" check + , Idempotent "repair" repair + , Idempotent "annotate" annotate + , Idempotent "pull 1000" $ pull 1000 ] Context: [Add a very incomplete report command. Eric Kow <[email protected]>**20100223223835 Ignore-this: fcc82bfcea92123bf3c2b64edc1ce06d - does not recognise variants - thinks all benchmarks are in milliseconds ] [Improve timings file. Eric Kow <[email protected]>**20100223223714 Ignore-this: 822f9cec2c0e4522b78f2fa89a926d58 ] [Dump result from each timing into a log file. Eric Kow <[email protected]>**20100223210259 Ignore-this: d438d115ec705fe27a0bdf545a894a9e ] [Fix shadowing warning. Eric Kow <[email protected]>**20100223200556 Ignore-this: 7a20535d1d7149d8da90d62b293f7f0d ] [Rearrange and break into sections. Eric Kow <[email protected]>**20100223200254 Ignore-this: ab65929f82425157824a04a62f333b38 ] [More suggestions for things to report. Eric Kow <[email protected]>**20100223144139 Ignore-this: 65fe7b5225669b2dbdc697815ea6330f ] [Open #3e0: darcs version threshold for variants. Eric Kow <[email protected]>**20100223143655 Ignore-this: 1168cce71d86f76653da05f3a5b4cabe ] [TAG 0.1.7 Eric Kow <[email protected]>**20100222172325 Ignore-this: b7aa43ef661226e5be1d955315b4bb2e ] Patch bundle hash: a3fc4d5ac879e618e5a175e42460cb6f78285dae
_______________________________________________ darcs-users mailing list [email protected] http://lists.osuosl.org/mailman/listinfo/darcs-users
