Here's my first draft of integrating my graphs directly into darcs-benchmark. It marks my first significant module in Haskell (not counting the surprisingly large xmonad.hs config file I've pieced together over the last couple of weeks).
Currently it doesn't do anything more than the python code I wrote that populated the graphs on the wiki, other than the fact that it appends the reST image directives directly into darcs-benchmark reports. There is more duped code between graphRepo and tabulateRepo than I would like, but I didn't feel confident enough in my Haskell-Fu to refactor it just yet. I wanted to try to keep my changes as separate as I could. (Similarly I didn't bother trying to better integrate the graphs into the tables -- one future tweak might be to embed the graphs directly in the rows they represent.) Comments welcome, of course. Tue May 4 04:13:04 EDT 2010 Max Battcher <[email protected]> * Add GChart graphs to reports
New patches: [Add GChart graphs to reports Max Battcher <[email protected]>**20100504081304 Ignore-this: d7c6beda21273afbb12f1319da5a04b4 ] addfile ./Graph.hs hunk ./Graph.hs 1 +module Graph ( graph ) where + +import Data.List +import Data.Maybe +import Graphics.GChart +import System.FilePath + +import Definitions +import Standard + +-- A Formatter with Double output instead of String +type Selector = TimeUnit -> Maybe MemTimeOutput -> Double + +selectMemory :: Selector +selectMemory _ (Just mt) = fromRational (mtMemMean mt) / (1024 * 1024) +selectMemory _ _ = 0.0 + +selectTime :: Selector +selectTime Milliseconds (Just mt) = mtTimeMean mt * 1000 +selectTime MinutesAndSeconds (Just mt) = mtTimeMean mt +selectTime _ _ = 0.0 + +graph :: Bool -- ^ Select for Time, else Memory + -> [(Test a, Maybe MemTimeOutput)] -> [(String, String)] +graph timeresults results = map graphs coreNames + where + graphs cname = (cname, mkGraphs cname) + coreNames = nub [ trCoreName r | (Test _ r _, _) <- results ] + mkGraphs cname = graphRepo timeresults cname results + +graphRepo :: Bool -- ^ Select for Time, else Memory + -> String -- ^ Repo name + -> [(Test a, Maybe MemTimeOutput)] -> String +graphRepo timeresults repo results = unlines $ map (graphDirective . graphUrl) rs + where + -- the results which correspond to the repo in question (or a variant thereof) + interesting = [ test | test@(Test _ r _, _) <- results, trCoreName r == repo ] + columns = nub [ (b, trName tr) | (Test _ tr b, _) <- interesting ] + rownames = map description . filter hasBenchmark $ benchmarks + hasBenchmark b = any (\ (Test tb _ _, _) -> description tb == description b) interesting + rows = [catMaybes [find (match row col) interesting | col <- columns] + | row <- rownames] + match bench (binary,name) (Test bench' tr binary', _) = + bench == description bench' && binary == binary' && trName tr == name + rs = [(title (head row) row, map mkColName columns, rowdata row) + | row <- rows] + mkColName (TestBinary b, tname) = + let v = nameToVariant tname + prefix = case vId v of + DefaultVariant -> "" + _-> vSuffix v ++ " " + in prefix ++ cutdown b + cutdown d | "darcs-" `isPrefixOf` d = cutdown (drop 6 d) + | takeExtension d == ".exe" = dropExtension d + | otherwise = d + rowdata row = [select (tu row) mt | (_, mt) <- row] + title (Test b _ _, _) row = description b ++ " (" ++ unitName (tu row) ++ ")" + select = if timeresults then selectTime else selectMemory + unitName tu = case (timeresults, tu) of + (True, Milliseconds) -> "ms" + (True, MinutesAndSeconds) -> "s" + (False, _) -> "MiB" + tu row = case timeresults of + True -> case mapMaybe getTime row of + [] -> Milliseconds + xs -> appropriateUnit (minimum xs) + False -> Milliseconds + getTime (_, Just mt) = Just $ mtTimeMean mt + getTime _ = Nothing + +graphDirective :: String -> String +graphDirective = (++) ".. image:: " + +graphUrl :: (String, [String], [Double]) -> String +graphUrl (title, labels, results) = getChartUrl $ do + setChartSize 200 200 + setDataEncoding simple + setChartType BarVerticalGrouped + setChartTitle title + addAxis $ makeAxis { axisType = AxisBottom, axisLabels = Just labels } + addAxis $ makeAxis { axisType = AxisLeft, + axisRange = Just $ Range (0, realToFrac range) Nothing } + setColors palette + addChartData row + where + range = case results of + [] -> 1 + xs -> maximum xs + row = [truncate (result * 61.0 / range) | result <- results] :: [Int] +-- Color palette for bars, based on the Tango palette (Butter, Orange, Choc) + -- TODO: Adjust palette size based on actual number of results? + palette = [intercalate "|" ["fce94f" + , "c4a000" + , "fcaf3e" + , "ce5c00" + , "e9b96e" + , "8f5902"]] hunk ./Report.hs 18 import System.IO import Definitions +import Graph import Shellish hiding ( run ) import Standard import TabularRST as TR hunk ./Report.hs 161 , "====================================================" , "" , intercalate "\n" (map showT m_tables) + , "Timing Graphs" + , "====================================================" + , "" + , intercalate "\n" (map showG t_graphs) + , "Memory Graphs" + , "====================================================" + , "" + , intercalate "\n" (map showG m_graphs) ] where machine_details = intercalate "\n" $ hunk ./Report.hs 188 , replicate (length r) '-' , "" , TR.render id id id t ] + -- + t_graphs = graph True results + m_graphs = graph False results + showG (r,g) = intercalate "\n" [ r + , replicate (length r) '-' + , "" + , g] printCumulativeReport :: Command () printCumulativeReport = do hunk ./darcs-benchmark.cabal 40 network == 2.2.*, split == 0.1.*, utf8-string == 0.3.*, + hs-gchart, tar, zlib main-is: main.hs hunk ./darcs-benchmark.cabal 49 Config Definitions Dist + Graph Report Run Standard Context: [Open #9bf: GoogleCharts integration. 'Eric Kow <[email protected]>'**20100321111031 Ignore-this: a95ee214b37a2ed9acf034d1592a9139 ] [Include the hostname in the parameters stamp. Eric Kow <[email protected]>**20100228225819 Ignore-this: d6747bed303faceb7ac6174cf3c4112a ] [Drop .exe from darcs binary name under Windows. Eric Kow <[email protected]>**20100228224552 Ignore-this: aed052b5198c2e63fb1b5f55dc030d95 ] [Restore variant-then-version order of row headers. Eric Kow <[email protected]>**20100228224437 Ignore-this: a1652b71035fe611e2b4bf8e80f7d447 ] [Pick time unit for whole row based on smallest value in that row. Eric Kow <[email protected]>**20100228224217 Ignore-this: 6491a506dc1af8ef84aeae5ff616238a ] [Refactor time-printing code. Eric Kow <[email protected]>**20100228204812 Ignore-this: e04c3ac9065fff437650964b541e04e ] [Clean up unused imports. Eric Kow <[email protected]>**20100228185054 Ignore-this: 9db5027c3ab521ee10dc527cb2d92839 ] [Separate timing and memory tables. Eric Kow <[email protected]>**20100228185052 Ignore-this: 8ff235c5cb1e7d8281d08cf69e92cf07 ] [First stab at pstamps and timestamps. Eric Kow <[email protected]>**20100225224835 Ignore-this: aa263b2ff030e36193fcf453dcc66ae5 ] [Constraint a type to fix a warning. Eric Kow <[email protected]>**20100225223723 Ignore-this: 9953809b43547e7d470ae9a706f1ff8 ] [Fill in type signatures for Report. Eric Kow <[email protected]>**20100225223557 Ignore-this: 9f8e6689b4113637673e15beeb05bfb7 ] [Print tables in a standard order. Eric Kow <[email protected]>**20100225110908 Ignore-this: e477aa66246da2bd615affef360cfa23 ] [Shuffle lots of code around. Eric Kow <[email protected]>**20100225110557 Ignore-this: e30f9a5c3d2287ebb384b614116d6f77 New modules: - Definitions (shared code - perhaps badly named) - Run (for the darcs-benchmark run command) This will be necessary if we want to avoid a future import cycle. ] [Characterise benchmarks so that we can moosh them into a single list. Eric Kow <[email protected]>**20100225102841 Ignore-this: 2177ac19c81e160bb557ae3db546f7ca ] [Minor cleanups. Eric Kow <[email protected]>**20100225102828 Ignore-this: 5d8a6ce39c6b150de0b94ebdd6b3737f ] [Make stddev column a little more visually distinct. Eric Kow <[email protected]>**20100225085511 Ignore-this: 76420ade616491cfb061bb72ec3ebc08 ] [Enable reporting with variants noted. Eric Kow <[email protected]>**20100224224516 Ignore-this: 3ef25ae7cede22de661129d8f50285e6 ] [Safer and simpler variant-sensitive tabulation. Eric Kow <[email protected]>**20100224224250 Ignore-this: d4c8922cf914e3bb958a31649232532d ] [Make timings files a bit more manageable. Eric Kow <[email protected]>**20100224215744 Ignore-this: 5d034aa10dfcf242c55b537327b4fcec Store timings as in files ~/.darcs-benchmark/<pstamp>.timings/<tstamp> The pstamp is some kind of representation of the unique paramaters that would make benchmarks with different pstamps hard to compare. Examples of things we may want to encode in the pstamp: - machine info - parameters like --cold The hope is to have a human readable ~/.darcs-benchmark/<pstamp>.info that provides details on the stamp. The tstamp is meant to be an ISO date/time. You can actually concatenate all the <tstamp> files if you want. They're just broken up for easy management (for example, if you wanted do delete all timings from 2009) Entries in each tstamp file are tab-separated lines <repo> <darcs> <benchmark> <mem> <time> ] [Drop the redundant "darcs-" prefix from column names. Petr Rockai <[email protected]>**20100224133722 Ignore-this: 3024b578cee99e9ec15def7801ec09b3 ] [Adaptively choose display units. Petr Rockai <[email protected]>**20100224133611 Ignore-this: 10b71be8e4ddf9775c9f8bed9fcecffb ] [Helper code to guess variant core names. Eric Kow <[email protected]>**20100224132846 Ignore-this: f119a6bab0cc0f24c5a61ab40f1f174a ] [Wibble. Eric Kow <[email protected]>**20100224132841 Ignore-this: c434de0d39706eafd9098cdb6997d4a6 ] [Minor code wibble. Eric Kow <[email protected]>**20100224130836 Ignore-this: 3f4d4f391b415a3c65cdc3147f5ed22f ] [Don't append variants to sdev columns. Eric Kow <[email protected]>**20100224125830 Ignore-this: 6f325c38b6a3bea66526d7f6d786ba2c ] [Oops, fix bug in column name handling. Eric Kow <[email protected]>**20100224121642 Ignore-this: 3e4339ea2530b3b67c5172b3b113b1ea ] [Cut some superfluous characters. Eric Kow <[email protected]>**20100224121304 Ignore-this: c17e8d6fabf41ce69ed23db4cbbed38e ] [Implement Petr's idea of display sample size indicators. Eric Kow <[email protected]>**20100224121202 Ignore-this: d20a6d7e9fbdbddca686b77aec3504d1 ] [Split stddev off into its own cell. Eric Kow <[email protected]>**20100224121201 Ignore-this: 7cd3871a3fab5892170b48b36806ef86 ] [Tweak formatting functions to return lists of cells. Eric Kow <[email protected]>**20100224121109 Ignore-this: b6931d8761c596256461ac61de86820f ] [Add a very incomplete report command. Eric Kow <[email protected]>**20100223223835 Ignore-this: fcc82bfcea92123bf3c2b64edc1ce06d - does not recognise variants - thinks all benchmarks are in milliseconds ] [Improve timings file. Eric Kow <[email protected]>**20100223223714 Ignore-this: 822f9cec2c0e4522b78f2fa89a926d58 ] [Dump result from each timing into a log file. Eric Kow <[email protected]>**20100223210259 Ignore-this: d438d115ec705fe27a0bdf545a894a9e ] [Fix shadowing warning. Eric Kow <[email protected]>**20100223200556 Ignore-this: 7a20535d1d7149d8da90d62b293f7f0d ] [Rearrange and break into sections. Eric Kow <[email protected]>**20100223200254 Ignore-this: ab65929f82425157824a04a62f333b38 ] [More suggestions for things to report. Eric Kow <[email protected]>**20100223144139 Ignore-this: 65fe7b5225669b2dbdc697815ea6330f ] [Open #3e0: darcs version threshold for variants. Eric Kow <[email protected]>**20100223143655 Ignore-this: 1168cce71d86f76653da05f3a5b4cabe ] [TAG 0.1.7 Eric Kow <[email protected]>**20100222172325 Ignore-this: b7aa43ef661226e5be1d955315b4bb2e ] Patch bundle hash: 39131ac917a40d97c35a5f4c35f69474d3aa8fee
_______________________________________________ darcs-users mailing list [email protected] http://lists.osuosl.org/mailman/listinfo/darcs-users
