Here are the results of some simple tests using four different versions
of darcs on my ubuntu box. A full transcript is included so you can spot
any problems with my method and see how to run these tests yourself, and
add more.
Also, this demonstrates bench.hs, a new tool based on Eric's scripts.
This one uses haskell only and aims to be simple to use and
cross-platform. It can only run simple darcs commands though. (I
extended it to handle shell commands last night, but the gods were
angered and I lost that work. :/)
I can't run the actual bigzoo tests on this machine, unfortunately, as
it would mean shutting down my cpu-hogging web app servers for several
hours. I did shut them down during the tests below, and monitored with
top to be sure the box was otherwise quiet.
-Simon
# darcs/bytestring benchmarking transcript
# 1. build four versions of darcs
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 6.8.2
$ ghc-pkg list bytestring
/usr/local/lib/ghc-6.8.2/package.conf:
bytestring-0.9.0.1
/home/simon/.ghc/x86_64-linux-6.8.2/package.conf:
$ darcs get --lazy http://darcs.net/unstable darcs-presprint
$ darcs --repodir darcs-presprint obliterate -a --matches 'date "after
2008-10-25"' # inclusive
$ darcs get --lazy darcs-presprint darcs-presprint-no-bytestring
$ (cd darcs-presprint-no-bytestring; autoconf; ./configure
--disable-bytestring; make)
$ darcs get --lazy darcs-presprint darcs-presprint-bytestring-0.9.0.1
$ (cd darcs-presprint-bytestring-0.9.0.1; autoconf; ./configure
--enable-bytestring; make)
$ darcs get --partial http://darcs.haskell.org/bytestring
$ (cd bytestring; runhaskell Setup.lhs configure; runhaskell Setup.lhs build;
sudo runhaskell Setup.lhs install)
$ sudo ghc-pkg hide bytestring-0.9.0.1
$ ghc-pkg list bytestring
/usr/local/lib/ghc-6.8.2/package.conf:
(bytestring-0.9.0.1), bytestring-0.9.1.4
/home/simon/.ghc/x86_64-linux-6.8.2/package.conf:
$ darcs get --lazy darcs-presprint darcs-presprint-bytestring-0.9.1.4
$ (cd darcs-presprint-bytestring-0.9.1.4; autoconf; ./configure
--enable-bytestring; make)
$ darcs get --lazy http://code.haskell.org/darcs/sprint
darcs-sprint-bytestring-0.9.1.4
$ (cd darcs-sprint-bytestring-0.9.1.4; autoconf; ./configure
--enable-bytestring; make)
# actually, had to build with cabal to work around a darcs changes hang
$ ln -s darcs-presprint-no-bytestring/darcs darcsnobs
$ ln -s darcs-presprint-bytestring-0.9.0.1/darcs darcsbs0901
$ ln -s darcs-presprint-bytestring-0.9.1.4/darcs darcsbs0914
$ ln -s darcs-sprint-bytestring-0.9.1.4/darcs darcssprint
$ ./darcsnobs --version; ./darcsbs0901 --version; ./darcsbs0914 --version;
./darcssprint --version
2.1.0 (+ 194 patches)
2.1.0 (+ 194 patches)
2.1.0 (+ 194 patches)
2.1.0 (+ 321 patches)
# 2. get latest bench.hs
$ darcs get --lazy http://joyful.com/repos/darcs-benchmark
$ (cd darcs-benchmark; sudo cabal install tabular; ghc --make bench.hs)
# 3. run some simple tests in the darcs repo
$ cat - >bench.tests
changes
changes -s
whatsnew -sl
annotate Setup.hs
$ darcs-benchmark/bench bench.tests 3 /repos/darcs-stable ./darcsnobs
./darcsbs0901 ./darcsbs0914 ./darcssprint
Running 4 tests 3 times in /repos/darcs-stable with 4 executables:
1: ./darcsnobs changes --repodir /repos/darcs-stable [1.9s]
2: ./darcsnobs changes --repodir /repos/darcs-stable [1.8s]
3: ./darcsnobs changes --repodir /repos/darcs-stable [1.9s]
1: ./darcsbs0901 changes --repodir /repos/darcs-stable [1.6s]
2: ./darcsbs0901 changes --repodir /repos/darcs-stable [1.5s]
3: ./darcsbs0901 changes --repodir /repos/darcs-stable [1.6s]
1: ./darcsbs0914 changes --repodir /repos/darcs-stable [1.7s]
2: ./darcsbs0914 changes --repodir /repos/darcs-stable [1.7s]
3: ./darcsbs0914 changes --repodir /repos/darcs-stable [1.6s]
1: ./darcssprint changes --repodir /repos/darcs-stable [1.6s]
2: ./darcssprint changes --repodir /repos/darcs-stable [1.6s]
3: ./darcssprint changes --repodir /repos/darcs-stable [1.7s]
1: ./darcsnobs changes -s --repodir /repos/darcs-stable [10.0s]
2: ./darcsnobs changes -s --repodir /repos/darcs-stable [9.9s]
3: ./darcsnobs changes -s --repodir /repos/darcs-stable [10.1s]
1: ./darcsbs0901 changes -s --repodir /repos/darcs-stable [7.8s]
2: ./darcsbs0901 changes -s --repodir /repos/darcs-stable [7.6s]
3: ./darcsbs0901 changes -s --repodir /repos/darcs-stable [7.7s]
1: ./darcsbs0914 changes -s --repodir /repos/darcs-stable [7.9s]
2: ./darcsbs0914 changes -s --repodir /repos/darcs-stable [7.8s]
3: ./darcsbs0914 changes -s --repodir /repos/darcs-stable [7.4s]
1: ./darcssprint changes -s --repodir /repos/darcs-stable [7.2s]
2: ./darcssprint changes -s --repodir /repos/darcs-stable [7.5s]
3: ./darcssprint changes -s --repodir /repos/darcs-stable [7.4s]
1: ./darcsnobs whatsnew -sl --repodir /repos/darcs-stable [8.3s]
2: ./darcsnobs whatsnew -sl --repodir /repos/darcs-stable [8.6s]
3: ./darcsnobs whatsnew -sl --repodir /repos/darcs-stable [9.1s]
1: ./darcsbs0901 whatsnew -sl --repodir /repos/darcs-stable [8.7s]
2: ./darcsbs0901 whatsnew -sl --repodir /repos/darcs-stable [8.3s]
3: ./darcsbs0901 whatsnew -sl --repodir /repos/darcs-stable [8.3s]
1: ./darcsbs0914 whatsnew -sl --repodir /repos/darcs-stable [8.4s]
2: ./darcsbs0914 whatsnew -sl --repodir /repos/darcs-stable [8.8s]
3: ./darcsbs0914 whatsnew -sl --repodir /repos/darcs-stable [8.0s]
1: ./darcssprint whatsnew -sl --repodir /repos/darcs-stable [8.6s]
2: ./darcssprint whatsnew -sl --repodir /repos/darcs-stable [8.4s]
3: ./darcssprint whatsnew -sl --repodir /repos/darcs-stable [8.2s]
1: ./darcsnobs annotate Setup.hs --repodir /repos/darcs-stable [47.6s]
2: ./darcsnobs annotate Setup.hs --repodir /repos/darcs-stable [47.3s]
3: ./darcsnobs annotate Setup.hs --repodir /repos/darcs-stable [48.1s]
1: ./darcsbs0901 annotate Setup.hs --repodir /repos/darcs-stable [38.8s]
2: ./darcsbs0901 annotate Setup.hs --repodir /repos/darcs-stable [38.9s]
3: ./darcsbs0901 annotate Setup.hs --repodir /repos/darcs-stable [38.8s]
1: ./darcsbs0914 annotate Setup.hs --repodir /repos/darcs-stable [38.4s]
2: ./darcsbs0914 annotate Setup.hs --repodir /repos/darcs-stable [38.6s]
3: ./darcsbs0914 annotate Setup.hs --repodir /repos/darcs-stable [38.3s]
1: ./darcssprint annotate Setup.hs --repodir /repos/darcs-stable [38.2s]
2: ./darcssprint annotate Setup.hs --repodir /repos/darcs-stable [38.5s]
3: ./darcssprint annotate Setup.hs --repodir /repos/darcs-stable [38.2s]
Summary (best iteration):
|| ./darcsnobs | ./darcsbs0901 | ./darcsbs0914 | ./darcssprint
==================++=============+===============+===============+==============
changes || 1.8 | 1.5 | 1.6 | 1.6
changes -s || 9.9 | 7.6 | 7.4 | 7.2
whatsnew || 0.1 | 0.1 | 0.1 | 0.1
whatsnew -sl || 8.3 | 8.3 | 8.0 | 8.2
annotate Setup.hs || 47.3 | 38.8 | 38.3 | 38.2
_______________________________________________
darcs-users mailing list
[email protected]
http://lists.osuosl.org/mailman/listinfo/darcs-users
