[webkit-changes] [137869] trunk/Tools

Sun, 16 Dec 2012 19:15:50 -0800

Title: [137869] trunk/Tools
Revision
137869
Author
m...@apple.com
Date
2012-12-16 19:17:35 -0800 (Sun, 16 Dec 2012)

Log Message

Add a script to compute the mean and 95% confidence interval (using two-sided t-test) of a sample set
https://bugs.webkit.org/show_bug.cgi?id=105148

Reviewed by Filip Pizlo.

Usage examples:
        
$ sampstat --help
Usage: sampstat [options] [FILES]
  Compute the mean and 95% confidence interval of a sample set.
  Standard input or files must contain two or more decimal numbers, one per line.
        
Options:
  -h, --help            show this help message and exit
  -u UNIT, --unit=UNIT  assume values are in units of UNIT
  -v, --verbose         print all values (with units)
        
$ sampstat -u MB memresults.txt
2356.90 MB +/- 101.34 MB (4.3%)
        
$ sampstat -v -u MB memresults.txt
      2318.21 MB
      2399.56 MB
      2352.93 MB
----------------
Mean: 2356.90 MB +/- 101.34 MB (4.3%)
        

* Scripts/sampstat: Added.
(sum): Helper function to compute the sum of a list.
(arithmeticMean): Compute the meam of a list.
(standardDeviation): Compute the sample standard deviation (unbiased estimator).
(standardError): Compute the sample standard error.
(tDist): Compute t(.025, n-1), the t-value for atwo-sided 95% confidence interval. 
(twoSidedConfidenceInterval): Compute the two-sided confidence interval range about the mean.

Modified Paths

Added Paths

Diff

Modified: trunk/Tools/ChangeLog (137868 => 137869)


--- trunk/Tools/ChangeLog	2012-12-17 02:43:44 UTC (rev 137868)
+++ trunk/Tools/ChangeLog	2012-12-17 03:17:35 UTC (rev 137869)
@@ -1,3 +1,41 @@
+2012-12-16  Maciej Stachowiak  <m...@apple.com>
+
+        Add a script to compute the mean and 95% confidence interval (using two-sided t-test) of a sample set
+        https://bugs.webkit.org/show_bug.cgi?id=105148
+
+        Reviewed by Filip Pizlo.
+
+        Usage examples:
+        
+        $ sampstat --help
+        Usage: sampstat [options] [FILES]
+          Compute the mean and 95% confidence interval of a sample set.
+          Standard input or files must contain two or more decimal numbers, one per line.
+        
+        Options:
+          -h, --help            show this help message and exit
+          -u UNIT, --unit=UNIT  assume values are in units of UNIT
+          -v, --verbose         print all values (with units)
+        
+        $ sampstat -u MB memresults.txt
+        2356.90 MB +/- 101.34 MB (4.3%)
+        
+        $ sampstat -v -u MB memresults.txt
+              2318.21 MB
+              2399.56 MB
+              2352.93 MB
+        ----------------
+        Mean: 2356.90 MB +/- 101.34 MB (4.3%)
+        
+
+        * Scripts/sampstat: Added.
+        (sum): Helper function to compute the sum of a list.
+        (arithmeticMean): Compute the meam of a list.
+        (standardDeviation): Compute the sample standard deviation (unbiased estimator).
+        (standardError): Compute the sample standard error.
+        (tDist): Compute t(.025, n-1), the t-value for atwo-sided 95% confidence interval. 
+        (twoSidedConfidenceInterval): Compute the two-sided confidence interval range about the mean.
+
 2012-12-15  Nima Ghanavatian  <nghanavat...@rim.com>
 
         [BlackBerry] Pass key modifiers with touch and mouse events

Added: trunk/Tools/Scripts/sampstat (0 => 137869)


--- trunk/Tools/Scripts/sampstat	                        (rev 0)
+++ trunk/Tools/Scripts/sampstat	2012-12-17 03:17:35 UTC (rev 137869)
@@ -0,0 +1,98 @@
+#!/usr/bin/env python
+
+#  Copyright (C) 2007, 2012 Apple Inc.  All rights reserved.
+#
+#  Redistribution and use in source and binary forms, with or without
+#  modification, are permitted provided that the following conditions
+#  are met:
+#  1. Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#  2. Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in the
+#     documentation and/or other materials provided with the distribution.
+# 
+#  THIS SOFTWARE IS PROVIDED BY APPLE COMPUTER, INC. ``AS IS'' AND ANY
+#  EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+#  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+#  PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE COMPUTER, INC. OR
+#  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+#  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+#  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+#  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+#  OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+
+import math
+import sys
+import re
+import fileinput
+from optparse import OptionParser
+
+usage = "usage: %prog [options] [FILES]\n  Compute the mean and 95% confidence interval of a sample set.\n  Standard input or files must contain two or more decimal numbers, one per line."
+parser = OptionParser(usage=usage)
+parser.add_option("-u", "--unit", dest="unit", default="",
+                  help="assume values are in units of UNIT", metavar="UNIT")
+parser.add_option("-v", "--verbose",
+                  action="" dest="verbose", default=False,
+                  help="print all values (with units)")
+(options, files) = parser.parse_args()
+
+def sum(items):
+    return reduce(lambda x,y: x+y, items)
+
+def arithmeticMean(items):
+    return sum(items) / len(items)
+
+def standardDeviation(mean, items):
+    deltaSquares = [(item - mean) ** 2 for item in items]
+    return math.sqrt(sum(deltaSquares) / (len(items) - 1))
+
+def standardError(stdDev, items):
+    return stdDev / math.sqrt(len(items))
+
+# t-distribution for 2-sided 95% confidence intervals
+tDistribution = [float('NaN'), float('NaN'), 12.71, 4.30, 3.18, 2.78, 2.57, 2.45, 2.36, 2.31, 2.26, 2.23, 2.20, 2.18, 2.16, 2.14, 2.13, 2.12, 2.11, 2.10, 2.09, 2.09, 2.08, 2.07, 2.07, 2.06, 2.06, 2.06, 2.05, 2.05, 2.05, 2.04, 2.04, 2.04, 2.03, 2.03, 2.03, 2.03, 2.03, 2.02, 2.02, 2.02, 2.02, 2.02, 2.02, 2.02, 2.01, 2.01, 2.01, 2.01, 2.01, 2.01, 2.01, 2.01, 2.01, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 2.00, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.99, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.98, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.97, 1.96]
+tMax = len(tDistribution)
+tLimit = 1.96
+
+def tDist(n):
+    if n > tMax:
+        return tLimit
+    return tDistribution[n]
+
+def twoSidedConfidenceInterval(items):
+    mean = arithmeticMean(items)
+    stdDev = standardDeviation(mean, items)
+    stdErr = standardError(stdDev, items)
+    return tDist(len(items)) * stdErr
+
+results = []
+
+decimalNumberPattern = re.compile(r"\d+\.?\d*")
+for line in fileinput.input(files):
+    match = re.search(decimalNumberPattern, line)
+    if match:
+        results.append(float(match.group(0)))
+
+if len(results) == 0:
+    parser.print_help()
+    quit()
+
+
+mean = arithmeticMean(results)
+confidenceInterval = twoSidedConfidenceInterval(results)
+confidencePercent = 100 * confidenceInterval / mean
+
+if options.verbose:
+    length = 7
+    for item in results:
+        line = "      %.2f %s" % (item, options.unit)
+        print line
+        length = len(line) if len(line) > length else length
+
+    print "-" * length
+
+prefix = "Mean: " if options.verbose else ""
+print "%s%.2f %s +/- %.2f %s (%.1f%%)" % (prefix, mean, options.unit, confidenceInterval, options.unit, confidencePercent)
+
Property changes on: trunk/Tools/Scripts/sampstat
___________________________________________________________________

Added: svn:executable

_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to