On Tue, 04 Mar 2008, Kees Cook wrote: > mplayer doesn't compile with PIE due to the various ASM routines. (I've > noted this failure mode in the wiki[2] now.) However, with everything > else enabled (including FORTIFY_SOURCE), there was no measurable > difference (it was below the percentage difference between runs): > > runtime in seconds > Mplayer Normal Hardened > 1 10.87 10.807 > 2 10.873 10.824 > 3 10.854 10.963 > 4 10.809 10.84 > 5 10.877 10.838
Just for future reference, it'd probably be better to run more than 5 tests of each population in the future, as 5 tests means you'll only detect very large differences in performance at any reasonable level of signifigance. FE: > t.test(x=c(10.87,10.873,10.854,10.809,10.877),y=c(10.807,10.824,10.963,10.84,10.838)) Welch Two Sample t-test data: c(10.87, 10.873, 10.854, 10.809, 10.877) and c(10.807, 10.824, 10.963, 10.84, 10.838) t = 0.0722, df = 5.561, p-value = 0.945 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -0.07382831 0.07822831 sample estimates: mean of x mean of y 10.8566 10.8544 > This one showed a possible difference: > > nexuiz Normal Hardened > 1 66.68 68.113 > 2 66.802 66.93 > 3 66.758 67.03 > 4 66.728 67.051 > 5 66.859 67.037 While there may be a possible difference here, it's not significant, given the sample size: > t.test(x=c(66.68,66.802,66.758,66.728,66.859),y=c(68.113,66.93,67.03,67.051,67.037)) Welch Two Sample t-test data: c(66.68, 66.802, 66.758, 66.728, 66.859) and c(68.113, 66.93, 67.03, 67.051, 67.037) t = -2.0899, df = 4.154, p-value = 0.1023 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -1.0779888 0.1443888 sample estimates: mean of x mean of y 66.7654 67.2322 But useful data nevertheless.[1] Don Armstrong 1: I won't even begin to discuss how many times I see benchmarks without SEM or sd reported. -- I'd sign up in a hot second for any cellular company whose motto was: "We're less horrible than a root canal with a cold chisel." -- Cory Doctorow http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]