1. I had just run 'pypy ./runner.py' not knowing what I was actually supposed to do<http://static.tumblr.com/tmrizjj/TS8lu94i7/1320363661170.jpg>. I have since re-run the benchmarks, after discussing how they should properly be run, and came up with the results below.
2. These are "normal" JiTting PyPys, however one is using the asmgcc gc where as the other is using the shadowstack gc. I thought it might be interesting to see if any significant info can be gathered from this benchmark (something. I compared my llvm/clang build to the prebuilt 64bit Linux binary provided on the PyPy downloads page. pypy ./runner.py --baseline=/usr/bin/pypy -p /home/chaos/pypy-1.7/bin/pypy The baseline being the clang build of pypy. Here are the results from stdout: Wrote /home/chaos/Desktop/benchmarks/lib/pypy/lib_pypy/ctypes_config_cache/_pyexpat_x86_64_.py. Wrote /home/chaos/Desktop/benchmarks/lib/pypy/lib_pypy/ctypes_config_cache/_locale_x86_64_.py. Wrote /home/chaos/Desktop/benchmarks/lib/pypy/lib_pypy/ctypes_config_cache/_resource_x86_64_.py. Wrote /home/chaos/Desktop/benchmarks/lib/pypy/lib_pypy/ctypes_config_cache/_syslog_x86_64_.py. Running ai... Running bm_chameleon... Running bm_mako... Running chaos... Running crypto_pyaes... Running django... Running fannkuch... Running float... Running go... Running html5lib... Running json_bench... Running meteor-contest... Running nbody_modified... Running pyflate-fast... Running raytrace-simple... Running richards... Running rietveld... Running slowspitfire... Running spambayes... Running spectral-norm... Running spitfire... Running spitfire_cstringio... Running sympy_expand... Running sympy_integrate... Running sympy_str... Running sympy_sum... Running telco... Running translate... Running twisted_iteration... Running twisted_names... Running twisted_pb... Running twisted_tcp... Report on Linux infinity 3.1.1-gentoo #1 SMP PREEMPT Sun Nov 20 03:57:03 PST 2011 x86_64 AMD Turion(tm) II Dual-Core Mobile M520 Total CPU cores: 2 ### ai ### Min: 0.139957 -> 0.134342: 1.0418x faster Avg: 0.156932 -> 0.151293: 1.0373x faster Not significant Stddev: 0.01776 -> 0.01588: 1.1185x smaller ### bm_chameleon ### Min: 0.051917 -> 0.050041: 1.0375x faster Avg: 0.064268 -> 0.060951: 1.0544x faster Not significant Stddev: 0.01868 -> 0.01755: 1.0643x smaller ### bm_mako ### Min: 0.146432 -> 0.136968: 1.0691x faster Avg: 0.164050 -> 0.153924: 1.0658x faster Not significant Stddev: 0.02840 -> 0.02575: 1.1028x smaller ### chaos ### Min: 0.024366 -> 0.025635: 1.0521x slower Avg: 0.037429 -> 0.038286: 1.0229x slower Not significant Stddev: 0.07200 -> 0.07000: 1.0286x smaller ### crypto_pyaes ### Min: 0.121551 -> 0.133157: 1.0955x slower Avg: 0.136914 -> 0.147496: 1.0773x slower Not significant Stddev: 0.05749 -> 0.05621: 1.0229x smaller ### django ### Min: 0.101975 -> 0.095751: 1.0650x faster Avg: 0.110838 -> 0.104973: 1.0559x faster Significant (t=3.146148, a=0.95) Stddev: 0.00960 -> 0.00904: 1.0616x smaller ### fannkuch ### Min: 0.511440 -> 0.446432: 1.1456x faster Avg: 0.521307 -> 0.451537: 1.1545x faster Significant (t=23.373959, a=0.95) Stddev: 0.01503 -> 0.01482: 1.0138x smaller ### float ### Min: 0.083709 -> 0.082221: 1.0181x faster Avg: 0.106045 -> 0.102772: 1.0318x faster Not significant Stddev: 0.01608 -> 0.01566: 1.0271x smaller ### go ### Min: 0.269653 -> 0.263512: 1.0233x faster Avg: 0.481460 -> 0.468852: 1.0269x faster Not significant Stddev: 0.19745 -> 0.19127: 1.0323x smaller ### html5lib ### Min: 5.790741 -> 5.692144: 1.0173x faster Avg: 7.893078 -> 7.687743: 1.0267x faster Not significant Stddev: 2.83904 -> 2.67228: 1.0624x smaller ### json_bench ### Min: 3.498221 -> 3.506862: 1.0025x slower Avg: 3.541327 -> 3.546107: 1.0013x slower Not significant Stddev: 0.14921 -> 0.13440: 1.1102x smaller ### meteor-contest ### Min: 0.338499 -> 0.313130: 1.0810x faster Avg: 0.346351 -> 0.320863: 1.0794x faster Significant (t=8.045312, a=0.95) Stddev: 0.01531 -> 0.01635: 1.0676x larger ### nbody_modified ### Min: 0.084227 -> 0.083003: 1.0147x faster Avg: 0.086098 -> 0.085613: 1.0057x faster Not significant Stddev: 0.00588 -> 0.00667: 1.1353x larger ### pyflate-fast ### Min: 1.049249 -> 1.025093: 1.0236x faster Avg: 1.099029 -> 1.069142: 1.0280x faster Significant (t=5.351335, a=0.95) Stddev: 0.02828 -> 0.02756: 1.0262x smaller ### raytrace-simple ### Min: 0.067093 -> 0.068943: 1.0276x slower Avg: 0.081748 -> 0.082917: 1.0143x slower Not significant Stddev: 0.02475 -> 0.02978: 1.2031x larger ### richards ### Min: 0.008263 -> 0.008141: 1.0150x faster Avg: 0.009398 -> 0.009126: 1.0298x faster Not significant Stddev: 0.00345 -> 0.00286: 1.2085x smaller ### rietveld ### Min: 0.243964 -> 0.241945: 1.0083x faster Avg: 0.612855 -> 0.594768: 1.0304x faster Not significant Stddev: 0.58823 -> 0.56197: 1.0467x smaller ### slowspitfire ### Min: 0.643957 -> 0.624110: 1.0318x faster Avg: 0.671090 -> 0.654331: 1.0256x faster Significant (t=3.642100, a=0.95) Stddev: 0.02335 -> 0.02265: 1.0310x smaller ### spambayes ### Min: 0.153035 -> 0.147505: 1.0375x faster Avg: 0.294454 -> 0.281956: 1.0443x faster Not significant Stddev: 0.12360 -> 0.11756: 1.0514x smaller ### spectral-norm ### Min: 0.028271 -> 0.029005: 1.0260x slower Avg: 0.032468 -> 0.033037: 1.0175x slower Not significant Stddev: 0.01273 -> 0.01247: 1.0204x smaller ### spitfire ### Min: 9.720000 -> 9.230000: 1.0531x faster Avg: 9.817600 -> 9.385200: 1.0461x faster Significant (t=16.544288, a=0.95) Stddev: 0.10499 -> 0.15209: 1.4486x larger ### spitfire_cstringio ### Min: 4.840000 -> 4.710000: 1.0276x faster Avg: 4.863400 -> 4.748400: 1.0242x faster Significant (t=8.894143, a=0.95) Stddev: 0.05294 -> 0.07454: 1.4081x larger ### sympy_expand ### Min: 2.006796 -> 1.881002: 1.0669x faster Avg: 2.503000 -> 2.402378: 1.0419x faster Not significant Stddev: 0.97556 -> 0.93094: 1.0479x smaller ### sympy_integrate ### Min: 4.551186 -> 4.328435: 1.0515x faster Avg: 6.496911 -> 6.308380: 1.0299x faster Not significant Stddev: 2.48643 -> 2.42822: 1.0240x smaller ### sympy_str ### Min: 0.896834 -> 1.034409: 1.1534x slower Avg: 1.944165 -> 1.933199: 1.0057x faster Not significant Stddev: 1.08961 -> 1.02921: 1.0587x smaller ### sympy_sum ### Min: 1.292385 -> 1.256345: 1.0287x faster Avg: 1.729903 -> 1.671316: 1.0351x faster Not significant Stddev: 0.53698 -> 0.50304: 1.0675x smaller ### telco ### Min: 0.098984 -> 0.093985: 1.0532x faster Avg: 0.119122 -> 0.113263: 1.0517x faster Not significant Stddev: 0.05782 -> 0.05571: 1.0379x smaller ### trans_annotate ### Raw results: [1377.2] None ### trans_rtype ### Raw results: [955.9] None ### trans_backendopt ### Raw results: [421.4] None ### trans_database ### Raw results: [568.0] None ### trans_source ### Raw results: [615.2] None ### twisted_iteration ### Min: 0.014090 -> 0.014660: 1.0404x slower Avg: 0.014315 -> 0.014838: 1.0366x slower Significant (t=-18.280629, a=0.95) Stddev: 0.00015 -> 0.00014: 1.0274x smaller ### twisted_names ### Min: 0.007479 -> 0.007457: 1.0030x faster Avg: 0.007976 -> 0.007883: 1.0118x faster Significant (t=2.202431, a=0.95) Stddev: 0.00023 -> 0.00018: 1.2883x smaller ### twisted_pb ### Min: 0.037453 -> 0.039920: 1.0659x slower Avg: 0.039451 -> 0.041383: 1.0490x slower Significant (t=-10.352263, a=0.95) Stddev: 0.00103 -> 0.00082: 1.2537x smaller ### twisted_tcp ### Min: 1.113818 -> 1.192250: 1.0704x slower Avg: 1.138590 -> 1.222393: 1.0736x slower Significant (t=-16.106458, a=0.95) Stddev: 0.02450 -> 0.02745: 1.1207x larger On Thu, Dec 8, 2011 at 01:16, Armin Rigo <[email protected]> wrote: > Hi Justin, > > On Thu, Dec 8, 2011 at 09:09, Justin Noah <[email protected]> wrote: > > Here are the llvm/clang build using the shadowstack gc. What do you > think? > > 1. What is the baseline you are comparing it with? > > 2. I assume these are normal JITting PyPys. In that case, then > for benchmarks like these, it doesn't really matter what C compiler we > used, because most of the time is spent inside JIT-generated assembler > anyway. > > > A bientôt, > > Armin. > -- - Justin Noah
_______________________________________________ pypy-dev mailing list [email protected] http://mail.python.org/mailman/listinfo/pypy-dev
