Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-29 Thread Maciej Fijalkowski
On Thu, Apr 28, 2011 at 11:10 PM, Stefan Behnel stefan...@behnel.de wrote: M.-A. Lemburg, 28.04.2011 22:23: Stefan Behnel wrote: DasIch, 28.04.2011 20:55: the CPython benchmarks have an extensive set of microbenchmarks in the pybench package Try not to care too much about pybench. There

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-29 Thread Mark Shannon
Maciej Fijalkowski wrote: On Thu, Apr 28, 2011 at 11:10 PM, Stefan Behnel stefan...@behnel.de wrote: M.-A. Lemburg, 28.04.2011 22:23: Stefan Behnel wrote: DasIch, 28.04.2011 20:55: the CPython benchmarks have an extensive set of microbenchmarks in the pybench package Try not to care too

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-29 Thread M.-A. Lemburg
Mark Shannon wrote: Maciej Fijalkowski wrote: On Thu, Apr 28, 2011 at 11:10 PM, Stefan Behnel stefan...@behnel.de wrote: M.-A. Lemburg, 28.04.2011 22:23: Stefan Behnel wrote: DasIch, 28.04.2011 20:55: the CPython benchmarks have an extensive set of microbenchmarks in the pybench package

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-29 Thread Michael Foord
On 29/04/2011 11:04, M.-A. Lemburg wrote: Mark Shannon wrote: Maciej Fijalkowski wrote: On Thu, Apr 28, 2011 at 11:10 PM, Stefan Behnelstefan...@behnel.de wrote: M.-A. Lemburg, 28.04.2011 22:23: Stefan Behnel wrote: DasIch, 28.04.2011 20:55: the CPython benchmarks have an extensive set of

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-29 Thread DasIch
Given those facts I think including pybench is a mistake. It does not allow for a fair or meaningful comparison between implementations which is one of the things the suite is supposed to be used for in the future. This easily leads to misinterpretation of the results from this particular

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-29 Thread M.-A. Lemburg
DasIch wrote: Given those facts I think including pybench is a mistake. It does not allow for a fair or meaningful comparison between implementations which is one of the things the suite is supposed to be used for in the future. This easily leads to misinterpretation of the results from

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-29 Thread Antoine Pitrou
On Fri, 29 Apr 2011 14:29:46 +0200 DasIch dasdas...@googlemail.com wrote: Given those facts I think including pybench is a mistake. It does not allow for a fair or meaningful comparison between implementations which is one of the things the suite is supposed to be used for in the future.

[Python-Dev] Proposal for a common benchmark suite

2011-04-28 Thread DasIch
Hello, As announced in my GSoC proposal I'd like to announce which benchmarks I'll use for the benchmark suite I will work on this summer. As of now there are two benchmark suites (that I know of) which receive some sort of attention, those are the ones developed as part of the PyPy project[1]

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-28 Thread Stefan Behnel
DasIch, 28.04.2011 20:55: the CPython benchmarks have an extensive set of microbenchmarks in the pybench package Try not to care too much about pybench. There is some value in it, but some of its microbenchmarks are also tied to CPython's interpreter behaviour. For example, the benchmarks

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-28 Thread M.-A. Lemburg
Stefan Behnel wrote: DasIch, 28.04.2011 20:55: the CPython benchmarks have an extensive set of microbenchmarks in the pybench package Try not to care too much about pybench. There is some value in it, but some of its microbenchmarks are also tied to CPython's interpreter behaviour. For

Re: [Python-Dev] Proposal for a common benchmark suite

2011-04-28 Thread Stefan Behnel
M.-A. Lemburg, 28.04.2011 22:23: Stefan Behnel wrote: DasIch, 28.04.2011 20:55: the CPython benchmarks have an extensive set of microbenchmarks in the pybench package Try not to care too much about pybench. There is some value in it, but some of its microbenchmarks are also tied to CPython's