[issue6422] timeit called from within Python should allow autoranging

[Cheryl Sabella]

Cheryl Sabella  added the comment:

The new ticket is #36461.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Steven,

Thank you.  Yes, I was thinking the same thing. But it might be better at this 
point for that change to have its own ticket, so I'll open a new issue for it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> Were you working on the additional functionality that you mentioned in 
> msg272704 or would that be open for someone else to do?  Thanks!

Please consider it open. I don't expect it to be difficult, it's just 
finding the Round Tuits. Perhaps an easy first issue for someone?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Hello Steven,

Were you working on the additional functionality that you mentioned in 
msg272704 or would that be open for someone else to do?  Thanks!

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Steven D'Aprano]

Steven D'Aprano added the comment:

Still to do (coming soon):

- make the 0.2s time configurable;
- have `timeit` and `repeat` methods (and functions) fall back 
  on `autorange` if the number is set to 0 or None.

--
assignee:  -> steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Roundup Robot]

Roundup Robot added the comment:

New changeset 424eb46f7f3a by Steven D'Aprano in branch 'default':
Issue6422 add autorange method to timeit.Timer
https://hg.python.org/cpython/rev/424eb46f7f3a

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Raymond Hettinger]

Raymond Hettinger added the comment:

I think the patch is good to go.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Steven D'Aprano]

Steven D'Aprano added the comment:

Nick gave a +1 to my auto-range patch with callback on 2016-05-13, and there's 
been no negative feedback since. Should I go ahead and check it in for 3.6?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Nick Coghlan]

Nick Coghlan added the comment:

Good point - given that, +1 from me for the callback based version, especially 
since exception chaining will still disambiguate failures in the callback from 
other errors.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Steven D'Aprano]

Steven D'Aprano added the comment:

On Thu, May 12, 2016 at 04:49:59AM +, Nick Coghlan wrote:
> The embedded side-effects were my main concern with Scott's original 
> patch, so Steven's callback-based approach strikes me as a definite 
> improvement. However, the awkwardness of the revised calling code in 
> main does make me wonder whether or not this might be better 
> implemented as a generator rather than as a function accepting a 
> callback:

I thought about a generator too, but then I thought about the *non* 
verbose case, where you don't care about the intermediate results, only 
the final (number, time_taken) pair.

# function with callback:
number, time_taken = t.autorange()

# generator
number, time_taken = list(t.autorange())[-1]

Which hints that your code snippet is buggy, or at least incomplete:

> try:
> results = list(t.autorange())
> except:
> t.print_exc()
> return 1
> if verbose:
> for number, time_taken in results:
> msg = "{} loops -> {:.{}g} secs"
> print(msg.format(number, time_taken, precision))

If verbose is False, you never set number and time_taken. So you need an 
else clause:

else:
number, time_taken = results[-1]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Nick Coghlan]

Nick Coghlan added the comment:

The embedded side-effects were my main concern with Scott's original patch, so 
Steven's callback-based approach strikes me as a definite improvement. However, 
the awkwardness of the revised calling code in main does make me wonder whether 
or not this might be better implemented as a generator rather than as a 
function accepting a callback:

try:
results = list(t.autorange())
except:
t.print_exc()
return 1
if verbose:
for number, time_taken in results:
msg = "{} loops -> {:.{}g} secs"
print(msg.format(number, time_taken, precision))

(Originally I had the "if verbose: print" embedded in a direct loop over 
t.autorange(), but writing it that way made it immediately clear that the scope 
of the exception handler was too broad, so I changed it to extract all the 
results and only then print them)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Antoine Pitrou]

Antoine Pitrou added the comment:

> I don't understand which repeat functionality you're referring to.

https://docs.python.org/3/library/timeit.html#timeit.Timer.repeat

(or, similarly, what timeit's __main__ does: report the minimum of all N runs)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Steven D'Aprano]

Steven D'Aprano added the comment:

> I would suggest making the 0.2 tunable as an optional argument.

Sounds like a good idea to me.

> I also notice the repeat functionality isn't included in the patch, is there 
> a reason?

I don't understand which repeat functionality you're referring to.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Antoine Pitrou]

Antoine Pitrou added the comment:

I would suggest making the 0.2 tunable as an optional argument. Different 
applications (benchmarks) may want different duration / precision tradeoffs.
I also notice the repeat functionality isn't included in the patch, is there a 
reason?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Steven D'Aprano]

Steven D'Aprano added the comment:

This issue seems to have lost momentum, I'd like to revive it by proposing a 
slightly different interface for the autorange function.

Attached is a proof-of-concept patch. I've moved the code which determines the 
number of loops out of the main function into a new Timer method, 
``autorange``. The main difference between my approach and Scott's is that my 
autorange method doesn't do any printing directly, it takes an optional 
callback function. This lets the caller take responsibility for all output.

If this approach is acceptable, I hope to:

- (in 3.6) add tests and docs for the new method;

- (in 3.6 if time permits, otherwise 3.7) modify the timeit and repeat methods 
and functions so that they can optionally call autorange(), e.g. if the caller 
passes 0 as the number.

I'm not sure that there's a good reason to add a top-level autorange() function 
to match the timeit() and repeat() functions. Especially not once they gain the 
ability to autorange themselves.

I think my approach will be compatible with cleaning up and refactoring the 
main() function. At the moment, main() is a mess IMO, it handles argument 
processing, autoranging, units of time, and unreliable timing detection all 
from one function.

--
nosy: +steven.daprano
versions: +Python 3.6 -Python 3.5
Added file: http://bugs.python.org/file42816/timeit-autorange.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Nick Coghlan]

Nick Coghlan added the comment:

The current patch moves print operations inside timeit() and repeat(), instead 
of leaving the main() function as the only one with side effects.

My counter-proposal was to instead extract the current main functionality out 
into a side-effect free public API of its own, and change the existing main 
function to call that new API and print the results.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Robert Collins]

Robert Collins added the comment:

Filed #23693 for the accuracy thing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Robert Collins]

Robert Collins added the comment:

I'm confused by the feedback on the patch. It adds a single new function, 
doesn't alter the public interface for any existing functions, and seems fit 
for purpose. Could someone help me understand how its deficient?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Robert Collins]

Changes by Robert Collins :


--
nosy: +rbcollins
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
assignee: amaury.forgeotdarc -> 
versions: +Python 3.5 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Antoine Pitrou]

Antoine Pitrou added the comment:

> In #5442, I proposed leaving the architecture of the module alone, and 
> simply exposing the main module functionality as a high level helper
> function:

Agreed with Nick's approach above.

Victor, if you want to improve timeit's reliability, please open a separate 
issue.

--
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Andrew Svetlov]

Changes by Andrew Svetlov :


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[STINNER Victor]

STINNER Victor  added the comment:

> The calibration function uses also the precision of the timer.

Oh, I forgot to mention that it computes the precision in Python, it doesn't 
read the precision announced by the OS or the precision of the C structure.
https://bitbucket.org/haypo/misc/src/bfacfb9a1224/python/benchmark.py#cl-66

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[STINNER Victor]

STINNER Victor  added the comment:

Hi, I wrote recently a similar function because timeit is not reliable by 
default. Results look random and require to run the same benchmark 3 times or 
more on the command line.

https://bitbucket.org/haypo/misc/src/tip/python/benchmark.py

By default, the benchmark takes at least 5 measures, one measure should be 
greater than 100 ms, and the benchmark should not be longer than 1 second. I 
chose these parameters to get reliable results on microbenchmarks like 
"abc".encode("utf-8").

The calibration function uses also the precision of the timer. The user may 
define a minimum time (of one measure) smaller than the timer precision, so the 
calibration function tries to solve such issue. The calibration computes the 
number of loops and the number of repetitions.

Look at BenchmarkRunner.calibrate_timer() and BenchmarkRunner.run_benchmark().
https://bitbucket.org/haypo/misc/src/bfacfb9a1224/python/benchmark.py#cl-362

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Nick Coghlan]

Changes by Nick Coghlan :


--
resolution: accepted -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Nick Coghlan]

Nick Coghlan  added the comment:

Oops, that link reference should have been to #5441.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Nick Coghlan]

Nick Coghlan  added the comment:

In #5442, I proposed leaving the architecture of the module alone, and simply 
exposing the main module functionality as a high level helper function:

def measure(stmt="pass", setup="pass", timer=default_timer,
repeat=default_repeat, number=default_number,
verbosity=0, precision=3)

The return value would simply be a (number, results) 2-tuple with the number of 
iterations per test (which may have been calculated automatically), and then a 
list of the results. To get "timeit" style behavior, simply set "repeat=1".

--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Looking at this again after more time has passes, I still think exposing 
autoranging is a good idea but I don't like the patch as it stands.  It 
"infects" the API in a number of places and makes the overall module harder to 
use and learn.   

Ideally, there should be a cleaner interface, or more limited API change, or a 
separate high level function that can autorange existing functions without 
changing their API.

Anyone care to propose a cleaner API?

--
versions: +Python 3.4 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Tshepang Lekhonkhobe]

Changes by Tshepang Lekhonkhobe :


--
nosy: +tshepang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Not sure why you chose 0.11 here. It should probably be 0.2 as in the 
command-line code.
As for applying the patch, this can't be done before 3.2 is released.

--
nosy: +pitrou
stage: unit test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Éric Araujo]

Changes by Éric Araujo :


--
nosy: +eric.araujo
versions: +Python 3.3 -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Raymond Hettinger]

Raymond Hettinger  added the comment:

This does not conflict with the other proposed changes to timeit and it is 
in-line with Guido's desire that to expose useful parts currently buried in the 
command-line logic.

Amaury, you've shown an interest.  Would you like to apply it?

--
assignee: rhettinger -> amaury.forgeotdarc
resolution:  -> accepted
versions:  -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I would like to look at this in context of all the other proposed build-
outs to timeit.

--
assignee:  -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Scott David Daniels]

Changes by Scott David Daniels :


--
keywords: +patch
Added file: http://bugs.python.org/file14472/timeit.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

You can still upload available patches to this tracker.

--
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Scott David Daniels]

Scott David Daniels  added the comment:

I've got the code "working" on trunk2 for my tests.
Should I port to py3K before checking in, and give diffs from there, or 
what?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Ezio Melotti]

Changes by Ezio Melotti :


--
priority:  -> normal
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6422] timeit called from within Python should allow autoranging

[Scott David Daniels]

New submission from Scott David Daniels :

timeit.main has a _very_ handy autoranging facility to pick an
appropriate number of repetitions when not specified.  The autoranging
code should be lifted to a method on Timer instances (so non-main code
can use it).  If number is specified as 0 or None, I would like to use
the results of that autoranging code in Timer.repeat and Timer.timeit.

Patch to follow.

--
components: Library (Lib)
messages: 90157
nosy: scott_daniels
severity: normal
status: open
title: timeit called from within Python should allow autoranging
type: feature request
versions: Python 2.7, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com