[issue14428] Implementation of the PEP 418

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I don't know what you two are arguing about, since the process_time() doc says 
"Return the value (in fractional seconds) of the sum of the system and user CPU 
time of the current process".

(while I'm not sure what "fractional" seconds are, they are probably seconds in 
the first place :-))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Charles-François Natali]

Charles-François Natali  added the comment:

> I'm closing again this issue.
>
> @neologix: Please open a new issue if you disagree with me on the definition 
> of "seconds" for time.process_time().

I won't reopen, but I still disagree with your definition.
process_time() returns second, as does the Unix 'time' command return:

"""
$ time sleep 1

real0m1.014s
user0m0.000s
sys 0m0.000s
"""

AFAICT, process_time() returns user + sys. Since the unit of those two
fields are second, the value returned is in second. I doesn't include
time spent in 'S'/'D'/whatever time, but it's still seconds.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

I'm closing again this issue.

@neologix: Please open a new issue if you disagree with me on the definition of 
"seconds" for time.process_time().

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

> It really looks like seconds to me, definitely not jiffies ;-)

time.process_time() uses maybe "seconds" on Linux, but it doesn't include time 
elapsed during sleep. See the test:

def test_process_time(self):
start = time.process_time()
time.sleep(0.1)
stop = time.process_time()
self.assertLess(stop - start, 0.01)

According to Wikipedia: "Since 1967, the second has been defined to be: the 
duration of 9,192,631,770 periods of the radiation corresponding to the 
transition between the two hyperfine levels of the ground state of the caesium 
133 atom."

The caesium 133 atom is not sleeping while your process is sleeping, so you 
cannot say the time.process_time() uses second. Do you see what I mean?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Charles-François Natali]

Charles-François Natali  added the comment:

> Yes, dt is not a number of seconds in the following example:
>
> t1=time.process_time(); (...); t2=time.process_time(); dt=t2-t1

OK, so what is it then?
It's not written anywhere - neither in the PEP nor in the
documentation - and it should.

Furthermore:
"""
$ cat /tmp/test_ptime.py
import sys
import time

rt = time.time()
pt = time.process_time()

for i in range(int(sys.argv[1])):
pass

print("real time: %f" % (time.time() - rt))
print("process time: %f" % (time.process_time() - pt))
$ ./python /tmp/test_ptime.py 10
real time: 0.168570
process time: 0.168425
"""

It really looks like seconds to me, definitely not jiffies ;-)
Also, I've had a quick look at the code, and ISTM that it does indeed
return seconds.

> I wrote the test to check if time.process_time() measures the total CPU of
> the process, and not the CPU time of only the current thread.
>
> I'm tired of this PEP, so I just removed the test. I don't think that it is
> really interesting and it looks difficult to write a reliable test.

Hum, right now, the only process_time test I see is test_process_time,
which just checks that a sleep almost doesn't consume "process time",
which is a bit light.

"""
# Use a factor of 0.75 because time.process_time() is maybe not precise
self.assertGreaterEqual(t2 - t1, busy * 0.75)
"""

It's not that process_time() is not precise (I assume you mean accurate ;-).
It's just that since it measures CPU time (user+system), it depends on
the scheduling: in fact, if you have let's say a real-time priority
task running at the same time, on a uniprocessor system, the thread
could in theory not even get a chance to run, which would yield a CPU
time of 0: that would still be accurate.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

> You mean that process_time() doesn't return seconds?

Yes, dt is not a number of seconds in the following example:

t1=time.process_time(); (...); t2=time.process_time(); dt=t2-t1

> I see two options: either increase the total running time,
> to make it really likely you'll get a chance to run
> (or decrase the lower threshold), or use times(2), and
> check the result against that (does Windows have such a syscall?).

I wrote the test to check if time.process_time() measures the total CPU of the 
process, and not the CPU time of only the current thread.

I'm tired of this PEP, so I just removed the test. I don't think that it is 
really interesting and it looks difficult to write a reliable test.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Roundup Robot]

Roundup Robot  added the comment:

New changeset ad3d6010379b by Victor Stinner in branch 'default':
Issue #14428: Remove test_process_time_threads() from test_time
http://hg.python.org/cpython/rev/ad3d6010379b

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 0bdf0727ee29 by Victor Stinner in branch 'default':
Issue #14428: Make test_process_time_threads() less strict
http://hg.python.org/cpython/rev/0bdf0727ee29

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Antoine Pitrou]

Antoine Pitrou  added the comment:

There are many sporadic failures on the buildbots:



==
FAIL: test_process_time_threads (test.test_time.TimeTestCase)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/test_time.py", 
line 441, in test_process_time_threads
self.assertGreaterEqual(t2 - t1, busy)
AssertionError: 0.012497368 not greater than or equal to 
0.01309399966854

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Charles-François Natali]

Charles-François Natali  added the comment:

> Hum, the problem is maybe that the thread is preempted, but the first
> problem is that the test considers that time.process_time() uses seconds.

Hum, what?
You mean that process_time() doesn't return seconds?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

"The test is a bit too optimistic: a thread is looping for 0.2s, and the test 
checks that the process time (user + system) increased of at least 0.1s. If the 
thread is preempted, there's a high chance you won't get even as low as 0.1s."

Hum, the problem is maybe that the thread is preempted, but the first problem 
is that the test considers that time.process_time() uses seconds.

I rewrote the test to make it more reliable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 1255cac63dfc by Victor Stinner in branch 'default':
Issue #14428: Rewrite test_process_time_threads() test
http://hg.python.org/cpython/rev/1255cac63dfc

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Charles-François Natali]

Charles-François Natali  added the comment:

test_process_time_threads is failing on one of the buildbots:
"""
==
FAIL: test_process_time_threads (test.test_time.TimeTestCase)
--
Traceback (most recent call last):
  File 
"/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/test_time.py",
 line 408, in test_process_time_threads
self.assertGreater(t2 - t1, 0.1)
AssertionError: 0.0804141256118 not greater than 0.1
"""

The test is a bit too optimistic: a thread is looping for 0.2s, and the test 
checks that the process time (user + system) increased of at least 0.1s. If the 
thread is preempted, there's a high chance you won't get even as low as 0.1s.
I see two options: either increase the total running time, to make it really 
likely you'll get a chance to run (or decrase the lower threshold), or use 
times(2), and check the result against that (does Windows have such a syscall?).

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Roundup Robot]

Roundup Robot  added the comment:

New changeset bd195749c0a2 by Victor Stinner in branch 'default':
Issue #14428: Use the new time.perf_counter() and time.process_time() functions
http://hg.python.org/cpython/rev/bd195749c0a2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Guido van Rossum accepted the PEP, let's commit the implementation.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 76d2e0761d18 by Victor Stinner in branch 'default':
Issue #14428, #14397: Implement the PEP 418
http://hg.python.org/cpython/rev/76d2e0761d18

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file25397/4255e3c4daf2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

667541bb315c.diff: Updated patch, last change: is_adjusted key of 
time.get_clock_info() is now mandatory.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file25396/667541bb315c.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25254/384190bb0bd5.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25395/9a93348e98e7.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file25395/9a93348e98e7.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25393/4ba64ca9abcf.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file25393/4ba64ca9abcf.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
>> Please leave the pybench default timers unchanged in case the
>> new APIs are not available.
> 
> Ok, done in the new patch: perf_counter_process_time-2.patch.

Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25210/pep418-9.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25202/perf_counter_process_time.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

> Please leave the pybench default timers unchanged in case the
> new APIs are not available.

Ok, done in the new patch: perf_counter_process_time-2.patch.

--
Added file: http://bugs.python.org/file25265/perf_counter_process_time-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

Please leave the pybench default timers unchanged in case the
new APIs are not available.

The perf_counter_process_time.patch currently changes them, even
though the new APIs are not available on older Python releases,
thus breaking pybench for e.g. Python 3.2 or earlier releases.

Ditto for the resolution changes: these need to be optional and
not cause a break when used in Python 3.1/3.2.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com


2012-04-28: PythonCamp 2012, Cologne, Germany  10 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! 

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

> can you let us know when you think the patch is ready for review?

Now!

I created a repository for the PEP. I integrated my own last comments. I tested 
the PEP on Linux 3.3, FreeBSD 8, OpenBSD 5, OpenSolaris and Windows Seven.

The implementation is ready for a review. The last patch is 384190bb0bd5.diff 
(presented as "Patch Set 12" in Rietveld).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25253/aac59a3c11ef.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file25254/384190bb0bd5.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file25253/aac59a3c11ef.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


--
hgrepos: +118

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Charles-François Natali]

Charles-François Natali  added the comment:

Victor, can you let us know when you think the patch is ready for
review?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

FreeBSD doesn't provide CLOCK_PROCESS_CPUTIME_ID, but CLOCK_PROF. CLOCK_PROF 
can be used instead of getrusage(), its precision can be read using 
clock_getres(). Something like "> #if defined(CLOCK_PROF) && 
defined(__FreeBSD__)" can be used. (By the way, "#if defined(__linux__)" may be 
enough)

I read somewhere than CLOCK_PROF is the user+system CPU time of the *current 
thread* on OpenBSD. I can be checked by the following unit test:

def test_process_time_threads(self):
class BusyThread(threading.Thread):
def run(self):
timeout = monotonic() + 1.0
loops = 1
while monotonic() < timeout:
i = 0
while i < loops:
i += 1
loops *= 2

t1 = process_time()
thread = BusyThread()
thread.start()
thread.join()
t2 = process_time()
self.assertGreater(t2 - t1, 0.9)

--

perf_counter() should remember if win32_perf_counter() failed or not, as the 
pseudo-code of the PEP. I prefer to leave clock() unchanged to keep backward 
compatibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

The precision of mach_absolute_time() is known: it is timebase.numer / 
timebase.denom * 1e-9.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25209/pep418-8.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25201/pep418-7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25126/pep418-6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Patch version 9: fixes for Windows (fix compilation and fix to code checking if 
GetTickCount64 is present).

--
Added file: http://bugs.python.org/file25210/pep418-9.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Patch version 8: time.process_time() uses times() if available. Rename also 
"function" key of time.get_clock_info() with "implementation".

--
Added file: http://bugs.python.org/file25209/pep418-8.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
> perf_counter_process_time.patch: replace "time.clock if windows else 
> time.time" with time.perf_counter, and getrusage/clock with time.process_time.
> 
> pybench and timeit now use time.perf_counter() by default. profile uses 
> time.proces_time() by default.
> 
> pybench uses time.get_clock_info() to display the precision and the 
> underlying C function (or the resolution if the precision is not available).
> 
> Tools/pybench/systimes.py and Tools/pybench/clockres.py may be removed: these 
> features are now available directly in the time module.

No changes to the pybench defaults, please. It has to stay backwards
compatible with older releases. Adding optional new timers is fine,
though.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com


2012-04-28: PythonCamp 2012, Cologne, Germany  15 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! 

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

perf_counter_process_time.patch: replace "time.clock if windows else time.time" 
with time.perf_counter, and getrusage/clock with time.process_time.

pybench and timeit now use time.perf_counter() by default. profile uses 
time.proces_time() by default.

pybench uses time.get_clock_info() to display the precision and the underlying 
C function (or the resolution if the precision is not available).

Tools/pybench/systimes.py and Tools/pybench/clockres.py may be removed: these 
features are now available directly in the time module.

--
Added file: http://bugs.python.org/file25202/perf_counter_process_time.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Patch version 7:
 - Add time.perf_counter() and time.process_time()
 - Replace "accuracy" key with "precision" in time.get_clock_info() result

--
Added file: http://bugs.python.org/file25201/pep418-7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

+if (has_getickcount64) {
+ULONGLONG ticks;
+ticks = Py_GetTickCount64();
+result = (double)ticks * 1e-3
+}

";" is missing after "1e-3", it does not compile on Windows because of this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[R. David Murray]

Changes by R. David Murray :


--
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25108/pep418-4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25115/pep418-5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Version 6: simplify the code handling GetTickCounter() integer overflow. We 
don't need a has_last_ticks variable.

--
Added file: http://bugs.python.org/file25126/pep418-6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

+#if defined(linux) || defined(__linux) || defined(__linux__)

Hum, a better check should be done in configure and/or a macro like MS_WINDOWS 
should be added.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Patch version 5, updated to the last version of the PEP:
 - drop time.highres()
 - time.monotonic() is always monotonic but it not always available
 - add a test on time.monotonic() setting the system clock (jump backward wtih 
a delta of 1 hour)

--
Added file: http://bugs.python.org/file25115/pep418-5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

> BTW: Are aware of the existing systimes.py module in pybench,
> which already provides interfaces to high resolution timers usable
> for benchmarking in a portable way ? Perhaps worth mentioning in
> the PEP.

Nope, I didn't know it. It mentioned it in the PEP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25103/pep418-3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25101/pep418-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file25053/pep418.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Patch version 4:
 - Rename time.monotonic() to time.steady()
 - Don't use CLOCK_MONOTONIC_RAW but CLOCK_MONOTONIC for time.highres() and 
time.steady()
 - Use CLOCK_HIGHRES, useful on Solaris
 - Rewrite time.highres() and time.steady() documentation

--
Added file: http://bugs.python.org/file25108/pep418-4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
>> I think you need to reconsider the time.steady() name you're using
>> in the PEP. For practical purposes, it's better to call it
>> time.monotonic()
> 
> I opened a new thread on python-dev to discuss this topic.
> 
>> and only make the function available if the OS provides
>> a monotonic clock.
> 
> Oh, I should explain this choice in the PEP. Basically, the idea is to
> provide a best-effort portable function.
> 
>> The fallback to time.time() is not a good idea, since then the programmer
>> has to check whether the timer really provides the features she's after
>> every time it gets used.
> 
> Nope, time.get_clock_info('steady') does not change at runtime. So it
> can only be checked once.

With "every time" I meant: in every application you use the function.
That pretty much spoils the idea of a best effort portable function.

It's better to use a try-except to test for availability of
functions than to have to (remember to) call a separate function
to find out the characteristics of the best effort approach.

>> Instead of trying to tweak all the different clocks and timers into
>> a single function, wouldn't it be better to expose each kind as a
>> different function and then let the programmer decide which fits
>> best ?!
> 
> This is a completly different approach. It should be discussed on
> python-dev, not in the bug tracker please. I think that Python can
> help the developer to write portable code by providing high-level
> functions because clock properties are well known (e.g. see
> time.get_clock_info).

Fair enough.

BTW: Are aware of the existing systimes.py module in pybench,
which already provides interfaces to high resolution timers usable
for benchmarking in a portable way ? Perhaps worth mentioning in
the PEP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

> I think you need to reconsider the time.steady() name you're using
> in the PEP. For practical purposes, it's better to call it
> time.monotonic()

I opened a new thread on python-dev to discuss this topic.

> and only make the function available if the OS provides
> a monotonic clock.

Oh, I should explain this choice in the PEP. Basically, the idea is to
provide a best-effort portable function.

> The fallback to time.time() is not a good idea, since then the programmer
> has to check whether the timer really provides the features she's after
> every time it gets used.

Nope, time.get_clock_info('steady') does not change at runtime. So it
can only be checked once.

> Instead of trying to tweak all the different clocks and timers into
> a single function, wouldn't it be better to expose each kind as a
> different function and then let the programmer decide which fits
> best ?!

This is a completly different approach. It should be discussed on
python-dev, not in the bug tracker please. I think that Python can
help the developer to write portable code by providing high-level
functions because clock properties are well known (e.g. see
time.get_clock_info).

> BTW: Thanks for the research you've done on the different clocks and
> timers. That's very useful information.

You're welcome.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

Hi Victor,

I think you need to reconsider the time.steady() name you're using
in the PEP. For practical purposes, it's better to call it
time.monotonic() and only make the function available if the OS provides
a monotonic clock.

The fallback to time.time() is not a good idea, since then the programmer
has to check whether the timer really provides the features she's after
every time it gets used.

Regardless of this functional problem, I'm also not sure what you want
to imply by the term "steady". A steady beat would mean that the timer
never stops and keeps a constant pace, but that's not the case for
the timers you're using to implement time.steady(). If you're after
a mathematical term, "continuous" would be a better term, but
again, time.time() is not always continuous.

Instead of trying to tweak all the different clocks and timers into
a single function, wouldn't it be better to expose each kind as a
different function and then let the programmer decide which fits
best ?!

BTW: Thanks for the research you've done on the different clocks and
timers. That's very useful information.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com


2012-04-03: Python Meeting Duesseldorf today

::: Try our new mxODBC.Connect Python Database Interface for free ! 

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/

--
nosy: +lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

TODO (pep418-3.patch):
 - Use CLOCK_HIGHRES on Solaris
 - Maybe implement gethrtime() for Solaris

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Solaris on x86_64:

>>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 1e-06,
 'function': 'clock()',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 1e-06}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 2e-09,
 'function': 'clock_gettime(CLOCK_HIGHRES)',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 2e-09,
 'function': 'clock_gettime(CLOCK_HIGHRES)',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 0.01,
 'function': 'clock_gettime(CLOCK_REALTIME)',
 'is_adjusted': True,
 'is_monotonic': False,
 'resolution': 1e-09}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Patch version 3:

 - fix compilation on non-Linux (e.g. on FreeBSD and OpenBSD)
 - time.monotonic() uses GetTickCount/GetTickCount64 on Windows
 - clock_getres() fills the accuracy field instead of the resolution field of 
time.get_clock_info()

Clock information on different operating systems.

Linux (Fedora 16) on x86_64:

>>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 1e-06,
 'function': 'clock()',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 1e-06}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 1e-09,
 'function': 'clock_gettime(CLOCK_MONOTONIC_RAW)',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 1e-09,
 'function': 'clock_gettime(CLOCK_MONOTONIC_RAW)',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 1e-09,
 'function': 'clock_gettime(CLOCK_REALTIME)',
 'is_adjusted': True,
 'is_monotonic': False,
 'resolution': 1e-09}

FreeBSD 8.2 on x86_64:

>>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 0.0078125,
 'function': 'clock()',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 0.0078125}
>>> pprint.pprint(time.get_clock_info('highres')) 
{'accuracy': 1.1001e-08,
 'function': 'clock_gettime(CLOCK_MONOTONIC)',
 'is_monotonic': True,
 'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 1.1001e-08,
 'function': 'clock_gettime(CLOCK_MONOTONIC)',
 'is_monotonic': True,
 'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 1.1001e-08,
 'function': 'clock_gettime(CLOCK_REALTIME)',
 'is_adjusted': True,
 'is_monotonic': False,
 'resolution': 1e-09}

OpenBSD on x86_64:

>>> pprint.pprint(time.get_clock_info('clock'))  
{'accuracy': 0.01,
 'function': 'clock()',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 0.01}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 0.01,
 'function': 'clock_gettime(CLOCK_MONOTONIC)',
 'is_monotonic': True,
 'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 0.01,
 'function': 'clock_gettime(CLOCK_MONOTONIC)',
 'is_monotonic': True,
 'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 0.01,
 'function': 'clock_gettime(CLOCK_REALTIME)',
 'is_adjusted': True,
 'is_monotonic': False,
 'resolution': 1e-09}

Python 32 bits on Windows 64 bits:

>>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 1e-08,
 'function': 'QueryPerformanceCounter()',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 1e-08}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 1e-08,
 'function': 'QueryPerformanceCounter()',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 1e-08}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 0.0156000999,
 'function': 'GetTickCount()',
 'is_adjusted': False,
 'is_monotonic': True,
 'resolution': 0.001}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 0.0156000999,
 'function': 'GetSystemTimeAsFileTime()',
 'is_adjusted': True,
 'is_monotonic': False,
 'resolution': 1e-07}

Some remarks:

 - these 4 platforms provide a monotonic clock
 - these 4 platforms provide the accurary of the 4 clocks (it looks like Linux 
is cheating)
 - on OpenBSD, time.highres() resolution is not so "high". It has only an 
accuracy of 10 ms.
 - as expected, the accuracy is very different from the resolution in some 
cases (ex: 0.0156 vs 1e-07 for time.time() on Windows)

--
Added file: http://bugs.python.org/file25103/pep418-3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

STINNER Victor  added the comment:

Updated patch:
 - Implement time.get_clock_info()
 - time.monotonic() never fails: fallback to time.time() if needed

TODO: time.monotonic() must use GetTickCount64() or GetTickCount() on Windows, 
with a detection of integer overflow on GetTickCount().

--
Added file: http://bugs.python.org/file25101/pep418-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[Charles-François Natali]

Charles-François Natali  added the comment:

Tiny review.

Two more details:
- since it's relatively straightforward to cache the last value returned using 
a static variable, it might be interesting to use this to make sure that the 
values returned are indeed monotonic
- I'm not a native speaker, but `hires` name bothers me a bit, I'd prefer 
`highres` (see for example 
http://www.kernel.org/doc/Documentation/timers/highres.txt, the highres kernel 
parameter and patch-set name), or maybe `hrtime`

--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14428] Implementation of the PEP 418

[STINNER Victor]

New submission from STINNER Victor :

Attached patch implements the PEP 418: add time.monotonic() and time.hires(). 
In practice, it replaces time.steady() by time.hires() and 
time.steady(strict=True) by time.monotonic().

--
components: Library (Lib)
files: pep418.patch
keywords: patch
messages: 156960
nosy: haypo
priority: normal
severity: normal
status: open
title: Implementation of the PEP 418
versions: Python 3.3
Added file: http://bugs.python.org/file25053/pep418.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com