Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On 03/04/2012 07:03, Cameron Simpson wrote: On 03Apr2012 07:51, Lennart Regebro wrote: | I like the aim of letting the user control what clock it get, but I | find this API pretty horrible: | |>clock = get_clock(T_MONOTONIC|T_HIRES) or get_clock(T_MONOTONIC) FWIW, the leading "T_" is now gone, so it would now read: clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) If the symbol names are not the horribleness, can you qualify what API you would like more? I reckon the API is ok given that you don't have to supply the flags, correct? A small point but I'm with (I think) Terry Reedy and Steven D'Aprano in that hires is an English word, could you please substitute highres and HIGHRES, thanks. -- Cheers. Mark Lawrence. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On 03Apr2012 09:03, Mark Lawrence wrote: | On 03/04/2012 07:03, Cameron Simpson wrote: | > On 03Apr2012 07:51, Lennart Regebro wrote: | > | I like the aim of letting the user control what clock it get, but I | > | find this API pretty horrible: | > | | > |>clock = get_clock(T_MONOTONIC|T_HIRES) or get_clock(T_MONOTONIC) | > | > FWIW, the leading "T_" is now gone, so it would now read: | > | >clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) | > | > If the symbol names are not the horribleness, can you qualify what API | > you would like more? | | I reckon the API is ok given that you don't have to supply the flags, | correct? That's right. And if the monotonic() or monotonic_clock() functions (or the hires* versions if suitable) do what you want you don't even need that. You only need the "or" style to choose your own fallback according to your own criteria. | A small point but I'm with (I think) Terry Reedy and Steven D'Aprano in | that hires is an English word, could you please substitute highres and | HIGHRES, thanks. I have the same issue and would be happy to do it. Victor et al, how do you feel about this? People have been saying "hires" throughout the threads I think, but I for one would be slightly happier with "highres". Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ I bested him in an Open Season of scouring-people's-postings-looking-for- spelling-errors.- [email protected] (Kevin Darcy) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?
> -Original Message- > From: [email protected] [mailto:[email protected]] On Behalf > Of Guido van Rossum > Sent: 2. apríl 2012 17:43 > To: Kristján Valur Jónsson > Cc: Cameron Simpson; Python Dev > Subject: Re: [Python-Dev] Use QueryPerformanceCounter() for > time.monotonic() and/or time.highres()? > > You seem to have missed the episode where I explained that caching the last > value in order to avoid going backwards doesn't work -- at least not if the > cached value is internal to the API implementation. > Yes, and I can't find it by briefly searching my mail. I haven't had the energy to follow every bit of this discussion because it has become completely insane. Of course we cannot promise not moving backwards, since there is a 64 bit wraparound some years in the future. Otherwise, evidence contradicts your claim. Here's actual code from production: BOOL WINAPI QueryPerformanceCounterCCP( LARGE_INTEGER* li ) { static LARGE_INTEGER last = {0}; BOOL ok = QueryPerformanceCounter(li); if( !ok ) { return FALSE; } if( li->QuadPart > last.QuadPart ) { last = *li; } else { *li = last; } return TRUE; } This has been running for many years on an incredible array of hardware and operating systems. However, we mostly don't do this caching anymore, this code is a rudiment. In all other places, a straight QPC is good enough for our purposes. Even negative delta values of time are usually harmless on the application level. A curiosity, but harmless. I am offering empirical evidence here from hundreds of thousands of computers over six years: For timing and benchmarking, QPC is good enough, and will only be as precise as the hardware and operating system permits, which in practice is good enough. Which is why am flabbergasted by all of this bikeshedding. My original submission (http://bugs.python.org/issue10278) Is merely a suggestion to provide a standardised clock function, useful for measuring the delta-t to the best abilities of the platform. This is incredibly useful in many areas and necessary because time.clock() currently means different things on different operating systems. There is no need to try to overspecify this to become something which it never can be. If someone wants a real time clock with no time slew to control a radio telescope, he better write his own interface to an atomic clock. What he definitely shouldn't be doing is using a built in timer on an old computer with an old operating system. Also, since you objected to the original suggestion of time.wallclock(), here is the definition from Wikipedia: http://en.wikipedia.org/wiki/Wall_clock_time . I actually never read this before, but it agrees with my original definition of relative passage of time in the real world. I got the term myself from using profiling tools, which measure program execution in cpu time or wallclock time. Cheers, Kristján ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
Hi,
I would to rename time.monotonic() to time.steady() in the PEP 418 for
the following reasons:
- time.steady() may fallback to the system clock which is not
monotonic, it's strange to have to check for
time.get_clock_info('monotonic')['is_monotonic']
- time.steady() uses GetTickCount() instead of
QueryPerformanceCounter() whereas both are monotonic, but
QueryPerformanceCounter() is not steady
Python steady clock will be different than the C++ definition.
You may argue that time.steady() is not always steady: it may fallback
to the system clock which is adjusted by NTP and can jump
backward/forward with a delta greater than 1 hour. In practice, there
is only one operating system that does not provide a monotonic clock:
GNU/Hurd.
I hesitate to add "is_steady" to time.get_clock_info(), but a boolean
is not very useful, it would be better to have a number.
Arguments for time.monotonic() name:
- Users are looking for the "monotonic" name
- Most of the time, time.monotonic() is a monotonic clock
--
On Linux, we might use CLOCK_MONOTONIC for time.steady() and
CLOCK_MONOTONIC_RAW for time.highres(). The NTP daemon Linux on Linux
uses a reliable clock to adjust CLOCK_MONOTONIC frequency and so
CLOCK_MONOTONIC is steady and it may go backward in a short period,
whereas CLOCK_MONOTONIC_RAW cannot go backward and so may fit closer
time.highres() requirements.
Currently, CLOCK_MONOTONIC is used for time.highres() and
time.steady() in the PEP.
--
NTP on Linux should only slew CLOCK_MONOTONIC, not step it. But it
looks like there was a bug in the Linux kernel 2.6.31: CLOCK_MONOTONIC
goes backward sometimes. Bug introduced in 2.6.31 by (August 14,
2009):
https://github.com/torvalds/linux/commit/0a54419836254a27baecd9037103171bcbabaf67
and fixed in the kernel 2.6.32 by (November 16, 2009):
https://github.com/torvalds/linux/commit/0696b711e4be45fa104c12329f617beb29c03f78
Someone had the bug:
http://stackoverflow.com/questions/3657289/linux-clock-gettimeclock-monotonic-strange-non-monotonic-behavior
Victor
PS: I already changed time.monotonic() to time.steady() in the PEP :-p
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?
>> You seem to have missed the episode where I explained that caching the last
>> value in order to avoid going backwards doesn't work -- at least not if the
>> cached value is internal to the API implementation.
>>
> Yes, and I can't find it by briefly searching my mail. I haven't had the
> energy to follow every bit of this discussion because it has become
> completely insane.
I'm trying to complete the PEP, but I didn't add this part yet.
> Of course we cannot promise not moving backwards, since there is a 64 bit
> wraparound some years in the future.
Some years? I computed 584.5 years, so it should not occur in
practice. 32-bit wraparound is a common issue which occurs in practice
on Windows (49.7 days wraparound), and I propose a workaround in the
PEP (already implemented in the related issue).
> Here's actual code from production:
>
> BOOL WINAPI QueryPerformanceCounterCCP( LARGE_INTEGER* li )
> {
> static LARGE_INTEGER last = {0};
> BOOL ok = QueryPerformanceCounter(li);
> if( !ok )
> {
> return FALSE;
> }
Did you already see it failing in practice? Python ignores the return
value and only uses the counter value.
> Even negative delta values of time are usually harmless on the application
> level.
> A curiosity, but harmless.
It depends on your usecase. For a scheduler or to implement a timeout,
it does matter. For a benchmark, it's not an issue because you usually
repeat a test at least 3 times. Most advanced benchmarked tools gives
a confidence factor to check if the benchmark ran fine or not.
> I am offering empirical evidence here from hundreds of thousands of computers
> over six years: For timing and benchmarking, QPC is good enough, and will only
> be as precise as the hardware and operating system permits, which in practice
> is good enough.
The PEP contains also different proofs that QPC is not steady,
especially on virtual machines.
Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?
Victor Stinner wrote:
>>> You seem to have missed the episode where I explained that caching the last
>>> value in order to avoid going backwards doesn't work -- at least not if the
>>> cached value is internal to the API implementation.
>>>
>> Yes, and I can't find it by briefly searching my mail. I haven't had the
>> energy to follow every bit of this discussion because it has become
>> completely insane.
>
> I'm trying to complete the PEP, but I didn't add this part yet.
>
>> Of course we cannot promise not moving backwards, since there is a 64 bit
>> wraparound some years in the future.
>
> Some years? I computed 584.5 years, so it should not occur in
> practice. 32-bit wraparound is a common issue which occurs in practice
> on Windows (49.7 days wraparound), and I propose a workaround in the
> PEP (already implemented in the related issue).
>
>> Here's actual code from production:
>>
>> BOOL WINAPI QueryPerformanceCounterCCP( LARGE_INTEGER* li )
>> {
>>static LARGE_INTEGER last = {0};
>>BOOL ok = QueryPerformanceCounter(li);
>>if( !ok )
>>{
>>return FALSE;
>>}
>
> Did you already see it failing in practice? Python ignores the return
> value and only uses the counter value.
>
>> Even negative delta values of time are usually harmless on the application
>> level.
>> A curiosity, but harmless.
>
> It depends on your usecase. For a scheduler or to implement a timeout,
> it does matter. For a benchmark, it's not an issue because you usually
> repeat a test at least 3 times. Most advanced benchmarked tools gives
> a confidence factor to check if the benchmark ran fine or not.
>
>> I am offering empirical evidence here from hundreds of thousands of
>> computers
>> over six years: For timing and benchmarking, QPC is good enough, and will
>> only
>> be as precise as the hardware and operating system permits, which in practice
>> is good enough.
>
> The PEP contains also different proofs that QPC is not steady,
> especially on virtual machines.
I'm not sure I understand what you are after here, Victor. For benchmarks
it really doesn't matter if one or two runs fail due to the timer having
a problem: you just repeat the run and ignore the false results (you
have such issues in all empirical studies). You're making things
needlessly complicated here.
Regarding the approach to try to cover all timing requirements into
a single time.steady() API, I'm not convinced that this is good
approach. Different applications have different needs, so it's
better to provide interfaces to what the OS has to offer and
let the application decide what's best.
If an application wants to have a monotonic clock, it should use
time.monotonic(). The OS doesn't provide it, you get an AttributeError
and revert to some other function, depending on your needs.
Having a time.steady() API make this decision for you, is not
going to make your application more portable, since the choice
will inevitably be wrong in some cases (e.g. going from
CLOCK_MONOTONIC to time.time() as fallback).
BTW: You might also want to take a look at the systimes.py module
in pybench. We've been through discussions related to
benchmark timing in 2006 already and that module summarizes
the best practice outcome :-)
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Apr 03 2012)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
2012-04-03: Python Meeting Duesseldorf today
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On Tue, Apr 3, 2012 at 08:03, Cameron Simpson wrote: > clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) > > If the symbol names are not the horribleness, can you qualify what API > you would like more? Well, get_clock(monotonic=True, highres=True) would be a vast improvement over get_clock(MONOTONIC|HIRES). I also think it should raise an error if not found. The clarity and easy of use of the API is much more important than how much you can do in one line. //Lennart ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
On Tue, Apr 3, 2012 at 13:26, Victor Stinner wrote:
> Hi,
>
> I would to rename time.monotonic() to time.steady() in the PEP 418 for
> the following reasons:
>
> - time.steady() may fallback to the system clock which is not
> monotonic, it's strange to have to check for
> time.get_clock_info('monotonic')['is_monotonic']
> - time.steady() uses GetTickCount() instead of
> QueryPerformanceCounter() whereas both are monotonic, but
> QueryPerformanceCounter() is not steady
Wait, what?
I already thought we, several days ago, decided that "steady" was a
*terrible* name, and that monotonic should *not* fall back to the
system clock.
It seems that we are going in circles with this. Now we are back to
where we started. Now we have a time.steady() which may not be steady
and a time.highres() which may not be high resolution.
//Lennart
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
> -Original Message- > From: [email protected] > [mailto:[email protected]] On > Behalf Of Lennart Regebro > Sent: 3. apríl 2012 14:14 > To: Victor Stinner > Cc: Python Dev > Subject: Re: [Python-Dev] PEP 418: rename time.monotonic() to > time.steady()? > > On Tue, Apr 3, 2012 at 13:26, Victor Stinner > wrote: > > Hi, > > > > I would to rename time.monotonic() to time.steady() in the PEP 418 for > > the following reasons: > > > > - time.steady() may fallback to the system clock which is not > > monotonic, it's strange to have to check for > > time.get_clock_info('monotonic')['is_monotonic'] > > - time.steady() uses GetTickCount() instead of > > QueryPerformanceCounter() whereas both are monotonic, but > > QueryPerformanceCounter() is not steady > > Wait, what? > I already thought we, several days ago, decided that "steady" was a > *terrible* name, and that monotonic should *not* fall back to the system > clock. > > It seems that we are going in circles with this. Now we are back to where we > started. Now we have a time.steady() which may not be steady and a > time.highres() which may not be high resolution. There is no such thing as steady time. I think we are trying to solve a non-existing problem here. K ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
The discussion has completed degenerated. There are several different clocks here, and several different agendas. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
On Tue, 03 Apr 2012 22:42:37 +0800, Matt Joiner wrote: > The discussion has completed degenerated. There are several different > clocks here, and several different agendas. It's probably time to do a reset. Read Victor's PEP, and help him edit it so that it accurately reflects the various arguments. Then we can bikeshed some more based on the language in the PEP :) --David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use QueryPerformanceCounter() for time.monotonic() and/or time.highres()?
> -Original Message- > From: [email protected] > Some years? I computed 584.5 years, so it should not occur in practice. Funny that you mention it. "should not occur in practice" is exactly my point. > > Here's actual code from production: > > > > BOOL WINAPI QueryPerformanceCounterCCP( LARGE_INTEGER* li ) { > > static LARGE_INTEGER last = {0}; > > BOOL ok = QueryPerformanceCounter(li); > > if( !ok ) > > { > > return FALSE; > > } > > Did you already see it failing in practice? Python ignores the return value > and > only uses the counter value. No, actually not. But we always check return codes. Always. > > > Even negative delta values of time are usually harmless on the application > level. > > A curiosity, but harmless. > > It depends on your usecase. For a scheduler or to implement a timeout, it > does matter. Does it? now = time.wallclock() if job.due_time <= now: job.do_it() So what if you get an early timeout? timeouts aren't guaranteed to wait _at least_ the specified time. Rather to wait _at most_ the specified time. > > The PEP contains also different proofs that QPC is not steady, especially on > virtual machines. What does "steady" mean? Sampled time on a computer will always differ from some ideal time measured by an atomic clock. On a virtual machine, the probability distribution of the error function can be different than on a "real" machine, but saying it is different is not enough. You have to quantify it somehow. And unless there is some "accepted" error PDF, there is no way to say that some platforms are ok, and others not. And what does it matter? A virtual machine is just another platform, one where we are providing a counter as good as that platform can provide. "Caveat emptor: Don't expect reliable benchmarks or smoothly running time on a virtual machine. The wallclock() function will contain some undetermined error depending on the quality of your platform." I think you are simply overcomplicating the problem and trying to promise too much, without even being able to properly quantify that promise. Just relax the specification and all will be well. K ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
Lennart Regebro wrote: On Tue, Apr 3, 2012 at 08:03, Cameron Simpson wrote: clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) If the symbol names are not the horribleness, can you qualify what API you would like more? Well, get_clock(monotonic=True, highres=True) would be a vast improvement over get_clock(MONOTONIC|HIRES). Allowing get_clock(True, True)? Ick. My nomination would be get_clock(MONOTONIC, HIGHRES) -- easier on the eyes with no |. I also think it should raise an error if not found. The clarity and easy of use of the API is much more important than how much you can do in one line. What's unclear about returning None if no clocks match? Cheers, ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] .{git,bzr}ignore in cpython HG repo
On Apr 03, 2012, at 12:44 AM, Antoine Pitrou wrote: >I don't think Barry still uses bzr, and who ever used git to manage their >patches against the CPython repo? I still use bzr, but not currently for Python development. I just use the standard hg repo. I'd like to go back to it though once the bzr-hg plugin can handle multiple branches in a single repo. -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Remove of w9xopen
I filed the issue http://bugs.python.org/issue14470 for removing w9xopen from subprocess as python 3.3 has declaration about finishing support of Windows 2000 and Win9x family. But, as I see, VC project for building w9xopen is still present. Should we remove it as well? -- Thanks, Andrew Svetlov ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove of w9xopen
On Tue, Apr 3, 2012 at 14:59, Andrew Svetlov wrote: > I filed the issue http://bugs.python.org/issue14470 for removing > w9xopen from subprocess as python 3.3 has declaration about finishing > support of Windows 2000 and Win9x family. > But, as I see, VC project for building w9xopen is still present. > Should we remove it as well? Please leave it in for the time being. Feel free to assign the issue to me and I'll take care of it once we've officially transitioned to VS2010. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove of w9xopen
Done. Thanks. On Tue, Apr 3, 2012 at 11:08 PM, Brian Curtin wrote: > On Tue, Apr 3, 2012 at 14:59, Andrew Svetlov wrote: >> I filed the issue http://bugs.python.org/issue14470 for removing >> w9xopen from subprocess as python 3.3 has declaration about finishing >> support of Windows 2000 and Win9x family. >> But, as I see, VC project for building w9xopen is still present. >> Should we remove it as well? > > Please leave it in for the time being. Feel free to assign the issue > to me and I'll take care of it once we've officially transitioned to > VS2010. -- Thanks, Andrew Svetlov ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
> Wait, what? > I already thought we, several days ago, decided that "steady" was a > *terrible* name, and that monotonic should *not* fall back to the > system clock. Copy of a more recent Guido's email: http://mail.python.org/pipermail/python-dev/2012-March/118322.html "Anyway, the more I think about it, the more I believe these functions should have very loose guarantees, and instead just cater to common use cases -- availability of a timer with minimal fuss is usually more important than the guarantees. So forget the idea about one version that falls back to time.time() and another that doesn't -- just always fall back to time.time(), which is (almost) always better than failing. Then we can design a separate inquiry API (doesn't have to be complex as long as it's extensible -- a dict or object with a few predefined keys or attributes sounds good enough) for apps that want to know more about how the timer they're using is actually implemented." I added time.get_clock_info() so the user can check if the clock is monotonic or not. Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On 03Apr2012 09:07, Ethan Furman wrote: | Lennart Regebro wrote: | > On Tue, Apr 3, 2012 at 08:03, Cameron Simpson wrote: | >> clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) | >> | >> If the symbol names are not the horribleness, can you qualify what API | >> you would like more? | > | > Well, get_clock(monotonic=True, highres=True) would be a vast | > improvement over get_clock(MONOTONIC|HIRES). | | Allowing get_clock(True, True)? Ick. My nomination would be | get_clock(MONOTONIC, HIGHRES) -- easier on the eyes with no |. get_clock already has two arguments - you can optionally hand it a clock list - that's used by monotonic_clock() and hires_clock(). Have a quick glance at: https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/clockutils.py (I finally found out how to point at the latest revision on BitBucket; it's not obvious from the web interface itself.) | > I also think it should | > raise an error if not found. The clarity and easy of use of the API is | > much more important than how much you can do in one line. How much you can do _clearly_ in one line is a useful metric. | What's unclear about returning None if no clocks match? The return of None is very deliberate. I _want_ user specified fallback to be concise and easy. The example: clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) seems to satisfy both these criteria to my eye. Raising an exception makes user fallback a royal PITA, with a horrible try/except cascade needed. Exceptions are all very well when there is just one thing to do: parse this or fail, divide this by that or fail. If fact they're the very image of "do this one thing or FAIL". They are not such a good match for do this thing or that thing or this other thing. When you want a simple linear cascade of choices, Python's short circuiting "or" operator is a very useful thing. Having an obsession with exceptions is IMO unhealthy. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Because of its special customs, crossposting between alt.peeves and normal newsgroups is discouraged. - Cameron Spitzer ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 14417: consequences of new dict runtime error
On Sat, Mar 31, 2012 at 7:45 PM, R. David Murray wrote: > On Sun, 01 Apr 2012 03:03:13 +1000, Nick Coghlan > wrote: > > On Sun, Apr 1, 2012 at 2:09 AM, Guido van Rossum > wrote: > > > Here's a different puzzle. Has anyone written a demo yet that provokes > > > this RuntimeError, without cheating? (Cheating would be to mutate the > > > dict from *inside* the __eq__ or __hash__ method.) If you're serious > > > about revisiting this, I'd like to see at least one example of a > > > program that is broken by the change. Otherwise I think the status quo > > > in the 3.3 repo should prevail -- I don't want to be stymied by > > > superstition. > > > > I attached an attempt to *deliberately* break the new behaviour to the > > tracker issue. It isn't actually breaking for me, so I'd like other > > folks to look at it to see if I missed something in my implementation, > > of if it's just genuinely that hard to induce the necessary bad timing > > of a preemptive thread switch. > > Thanks, Nick. It looks reasonable to me, but I've only given it a quick > look so far (I'll try to think about it more deeply later today). > > If it is indeed hard to provoke, then I'm fine with leaving the > RuntimeError as a signal that the application needs to add some locking. > My concern was that we'd have working production code that would start > breaking. If it takes a *lot* of threads or a *lot* of mutation to > trigger it, then it is going to be a lot less likely to happen anyway, > since such programs are going to be much more careful about locking > anyway. > > --David > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com > Hm I might be missing something, but if you have multiple threads accessing a dict, already this program: http://paste.pocoo.org/show/575776/ raises RuntimeError. You'll get slightly more obscure cases than changing a size raise RuntimeError during iteration under PyPy. As far as I understood, if you're mutating while iterating, you *can* get a runtime error. This does not even have a custom __eq__ or __hash__. Are you never iterating over dicts? Cheers, fijal ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
On 03Apr2012 13:26, Victor Stinner wrote:
| I would to rename time.monotonic() to time.steady() in the PEP 418 for
| the following reasons:
|
| - time.steady() may fallback to the system clock which is not
| monotonic, it's strange to have to check for
| time.get_clock_info('monotonic')['is_monotonic']
| - time.steady() uses GetTickCount() instead of
| QueryPerformanceCounter() whereas both are monotonic, but
| QueryPerformanceCounter() is not steady
|
| Python steady clock will be different than the C++ definition.
|
| You may argue that time.steady() is not always steady: it may fallback
| to the system clock which is adjusted by NTP and can jump
| backward/forward with a delta greater than 1 hour.
An HOUR ?!?!?
I have to say I'm -100 on any proposal where time.monotonic() returns
non-monotonic time and likewise for time.steady() returning unsteady
time.
| In practice, there
| is only one operating system that does not provide a monotonic clock:
| GNU/Hurd.
I'd have thought practically any early UNIX falls into this category.
And any number of other niche things. (Yes I know Python doesn't run on
everything anyway.) Are we only considering Linux/Mac/Windows, and
only recent versions of those?
What's the status of Java and Jython?
| I hesitate to add "is_steady" to time.get_clock_info(), but a boolean
| is not very useful, it would be better to have a number.
|
| Arguments for time.monotonic() name:
|
| - Users are looking for the "monotonic" name
| - Most of the time, time.monotonic() is a monotonic clock
Again, here, I'm -100 on "most". If I ask for monotonic, it is because I
need one. Given me monotonic or give me death! (Well, an exception or
at any rate something unusable like None.)
[...]
| PS: I already changed time.monotonic() to time.steady() in the PEP :-p
Sigh. They're different things! For all that "steady" is a slightly
vague term, steady and hires and monotonic are independent concepts. Of
course a lot of high quality clocks will embody hires and ideally steady
or monotonic.
This kind of offer-just-one-thing embedded policy is why I feel the API
needs more user control and a polciy free interface, with montonic() et
al providing handy prepackaged policy for the common uses.
If you can provide monotonic (for example, on Linux as you outline),
which _not_ offer it? Offering steady() provides no way for the user to
ask for higher guarentees.
Cheers,
--
Cameron Simpson DoD#743
http://www.cskk.ezoshosting.com/cs/
But in our enthusiasm, we could not resist a radical overhaul of the
system, in which all of its major weaknesses have been exposed, analyzed,
and replaced with new weaknesses. - Bruce Leverett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 14417: consequences of new dict runtime error
I'm confused. Are you saying that that program always raised RuntimeError, or that it started raising RuntimeError with the new behavior (3.3 alpha 2)? On Tue, Apr 3, 2012 at 2:47 PM, Maciej Fijalkowski wrote: > On Sat, Mar 31, 2012 at 7:45 PM, R. David Murray > wrote: >> >> On Sun, 01 Apr 2012 03:03:13 +1000, Nick Coghlan >> wrote: >> > On Sun, Apr 1, 2012 at 2:09 AM, Guido van Rossum >> > wrote: >> > > Here's a different puzzle. Has anyone written a demo yet that provokes >> > > this RuntimeError, without cheating? (Cheating would be to mutate the >> > > dict from *inside* the __eq__ or __hash__ method.) If you're serious >> > > about revisiting this, I'd like to see at least one example of a >> > > program that is broken by the change. Otherwise I think the status quo >> > > in the 3.3 repo should prevail -- I don't want to be stymied by >> > > superstition. >> > >> > I attached an attempt to *deliberately* break the new behaviour to the >> > tracker issue. It isn't actually breaking for me, so I'd like other >> > folks to look at it to see if I missed something in my implementation, >> > of if it's just genuinely that hard to induce the necessary bad timing >> > of a preemptive thread switch. >> >> Thanks, Nick. It looks reasonable to me, but I've only given it a quick >> look so far (I'll try to think about it more deeply later today). >> >> If it is indeed hard to provoke, then I'm fine with leaving the >> RuntimeError as a signal that the application needs to add some locking. >> My concern was that we'd have working production code that would start >> breaking. If it takes a *lot* of threads or a *lot* of mutation to >> trigger it, then it is going to be a lot less likely to happen anyway, >> since such programs are going to be much more careful about locking >> anyway. >> >> --David >> ___ >> Python-Dev mailing list >> [email protected] >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com > > > Hm > > I might be missing something, but if you have multiple threads accessing a > dict, already this program: http://paste.pocoo.org/show/575776/ raises > RuntimeError. You'll get slightly more obscure cases than changing a size > raise RuntimeError during iteration under PyPy. As far as I understood, if > you're mutating while iterating, you *can* get a runtime error. > > This does not even have a custom __eq__ or __hash__. Are you never iterating > over dicts? > > Cheers, > fijal -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
Cameron Simpson wrote: get_clock already has two arguments - you can optionally hand it a clock list - that's used by monotonic_clock() and hires_clock(). def get_clock(*flags, *, clocklist=None): ''' Return a Clock based on the supplied `flags`. The returned clock shall have all the requested flags. If no clock matches, return None. ''' wanted = 0 for flag in flags: wanted |= flag if clocklist is None: clocklist = ALL_CLOCKS for clock in clocklist: if clock.flags & wanted == wanted: return clock.factory() return None Would need to make *flags change to the other *_clock functions. Have a quick glance at: https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/clockutils.py Thanks. The return of None is very deliberate. I _want_ user specified fallback to be concise and easy. The example: clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) Which would become: clock = get_clock(MONOTONIC, HIGHRES) or get_clock(MONOTONIC) +1 to returning None Exceptions are all very well when there is just one thing to do: parse this or fail, divide this by that or fail. If fact they're the very image of "do this one thing or FAIL". They are not such a good match for do this thing or that thing or this other thing. When you want a simple linear cascade of choices, Python's short circuiting "or" operator is a very useful thing. Having an obsession with exceptions is IMO unhealthy. Another +1. ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 14417: consequences of new dict runtime error
Never mind, I got it. This always raised RuntimeError. I see this should be considered support in favor of keeping the change, since sharing dicts between threads without locking is already fraught with RuntimeErrors. At the same time, has anyone looked at my small patch (added to the issue) that restores the retry loop without recursion? On Tue, Apr 3, 2012 at 3:17 PM, Guido van Rossum wrote: > I'm confused. Are you saying that that program always raised > RuntimeError, or that it started raising RuntimeError with the new > behavior (3.3 alpha 2)? > > On Tue, Apr 3, 2012 at 2:47 PM, Maciej Fijalkowski wrote: >> On Sat, Mar 31, 2012 at 7:45 PM, R. David Murray >> wrote: >>> >>> On Sun, 01 Apr 2012 03:03:13 +1000, Nick Coghlan >>> wrote: >>> > On Sun, Apr 1, 2012 at 2:09 AM, Guido van Rossum >>> > wrote: >>> > > Here's a different puzzle. Has anyone written a demo yet that provokes >>> > > this RuntimeError, without cheating? (Cheating would be to mutate the >>> > > dict from *inside* the __eq__ or __hash__ method.) If you're serious >>> > > about revisiting this, I'd like to see at least one example of a >>> > > program that is broken by the change. Otherwise I think the status quo >>> > > in the 3.3 repo should prevail -- I don't want to be stymied by >>> > > superstition. >>> > >>> > I attached an attempt to *deliberately* break the new behaviour to the >>> > tracker issue. It isn't actually breaking for me, so I'd like other >>> > folks to look at it to see if I missed something in my implementation, >>> > of if it's just genuinely that hard to induce the necessary bad timing >>> > of a preemptive thread switch. >>> >>> Thanks, Nick. It looks reasonable to me, but I've only given it a quick >>> look so far (I'll try to think about it more deeply later today). >>> >>> If it is indeed hard to provoke, then I'm fine with leaving the >>> RuntimeError as a signal that the application needs to add some locking. >>> My concern was that we'd have working production code that would start >>> breaking. If it takes a *lot* of threads or a *lot* of mutation to >>> trigger it, then it is going to be a lot less likely to happen anyway, >>> since such programs are going to be much more careful about locking >>> anyway. >>> >>> --David >>> ___ >>> Python-Dev mailing list >>> [email protected] >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com >> >> >> Hm >> >> I might be missing something, but if you have multiple threads accessing a >> dict, already this program: http://paste.pocoo.org/show/575776/ raises >> RuntimeError. You'll get slightly more obscure cases than changing a size >> raise RuntimeError during iteration under PyPy. As far as I understood, if >> you're mutating while iterating, you *can* get a runtime error. >> >> This does not even have a custom __eq__ or __hash__. Are you never iterating >> over dicts? >> >> Cheers, >> fijal > > > > -- > --Guido van Rossum (python.org/~guido) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
Cameron Simpson wrote: Sigh. They're different things! For all that "steady" is a slightly vague term, steady and hires and monotonic are independent concepts. Of course a lot of high quality clocks will embody hires and ideally steady or monotonic. This kind of offer-just-one-thing embedded policy is why I feel the API needs more user control and a polciy free interface, with montonic() et al providing handy prepackaged policy for the common uses. +1 ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Issue 11734: Add half-float (16-bit) support to struct module
Hello, I worked on a patch to support half-floats about a year ago, and the impression I got from the python-dev list was that there wasn't anyone with objections to the patch, and from the reviewers was that it was ready for inclusion, but it never moved beyond that stage (I should have pushed it harder, I suspect). Is there still time to get it in the 3.3 cycle? The corresponding patch for NumPy has been accepted, and IIRC, is present in the 1.6 release. http://bugs.python.org/issue11734 The issue has links to the various discussions surrounding the patch for context. What should happen next? Thanks, Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
[ Returning at more leisure... ]
On 04Apr2012 07:53, I wrote:
| On 03Apr2012 13:26, Victor Stinner wrote:
| | I would to rename time.monotonic() to time.steady() in the PEP 418 for
| | the following reasons:
| | - time.steady() may fallback to the system clock which is not
| | monotonic, it's strange to have to check for
| | time.get_clock_info('monotonic')['is_monotonic']
This I agree with. You should never need to do that.
| | - time.steady() uses GetTickCount() instead of
| | QueryPerformanceCounter() whereas both are monotonic, but
| | QueryPerformanceCounter() is not steady
This is an example of where I think my pick-a-clock API can help people;
we should in some fashion offer all or most of the system clocks. Of
course monotonic() or steady() stould itself pick one, whatever people
agree is the best choice for those modes. But we may as well offer the
rest if it is easy; not with their own functions - that would be
platform specific - but findable.
| | Python steady clock will be different than the C++ definition.
[ BTW, you've a typo in here:
http://www.python.org/dev/peps/pep-0418/#id22
with the word "Specifiction", however apt that exciting new word may
seem:-)
]
You say "will be different", and since the C++ may not be adjusted maybe that
is reasonable, but there's no Python definition in the PEP for "steady" at
present. Of course, people are still bickering, but perhaps you should whack
one in as a reference for the bickering.
| | You may argue that time.steady() is not always steady: it may fallback
| | to the system clock which is adjusted by NTP and can jump
| | backward/forward with a delta greater than 1 hour.
|
| An HOUR ?!?!?
I'd like to apologise for my shrill tone here.
I still think a clock that stepped by an hour is grotesquely
non-steady. (Why an hour, BTW? I'd hope it is not related to any
timezone summer/winter localtime presentation shift notions.)
I think Kristj\341n Valur J\363nsson is on point when he says "There is
no such thing as steady time", but the notion is very attractive. If
you're going to return a "steady" clock you should be able to find out
how steady that is, for example in maximum step size (adjustment in
alignment with "real time") in seconds. I think if I got 3600 from such
a query I'd decide it was not steady enough and choose not to rely on
it. (Or print all results output in blinking red text:-)
Cheers,
--
Cameron Simpson DoD#743
http://www.cskk.ezoshosting.com/cs/
186,282 miles per second - Not just a good idea, It's the Law!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On 03Apr2012 15:08, Ethan Furman wrote: | Cameron Simpson wrote: | > get_clock already has two arguments - you can optionally hand it a clock | > list - that's used by monotonic_clock() and hires_clock(). | | def get_clock(*flags, *, clocklist=None): I presume that bare "*," is a typo. Both my python2 and python3 commands reject it. [...] | wanted = 0 | for flag in flags: | wanted |= flag [...] I could do this. I think I'm -0 on it, because it doesn't seem more expressive to my eye than the straight make-a-bitmask "|" form. Other opinions? | Would need to make *flags change to the other *_clock functions. Yep. | > The return of None is very deliberate. I _want_ user specified fallback | > to be concise and easy. The example: | > clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) | | Which would become: | clock = get_clock(MONOTONIC, HIGHRES) or get_clock(MONOTONIC) | | +1 to returning None | | > Exceptions are all very well when there is just one thing to do: parse | > this or fail, divide this by that or fail. If fact they're the very | > image of "do this one thing or FAIL". They are not such a good match for do | > this thing or that thing or this other thing. Another thought that occurred in the shower was that get_clock() et al are inquiry functions, and returning None is very sensible there. monotonic() et al are direct use functions, which should raise an exception if unavailable so that code like: t0 = monotonic() ... t1 = monotonic() does not become littered with checks for special values like None. I consider this additional reason to return None from get_clock(). Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ DON'T DRINK SOAP! DILUTE DILUTE! OK! - on the label of Dr. Bronner's Castile Soap ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
> | get_clock() returns None if no clock has the requested flags, whereas
> | I expected an exception (LookupError or NotImplementError?).
>
> That is deliberate. People can easily write fallback like this:
>
> clock = get_clock(T_MONOTONIC|T_HIRES) or get_clock(T_MONOTONIC)
Why not passing a a list of set of flags? Example:
haypo_steady = get_clock(MONOTONIC|STEADY, STEADY, MONOTONIC, REALTIME)
# try to get a monotonic and steady clock,
# or fallback to a steady clock,
# or fallback to a monotonic clock,
# or fallback to the system clock
haypo_perf_counter = get_clock(HIGHRES, MONOTONIC|STEADY, STEADY,
MONOTONIC, REALTIME)
# try to get a high-resolution clock
# or fallback to a monotonic and steady clock,
# or fallback to a steady clock,
# or fallback to a monotonic clock,
# or fallback to the system clock
On Windows, haypo_steady should give GetTickCount (MONOTONIC|STEADY)
and haypo_perf_counter should give QueryPerformanceCounter
(MONOTONIC|HIGHRES).
Hum, I'm not sure that haypo_highres uses the same clocks than
time.perf_counter() in the PEP.
> If one wants an exception it is easy to follow up with:
>
> if not clock:
> raise RunTimeError("no suitable clocks on offer on this platform")
And if don't read the doc carefuly and forget the test, you can a
"NoneType object is not callable" error.
> | get_clock() doesn't remember if a clock works or not (if it raises an
> | OSError) and does not fallback to the next clock on error. See
> | "pseudo-codes" in the PEP 418.
>
> I presume the available clocks are all deduced from the platform. Your
> pseudo code checks for OSError at fetch-the-clock time. I expect that
> to occur once when the module is loaded, purely to populate the table
> of avaiable platform clocks.
It's better to avoid unnecessary system calls at startup (when the
time module is loaded), but you may defer the creation of the clock
list, or at least of the flags of each clock.
> Note that you don't need to provide a clock list at all; get_clock(0
> will use ALL_CLOCKS by default, and hires() and monotonic() should each
> have their own default list.
A list of clocks and a function are maybe redundant. Why not only
providing a function?
> Regarding the choice itself: as the _caller_ (not the library author),
> you must decide what you want most. You're already planning offering
> monotonic() and hires() calls without my proposal!
My PEP starts with use cases: it proposes one clock per use case.
There is no "If you need a monotonic, steady and high-resolution clock
..." use case.
The "highres" name was confusing, I just replaced it with
time.perf_counter() (thanks Antoine for the name!).
time.perf_counter() should be used for benchmarking and profiling.
> Taking your query "Should
> I use MONTONIC_CLOCKS or HIRES_CLOCKS when I would like a monotonic and
> high-resolution clock" is _already_ a problem. Of course you must call
> monotonic() or hires() first under the current scheme, and must answer this
> question anyway. Do you prefer hires? Use it first! No preference? Then the
> question does not matter.
I mean having to choose the flags *and* the list of clocks is hard. I
would prefer to only have to choose flags or only the list of clocks.
The example was maybe not the best one.
> | If you have only one list of clocks, how do sort the list to get
> | QueryPerformanceCounter when the user asks for highres and
> | GetTickCount when the user asks for monotonic?
>
> This is exactly why there are supposed to be different lists.
> You have just argued against your objection above.
You can solve this issue with only one list of clocks if you use the
right set of flags.
> | So we would have:
> |
> | GetTickCount.flags = T_MONOTONIC | T_STEADY | T_HIGHRES
> |
> | Even if GetTickCount has only an accuracy of 15 ms :-/
>
> T_HIGHRES is a quality call, surely? If 15ms is too sloppy for a "high
> resolution, the is should _not_ have the T_HIRES flag.
So what is the minimum resolution and/or accuracy of the HIGHRES flag?
> | Could you please update your code according to my remarks? I will try
> | to integrate it into the PEP. A PEP should list all alternatives!
>
> Surely.
>
> The only updates I can see are to provide the flat interface
> (instead of via clock-object indirection) and the missing hires_clock()
> and monotonic_clock() convenience methods.
A full implementation would help to decide which API is the best one.
"Full" implementation:
- define all convinience function
- define all list of clocks
- define flags of all clocks listed in the PEP 418: clocks used in
the pseudo-code of time.steady and time.perf_counter, and maybe also
time.time
Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On Wed, Apr 4, 2012 at 9:38 AM, Cameron Simpson wrote:
> I could do this. I think I'm -0 on it, because it doesn't seem more
> expressive to my eye than the straight make-a-bitmask "|" form.
> Other opinions?
Yes. I've been mostly staying out of the PEP 418 clock discussion
(there are enough oars in there already), but numeric flags are
unnecessarily hard to debug. Use strings as your constants unless
there's a compelling reason not to.
Seeing "('MONOTONIC', 'HIGHRES')" in a debugger or exception message
is a lot more informative than seeing "3".
Regards,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Scalability micro-conference topic proposals (LPC2012)
Hi, We are organizing a micro-conference on scaling both upwards (many cores) and downwards (low footprint, energy efficiency) that targets all layers of the software stack. Our intent is to bring together application, libraries and kernel developers to discuss the scalability issues they currently face, and get exposure for the ongoing work on scalability infrastructure. Suggestions of topics are welcome. If you would like to present, please let us know: we have lightnening-talk slots and a few 30 minutes slots available. Presentations should be oriented towards stimulating discussion over currently faced scalability problems and/or work in progress in the area of scalability. The micro-conference will be held between August 29-31, at LinuxCon North America 2012, in San Diego. http://www.linuxplumbersconf.org/2012/ The Scaling Micro-Conference page is available at: http://wiki.linuxplumbersconf.org/2012:scaling Best Regards, Mathieu Desnoyers & Paul E. McKenney -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] .{git,bzr}ignore in cpython HG repo
On Mon, Apr 2, 2012 at 2:54 PM, Stefan Behnel wrote: > Antoine Pitrou, 02.04.2012 13:50: >> On Sun, 1 Apr 2012 19:44:00 -0500 >> Brian Curtin wrote: >>> On Sun, Apr 1, 2012 at 17:31, Matěj Cepl wrote: On 1.4.2012 23:46, Brian Curtin wrote: > For what reason? Are the git or bzr files causing issues on HG? No, but wrong .gitignore causes issues with git repo obtained via hg-fast-import. If it is meant as an intentional sabotage of using git (and bzr) for cpython, then that's the only explanation I can understand, otherwise it doesn't make sense to me why these files are in HG repository at all. >>> >>> Then you won't understand. Sometimes things get out of date when they >>> aren't used or maintained. >>> >>> You're welcome to fix the problem if you're a Git user, as suggested >>> earlier. >> >> That said, these files will always be outdated, so we might as well >> remove them so that at least git / bzr users don't get confused. > > How often is anything added to the .hgignore file? I doubt that these files > will "sufficiently always" be outdated to be unhelpful. How about using symlinks and only using a common syntax in .hgignore that git also understands? Greetings, Tom ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
Lennart Regebro wrote: On Tue, Apr 3, 2012 at 08:03, Cameron Simpson wrote: clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) If the symbol names are not the horribleness, can you qualify what API you would like more? Well, get_clock(monotonic=True, highres=True) would be a vast improvement over get_clock(MONOTONIC|HIRES). I also think it should raise an error if not found. The clarity and easy of use of the API is much more important than how much you can do in one line. That's a matter of opinion. I'm not particularly fond of this get_clock idea, but of the two examples given, I much prefer the first of these: get_clock(MONOTONIC|HIRES) get_clock(monotonic=True, highres=True) and not just because it is shorter. The API is crying out for enum arguments, not a series of named flags. But frankly I think this get_clock API sucks. At some earlier part of this thread, somebody listed three or four potential characteristics of clocks. If we offer these as parameters to get_clock(), that means there's eight or sixteen different clocks that the user can potentially ask for. Do we really offer sixteen different clocks? Or even eight? I doubt it -- there's probably only two or three. So the majority of potential clocks don't exist. With get_clock, discoverability is hurt. How does the caller know what clocks are available? How can she look for documentation for them? A simple, obvious, discoverable API is best. If we offer three clocks, we have three named functions. If some of these clocks aren't available on some platform, and we can't emulate them, then simply don't have that named function available on that platform. That's easy to discover: trying to use that clock will give a NameError or AttributeError, and the caller can then fall back on an alternative, or fail, whichever is appropriate. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
> Lennart Regebro wrote: >> Well, get_clock(monotonic=True, highres=True) would be a vast >> improvement over get_clock(MONOTONIC|HIRES). I don't like this keyword API because you have to use a magically marker (True). Why True? What happens if I call get_clock(monotonic=False) or get_clock(monotonic="yes")? Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
Cameron Simpson wrote: People have been saying "hires" throughout the threads I think, but I for one would be slightly happier with "highres". hirez? -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: rename time.monotonic() to time.steady()?
On 04/04/2012 00:31, Cameron Simpson wrote: [ Returning at more leisure... ] I think Kristj\341n Valur J\363nsson is on point when he says "There is no such thing as steady time", but the notion is very attractive. If you're going to return a "steady" clock you should be able to find out how steady that is, for example in maximum step size (adjustment in alignment with "real time") in seconds. I think if I got 3600 from such a query I'd decide it was not steady enough and choose not to rely on it. (Or print all results output in blinking red text:-) Cheers, IIRC time.steady() has been rejected umpteen times. Someone (apologies, it's a long thread :) suggested time.steadier() [or time.steadiest() ?] implying that it's the best that can be done. I'd go with that unless someboby has a better option, but there's been so many suggested that I might even have to read the PEP :) The more that this thread runs, the more I get the impression that we're trying to make mountains out of electrons. Having said that, I know that the conservative nature of Python development is the best approach here wrt the API. Rather short term pain and long term gain than vice versa. Just my 2p worth. -- Cheers. Mark Lawrence. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On 04/04/2012 01:04, Greg Ewing wrote: Cameron Simpson wrote: People have been saying "hires" throughout the threads I think, but I for one would be slightly happier with "highres". hirez? IMHO still too easy to read as hires. Or is it? Bah I'm going to bed and will think about it, night all. -- Cheers. Mark Lawrence. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Judging by the hundreds of emails regarding PEP 418, the disagreements about APIs, namings, and even what characteristics clocks should have, I believe that the suggestion is too divisive (and confusing!) to be accepted or rejected at this time. Everyone has a different opinion, everyone believes their opinion holds for the majority, and it isn't practical for anyone to read the entire discussion. I propose for Python 3.3: 1) the os module should expose lightweight wrappers around whatever clocks the operating system provides; 2) the time module should NOT provide any further clocks other than the existing time() and clock() functions (but see point 4 below); 3) we postpone PEP 418 until there is some real-world experience with using the os clocks from Python and we can develop a consensus of what is actually needed rather than what people think we need (i.e. probably in 3.4); 4) if the standard library has need for a "use the best clock available, for some definition of best, and fall back to time() if not" clock, then the time module should do the simplest thing that could possible work, flagged as a private function: try: from os import bestclock as _bestclock except ImportError: _bestclock = time This can always be promoted to a public function later, if necessary. Python has worked pretty well without high res and monotonic clocks for 20 years. Let's not rush into a suboptimal design based on who can outlast everyone else in this discussion. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Wed, Apr 4, 2012 at 10:33 AM, Steven D'Aprano wrote: > Python has worked pretty well without high res and monotonic clocks for 20 > years. Let's not rush into a suboptimal design based on who can outlast > everyone else in this discussion. +1 FWIW, I'd be fine with underscore prefixes on *any* additions to the relevant module APIs for 3.3. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
On 04Apr2012 09:53, Steven D'Aprano wrote: | Lennart Regebro wrote: | > On Tue, Apr 3, 2012 at 08:03, Cameron Simpson wrote: | >> clock = get_clock(MONOTONIC|HIRES) or get_clock(MONOTONIC) | >> If the symbol names are not the horribleness, can you qualify what API | >> you would like more? | > | > Well, get_clock(monotonic=True, highres=True) would be a vast | > improvement over get_clock(MONOTONIC|HIRES).[...] | | That's a matter of opinion. I'm not particularly fond of this get_clock idea, | but of the two examples given, I much prefer the first of these: | | get_clock(MONOTONIC|HIRES) | get_clock(monotonic=True, highres=True) | | and not just because it is shorter. The API is crying out for enum arguments, | not a series of named flags. Enums would be ok with me. I went with a bitmask because it is natural to me and very simple. But anything symbolicly expression will do. | But frankly I think this get_clock API sucks. At some earlier part of this | thread, somebody listed three or four potential characteristics of clocks. If | we offer these as parameters to get_clock(), that means there's eight or | sixteen different clocks that the user can potentially ask for. Do we really | offer sixteen different clocks? Or even eight? I doubt it -- there's probably | only two or three. So the majority of potential clocks don't exist. That's not the point. I think we should offer all the platform system clocks, suitably described. That there are up to 8 or 16 flag combinations is irrelevant; no user is going to try them all. A user will have requirements for their clock. They ask for them either blandly via get_clock() or (for example considering monotonic most important) via monotonic_clock(). In the latter case, the supported clocks can be considered in a more apt order via a different internal clock list. | With get_clock, discoverability is hurt. No, because the other calls still exist. (In my proposal. I see Victor's characterised this as either/or in the PEP, never my intent.) | How does the caller know what clocks | are available? I would definitely want either: - the module clock lists available via public names, for example as in my sample clockutils.py code (ALL_CLOCKS, MONTONIC_CLOCKS etc) or via some map (eg clocks['monotonic']). - a get_clocks() function to return matching clocks, like get_clock() but not stopping on the first match - an all_clocks=False parameter to get_clock() to get an iterable of the suitable clocks | How can she look for documentation for them? There is good text in the PEP. That could be easily moved into the module doco in a "clocks" section. Since my clocks proposal wraps clocks in an object, they _can_ have nice class names and good docstrings and more metadata in the object (possibilities including .epoch, .precision, .is_steady() methods, .os_clock_name (eg "QueryPerformanceCounter"), etc). | A simple, obvious, discoverable API is best. If we offer three clocks, we have | three named functions. If some of these clocks aren't available on some | platform, and we can't emulate them, then simply don't have that named | function available on that platform. That's easy to discover: trying to use | that clock will give a NameError or AttributeError, and the caller can then | fall back on an alternative, or fail, whichever is appropriate. And I hate this. Because many platforms offer several OS clocks. The time module SHOULD NOT dictate what clocks you get to play with, and you should not need to have platform specific knowledge to look for a clock with your desired characteristics. If you just want montonic() and trust the module authors' policy decisions you can go with monotonic(), have it do AttributeError if unavailable and never worry about discoverability or the inspectable object layer. Many will probaby be happy with that. But without get_clock() or something like it, there is no discoverability and not ability for a user to decide their own clock choice policy. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Your modesty is typically human, so I will overlook it. - a Klingon ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Le 04/04/2012 02:33, Steven D'Aprano a écrit : Judging by the hundreds of emails regarding PEP 418, the disagreements about APIs, namings, and even what characteristics clocks should have, I believe that the suggestion is too divisive (and confusing!) to be accepted or rejected at this time. Oh, I just "rewrote" the PEP before reading your email. Sorry for the noise with this PEP :-) I just read again all emails related to this PEP to complete the PEP. The PEP should now list all proposed API designs. I hope that I did not forget anything. I failed to propose a consistent and clear API because I (and Guido!) wanted to fallback to the system clock. Falling back to the system clock is a problem when you have to define the function in the documentation or if you don't want to use the system clock (but do something else) if no monotonic clock is available. So I rewrote the PEP to simplify it: * Don't fallback to system clock: time.monotonic() is always monotonic (cannot go backward), but it is not always available. You have to write a classic try/except ImportError which has a nice advantage: your program will work on Python older than 3.3 ;-) * Remove the time.perf_counter() function (it was called time.highres() before). "highres" notion was confusing. I only wrote the function to expose QueryPerformanceCounter (while it was already accessible using os.clock()). The function was not well defined. Another PEP should be written or at least the subject should be discussed after the PEP 418 (monotonic clock). * Rename time.steady() to time.monotonic(), again :-) Everyone has a different opinion, everyone believes their opinion holds for the majority, and it isn't practical for anyone to read the entire discussion. I read most emails and I can say that: * There is a need for a monotonic clock * Most people prefer to handle explicitly the fallback if no monotonic clock is available * Most people don't want to call the new function "steady" because it stands for something different I propose for Python 3.3: 1) the os module should expose lightweight wrappers around whatever clocks the operating system provides; Python 3.3 has already time.clock_gettime() and time.clock_getres() with CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW, CLOCK_HIGHRES. mach_absolute_time() and GetTickCount/GetTick64 are not available yet. 3) we postpone PEP 418 until there is some real-world experience with using the os clocks from Python and we can develop a consensus of what is actually needed rather than what people think we need (i.e. probably in 3.4); Many applications already implement their own "monotonic" clock". Some libraries provide also such clock for Python. On UNIX, it's always using clock_gettime(MONOTONIC). On Windows, it's sometimes GetTickCount, sometimes QueryPerformanceCounter. On Mac OS X, it's always mach_absolute_time(). I didn't find a library supporting Solaris. 4) if the standard library has need for a "use the best clock available, for some definition of best, and fall back to time() if not" clock, then the time module should do the simplest thing that could possible work, flagged as a private function: In the last version of my PEP, time.monotonic() is simply defined as "a monotonic clock (cannot go backward)". There is no more "... best ..." in its definition. Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 2012-04-03, at 8:33 PM, Steven D'Aprano wrote: > 1) the os module should expose lightweight wrappers around whatever clocks > the operating system provides; +1. That should make it flexible enough to those who really need it. > 2) the time module should NOT provide any further clocks other than the > existing time() and clock() functions (but see point 4 below); > > 3) we postpone PEP 418 until there is some real-world experience with using > the os clocks from Python and we can develop a consensus of what is actually > needed rather than what people think we need (i.e. probably in 3.4); > > 4) if the standard library has need for a "use the best clock available, for > some definition of best, and fall back to time() if not" clock, then the time > module should do the simplest thing that could possible work, flagged as a > private function: +1 on overall idea too. - Yury ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Finally! We've come full circle. +1 for monotonic as just described by Victor. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Lock it in before the paint dries. On Apr 4, 2012 10:05 AM, "Matt Joiner" wrote: > Finally! We've come full circle. > > +1 for monotonic as just described by Victor. > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On 04Apr2012 01:45, Victor Stinner wrote:
| > | get_clock() returns None if no clock has the requested flags, whereas
| > | I expected an exception (LookupError or NotImplementError?).
| >
| > That is deliberate. People can easily write fallback like this:
| >
| > clock = get_clock(T_MONOTONIC|T_HIRES) or get_clock(T_MONOTONIC)
|
| Why not passing a a list of set of flags? Example:
|
| haypo_steady = get_clock(MONOTONIC|STEADY, STEADY, MONOTONIC, REALTIME)
| # try to get a monotonic and steady clock,
| # or fallback to a steady clock,
| # or fallback to a monotonic clock,
| # or fallback to the system clock
That's interesting. Ethan Furman suggested multiple arguments to be
combined, whereas yours bundles multiple search criteria in one call.
While it uses a bitmask as mine does, this may get cumbersome if we went
with Nick's "use strings!" suggestion.
| haypo_perf_counter = get_clock(HIGHRES, MONOTONIC|STEADY, STEADY,
| MONOTONIC, REALTIME)
| # try to get a high-resolution clock
| # or fallback to a monotonic and steady clock,
| # or fallback to a steady clock,
| # or fallback to a monotonic clock,
| # or fallback to the system clock
|
| On Windows, haypo_steady should give GetTickCount (MONOTONIC|STEADY)
| and haypo_perf_counter should give QueryPerformanceCounter
| (MONOTONIC|HIGHRES).
Sounds ok to me. I am not familiar with the Windows counters and am
happy to take your word for it.
| Hum, I'm not sure that haypo_highres uses the same clocks than
| time.perf_counter() in the PEP.
|
| > If one wants an exception it is easy to follow up with:
| > if not clock:
| > raise RunTimeError("no suitable clocks on offer on this platform")
|
| And if don't read the doc carefuly and forget the test, you can a
| "NoneType object is not callable" error.
Excellent! An exception either way! Win win!
| > | get_clock() doesn't remember if a clock works or not (if it raises an
| > | OSError) and does not fallback to the next clock on error. See
| > | "pseudo-codes" in the PEP 418.
| >
| > I presume the available clocks are all deduced from the platform. Your
| > pseudo code checks for OSError at fetch-the-clock time. I expect that
| > to occur once when the module is loaded, purely to populate the table
| > of avaiable platform clocks.
|
| It's better to avoid unnecessary system calls at startup (when the
| time module is loaded), but you may defer the creation of the clock
| list, or at least of the flags of each clock.
Yes indeed. I think this should be deferred until use.
| > Note that you don't need to provide a clock list at all; get_clock(0
| > will use ALL_CLOCKS by default, and hires() and monotonic() should each
| > have their own default list.
|
| A list of clocks and a function are maybe redundant. Why not only
| providing a function?
Only because the function currently only returns one clock.
The picky user may want to peruse all the clocks inspecting other
metadata (precision etc) than the coarse flag requirements.
There should be a way to enumerate the available clock implementation;
in my other recent post I suggest either lists (as current), a
get_clocks() function, or a mode parameter to get_clock() such as
_all_clocks, defaulting to False.
| > Regarding the choice itself: as the _caller_ (not the library author),
| > you must decide what you want most. You're already planning offering
| > monotonic() and hires() calls without my proposal!
|
| My PEP starts with use cases: it proposes one clock per use case.
| There is no "If you need a monotonic, steady and high-resolution clock
| ..." use case.
Yes. but this is my exact objection to the "just provide hires() and
steady() and/or monotonic()" API; the discussion to date is littered
with "I can't imagine wanting to do X" style remarks. We should not be
trying to enumerate the user case space exhaustively. I'm entirely in
favour of your list of use cases and the approach of providing hires() et
al to cover the thought-common use cases. But I feel we really _must_
provide a way for the user with a not-thought-of use case to make an
arbitrary decision.
get_clock() provides a simple cut at the "gimme a suitable clock"
approach, with the lists or other "get me an enumeration of the
available clocks" mechanism for totally ad hoc perusal if the need
arises.
This is also my perhaps unstated concern with Guido's "the more I think about
it, the more I believe these functions should have very loose guarantees, and
instead just cater to common use cases -- availability of a timer with
minimal fuss is usually more important than the guarantees"
http://www.mail-archive.com/[email protected]/msg66173.html
The easy to use hires() etc must make very loose guarentees or they will
be useless too often. That looseness is fine in some ways - it provides
availability on many platforms (all?) and discourages the user from
hoping for too much and thus writing fragile code. But it also PREVENTS
the user from obtaining a really good clock if it is availabl
