Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Cameron Simpson wrote: I maintain that "monotonic" still means what I said, and that it is the combination of the word with "clock" that brings in your other criteria. I'm not trying to redefine the word "monotonic" in general. All I'm saying is that *if* the PEP is going to talk about a "monotonic clock" or "monotonic time", it should clearly define what this means, because it's not obvious to everyone that it implies something more than the mathematical meaning. Alternatively, don't use the word "monotonic" at all, and find a better term. CLOCK_MONOTONIC Clock that cannot be set and represents monotonic time since some unspecified starting point. Which doesn't help very much, because it talks about "monotonic time" without saying what that means. Googling for that phrase doesn't seem turn up anything very useful. Apparently we're supposed to just know. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Steven D'Aprano wrote: Greg Ewing wrote: the important thing about a clock that it *keeps going forward* That would be a *strictly* monotonic clock, as opposed to merely monotonic. Well, yes, but even that's not enough -- it needs to go forward at a reasonably constant rate, otherwise it's just as useless. If it had enough resolution, it could go forward by one femtosecond every hour for a while and still call itself strictly monotonic... -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
On Fri, Apr 6, 2012 at 3:39 PM, Glyph Lefkowitz wrote: > There seems to be a persistent desire in this discussion to specify and > define these flaws out of existence, where this API really should instead be > embracing the flaws and classifying them. That seems to be precisely what Cameron is advocating. > I think it's better to learn the local jargon and try to apply it > consistently. If you search around the web for the phrase "monotonic > clock", it's applied in a sense closest to the one you mean on thousands and > thousands of web pages. But is "a sense" the *same* sense on all of those pages? If not, then some people are going to be upset by anything we label a "monotonic" clock, because it will suffer from some flaw that's unacceptable in their applications for "monotonic" clocks. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
Glyph Lefkowitz wrote: On Apr 5, 2012, at 8:07 PM, Zooko Wilcox-O'Hearn wrote: 2. Those who think that "monotonic clock" means a clock that never jumps, and that runs at a rate approximating the rate of real time. This is a very useful kind of clock to have! It is what C++ now calls a "steady clock". It is what all the major operating systems provide. All clocks run at a rate approximating the rate of real time. That is very close to the definition of the word "clock" in this context. All clocks have flaws in that approximation, and really those flaws are the whole point of access to distinct clock APIs. Different applications can cope with different flaws. I think that this is incorrect. py> time.clock(); time.sleep(10); time.clock() 0.41 0.41 -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
Cameron Simpson wrote: On 05Apr2012 08:50, Steven D'Aprano wrote: | Although I don't like the get_clock() API, I don't think this argument against | it is a good one. Just to divert briefly; you said in another post you didn't like the API and (also/because?) it didn't help discoverability. My core objective was to allow users to query for clocks, and ideally enumerate and inspect all clocks. Without the caller having platform specific knowledge. Clocks *are* platform specific -- not just in their availability, but also in the fine details of their semantics and behaviour. I don't think we can or should try to gloss over this. If people are making decisions about timers without knowledge of what their platform supports, they're probably making poor decisions. Even the venerable time.time() and time.clock() differ between Linux and Windows. Allowing for the sake of discussion that this is desirable, what would you propose as an API instead of get_clock() (and its friend, get_clocks() for enumeration, that I should stuff into the code). The old ways are the best. We don't have math.get_trig() and math.get_trigs() functions for querying trigonometric functions, we just expose the functions directly. I think the way to enumerate and inspect all clocks is with the tried and true Python introspection tools that people use on all other functions: * use dir(time) to see a list of names available in the module * use help(time) to read their help * read the Fine Manual to find out more * use try... except... to detect the existence of a clock There's nothing special about clocks that needs anything more than this. get_clock() looks like a factory function, but it actually isn't. It just selects from a small number of pre-existing clocks. We should just expose those pre-existing clocks directly. I don't see any advantage in adding that extra level of indirection or the addition of all this complexity: * a function get_clock() to select a clock * a function get_clocks() to enumerate all the clocks * another function for querying the properties of a clock All those functions accomplish is to increase the complexity of the API, the documentation and the implementation. It's one more special case for the user to learn: "To find out what functions are available, use dir(module), except for clocks, where you have to use time.get_clocks()." Blah. Another problem with get_clock() -- it will be an attractive nuisance for the sort of person who cares about symmetry and completeness. You will have a steady trickle of "feature requests" from users who are surprised that not every combination of features is supported. Out of the eight or sixteen or thirty-two potential clocks that get_clock() tempts the user with, only three or five will actually exist. The only advantage of get_clock is that you don't need to know the *name* of a platform clock in order to use it, you can describe it with a series of flags or enums. But in practice, that's not an advantage, that's actually a disadvantage. Consider: "Which clock should I use for such-and-such a task, foo or bar?" versus "Which clock should I use for such-and-such a task, get_clock(spam, eggs, cheese) or get_clock(ham, eggs, truffles)?" The mere mechanics of talking about these clocks will suffer because they aren't named. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 6 April 2012 02:50, Cameron Simpson wrote: (Quoted from the Linux manpage) > All implementations support the system-wide real-time clock, which > is identified by CLOCK_REALTIME. Its time represents seconds and > nanoseconds since the Epoch. When its time is changed, timers for > a relative interval are unaffected, but timers for an absolute point > in time are affected. > This made me think. They make a distinction between "timers for a relative interval" and "timers for an absolute point in time". But that's not right - *all* of the clock calls we are talking about here return a *single* number. Interpreting that as an absolute time needs an epoch. Or to put it another way, clock values are always meaningless without context - whereas clock *differences* are what actually carry meaning (in terms of a duration). On that basis, I'd say that - A clock that doesn't get adjusted or slewed is a tick counter (which technically doesn't have any relationship to time, although the tick frequency can be used to convert to seconds, but see the next entry) - A clock that gets slewed but not adjusted is a seconds counter (or at least, the nearest approximation the system can provide - presumably better than a tick counter) - A clock that gets adjusted is not an interval timer at all, but an absolute timer (which therefore shouldn't really be used for benchmarking or timeouts) It seems to me that what *I* would most often need are the second two of these (to at least as high a precision as my app needs, which may vary but "to the highest precision possible" would do :-)) I'd be happy for a seconds counter to fallback to a tick counter converted to seconds using its frequency - slewing is simply an accuracy improvement process, as far as I can see. It seems to me that the current time.time() and time.wallclock() are the right names for my "absolute timer" and "seconds timer" above. Whether their implementations match my definitions I'm not sure, but that's what I'd hope. One thing I would expect is that time.wallclock() would never go backwards (so differences are always positive). The various other debates about monotonic, steady, etc, seem to me to be only relevant for specialist uses that I don't care about. As regards suspension, if I'm timing intervals and the system suspends, I'd be happy to say all bets are off. Similarly with timeouts. If I cared, I'd simply make sure the system didn't suspend :-) As far as comparability between different threads or processes are concerned, I would expect absolute time (time.time) to be the same across threads or processes (but wouldn't generally write apps that were affected if it weren't - at least by small amounts), but I wouldn't expect time.wallclock values obtained in different threads or processes to be comparable (mostly because I can't think of a case where I'd compare them). Where VMs or multiple machines are involved, I wouldn't even expect absolute time to match (but that's the job of NTP, and if time.time follows NTP, there's no reason why there would be an issue even there). Summary: I'm happy with time.time and time.wallclock. The rest of this debate doesn't matter for my usecases (and I suspect many other people's in practice). [Update, after I downloaded and installed 3.3a2] Bah, looks like time.wallclock is gone. (Actually, looks like it was documented but not implemented in 3.3a1!). Actually, the docs and the implementation don't match - clock_gettime is documented as available, but it's not (at least on Windows). I still prefer time.wallclock() as described above and in the 3.3a1 documentation. I thought I knew what was going on, but now I'm confused. My comments above still stand, though. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418 (was: PEP 418: Add monotonic clock)
On Thu, Apr 5, 2012 at 21:57, Stephen J. Turnbull wrote: > I might have chosen to implement a 'None' return if I had designed > open(), but I can't get too upset about raising an Exception as it > actually does. One fundamental difference is that there are many reasons one might fail to open a file. It may not exist. It may not have permissions allowing the request. It may be locked. If open() returned None, this information would have to be retrievable through another function. However since it returns an exception, that information is already wrapped up in the exception object, should you choose to catch it, and likely to be logged otherwise. In the case of the clocks, I'm assuming the only reason you would fail to get a clock is because it isn't provided by hardware and/or OS. You don't have to worry about transient scenarios on multi-user systems where another user has locked the clock. Thus the exception cannot tell you anything more than None tells you. (Of course, if my assumption is wrong, I'm not sure whether my reasoning still applies.) -- Michael Urman ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
On 6 April 2012 11:12, Steven D'Aprano wrote: > Glyph Lefkowitz wrote: > >> On Apr 5, 2012, at 8:07 PM, Zooko Wilcox-O'Hearn wrote: >> > > 2. Those who think that "monotonic clock" means a clock that never jumps, >>> and that runs at a rate approximating the rate of real time. This is a >>> very useful kind of clock to have! It is what C++ now calls a "steady >>> clock". It is what all the major operating systems provide. >>> >> >> All clocks run at a rate approximating the rate of real time. That is >> very >> close to the definition of the word "clock" in this context. All clocks >> have flaws in that approximation, and really those flaws are the whole >> point of access to distinct clock APIs. Different applications can cope >> with different flaws. >> > > I think that this is incorrect. > > py> time.clock(); time.sleep(10); time.clock() > 0.41 > 0.41 > Blame Python's use of CPU time in clock() on Unix for that. On Windows: >>> time.clock(); time.sleep(10); time.clock() 14.879754156329385 24.879591008462793 That''s a backward compatibility issue, though - I'd be arguing that time.clock() is the best name for "normally the right clock for interval, benchmark or timeout uses as long as you don't care about oddities like suspend" otherwise. Given that this name is taken, I'd argue for time.wallclock. I'm not familiar enough with the terminology to know what to expect from terms like monotonic, steady, raw and the like. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
On Fri, Apr 06, 2012 at 11:57:20AM +0900, "Stephen J. Turnbull" wrote: > What I want to know is why you're willing to assert that absence of a > clock of a particular configuration is an Exception, when that absence > clearly documented to be a common case? An error or not an error depends on how people will use the API. I usually don't like error codes -- people tend to ignore them or check lazily. If some library would do (get_clock(THIS) or get_clock(THAT)).clock() I want to get a clearly defined and documented clock-related error, not some vague "AttributeError: 'NoneType' object has no attribute 'clock'". Oleg. -- Oleg Broytmanhttp://phdru.name/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pep 393 and debugging
I just had my first fun with Pep 393 strings and debuggers. Trying to debug a deadlocked python program, I'm trying to figure out the callstack of the thread in the debugger. I ended up with something like: (char*)&((PyASCIIObject*)(tstate->frame->f_code->co_filename))[1] while previously, it was sufficient to do (PyUnicodeObject*)(tstate->frame->f_code->co_filename) Obviously this won't work for non-ASCII objects. I wonder if there is a way to make this situation easier? Perhaps for "debug" builds, we can store some debug information in the frame object, e.g. utf8 encoding of the filename and function? K ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pep 393 and debugging
2012/4/6 Kristján Valur Jónsson : > I wonder if there is a way to make this situation easier? Perhaps for > "debug" builds, we can store some debug information in the frame object, > e.g. utf8 encoding of the filename and function? Have you tried using the cpython gdb plugin? It should repr these things for you. -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
> 2. Those who think that "monotonic clock" means a clock that never > jumps, and that runs at a rate approximating the rate of real time. > This is a very useful kind of clock to have! It is what C++ now calls > a "steady clock". It is what all the major operating systems provide. Python cannot give such guarantee. Extract of time.monotonic() function proposed in the PEP 418: "The elapsed time may or may not include time the system spends in sleep or hibernation; this depends on the operating system." The C++ Timeout Specification uses the following definition: "Objects of class steady_clock represent clocks for which values of time_point advance at a steady rate relative to real time. That is, the clock may not be adjusted." Proposed time.monotonic() doesn't respect this definition because CLOCK_MONOTONIC *is* adjusted (slewed) on Linux. We might provide a steady clock, but it would be less portable than the monotonic clock. I'm not sure that we need such clock, which use case requires a steady and not a monotonic clock? On Linux, I now prefer to use CLOCK_MONOTONIC (monotonic) than CLOCK_MONOTONIC_RAW (monotonic and steady as defined by C++) *because* its frequency is adjusted. Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
Oleg Broytman wrote: On Fri, Apr 06, 2012 at 11:57:20AM +0900, "Stephen J. Turnbull" wrote: What I want to know is why you're willing to assert that absence of a clock of a particular configuration is an Exception, when that absence clearly documented to be a common case? An error or not an error depends on how people will use the API. I usually don't like error codes -- people tend to ignore them or check lazily. If some library would do (get_clock(THIS) or get_clock(THAT)).clock() I want to get a clearly defined and documented clock-related error, not some vague "AttributeError: 'NoneType' object has no attribute 'clock'". The error won't be that vague -- it will include that offending line, making the problem easy to track. ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
I'd like to veto wall clock because to me that's the clock on my wall, i.e. local time. Otherwise I like the way this thread is going. --Guido van Rossum (sent from Android phone) On Apr 6, 2012 4:57 AM, "Paul Moore" wrote: > On 6 April 2012 11:12, Steven D'Aprano wrote: > >> Glyph Lefkowitz wrote: >> >>> On Apr 5, 2012, at 8:07 PM, Zooko Wilcox-O'Hearn wrote: >>> >> >> 2. Those who think that "monotonic clock" means a clock that never jumps, and that runs at a rate approximating the rate of real time. This is a very useful kind of clock to have! It is what C++ now calls a "steady clock". It is what all the major operating systems provide. >>> >>> All clocks run at a rate approximating the rate of real time. That is >>> very >>> close to the definition of the word "clock" in this context. All clocks >>> have flaws in that approximation, and really those flaws are the whole >>> point of access to distinct clock APIs. Different applications can cope >>> with different flaws. >>> >> >> I think that this is incorrect. >> >> py> time.clock(); time.sleep(10); time.clock() >> 0.41 >> 0.41 >> > > Blame Python's use of CPU time in clock() on Unix for that. On Windows: > > >>> time.clock(); time.sleep(10); time.clock() > 14.879754156329385 > 24.879591008462793 > > That''s a backward compatibility issue, though - I'd be arguing that > time.clock() is the best name for "normally the right clock for interval, > benchmark or timeout uses as long as you don't care about oddities like > suspend" otherwise. Given that this name is taken, I'd argue for > time.wallclock. I'm not familiar enough with the terminology to know what > to expect from terms like monotonic, steady, raw and the like. > > Paul. > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2012-03-30 - 2012-04-06) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open3360 ( +1) closed 22935 (+64) total 26295 (+65) Open issues with patches: 1430 Issues opened (38) == #9634: Add timeout parameter to Queue.join() http://bugs.python.org/issue9634 reopened by ncoghlan #14258: Better explain re.LOCALE and re.UNICODE for \S and \W http://bugs.python.org/issue14258 reopened by orsenthil #14453: profile.Profile.calibrate can produce incorrect numbers in som http://bugs.python.org/issue14453 opened by adamtj #14455: plistlib unable to read json and binary plist files http://bugs.python.org/issue14455 opened by d9pouces #14457: Unattended Install doesn't populate registry http://bugs.python.org/issue14457 opened by Paul.Klapperich #14458: Non-admin installation fails http://bugs.python.org/issue14458 opened by toughy #14460: In re's positive lookbehind assertion repetition works http://bugs.python.org/issue14460 opened by py.user #14461: In re's positive lookbehind assertion documentation match() ca http://bugs.python.org/issue14461 opened by py.user #14462: In re's named group the name cannot contain unicode characters http://bugs.python.org/issue14462 opened by py.user #14465: xml.etree.ElementTree: add feature to prettify XML output http://bugs.python.org/issue14465 opened by tshepang #14468: Update cloning guidelines in devguide http://bugs.python.org/issue14468 opened by eric.araujo #14469: Python 3 documentation links http://bugs.python.org/issue14469 opened by storchaka #14470: Remove using of w9xopen in subprocess module http://bugs.python.org/issue14470 opened by asvetlov #14472: .gitignore is outdated http://bugs.python.org/issue14472 opened by mcepl #14475: codecs.StreamReader.read behaves differently from regular file http://bugs.python.org/issue14475 opened by tdb #14477: Rietveld test issue http://bugs.python.org/issue14477 opened by loewis #14478: Decimal hashing very slow, could be cached http://bugs.python.org/issue14478 opened by Jimbofbx #14480: os.kill on Windows should accept zero as signal http://bugs.python.org/issue14480 opened by asvetlov #14483: inspect.getsource fails to read a file of only comments http://bugs.python.org/issue14483 opened by Sean.Grider #14484: missing return in win32_kill? http://bugs.python.org/issue14484 opened by pitrou #14486: Add some versionchanged notes in threading docs http://bugs.python.org/issue14486 opened by ncoghlan #14488: Can't install Python2.7.2 http://bugs.python.org/issue14488 opened by kiwii128 #14494: __future__.py and its documentation claim absolute imports bec http://bugs.python.org/issue14494 opened by smarnach #14499: Extension module builds fail with Xcode 4.3 on OS X 10.7 due t http://bugs.python.org/issue14499 opened by ned.deily #14500: test_importlib fails in refleak mode http://bugs.python.org/issue14500 opened by pitrou #14501: Error initialising BaseManager class with 'authkey' argument o http://bugs.python.org/issue14501 opened by Drauger #14503: docs:Code not highlighted http://bugs.python.org/issue14503 opened by ramchandra.apte #14504: Suggestion to improve argparse's help messages for "store_cons http://bugs.python.org/issue14504 opened by Amnon.Harel #14507: Segfault with starmap and izip combo http://bugs.python.org/issue14507 opened by progrper #14508: gprof2html is broken http://bugs.python.org/issue14508 opened by Popa.Claudiu #14509: Build failures in non-pydebug builds without NDEBUG. http://bugs.python.org/issue14509 opened by twouters #14511: _static/opensearch.xml for Python 3.2 docs directs searches to http://bugs.python.org/issue14511 opened by zach.ware #14512: Pydocs module docs server not working on Windows. http://bugs.python.org/issue14512 opened by terry.reedy #14513: IDLE icon switched and switches on Windows taskbar http://bugs.python.org/issue14513 opened by terry.reedy #14514: Equivalent to tempfile.NamedTemporaryFile that deletes file at http://bugs.python.org/issue14514 opened by r.david.murray #14515: tempfile.TemporaryDirectory documented as returning object but http://bugs.python.org/issue14515 opened by r.david.murray #14516: test_tools assumes BUILDDIR=SRCDIR http://bugs.python.org/issue14516 opened by ronaldoussoren #14517: Recompilation of sources with Distutils http://bugs.python.org/issue14517 opened by cbenoit Most recent 15 issues with no replies (15) == #14517: Recompilation of sources with Distutils http://bugs.python.org/issue14517 #14515: tempfile.TemporaryDirectory documented as returning object but http://bugs.python.org/issue14515 #14512: Pydocs module docs server not working on Windows. http://bugs.python.org/issue14512 #14511: _static/opensearch.xml for Python 3.2 docs directs sear
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
This is the most amusing of discussions. Teh key sentence here is "the clock may not be adjusted". Slewing or accelerating a clock is nerely adding to the already present error of the pace of the clock. Sometimes a clock runs fast, sometimes it runs slow. This is without any purposeful slewing or accelerating made by the OS. Notice how the C++ standard specifies nothing about the error of this steady rate, which is however always nonzero. This implies that the error in the time, or the variations of its rate, are not important to the meaning of the standard. The thing wich matters here is that the clock progresses forwards, matching the progress of real times, to some (unsopecified) precision. It must not suddenly jump backwards or forwards because someone changed the timezone. That is all that is implied. Since the error of the clock is unspecified, (that is, the error in its rate of progress) it cannot matter if this rate is temporarily adjusted by hand. K Frá: [email protected] [[email protected]] fyrir hönd Victor Stinner [[email protected]] Sent: 6. apríl 2012 14:32 To: Zooko Wilcox-O'Hearn Cc: Python-Dev Efni: Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed) The C++ Timeout Specification uses the following definition: "Objects of class steady_clock represent clocks for which values of time_point advance at a steady rate relative to real time. That is, the clock may not be adjusted." Proposed time.monotonic() doesn't respect this definition because CLOCK_MONOTONIC *is* adjusted (slewed) on Linux. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
This is the original reason for the original defect (issue 10278) unix' clock() doesn't actually provide a clock in this sense, it provides a resource usage metric. K Frá: [email protected] [[email protected]] fyrir hönd Steven D'Aprano [[email protected]] Sent: 6. apríl 2012 10:12 To: Python-Dev Efni: Re: [Python-Dev] this is why we shouldn't call it a "monotonicclock" (was: PEP 418 is too divisive and confusing and should be postponed) I think that this is incorrect. py> time.clock(); time.sleep(10); time.clock() 0.41 0.41 -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Possible change to logging.handlers.SysLogHandler
There is a problem with the way logging.handlers.SysLogHandler works when presented with Unicode messages. According to RFC 5424, Unicode is supposed to be sent encoded as UTF-8 and preceded by a BOM. However, the current handler implementation puts the BOM at the start of the formatted message, and this is wrong in scenarios where you want to put some additional structured data in front of the unstructured message part; the BOM is supposed to go after the structured part (which, therefore, has to be ASCII) and before the unstructured part. In that scenario, the handler's current behaviour does not strictly conform to RFC 5424. The issue is described in [1]. The BOM was originally added / position changed in response to [2] and [3]. It is not possible to achieve conformance with the current implementation of the handler, unless you subclass the handler and override the whole emit() method. This is not ideal. For 3.3, I will refactor the implementation to expose a method which creates the byte string which is sent over the wire to the syslog daemon. This method can then be overridden for specific use cases where needed. However, for 2.7 and 3.2, removing the BOM insertion would bring the implementation into conformance to the RFC, though the entire message would have to be regarded as just a set of octets. A Unicode message would still be encoded using UTF-8, but the BOM would be left out. I am thinking of removing the BOM insertion in 2.7 and 3.2 - although it is a change in behaviour, the current behaviour does seem broken with regard to RFC 5424 conformance. However, as some might disagree with that assessment and view it as a backwards-incompatible behaviour change, I thought I should post this to get some opinions about whether this change is viewed as objectionable. Regards, Vinay Sajip [1] http://bugs.python.org/issue14452 [2] http://bugs.python.org/issue7077 [3] http://bugs.python.org/issue8795 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, Apr 5, 2012 at 12:32, Victor Stinner wrote: > I prefer to use CLOCK_MONOTONIC, not because it is also available for > older Linux kernels, but because it is more reliable. Even if the > underlying clock source is unstable (unstable frequency), a delta of > two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, > whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or > smaller than a second. Aha. OK, CLOCK_MONOTONIC it is then. //Lennart ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
On Fri, Apr 6, 2012 at 00:17, Cameron Simpson wrote: > Gah! ALL functions are like that! How often do we see questions about > max() or split() etc that a close reading of the docs obviate? My point exactly. //Lennart ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
Lennart Regebro wrote: On Fri, Apr 6, 2012 at 00:17, Cameron Simpson wrote: Good point, but the same does for using flags. If you don't pass in the MONOTONIC flag, what happens? Only reading the documentation will tell you. As such this, if anything, is an indication that the get_clock() API isn't ideal in any incarnation. Gah! ALL functions are like that! How often do we see questions about max() or split() etc that a close reading of the docs obviate? My point exactly. Huh? Your point is that all APIs are less than ideal because you have to read the docs to know for certain how they work? ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
I don't know who started this, but the PEP 418 threads have altogether too much snarkiness and not enough content. It's bad enough that we're bikeshedding so intensely; we don't need clever comebacks in triplicate to every out-of-context argument. --Guido On Fri, Apr 6, 2012 at 2:26 PM, Ethan Furman wrote: > Lennart Regebro wrote: >> >> On Fri, Apr 6, 2012 at 00:17, Cameron Simpson wrote: >>> >>> >> Good point, but the same does for using flags. If you don't pass in >> >> the MONOTONIC flag, what happens? Only reading the documentation will >> tell you. As such this, if anything, is an indication that the >> >> get_clock() API isn't ideal in any incarnation. >>> >>> >>> Gah! ALL functions are like that! How often do we see questions about >>> max() or split() etc that a close reading of the docs obviate? >> >> >> My point exactly. > > > Huh? Your point is that all APIs are less than ideal because you have to > read the docs to know for certain how they work? > > ~Ethan~ > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
On 06Apr2012 15:19, I wrote: | On 06Apr2012 14:31, Steven D'Aprano wrote: | | Here is a non-monotonic sequence: | | | | 1, 2, 3, 4, 5, 6, 7, 2, 3, 4, 5, 6, 7, 8 | | | | This isn't steady either, because it jumps backwards. | | | | To be steady, it MUST also be monotonic. If you think that it is appropriate | | to call a non-monotonic clock "steady", then I think you should tell us what | | you mean by "steady but not monotonic". | | I took steady to mean "never jumps more than x", for "x" being "small", | and allowing small negatives. If steady implies monotonic and people | agree that that is so, I'm happy too, and happy that steady is a better | aspiration than merely monotonic. I've had some sleep. _Of course_ steady implies monotonic, or it wouldn't steadily move forwards. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ I went to see a psychiatrist. He told me I was crazy. I told him I wanted a second opinion, so he said, "Ok, you're ugly, too." - Rodney Dangerfield ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 4 April 2012 10:33, Steven D'Aprano wrote:
> try:
>from os import bestclock as _bestclock
> except ImportError:
>_bestclock = time
>
My problem here is that "best clock" means different things to different
people (as the number of emails shows).
I think exposing specific clocks is also useful (sometimes people may need
a steady clock, and early failure is better than clock skew). However, I
propose a loosely classified set of clocks built on top of the specific
clocks, all of which can fall back to the lowest precision/non-monotonic
clock if needed.
1. The "steadiest" clock on the system. Ideally this would be a steady
clock, but may not be.
2. The "most precise" clock on the system. This would have the finest-grain
tick available on the system.
3. The "highest performance" (or maybe "lowest latency") clock. This would
be whichever clock on the system returned its results fastest.
I'm not sure if there are more that would be needed ("most accurate" comes
to mind, but feels like it's arbitrarily choosing between steadiest and
most precise, so I don't think it's valid).
By choosing relative terms, it caters to people's desire to have the "best"
clock, but doesn't set the expectation that the behaviour is guaranteed.
Tim Delaney
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
On 06Apr2012 17:07, Kristj�n Valur J�nsson wrote: | Steven D'Aprano: | > I think that this is incorrect. | > py> time.clock(); time.sleep(10); time.clock() | > 0.41 | > 0.41 | | This is the original reason for the original defect (issue 10278) | unix' clock() doesn't actually provide a clock in this sense, it provides a resource usage metric. Yeah:-( Its help says "Return the CPU time or real time since [...]". Two very different things, as demonstrated. I suppose neither goes backwards, but this seems like a classic example of the "useless monotonic clock" against which Greg Ewing railed. And why? For one thing, because one can't inspect its metadata to find out what it does. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Tens of thousands of messages, hundreds of points of view. It was not called the Net of a Million Lies for nothing. - Vernor Vinge, _A Fire Upon The Deep_ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
> 1. The "steadiest" clock on the system. Ideally this would be a steady > clock, but may not be. time.monotonic() as proposed in the PEP 418 *is* the steadiest available clock, but we cannot say that it is steady :-) > 2. The "most precise" clock on the system. This would have the finest-grain > tick available on the system. It's discussed in the "Deferred API: time.perf_counter()" section. It would be nice to provide such clock, but I don't feel able right now to propose ab API for such requirement. It's unclear to me if it must be monotonic, steady, count elapsed time during a sleep or not, etc. It is already very hard to propose one single time function (time.monotonic), so I chose to simplify the PEP and not propose two functions but only one :-) > 3. The "highest performance" (or maybe "lowest latency") clock. This would > be whichever clock on the system returned its results fastest. Linux provides CLOCK_REALTIME_COARSE and CLOCK_MONOTONIC_COARSE clocks, reading the ACPI Power Management clock is known to be slow. But should the clock be monotonic or not? Return seconds or CPU ticks? If the clock is not well defined, it's useless or at least, not portable. Exposing CLOCK_REALTIME_COARSE and CLOCK_MONOTONIC_COARSE constants should be enough. Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
On 06Apr2012 20:25, Steven D'Aprano wrote:
| Cameron Simpson wrote:
| > My core objective was to allow users to query for clocks, and ideally
| > enumerate and inspect all clocks. Without the caller having platform
| > specific knowledge.
|
| Clocks *are* platform specific -- not just in their availability, but also in
| the fine details of their semantics and behaviour. I don't think we can or
| should try to gloss over this.
This is why get_clock() returns a clock object, which can have metadata
exposing such details. Up to and including the name of the platform specific
library/system-call at its core.
The issue with monotonic() on its own is that the guarentees in the doco
will have to be fairly loose. That prevents the user learning about
"fine details of their semantics and behaviour". Glossing over this
stuff is exactly what offering _only_ a few genericly characterised
clock names (monotonic() et al) does.
| If people are making decisions about timers
| without knowledge of what their platform supports, they're probably making
| poor decisions. Even the venerable time.time() and time.clock() differ
between
| Linux and Windows.
time.clock() does, as (you?) clearly demonstrated elsewhere.
time.time()? (Aside from precision?)
| > Allowing for the sake of discussion that this is desirable, what would
| > you propose as an API instead of get_clock() (and its friend, get_clocks()
| > for enumeration, that I should stuff into the code).
|
| The old ways are the best. We don't have math.get_trig() and math.get_trigs()
| functions for querying trigonometric functions, we just expose the functions
| directly.
|
| I think the way to enumerate and inspect all clocks is with the tried and
true
| Python introspection tools that people use on all other functions:
|
| * use dir(time) to see a list of names available in the module
So, they see "monotonic". Does that tell them much about fine details?
| * use help(time) to read their help
Useful only to humans, not programs.
| * read the Fine Manual to find out more
Useful only to humans, not programs.
| * use try... except... to detect the existence of a clock
Useful only for a fixed list of defined name. Works fine for monotonic,
highres, steady or whatever. And I would be ok with the module
presenting these only where available and concealing them otherwise,
thus raising AttributeError. Or ImportError ("from time import
monotonic").
| There's nothing special about clocks that needs anything more than this.
This I think is false. In fact, I think your own statement at the start
about glossing over fine details goes against this.
If I ask for a highres clock, I might well care _how_ precise it was.
If I ask for a steady clock, I might well care how large its slews were.
If I ask for a monotonic clock, I might well want to know if it tracks
wall clock time (even if by magic) or elapsed system run time (eg time
that stops increasing if the system is suspended, whereas wallclocks do
not). Example: a wallclock is nice for log timestamps. A system run time
clock is nice for profiling. They're both monotonic in some domain.
| get_clock() looks like a factory function, but it actually isn't. It just
| selects from a small number of pre-existing clocks.
That number may still be a few. Victor's made it clear that Windows
has a choice of possible highres clocks, UNIX clock_getres() offers
several possible clock behaviours and an indication that a platform may
have several clocks embodying a subset of these, and may indeed offer
more clocks.
| We should just expose
| those pre-existing clocks directly.
But exposing them _purely_ _by_ _name_ means inventing names for every single
platform clock, and knowing those names per platform. time.clock() is a
fine example where the name tells you nearly nothing about the clock
behaviour. If the user cares about fine detail as you suggest they need
to know their platform and have _external_ knowledge of the platform
specifics; they can't inspect from inside the program.
| I don't see any advantage in adding that
| extra level of indirection or the addition of all this complexity:
| * a function get_clock() to select a clock
| * a function get_clocks() to enumerate all the clocks
These are only two functions because the next alternative seemed an
all_clocks= mode parameter, which changed the signature of the function
return.
Another alternative is the public lists-of-clocks.
The point it to be able to enumerate all available clocks for
consideration of their properties; get_clock() provides a simple way
to coarsely say "a clock like _this_" for the common instances of
"this".
| * another function for querying the properties of a clock
No, that's why you get a clock object back. You can examine it directly
for defined metadata names (epoch, precision, underlying-os-clock-name,
etc). In exactly the fashion you appear to want for the top level
offerings: by knowing the metadata property names.
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
> | This is the original reason for the original defect (issue 10278)
> | unix' clock() doesn't actually provide a clock in this sense, it provides a
> resource usage metric.
>
> Yeah:-( Its help says "Return the CPU time or real time since [...]".
> Two very different things, as demonstrated. I suppose neither goes
> backwards, but this seems like a classic example of the "useless
> monotonic clock" against which Greg Ewing railed.
>
> And why? For one thing, because one can't inspect its metadata to find
> out what it does.
Should I add another key to the result of
time.get_clock_info('clock')? How can we define "clock on Windows"
(almost monotonic and steady clock) vs "clock on UNIX" (CPU time) with
a flag or a value?
Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] this is why we shouldn't call it a "monotonic clock" (was: PEP 418 is too divisive and confusing and should be postponed)
> 2. Those who think that "monotonic clock" means a clock that never > jumps, and that runs at a rate approximating the rate of real time. > This is a very useful kind of clock to have! It is what C++ now calls > a "steady clock". It is what all the major operating systems provide. For the "C++" part, I suppose that you are thinking to: "Objects of class steady_clock represent clocks for which values of time_point advance at a steady rate relative to real time. That is, the clock may not be adjusted." http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3128.html#time.clock.steady I don't understand this definition. All clocks have a clock drift. This is just one exception: atomic clocks, but such clocks are rare and very expensive. http://www.clocktypes.com/buy_atomic_clocks.html "Atomic clocks can have a high price, but if you really want to buy one there is at least one place you can purchase an atomic clock. Agilent Technologies (www.agilient.com) Model number 5071A atomic clock with a long-term stability better than 1 x 10-14 price - $50,390" There is a simple "trick" to get a very cheap steady clock: adjust the clock manually. Extract of a Wikipedia article: "More advanced clocks and old mechanical clocks often have some kind of speed trimmer where one can adjust the speed of the clock and thus reduce the clock drift. For instance, in pendulum clocks the clock drift can be manipulated by slightly changing the length of the pendulum." http://en.wikipedia.org/wiki/Clock_drift Or you can use a NTP daemon to adjust automatically using a free farm of atomic clocks distributed around the world. So you can get a cheap steady clock if you accept that (OMG!) it can be adjusted. Or did I misunderstand "the clock may not be adjusted"? Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 4/5/2012 6:32 AM, Victor Stinner wrote: Since the only monotonic clock that can be adjusted by NTP is Linux' CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always give a clock that isn't adjusted by NTP. I thought we decided that NTP adjustment isn't an issue, because it's always gradual. Well, in timings it is an issue, but perhaps not a big one. :-) In any case, which one we use will not change the API, so if it is decided it is an issue, we can always more to CLOCK_MONOTONIC_RAW in the future, once Linux< 2.6.26 (or whatever it was) is deemed unsupported. I prefer to use CLOCK_MONOTONIC, not because it is also available for older Linux kernels, but because it is more reliable. Even if the underlying clock source is unstable (unstable frequency), a delta of two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or smaller than a second. time.monotonic() unit is the second, as written in its documentation. I believe the above is only true for sufficiently large time deltas. One of the major purposes of NTP slewing is to give up some short term accuracy in order to achieve long term accuracy (e.g. whenever the clock is found to be ahead of real time it is purposefully ticked slower than real time). So for benchmarking it would not be surprising to be better off with the non-adjusted clock. Ideally there would be a clock that was slewed "just enough" to try and achieve short term accuracy, but I don't know of anything providing that. Janzert ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
On 4/6/2012 4:11 PM, Cameron Simpson wrote: Another alternative is the public lists-of-clocks. After watching this thread with amusement and frustration, amusement because it is so big, and so many people have so many different opinions, frustration, because it seems that few of the clocks that are available are anywhere near ideal for any particular stated characteristic, and because none of the APIs presented provide a way for the user to specify the details of the characteristics of the desired clock, I think this idea of a list-of-clocks sounds better and better. Hopefully, for each system, the characteristics of each clock can be discovered, and fully characterized in available metadata for the clock... tick rate, or list of tick rates maximum variation of tick rate precision maximum "helicopter drop" jump delta monotonicity frequency of rollover or None base epoch value or None behavior during system sleep, hibernate, suspend, shutdown, battery failure, flood, wartime events, and acts of God. These last two may have values that are long prose texts full of political or religious rhetoric, such as the content of this thread :) any other characteristics I forgot to mention Of course, it is not clear that all of these characteristics can be determined based on OS/Version; hardware vendors may have different implementations. There should be a way to add new clock objects to the list, given a set of characteristics, and an API to retrieve them, at least by installing a submodule that provides access to an additional clock. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
On 06Apr2012 17:30, Glenn Linderman wrote: | On 4/6/2012 4:11 PM, Cameron Simpson wrote: | > Another alternative is the public lists-of-clocks. | | After watching this thread with amusement and frustration, amusement | because it is so big, and so many people have so many different | opinions, frustration, because it seems that few of the clocks that are | available are anywhere near ideal for any particular stated | characteristic, My partner has occasionally opined that most Prolog programs simply result in "*** NO ***". We could optimise for that and simplify the implementation enormously. It would also let us provide very strong guarrentees about the offered clocks on the basis that no suitable clock would ever provided:-) | and because none of the APIs presented provide a way for | the user to specify the details of the characteristics of the desired | clock, I think this idea of a list-of-clocks sounds better and better. | | Hopefully, for each system, the characteristics of each clock can be | discovered, and fully characterized in available metadata for the clock... Victor has asked me to do that for my skeleton, based on the tables he has assembled. I'll see what i can do there... | Of course, it is not clear that all of these characteristics can be | determined based on OS/Version; hardware vendors may have different | implementations. If you can look up the kernel revision you can do fairly well. In principle. | There should be a way to add new clock objects to the list, given a set | of characteristics, and an API to retrieve them, at least by installing | a submodule that provides access to an additional clock. Returning to seriousness, the get_clock() call admits a clocklist. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Principles have no real force except when one is well fed. - Mark Twain ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
