Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
On Tue, Feb 14, 2012 at 2:25 PM, Eli Bendersky wrote: > With the deprecation warning being silent, is there much to lose, though? Yes, it creates problems for anyone that deliberately converts all warnings to errors when running their test suites. This forces them to spend time switching over to a Python version dependent import of either cElementTree or ElementTree that could have been spent doing something actually productive instead of mere busywork. And, of course, even people that *don't* convert warnings to errors when running tests will have to make the same switch when the module is eventually removed. > Cleanups help lower the clutter and mental burden on maintainers in the long > run. If nothing is ever cleaned up don't we end up with PHP :-) ? It's a balancing act, sure. But when the maintenance burden for us is low and the cost to third parties clear, documented deprecation for eventual removal in the next release series is the better choice. >> Programmatic deprecation is a significant imposition on third party >> developers and should really be reserved for APIs that actively >> encourage writing broken code (e.g. contextlib.nested) or are >> seriously problematic for python-dev to maintain. For cleanup stuff, >> documented deprecation is sufficient. > > A quick search of the sources for DeprecationWarning show that it's being > used much more liberally than solely for stuff that encourages writing > broken code. Has there been a recent policy change with regards to what's > considered deprecated? It's always been judged on a case-by-case basis, but yes, there's been a deliberate push in favour of purely documented deprecations in recent years (initially mostly from Raymond Hettinger, more recently from me as well as I came to appreciate the merit of Raymond's point of view). It mainly started with the decision to leave optparse alone (aside from a deprecation note in the docs) even after argparse was added to the standard library. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
> Currently in 3.3 the whole cElementTree module is: > > > > # Deprecated alias for xml.etree.ElementTree > > > > from xml.etree.ElementTree import * > > > > Would it be alright to issue a DeprecationWarning if this module is > > imported? Then hopefully a couple of releases after 3.3 we can just dump > it. > > What do we really gain by dumping it, though? Just add a CPython > specific test that ensures: > >for key, value in xml.etree.ElementTree.__dict__.items(): >self.assertIs(getattr(xml.etree.cElementTree, key), value) > > and then ignore it for the next decade or so. > > With the deprecation warning being silent, is there much to lose, though? Cleanups help lower the clutter and mental burden on maintainers in the long run. If nothing is ever cleaned up don't we end up with PHP :-) ? > Programmatic deprecation is a significant imposition on third party > developers and should really be reserved for APIs that actively > encourage writing broken code (e.g. contextlib.nested) or are > seriously problematic for python-dev to maintain. For cleanup stuff, > documented deprecation is sufficient. > > A quick search of the sources for DeprecationWarning show that it's being used much more liberally than solely for stuff that encourages writing broken code. Has there been a recent policy change with regards to what's considered deprecated? Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
On Tue, Feb 14, 2012 at 1:42 PM, Eli Bendersky wrote: > An open question remains on whether to deprecate cElementTree, now that this > change is in place. > > Currently in 3.3 the whole cElementTree module is: > > # Deprecated alias for xml.etree.ElementTree > > from xml.etree.ElementTree import * > > Would it be alright to issue a DeprecationWarning if this module is > imported? Then hopefully a couple of releases after 3.3 we can just dump it. What do we really gain by dumping it, though? Just add a CPython specific test that ensures: for key, value in xml.etree.ElementTree.__dict__.items(): self.assertIs(getattr(xml.etree.cElementTree, key), value) and then ignore it for the next decade or so. Programmatic deprecation is a significant imposition on third party developers and should really be reserved for APIs that actively encourage writing broken code (e.g. contextlib.nested) or are seriously problematic for python-dev to maintain. For cleanup stuff, documented deprecation is sufficient. Something that might be worth doing (although it would likely scare the peanut gallery) is to create a PEP 4000 to record the various cleanup tasks (like dropping cElementTree) that are being deliberately excluded from the 3.x series. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
> > The change was committed to the default branch. In 3.3, "import > xml.etree.ElementTree" will automatically use the _elementtree accelerator, > if available, and will fall back to a Python implementation otherwise. The > documentation of ElementTree has also been updated to reflect this fact. > An open question remains on whether to deprecate cElementTree, now that this change is in place. Currently in 3.3 the whole cElementTree module is: # Deprecated alias for xml.etree.ElementTree from xml.etree.ElementTree import * Would it be alright to issue a DeprecationWarning if this module is imported? Then hopefully a couple of releases after 3.3 we can just dump it. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)
On Tue, Feb 14, 2012 at 8:08 AM, Barry Warsaw wrote: > On Feb 13, 2012, at 12:31 PM, Nick Coghlan wrote: > >>I think Antoine makes a good point about ease of introspection when >>you have multiple versions in the same series installed, so I'd be >>fine with: >>- updating the PEP recommendation to say that either form of link is >>fine (with hard links marginally faster, but harder to introspect) >>- noting that python.org releases will consistently use symlinks for >>easier introspection via "ls -l" >>- updating Makefile.pre.in to ensure that we really do consistently use >>symlinks > > +1, and +1 for the PEP to be accepted. > >>This does mean that launching Python may involve a slightly longer >>symlink chain in some cases (python -> python2 -> python2.7), but the >>impact of that is always going to be utterly dwarfed by other startup >>costs. > > Agreed about startup times. However, does the symlink chain have to go in > this order? Couldn't python -> python2.7 and python2 -> python2.7? OTOH, I > seriously doubt removing one level of symlink chasing will have any noticeable > effect on startup times. I considered that, but thought it would be odd to make people double-key a manual default version change within a series. It seemed more logical to have "python" as a binary "python2/3" switch and then have the python2/3 symlinks choose which version is the default for that series. (I'll add that rationale to the PEP, though) > One other thing I'd like to see the PEP address is a possible migration > strategy to python->python3. Even if that strategy is "don't do it, man!". > IOW, can a distribution change the 'python' symlink once it's pointed to > python2? What is the criteria for that? Is it up to a distribution? Will > the PEP get updated when our collective wisdom says its time to change the > default? etc. I have no idea, and I'm not going to open that can of worms for this PEP. We need to say something about the executable aliases so that people can eventually write cross-platform python2 shebang lines, but how particular distros actually manage the transition is going to depend more on their infrastructure and community than it is anything to do with us. > Also, if Python 2.7 is being changed to add this feature, why can't Python 3.2 > also be changed? (And if there's a good reason for not doing it there, that > should be added to the PEP.) Because Python 3.2 already installs itself as python3 and doesn't touch the python symlink. Aside from potentially cleaning up the choice of symlinks vs hardlinks in a couple of cases, the PEP really doesn't alter Python 3 deployment at all. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)
On Tue, Feb 14, 2012 at 7:07 AM, "Martin v. Löwis" wrote: >> I think Antoine makes a good point about ease of introspection when >> you have multiple versions in the same series installed, so I'd be >> fine with: >> - updating the PEP recommendation to say that either form of link is >> fine (with hard links marginally faster, but harder to introspect) >> - noting that python.org releases will consistently use symlinks for >> easier introspection via "ls -l" >> - updating Makefile.pre.in to ensure that we really do consistently use >> symlinks > > Sounds fine to me as well. When you update the PEP, please also update > the mark with the actual issue number (or add it to the References). Hmm, the PEP builder on python.org may need a kick. I added the tracker reference before starting this thread (http://hg.python.org/peps/rev/78b94f8648fa), but didn't comment on it since I expected the site to update in fairly short order. > For the patch, it seems that one open issue is OSX support, although > I'm unsure what exactly the issue is. I don't know either, but I'll take Ned's word for it if he says there's something more he needs to do to make it work in the OS X binaries. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review
On Tue, Feb 14, 2012 at 4:33 AM, Victor Stinner wrote: >> However, I am still -1 on the solution proposed by the PEP. I still think >> that migrating to datetime use is a better way to go, rather than a >> proliferation of the data types used to represent timestamps, along with an >> API to specify the type of data returned. >> >> Let's look at each item in the PEPs rationale for discarding the use of >> datetimes: > > Oh, I forgot to mention my main concern about datetime: many functions > returning timestamp have an undefined starting point (an no timezone > information ), and so cannot be converted to datetime: > - time.clock(), time.wallclock(), time.monotonic(), > time.clock_gettime() (except for CLOCK_REALTIME) > - time.clock_getres() > - signal.get/setitimer() > - os.wait3(), os.wait4(), resource.getrusage() > - etc. > > Allowing datetime.datetime type just for few functions (like > datetime.datetime or time.time) but not the others (raise an > exception) is not an acceptable solution. A datetime module based approach would need to either use a mix of datetime.datetime() (when returning an absolute time) and datetime.timedelta() (when returning a time relative to an unknown starting point), or else just always return datetime.timedelta (even when we know the epoch and could theoretically make the time absolute). In the former case, it may be appropriate to adopt a boolean flag API design and the "I want high precision time" request marker would just be "datetime=True". You'd then get back either datetime.datetime() or datetime.timedelta() as appropriate for the specific API. In the latter case, the design would be identical to the current PEP, only with "datetime.timedelta" in place of "decimal.Decimal". The challenge relative to the current PEP is that any APIs that wanted to *accept* either of these as a timestamp would need to do some specific work to avoid failing with a TypeError. For timedelta values, we'd have to define a way to easily extract the full precision timestamp as a number (total_seconds() currently returns a float, and hence can't handle nanosecond resolutions), as well as improving interoperability with algorithms that expected a floating point value. If handed a datetime value, you need to know the correct epoch value, do the subtraction, then extract the full precision timestamp from the resulting timedelta object. To make a datetime module based counter-proposal acceptable, it would need to be something along the following lines: - to avoid roundtripping problems, only return timedelta() (even for cases where we know the epoch and could theoretically return datetime instead) - implement __int__ and __float__ on timedelta (where the latter is just "self.total_seconds()" and the former "int(self.total_seconds())") It may also take some fancy footwork to avoid a circular dependency between time and datetime while supporting this (Victor allowed this in an earlier version of his patch, but he did it by accepting datetime.datetime and datetime.time_delta directly as arguments to the affected APIs). That's a relatively minor implementation concern, though (at worst it would require factoring out a support module used by both datetime and time). The big problem is that datetime and timedelta pose a huge problem for compatibility with existing third party APIs that accept timestamp values. This is in stark contrast to what happens with decimal.Decimal: coercion to float() or int() will potentially lose precision, but still basically works. While addition and subtraction of floats will fail, addition and subtraction of integers works fine. To avoid losing precision, it's sufficient to just avoid the coercion. I think the outline above really illustrates why the *raw* data type for timestamps should just be a number, not a higher level semantic type like timedelta or datetime. Eventually, you want to be able to express a timestamp as a number of seconds relative to a particular epoch. To do that, you want a number. Originally, we used ints, then, to support microsecond resolution, we used floats. The natural progression to support arbitrary resolutions is to decimal.Decimal. Then, the higher level APIs can be defined in *terms* of that high precision number. Would it be nice if there was a PyPI module that provided APIs that converted the raw timestamps in stat objects and other OS level APIs into datetime() and timedelta() objects as appropriate? Perhaps, although I'm not sure it's necessary. But are those types low-level enough to be suitable for the *OS* interface definition? I don't think so - we really just want a number to express "seconds since a particular time" that plays fairly nicely with other numbers, not anything fancier than that. Notice that PEP 410 as it stands can be used to *solve* the problem of how to extract the full precision timestamp from a timedelta object as a number: timedelta.total_seconds() can be updated to accept a "timestamp" argument,
Re: [Python-Dev] peps: Update with bugfix releases.
In article , Ned Deily wrote: > In article , > "Russell E. Owen" wrote: > > One problem I've run into is that the 64-bit Mac python 2.7 does not > > work properly with ActiveState Tcl/Tk. One symptom is to build > > matplotlib. The results fail -- both versions of Tcl/Tk somehow get > > linked in. > > The 64-bit OS X installer is built on and tested on systems with A/S > Tcl/Tk 8.5.x and we explicitly recommend its use when possible. > > http://www.python.org/download/mac/tcltk/ > > Please open a python bug for this and any other issues you know of > regarding the use with current A/S Tcl/Tk 8.5.x with current 2.7.x or > 3.2.x installers on OS X 10.6 or 10.7. Yes. I apologize. See the discussion in the Mac python mailing list (I replied to your email there). I was trying to build a matplotlib binary installer and ran into problems. I don't know where the problem comes from, and it may well not have anything to do with the python build. -- Russell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)
On Feb 13, 2012, at 12:31 PM, Nick Coghlan wrote: >I think Antoine makes a good point about ease of introspection when >you have multiple versions in the same series installed, so I'd be >fine with: >- updating the PEP recommendation to say that either form of link is >fine (with hard links marginally faster, but harder to introspect) >- noting that python.org releases will consistently use symlinks for >easier introspection via "ls -l" >- updating Makefile.pre.in to ensure that we really do consistently use >symlinks +1, and +1 for the PEP to be accepted. >This does mean that launching Python may involve a slightly longer >symlink chain in some cases (python -> python2 -> python2.7), but the >impact of that is always going to be utterly dwarfed by other startup >costs. Agreed about startup times. However, does the symlink chain have to go in this order? Couldn't python -> python2.7 and python2 -> python2.7? OTOH, I seriously doubt removing one level of symlink chasing will have any noticeable effect on startup times. One other thing I'd like to see the PEP address is a possible migration strategy to python->python3. Even if that strategy is "don't do it, man!". IOW, can a distribution change the 'python' symlink once it's pointed to python2? What is the criteria for that? Is it up to a distribution? Will the PEP get updated when our collective wisdom says its time to change the default? etc. Also, if Python 2.7 is being changed to add this feature, why can't Python 3.2 also be changed? (And if there's a good reason for not doing it there, that should be added to the PEP.) Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review
Antoine Pitrou conviced me to drop simply the int type: float and Decimal are just enough. Use an explicit cast using int() to get int. os.stat_float_times() is still deprecated by the PEP. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)
> I think Antoine makes a good point about ease of introspection when > you have multiple versions in the same series installed, so I'd be > fine with: > - updating the PEP recommendation to say that either form of link is > fine (with hard links marginally faster, but harder to introspect) > - noting that python.org releases will consistently use symlinks for > easier introspection via "ls -l" > - updating Makefile.pre.in to ensure that we really do consistently use > symlinks Sounds fine to me as well. When you update the PEP, please also update the mark with the actual issue number (or add it to the References). For the patch, it seems that one open issue is OSX support, although I'm unsure what exactly the issue is. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review
> However, I am still -1 on the solution proposed by the PEP. I still think > that migrating to datetime use is a better way to go, rather than a > proliferation of the data types used to represent timestamps, along with an > API to specify the type of data returned. > > Let's look at each item in the PEPs rationale for discarding the use of > datetimes: Oh, I forgot to mention my main concern about datetime: many functions returning timestamp have an undefined starting point (an no timezone information ), and so cannot be converted to datetime: - time.clock(), time.wallclock(), time.monotonic(), time.clock_gettime() (except for CLOCK_REALTIME) - time.clock_getres() - signal.get/setitimer() - os.wait3(), os.wait4(), resource.getrusage() - etc. Allowing datetime.datetime type just for few functions (like datetime.datetime or time.time) but not the others (raise an exception) is not an acceptable solution. > I'm looking at a use case from my flufl.lock library: > >return datetime.datetime.fromtimestamp( >os.stat(self._lockfile).st_mtime) Keep your code but just add timestamp=decimal.Decimal argument to os.stat() to get high-resolution timestamps! (well, you would at least avoid loss of precision loss if datetime is not improved to support nanosecond.) > * datetime.datetime has ordering issues with daylight saving time (DST) in > the duplicate hour of switching from DST to normal time. > > Sure, but only for timezone-ful datetimes, right? I don't know enough this topic to answer. Martin von Loewis should answer to this question! > * datetime.datetime is not as well integrated than Epoch timestamps, some > functions don't accept this type as input. For example, os.utime() expects > a tuple of Epoch timestamps. > > So, by implication, Decimal is better integrated by virtue of its ability to > be coerced to floats and other numeric stack types? Yes. decimal.Decimal is already supported by all functions accepting float (all functions expecting timestamps). > Will users ever have to explicitly convert Decimal types to use other APIs? Sorry, I don't understand. What do you mean? > It bothers me that the PEP is proposing that users will now have to be > prepared to handle yet another (and potentially *many* more) data types coming > from what are essentially datetime-like APIs. Users only get decimal.Decimal if they ask explicitly for decimal.Decimal. By default, they will still get float. Most users don't care of nanoseconds :-) If a library choose to return Decimal instead of float, it's a change in the library API unrelated to the PEP. > If it really is impossible or suboptimal to build high resolution datetimes > and timedeltas, and to use them in these APIs, then at the very least, the PEP > needs a stronger rationale for why this is. IMO supporting nanosecond in datetime and timedelta is an orthogonal issue. And yes, the PEP should maybe give better arguments against datetime :-) I will update the PEP to mention the starting point issue. > In any case, thanks for your work in this (and so many other!) areas. You're welcome :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review
On Feb 13, 2012, at 01:28 AM, Victor Stinner wrote: >I'm still waiting for Nick Coghlan and Guido van Rossum for their >decision on the PEP. Thanks for continuing to work on this Victor. I agree with the general motivation behind the PEP, and appreciate your enthusiasm for improving Python here. However, I am still -1 on the solution proposed by the PEP. I still think that migrating to datetime use is a better way to go, rather than a proliferation of the data types used to represent timestamps, along with an API to specify the type of data returned. Let's look at each item in the PEPs rationale for discarding the use of datetimes: * datetime.datetime only supports microsecond resolution, but can be enhanced to support nanosecond. Great! JFDI! * datetime.datetime has issues with timezone. For example, a datetime object without timezone and a datetime with a timezone cannot be compared. This may be so, but I don't think it causes fatal problems with this approach. The APIs returning high resolution datetimes should return naive (i.e. timezone-less) datetimes. If this is done consistently, then most math on such datetimes (along with high resolution timedeltas) should Just Work. I'm looking at a use case from my flufl.lock library: return datetime.datetime.fromtimestamp( os.stat(self._lockfile).st_mtime) and later, this value is compared: datetime.datetime.now() > release_time So with higher resolution naive datetimes, this would still work. I think it's fine if the user wants to mix naive and timezone-ful datetimes, they will have to resolve the compatibility issues, but I think that will be the minority of cases. So this issue should not be a blocker for high resolution datetimes (and timedeltas). * datetime.datetime has ordering issues with daylight saving time (DST) in the duplicate hour of switching from DST to normal time. Sure, but only for timezone-ful datetimes, right? I can live with that, since I have to live with that for all non-naive datetimes anyway, and as I mentioned, I don't think in general it will be a practical problem when using high resolution datetime.s * datetime.datetime is not as well integrated than Epoch timestamps, some functions don't accept this type as input. For example, os.utime() expects a tuple of Epoch timestamps. So, by implication, Decimal is better integrated by virtue of its ability to be coerced to floats and other numeric stack types? Will users ever have to explicitly convert Decimal types to use other APIs? I don't think this one is insurmountable either. We could certainly improve the compatibility of datetimes with other APIs, and in fact, I think we should regardless of which direction this PEP takes. You could even argue for EIBTI in converting from datetimes to types acceptable to those other APIs, many of which derive their argument types by virtue of the C APIs underneath. It bothers me that the PEP is proposing that users will now have to be prepared to handle yet another (and potentially *many* more) data types coming from what are essentially datetime-like APIs. If it really is impossible or suboptimal to build high resolution datetimes and timedeltas, and to use them in these APIs, then at the very least, the PEP needs a stronger rationale for why this is. But I think ultimately, it would be better for Python to improve the resolution, and API support for datetimes and timestamps. In any case, thanks for your work in this (and so many other!) areas. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
Eli Bendersky, 13.02.2012 12:35: >> Since there appeared to be an overall positive response for making >> this change in Python 3.3, and since there isn't longer any doubt >> about the ownership of the package *in Python's stdlib* (see >> http://mail.python.org/pipermail/python-dev/2012-February/116389.html), >> I've opened issue 13988 on the bug tracker to follow the >> implementation. > > The change was committed to the default branch. In 3.3, "import > xml.etree.ElementTree" will automatically use the _elementtree accelerator, > if available, and will fall back to a Python implementation otherwise. The > documentation of ElementTree has also been updated to reflect this fact. Thanks! Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] How to round timestamps and durations?
Hi, My work on the PEP 410 tries to unify the code to manipulate timestamps. The problem is that I'm unable to decide how to round these numbers. Functions using a resolution of 1 second (e.g. time.mktime) expects rounding towards zero (ROUND_HALF_DOWN), as does int(float). Example: >>> time.mktime(time.localtime(-1.9)), time.mktime(time.localtime(1.9)) (-1.0, 1.0) datetime.datetime.fromtimestamp() rounds to nearest with ties going away from zero (ROUND_HALF_UP). Example: >>> datetime.datetime.fromtimestamp(-1.1e-6), >>> datetime.datetime.fromtimestamp(1.1e-6) (datetime.datetime(1970, 1, 1, 0, 59, 59, 99), datetime.datetime(1970, 1, 1, 1, 0, 0, 1)) >>> datetime.datetime.fromtimestamp(-1.9e-6), >>> datetime.datetime.fromtimestamp(1.9e-6) (datetime.datetime(1970, 1, 1, 0, 59, 59, 98), datetime.datetime(1970, 1, 1, 1, 0, 0, 2)) datetime.timedelta * float and datetime.timedelta / float rounds to nearest with ties going to nearest even integer (ROUND_HALF_EVEN), as does round(). Example: >>> [(datetime.timedelta(microseconds=x) / 2.0).microseconds for x in range(6)] [0, 0, 1, 2, 2, 2] Should I also support multiple rounding methods depending on the operation and of the Python function? Should we always use the same rounding method? Antoine pointed me that ROUND_HALF_UP can produce timestamps "in the future", which is especially visible when using a resolution of 1 second. I like this rounding method because it limits the loss of precision to an half unit: abs(rounded - timestamp) <= 0.5. But it can be "surprising". The rounding method should maybe be the same than int(float) (so ROUND_HALF_DOWN) to avoid surprising results for applications using int(time.time()) for example (I had such problem with rotated logs and test_logging). -- There is an issue on rounding timedelta: http://bugs.python.org/issue8860 Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for new dictionary implementation
Revised PEP for new dictionary implementation, PEP 412?
is attached.
Cheers,
Mark.
PEP: XXX
Title: Key-Sharing Dictionary
Version: $Revision$
Last-Modified: $Date$
Author: Mark Shannon
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 08-Feb-2012
Python-Version: 3.3 or 3.4
Post-History: 08-Feb-2012
Abstract
This PEP proposes a change in the implementation of the builtin dictionary
type ``dict``. The new implementation allows dictionaries which are used as
attribute dictionaries (the ``__dict__`` attribute of an object) to share
keys with other attribute dictionaries of instances of the same class.
Motivation
==
The current dictionary implementation uses more memory than is necessary
when used as a container for object attributes as the keys are
replicated for each instance rather than being shared across many instances
of the same class.
Despite this, the current dictionary implementation is finely tuned and
performs very well as a general-purpose mapping object.
By separating the keys (and hashes) from the values it is possible to share
the keys between multiple dictionaries and improve memory use.
By ensuring that keys are separated from the values only when beneficial,
it is possible to retain the high-performance of the current dictionary
implementation when used as a general-purpose mapping object.
Behaviour
=
The new dictionary behaves in the same way as the old implementation.
It fully conforms to the Python API, the C API and the ABI.
Performance
===
Memory Usage
Reduction in memory use is directly related to the number of dictionaries
with shared keys in existence at any time. These dictionaries are typically
half the size of the current dictionary implementation.
Benchmarking shows that memory use is reduced by 10% to 20% for
object-oriented programs with no significant change in memory use
for other programs.
Speed
-
The performance of the new implementation is dominated by memory locality
effects. When keys are not shared (for example in module dictionaries
and dictionary explicitly created by dict() or {} ) then performance is
unchanged (within a percent or two) from the current implementation.
For the shared keys case, the new implementation tends to separate keys
from values, but reduces total memory usage. This will improve performance
in many cases as the effects of reduced memory usage outweigh the loss of
locality, but some programs may show a small slow down.
Benchmarking shows no significant change of speed for most benchmarks.
Object-oriented benchmarks show small speed ups when they create large
numbers of objects of the same class (the gcbench benchmark shows a 10%
speed up; this is likely to be an upper limit).
Implementation
==
Both the old and new dictionaries consist of a fixed-sized dict struct and
a re-sizeable table.
In the new dictionary the table can be further split into a keys table and
values array.
The keys table holds the keys and hashes and (for non-split tables) the
values as well. It differs only from the original implementation in that it
contains a number of fields that were previously in the dict struct.
If a table is split the values in the keys table are ignored, instead the
values are held in a separate array.
Split-Table dictionaries
When dictionaries are created to fill the __dict__ slot of an object, they are
created in split form. The keys table is cached in the type, potentially
allowing all attribute dictionaries of instances of one class to share keys.
In the event of the keys of these dictionaries starting to diverge,
individual dictionaries will lazily convert to the combined-table form.
This ensures good memory use in the common case, and correctness in all cases.
When resizing a split dictionary it is converted to a combined table.
If resizing is as a result of storing an instance attribute, and there is
only instance of a class, then the dictionary will be re-split immediately.
Since most OO code will set attributes in the __init__ method, all attributes
will be set before a second instance is created and no more resizing will be
necessary as all further instance dictionaries will have the correct size.
For more complex use patterns, it is impossible to know what is the best
approach, so the implementation allows extra insertions up to the point
of a resize when it reverts to the combined table (non-shared keys).
A deletion from a split dictionary does not change the keys table, it simply
removes the value from the values array.
Combined-Table dictionaries
---
Explicit dictionaries (dict() or {}), module dictionaries and most other
dictionaries are created as combined-table dictionaries.
A combined-table dictionary never becomes a split-table dictionary.
Combined tables are laid out in much the same way as the tables in the old
dictionary, resulting in very similar performance.
Implementatio
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
> Since there appeared to be an overall positive response for making > this change in Python 3.3, and since there isn't longer any doubt > about the ownership of the package *in Python's stdlib* (see > http://mail.python.org/pipermail/python-dev/2012-February/116389.html), > I've opened issue 13988 on the bug tracker to follow the > implementation. > > The change was committed to the default branch. In 3.3, "import xml.etree.ElementTree" will automatically use the _elementtree accelerator, if available, and will fall back to a Python implementation otherwise. The documentation of ElementTree has also been updated to reflect this fact. Thanks a lot to Florent Xicluna for the great co-operation, and all the others who submitted opinions in the issue. For more details see http://bugs.python.org/issue13988 Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
