Re: [Python-Dev] [python-committers] [RELEASE] Python 3.6.0b1 is now available
On 14 September 2016 at 11:32, Serhiy Storchaka wrote: > On 13.09.16 02:35, Ned Deily wrote: >> >> On behalf of the Python development community and the Python 3.6 release >> team, I'm happy to announce the availability of Python 3.6.0b1. 3.6.0b1 >> is the first of four planned beta releases of Python 3.6, the next major >> release of Python, and marks the end of the feature development phase >> for 3.6. > > > There is no mention on https://www.python.org/news/. The last release mentioned there is 3.4.0rc1... Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
On Sep 9, 2016 1:35 AM, "Benjamin Peterson" wrote: > On Thu, Sep 8, 2016, at 22:33, Tim Delaney wrote: > > Are sets also ordered by default now? None of the PEPs appear to mention > > it. > > No. Is there anyone working to move sets in the same direction for 3.6? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
On 14 September 2016 at 13:18, Franklin? Lee wrote: > On Sep 9, 2016 1:35 AM, "Benjamin Peterson" wrote: >> On Thu, Sep 8, 2016, at 22:33, Tim Delaney wrote: >> > Are sets also ordered by default now? None of the PEPs appear to mention >> > it. >> >> No. > > Is there anyone working to move sets in the same direction for 3.6? It won't happen for 3.6, as we're now in feature freeze. So it'd be 3.7 at the earliest. What exactly do you mean by "in the same direction" anyway? Remember that ordering isn't guaranteed (it's an implementation detail), so are you just saying "can sets benefit from the improvements this change provided to dicts"? If you *are* hoping for ordered sets, what's your use case? (Dictionaries had particular use cases - retaining ordering of keyword arguments and class definitions, the existence of OrderedDict, ...) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
On Sep 14, 2016 8:29 AM, "Paul Moore" wrote: > > On 14 September 2016 at 13:18, Franklin? Lee > wrote: > > On Sep 9, 2016 1:35 AM, "Benjamin Peterson" wrote: > >> On Thu, Sep 8, 2016, at 22:33, Tim Delaney wrote: > >> > Are sets also ordered by default now? None of the PEPs appear to mention > >> > it. > >> > >> No. > > > > Is there anyone working to move sets in the same direction for 3.6? > > It won't happen for 3.6, as we're now in feature freeze. So it'd be > 3.7 at the earliest. > > What exactly do you mean by "in the same direction" anyway? Remember > that ordering isn't guaranteed (it's an implementation detail), so are > you just saying "can sets benefit from the improvements this change > provided to dicts"? If you *are* hoping for ordered sets, what's your > use case? (Dictionaries had particular use cases - retaining ordering > of keyword arguments and class definitions, the existence of > OrderedDict, ...) > > Paul I mean using a compact representation, if not an ordered one. I have no particular usecase in mind. As far as I understand the compact implementation, sets can do it just as well. The original discussion proposed trying to implement it for sets first. Like dict, they would (probably) use less memory, and would usually have a more readable (i.e. less jarring to read) print order. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
> > I mean using a compact representation, if not an ordered one. > > I have no particular usecase in mind. As far as I understand the compact > implementation, sets can do it just as well. The original discussion > proposed trying to implement it for sets first. > > Like dict, they would (probably) use less memory, and would usually have a > more readable (i.e. less jarring to read) print order. > I'll improve OrderedDict after dict in 3.6 is stable enough. Then, I'll do same to sets. While compact ordered split dict is very hard to implement right, OrderedDict and set must be easier than dict. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] [RELEASE] Python 3.6.0b1 is now available
Fortunately that page isn't linked from anywhere on the home page AFAIK. If it is, could someone file an issue in the pydotorg tracker? The url is at the bottom of every page. On Wed, Sep 14, 2016 at 3:41 AM, Paul Moore wrote: > On 14 September 2016 at 11:32, Serhiy Storchaka wrote: >> On 13.09.16 02:35, Ned Deily wrote: >>> >>> On behalf of the Python development community and the Python 3.6 release >>> team, I'm happy to announce the availability of Python 3.6.0b1. 3.6.0b1 >>> is the first of four planned beta releases of Python 3.6, the next major >>> release of Python, and marks the end of the feature development phase >>> for 3.6. >> >> >> There is no mention on https://www.python.org/news/. > > The last release mentioned there is 3.4.0rc1... > > Paul > ___ > python-committers mailing list > python-committ...@python.org > https://mail.python.org/mailman/listinfo/python-committers > Code of Conduct: https://www.python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Code Complexity Survey
Dear Python developers, We have undertaken a task to assess code complexity triggers and generate recommendations for developing simple and understandable code. Our intension is to share the results with you, developers, so everyone can learn the triggers behind complex software. We need your help for rigorous results. My request to you is - if you get 5-10 min. time, would you please consider to answer the questions of this survey? https://goo.gl/forms/h9WXZ8VSEw7BUyHg1 You are welcome to learn preliminary results through this link: https://www.facebook.com/SoftwareCodeQuality/photos/?tab=album&album_id=1639816749664288 The results will be shared in a public webpage and everyone possible will be invited to learn and discuss them. Your knowledge and experience is vital for achieving substantial and generalizable results, and your effort is much appreciated! Sincerely Vard Antinyan PhD candidate in University of Gothenburg, Sweden Tel: 0046317725707 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
On 9/14/2016 9:33 AM, INADA Naoki wrote: I mean using a compact representation, if not an ordered one. I have no particular usecase in mind. As far as I understand the compact implementation, sets can do it just as well. The original discussion proposed trying to implement it for sets first. Like dict, they would (probably) use less memory, and would usually have a more readable (i.e. less jarring to read) print order. I'll improve OrderedDict after dict in 3.6 is stable enough. Then, I'll do same to sets. While compact ordered split dict is very hard to implement right, OrderedDict and set must be easier than dict. Frozensets, and even more, sets, have lots of operations that dicts do not. Making sets more compact without slowing down the various operations should be an interesting challenge. Insert order is not meaningful for at least some of the operations. Moreover, I believe repeated inserts and deletions are much more common for sets than dicts. So I think that any initial ordering should be considered a side-effect and documented as such. The operations would then be optimized for speed and compactness without regard to ordering. This might mean keeping a linked list of free slots so slots can be reused and costly compaction avoided as much and for long as possible. We already have compact mutable collection types that can be kept insert-ordered if one chooses -- lists and collections.deque -- and they are not limited to hashables. Before sets were added, either lists or dicts with None values were used as sets. The latter is obsolete but lists are still sometimes used for their generality, as in a set of lists. We now also have enums for certain small frozensets where the set opertions are not needed. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
On Wed, Sep 14, 2016 at 7:33 AM, INADA Naoki wrote: > I'll improve OrderedDict after dict in 3.6 is stable enough. +1 and if it's done carefully we could even utilize the pure Python OrderedDict and get rid of odictobject.c (and fold dict-common.h back into dictobject.c). We'd need to leave the current implementation as the fallback for implementations that don't have an ordered dict. However, we'd first try a compact-dict-based variant. Doing so would probably require a new field in sys.implementation that indicates dict is ordered. > Then, I'll do same to sets. Unless I've misunderstood, Raymond was opposed to making a similar change to set. -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
On 15 September 2016 at 05:02, Terry Reedy wrote: > > We already have compact mutable collection types that can be kept > insert-ordered if one chooses -- lists and collections.deque -- and they > are not limited to hashables. Before sets were added, either lists or > dicts with None values were used as sets. The latter is obsolete but lists > are still sometimes used for their generality, as in a set of lists. We > now also have enums for certain small frozensets where the set opertions > are not needed. One use case that isn't covered by any of the above is removing duplicates whilst retaining order (of the first of the matching elements). With an OrderedSet (or ordered by default sets) it would be as simple as: a = OrderedSet(iterable) Probably the best current option would be: a = list(OrderedDict(k, None for k in iterable)) The other use I have for an ordered set is to build up an iterable of unique values whilst retaining order. It's a lot more efficient than doing a linear search on a list when adding each element to see if it's already present. In many cases the order is primarily important for debugging purposes, but I can definitely find cases in my current java codebase where I've used the pattern (LinkedHashSet) and the order is important to the semantics of the code. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
> On Sep 14, 2016, at 3:50 PM, Eric Snow wrote: > >> >> Then, I'll do same to sets. > > Unless I've misunderstood, Raymond was opposed to making a similar > change to set. That's right. Here are a few thoughts on the subject before people starting running wild. * For the compact dict, the space savings was a net win with the additional space consumed by the indices and the overallocation for the key/value/hash arrays being more than offset by the improved density of key/value/hash arrays. However for sets, the net was much less favorable because we still need the indices and overallocation but can only offset the space cost by densifying only two of the three arrays. In other words, compacting makes more sense when you have wasted space for keys, values, and hashes. If you lose one of those three, it stops being compelling. * The use pattern for sets is different from dicts. The former has more hit or miss lookups. The latter tends to have fewer missing key lookups. Also, some of the optimizations for the set-to-set operations make it difficult to retain set ordering without impacting performance. * I pursued alternative path to improve set performance. Instead of compacting (which wasn't much of space win and incurred the cost of an additional indirection), I added linear probing to reduce the cost of collisions and improve cache performance. This improvement is incompatible with the compacting approach I advocated for dictionaries. * For now, the ordering side-effect on dictionaries is non-guaranteed, so it is premature to start insisting the sets become ordered as well. The docs already link to a recipe for creating an OrderedSet ( https://code.activestate.com/recipes/576694/ ) but it seems like the uptake has been nearly zero. Also, now that Eric Snow has given us a fast OrderedDict, it is easier than ever to build an OrderedSet from MutableSet and OrderedDict, but again I haven't observed any real interest because typical set-to-set data analytics don't really need or care about ordering. Likewise, the primary use of fast membership testings is order agnostic. * That said, I do think there is room to add alternative set implementations to PyPI. In particular, there are some interesting special cases for orderable data where set-to-set operations can be sped-up by comparing entire ranges of keys (see https://code.activestate.com/recipes/230113-implementation-of-sets-using-sorted-lists for a starting point). IIRC, PyPI already has code for set-like bloom filters and cuckoo hashing. * I understanding that it is exciting to have a major block of code accepted into the Python core but that shouldn't open to floodgates to engaging in more major rewrites of other datatypes unless we're sure that it is warranted. Raymond Hettinger ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] [RELEASE] Python 3.6.0b1 is now available
On Sep 14, 2016, at 10:36, Guido van Rossum wrote: > Fortunately that page isn't linked from anywhere on the home page > AFAIK. If it is, could someone file an issue in the pydotorg tracker? > The url is at the bottom of every page. > > On Wed, Sep 14, 2016 at 3:41 AM, Paul Moore wrote: >> On 14 September 2016 at 11:32, Serhiy Storchaka wrote: >>> On 13.09.16 02:35, Ned Deily wrote: On behalf of the Python development community and the Python 3.6 release team, I'm happy to announce the availability of Python 3.6.0b1. 3.6.0b1 is the first of four planned beta releases of Python 3.6, the next major release of Python, and marks the end of the feature development phase for 3.6. >>> >>> >>> There is no mention on https://www.python.org/news/. >> >> The last release mentioned there is 3.4.0rc1... https://github.com/python/pythondotorg/issues/1008 [closed] -> duplicate of https://github.com/python/pythondotorg/issues/807 [open] Also, https://www.python.org/news/ has been manually updated by Ewa. -- Ned Deily n...@python.org -- [] ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
On 08.09.16 23:22, Victor Stinner wrote: I pushed INADA Naoki's implementation of the "compact dict". The hash table now stores indices pointing to a new second table which contains keys and values: it adds one new level of indirection. The table of indices is "compact": use 1, 2, 4 or 8 bytes per indice depending on the size of the dictionary. Moreover, the keys/values table is also more compact: its size is 2/3 of the indices table. A nice "side effect" of compact dict is that the dictionary now preserves the insertion order. It means that keyword arguments can now be iterated by their creation order: Note that this is made at the expense of the 20% slowing down an iteration. $ ./python -m timeit -s "d = dict.fromkeys(range(10**6))" -- "list(d)" Python 3.5: 66.1 msec per loop Python 3.6: 82.5 msec per loop Fortunately the cost of the lookup (the most critical operation for dicts) seems left the same. But this can be an argument against using this technique in sets. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.6.0b1 is now available
On 14.09.16 17:36, Guido van Rossum wrote: Fortunately that page isn't linked from anywhere on the home page AFAIK. If it is, could someone file an issue in the pydotorg tracker? The url is at the bottom of every page. This is on of the first results (actually the first besides manually edited news) of googling "python news". ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] [RELEASE] Python 3.6.0b1 is now available
On Thu, Sep 15, 2016 at 9:35 AM, Serhiy Storchaka wrote: > On 14.09.16 17:36, Guido van Rossum wrote: >> >> Fortunately that page isn't linked from anywhere on the home page >> AFAIK. If it is, could someone file an issue in the pydotorg tracker? >> The url is at the bottom of every page. > > > This is on of the first results (actually the first besides manually edited > news) of googling "python news". Fixed, it should redirect to https://www.python.org/blogs/ now. Thanks for noticing this! --Berker ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
