Re: [Python-ideas] Fwd: Replacing Infinite while Loops with an Iterator: async edition

On Sunday, June 24, 2018 at 7:48:34 PM UTC-4, Greg Ewing wrote: > > [email protected] wrote: > > On Jun 23, 2018, at 21:11, Nathaniel Smith > > wrote: > > > > He's asking for an async version of the 'iter' builtin, presumably > > something like: > > async

[Python-ideas] incremental hashing in __hash__

Suppose you have implemented an immutable Position type to represent the state of a game played on an MxN board, where the board size can grow quite large. Or suppose you have implemented an immutable, ordered collection type. For example, the collections-extended package provides a frozensetlist[

Re: [Python-ideas] incremental hashing in __hash__

I actually have been poking around that code already. I also found https://github.com/vperron/python-superfasthash/blob/master/superfasthash.py in case of interest. But it still seems like library authors with this use case should keep their library code free of implementation details like this, a

Re: [Python-ideas] incremental hashing in __hash__

On Wed, Dec 28, 2016 at 11:48 AM, Ned Batchelder wrote: > You can write a simple function to use hash iteratively to hash the entire > stream in constant space and linear time: > > def hash_stream(them): > val = 0 > for it in them: > val = hash((val, it)) >

Re: [Python-ideas] incremental hashing in __hash__

On Wed, Dec 28, 2016 at 12:10 PM, Ethan Furman wrote: > In other words, objects that do not compare equal can also have the same > hash value (although too much of that will reduce the efficiency of > Python's containers). > Yes, I realize that unequal objects can return the same hash value with

Re: [Python-ideas] incremental hashing in __hash__

in the meantime. Prompted by your helpful comparison, I just put together https://gist.github.com/jab/fd78b3acd25b3530e0e21f5aaee3c674 to further compare hash_tuple vs. hash_incremental. Based on that, hash_incremental seems to have a comparable distribution to hash_tuple. I'm not sure if the

Re: [Python-ideas] incremental hashing in __hash__

Updating the docs sounds like the more important change for now, given 3.7+. But before the docs make an official recommendation for that recipe, were the analyses that Steve and I did sufficient to confirm that its hash distribution and performance is good enough at scale, or is more rigorous anal

Re: [Python-ideas] incremental hashing in __hash__

On Fri, Dec 30, 2016 at 3:54 PM, Christian Heimes wrote: > Hi, > > I'm the author of PEP 456 (SipHash24) and one of the maintainers of the > hashlib module. > > Before we come up with a new API or recipe, I would like to understand > the problem first. Why does the initial op consider hash(large_

Re: [Python-ideas] incremental hashing in __hash__

On Fri, Dec 30, 2016 at 7:20 PM, Ethan Furman wrote: > On 12/30/2016 03:36 PM, [email protected] wrote: > > In the use cases I described, the objects' members are ordered. So in the >> unlikely event that two objects hash to the same value but are unequal, the >> __eq__ call should be cheap, be

Re: [Python-ideas] incremental hashing in __hash__

On Fri, Dec 30, 2016 at 8:04 PM, Ethan Furman wrote: > No. It is possible to have two keys be equal but different -- an easy > example is 1 and 1.0; they both hash the same, equal the same, but are not > identical. dict has to check equality when two different objects hash the > same but have n

Re: [Python-ideas] incremental hashing in __hash__

On Fri, Dec 30, 2016 at 8:10 PM, wrote: > On Fri, Dec 30, 2016 at 8:04 PM, Ethan Furman wrote: > >> No. It is possible to have two keys be equal but different -- an easy >> example is 1 and 1.0; they both hash the same, equal the same, but are not >> identical. dict has to check equality when

Re: [Python-ideas] incremental hashing in __hash__

On Fri, Dec 30, 2016 at 9:21 PM, Ethan Furman wrote: > I don't think so. As someone else said, a hash can be calculated once and > then cached, but __eq__ has to be called every time. Depending on the > clustering of your data that could be quick... or not. > __eq__ only has to be called when

Re: [Python-ideas] incremental hashing in __hash__

On Fri, Dec 30, 2016 at 10:08 PM, Ethan Furman wrote: > So maybe this will work? > > def __hash__(self): > return hash(self.name) * hash(self.nick) * hash(self.color) > > In other words, don't create a new tuple, just use the ones you already > have and toss in a couple maths operatio

Re: [Python-ideas] incremental hashing in __hash__

On Fri, Dec 30, 2016 at 10:30 PM, Nathaniel Smith wrote: > ... "Most hash schemes depend on having a "good" hash function, in the sense of > simulating randomness. Python doesn't ..." https://github.com/python/cpython/blob/d0a2f68a/Objects/dictobject.c#L133 ... Thanks for that link, fascinat

Re: [Python-ideas] incremental hashing in __hash__

of your data so you can use the fast C routine, or streaming your data to a slower Python implementation. At least in this case the Python implementation is built-in though. Given my current shortage of information, for now I'm thinking of handling this problem in my library by exposing a

Re: [Python-ideas] incremental hashing in __hash__

On Sat, Dec 31, 2016 at 5:39 PM, wrote: > a set hashing algorithm is exposed as collections.Set._hash() in > _collections_abc.py, which can be passed an iterable (By which I meant to say, "which can be passed a set-like iterable such as a Keys- or ItemsView of an existing mapping, so that the h

Re: [Python-ideas] incremental hashing in __hash__

Instead of the proposals like "hash.from_iterable()", would it make sense to allow tuple.__hash__() to accept any iterable, when called as a classmethod? (And similarly with frozenset.__hash__(), so that the fast C implementation of that algorithm could be used, rather than the slow collections.Set

[Python-ideas] Replacing Infinite while Loops with an Iterator: async edition

I first learned the ``for chunk in iter(lambda: sock.recv(N), b'')`` trick from page 138 of Dave Beazley’s fantastic Python Cookbook (“4.16. Replacing Infinite while Loops with an Iterator”), and never looked back. When I started to play with consuming sockets asynchronously, it occurred to me tha

[Python-ideas] Fwd: Replacing Infinite while Loops with an Iterator: async edition

On Jun 23, 2018, at 21:11, Nathaniel Smith wrote: > He's asking for an async version of the 'iter' builtin, presumably > something like: > async def aiter(async_callable, sentinel): >while True: >value = await async_callable() >if value == sentinel: >break >

Re: [Python-ideas] Fwd: Replacing Infinite while Loops with an Iterator: async edition

On Sun, Jun 24, 2018 at 3:34 PM Jelle Zijlstra wrote: > There is an open issue for this: https://bugs.python.org/issue31861. It > proposes adding aiter() and anext() as builtins. > Oh, great to see that, thanks! I'll follow along on that issue, and maybe even contribute a PR if I can. __

Re: [Python-ideas] Replacing Infinite while Loops with an Iterator: async edition

Yes, exactly. Wouldn't that be a useful built-in? (And, is this really the first time this idea is being discussed?) Thanks, Josh On Saturday, June 23, 2018 at 9:12:35 PM UTC-4, Nathaniel Smith wrote: > > On Sat, Jun 23, 2018 at 5:58 PM, Greg Ewing > wrote: > > [email protected] wrote: > >