[Python-ideas] Re: Set operations with Lists

[Tim] >> Something I haven't seen mentioned here, but I may have missed it: >> when timing algorithms with sets and dicts, people often end up merely >> measuring the bad effects of hardware data cache misses when the >> containers get large, and especially in contrived benchmarks. >> >> In those t

[Python-ideas] Re: Set operations with Lists

Andrew Barnert wrote: > On Sep 20, 2019, at 14:51, Richard Higginbotham higgi...@gmail.com wrote: > > Andrew Barnert wrote: > > What’s the goal here? > > I'm not concerned about micro op, I'm concerned with macro op on large data > > sets. > > Right now if someone comes to you with that simple use

[Python-ideas] Re: Set operations with Lists

Let me expand on the reasons for my post here. It seems like motivation has turned into a bit of a bike-shead. I have an algorithm I wanted to see if the python devs would be interested in. It fulfills my use case as is. I've found it useful for different problem areas and I haven't seen anythi

[Python-ideas] Re: Set operations with Lists

On Sep 20, 2019, at 14:51, Richard Higginbotham wrote: > > Andrew Barnert wrote: >> What’s the goal here? > > I'm not concerned about micro op, I'm concerned with macro op on large data > sets. Right now if someone comes to you with that simple use case of two > large lists you have to tell t

[Python-ideas] Re: Set operations with Lists

On Sat, Sep 21, 2019 at 7:54 AM Richard Higginbotham wrote: > I'm not concerned about micro op, I'm concerned with macro op on large data > sets. Right now if someone comes to you with that simple use case of two > large lists you have to tell them to convert to sets and then back again. > Forg

[Python-ideas] Re: Set operations with Lists

Andrew Barnert wrote: > On Sep 20, 2019, at 09:21, Richard Higginbotham higgi...@gmail.com wrote: > > Richard Musil wrote: > > On Fri, > > Sep 20, 2019 at 1:52 AM Andrew Barnert abarn...@yahoo.com wrote: > > On Sep 19, 2019, at 15:18, Richard Musil risa20...@gmail.com wrote: > > After some thought

[Python-ideas] Re: Set operations with Lists

Chris Angelico wrote: > On Fri, Sep 20, 2019 at 6:31 AM Richard Higginbotham higgi...@gmail.com wrote: > > > > That's exactly what I expected. It's the opposite > > though. The time to check one list for all the elements > > Please quote some text to give context. WHAT is exactly what you expected

[Python-ideas] Re: Set operations with Lists

BTW, this thread seems a good place to mention the under-appreciated SortedContainers package from Grant Jenks: http://www.grantjenks.com/docs/sortedcontainers/ This supplies sorted containers (lists, dicts, sets) coded in pure Python, which generally run at least as fast as C extensions impl

[Python-ideas] Re: Set operations with Lists

On Sep 20, 2019, at 09:21, Richard Higginbotham wrote: > > Richard Musil wrote: >>> On Fri, Sep 20, 2019 at 1:52 AM Andrew Barnert abarn...@yahoo.com wrote: >>> On Sep 19, 2019, at 15:18, Richard Musil risa20...@gmail.com wrote: > > After some thought I don't think the the time comparisons are u

[Python-ideas] Re: Set operations with Lists

On Sep 19, 2019, at 20:25, Tim Peters wrote: > > Something I haven't seen mentioned here, but I may have missed it: > when timing algorithms with sets and dicts, people often end up merely > measuring the bad effects of hardware data cache misses when the > containers get large, and especially in

[Python-ideas] Re: Set operations with Lists

Richard Musil wrote: > On Fri, Sep 20, 2019 at 1:52 AM Andrew Barnert abarn...@yahoo.com wrote: > > On Sep 19, 2019, at 15:18, Richard Musil risa20...@gmail.com wrote: After some thought I don't think the the time comparisons are useful unless we include the set init time. If we already have two

[Python-ideas] Re: Set operations with Lists

On Sep 20, 2019, at 03:18, Richard Musil wrote: > > This is an interesting point, which is difficult to deduce without an > implementation insight. I would for example assume that there is a "list > construct" which holds just the references to the actual objects (strings in > our example here

[Python-ideas] Re: Set operations with Lists

On Sep 20, 2019, at 02:41, Richard Musil wrote: > > I just added another implementation to the test (which I believe you had on > mind): > ``` > def relcomp_set_list(a, b): > bset = set(b) > return [ia for ia in a if ia not in bset] Or just `set(b).intersection(a)` and maybe checking th

[Python-ideas] Re: Set operations with Lists

Tim Peters wrote: > RE [Richard Higginbotham higgi...@gmail.com] .. > Traversing via a set or dict instead reads the objects' data in > pseudo-random memory order instead, which can give a very high rate of > cache misses. In addition, the internal hash table probes also occur > in pseudo-rand

[Python-ideas] Re: Set operations with Lists

Andrew Barnert wrote: > On Sep 19, 2019, at 19:18, Richard Higginbotham higgi...@gmail.com wrote: > > It's not really constant though. > > It’s really hard to have a discussion when all of your posts are all > > replies, but > you don’t give us any clue what you’re replying to. There are multiple

[Python-ideas] Re: Need a core developer sponsor for a PEP

Response inline Le jeu. 19 sept. 2019 à 19:18, Andrew Barnert a écrit : > Steven already answered many of these, so I’ll just snip the ones he > didn’t. > > On Sep 19, 2019, at 00:03, Philippe Prados > wrote: > > > * The static types in typing are not instances of type, so you need to > work

[Python-ideas] Re: Set operations with Lists

You know, if anyone wanted a good example of why Computer Science courses should carry on teaching about sorting algorithms despite them being already implemented for you these days, this thread is it. Very nicely explained, folks! -- Rhodri James *-* Kynesim Ltd _

[Python-ideas] Re: Set operations with Lists

> > On Thu, Sep 19, 2019 at 10:25:20PM -0500, Tim Peters wrote: > [...] > > Something I haven't seen mentioned here, but I may have missed it: > > when timing algorithms with sets and dicts, people often end up merely > > measuring the bad effects of hardware data cache misses when the > > containe

[Python-ideas] Re: Set operations with Lists

On Fri, Sep 20, 2019 at 1:52 AM Andrew Barnert wrote: > On Sep 19, 2019, at 15:18, Richard Musil wrote: > > > > Ok, I misread the original code, the lists were not sorted in the > previous results (and their should be). So with the correction to have them > sorted, > > I think to be fair, you wa

[Python-ideas] Discussions-To PEP-0604: Complementary syntax for Union[] and Optional[]

Hello, I just publish the PEP-0604 for discussions (https://www.python.org/dev/peps/pep-0604/). Because this proposition is between python and typing, and because the discussion has already been made here, I start a new discussion

[Python-ideas] Re: Set operations with Lists

On Thu, Sep 19, 2019 at 10:25:20PM -0500, Tim Peters wrote: [...] > Something I haven't seen mentioned here, but I may have missed it: > when timing algorithms with sets and dicts, people often end up merely > measuring the bad effects of hardware data cache misses when the > containers get large,