Re: [Python-ideas] PEP: Dict addition and subtraction
Just in case I'm not the only one that had a hard time finding the latest version of this PEP, here it is in the PEPS Repo: https://github.com/python/peps/blob/master/pep-0584.rst -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Attribute-Getter Syntax Proposal
> > Rather than using map in this way, I would recommend a list comprehension: Exactly! I really don’t get why folks want to use map() so much when the comprehension syntax is often cleaner and easier. It was added for a reason :-) -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 8 update on line length
On Wed, Feb 27, 2019 at 5:53 AM Rhodri James wrote: > > To be a bit less harsh -- that is, change the clause: > > > > "For code maintained exclusively or primarily by a team that can reach > > agreement on this issue, it is okay ..." > > > > To maybe just > > > > "For code maintained by a team that can reach agreement on this issue, > it > > is okay ..." > > I would prefer not. The wording as given emphasizes that the team isn't > expecting outsiders to work on their code. I think that's quite > important, especially when you consider that PEP8 only really has force > for the standard library. > Huh? in the case of the stdlib, the "team" is the python core devs -- they could decide to increase the line length if they want (though I imagine if they did, they's probably change the PEP :- ) Anyway, not a biggie -- just might get folks to chill out a bit and stop pushing to change the PEP ... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP 8 update on line length
On Thu, Feb 21, 2019 at 12:01 PM Raymond Hettinger <
raymond.hettin...@gmail.com> wrote:
> I concur. We now put expressions in f-strings and have assignment
> expressions that easily spill over 80 characters if one uses all but the
> most compact variable names. Comprehensions tend to make expressions
> longer. Queries and filters in Pandas also easily spill over. The 80
> character limit pre-dates these evolutions of the language.
>
> In particular, the case where I most want to let people run with long
> lines is in forming messages to be displayed to a user.
I agree here -- and if it's messages (also comments - a twenty char comment
after a 70 char line should be fine!) then it's not part of the logic of
the code -- so not as bd if there is some spill off the screen for those
folks that do code on tablets ;-) Actually for me, the issue s comes up
when I'm showing code on a projector -- I use a huge font so folks in the
back can see)
> class Frabawidget:
> ...
> @wozzle.setter
> def (self, woozle):
> if not (self.min_woozle < woozle < self.max_woozle):
> raise ValueError(f"Expected woozle to be between
> {self.min_woozle} and {self.max_woozle}")
> self._wozzle = normalize(woozle)
>
That's 103 chars long -- and very readable. But, is this that much worse?
class Frabawidget:
...
@wozzle.setter
def (self, woozle):
if not (self.min_woozle < woozle < self.max_woozle):
raise ValueError(f"Expected woozle to be between"
"{self.min_woozle} and {self.max_woozle}")
self._wozzle = normalize(woozle)
(it IS harder to write, that's for sure)
In doing code reviews, I see many fewer atrocities from long lines than I
> do from weird line-wraps and from variable names that have been
> over-shortened to make the line fit in 80 characters. To avoid these
> issues, my clients typically set their line limits at 90 or 100
and yet the above example was 103 ... you do need a limit somewhere.
I actually would really like the "limit" to depend on what the line is --
that is, it's OK to go longer if it's essentially text -- message to the
user, comment, etc., rather than part of the code logic.
In fact, now that I write that, I think I actually DO do that -- and even
add a # noqa sometimes so my linter will stop bugging me. A smart linter
would be nice here.
PEP 8 is mostly about readability. However, the line length limit often
> seems to cause less readable code.
>
So what do you suggest? simply increase the recommended line length? Or
would softening the language about extending it be enough?
Maybe something along the lines of:
A line length limit should be well defined for each project / organization
-- 80 char per line is safest for all users, but anything up to 100 is
appropriate if the team maintaining the code agrees.
Raymond: As a core dev -- are suggesting extending the line limit for the
standard library?
To all the folks quoting theory: let's be honest. Yes, really long lines
are harder to read, but the 80 char limit comes from old terminals, NOT any
analysis that somehow that is optimum for readability.
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Stack traces ought to flag when a module has been changed on disk
On Wed, Jan 30, 2019 at 3:43 AM Jonathan Fine wrote: > I think Steve's suggestion fails in this situation. Suppose wibble.py > contains a function fn. Now do >import wibble >fn = wibble.fn ># Modify and save wibble.py >reload(wibble) >fn() > Sure -- but this is just a warning that there *might* be an issue -- if a user doesn't get the warning in some cases (particularly cases in which they have used an "advanced" feature like reload) we aren't any worse off. +1 on the idea -- I don't see a downside. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add list.join() please
> Alex Shafer via Python-ideas wrote:
>> 1) I'm in favor of adding a stringify method to all collections
>
> Are you volunteering to update all the collection
> classes in the world written in Python? :-)
To be fair, we could add an implementation to the sequence ABC, and
get pretty far.
Not that I’m suggesting that — as I said earlier, Python is all about
iterables, not sequences, anyway.
Also, despite some folks’ instance that this “stringify’ method is
something many folks want -.I’m not even sure what it is.
I was thinking it was:
def stringify(self, sep):
return sep.join(str(i) for i in self)
Which, by the way would work for any iterable :-)
If you want a language designed specifically for text processing, use Perl.
Python is deliberately strongly typed, so that:
2 + “2”
Raises an error. Why should:
“”.join([2, “2”]) not raise an error as well?
And aside from repr or ascii, when I turn numbers into text, I usually
want to control the formatting anyway:
“ “.join(f”{n:.2f}” for n in seq)
So having str() called automatically for a join wouldn’t be that useful.
-CHB
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add list.join() please
A couple notes:
On Tue, Jan 29, 2019 at 5:31 AM Jamesie Pic wrote:
> can you clarify the documentation
> topic you think should be improved or created ? "Assembling strings"
>
I would think "assembling strings", though there is a lot out there already.
> or "inconsistencies between os.path.join and str.join" ?
>
well, if we're talking about moving forward, then the Path object is
probably the "right" way to join paths anyway :-)
a_path / "a_dir" / "a_filename"
But to the core language issue -- I started using Python with 1.5.* and
back then join() was in the string module (and is there in 2.7 still)
And yes, I did expect it to be a list method...
Then it was added as a method of the string object.
And I thought THAT was odd -- be really appreciated that I didn't need to
import a module to do something fundamental.
But the fact is, that joining strings is fundamentally a string operation,
so it makes sense for it to be there.
In earlier py2, I would have thought, maybe it should be a list method --
it's pretty darn common to join lists of strings, yes? But what about
tuples? Python was kind of all about sequences -- so maybe all sequences
could have that method -- i.e part of the sequence ABC.
But with > py3k, Python is more about iterables than sequences -- and join
(and many other methods and functions) operate on any iterable -- and this
is a really good thing.
So add join to ALL iterables? That makes little sense, and really isn't
possible -- an iterable is something that conforms to the iterator protocol
-- it's not a type, or even an ABC.
So in the end, join really does only make sense as string method.
Or Maybe as a built in -- but we really don't need any more of those.
If you want to argue that str.join() should take multiple arguments, like
os.path.join does, then, well we could do that -- it currently takes one
and only one argument, so it could be extended to join multiple arguments
-- but I have hardly ever seem a use case for that.
The mistake I'm still doing after 10 years of Python
>
hmm -- I've seen a lot of newbies struggle with this, but haven't had an
issue with it for years myself.
> >>> '/'.join('some', 'path')
> TypeError: join() takes exactly one argument (2 given)
>
pathlib aside, that really isn't the right way to join paths .
os.path.jon exists for a (good) reasons. One of which is this:
In [22]: os.path.join("this/", "that")
Out[22]: 'this/that'
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Possible PEP regarding the use of the continue keyword in try/except blocks
There is already a much simpler way of doing this:
> try:
> i = int("string")
> except ValueError as e:
> print(e)
> print("continued on")
> j = int(9.0)
>
> The point of the 'try' block is to encapsulate the code you want to *stop*
> executing if an exception is raised. If you want code to be run regardless
> of whether an exception is raised, move it past the try-except.
>
To be fair, I suspect the issue was there were two calls to int() there
that might raise a ValueError, and the OP wanted to catch them with one
except, so you would need to do somethign like:
try:
i = int("string")
except ValueError as e:
print(e)
print("continued on")
try:
j = int(9.0)
except ValueError as e:
print(e)
Which can seem a bit verbose, but in fact, there are a number of ways one
might want to proceed with/without an error, and the current except,
finally, else options cover them all in a clearly defined way.
-CHB
>
> ~Amber
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Suggested MapView object (Re: __len__() for map())
>>> and the test for an iterator is: >>> >>> obj is iter(obj) Is that a hard and fast rule? I know it’s the vast majority of cases, but I imagine you could make an object that behaved exactly like an iterator, but returned some proxy object rather that itself. Not sure why one would do that, but it should be possible. - CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Suggested MapView object (Re: __len__() for map())
On Tue, Dec 11, 2018 at 11:10 AM Terry Reedy wrote: > > I _think_ someone may be advocating that map() could return an > > iterable if it is passed a iterable, > > I believe you mean 'iterator' rather than 'iterable' here and below as a > sequence is an iterable. > well, the iterator / iterable distinction is important in this thread in many places, so I should have been more careful about that -- but not for this reason. Yes, a a sequence is an iterable, but what I meant was an "iterable-that-is-not-a-sequence". -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Suggested MapView object (Re: __len__() for map())
Perhaps I got confused by the early part of this discussion. My point was that there is no “map-like” object at the Python level. (That is no Map abc). Py2’s map produced a sequence. Py3’s map produced an iterable. So any API that was expecting a sequence could accept the result of a py2 map, but not a py3 map. There is absolutely nothing special about map here. The example of range has been brought up, but I don’t think it’s analogous — py2 range returns a list, py3 range returns an immutable sequence. Because that’s as close as we can get to a sequence while preserving the lazy evaluation that is wanted. I _think_ someone may be advocating that map() could return an iterable if it is passed a iterable, and a sequence of it is passed a sequence. Yes, it could, but that seems like a bad idea to me. But folks are proposing a “map” that would produce a lazy-evaluated sequence. Sure — as Paul said, put it up on pypi and see if folks find it useful. Personally, I’m still finding it hard to imagine a use case where you need the sequence features, but also lazy evaluation is important. Sure: range() has that, but it came at almost zero cost, and I’m not sure the sequence features are used much. Note: the one use-case I can think of for a lazy evaluated sequence instead of an iterable is so that I can pick a random element with random.choice(). (Try to pick a random item from. a dict), but that doesn’t apply here—pick a random item from the source sequence instead. But this is specific example of a general use case: you need to access only a subset of the mapped sequence (or access it out of order) so using the iterable version won’t work, and it may be large enough that making a new sequence is too resource intensive. Seems rare to me, and in many cases, you could do the subsetting before applying the function, so I think it’s a pretty rare use case. But go ahead and make it — I’ve been wrong before :-) -CHB Sent from my iPhone > On Dec 11, 2018, at 6:47 AM, Steven D'Aprano wrote: > >> On Mon, Dec 10, 2018 at 05:15:36PM -0800, Chris Barker via Python-ideas >> wrote: >> [...] >> I'm still confused -- what's so wrong with: >> >> list(map(func, some_iterable)) >> >> if you need a sequence? > > You might need a sequence. Why do you think that has to be an *eager* > sequence? > > I can think of two obvious problems with eager sequences: space and > time. They can use too much memory, and they can take too much time to > generate them up-front and too much time to reap when they become > garbage. And if you have an eager sequence, and all you want is the > first item, you still have to generate all of them even though they > aren't needed. > > We can afford to be profligate with memory when the data is small, but > eventually you run into cases where having two copies of the data is one > copy too many. > > >> You can, of course mike lazy-evaluated sequences (like range), and so you >> could make a map-like function that required a sequence as input, and would >> lazy evaluate that sequence. This could be useful if you weren't going to >> work with the entire collection, > > Or even if you *are* going to work with the entire collection, but you > don't need them all at once. I once knew a guy whose fondest dream was > to try the native cuisine of every nation of the world ... but not all > in one meal. > > This is a classic time/space tradeoff: for the cost of calling the > mapping function anew each time we index the sequence, we can avoid > allocating a potentially huge list and calling a potentially expensive > function up front for items we're never going to use. Instead, we call > it only on demand. > > These are the same principles that justify (x)range and dict views. Why > eagerly generate a list up front, if you only need the values one at a > time on demand? Why make a copy of the dict keys, if you don't need a > copy? These are not rhetorical questions. > > This is about avoiding the need to make unnecessary copies for those > times we *don't* need an eager sequence generated up front, keeping the > laziness of iterators and the random-access of sequences. > > map(func, sequence) is a great candidate for this approach. It has to > hold onto a reference to the sequence even as an iterator. The function > is typically side-effect free (a pure function), and if it isn't, > "consenting adults" applies. We've already been told there's at least > one major Python project, Sage, where this would have been useful. > > There's a major functional language, Haskell, where nearly all sequence > processing follows this approach. > > I suggest we provide a separate mapview() type that offers only the lazy > sequence API, without trying to be an iterator at the same ti
Re: [Python-ideas] Using sha512 instead of md5 on python.org/downloads
On Sun, Dec 9, 2018 at 10:32 PM Ronald Oussoren via Python-ideas < python-ideas@python.org> wrote: > BTW. I wonder how many actually verify these checksums, > Hardly anyone -- most of us verify the download by trying to use it :-) Which doesn't mean that we shouldn't have it -- but it will indeed make very little difference to the vast majority of users -- and those that do check it are generally pretty sophisticated -- shouldn't be hard to use a different hash algorithm. Though people would have to update their workflows, which could be annoying. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Suggested MapView object (Re: __len__() for map())
On Mon, Dec 10, 2018 at 5:23 AM E. Madison Bray wrote: > Indeed; I believe it is very useful to have a map-like object that is > effectively an augmented list/sequence. but what IS a "map-like object" -- I'm trying to imagine what that actually means. "map" takes a function and maps it onto a interable, returning a new iterable. So a map object is an iterable -- what's under the hood being used to create it is (and should remain) opaque. Back in the day, Python was "all about sequences" -- so map() took a sequence and returned a sequence (an actual list, but that's not the point here). And that's pretty classic "map". With py3, there was a big shift toward iterables, rather than sequences as the core type to work with. There are a few other benefits, but the main one is that often sequences were made, simply so that they could be immediately iterated over, and that was a waste of resources. for i, item in enumerate(a_sequence): ... for x, y in zip(seq1, seq2): ... These two are pretty obvious, but the same approach was taken over much of python: dict.keys(), map(), range(), So now in Python, you need to decide, when writing code, what your API is -- does your function take a sequence? or does it take an iterable? Of course, a sequence is an iterable, but a iterable is not (necessarily) a sequence. -- so back in the day, you din't really need to make the decision. So in the case of the Sage example -- I wonder what the real problem is -- if you have an API that requires a sequence, on Py2, folks may have well been passing it the result of a map() call. -- note that they weren't passing a "map object" that is now somehow different than it used to be -- they were passing a list plain and simple. And there are all sorts of places, when converting from py2 to py3, where you will now get an iterable that isn't a proper sequence, and if the API you are using requires a sequence, you need to wrap a list() or tuple() or some such around it to make the sequence. Note that you can write your code to work under either 2 or 3, but it's really hard to write a library so that your users can run it under either 2 or 3 without any change in their code! But note: the fact that it's a map object is just one special case. I suppose one could write an API now that actually expects a map object (rather than a generic sequence or iterable) but it was literally impossible in py2 -- there was no such object. I'm still confused -- what's so wrong with: list(map(func, some_iterable)) if you need a sequence? You can, of course mike lazy-evaluated sequences (like range), and so you could make a map-like function that required a sequence as input, and would lazy evaluate that sequence. This could be useful if you weren't going to work with the entire collection, but really wanted to only index out a few items, but I'm trying to imagine a use case for that, and I haven't. And I don't think that's the use case that started this thread... -CHB > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Relative Imports
This is somewhat unpleasant to me, especially while developing something and trying to test it quickly. I just want to be able to use same relative imports and run single file with `python3 test_main.py` for example. I had the same frustration when I first tried to use relative imports. Then I discovered setuptools’ develop mode (now pip editable install) It is the right way to run code in packages under development. -CHB Running files as modules every time is tiring. This is my problem. I could not come up with a concrete solution idea yet i am thinking on it. Open to suggestions. Thank you all for your help! On Fri, Nov 9, 2018 at 4:16 PM Steven D'Aprano wrote: > On Fri, Nov 09, 2018 at 03:51:46PM -0800, danish bluecheese wrote: > > └── src > > ├── __init__.py > > ├── main.py > > └── test > > ├── __init__.py > > └── test_main.py > > > > assume the structure above. To be able to use relative imports with such > > fundamental structure either i can go for sys.path hacks or could run as > a > > module from one further level parent. > > I don't understand. From the top level of the package, running inside > either __init__ or main, you should be able to say: > > from . import test > from .test import test_main > > From the test subpackage, you should be able to say: > > from .. import main > > to get the src/main module, or > > from . import test_main > > to get the test/test_main module from the test/__init__ module. > > (Disclaimer: I have not actually run the above code to check that it > works, beyond testing that its not a SyntaxError.) > > What *precisely* is the problem you are trying to solve, and your > proposed solution? > > > > -- > Steve > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Serialization of CSV vs. JSON
On Fri, Nov 2, 2018 at 12:17 PM, Wes Turner wrote: > JSON5 supports comments in JSON. > https://github.com/json5/json5/issues/3 > and other nifty things -- any plans to support JSON5 in the stdlib json library? I think that would be great. -CHB > ... Some form of schema is necessary to avoid having to try parsing every > string value as a date time (and to specify precision: "2018" is not the > same as "2018 00:00:01") > > On Friday, November 2, 2018, Chris Barker via Python-ideas < > python-ideas@python.org> wrote: > >> On Fri, Nov 2, 2018 at 9:31 AM, M.-A. Lemburg wrote: >> >>> Serialization of those data types is not defined in the JSON standard: >>> >>> https://www.json.org/ >> >> >> That being said, ISO 8601 is a standard for datetime stamps, and a >> defacto one for JSON >> >> So building encoding of datetime into Python's json encoder would be >> pretty useful. >> >> (I would not have any automatic decoding though -- as an ISO8601 string >> would still be just a string in JSON) >> >> Could we have a "pedantic" mode for "fully standard conforming" JSON, and >> then add some extensions to the standard? >> >> As another example, I would find it very handy if the json decoder would >> respect comments in JSON (I know that they are explicitly not part of the >> standard), but they are used in other applications, particularly when JSON >> is used as a configuration language. >> >> -CHB >> >> >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR(206) 526-6959 voice >> 7600 Sand Point Way NE (206 >> <https://maps.google.com/?q=7600+Sand+Point+Way+NE+%C2%A0%C2%A0(206=gmail=g>) >> 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> chris.bar...@noaa.gov >> > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.setdefault_call(), or API variations thereupon
On Fri, Nov 2, 2018 at 7:49 PM, Steven D'Aprano wrote:
> Consider the use-case where you want to pass a different default value
> to the dict each time:
>
exactly - the "default" is per call, not the same for the whole dict.
though again, how common is this?
> d.setdefault(key, expensive_function(1, 2, 3))
> d.setdefault(key, expensive_function(4, 8, 16))
> d.setdefault(key, expensive_function(10, 100, 1000))
>
also -- aside from performance, if expensive_function() has side effects,
you may really not want to call it when you don't need to (not that that
would be well-designed code, but...)
and of course, you can always simply do:
if key in d:
val = d[key]
else:
val = expensive_function(4, 8, 16)
d[key] = val
sure, it requires looking up the key twice, but doesn't call the function
unnecessarily.
So it's a pretty small subset of cases, where this would be needed.
defaultdict won't help, because your factory function takes no
> arguments: there's no way to supply arguments for the factory.
>
maybe that's a feature defaultdict should have?
-CHB
> __missing__ won't help, because it only receives the key, not arbitrary
> arguments.
>
> We can of course subclass dict and give it a method with the semantics
> we want:
>
> d.my_setdefault(key, expensive_function, args=(1, 2, 3), kw={})
>
> but it would be nicer and more expressive if we could tell the
> interpreter "don't evaluate expensive_function(...) unless you really
> need it".
>
> Other languages have this -- I believe it is called "Call By Need" or
> "Call By Name", depending on the precise details of how it works. I call
> it delayed evaluation, and Python already has it, but only in certain
> special syntactic forms:
>
> spam and
> spam or
> if condition else
>
> There are others: e.g. the body of functions, including lambda. But
> functions are kinda heavyweight to make and build and call.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Serialization of CSV vs. JSON
On Fri, Nov 2, 2018 at 9:31 AM, M.-A. Lemburg wrote: > Serialization of those data types is not defined in the JSON standard: > > https://www.json.org/ That being said, ISO 8601 is a standard for datetime stamps, and a defacto one for JSON So building encoding of datetime into Python's json encoder would be pretty useful. (I would not have any automatic decoding though -- as an ISO8601 string would still be just a string in JSON) Could we have a "pedantic" mode for "fully standard conforming" JSON, and then add some extensions to the standard? As another example, I would find it very handy if the json decoder would respect comments in JSON (I know that they are explicitly not part of the standard), but they are used in other applications, particularly when JSON is used as a configuration language. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add "default" kwarg to list.pop()
On Fri, Nov 2, 2018 at 3:59 AM, Steven D'Aprano wrote: > - I'm not volunteering to do the work (I don't know enough C to write > a patch). Unless somebody has a patch, we can't expect the core devs > who aren't interested in this feature to write it. > > (Hence, status quo wins a stalemate.) > Well, it would be frustrating to have a feature accepted but not implemented, but the steps are separate. And it wouldn't have to be a core dev that implements it -- anyone with the C chops (not me :-) ) could do it. As an example, a good chunk of PEP 485 was implemented by someone else (I wrote the first prototype, but it was not complete), who I'm pretty sure is not a core dev. A core dev has to actually merge it, of course, and that is a bottleneck, but not a show stopper. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.setdefault_call(), or API variations thereupon
On Thu, Nov 1, 2018 at 8:34 PM, Steven D'Aprano wrote:
> The bottom line is, if I understand your proposal, the functionality
> already exists. All you need do is subclass dict and give it a
> __missing__ method which does what you want.
or subclass dict and give it a "setdefault_call") method :-)
But as I think Guido wasa pointing out, the real difference here is that
DefaultDict, or any other subclass, is specifying what the default callable
is for the entire dict, rather than at time of use. Personally, I'm pretty
sure I"ve only used one default for any given dict, but I can imaige the
are use cases for having different defaults for the same dict depending on
context.
As for the OP's justification:
"""
If it's not clear, the purpose is to eliminate the overhead of creating an
empty list or similar in situations like this:
d = {}
for i in range(100): # some large loop
l = d.setdefault(somekey, [])
l.append(somevalue)
# instead...
for i in range(100):
l = d.setdefault_call(somekey, list)
l.append(somevalue)
"""
I presume the point is that in the first case, somekey might be often the
same, and setdefault requires creating an actual empty list even if the
key is alredy there. whereas case 2 will only create the empty list if the
key is not there. doing some timing with defaultdict:
In [19]: def setdefault():
...: d = {}
...: somekey = 5
...: for i in range(100): # some large loop
...: l = d.setdefault(somekey, [])
...: l.append(i)
...: return d
In [20]: def default_dict():
...: d = defaultdict(list)
...: somekey = 5
...: for i in range(100): # some large loop
...: l = d[somekey]
...: l.append(i)
...: return d
In [21]: % timeit setdefault()
185 ms ± 1.23 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [22]: % timeit default_dict()
128 ms ± 1.65 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
so yeah, it's a little more performant, and I suppose if you were using a
more expensive constructor, it would make a lot more difference. But then,
how much is it likely to matter in a real use cases -- this was 1 million
calls for one key and you got a 50% speed up -- is that common?
So it seems this would give us slightly better performance than
.setdefault() for the use cases where you are using more than one default
for a given dict.
BTW:
+1 for a mention of defaultdict in the dict.setdefault docs -- you can't do
everything with defaultdict that you can with setdefault, but it is a very
common use case.
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Contracts in python -- a report & next steps
On Wed, Oct 24, 2018 at 12:57 AM, Chris Angelico wrote: > For the sake of those of us who REALLY don't feel like diving back > into the extensive threads on this subject, can you please summarize > the benefits of having this in the stdlib rather than as a third-party > library? > I'm not (currently) a fan of design by contract, and don't expect I'll end up using it, whether or not we get a nifty standard library for it That being said -- this is analogous to why PEP 484 -- Type Hints was adopted, even though the syntax changes in Python had long been adopted to enable it. That is: the IF folks are going to use DbC in Python, the community is MUCH better off if everyone does it in the same way. And the way to make that happen is to put it in the stdlib. However, I'm not sure it's anywhere near time to actually do that -- before we get there, there needs to be a pretty good community of folks using icontract (or maybe something else?) and ideally some interest from a core developer or two. After all, PEP 484 didn't get rollling until Guido got interested in MyPy. But by all means -- start building that community -- maybe some of us skeptics will be drawn in > Also - have you benchmarked the performance cost of adding contracts? > Particularly: if you're planning to contractify the stdlib, what is > the impact on startup performance? > Contractifying the stdlib is an entirely independent question -- yes, if we wanted to add contract to the stdlib, we'd need a contract library in the stdlib, but there's no need for the other way around to be true. Is the stdlib being fully typhinted since PEP 484 was added? a quick scan of the PEP didn't seem to mention it at all. Performance (particularly start up time) would absolutely be a consideration if and when we get there, but I don't know that it has any impact to anything anyone might do now. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting Immutable Mappings
On Thu, Oct 18, 2018 at 10:12 AM, Sven R. Kunze wrote: > On 18.10.18 18:49, Anders Hovmöller wrote: > >> If it's AND, shouldn't it be "hasinterfaces" (notice the s!)? > > yeah, that would make sense. Is someone proposing something here? The point I was making, is in the case of ABCs: issubclass(a_class, an_abc) isn't "Really" testing a subclass, but really an interface. That is - the ABC exists solely to define an interface, not provide any functionality (though there is some in there), and you can, in fact, use ABCs to provide a way to check a duck-typed class that isn't an actual subclass. So do we need a new function that makes that clear? would it functionally be an different than issubclass with an ABC? I honestly don't know. Though an easy way to get the "and" behaviour -- i.e. interfaceS would be nice. -CHB To be sure, we are on the same page here: "interface" refers to a set of > attributes of the object in question, does it? > > E.g. like the __iter__ iterface. I usually don't care about the actual > inheritance hierarchy but care about functionality. > > One could also imagine that isinstance and issubclass taking a keyword >> argument for the logical operator. Maybe just something as simple as >> "isinstance(foo, (a, b), all=True)" >> > > Does AND even make sense for isinstance/issubclass? > > Cheers, > Sven > > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting Immutable Mappings
On Thu, Oct 11, 2018 at 3:35 PM, Steven D'Aprano wrote: > On Thu, Oct 11, 2018 at 12:34:13PM -0700, Chris Barker via Python-ideas > wrote: > > > I don't care what is or isn't a subclass of what -- I don't think that's > a > > Pythonic question. But I do think : > > > > issubclass(frozendict, abc.Mapping) and issubclass(frozendict, > abc.Hashable) > > > > would be useful. > > So you do care about what is and isn't a subclass :-) > well, kinda -- I don't care whether dict and frozen dict have a subclassing relationship, and I don't care what class a given object is that gets passed to my code. What I *might* care about is what interface it presents, which is what ABCs are for. If Python had a way to check ABCs without issubclass, then I wouldn't care about subclasses at all. I'd actually kind of like to have: hasinterface(an_object, (ABC1, ABC2, ABC3)) Even though, it would be the same as issubclass() (though I'd like an AND relationship with the tuple of ABCs..) Maybe I'm making a distinction that isn't really there, but I see a subclass of an ABC is quite different than a subclass of another concrete class. > It is 2018 and we've had isinstance and issubclass in the language since > Python 2.2 in 2002, I really wish that we could get away from the idea > that checking types is unPythonic and duck-typing is the One True Way to > do things. The old, Python 1.5 form of type-checking: > > type(spam) is dict > > was not so great, since it tested only for a specific type by identity. > But with ABCs and virtual subclasses and inheritance, isinstance is a > great way to check that your duck both quacks and swims, rather than > watching it sink and drown in the middle of your function :-) > yeah, I'm not so sure -- I"ve been watching Python go more and more in that direction -- and I don't think I'm happy about it. Personally, the only time I check a type is the darn list-of-strings vs single-string issue -- ever since we got true_division, that's the only type error I commonly see (that isn't trivial to debug). And I still wish Python had a Character type, and then I wouldn't need to do that either :-) And actually, if you're talking "modern" Python, isn't static type checking the way to do it now ? Now that I think about it - how does MyPy/Typeshed handle the iterable_of_strings problem? Since a single string IS and iterable of strings? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting Immutable Mappings
BTW: In [7]: issubclass(set, frozenset) Out[7]: False In [8]: issubclass(frozenset, set) Out[8]: False no reason to do anything different here. and: In [13]: issubclass(MappingProxyType, abc.Hashable) Out[13]: False so yes, there is a need for something different -CHB On Thu, Oct 11, 2018 at 12:34 PM, Chris Barker wrote: > On Thu, Oct 11, 2018 at 9:54 AM, Jonathan Fine > wrote: > >> Summary: Long post. Because of LSP, neither dict nor frozendict are a >> subclass of the other. > > > given Python's dynamic typing, these issues are kinda academic :-) > > > Intuition tells me that a frozen dictionary is a form of dictionary >> > that adds restrictions, not that a dictionary is a frozen dictionary >> > that you left out to thaw. >> > > well, IIUC the Liskov principle correctly then a subclass is never a > version of a class that does less, but rather always one that does more. > > Think you intuition may be driven by the choice of names and history: yes, > a frozen dict sounds like a regular dict that has been altered > (specifically frozen) -- but if we called them "hash_table" and > "mutable_has_table", then your intuition may be different :-) > > As for subclassing or not, for most Python code is makes no difference -- > polymorphism is not achieved through subclassing. and the "Pythonic" way to > test for type is through ABCs, and we already have Mapping and > MutableMapping, which kind of surprised me, as there is no builtin Mapping > that isn't also a MutableMapping. > > Notice that I said USEFUL property P. The negation Q of property P is >> also a property. > > > well, yeah, but I think the concept of "useful" is pretty vague. > > in this case, is "imutablilty" the "useful" property -- or is > "hashability"?, which would want us to use abc.Hashable. > > So: > > I don't care what is or isn't a subclass of what -- I don't think that's a > Pythonic question. But I do think : > > issubclass(frozendict, abc.Mapping) and issubclass(frozendict, > abc.Hashable) > > would be useful. > > BTW, I just noticed that: > > A tuple, as in ``issubclass(x, (A, B, ...))``, may be given as the target > to check against. This is equivalent to ``issubclass(x, A) or issubclass(x, > B) or ...`` etc. > > which seems less useful than "and" -- at least for ABCs > > I suppose that API pre-dates ABCs > > One way to fix this in Python 4 would be to create a common ancestor, >> which has only the shared methods of dict and frozendict. > > > which would be an immutable, but not hashable, mapping ?!? > > Or do >> something similar with abstract base classes. >> > > already done -- see above. > > -CHB > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting Immutable Mappings
On Thu, Oct 11, 2018 at 9:54 AM, Jonathan Fine wrote: > Summary: Long post. Because of LSP, neither dict nor frozendict are a > subclass of the other. given Python's dynamic typing, these issues are kinda academic :-) > Intuition tells me that a frozen dictionary is a form of dictionary > > that adds restrictions, not that a dictionary is a frozen dictionary > > that you left out to thaw. > well, IIUC the Liskov principle correctly then a subclass is never a version of a class that does less, but rather always one that does more. Think you intuition may be driven by the choice of names and history: yes, a frozen dict sounds like a regular dict that has been altered (specifically frozen) -- but if we called them "hash_table" and "mutable_has_table", then your intuition may be different :-) As for subclassing or not, for most Python code is makes no difference -- polymorphism is not achieved through subclassing. and the "Pythonic" way to test for type is through ABCs, and we already have Mapping and MutableMapping, which kind of surprised me, as there is no builtin Mapping that isn't also a MutableMapping. Notice that I said USEFUL property P. The negation Q of property P is > also a property. well, yeah, but I think the concept of "useful" is pretty vague. in this case, is "imutablilty" the "useful" property -- or is "hashability"?, which would want us to use abc.Hashable. So: I don't care what is or isn't a subclass of what -- I don't think that's a Pythonic question. But I do think : issubclass(frozendict, abc.Mapping) and issubclass(frozendict, abc.Hashable) would be useful. BTW, I just noticed that: A tuple, as in ``issubclass(x, (A, B, ...))``, may be given as the target to check against. This is equivalent to ``issubclass(x, A) or issubclass(x, B) or ...`` etc. which seems less useful than "and" -- at least for ABCs I suppose that API pre-dates ABCs One way to fix this in Python 4 would be to create a common ancestor, > which has only the shared methods of dict and frozendict. which would be an immutable, but not hashable, mapping ?!? Or do > something similar with abstract base classes. > already done -- see above. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting Immutable Mappings
> This violates the Liskov Substitution Principle. If we REALLY had a time machine, then dict would subclass frozendict, and we’d be all set. But to what extent do we need to support ALL the ways to check for an interface? Personally, I think EAFTP is the most “Pythonic”, but if folks want to check up front, then isn’t that what ABCs are for? In which case , we already have Mapping and MutableMapping. So if you have code that checks for Mapping when you need it mutable, then that’s arguably a bug. And if you code checks for dict directly, then it’s arguably unpythonic. That being said, it probably is best not to break working code. Would there be unpleasant consequences to making dict a subclass of FrozenDict ? Or maybe come up with a new name: after all, lists and tuples are independent, even though you *could* think of a tuple as a FrozenList ... -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] support toml for pyproject support
If I had the energy to argue it I would also argue against using TOML > in those PEPs. I partook in that discussion, and I still have no idea why TOML was chosen, over, say, a defined subset of YAML, or a slightly extended JSON. But the folks that were highly invested and putting the work in made a choice, so here we are. But if it’s in the PEPs, it seems time to define a version used ( 1.0 would be good, but often that’s actually pretty arbitrary) and get an implementation into the stdlib. If the TOML folks don’t think it’s stable enough for that, I’ve got to wonder if it was a good choice! We’re have enough trouble with really slow adoption of the new ways of doing packaging, not even providing the tools seems to really defeat the purpose. -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Why is design-by-contracts not widely adopted?
On Tue, Sep 25, 2018 at 10:10 PM Lee Braiden wrote: > It's the reason why type checking exists, and why bounds checking exists, > and why unit checking exists too. > And yet Python has none of those. They all provide safety, but also place a burden on the developer. So why use Python? I’m not arguing that all those features don’t have their advantages, but I am wondering why add them all to Python, rather than using a language designed for safety? But Python has such a great ecosystem of packages! Yes, it does — but you might ask yourself why that is. All that being said — do go and make a nice DbC package for Python — maybe all us naysayers will grow to love it! But could we please stop cluttering this list with discussion of how great or not great it is? These meta-conversations are getting really tiresome. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Moving to another forum system where
On Fri, Sep 21, 2018 at 1:24 PM James Lu wrote: > One of the reasons Guido left was the insane volume of emails he had to > read on Python-ideas. > You'd have to ask Guido directly, but I don't think so. It wasn't the volume, but the nature and timing of the discussion that was so difficult. It went on for a LONG time, with many, many circular arguments, and people commenting on issues that has already been brought up and maybe resolved. Then the kicker -- after a decision was made, there were very strong objections -- the whole process was rather ugly. One can certainly make a good case that a different system for having such discussion might have made it much better -- but I'm not so sure. But maybe it's a good case-study t guide a decision. Frankly, I'm more concerned about how an important technical discussion like that goes than I am about than issues like the recent "beautiful - ugly" thread. Maybe we need something in-between python-idea and python-dev -- a place to discuss "serious" proposals, where "serious" means somewhat fleshed out, and with the support of at least a couple key people. One of the problems with the assignment expression discussion is that it got pretty far on python-ideas, then moved to python-dev, where is was further discussed (and there were parallel thread on the two lists) -- but the two list have overlapping, but different members, so some folks were surprised at the outcome. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Moving to another forum system where
A point here: any proposal that is an actual proposal, rather than a idea that needs fleshing out, can benefit from being written down in a single document. It does NOT have to be an official PEP in order for that to happen. If you are advocating something, then write it down, post it gitHbu or some such, and point people to it -- that's it. Using this list alone (or any forum-like system) is not a good way to work out the details of the proposal -- if you have a written out version, then folks can debate what's actually in the proposal, and not the various already discussed and rejected idea that float around in multiple discussion threads... -CHB On Thu, Sep 20, 2018 at 7:46 PM, Michael Selik wrote: > On Thu, Sep 20, 2018 at 10:25 AM Anders Hovmöller > wrote: > > >>> Not for drafting, but for submitting. > > >> > > >> Can you quote pep1? I think you’re wrong. > > > > > > I can't remember if I pulled this quote previously (that's one of the > > > troubles with emails): "Following a discussion on python-ideas, the > > > proposal should be submitted as a draft PEP ..." > > > > > > Could you clarify what you think is inaccurate in the previous > statements? > > > > It later states this is just to avoid submitting bad ideas. It’s not > actually a requirement but a (supposed) kindness. > > Some regulations are de jure, others are de facto. > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Moving to another forum system where moderation is possible
On Thu, Sep 20, 2018 at 6:24 PM, Stephen J. Turnbull < turnbull.stephen...@u.tsukuba.ac.jp> wrote: > > And that's exactly what a mute on replies does. Most people will just > give up, which is appropriate. People who have (what they think is) a > good reason to continue can start a new thread with a link to the old > one. Hardly a prohibitive barrier, if you're willing to risk banning. > > I think this "feature" will do what people generally think it does: > provide a strong signal to stop the discussion, and back that up with > a fail-safe (if you *do* hit reply, it won't work). > Hmm -- I don't suppose Mailman has a way to filter out threads, does it? If not, maybe we could add that -- might work well in cases like this. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Moving to another forum system where moderation is possible
On Thu, Sep 20, 2018 at 1:39 PM, James Lu wrote: > In a forum, the beautiful is better than ugly issue would be locked. No > more posts can be added. > Exactly -- but that means we are stopping the discussion -- but we don't want to stop the discussion altogether, we want to have the productive parts of the discussion, without the non-productive parts -- not sure there is any technical solution to that problem. - CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Moving to another forum system where moderation is possible
On Tue, Sep 18, 2018 at 9:05 PM, Franklin? Lee < leewangzhong+pyt...@gmail.com> wrote: > > What people may really be clamoring for is a larger moderation team, > or a heavier hand. They want more enforcement, not more effective > enforcement. > Or maybe clamoring for nothing -- it's just not that hard to ignore a thread Frankly, I think the bigger issue is all too human -- we get sucked in and participate when we really know we shouldn't (or maybe that's just me). And I'm having a hard time figuring out how moderation would actually result in the "good" discussion we really want in an example like the "beautiful is better than ugly" issue, without someone trusted individual approving every single post -- I don't imagine anyone wants to do that. Let's just keep it on email -- I, at least, find i never participate in any other type of discussion forum regularly. -CHB > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Retire or reword the "Beautiful is better than ugly" Zen clause
On Sat, Sep 15, 2018 at 7:38 PM, Antoine Pitrou wrote: > To be fair, in my > > experience this has been a source of confusion to many Python > > newcomers, as the notion of "beauty", as with any other value > > judgment, is highly relative to the subject evaluating it. Indeed is *is* subjective -- as is "Pythonic", or "elegant", or other concept of that nature -- that is intentional. "efficient is better than inefficient" kind of goes without saying... What this merely shows, IMHO, is that writing programming slogans or > jokes on clothing you wear in public is stupid. Most people who see > them won't understand a word of them, and in some cases may badly > misinterpret them as your example shows. > > I used to think I was the only one for whom conference t-shirts could > only serve as pyjamas, well, I see them as my "geek cred" t-shirts, and part of the point is that only those those "in the know" will get it. So I don't think this says anything about wearing clothing that refers to a particular group is bad, but that one shoudl be caefule about whicj slogans you display out of context. If teh shirt said" "beuatiful code is better than ugly code" I don't think there would be an issue. As to the OP's point: We now have anecdotal evidence that "beautiful is better than ugly" can be offensive out of context. Other than that, we have people "suspecting" or "imagining" that some people "may" find it offensive in context. I try never to speak for others when saying whether something is troublesome to a community, but if we have exactly zero actual cases of someone finding it personally offensive (in context), I think we'd be going a bit overboard in making any changes. Is it any better to make a change that has not been asked for by imagining other's sensitivities than it is to ignore others' sensitivities? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Keyword only argument on function call
Did you mean to take this off-list? I hope not, as I'm bringing it back on. On Tue, Sep 11, 2018 at 8:09 AM, Anders Hovmöller wrote: > I've spent this whole thread thinking: "who in the world is writing code > with a lot of spam=spam arguments? If you are transferring that much state > in a function call, maybe you should have a class that holds that state? Or > pass in a **kwargs dict? > > > Kwargs isn’t good because it breaks static analysis which we really want. > well, Python isn't a static language, and I personally have my doubts about trying to make it more so -- but that makes it sound like we need some solution for type annotations, rather than executable code. But see my other note -- I do agree that a well specified function signature is a good thing. > for .format() -- that's why we now have f-strings -- done. > > > Not really no. F-strings can’t be used for many strings if you need to > localize your app. If you don’t have that need then yes f-strings are great > and I’m fortunate to work on an app where we don’t need to but we shouldn’t > dismiss people who have this need. > Darn -- I hope this was brought up in the original f-string conversation. > for templates -- are you really passing all that data in from a bunch of > variables?? as opposed to, say, a dict? That strikes me as getting code and > data confused (which is sometimes hard not to do...) > > Yes. For example we have decorators on our views that fetch up objects > from our DB based on url fragments or query parameters and then pass to the > function. This means we have all access control in our decorators and you > can’t forget to do this because a view without access control decorators > are stopped by a middleware. So several of variables are there before we > even start the actual view function. > hmm -- this is a bit of what I mean by mixing data and code -- in my mind a database record is more data than code, so better to use a dict than a class with attributes matching teh fields. However, I also see the advantage of mixing the code -- providing additional logic, and pre and post processing, like it sounds like you are doing. But python is a nifty dynamic language -- could your views have an "as_dict" property that provided just the fields in the existing instance? If you were in Stockholm I’d offer you to come by our office and I could > show some code behind closed doors :) > As it happens, I am much closer than usual -- in the Netherlands, but still not Stockholm :-) - CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Keyword only argument on function call
On Mon, Sep 10, 2018 at 11:00 PM, Ethan Furman wrote: > > In my day job I spend a lot of time writing/customizing modules for a > framework called OpenERP (now Odoo*). Those modules are all subclasses, > and most work will require updating at least a couple parent methods -- so > most calls look something like: > > def a_method(self, cr, uid, ids, values, context=None): > ... > super(self, parent).a_method(cr, uid, ids, values, context=context) > hmm -- this is a trick -- in those cases, I find myself using *args, **kwargs when overloading methods. But that does hide the method signature, which is really unfortunate. IT works pretty well for things like GUI toolkits, where you might be subclassing a wx.Window, and the docs for wx.Window are pretty easy to find, but for you own custom classes with nested subclassing, it does get tricky. For this case, I kinda like Steve Barnes idea (I think it is his) to have a "magic object of some type, so you can have BOTH specified parameters, and easy access to the *args, **kwargs objects. Though I'm also wary of the magic... Perhaps there's some way to make it explicit, like "self": def fun(a, b, c, d=something, e=something, , &): (I'm not sure I like the &, so think of it as a placeholder) In this case, then would be the *args tuple, and & would be the **kwargs dict (as passed in) -- completely redundant with the position and keyword parameters. So the above could be: def a_method(self, cr, uid, ids, values, context=None, , &): super(self, parent).a_method(*args, **kwargs) do_things_with(cr, uid, ...) So you now have a clear function signature, access to the parameters, and also a clear an easy way to pass the whole batch on to the superclass' method. I just came up with this off teh top of my head, so Im sure there are big issues, but maybe it can steer us in a useful direction. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Keyword only argument on function call
On Sun, Sep 9, 2018 at 7:37 AM, Anders Hovmöller wrote: I've spent this whole thread thinking: "who in the world is writing code with a lot of spam=spam arguments? If you are transferring that much state in a function call, maybe you should have a class that holds that state? Or pass in a **kwargs dict? Note: I write a lot of methods (mostly __init__) with a lot of keyword parameters -- but they all tend have sensible defaults, and/or will have many values specified by literals. Then this: > It would almost certainly become the strongly preferred way to do it for > some cases like .format() and sending a context to a template renderer in > web apps. But that’s because in those cases it is very important to match > the names. OK -- those are indeed good use cases, but: for .format() -- that's why we now have f-strings -- done. for templates -- are you really passing all that data in from a bunch of variables?? as opposed to, say, a dict? That strikes me as getting code and data confused (which is sometimes hard not to do...) So still looking for a compelling use-case -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add recordlcass to collections module
Chiming in here:
dataclasses was just added to the stdlib.
I understand that record class is not the same thing, but the use cases do
overlap a great deal.
So I think the cord goal for anyone that wants to see this in the stdlib is
to demonstrate tbat recordclass
Adds significant enough value to justify something so similar.
Personally, I don’t see it.
-CHB
On Tue, Sep 4, 2018 at 2:04 PM Zaur Shibzukhov wrote:
>
>
> ---
> *Zaur Shibzukhov*
>
>
> 2018-09-03 1:02 GMT+03:00 Wes Turner :
>
>>
>> On Sunday, September 2, 2018, Zaur Shibzukhov wrote:
>>
>>>
>>>
>>> ---
>>> *Zaur Shibzukhov*
>>>
>>>
>>> 2018-09-02 22:11 GMT+03:00 Wes Turner :
>>>
Does the value of __hash__ change when attributes of a recordclass
change?
>>>
>>> Currently recordclass's __hash__ didn't implemented.
>>>
>>
>> https://docs.python.org/3/glossary.html#term-hashable
>>
>> https://docs.python.org/3/reference/datamodel.html#object.__hash__
>>
>> http://www.attrs.org/en/stable/hashing.html
>>
>
> There is correction:
> recordclass and it's base memoryslots didn't implement __hash__, but
> memoryslots implement richcompare (almost as python's list).
>
>>
>>
>>>
On Sunday, September 2, 2018, Zaur Shibzukhov wrote:
> As the author of `recordclass` I would like to shed some light...
>
> Recorclass originated as a response to the [question](
> https://stackoverflow.com/questions/29290359/existence-of-mutable-named-tuple-in-python/29419745#29419745)
> on stackoverflow.
>
> `Recordclass` was conceived and implemented as a type that, by api,
> memory and speed, would be completely identical to` namedtuple`, except
> that it would support an assignment in which any element could be replaced
> without creating a new instance, as in ` namedtuple`. Those. would be
> almost identical to `namedtuple` and support the assignment (`
> __setitem__`
> / `setslice__`).
>
> The effectiveness of namedtuple is based on the effectiveness of the
> `tuple` type in python. In order to achieve the same efficiency it
> was necessary to create a type `memoryslots`. Its structure
> (`PyMemorySlotsObject`) is identical to the structure of` tuple`
> (`PyTupleObject`) and therefore takes up the same amount of memory as`
> tuple`.
>
> `Recordclass` is defined on top of` memoryslots` just like
> `namedtuple` above` tuple`. Attributes are accessed via a descriptor
> (`itemgetset`), which supports both` __get__` and `__set__` by the element
> index.
>
> The class generated by `recordclass` is:
>
> `` `
> from recordclass import memoryslots, itemgetset
>
> class C (memoryslots):
> __slots__ = ()
>
> _fields = ('attr_1', ..., 'attr_m')
>
> attr_1 = itemgetset (0)
> ...
> attr_m = itemgetset (m-1)
>
> def __new __ (cls, attr_1, ..., attr_m):
> 'Create new instance of {typename} ({arg_list})'
> return memoryslots .__ new __ (cls, attr_1, ..., attr_m)
> `` `
> etc. following the `namedtuple` definition scheme.
>
> As a result, `recordclass` takes up as much memory as` namedtuple`, it
> supports quick access by `__getitem__` /` __setitem__` and by attribute
> name via the protocol of the descriptors.
>
> Regards,
>
> Zaur
>
> суббота, 1 сентября 2018 г., 10:48:07 UTC+3 пользователь Martin Bammer
> написал:
>>
>> Hi,
>>
>> what about adding recordclass
>> (https://bitbucket.org/intellimath/recordclass) to the collections
>> module
>>
>> It is like namedtuple, but elements are writable and it is written in
>> C
>> and thus much faster.
>>
>> And for convenience it could be named as namedlist.
>>
>> Regards,
>>
>> Martin
>>
>>
>> ___
>> Python-ideas mailing list
>> python...@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A GUI for beginners and experts alike
> If you're talking about my PyGUI project, using it with > wPython doesn't really make sense. Now that you say that, I think I’m mingling memories — there was another project that attempted to wrap TKInter, wxPython, QT .. I always thought that was ill advised. > The goal is to provide exactly one high-quality > implementation for each platform, with as few layers as > practicable between the Python API and the platform's native > GUI facilities. So kinda like wxWidgets but in Python — which would be nice. wxPython definitely suffers from its C++ underpinnings. > Implementations on top > of wxPython, Qt etc. could probably be created, QT might make some sense— you need something on top of X, don’t you? >> I'd also make sure that you CAN "drop down" into the lower level toolkit >> fairly smoothly, if you do need something more complex that the basics. > > PyGUI provides everything you need to create a custom widget, > without needing to go down to a lower level. In that context, I was thinking about the OP’s concept— very high level, pretty much declarative. It’s going to run into limitations fast. So there should be s way to customize user interaction. Too bad all the cool kids are doing web dev these days — hard to get help with a desktop GUI project :-( > Pyjamas seems to be something like that: > > https://pypi.org/project/Pyjamas/ Or it was 6 years ago :-( -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A GUI for beginners and experts alike
while True: button, (a,b) = form.Read() try: answer = int(a) + int(b) output.Update(answer) except: pass Whoa! You really want people to write their own event loop? That seems like a bad idea to me. If you want people to not have to think about events much, maybe look at traitsui for ideas. http://docs.enthought.com/traitsui/ -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A GUI for beginners and experts alike
On Fri, Aug 24, 2018 at 10:07 AM, MRAB wrote: > A canvas can contain images, lines and shapes, which is useful for > diagrams and drawings. And the TK Canvas is kinda the "killer app" of TK. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A GUI for beginners and experts alike
A couple thoughts: You're essentially writing a new API on top of tkINter, you could probably have multiple "back-ends" -- i.e. be able to use the same code with wxPython or PySide behind it. In fact, there was an effort along these lines a few years back: http://www.cosc.canterbury.ac.nz/greg.ewing/python_gui/ I don't know that it's seen much development lately. Nope. I don't even have a need for a canvas. But what I have found in the > past is that *anything* which is a "simple wrapper over X" always ends up > in a situation where you need some feature from X that the simple wrapper > doesn't cover, and you're left with the unpalatable choice of whether to > omit functionality you want to provide, or rewrite your code to use X > directly. So my question is more about "if you hit a need for something > PySimpleGUI doesn't cover, what are your options? > This is key -- and one of the real issues around wrapping multiple GUI back-ends -- you end up having to do a lot of re-implementation of details. In fact, I always thought Pyton_guiu above really suffered from that conceptually. For example, if you used pyton-gui with a wxPython back end, you had: A python wrapper around a python wrapper around a C++ wrapper around each native toolkit. That's a lot of layers! So it's probably better to have a simpler stack -- and at the bottom, you probably need something with at least some C/C++ in it. And there are two options there: 1) re-implement widgets with native basic drawing and event handling. - this is what QT GTK, and TK do 2) Wrap the native widgets - this is what wxWidgets does The advantage of (1) is that most of your code is the same on all platform, widgets behave the same on all platforms, and there is less code to write to support a new platform. The advantage of (2) is that you get more native results -- the widgets ARE native. And it's easier to support the first platform -- you aren't writing a while GUI toolkit. But there is a lot more wrapper code to write for each new platform. And the results ARE going to be a bit platform-dependent in some places (though I've found that I need to write very little platform dependent code with wx) TK is essentially (1), though AIUI, it was originally written for X-windows, and the other platform have an X-windows emulation layer, and then all the TK code works with that. But the issue with TK is that it's really pretty old and krufty. It's great that it comes with the standard library, but now that yu can pip install pySide and wxPython, I'd consider working with one of those. I'd also make sure that you CAN "drop down" into the lower level toolkit fairly smoothly, if you do need something more complex that the basics. Finally -- I'm not sure the desktop is dead, but there is a heck of a lot going on in the browser. And if someone could write a simple GUI for a desktop app, and then easily prt that to a WebApp -- that would be great. I'm sure there are toolkits that do maybe it possible to write pure-python, and have all the javascript pre-written (Or generated) for you. While its not a complete solution, Jupyter Widgets does this within the Jupyter framework, for instance. -CHB > > >> 3. It doesn't seem to use native widgets (the buttons have a non-standard >> look on my Windows PC). >> The defaults can be easily changed. The default buttons are the one >> widget that I modify from the system default. The reason was that the >> system default is a gray button. It pretty much matches the background. >> >> If you want your buttons to all look like the system default, slip this >> line of code at the top: >> >> sg.SetOptions(button_color=sg.COLOR_SYSTEM_DEFAULT) >> > > OK. Personally, I'd argue quite strongly that "match the OS default" is > the right default for a GUI library, but that's a hugely subjective area, > and you're never going to please everyone. So do whatever works for you. > > Thank you again Paul... I learn something new from every reply > > > No problem - glad it helped. > > Paul > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A GUI for beginners and experts alike (Mike Barnett)
On Fri, Aug 24, 2018 at 8:17 AM, Barry Scott wrote: > > A graphical "hello world" in QT is only half a dozen or less lines of > > code in total, so > > meets the simplicity requirement. > > I think 2 lines is simple, 12 is not really simple, is it? > well, if all you need is a single dialog box with one or two entry fields, then yes, 2-4 lines is better than 12-14 lines -- but a lot. But if you are building an actual application, I'm not sure that the extra ten lines of startup code matters at all. And if those ten lines are essentially boilerplate that you can copy and paste from somewhere (Or have a gui-builder write for you), then even less so. That's not to say that the GUI toolkit slike wxPython, PySide, pyGTK aren't a bit more complex than they need to be, but much of that complex is there to support complex needs. I do think there is a place for tools that make "the easy stuff easy", but make sure that the complex stuff is still possible. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A simple proposal concerning lambda
On Wed, Aug 22, 2018 at 9:51 AM, Stephan Houben wrote: > Let me stand up and say that I personally like lambda. It's the standard > terminology and not easily confused. > I agree. And secondly, even if I didn't like it, changing the name of something because it's a slightly less confusing name to newbies is a really low priority! If you look back at previous discussion, the real push was to exp[and lambda's capabilities -- maybe allow more than one expression, or ??? But that would make it MORE confusing to newbies, not less. I actually think it makes a strikes a pretty good balance -- to use the idiomatic phrase :-) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Jump to function as an an alternative to call function
On Mon, Aug 20, 2018 at 7:21 AM, Steven D'Aprano wrote: > * Introducing a warning makes it clear that this is not a de facto > language standard, but a mere implementation detail subject to > change if somebody comes up with a better optimization for locals. > defacto standards are sub-optimum -- the docs say "may not" -- that seems really sketchy to me. Even if there is no change to the implementation of cPython, I'd like to see the behavior clearly defined -- if I pass a object returned by "locals()" to a function, and that function modifies that object -- will, or will not, the local namespace be altered? Saying it "may" be altered is kind of crazy! Does that mean the same code will have a different effect if run in two different (compliant) implementations of Python? That sure seems like a bad idea... > more so, I do not believe that anyone will rely or use such a feature. > well, this thread exists because someone wanted to do something like that -- i.e. manipulate the calling namespace from within a function. I suggested that passing locals() in might work -- it didn't work (in cPython), but if it HAD worked in whatever implementation the OP is using, then, yes, someone would now be relying on that feature. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Off topic: 'strike a balance' - second language English
>
> > Summary: I look at the phrase 'strike a balance' in different languages,
>
> It is interesting that you picked up on "strike a balance" which has
> been a standard English phrase for a very long time rather than the much
> more resent, (and itself a form of jargon), "dumbing down".
>
> The other point is that the use of Jargon is often as a form of
> shorthand so as to avoid excessive verbosity, (or long windedness).
>
We are (maybe) mingling two issues here -- there is an important
distinction between idiomatic expressions ("striking a balance", "dumbing
down") and technical terms (jargon).
If you want to make it easier for non-native english speakers to understand
-- minimal use of idiomatic expressions is a good idea. They really don't
serve much real purpose, other than making the prose more colorful and
friendly (to those that understand it). Sometimes a bit of brevity is
gained, but not much.
Technical jargon, on the other hand, can be very helpful for precision and
compactness.
(side note -- are all domain-specific technical term "jargon"? I tend to
see "jargon" as having a negative connotation -- specifically that it isn't
required for technical specificity. That is, "jargon" is language that is
unnecessarily domain specific)
I think it's pretty important to use the common domain specific terms in
introductory texts -- how else will folks learn them? So I make a
distinction between *using* a technical term, and *introducing* a technical
term.
In fact, in my PR on Jonathan's doc on None, I deliberately introduced the
term "Singleton" -- not because it was necessary to understand the idea at
hand, but because people are likely to encounter the term elsewhere.
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Jump to function as an an alternative to call function
By the way— really kludgy, doesn’t exec() do what you want here: Note The default *locals* act as described for function locals() <https://docs.python.org/3/library/functions.html#locals> below: modifications to the default *locals* dictionary should not be attempted. Pass an explicit *locals*dictionary if you need to see effects of the code on *locals* after function exec() <https://docs.python.org/3/library/functions.html#exec> returns. Though great still may not effect the current local namespace. But I’d be really surprised if there were no way to modify the local namespace — you can mess with pretty much anything else in Python. -CHB Sent from my iPhone On Aug 17, 2018, at 12:46 PM, Chris Barker wrote: On Thu, Aug 16, 2018 at 12:37 PM, Chris Angelico wrote: > > I've no idea what interpreter you're using, but it doesn't work for me. > That was in iPython, with python3.6.2 -- I wouldn't have expected it to be different in this case though -- really odd. OK -- tired again, and it indeed it failed -- so I'm really confused. You can see my console output -- it did indeed work at least once. > You've changed a cached dictionary but haven't actually created a local > which still makes me wonder WHY locals() returns a writable dict... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Jump to function as an an alternative to call function
On Thu, Aug 16, 2018 at 12:37 PM, Chris Angelico wrote: > > I've no idea what interpreter you're using, but it doesn't work for me. > That was in iPython, with python3.6.2 -- I wouldn't have expected it to be different in this case though -- really odd. OK -- tired again, and it indeed it failed -- so I'm really confused. You can see my console output -- it did indeed work at least once. > You've changed a cached dictionary but haven't actually created a local > which still makes me wonder WHY locals() returns a writable dict... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Jump to function as an an alternative to call function
one more thought: given that: > > The contents of this dictionary should not be modified; changes may not >> affect the values of local and free variables used by the interpreter. >> > > and: > """ > locals() > > Update and return a dictionary representing the current local symbol table. > I wonder why locals doesn't return a Mapping Proxy, or other read-only mapping object? If it's not guaranteed to be THE locals dict, and changes *may* not affect the real one (but may), a read-only object seems like much safer idea. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Jump to function as an an alternative to call function
On Thu, Aug 16, 2018 at 10:33 AM, Jonathan Fine wrote:
> >> there
> >> are times where I have really wanted to access the caller's environment,
> >> not the environment where my function was defined.
>
> > what am I missing? can't you get that by passing locals() in to a
> function?
>
> I think this will fail when values are changed. According to
> https://docs.python.org/3/library/functions.html#locals
> > The contents of this dictionary should not be modified; changes may not
> affect the values of local and free variables used by the interpreter.
>
and:
"""
locals()
Update and return a dictionary representing the current local symbol table.
Free variables are returned by locals() when it is called in function
blocks, but not in class blocks.
"""
it a dictionary *representing* the current local symbol table, so not the
actual symbol table -- which surprised me, I thought that is was -- and
sometimes it appears to be -- see later example:
> To finish, here's an interactive example of changing the value of
locals().
>
> >>> def f(a=1): loc = locals(); yield loc; yield a
> >>> it = f()
> >>> ctx = next(it)
> >>> ctx
> {'a': 1} ## This surprised me.
>
me too I would have thought "loc" would be in there.
> >>> def f(a=1): loc = locals(); yield locals(); yield a
> >>> it = f()
> >>> ctx = next(it)
> >>> ctx
> {'a': 1, 'loc': {...}}
>
now "loc is there"
> >>> ctx['a'] = 3
> >>> ctx['loc']['a'] = 5
> >>> next(it) ## Is it 1 or 3 or 5?
> 1 ## The value of 'a' hasn't changed.
hmm -- made me think that generators are doing something different here --
and indeed they are. If you use regular functions:
In [30]: def local_modifying(loc):
...: """
...: adds a "fred" key to the dict passed in
...: """
...: print("locals passed in:", loc)
...: loc['fred'] = 5
...: print("locals after adding", loc)
...:
In [31]: def test_locals():
...: """
...: a simple local namespace to use
...: """
...: a = 1
...: b = 2
...: local_modifying(locals())
...: # does "fred" exist?
...: print(locals())
...: # and we can access it the usual way
...: print("fred:", fred)
...:
In [32]: test_locals()
locals passed in: {'b': 2, 'a': 1}
locals after adding {'b': 2, 'a': 1, 'fred': 5}
{'b': 2, 'a': 1, 'fred': 5}
fred: 5
It seems you CAN modify the locals dict passed in, and the change will show
up in the enclosing scope.
But it sounds like that is not guaranteed by the language.
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Jump to function as an an alternative to call function
On Thu, Aug 16, 2018 at 1:28 AM, Steven D'Aprano wrote: > there > are times where I have really wanted to access the caller's environment, > not the environment where my function was defined. > what am I missing? can't you get that by passing locals() in to a function? A bit clunky, sure, but everyone seems to agree that this a fairly rarely used approach. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Python docs: page: In what ways in None special
Since I already spent a bunch of time on this, I did a PR: https://github.com/jfine2358/py-jfine2358/pull/2 further discussion should probably be in that PR / repo -CHB On Wed, Aug 15, 2018 at 9:02 AM, Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: > > None is keyword, and just like any other keyword, it can't be re-bound. > > > >> it's only a keyword because Python doesn't otherwise have a way of > creating non-rebindable names. It's purpose is to represent the singular > object of NoneType, and in that sense it's a literal as much as [] or "". > > We’re getting kind of pedantic here, but no, it’s not “as much as” — [] > and “” create new instances of a list or string. > > For the purposes of this document, however, these are pretty esoteric > distinctions. > > What the docs should make clear is that None ( and True and False ) is a > singleton— None will always refer to the SAME None object. > > And that can be demonstrated by showing that you can’t rebind the name > None. > > But I think it’s misleading to say that that is the same as: > > 42 = “something else” > > None is syntactical a name like any other — what’s special about is that > it can’t be rebound. > > -CHB > > > > > -- > > Rhodri James *-* Kynesim Ltd > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Python docs: page: In what ways in None special
> None is keyword, and just like any other keyword, it can't be re-bound. >> it's only a keyword because Python doesn't otherwise have a way of creating >> non-rebindable names. It's purpose is to represent the singular object of >> NoneType, and in that sense it's a literal as much as [] or "". We’re getting kind of pedantic here, but no, it’s not “as much as” — [] and “” create new instances of a list or string. For the purposes of this document, however, these are pretty esoteric distinctions. What the docs should make clear is that None ( and True and False ) is a singleton— None will always refer to the SAME None object. And that can be demonstrated by showing that you can’t rebind the name None. But I think it’s misleading to say that that is the same as: 42 = “something else” None is syntactical a name like any other — what’s special about is that it can’t be rebound. -CHB > > -- > Rhodri James *-* Kynesim Ltd ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Python docs: page: In what ways in None special
On Tue, Aug 14, 2018 at 10:45 AM, Rhodri James wrote: > On 'None is a constant': > > Erm. I think you've got carried away with simplifying this and gone down > a blind alley. None is a literal, and like any other literal can't be > rebound. no, it's not -- None is keyword, and just like any other keyword, it can't be re-bound. However, every other keyword I tried to rebind results in a generic: SyntaxError: invalid syntax (except None, True, and False) which I suppose is because while None is a keyword, it can be used pretty much anywhere any other name can be used (as opposed to say, def) Either this entire section is irrelevant or you meant to explain that there > is only one "NoneType" object. > > Constant is a bit of a loaded term in Python, and I think you've fallen > foul of it here. > yes, I think "singleton" is the word you want here, though it is a bi CS-y :-( -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Python docs page: In what ways is None special
On Tue, Aug 14, 2018 at 10:25 AM, David Mertz wrote: > Great work! There are a few typos, I'll try to get to a PR on those. > > I wonder if it's worth noting that None is a singleton, while 42 is just a > value. I.e. there might be several distinct objects that happen to be the > int 42, but not so with None. > very much worth nothing -- and I think re-wording that example. The fact that you can't assign to None is orthogonal to the fact that it's immutable. in fact, [42] is a llteral for a list with one element, the integer with the value of 42 in it. It is very much a mutable. and yet: In [7]: [42] = 34 File "", line 1 [42] = 34 SyntaxError: can't assign to literal whereas: [5].append(3) works -- at least it does not result in an error -- even though it's useless. And the reason this matters is that name binding (and rebinding) is very much a different operation than mutating -- we should not conflate those. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Syntactic sugar to declare partial functions
> > Do we often call functools.partial on arbitrary callable objects that we > don't know in advance? For my part, I don’t think I’ve ever used partial in production code. It just seems easier to simply fo it by hand with a closure ( Am I using that term right? I still don’t quite get the terminology) And if I had a callable class instance I wanted to “customize”, I’d likely use an OO approach — parameterize the instance, or subclass. So yeah, partial is probably used primarily with “regular” functions. But then, I don’t know that we need any easier syntax anyway — maybe I’d be more likely to use it, but it kind of feels like a feature that’s there so we can write more functional code for the sake of writing more functional code. If someone’s really interested in this, a search on gitHub or something to see how it’s used in the wild could be enlightening. -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Python certification
On Thu, Aug 9, 2018 at 10:40 AM, Jonathan Fine wrote: > There is, implicitly, a > syllabus in https://docs.python.org/3/tutorial/. > The tutorial, is, well, a tutorial, it is by no means a complete course of study. So no, I don't think it's an appropriate place to start to develop a certification. Here's a couple of Python syllabuses (the first proprietary, the > second perhaps open) > > https://www.microsoft.com/en-us/learning/exam-98-381.aspx > https://pythoninstitute.org/pcap-exam-syllabus/ I am developing this: https://uwpce-pythoncert.github.io/PythonCertDevel/index.html Which we use for a "Certificate" program, which is NOT a "certification". But I like to think we've put together a pretty good curriculum. I hope this is enough to persuade you that this topic is appropriate > for python-ideas. Of course, if you know a better forum for this, that > would be welcome. This really isn't the right forum -- maybe the tutor list? https://mail.python.org/mailman/listinfo/tutor But if you think you want to develop a certification, you need to find like-minded people to do it. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting dedicated overloadable boolean operators
On Mon, Aug 6, 2018 at 11:11 AM, Chris Barker wrote: > So any new class that doesn't already make use of the bitwise operators > can do that. > just like set() -- which I think has been mentioned here already. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting dedicated overloadable boolean operators
On Fri, Aug 3, 2018 at 9:13 PM, Todd wrote: > >> Also, in a common use-case, bitwise-and behaves the same as logical_and, >> e.g. >> >> if (arr > x) & (arr2 == y) >> >> This "works" because both arrays being bitwise-anded are boolean arrays. >> > > There are a few problems with using the bitwise operators. > > First, and most important in my opinion, is that the precedence is > significantly off from that of the logical operators. > yes, that's true, and perhaps too bad, but as they are spelled differently, not a killer. if you are switching back and forth between, say, array logical operations > and "normal" logical operations it is easy to mess up. > well, as you generally are working with arrays or not, again, not too bad. > Third is that it allows both boolean and bitwise operations to be carried > out on the same data types. Numpy is a special case where the two > basically are equivalent if you are working with boolean arrays. But that > is a special case. > I kind of muddled my point -- the main trust was that overloading the bitwise operators to do logical operations is a fine idea -- many objects will have no or limited use for bitwise operations. In fact, if I were to re-design the numpy API, I would overload the bitwise operators to do logic, and use the special functions for bitwise operations: np.bitwise_and etc. rather than having to use logical_and and friends the way we do now. So any new class that doesn't already make use of the bitwise operators can do that. (yes, still the precedence issue, but what can you do?) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting dedicated overloadable boolean operators
On Fri, Aug 3, 2018 at 1:02 PM, Nicholas Chammas wrote: > The project overloaded the bitwise operators &, |, and ~ since they > could not > override the boolean operators and, or, and not. > > I actually think that is a good solution to this problem -- the fact is that for most data types bitwise operators are useless -- and for even more not-very-useful. numpy did not do this, because, as it happens, bitwise operators can be useful for numpy arrays of integers (though as I write this, bitwise operations really aren't that common -- maybe requiring a function call for them would be a good way to go -- too late now). Also, in a common use-case, bitwise-and behaves the same as logical_and, e.g. if (arr > x) & (arr2 == y) This "works" because both arrays being bitwise-anded are boolean arrays. So you really don't need to call: np.logical_and and friends very often. so -1 on yet another set of operartors. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] With expressions
On Fri, Aug 3, 2018 at 3:49 AM, Robert Vanden Eynde wrote: > > When I say "functional programming", I speak about the paradigm used in > language like Haskell. In language like those, all constructs are > "expression-based". > sure -- but Python is explicitly NOT a functional language in that sense. It does support some functional paradigms, but features are not added to conform to the functional paradigm per-se, but because they are considered useful features. So one needs to defend a new proposal with arguments about how it makes python programming better (by SOME definition of better) on its own merits. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Redefining method
On Mon, Jul 30, 2018 at 9:10 AM, Nick Coghlan wrote: If you need to replace them for some reason, it will preferably be > within a temporary bounded scope, using a tool like > unittest.mock.patch, rather than as a permanent change that affects > every other use of the class within the process. yup -- but you can still do a raw monkey-patch anyway. You asked for: > Thank you for your question. You asked why not > >> c = MyClass > >> o = c() > >> > >> def c.foo(cls): ... > do you mean this? you want a classmethod? or is this what you mean? -- which you can do: C = MyClass def foo(self, some_param): some_code() C.foo = foo (note that I used a capital C -- capitalized names are traditional for classes -- see PEP 8) In that case, any existing, and new instances of the C class will now have the foo method. Also -- that line: C = MyClass -- binds the name "C" to the very same class object as MyClass -- so you will have just monkey=patched MyClass -- I"m guessing you didn't want that. If not, and you wanted C t be a copy of MyClass that you could then change/add/remove methods from, then you want subclassing -- that is exactly what it is for. Then there is this: > >> def o.bar(self): ... > which is monkey patching an instance object, which you can also do, but you don't get a bound method -- i.e. it doesn't get self passed in automatically. -CHB > > I've the same same query, but never had the courage to ask. So that > > you for asking. And also giving me a chance to share my thoughts. > > It's essentially due to the fact that while we deliberately allow > runtime monkeypatching (as it's sometimes the best available answer), > we also strongly encourage the idea of treating it as a last resort > option (outside specific use cases like testing). > > So if you want to define methods on a class, the strongly preferred > place to define them is inside the class definition, where they're > easy for future maintainers of that class to find. > > > Cheers, > Nick. > > P.S. While it's *not* explicitly part of Python's design rationale, > http://connascence.io/locality.html and the rest of that site provide > some good info on the kinds of problems that "action at a distance" > effects, like monkeypatching class definitions, can cause in a code > base. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Python docs page: In what ways is None special
I agree that some more docs on the specialness of None (and, to a
lessor extent, True and False).
A few comments:
> None is a keyword
> ==
None = 0
> SyntaxError: can't assign to keyword
One of the implications of this is that “None” will always be the
Singleton None object — so you can (and should) use:
Something is None
To test for None.
> The Command Line Interpreter hides None
> =
None
That’s a good one to highlight!
>
> None is false in a boolean context
> ==
bool(None)
> False
Maybe this belongs more in a discussion of “Falseyness”
> Procedures return None
> ==
a = [3,1,2]
b = a.sort()
a, b
> ([1, 2, 3], None)
This is less about None than about the convention that mutating
methods return None. Maybe that discussion belongs elsewhere.
> Dictionary get returns None if not found
> ==
{}.get('dne') is None
> True
Belongs with dict docs really, and not really true — dict.get()
returns the default value, which is None be default.
> None is default return value
> =
def fn(): pass
> ...
fn() # No response!
print(fn()) # Here's why.
> None
Yup.
> None is used as a sentinel default value
> ==
> Particularly useful when default value must be determined
> in body of function.
> ---
> def insort_right(a, x, lo=0, hi=None):
># ...
>if hi is None:
>hi = len(a)
> ---
This is also a convention — and primarily applies to mutable defaults,
which you hardly ever want to assign directly.
So a good example of None being used as a sentinel, but nog really
anything special about None.
-CHB
> /
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Python docs page: In what ways is None special
>lot. Actually, the ?. and ?[ > operators seem like they'd be much more useful if I did more JSON > processing - This has been mentioned a lot in this discussion— Maybe what we need is a smarter JSON processing package, rather than new operators. Granted, these operators would help the authors of suck a package(s), but if the bulk of users didn’t need them, then no point in adding them to the language. -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Revisiting str.rreplace()
I starting g reading this thread in the middle, on a phone. But was very confused for a while because I didn’t notice that there were two ‘r’s at the beginning of .rreplace Just sayin’ -CHB Sent from my iPhone > On Jul 19, 2018, at 9:29 AM, Paul Moore wrote: > >> On 19 July 2018 at 16:25, Eric V. Smith wrote: >> It currently does something: it replaces all instances, just as if you >> hadn't supplied a count (see my example below). You can't change its >> behavior. > > ... without a deprecation cycle. Which is of course not worth it for > something which could much more easily be done by adding an rreplace > function - which is the real point of the comment. > Paul > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Performance improvements via static typing
A note here:
Earlier in the conversation about standardizing type hinting, I (among
others) was interested in applying it to C-level static typing (e.g.
Cython).
Guido made it very clear that that was NOT a goal of type hints —
rather, they were to be used for dynamic, python style types — so a
“list” is guaranteed to act like a list in python code, but not to
have the same underlying binary representation.
We could still use the same syntax for things like Cython, but they
would have a different meaning.
And a JIT compiler may not benefit from the hints at all, as it would
have to check the actual type at run-time anyway.
-CHB
Sent from my iPhone
> On Jul 20, 2018, at 2:32 AM, Stefan Behnel wrote:
>
> Michael Hall schrieb am 19.07.2018 um 15:51:
>> While I am aware of projects like Cython and mypy, it seems to make sense
>> for CPython to allow optional enforcement of type hints, with compiler
>> optimizations related to it to be used. While this would not receive the
>> same level of performance benefits as using ctypes directly, there do
>> appear to be various gains available here.
>
> Well, first of all, a C level type check at runtime is quite fast compared
> to a byte code dispatch with stack argument handling, and can be done
> regardless of any type hints. There are various code patterns that would
> suggest a certain type ("x.append()" probably appends to a list, "x.get()"
> will likely be a dict lookup, etc.), and that can be optimised for without
> static type declarations and even without any type hints.
>
> Then, the mere fact that user code says "a: List" does not help in any way.
> Even "a: list" would not help, because any behaviour of that "list" might
> have been overridden by the list subtype that the function eventually
> receives.
>
> The same applies to "x: float". Here, in order to gain speed, the compiler
> would have to generate two separate code paths through the entire function,
> one for C double computations, and one for Python float subtypes. And then
> exponentiate that by the number of other typed arguments that may or may
> not contain subtypes. Quite some overhead.
>
> It's unclear if the gain would have any reasonable relation to the effort
> in the end. Sure, type hints could be used as a bare trigger for certain
> optimistic optimisations, but then, it's cheap enough to always apply these
> optimisations regardless, via a C type check.
>
> Stefan
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Multi-core reference count garbage collection
> You wrote: > > I'd argue that the ref counts are not interesting at all, only a > > side effect of one possible solution to the object life time problem. > > I'm happy for you to regard multi-core reference counting (MCRC) as a toy > problem, which won't become part of useful software. Perhaps the point was that reference counting is only one of the issues (race conditions?) the GIL solves. So a GIL - free reference counting scheme may not prove to be helpful. If that were it, we could (optionally) turn off reference counting in multi-threaded code, and run a garbage collector once in a while instead. After all, cPython’s (mostly) deterministic garbage collection is not guaranteed by the language standard. -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] if we were to ever shorten `A if A else B` (was: PEP 505: None-aware operators)
> my vote would go to `A otherwise B` since it's unambiguous, the case you care > about the state of comes first, and it doesn't trip your brain up looking > for 'if'. :) And I’d hope “otherwise” is a rare variable name :-) - CHB > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Including the unparse module in the standard library
On Mon, Jul 16, 2018 at 12:24 PM, Brett Cannon wrote: > > Since it isn't necessary for Python to function, I would say we probably > don''t want to pull it up. Then the maintenance burden grows much more. > might make sense to put it on pypi though, if someone want to take responsibility for it. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Fwd: grouping / dict of lists
On Fri, Jul 13, 2018 at 12:38 PM, Michael Selik wrote:
> Thanks for linking to these.
>
yup -- real use cases are really helpful.
Though the other paradigm for grouping is use of setdefault() rather than
defaultdict. So it would be nice to look for those, too.
> I looked at many of them in my own research, but for some reason didn't
> think to write down the links. I'll respond to each one separately.
>
> Throughout, I'm going to use my proposed ``grouped`` builtin to
> demonstrate possible revisions. Note that I am *not* suggesting a
> replacement to defaultdict. The proposal is to make a common task easier
> and more reliable. It does not satisfy all uses of defaultdict.
>
agreed -- and it shouldn't.
I"d like to see how some of these pan our with my proposed API:
either a Grouped class, or at least (key, value) iterables and/or a value
function.
I don't have time now to do them all, but for the moment:
I noticed recently that *all* examples for collection.defaultdict (
>> https://docs.python.org/3.7/library/collections.html#
>> collections.defaultdict) are cases of grouping (for an int, a list and a
>> set) from an iterator with a key, value output.
>>
>
and yet others on this thread think a (key, value) input would be rare -- I
guess it depends on whether you are thinking dict-like already
>
>> https://frama.link/o3Hb3-4U,
>>
>
> accum = defaultdict(list)
> garbageitems = []
>
> for item in root:
> filename = findfile(opts.fileroot, item.attrib['classname'])
> accum[filename].append(float(item.attrib['time']))
> if filename is None:
> garbageitems.append(item)
>
>
> This might be more clear if separated into two parts.
>
> def keyfunc(item):
> return findfile(opts.fileroot, item.attrib['classname'])
> groups = grouped(root, keyfunc)
> groups = {k: [float(v.attrib['time']) for v in g] for k, g in
> groups.items()}
> garbage = groups.pop(None, [])
>
so this one is a prime case for a value function -- I think post-processing
the groups is a pretty common case -- why make people post-process it?
def keyfunc(item):
return findfile(opts.fileroot, item.attrib['classname'])
def valuefunc(item):
float(item.attrib['time'])
groups = grouped(root, keyfunc, valuefunc)
garbage = groups.pop(None, [])
And the post-processing is then mixing comprehension style with key
function style (what to call that -- "functional" style?), so why not use a
(key, value) iterable:
groups = grouped((findfile(opts.fileroot, item.attrib['classname']),
item.attrib['time'])
for item in root))
OK -- that's packing a bit too much into a line, so how about:
def keyfunc(item):
return findfile(opts.fileroot, item.attrib['classname'])
groups = grouped( (keyfunc(item), item.attrib['time']) for item in root)
>
> self.mapping = collections.defaultdict(set)
> for op in (op for op in graph.get_operations()):
> if op.name.startswith(common.SKIPPED_PREFIXES):
> continue
> for op_input in op.inputs:
> self.mapping[op_input].add(op)
>
>
> This is a case of a single element being added to multiple groups, which
> is your section B, below. The loop and filter could be better. It looks
> like someone intended to convert if/continue to a comprehension, but
> stopped partway through the revision.
>
yeah, this is weird --
But it does make a case for having a class with the option f using a set to
collect (which I have in an older commit of my prototype:
inputs = ((op_input, op) for op in ops for op_input in op.inputs)
groups = Grouping(inputs, key=itemgetter(0), collection=set)
otherwise, you could have a method to do it:
groups.map_on_groups(set)
(not sure I like that method name, but I hope you get the idea)
OK, back to work.
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add the imath module
On Thu, Jul 12, 2018 at 11:05 AM, Steven D'Aprano wrote: > I'm not sure that we need a specific module for integer-valued maths. I > think a more important change would be to allow math functions to be > written in Python, not just C: > > - move the current math library to a private _math library; > > - add math.py, which calls "from _math import *". > FWIW, Guido rejected that idea when we added math.isclose() -- Victor had even written it up already. Not that people never change their mind... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Fwd: grouping / dict of lists
On Mon, Jul 9, 2018 at 5:55 PM, Franklin? Lee wrote: > >> - The storage container. > > > > > > so this means you'r passing in a full set of storage containers? I'm a > vit > > confused by that -- if they might be pre-populated, then they would need > to > > be instance,s an you'd need to have one for every key -- how would you > know > > in advance aht you needed??? > > No, I mean the mapping (outer) container. For example, I can pass in > an empty OrderedDict, or a dict that already contained some groups > from a previous call to the grouping function. > Sure -- that's what my prototype does if you pass a Mapping in (or use .update() ) why not? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Fwd: grouping / dict of lists
On Mon, Jul 9, 2018 at 3:38 PM, David Mertz wrote: > In my mind, I *rarely* (which is more than never) have my data in the form > of a sequence of key/value pairs. The version of the API that assumes data > starts that way feels like either a niche case, or demands preprocessing > before it's ready to pass to grouping() or collections.Grouping(). > sure, but it makes it easy to use a different approach -- i.e. a comprehension with expressions rather than a key (and maybe value) function. AT least two people have expressed a preference for that. That said, an identity key is rarely interesting either. > is it EVER interesting?? wouldn't it be essentially a Counter, without the counting? :-) > So I think have key=None mean "assume we get key/val pairs is harmless to > the more common case where we give an explicit key function. > I'm not sure we'll know what's more common 'till it's loose in the wild -- any problem can be solved either way -- which one will people prefer? Given that there Is a preference for comprehensions over map() -- I think it will see a lot of use. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Fwd: grouping / dict of lists
On Fri, Jul 6, 2018 at 12:26 PM, Franklin? Lee I use this kind of function in several different projects over the > > years, and I rewrote it many times as needed. > > I added several options, such as: > - key function > - value function > - "ignore": Skip values with these keys. > - "postprocess": Apply a function to each group after completion. > - Pass in the container to store in. For example, create an > OrderedDict and pass it in. It may already hold items. > - Specify the container for each group. > - Specify how to add to the container for each group. > interesting... > Then I cut it down to two optional parameters: > - key function. If not provided, the iterable is considered to have > key-value pairs. > OK -- seems we're all converging on that :-) > - The storage container. > so this means you'r passing in a full set of storage containers? I'm a vit confused by that -- if they might be pre-populated, then they would need to be instance,s an you'd need to have one for every key -- how would you know in advance aht you needed??? I played around with passing in a optional storage object: https://github.com/PythonCHB/grouper/commit/d986816905406ec402724beaed2b88c96df64469 but as we might want a list or a set, or a Counter, or ??? it got pretty ugly, as lists and sets and Counters all have different APIs for adding stuff. So I gave up and figured just saying "it's always a list) made the most sense. > Finally, I removed the key function, and only took pairs and an > optional container. However, I don't remember why I removed the key > function. It may be that I was writing throwaway lambdas, and I > decided I might as well just write the transformation into the > comprehension. exactly -- but I suspect hat may be because you where writing a comprehension anyway, as you needed to manipulate the values, also -- so if there were a value function, you could use either API. > I think a key function is worth having. > I think there's more or less consensus on that too. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Where should grouping() live
On Fri, Jul 6, 2018 at 5:13 PM, Michael Selik wrote:
> On Tue, Jul 3, 2018 at 10:11 PM Chris Barker via Python-ideas <
> python-ideas@python.org> wrote:
>
>> * There are types of data well suited to the key function approach, and
>> other data not so well suited to it. If you want to support the not as well
>> suited use cases, you should have a value function as well and/or take a
>> (key, value) pair.
>>
>> * There are some nice advantages in flexibility to having a Grouping
>> class, rather than simply a function.
>>
>
> The tri-grams example is interesting and shows some clever things you can
> do. The bi-grams example I wrote in my draft PEP could be extended to
> handle tri-grams with just a key-function, no value-function.
>
hmm, I'll take a look -- 'cause I found that I was really limited to only a
certain class of problems without a way to get "custom" values.
Do you mean the "foods" example?
>>> foods = [
... ('fruit', 'apple'),
... ('vegetable', 'broccoli'),
... ('fruit', 'clementine'),
... ('vegetable', 'daikon')
... ]
>>> groups = grouping(foods, key=lambda pair: pair[0])
>>> {k: [v for _, v in g] for k, g in groups.items()}
{'fruit': ['apple', 'clementine'], 'vegetable': ['broccoli', 'daikon']}
Because that one, I think, makes my point well. To get what you want, you
have to post-processthe Grouping with a (somewhat complex) comprehension.
If someone is that adept with comprehensions, and want to do it that way,
the grouping function isn't really buying them much at all, over
setdefault, or defaultdict, or roll your own.
Contrast this with:
groups = grouping(foods,
key=lambda pair: pair[0],
value=lambda pair: pair[1])
and you're done.
or:
groups = grouping(foods,
key=itemgetter(0),
value=itemgetter0))
Or even better:
groups = grouping(foods)
:-)
However, because this example is fun it may be distracting from the core
> value of ``grouped`` or ``grouping``.
>
Actually, I think it's the opposite -- it opens up the concept to be more
general purpose -- I guess I'm thinking of this a "dict with lists as the
values" that has many purposes beyond strictly "groupby". Maybe that's
because I'm a general python programmer, and not a database guy, but if
something is going to be added to the stdlib, why not add a more general
purpose class?
> I don't think we need a nicer API for complex grouping tasks. As the tasks
> get increasingly sophisticated, any general-purpose API will be less nice
> than something built for that specific task.
>
I guess this is where we disagree -- I think we've found an API that is
general purpose, and cleanly supports multiple tasks.
Instead, I want the easiest possible interface for making groups for
> every-day use cases. The wide range of situations that ``sorted`` covers
> with just a key-function suggests that ``grouped`` should follow the same
> pattern.
>
not at all -- sorted() is about, well, sorting -- which means rearranging
items. I certainly don't expect it to break up the items for me.
Again, this is a matter of perspective -- if you you start with "groupby"
as a concept, then I can see how you see the parallel with sorted -- you
are rearranging the items, but this time into groups.
But if you start with "a dict of lists", then you take a wider perspective:
- It can naturally an easily be used to group things
- It can do another nifty things
- And as a "dict of something", it's natural to think of keys AND values,
and to want a dict-like API -- i.e. pass in (key, value) pairs.
I do think that the default, key=None, could be set to handle (key, value)
> pairs.
>
OK, so for my part, if you provide the (key, value) pair API, then you
don't really need a value_func. But as the "pass in a function to process
the data" model IS well suited to some tasks, and some people simply like
the style, why not?
And it creates an asymetry: or you have a (key, the_item) problem, you can
use either the key function API or the (key, value) API -- but if you have
a (key, value) problem, you can only use the (key, value) API
But I'm still reluctant to break the standard of sorted, min, max, and
> groupby.
>
This is the power of Python's keyword parameters -- anyone coming to this
from a perspective of "I expect this to be like sorted, min, max, and
groupby" can simply ignore the value parameter :-)
One more argument :-)
There have been comments a bout how maybe some of the classes in
collections are maybe not needed -- Counter, in particular. I tend to
agree, but i think the reason Counter is not-that-useful is because it
doesn't do enough -- not that it isn't useful -- it's just such a thin
wrapper around a dict, that I hardly
Re: [Python-ideas] Where should grouping() live (was: grouping / dict of lists)
On Jul 6, 2018, at 2:10 AM, Steven D'Aprano wrote: I would consider statistics to have similarities - median, mean etc are aggregate functions. Not really, more like reduce, actually -/ you get a single result. Histograms are also doing something similar to grouping. .(Yes, a few statistics apply to nominal and ordinal data too, And for that, a generic grouping function could be used. In fact, allowing Counter to be used as the accumulater was one suggestion in this thread, and would build s histogram. Now that I think about it, you could write a key function that built a histogram for continuous data as well. Though that might be a bit klunky. But if someone thinks that’s a good idea, a PR for an example would be accepted: https://github.com/PythonCHB/grouper -CHB -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Where should grouping() live (was: grouping / dict of lists)
On Thu, Jul 5, 2018 at 3:26 AM, David Mertz wrote: > Yes, he said a definite no to a built-in. But he expressed a less specific > lack of enthusiasm for collections classes (including Counter, which exists > and which I personally use often). > And a Grouping class would do more than Counter, which I find trivial enough that I generally don't bother to use it. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Fwd: grouping / dict of lists
On Wed, Jul 4, 2018 at 6:34 AM, David Mertz wrote: > You've misunderstood part of the discussion. There are two different > signatures being discussed/proposed for a grouping() function. > > The one you show we might call grouping_michael(). The alternate API we > might call grouping_chris(). These two calls will produce the same result > (the first output you show) > > grouping_michael(words, keyfunc=len) > grouping_chris((len(word), word) for word in words) > > I happen to prefer grouping_michael(), but recognize they each make > slightly different things obvious. > I starting thinking grouping_chris was the obvious and natural thing to do, but his discussion has made it clear that grouping_michael is more natural for some kinds of data. and in some cases, it really comes down to taste, after all, who's to say which of these is "better" map(func, iterable) or (expression for item in iterable) given that map existed in Python when comprehensions were added, I tend to see the latter as more "Pythonic" but that's just me. So I'm currently lobbying for both :-) The default is iterable of (key. value) pairs, but the use can specify a key function is they want to do it that way. While a bit of a schizophrenic API, it makes sens (to me), because grouping_mikael isn't useful with a default key function anyway. The other enhancement I suggest is that an (optional) value function be added, as there are use cases where that would be really helpful. -CHB NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Where should grouping() live (was: grouping / dict of lists)
On Tue, Jul 3, 2018 at 6:23 AM, David Mertz wrote: > Guido said he has mooted this discussion > ... But before putting it on auto-archive, the BDFL said (1) NO GO on getting a new builtin; (2) NO OBJECTION to putting it in itertools. I don't recall him offering an opinion on a class in collections, did he? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Where should grouping() live (was: grouping / dict of lists)
On Wed, Jul 4, 2018 at 3:53 AM, INADA Naoki wrote: > But if it happens, I'm -1 on functools and collections. > They are used very much. Every Python tool import them regardless how > much of their contents are used. > really? collections? what for? I'm guessing namedtuple and maybe deque. But collections already has 9 classes (well, things) in it so we'd be adding a bit less than 10% more to it. what is the concern? import time, memory? In either case, it seems like the wrong driver for deciding where to put new things. > If you really want to add it in collections, I suggests from collections.groupdict import GroupDict. Perhaps the stdlib should have a deeper namespaces in general -- if that is established as a policy, then this could be the first thing to follow that policy. But I thought "flat is better than nested" -- sigh. So maybe we need to bite the bullet and solve the problem at another level: 1) if, say, namedtuple has gotten very popular, maybe it should move to builtins. 2) Whatever happened to the proposals to make it easier to lazy-load stuff in modules? If that gets implemented, then we can speed up startup in general, and not have to be too worried about adding "too much" to a module because one thing in it is common use. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Where should grouping() live
So this ended up a long post, so the TL;DR
* There are types of data well suited to the key function approach, and
other data not so well suited to it. If you want to support the not as well
suited use cases, you should have a value function as well and/or take a
(key, value) pair.
* There are some nice advantages in flexibility to having a Grouping class,
rather than simply a function.
So: I propose a best of all worlds version: a Grouping class (subclass of
dict):
* The constructor takes an iterable of (key, value) pairs by default.
* The constructor takes an optional key_func -- when not None, it is used
to determine the keys in the iterable instead.
* The constructor also takes a value_func -- when specified, it processes
the items to determine the values.
* a_grouping[key] = value
adds the value to the list corresponding to the key.
* a_grouping.add(item) -- applies the key_func and value_func to add a new
value to the appropriate group.
Prototype code here:
https://github.com/PythonCHB/grouper
Now the lengthy commentary and examples:
On Tue, Jul 3, 2018 at 5:21 PM, Steven D'Aprano wrote:
> On Wed, Jul 04, 2018 at 10:44:17AM +1200, Greg Ewing wrote:
> > Steven D'Aprano wrote:
>
> > Unless we *make* it a data type. Then not only would it fit
> > well in collections, it would also make it fairly easy to do
> > incremental grouping if you really wanted that.
indeed -- one of motivations for my prototype:
https://github.com/PythonCHB/grouper
(Did none of my messages get to this list??)
> > Usual case:
> >
> >g = groupdict((key(val), val) for val in things)
>
>
> How does groupdict differ from regular defaultdicts, aside from the
> slightly different constructor?
>
* You don't need to declare the defaultdict (and what the default is) first
* You don't need to call .append() yourself
* It can have a custom .init() and .update()
* It can have a .add() method
* It can (optionally) use a key function.
* And you can have other methods that do useful things with the groupings.
>g = groupdict()
> >for key(val), val in things:
> > g.add(key, val)
> > process_partial_grouping(g)
>
> I don't think that syntax works. I get:
>
> SyntaxError: can't assign to function call
>
looks like untested code :-)
with my prototype it would be:
g = groupdict()
for key, val in things:
g[key] = val
process_partial_grouping(g)
(this assumes your things are (key, value) pairs)
Again, IF you data are a sequence of items, and the value is the item
itself, and the key is a simple function of the item, THEN the key function
method makes more sense, which for the incremental adding of data would be:
g = groupdict(key_fun=a_fun)
for thing in things:
g.add(thing)
process_partial_grouping(g)
Even if it did work, it's hardly any simpler than
>
> d = defaultdict(list)
> for val in things:
> d[key(val)].append(val)
>
> But then Counter is hardly any simpler than a regular dict too.
>
exactly -- and counter is actually a little annoyingly too much like a
regular dict, in my mind :-)
In the latest version of my prototype, the __init__ expects a (key, value)
pair by default, but you can also pass in a key_func, and then it will
process the iterable passes in as (key_func(item), item) pairs.
And the update() method will also use the key_func if one was provided.
So a best of both worlds -- pick your API.
In this thread, and in the PEP, there various ways of accomplishing this
task presented -- none of them (except using a raw itertools.groupby in
some cases) is all that onerous.
But I do think a custom function or even better, custom class, would create
a "one obvious" way to do a common manipulation.
A final (repeated) point:
Some data are better suited to a (key, value) pair style, and some to a key
function style. All of the examples in the PEP are well suited to the key
function style. But the example that kicked off this discussion was about
data already in (key, value) pairs (actual in that case, (value, key) pairs.
And there are other examples. Here's a good one for how one might want to
use a Grouping dict more like a regular dict -- of maybe like a simple
function constructor:
(code in: https://github.com/PythonCHB/grouper/blob/master/examples/
trigrams.py)
#!/usr/bin/env python3
"""
Demo of processing "trigrams" from Dave Thomas' Coding Kata site:
http://codekata.com/kata/kata14-tom-swift-under-the-milkwood/
This is only addressing the part of the problem of building up the trigrams.
This is showing various ways of doing it with the Grouping object.
"""
from grouper import Grouping
from operator import itemgetter
words = "I wish I may I wish I might".split()
# using setdefault with a regular dict:
# how I might do it without a Grouping class
trigrams = {}
for i in range(len(words) - 2):
pair = tuple(words[i:i + 2])
follower = words[i + 2]
trigrams.setdefault(pair, []).append(follower)
print(trigrams)
# using a Grouping
Re: [Python-ideas] Where should grouping() live (was: grouping / dict of lists)
On Tue, Jul 3, 2018 at 12:01 PM, David Mertz wrote: > ... but I STILL like a new collections.Grouping (or collections.Grouper) > the best. > me too. > It might overcome Guido's reluctance... and what goes there is really > delegated by him, not his own baby. > Is collections anyone in particular's baby? like itertools "belongs" to Raymond? -CHB > On Tue, Jul 3, 2018 at 12:19 PM Chris Barker via Python-ideas < > python-ideas@python.org> wrote: > >> On Tue, Jul 3, 2018 at 8:24 AM, Steven D'Aprano >> wrote: >> >>> On Tue, Jul 03, 2018 at 09:23:07AM -0400, David Mertz wrote: >>> >> >> >>> > My problem with the second idea is that *I* find it very wrong to have >>> > something in itertools that does not return an iterator. It wrecks the >>> > combinatorial algebra of the module. >>> >> >> hmm -- that seems to be a pretty pedantic approach -- practicality beats >> purity, after all :-) >> >> I think we should first decide if a grouping() function is a useful >> addition to the standard library (after all: "not every two line function >> needs to in the stdlib"), and f so, then we can find a home for it. >> >> personally, I'm wondering if a "dicttools" or something module would make >> sense -- I imagine there are all sorts of other handy utilities for working >> with dicts that could go there. (though, yeah, we'd want to actually have a >> handful of these before creating a new module :-) ) >> >> > That said, it's easy to fix... and I believe independently useful. Just >>> > make grouping() a generator function rather than a plain function. >>> This >>> > lets us get an incremental grouping of an iterable. >>> >>> We already have something which lazily groups an iterable, returning >>> groups as they are seen: groupby. >>> >>> What makes grouping() different from groupby() is that it accumulates >>> ALL of the subgroups rather than just consecutive subgroupings. >> >> >> well, yeah, but it wont actually get you those until you exhaust the >> iterator -- so while it's different than itertools.groupby, it is different >> than itertools.groupby(sorted(iterable))? >> >> In short, this wouldn't really solve the problems that itertools.groupby >> has for this sort of task -- so what's the point? >> >> > As for where it belongs, perhaps the collections module is the least >> worst fit. >> >> That depends some on whether we go with a simple function, in which case >> collections is a pretty bad fit (but maybe still the least worse). >> >> Personally I still like the idea of having this be special type of dict, >> rather than "just a function" -- and then it's really obvious where to put >> it :-) >> >> -CHB >> >> >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR(206) 526-6959 voice >> 7600 Sand Point Way NE >> <https://maps.google.com/?q=7600+Sand+Point+Way+NE=gmail=g> >> (206) 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> chris.bar...@noaa.gov >> ___ >> Python-ideas mailing list >> Python-ideas@python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > Keeping medicines from the bloodstreams of the sick; food > from the bellies of the hungry; books from the hands of the > uneducated; technology from the underdeveloped; and putting > advocates of freedom in prisons. Intellectual property is > to the 21st century what the slave trade was to the 16th. > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Where should grouping() live (was: grouping / dict of lists)
It seems a really stupid reason to make this choice, but: If we make a Grouping class, it has an obvious home in the collections module If we make a grouping (or grouped) function, we don't know where to put it But since I like the Grouping class idea anyway, it's one more reason... -CHB On Tue, Jul 3, 2018 at 9:15 AM, Chris Barker wrote: > On Tue, Jul 3, 2018 at 8:24 AM, Steven D'Aprano > wrote: > >> On Tue, Jul 03, 2018 at 09:23:07AM -0400, David Mertz wrote: >> > > >> > My problem with the second idea is that *I* find it very wrong to have >> > something in itertools that does not return an iterator. It wrecks the >> > combinatorial algebra of the module. >> > > hmm -- that seems to be a pretty pedantic approach -- practicality beats > purity, after all :-) > > I think we should first decide if a grouping() function is a useful > addition to the standard library (after all: "not every two line function > needs to in the stdlib"), and f so, then we can find a home for it. > > personally, I'm wondering if a "dicttools" or something module would make > sense -- I imagine there are all sorts of other handy utilities for working > with dicts that could go there. (though, yeah, we'd want to actually have a > handful of these before creating a new module :-) ) > > > That said, it's easy to fix... and I believe independently useful. Just >> > make grouping() a generator function rather than a plain function. This >> > lets us get an incremental grouping of an iterable. >> >> We already have something which lazily groups an iterable, returning >> groups as they are seen: groupby. >> >> What makes grouping() different from groupby() is that it accumulates >> ALL of the subgroups rather than just consecutive subgroupings. > > > well, yeah, but it wont actually get you those until you exhaust the > iterator -- so while it's different than itertools.groupby, it is different > than itertools.groupby(sorted(iterable))? > > In short, this wouldn't really solve the problems that itertools.groupby > has for this sort of task -- so what's the point? > > > As for where it belongs, perhaps the collections module is the least > worst fit. > > That depends some on whether we go with a simple function, in which case > collections is a pretty bad fit (but maybe still the least worse). > > Personally I still like the idea of having this be special type of dict, > rather than "just a function" -- and then it's really obvious where to put > it :-) > > -CHB > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] grouping / dict of lists
On Tue, Jul 3, 2018 at 8:33 AM, Steven D'Aprano wrote:
> but why are we using key values by hand when grouping ought to do it for
>
us, as Michael Selik's version does?
>
> grouping(words, key=len)
because supplying a key function is sometimes cleaner, and sometimes uglier
than building up a comprehension -- which I think comes down to:
1) taste (style?)
2) whether the key function is as simple as the expression
3) whether you ned to transform the value in any way.
This argument is pretty much the same as whether you should use a
comprehension or map:
map(len, words)
vs
(len(word) for word in words)
In that case, map() looks cleaner and easier, but when you have something
less simple:
map(operator.attrgetter('something'), some_objects)
vs
(object.something for object in some_objects)
I like the comprehension better.
add a filter, and comps really get nicer -- after all they were added to
the language for a reason.
Then when you add the additional complication of needing to "transform" the
value as well, it's easy to do with the comprehension, but there is no way
to do it with only a key function.
I think the "confilct" here is that Micheal started with a bunch of
examples that area ll well suited to the key_function approach, and Nicolas
started with a use-case that is better suited to the comprehension /
(key,value) approach.
However, while the key, value approach can be reasonably (if a bit klunky)
used everywhere the key function approach can, the opposite is not true
(for when the value needs to be transformed as well.
But in the spirit of "Python has both map and comprehensions", I say let's
use both!
* The default behavior is to process a (key.value) pair.
* A key function can be provided in which case it is used, and the value is
the full item.
* A value function can be provided, in which case, it is used to "process"
the value.
If this is too confusing an interface, we could forget the value function,
and folks would have to use the (key, value) interface if they need to
transform the value.
What makes no sense to me is having the identify function as the default
key (and yes, it is the identity function, it would return the actual
object, or not be there at all) -- the grouping would be done by the hash
of key after passing through the key function).
That's because having a default that is (almost) completely useless makes
no sense -- it might as well be a required parameter.
(unless there was a value function as well, in which case, it's not a
completely useless default).
- CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Where should grouping() live (was: grouping / dict of lists)
On Tue, Jul 3, 2018 at 8:24 AM, Steven D'Aprano wrote: > On Tue, Jul 03, 2018 at 09:23:07AM -0400, David Mertz wrote: > > > My problem with the second idea is that *I* find it very wrong to have > > something in itertools that does not return an iterator. It wrecks the > > combinatorial algebra of the module. > hmm -- that seems to be a pretty pedantic approach -- practicality beats purity, after all :-) I think we should first decide if a grouping() function is a useful addition to the standard library (after all: "not every two line function needs to in the stdlib"), and f so, then we can find a home for it. personally, I'm wondering if a "dicttools" or something module would make sense -- I imagine there are all sorts of other handy utilities for working with dicts that could go there. (though, yeah, we'd want to actually have a handful of these before creating a new module :-) ) > That said, it's easy to fix... and I believe independently useful. Just > > make grouping() a generator function rather than a plain function. This > > lets us get an incremental grouping of an iterable. > > We already have something which lazily groups an iterable, returning > groups as they are seen: groupby. > > What makes grouping() different from groupby() is that it accumulates > ALL of the subgroups rather than just consecutive subgroupings. well, yeah, but it wont actually get you those until you exhaust the iterator -- so while it's different than itertools.groupby, it is different than itertools.groupby(sorted(iterable))? In short, this wouldn't really solve the problems that itertools.groupby has for this sort of task -- so what's the point? > As for where it belongs, perhaps the collections module is the least worst fit. That depends some on whether we go with a simple function, in which case collections is a pretty bad fit (but maybe still the least worse). Personally I still like the idea of having this be special type of dict, rather than "just a function" -- and then it's really obvious where to put it :-) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] grouping / dict of lists
On Mon, Jul 2, 2018 at 11:50 PM, Chris Barker wrote: > - keep the key function optional parameter. > - add a value function optional parameter. -- it really makes any case > where you don't want to store the whole item a lot easier. > > - Have the default key function be itemgetter(0) and the default value > function be itemgetter(1) (or some similar way to implement default support > for processing an iterable of (key, value) pairs. > > Having no value function and an equality default for the key function may > be "common", but it's also pretty much useless -- why have a useless > default? > > Thinking this through now I do see that having key and value default to to > the pair method means that if you specify key function, you will probably > have to specify a value function as well -- so maybe that's not ideal. > OK, I prototyped a class solution that defaults to key, value pairs, but you can specify a key and/or value function. and with convoluted logic, if you specify just a key, then the value defaults to the entire item. So folks will pretty much get what they expect. I think it's actually pretty slick -- best of both worlds? Code here: https://github.com/PythonCHB/grouper/blob/master/grouper/grouper.py -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] grouping / dict of lists
On Mon, Jul 2, 2018 at 9:39 AM, Michael Selik wrote:
> On Mon, Jul 2, 2018 at 2:32 AM Nicolas Rolin
>>> wrote:
>>>
For example the default could be such that grouping unpack tuples (key,
value) from the iterator and do what's expected with it (group value by
key). It is quite reasonable, and you have one example with (key, value) in
your example, and no example with the current default.
>>>
I agree, the default should do something that has some chance of being
useful on its own, and ideally, the most "common" use, if we can identify
that.
> On Mon, Jul 2, 2018 at 3:22 AM Nicolas Rolin
> wrote:
>
>> My question would be: does it have to be a key function? Can't we just
>> remove the "key" argument?
>>
>
> In the examples or from the parameters? A key function is necessary to
> support a wide variety of uses.
>
not if you have the expectation of an iterable of (key, value) pairs as the
input -- then any processing required to get a different key happens before
hand, allowing folks to use comprehension syntax.
as so: :-)
Because for pretty much all the given examples, I would find my default as
>> readable and nearly as short as the "key" syntax :
>>
>> > grouping(words, key=len)
>> grouping((len(word), word for word in words))
>>
>
> I think the fact that you misplaced a closing parenthesis demonstrates how
> the key-function pattern can be more clear.
>
I think it demonstrates that you shouldn't post untested code :-) -- the
missing paren is a syntax error -- it would be caught right away in real
life.
>
> The code is slightly more verbose, but it is akin to filter(iterable,
>> function) vs (i for i in iterable if function(i)).
>>
>
> Sometimes I prefer ``map`` and sometimes I prefer a list comprehension.
>
That is a "problem" with python: there are two ways to do things like map
and filter, and one way is not always the clearest choice.
But I wonder if map and filter would exist if they didn't pre-date
comprehensions. That aside, the comprehension approach is pretty well
liked and useful. And almost always prefer it -- an expression is simple on
the eyes to me :-)
But when it's really a win is when you don't have a handy built-in function
to do what you want, even though it's simple expression.
With the map, filter, key approach, you have to write a bunch of little
utility functions or lambdas, which can really clutter up the code.
If so, I like to write out the comprehension to provide that extra variable
> name for clarity.
>
> I'd write:
> map(len, words)
>
> But I'd also write
> [len(fullname) for fullname in contacts]
>
A key (pun intended) issue is that passing functions around looks so neat
and clean when it's a simple built in function like "len" -- but if not,
the it gets uglier, like:
map(lambda name: name.first_name, all_names)
vs
[name.first_name for nam in all names]
I really like the comprehension form much better when what you really want
is a simple expression like an attribute access or index or simple
calculation, or
I appreciate that defaulting the grouping key-function to ``itemgetter(0)``
> would enable a pleasant flexibility for people to make that same choice for
> each use. I haven't fully come around to that, yet, because so many other
> tools use the equality function as the default.
>
well, kinda, but maybe those aren't "pythonic" :-)
(and yes, itemgetter() is probably a better option than lambda in many
cases -- but I always forget that exists)
I started out in this topic answering a question about how to do a grouping
for a list of tuples, in that case the OP wanted a comprehension. I don't
think there's a good way to get a direct comprehension, but I do think we
can make a class of function that takes something you could build with a
comprehension.
And I took a look at itertools.groupby, and found it very, very awkward,
ultimately arriving at:
student_school_list = [('Fred', 'SchoolA'),
('Bob', 'SchoolB'),
('Mary', 'SchoolA'),
('Jane', 'SchoolB'),
('Nancy', 'SchoolC')]
grouped = {a:[t[0] for t in b] for a,b in groupby(sorted(student_school_list,
key=lambda t: t[1]), key=lambda t: t[1])}
{'SchoolA': ['Fred', 'Mary'], 'SchoolB': ['Bob', 'Jane'], 'SchoolC':
['Nancy']}
So why is that so painful?
("it" is itertools.groupby)
a) it returns an iterable of tuples, so to get a dict you need to do the
dict comp
b) it requires the data to be sorted -- so you ned to sort it first
c) I need to provide a key function to sort by
d) I need to provide (the same, always?) key function to group by
e) when I make the dict, I need to make the list, and use an expression to
get the value I want.
f) because I need those key functions, I need to use lambda for what could
be a simple expression
So the current proposal in the PEP makes that a lot better:
a) It makes a dict, so that step is done
b) It doesn't require the data to
Re: [Python-ideas] Fwd: grouping / dict of lists
On Sun, Jul 1, 2018 at 9:36 PM, Chris Barker wrote: > hmm, makes we wonder if it would make sense to update my implementation to > allow mapping types as well for the collection > general mapping types don't make sense -- but I added Counter. Which is a pretty special case, so I think it probably makes that case that it should just always be a list, and you can convert to others later. Though maybe list, set and Counter are the ones you'd want to use Again, real world use cases are needed! -CHB code here: https://github.com/PythonCHB/grouper/blob/master/grouper/grouper.py -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Fwd: grouping / dict of lists
On Sun, Jul 1, 2018 at 7:12 PM, David Mertz wrote:
> Michael changed from set to list at my urging. A list is more general. A
> groupby in Pandas or SQL does not enforce uniqueness, but DOES preserve
> order.
>
It really is better to construct the collection using lists—in the fully
general manner—and then only throw away the generality when that
appropriate.
well, yes -- if there were only one option, then list is pretty obvious.
but whether converting to sets after the fact is just as good or not -- I
don't think so.
It's only just as good if you think of it as a one-time operation --
process a bunch of data all at once, and get back a dict with the results.
But I'm thinking of it in a different way:
Create a custom class derived from dict that you can add stuff to at any
time --much more like the current examples in the collections module.
If you simply want a groupby function that returns a regular dict, then you
need a utility function (or a few), not a new class.
If you are making a class that enforces the the values to be a collection
of items, then list is the obvious default, but of someone wants a set --
they want it built in to the class, not converted after the fact.
I've extended my prototype to do just that:
class Grouping(dict):
...
def __init__(self, iterable=(), *, collection=list):
"collection" is a class that's either a Mutable Sequence (has .append and
.extend methods) or Set (has .add and .update methods).
Once you create a Grouping instance, the collection class you pass in is
used everywhere.
I've put the prototype up on gitHub if anyone wants to take a look, try it
out, suggest changes, etc:
https://github.com/PythonCHB/grouper
(and enclosed here)
Note that I am NOT proposing this particular implementation or names, or
anything. I welcome feedback on the implementation, API and naming scheme,
but it would be great if we could all be clear on whether the critique is
of the idea or of the implementation.
This particular implementation uses pretty hack meta-class magic (or the
type constructor anyway) -- if something set-like is passed in, it creates
a subclass that adds .append and .extend methods, so that the rest of the
code doesn't have to special case. Not sure if that's a good idea, it feels
pretty kludgy -- but kinda fun to write.
It also needs more test cases and example use cases for sure.
And before we go much farther with this discussion, it would be great to
see some more real-world use cases, if anyone has some in mind.
-CHB
---
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
from collections.abc import Mapping
import heapq
# extra methods to tack on to set to make it "act" like a list.
extra_methods = {"append": lambda self, value: self.add(value),
"extend": lambda self, sequence: self.update(set(sequence))}
class Grouping(dict):
"""
Dict subclass for grouping elements of a sequence.
The values in the dict are a list of all items that have
corresponded to a given key
essentially, adding an new item is the same as:
dict.setdefault(key, []).append(value)
In other words, for each item added:
grouper['key'] = 'value'
If they key is already there, the value is added to the corresponding list.
If the key is not already there, a new entry is added, with the key as the
key, and the value is a new list with the single entry of the value in it.
The __init__ (and update) can take either a mapping of keys to lists,
or an iterable of (key, value) tuples.
If the initial data is not in exactly the form desired, an generator
expression can be used to "transform" the input.
For example:
>>> Grouping(((c.casefold(), c) for c in 'AbBa'))
Grouping({'a': ['A', 'a'], 'b': ['b', 'B']})
"""
def __init__(self, iterable=(), *, collection=list):
"""
Create a new Grouping object.
:param iterable: an iterable or mapping with initial data.
"""
if hasattr(collection, "append") and hasattr(collection, "extend"):
self.collection = list
elif hasattr(collection, "add") and hasattr(collection, "update"):
# this is very kludgy -- adding append and extend methods to a
# set or set-like object
self.collection = type("appendset", (set,), extra_methods)
else:
raise TypeError("collection has to be a MutableSequence or set-like object")
super().__init__()
self.update(iterable)
# Override a few dict methods
def __setitem__(self, key, value):
self.setdefault(key, self.collection()).append(value)
def __repr__(self):
return f"Grouping({super().__repr__()})"
@classmethod
def fromkeys(cls, iterable,
Re: [Python-ideas] Fwd: grouping / dict of lists
On Sun, Jul 1, 2018 at 7:28 PM, David Mertz wrote:
> But it's pretty simple. Whether my idea of collections.Grouping is adapted
> or whether a function/classmethod grouping() produces a plain dictionary,
>
or my custom class...
> the casting would be the same:
>
> {k:set(v) for k,v in deps.items()}
>
> {k:Counter(v) for k,v in deps.items()}
>
hmm, makes we wonder if it would make sense to update my implementation to
allow mapping types as well for the collection --now we'd be getting really
magic
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Fwd: grouping / dict of lists
Ivan, Did you mean this to go to the list? I hope so, as I've cc-d it this time :-) On Sun, Jul 1, 2018 at 1:20 AM, Ivan Levkivskyi wrote: > On 1 July 2018 at 06:18, Chris Barker via Python-ideas < > python-ideas@python.org> wrote: > >> I'm really warming to the: >> >> Alternate: collections.Grouping >> >> version -- I really like this as a kind of custom mapping, rather than >> "just a function" (or alternate constructor) -- >> > I wanted the group to be represented as a set, not a list. I however > understand that list may be more common. Can we design an API that > would make this configurable? Something like: > > from collections import Grouping > > deps = Grouping(set) # list can be the default > deps.update(other_deps) # uses set.update or list.extend for every key > deps.add(trigger, target) # uses set.add or list.append > yeah, I thought about that too -- Michael was using set() in some of his examples. But the question is -- do we have a single switchable version or just too classes? > Probably allowing an arbitrary collection for values is to general/hard. > maybe not -- if we had the criteria that you pass in any collection you wanted, as long as it had either an .append() or .add()method, it would be pretty easy to do with duck typing magic. Sure -- a user could make a mess easily enough by passing in a weird custom class, but so what? Using something other than a set or list would be a "at your own risk" thing anyway. > Maybe we can just add a flag `unique=True` to the constructor, that will > cause using sets instead of lists for groups? > That's another, more robust, but less flexible option. Stay tuned for a prototype, if I can get it done fast -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] grouping / dict of lists
On Fri, Jun 29, 2018 at 10:53 AM, Michael Selik wrote:
> I've drafted a PEP for an easier way to construct groups of elements from
> a sequence. https://github.com/selik/peps/blob/master/pep-.rst
>
> I'm really warming to the:
Alternate: collections.Grouping
version -- I really like this as a kind of custom mapping, rather than
"just a function" (or alternate constructor) -- and I like your point that
it can have a bit of functionality built in other than on construction.
But I think it should be more like the other collection classes -- i.e. a
general purpose class that can be used for grouping, but also used more
general-purpose-y as well. That way people can do their "custom" stuff (key
function, etc.) with comprehensions.
The big differences are a custom __setitem__:
def __setitem__(self, key, value):
self.setdefault(key, []).append(value)
And the __init__ and update would take an iterable of (key, value) pairs,
rather than a single sequence.
This would get away from the itertools.groupby approach, which I find kinda
awkward:
* How often do you have your data in a single sequence?
* Do you need your keys (and values!) to be sortable???)
* Do we really want folks to have to be writing custom key functions and/or
lambdas for really simple stuff?
* and you may need to "transform" both your keys and values
I've enclosed an example implementation, borrowing heavily from Michael's
code.
The test code has a couple examples of use, but I'll put them here for the
sake of discussion.
Michael had:
Grouping('AbBa', key=c.casefold))
with my code, that would be:
Grouping(((c.casefold(), c) for c in 'AbBa'))
Note that the key function is applied outside the Grouping object, it
doesn't need to know anything about it -- and then users can use an
expression in a comprehension rather than a key function.
This looks a tad clumsier with my approach, but this is a pretty contrived
example -- in the more common case [*], you'd be writing a bunch of
lambdas, etc, and I'm not sure there is a way to get the values customized
as well, if you want that. (without applying a map later on)
Here is the example that the OP posted that kicked off this thread:
In [37]: student_school_list = [('Fred', 'SchoolA'),
...:('Bob', 'SchoolB'),
...:('Mary', 'SchoolA'),
...:('Jane', 'SchoolB'),
...:('Nancy', 'SchoolC'),
...:]
In [38]: Grouping(((item[1], item[0]) for item in student_school_list))
Out[38]: Grouping({'SchoolA': ['Fred', 'Mary'],
'SchoolB': ['Bob', 'Jane'],
'SchoolC': ['Nancy']})
or
In [40]: Grouping((reversed(item) for item in student_school_list))
Out[40]: Grouping({'SchoolA': ['Fred', 'Mary'], 'SchoolB': ['Bob', 'Jane'],
'SchoolC': ['Nancy']})
(note that if those keys and values were didn't have to be reversed, you
could just pass the list in raw.
I really like how I can use a generator expression and simple expressions
to transform the data in the way I need, rather than having to make key
functions.
And with Michael's approach, I think you'd need to call .map() after
generating the grouping -- a much klunkier way to do it. (and you'd get
plain dict rather than a Grouping that you could add stuff too later...)
I'm sure there are ways to improve my code, and maybe Grouping isn't the
best name, but I think something like this would be a nice addition to the
collections module.
-CHB
[*] -- before making any decisions about the best API, it would probably be
a good idea to collect examples of the kind of data that people really do
need to group like this. Does it come in (key, value) pairs naturally? or
in one big sequence with a key function that's easy to write? who knows
without examples of real world use cases.
I will show one "real world" example here:
In my Python classes, I like to use Dave Thomas' trigrams: "code kata":
http://codekata.com/kata/kata14-tom-swift-under-the-milkwood/
A key piece of this is building up a data structure with word pairs, and a
list of all the words that follow the pair in a piece of text.
This is a nice exercise to help people think about how to use dicts, etc.
Currently the most clean code uses .setdefault:
word_pairs = {}
# loop through the words
# (rare case where using the index to loop is easiest)
for i in range(len(words) - 2): # minus 2, 'cause you need a pair
pair = tuple(words[i:i + 2]) # a tuple so it can be a key in the
dict
follower = words[i + 2]
word_pairs.setdefault(pair, []).append(follower)
if this were done with my Grouping class, it would be:
In [53]: word_pairs = Grouping()
In [54]: for i in range(len(words) - 2):
...: pair = tuple(words[i:i + 2]) # a tuple so it can be a key in
the dict
...: follower = words[i + 2]
...: word_pairs[pair] = follower
...:
In [55]: word_pairs
Re: [Python-ideas] Allow a group by operation for dict comprehension
> On Jun 28, 2018, at 5:30 PM, Chris Barker - NOAA Federal > wrote: > > So maybe a solution is an accumulator special case of defaultdict — it uses a > list be default and appends by default. > > Almost like counter... Which, of course, is pretty much what your proposal is. Which makes me think — a new classmethod on the builtin dict is a pretty heavy lift compared to a new type of dict in the collections module. -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow a group by operation for dict comprehension
> I think you accidentally swapped variables there: > student_school_list > vs student_by_school Oops, yeah. That’s what I get for whipping out a message before catching a bus. (And on a phone now) But maybe you could wrap the defaultdict constructor around a generator expression that transforms the list first. That would get the keys right. Though still not call append for you. So maybe a solution is an accumulator special case of defaultdict — it uses a list be default and appends by default. Almost like counter... -CHB ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow a group by operation for dict comprehension
Hold the phone!
On Thu, Jun 28, 2018 at 8:25 AM, Nicolas Rolin
wrote:
> student_by_school = defaultdict(list)
> for student, school in student_school_list:
> student_by_school[school].append(student)
>
> What I would expect would be a syntax with comprehension allowing me to
> write something along the lines of:
>
> student_by_school = {group_by(school): student for school, student in
> student_school_list}
>
OK -- I agreed that this could/should be easier, and pretty much like using
setdefault, but did like the single expression thing, so went to "there
should be a way to make a defaultdict comprehension" -- and played with
itertools.groupby (which is really really awkward for this), but then light
dawned on Marblehead:
I've noticed (and taught) that dict comprehensions are kinda redundant with
the dict() constructor, and _think_, in fact, that they were added before
the current dict() constructor was added.
so, if you think "dict constructor" rather than dict comprehensions, you
realize that defaultdict takes the same arguments as the dict(), so the
above is:
defaultdict(list, student_by_school)
which really couldn't be any cleaner and neater.
Here it is in action:
In [97]: student_school_list
Out[97]:
[('Fred', 'SchoolA'),
('Bob', 'SchoolB'),
('Mary', 'SchoolA'),
('Jane', 'SchoolB'),
('Nancy', 'SchoolC')]
In [98]: result = defaultdict(list, student_by_school)
In [99]: result.items()
Out[99]: dict_items([('SchoolA', ['Fred', 'Mary']), ('SchoolB', ['Bob',
'Jane']), ('SchoolC', ['Nancy'])])
So: never mind
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow a group by operation for dict comprehension
On Thu, Jun 28, 2018 at 4:59 PM, Steven D'Aprano
wrote:
> Can I make a plea for people to not post code with source highlighting
> as HTML please? It is rendered like this for some of us:
>
> On Thu, Jun 28, 2018 at 10:01:00AM -0700, Chris Barker via Python-ideas
> wrote:
>
> In [*46*]: {a:[t[0] *for* t *in* b] *for* a,b *in*
> groupby(sorted(student_school_list,
> key=*lambda* t: t[1]), key=*lambda* t: t[
>
Oh god -- yeach!! -- sorry about that -- that was copy an pasted from
iPython -- I was assuming it would strip out the formatting and give
reasonable plain text -- but apparently not.
I'll stop that.
-CHB
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow a group by operation for dict comprehension
On Thu, Jun 28, 2018 at 4:23 PM, Greg Ewing
wrote:
> Nicolas Rolin wrote:
>
>> student_by_school = {group_by(school): student for school, student in
>> student_school_list}
>>
>
> In the spirit of making the target expression look like
> a template for the generated elements,
>
>{school: [student...] for school, student in student_school_list}
hmm -- this seems a bit non-general -- would this only work for a list?
maybe you would want a set, or???
so could be get a defaultdict comprehension with something like:
{ school: (default_factory=list, student) for school, student in
student_school_list }
But I can't think of an reasonable syntax to make that work.
-CHB
>
> --
> Greg
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow a group by operation for dict comprehension
On Thu, Jun 28, 2018 at 1:34 PM, David Mertz wrote:
> I'd add one more option. You want something that behaves like SQL. Right
> in the standard library is sqlite3, and you can create an in-memory DB to
> hope the data you expect to group.
>
There are also packages designed to make DB-style queries easier.
Here's one I found with a quick google.
-CHB
> On Thu, Jun 28, 2018, 3:48 PM Wes Turner wrote:
>
>> PyToolz, Pandas, Dask .groupby()
>>
>> toolz.itertoolz.groupby does this succinctly without any
>> new/magical/surprising syntax.
>>
>> https://toolz.readthedocs.io/en/latest/api.html#toolz.itertoolz.groupby
>>
>> From https://github.com/pytoolz/toolz/blob/master/toolz/itertoolz.py :
>>
>> """
>> def groupby(key, seq):
>> """ Group a collection by a key function
>> >>> names = ['Alice', 'Bob', 'Charlie', 'Dan', 'Edith', 'Frank']
>> >>> groupby(len, names) # doctest: +SKIP
>> {3: ['Bob', 'Dan'], 5: ['Alice', 'Edith', 'Frank'], 7: ['Charlie']}
>> >>> iseven = lambda x: x % 2 == 0
>> >>> groupby(iseven, [1, 2, 3, 4, 5, 6, 7, 8]) # doctest: +SKIP
>> {False: [1, 3, 5, 7], True: [2, 4, 6, 8]}
>> Non-callable keys imply grouping on a member.
>> >>> groupby('gender', [{'name': 'Alice', 'gender': 'F'},
>> ...{'name': 'Bob', 'gender': 'M'},
>> ...{'name': 'Charlie', 'gender': 'M'}]) #
>> doctest:+SKIP
>> {'F': [{'gender': 'F', 'name': 'Alice'}],
>> 'M': [{'gender': 'M', 'name': 'Bob'},
>>{'gender': 'M', 'name': 'Charlie'}]}
>> See Also:
>> countby
>> """
>> if not callable(key):
>> key = getter(key)
>> d = collections.defaultdict(lambda: [].append)
>> for item in seq:
>> d[key(item)](item)
>> rv = {}
>> for k, v in iteritems(d):
>> rv[k] = v.__self__
>> return rv
>> """
>>
>> If you're willing to install Pandas (and NumPy, and ...), there's
>> pandas.DataFrame.groupby:
>>
>> https://pandas.pydata.org/pandas-docs/stable/generated/
>> pandas.DataFrame.groupby.html
>>
>> https://github.com/pandas-dev/pandas/blob/v0.23.1/pandas/
>> core/generic.py#L6586-L6659
>>
>>
>> Dask has a different groupby implementation:
>> https://gist.github.com/darribas/41940dfe7bf4f987eeaa#
>> file-pandas_dask_test-ipynb
>>
>> https://dask.pydata.org/en/latest/dataframe-api.html#
>> dask.dataframe.DataFrame.groupby
>>
>>
>> On Thursday, June 28, 2018, Chris Barker via Python-ideas <
>> python-ideas@python.org> wrote:
>>
>>> On Thu, Jun 28, 2018 at 8:25 AM, Nicolas Rolin
>>> wrote:
>>>>
>>>> I use list and dict comprehension a lot, and a problem I often have is
>>>> to do the equivalent of a group_by operation (to use sql terminology).
>>>>
>>>
>>> I don't know from SQL, so "group by" doesn't mean anything to me, but
>>> this:
>>>
>>>
>>>> For example if I have a list of tuples (student, school) and I want to
>>>> have the list of students by school the only option I'm left with is to
>>>> write
>>>>
>>>> student_by_school = defaultdict(list)
>>>> for student, school in student_school_list:
>>>> student_by_school[school].append(student)
>>>>
>>>
>>> seems to me that the issue here is that there is not way to have a
>>> "defaultdict comprehension"
>>>
>>> I can't think of syntactically clean way to make that possible, though.
>>>
>>> Could itertools.groupby help here? It seems to work, but boy! it's ugly:
>>>
>>> In [*45*]: student_school_list
>>>
>>> Out[*45*]:
>>>
>>> [('Fred', 'SchoolA'),
>>>
>>> ('Bob', 'SchoolB'),
>>>
>>> ('Mary', 'SchoolA'),
>>>
>>> ('Jane', 'SchoolB'),
>>>
>>> ('Nancy', 'SchoolC')]
>>>
>>>
>>> In [*46*]: {a:[t[0] *for* t *in* b] *for* a,b *in* groupby(sorted
>>> (student_school_list, key=*lambda* t: t[1]), key=*lambda* t: t[
>>>
>>> ...: 1])}
>>>
>>> ...:
>>>
>>> ...:
>>>
>>> ...:
>>>
>>> ...:
>>>
>>> ...:
>>>
>>> ...:
>>
Re: [Python-ideas] Allow a group by operation for dict comprehension
On Thu, Jun 28, 2018 at 3:17 PM, Chris Barker wrote: > There are also packages designed to make DB-style queries easier. > > Here's one I found with a quick google. > opps -- hit send too soon: http://178.62.194.22/ https://github.com/pythonql/pythonql -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow a group by operation for dict comprehension
On Thu, Jun 28, 2018 at 8:25 AM, Nicolas Rolin
wrote:
>
> I use list and dict comprehension a lot, and a problem I often have is to
> do the equivalent of a group_by operation (to use sql terminology).
>
I don't know from SQL, so "group by" doesn't mean anything to me, but this:
> For example if I have a list of tuples (student, school) and I want to
> have the list of students by school the only option I'm left with is to
> write
>
> student_by_school = defaultdict(list)
> for student, school in student_school_list:
> student_by_school[school].append(student)
>
seems to me that the issue here is that there is not way to have a
"defaultdict comprehension"
I can't think of syntactically clean way to make that possible, though.
Could itertools.groupby help here? It seems to work, but boy! it's ugly:
In [*45*]: student_school_list
Out[*45*]:
[('Fred', 'SchoolA'),
('Bob', 'SchoolB'),
('Mary', 'SchoolA'),
('Jane', 'SchoolB'),
('Nancy', 'SchoolC')]
In [*46*]: {a:[t[0] *for* t *in* b] *for* a,b *in*
groupby(sorted(student_school_list,
key=*lambda* t: t[1]), key=*lambda* t: t[
...: 1])}
...:
...:
...:
...:
...:
...:
...:
Out[*46*]: {'SchoolA': ['Fred', 'Mary'], 'SchoolB': ['Bob', 'Jane'],
'SchoolC': ['Nancy']}
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] list configuration
Hey all, I've been replying to messages lately, and getting a bounce back: """ Hello chris.bar...@noaa.gov, We're writing to let you know that the group you tried to contact (python-ideas) may not exist, or you may not have permission to post messages to the group. A few more details on why you weren't able to post: """ And it's not quite clar to me if the message actually got through. IIUC, this is a Mailman list -- so it must be getting mirrored through google groups, and at least with some people's posts, the reply-to header is getting messed up. Anyone know what's going on? It would be nice to fix this... -CHB PS: I've seen a couple other notes about this -- I'm not the only one. -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
