Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Thu, Mar 21, 2019 at 09:36:20PM -0400, Terry Reedy wrote:
> I counted what I believe to be 10 instances of copy-update in the top
> level of /lib. Do either of you consider this to be enough that any
> addition would be worthwhile.
I think you're referring to Guido and Antoine? But for what it's worth,
I think that's a good indication that there are uses for a merge
operator.
> There are 3 in idlelib that I plan to replace with {**a, **b} and be
> done with the issue. I did not check any other packages.
If a+b already worked for dicts, would you still prefer {**a, **b}?
How about if it were spelled a|b?
--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On 2019-03-22 02:40, Dan Sommers wrote: On 3/21/19 9:19 PM, Christopher Barker wrote: https://docs.python.org/3.8/library/collections.html has some examples using collections.Counter, which is clearly described as being a subclass of dict. Amongst the examples: c + d # add two counters together: c[x] + d[x] That's the + operator operating on two dicts (don't make me quote the Liskov Substitution Principle), but doing something really different than the base operator. So if I know that c and d (or worse, that one of them) is a dict, then interpreting c + d becomes much more interesting, Killing a use of a common operator with a very common built in data type because the operator is used in a different way by a specialized object in the stdlib seems a bit backwards to me. Perhaps. Note that Counter also uses | and & for other operations that probably wouldn't make much sense on base dicts. Frankly, I think considering Counter as a dict subclass is the mistake here, even if it is true. I had the same thought that Counter is misdesigned in one way or another, but (a) that ship has long sailed, and (b) I didn't want to run off on that tangent. [snip] Counter is trying to provide the functionality of 2 kinds of container: 1. A counting container. 2. A multi-set. + makes sense for counting (sum); | makes sense for multi-sets (union). ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019, 10:15 PM Steven D'Aprano wrote:
> What would be the most useful behaviour for dict "addition" in your
> opinion?
>
Probably what I would use most often was a "lossless" merging in which
duplicate keys resulted in the corresponding value becoming a set
containing all the merged values. E.g.
>>> d1 = {1: 55, 2: 77, 3: 88}
>>> d2 = {3: 99, 4: 22}
>>> add(d1, d2)
{1: 55, 2: 77, 3: {88, 99}, 4:22}
I'm sure most users would hate this too. It changes the type of values
between a thing and a set of things, and that has to be sorted out
downstream. But it is lossless in a similar way to Counter or sequence
addition.
I can write what I want perfectly well. Perhaps useing defaultdict as a
shortcut to get there. And I know there are some behaviors I have not
specified here, but my function can do whatever I want in the edge cases.
If we're to see 'd1 + d2' for the first time without having followed this
discussion, my guess would be behavior similar to what I show.
> Of course I could learn it and teach it, but it will always feel
> > like a wart in the language.
>
> Would that wartness be lessoned if it were spelled | or << instead?
>
Yes, definitely. Both those spellings feel pretty natural to me. They don't
have the misleading associations '+' carries. I'm kinda fond of '<<'
because it visitation resembles an arrow that I can think of as "put the
stuff here into there".
> In contrast, once you tell me about the special object "vectorised
> arrays",
> > `arr1 + arr2` does exactly what is expect in NumPy.
>
> I don't know Numpy well enough to know whether that is elementwise
> addition or concatenation or something else, so that example doesn't
> resonate with me. I can't guess what you expect, and I have no confidence
> that my guess (matrix addition of equal-sized arrays, an exception if
> unequal) will be what Numpy does
>
Fair enough. I've worked with NumPy long enough that perhaps I forget what
my first intuition was. I accept that it's non-obvious to many users.
FWIW, I really love NumPy behavior, but it's a shift in thinking vs lists.
E.g.
>>> a = array([1, 2, 3])
>>> b = array([[10, 11, 12], [100, 200, 300]])
>>> a + b
[[ 11 13 15 ]
[ 101 202 303]]
This is "broadcasting" of compatible shapes.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On 3/21/19 9:19 PM, Christopher Barker wrote: https://docs.python.org/3.8/library/collections.html has some examples using collections.Counter, which is clearly described as being a subclass of dict. Amongst the examples: c + d # add two counters together: c[x] + d[x] That's the + operator operating on two dicts (don't make me quote the Liskov Substitution Principle), but doing something really different than the base operator. So if I know that c and d (or worse, that one of them) is a dict, then interpreting c + d becomes much more interesting, Killing a use of a common operator with a very common built in data type because the operator is used in a different way by a specialized object in the stdlib seems a bit backwards to me. Perhaps. Note that Counter also uses | and & for other operations that probably wouldn't make much sense on base dicts. Frankly, I think considering Counter as a dict subclass is the mistake here, even if it is true. I had the same thought that Counter is misdesigned in one way or another, but (a) that ship has long sailed, and (b) I didn't want to run off on that tangent. My point remains: because Counter is a subclass of dict, and Counter uses the + operator for something that doesn't apply to base dicts, adding + to dicts *may* cause confusion that wasn't there before. Presently, +, -, |, and & all raise an exception when given a Counter and a dict. This (raising an exception) is probably still the Right Thing to do in that case, even with a + operator on dicts, but that violates the LSP and IMO the PLS. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
> > https://docs.python.org/3.8/library/collections.html has some > examples using collections.Counter, which is clearly described > as being a subclass of dict. Amongst the examples: > > c + d # add two counters together: c[x] + d[x] > > That's the + operator operating on two dicts (don't make me > quote the Liskov Substitution Principle), but doing something > really different than the base operator. > > So if I know that c and d (or worse, that one of them) is a > dict, then interpreting c + d becomes much more interesting, Killing a use of a common operator with a very common built in data type because the operator is used in a different way by a specialized object in the stdlib seems a bit backwards to me. Frankly, I think considering Counter as a dict subclass is the mistake here, even if it is true. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019 at 08:13:01PM -0400, David Mertz wrote: > On Thu, Mar 21, 2019, 7:48 PM Steven D'Aprano wrote: [...] > Nonetheless, if I see `dict1 + dict2` the meaning you intend in the PEP > does not jump out as the obvious behavior. Nor even as the most useful > behavior. What would be the most useful behaviour for dict "addition" in your opinion? > Of course I could learn it and teach it, but it will always feel > like a wart in the language. Would that wartness be lessoned if it were spelled | or << instead? > In contrast, once you tell me about the special object "vectorised arrays", > `arr1 + arr2` does exactly what is expect in NumPy. I don't know Numpy well enough to know whether that is elementwise addition or concatenation or something else, so that example doesn't resonate with me. I can't guess what you expect, and I have no confidence that my guess (matrix addition of equal-sized arrays, an exception if unequal) will be what Numpy does. > > And your subjective feeling is well-noted :-) > > This is more than "merely subjective." If it is more than subjective, then there must be an objective test that anyone, or a computer program, could do to tell whether or not the + operator on dicts will be ... um, what? A wart? Ugly? Both of those are subjective value judgements, so I'm not sure what objective claim you believe you are making which is "more than" subjective. The point is, I'm not *discounting* the subjective claims that + on dicts is ugly. I've acknowledged them, and the next draft of the PEP will do so too. But repetition doesn't make a subjective value judgement objective. It might boil down to a subjective preference for + over | or visa versa, or another operator, or no operator at all. That's fine: language design is partly subjective. But I'd like to see more comments based on objective reasons we can agree on, and fewer arguments that boil down to "I just don't like it". -- Steven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Fri, Mar 22, 2019 at 1:21 AM Steven D'Aprano wrote: > > > How about dict.merged(*args, **kw)? Or dict.updated()? > +1 on "merged". I feel the word "update" indicating mutating, and it's difficult to distinguish between "update" and "updated". > That would eliminate some of the difficulties with an operator, such as > the difference between + which requires both operands to be a dict > but += which can take any mapping or (key,value) iterable. > > -- > Steven -- Inada Naoki ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On 3/21/2019 12:11 PM, Guido van Rossum wrote:
On Thu, Mar 21, 2019 at 7:45 AM Antoine Pitrou
One should also be able to write `d = dict.merge(d1, d2, ...)`
If dict merging is important enough to get a new spelling, then I think
this proposal is the best: explicit, unambiguous, immediately
understandable and easy to remember.
I don't find it easy to understand or remember that d1.update(d2)
modifies d1 in place, while d1.merge(d2) first copies d1.
Maybe the name can indicate the copying stronger? Like we did with
sorting: l.sort() sorts in-place, while sorted(l) returns a sorted copy.
I counted what I believe to be 10 instances of copy-update in the top
level of /lib. Do either of you consider this to be enough that any
addition would be worthwhile.
There are 3 in idlelib that I plan to replace with {**a, **b} and be
done with the issue. I did not check any other packages.
--
Terry Jan Reedy
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On 3/21/19 6:46 PM, Steven D'Aprano wrote: Antoine and Serhiy seem to worry that there are existing uses of + which are currently easy to understand but will become less so if dict.__add__ is added. I respect that worry, even if I doubt that they are correct. If someone can demonstrate that their fear is well-founded, that would be an excellent counter-argument to the PEP's proposal to use +. https://docs.python.org/3.8/library/collections.html has some examples using collections.Counter, which is clearly described as being a subclass of dict. Amongst the examples: c + d # add two counters together: c[x] + d[x] That's the + operator operating on two dicts (don't make me quote the Liskov Substitution Principle), but doing something really different than the base operator. So if I know that c and d (or worse, that one of them) is a dict, then interpreting c + d becomes much more interesting, but arguably no worse than c.update(d). Yes, it's "just" polymorphism, but IMO it violates the Principle of Least Surprise. My apologies if this is covered elsewhere in this thread, or it doesn't meet the bar Steven set. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019, 7:48 PM Steven D'Aprano wrote: > A number of people including Antoine and Serhiy seem to have taken the > position that merely adding dict.__add__ will make existing code using + > harder to understand, as you will need to consider not just numeric > addition and concatenation, but also merging, when reading code. > > Would you agree that this example of + is perfectly clear today? > > for digit in digits: > num = num*10 + digit > > By both the naming (digit, num) and presence of multiplication by the > literal 10, it should be pretty obvious that this is probably doing > integer addition. > Yep. This is clear and will not become less clear if some more objects grow an .__add__() methods. Already, it is POSSIBLE that `num` and `digit` mean something other than numbers. Bad naming of variables if so, but not prohibited. For example, NumPy uses '+' and '*' for elementwise operations, often with broadcasting to different areas shapes. Maybe that's code dealing with vectorised arrays... But probably not. Holoviews users '+' and '*' to combine elements of graphs. E.g labelled = low_freq * high_freq * linpoints overlay + labelled + labelled.Sinusoid.Low_Frequency ggplot in R has similar behavior. Maybe your loop is composing a complex graph... But probably not. Nonetheless, if I see `dict1 + dict2` the meaning you intend in the PEP does not jump out as the obvious behavior. Nor even as the most useful behavior. Of course I could learn it and teach it, but it will always feel like a wart in the language. In contrast, once you tell me about the special object "vectorised arrays", `arr1 + arr2` does exactly what is expect in NumPy. And your subjective feeling is well-noted :-) > This is more than "merely subjective." I teach Python. I write books about Python. I've had tens of millions of readers of articles I've written about Python. I'm not the only person in this discussion with knowledge of learners and programmers and scientists... But the opinions I'm expressing ARE on their behalf too (as I perceive likely surprise and likely bugs). I like most of the design of Python. Almost all, even. But there are a few warts in it. This would be a wart. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019 at 03:10:48PM -0700, Brandt Bucher wrote: > For anyone interested in "trying it out": if you're not against cloning and > compiling CPython yourself, here is a PEP 584 C implementation I have PR'd > against master right now. I'm keeping it in sync with the draft PEP as it > changes, so subtraction performance is not overly optimized yet, but it > will show you the *exact* behavior outlined in the PEP on the dict builtin > and its subclasses. The relevant branch is called "addiction". You can > clone it from: That's great, thank you! For the sake of comparisons, could you support | as an alias? That will allow people to get a feel for whether a+b or a|b looks nicer. (For the record, the PEP isn't set in stone in regards to the choice of operator. > https://github.com/brandtbucher/cpython.git -- Steven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019 at 06:02:05PM -0400, David Mertz wrote:
> I dislike the symbol '+' to mean "dictionary merging with value updates." I
> have no objection to, and mildly support, adding '|' with this meaning.
>
> It's not really possible to give "that one example" where + for meeting
> makes code less clear... In my eyes it would be EVERY such use.
I suspect that I may not have explained myself properly. Sorry. Let me
try to explain again.
A number of people including Antoine and Serhiy seem to have taken the
position that merely adding dict.__add__ will make existing code using +
harder to understand, as you will need to consider not just numeric
addition and concatenation, but also merging, when reading code.
*If this were true* it would be an excellent argument against using +
for dict merges. But is it true?
Would you agree that this example of + is perfectly clear today?
for digit in digits:
num = num*10 + digit
By both the naming (digit, num) and presence of multiplication by the
literal 10, it should be pretty obvious that this is probably doing
integer addition.
(I suppose it is conceivable that this is doing sequence repetition and
concatenation, but given the names that interpretation would be rather
unexpected.)
We shouldn't find it hard to understand that code, using nothing more
than *local* context. There's no need to search the global context to
find out what num and digits are.
(Although in the specific example I copied that snippet from, that
information is only two or three lines away. But in principle, we might
have needed to search an arbitrarily large code base to determine what
they were.)
Adding dict.__add__ isn't going to make that example harder to
understand. If it did, that would be a big blow to the + proposal.
Antoine and Serhiy seem to worry that there are existing uses of + which
are currently easy to understand but will become less so if dict.__add__
is added. I respect that worry, even if I doubt that they are correct.
If someone can demonstrate that their fear is well-founded, that would
be an excellent counter-argument to the PEP's proposal to use +.
What *doesn't* count as a demonstration:
1. Toy examples using generic names don't count. With generic,
meaningless names, they're not meaningful now and so adding dict.__add__
won't make them *less* meaningful:
# is this concatenation or numeric addition? who can tell?
for spam in spammy_macspamface:
eggs += spam
Regardless of whether dicts support + or not, we would still have to
search the global context to work out what eggs and spam are.
Adding dict.__add__ doesn't make this harder.
2. Purely opinion-based subjective statements, since they basically boil
down to "I don't like the use of + for dict merging." That point has
been made, no need to keep beating that drum.
3. Arguments based on unfamiliarity to the new operator:
preferences += {'EDITOR': 'ed', 'PAGESIZE': 'A4'}
might give you a bit of a double-take the first time you see it, but it
surely won't still be surprising you in five years time.
I realise that this is a high bar to reach, but if somebody does reach
it, and demonstrates that Antoine and Serhiy's fears are well-founded,
that would be a very effective and convincing argument.
> Every
> example presented in this thread or in the PEP feels wrong to me. I know
> about operator overloading and dunder methods and custom classes. My
> intuition about '+' from math, other programming languages, and Python,
> simply does not lead me to expect the proposed meaning.
And your subjective feeling is well-noted :-)
--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Fri, Mar 22, 2019 at 3:42 AM Steven D'Aprano wrote:
> And to those who support this PEP, code examples where a dict merge
> operator will help are most welcome!
Since Python examples don't really exist yet, I'm reaching for another
language that DOES have this feature. Pike's mappings (broadly
equivalent to Python's dicts) can be added (actually, both + and | are
supported), with semantics equivalent to PEP 584's. Translated into
Python syntax, here's a section from the implementation of
Process.run():
def run(cmd, modifiers={}):
...
...
p = Process(cmd, modifiers + {
"stdout": mystdout->pipe(),
"stderr": mystderr->pipe(),
"stdin": mystdin->pipe(),
})
In Val.TimeTZ, a subclass that adds a timezone attribute overrides a
mapping-returning method to incorporate the timezone in the result
mapping. Again, translated into Python syntax:
def tm(self):
return super().tm() + {"timezone": self.timezone}
To spawn a subprocess with a changed environment variable:
//from the Process.create_process example
Process.create_process(({ "/usr/bin/env" }),
(["env" : getenv() + (["TERM":"vt100"]) ]));
# equivalent Python code
subprocess.Popen("/usr/bin/env",
env=os.environ + {"TERM": "vt100"})
All of these examples could be done with the double-star syntax, as
they all use simple literals. But addition looks a lot cleaner IMO,
and even more so if you're combining multiple variables rather than
using literals.
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
I dislike the symbol '+' to mean "dictionary merging with value updates." I have no objection to, and mildly support, adding '|' with this meaning. It's not really possible to give "that one example" where + for meeting makes code less clear... In my eyes it would be EVERY such use. Every example presented in this thread or in the PEP feels wrong to me. I know about operator overloading and dunder methods and custom classes. My intuition about '+' from math, other programming languages, and Python, simply does not lead me to expect the proposed meaning. On Thu, Mar 21, 2019, 12:43 PM Steven D'Aprano wrote: > I'd like to make a plea to people: > > I get it, there is now significant opposition to using the + symbol for > this proposed operator. At the time I wrote the first draft of the PEP, > there was virtually no opposition to it, and the | operator had very > little support. This has clearly changed. > > At this point I don't think it is productive to keep making subjective > claims that + will be more confusing or surprising. You've made your > point that you don't like it, and the next draft^1 of the PEP will make > that clear. > > But if you have *concrete examples* of code that currently is easy to > understand, but will be harder to understand if we add dict.__add__, > then please do show me! > > For those who oppose the + operator, it will help me if you made it > clear whether it is *just* the + symbol you dislike, and would accept > the | operator instead, or whether you hate the whole operator concept > regardless of how it is spelled. > > And to those who support this PEP, code examples where a dict merge > operator will help are most welcome! > > > > > ^1 Coming Real Soon Now™. > > > -- > Steven > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Fri, Mar 22, 2019 at 8:44 AM Jonathan Fine wrote: > > Antoine Pitrou wrote: > > Note that, if you're able to live with a third-party dependency, the > > `toolz` package has what you need (and lots of other things too): > > https://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoolz.merge > > I suggest that the supporters of dict + dict make (and put up on PyPi) > a pure-Python subclass of dict that has the desired properties. This > would > > 1. Clarify and document the syntax and semantics. > 2. Help with exploration and testing. > 3. Provide a 'back-port' mechanism to current Python. > 4. Give the proposal the benefit of practical experience. > The trouble with that is that you can't always use a dict subclass (or a non-subclass MutableMapping implementation, etc, etc, etc). There are MANY situations in which Python will give you an actual real dict, and it defeats the purpose if you then have to construct an AddableDict out of it just so you can add something to it. Not every proposed change makes sense on PyPI, and it definitely won't get a fair representation in "practical experience". If someone's proposing adding a new module to the standard library, then by all means, propose PyPI. But changes to core types can't be imported from other modules. Python is not Ruby. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
Antoine Pitrou wrote: > Note that, if you're able to live with a third-party dependency, the > `toolz` package has what you need (and lots of other things too): > https://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoolz.merge I suggest that the supporters of dict + dict make (and put up on PyPi) a pure-Python subclass of dict that has the desired properties. This would 1. Clarify and document the syntax and semantics. 2. Help with exploration and testing. 3. Provide a 'back-port' mechanism to current Python. 4. Give the proposal the benefit of practical experience. I find this last very important, when we can do it. And we can, in this case. Language changes are 'cast in stone' and hard to reverse. And afterwards, on this list, we're sometime told that we've 'missed the boat' for a particular change. Let's take the benefit of a reference pure Python implementation, when we can. Steven D'A. Please would you include or respond to this suggestion, in the next revision of the PEP. -- Jonathan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Report an issue of Python
Hello, On Thu, 21 Mar 2019 15:31:30 -0400 Hardik Patel wrote: > Hello, > Can you please help me to contact a core team if it is possible. > I would like to report an issue. Issues can be reported at https://bugs.python.org/ Regards Antoine. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Report an issue of Python
Hello, Can you please help me to contact a core team if it is possible. I would like to report an issue. Thank you, Hardik Patel ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, 21 Mar 2019 17:59:41 + Rhodri James wrote: > > >> And to those who support this PEP, code examples where a dict merge > >> operator will help are most welcome! > > I don't use Python often enough to have much to offer, I'm afraid. The > sort of occasion I would use dict merging is passing modified > environments to subcommands. Something like: > > def process(): > if time_to_do_thing1(): > thing1(base_env + thing1_env_stuff + env_tweaks) > if time_to_do_thing2(): > thing2(base_env + thing2_env_stuff + env_tweaks) > > ...and so on. The current syntax for doing this is a tad verbose: > > def process(): > if time_to_do_thing1(): > env = base_env.copy() > env.update(thing1_env_stuff) > env.update(env_tweaks) > thing1(env) > del env > if time_to_do_thing2(): > env = base_env.copy() > env.update(thing2_env_stuff) > env.update(env_tweaks) > thing2(env) > del env Ah, you convinced me there is a use case indeed (though `del env` isn't necessary above). I would still prefer something that's not an operator, but I agree there is potential to improve the current state of affairs. Note that, if you're able to live with a third-party dependency, the `toolz` package has what you need (and lots of other things too): https://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoolz.merge Regards Antoine. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On 21/03/2019 17:59, Rhodri James wrote:
def process():
if time_to_do_thing1():
thing1(base_env + thing1_env_stuff + env_tweaks)
if time_to_do_thing2():
thing2(base_env + thing2_env_stuff + env_tweaks)
...and so on. The current syntax for doing this is a tad verbose:
def process():
if time_to_do_thing1():
env = base_env.copy()
env.update(thing1_env_stuff)
env.update(env_tweaks)
thing1(env)
del env
if time_to_do_thing2():
env = base_env.copy()
env.update(thing2_env_stuff)
env.update(env_tweaks)
thing2(env)
del env
Of course I forgot:
def process():
if time_to_do_thing1():
thing1({**base_env, **thing1_env_stuff, **env_tweaks})
if time_to_do_thing2():
thing2({**base_env, **thing2_env_stuff, **env_tweaks})
...which says something about how memorable that syntax is.
--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
I'd also like to add what I consider to be another point in favor of an
operator:
Throughout all of these related threads, I have seen many typos and
misspellings of current dict merging idioms, from messing up the number of
asterisks in "{**a, **b}", to even Guido(!) accidentally writing the common
copy/update idiom as
d = d1.copy()
d = d1.update(d2)
in a thoughtful email... and it was then copied-and-pasted (unquoted and
verbatim) by others!
I still have yet to see somebody (even those who claim to be confused by
it) mess up the PEP's current definition of "+" or "+=" in this context.
Brandt
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Thu, Mar 21, 2019 at 1:54 PM Stefan Behnel wrote: > Steven D'Aprano schrieb am 21.03.19 um 17:21: > > On Thu, Mar 21, 2019 at 09:11:18AM -0700, Guido van Rossum wrote: > > > >> I don't find it easy to understand or remember that d1.update(d2) > modifies > >> d1 in place, while d1.merge(d2) first copies d1. > >> > >> Maybe the name can indicate the copying stronger? Like we did with > sorting: > >> l.sort() sorts in-place, while sorted(l) returns a sorted copy. > > > > How about dict.merged(*args, **kw)? Or dict.updated()? > > And then users would accidentally type > > d.updated(items) > > and lack the tests to detect that this didn't do anything (except wasting > some time and memory). > > Stefan > Generally when I call a method named with a verb on an instance of something mutable, I expect it to do something on that instance and return None. So merged() or updated() feels more like a built-in or a function to import from somewhere, akin to sorted(). Perhaps dict.union(d2) could be considered? Three points in favor: 1) Not a verb, therefore makes it clearer that it returns something new. 2) Not confusable with existing dict methods. 3) It matches the name and behavior of set.union (modulo value conflicts), so will be easier to grok. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On 21/03/2019 17:06, Antoine Pitrou wrote: On Fri, 22 Mar 2019 03:42:00 +1100 Steven D'Aprano wrote: For those who oppose the + operator, it will help me if you made it clear whether it is *just* the + symbol you dislike, and would accept the | operator instead, or whether you hate the whole operator concept regardless of how it is spelled. I'd rather see a method. Dict merging just doesn't occur often enough that an operator is desirable for it. Analogous to the relationship between list.sort() and sorted(), I can't help but think that a dict.merge() method would be a terrible idea. A merged() function is more defensible. And to those who support this PEP, code examples where a dict merge operator will help are most welcome! I don't use Python often enough to have much to offer, I'm afraid. The sort of occasion I would use dict merging is passing modified environments to subcommands. Something like: def process(): if time_to_do_thing1(): thing1(base_env + thing1_env_stuff + env_tweaks) if time_to_do_thing2(): thing2(base_env + thing2_env_stuff + env_tweaks) ...and so on. The current syntax for doing this is a tad verbose: def process(): if time_to_do_thing1(): env = base_env.copy() env.update(thing1_env_stuff) env.update(env_tweaks) thing1(env) del env if time_to_do_thing2(): env = base_env.copy() env.update(thing2_env_stuff) env.update(env_tweaks) thing2(env) del env -- Rhodri James *-* Kynesim Ltd ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On 3/21/19 1:01 PM, Jonathan Fine wrote: Rémi Lapeyre wrote: Not matter the notation you end up choosing, I think this code: https://github.com/jpadilla/pyjwt/blob/master/jwt/utils.py#L71-L81 [...] would greatly benefit from a new merge to merge dicts. I've looked at the merge_dict defined in this code. It's similar to def gapfill(self, other): # See also: https://cobrapy.readthedocs.io/en/latest/gapfilling.html # Cobra's gapfill adds items to a model, to meet a requirement. for key in other.keys(): if key not in self: self[key] = other[key] (This is code I've written, that's not yet on PyPi.) The usage is different. Instead of writing one of aaa = merge_dict(aaa, bbb) ccc = merge_dict(aaa, bbb) you write one of gapfill(aaa, bbb) aaa.gapfill(bbb) # If gapfill added to dict methods. With merge_dict, you never really know if ccc is the same object as aaa, or a different one. Sometimes this is important. With gapfill, you get the same behaviour as the already familiar and loved dict.update. But of course with a different merge rule. With gapfill, I can never remeber whether it's gapfill(aaa, bbb) or gapfill(bbb, aaa). This is always important. :-) At least with aaa.gapfill(bbb), I have some sense of the "direction" of the asymmetry, or I would if I had some frame of reference into which to put the "gapfill" operation. (With the proposed + or | operator syntax, that gets lost.) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
Chris Angelico writes:
> ... then, in the interests of productive discussion, could you please
> explain? What is it about dict addition that makes it harder to
> understand than other addition?
Antoine didn't say what dict addition does is harder to understand
than other addition. He said he wants to understand it without
knowing what it does.
I can't say for sure what he means precisely, but I take it that he
wants dict "+" to obey certain regularities that other instances of
"+" do, possibly including outside of Python. As you'll see I find it
hard to make this precise, but it's a pretty strong feeling for me, as
well.
To me, those regularities include associativity (satisfied in Python
except for floats) and commutativity where possible (integers and I
believe floats do satisfy it, while strings cannot and other sequences
in Python in general do not, although they very often do in
mathematics).
For mappings, the mathematical meaning of "+" is usually pointwise.
This wouldn't make sense for strings (interpreted as mappings from a
prefix of the natural numbers) at all except that by accident in
Python s1[n] + s2[n] does make sense, but not pointwise (because the
length of the result is 2, not 1, for each n).
For sequences in general pointwise doesn't make sense (there's no
restriction to homogeneous sequences, and if there were, like strings
it's not clear that the elements would be summable in an appropriate
sense). But concatenation always makes sense, especially by analogy
to the somehow (IMO) canonical case of strings. For sets, the only
plausible interpretation of "addition" is union, but in fact Python
used .add asymmetrically as "add to", not "add together" (self is a
set, argument is a generic object), and the union operator is "|", not
"+".
For dictionaries, neither pointwise addition nor concatenation makes
sense in general, and update is "too asymmetric" for my taste, and has
no analog in the usual algebras of mappings.
In some sense string concatenation, though noncommutative, doesn't
lose information, and it does obey a sort of antisymmetry in that
a + b == reversed(reversed(b) + reversed(a)). Dictionary update does
lose the original settings.
If people really think it's so important to spell
d = d0.copy()
d.update(d1)
as "d0 + d1" despite the noncommutativity (and the availability of
"{**d0, **d1}" for "true" dicts), and by extension the redundant "d |=
d1" for "d.update(d1)", I won't get terribly upset, but I will be sad
because it offends my sense of "beautiful code" (including TOOWTDI,
where "+" for dicts would violate both the "obvious" and the
parenthetical "only one" conditions IMO). I would consider it a wart
in the same way that many people consider str.join a wart, as it
breaks even more of the regularities I associate with "+" than string
concatenation does.
Again, I don't know what Antoine meant, but I might say the same kind
of thing in the same words, and the above is what I would mean.
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
Brandt Bucher wrote:
> Before:
>
> tmp = keep.copy()
> tmp.update(separate)
> result = function(param=tmp)
> del tmp
> After:
>
> result = f(param=keep+separate)
I'd rewrite this example as:
Before:
fn(param={**keep, **separate})
After:
fn(param=keep + separate)
--
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
Rémi Lapeyre wrote: > Not matter the notation you end up choosing, I think this code: > https://github.com/jpadilla/pyjwt/blob/master/jwt/utils.py#L71-L81 > [...] would greatly benefit from a new merge to merge dicts. I've looked at the merge_dict defined in this code. It's similar to def gapfill(self, other): # See also: https://cobrapy.readthedocs.io/en/latest/gapfilling.html # Cobra's gapfill adds items to a model, to meet a requirement. for key in other.keys(): if key not in self: self[key] = other[key] (This is code I've written, that's not yet on PyPi.) The usage is different. Instead of writing one of aaa = merge_dict(aaa, bbb) ccc = merge_dict(aaa, bbb) you write one of gapfill(aaa, bbb) aaa.gapfill(bbb) # If gapfill added to dict methods. With merge_dict, you never really know if ccc is the same object as aaa, or a different one. Sometimes this is important. With gapfill, you get the same behaviour as the already familiar and loved dict.update. But of course with a different merge rule. -- Jonathan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
Steven D'Aprano schrieb am 21.03.19 um 17:21: > On Thu, Mar 21, 2019 at 09:11:18AM -0700, Guido van Rossum wrote: > >> I don't find it easy to understand or remember that d1.update(d2) modifies >> d1 in place, while d1.merge(d2) first copies d1. >> >> Maybe the name can indicate the copying stronger? Like we did with sorting: >> l.sort() sorts in-place, while sorted(l) returns a sorted copy. > > How about dict.merged(*args, **kw)? Or dict.updated()? And then users would accidentally type d.updated(items) and lack the tests to detect that this didn't do anything (except wasting some time and memory). Stefan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
> > And to those who support this PEP, code examples where a dict merge > operator will help are most welcome! I would definitely include the example you alluded to in the operators thread: Before: tmp = keep.copy() tmp.update(separate) result = function(param=tmp) del tmp After: result = f(param=keep+separate) Thanks for drafting the PEP for this. There seems to be a bit of an echo in these 5+ threads, and your commentary has definitely been more constructive/original than most. Looking forward to the next revision! Brandt On Thu, Mar 21, 2019 at 9:42 AM Steven D'Aprano wrote: > I'd like to make a plea to people: > > I get it, there is now significant opposition to using the + symbol for > this proposed operator. At the time I wrote the first draft of the PEP, > there was virtually no opposition to it, and the | operator had very > little support. This has clearly changed. > > At this point I don't think it is productive to keep making subjective > claims that + will be more confusing or surprising. You've made your > point that you don't like it, and the next draft^1 of the PEP will make > that clear. > > But if you have *concrete examples* of code that currently is easy to > understand, but will be harder to understand if we add dict.__add__, > then please do show me! > > For those who oppose the + operator, it will help me if you made it > clear whether it is *just* the + symbol you dislike, and would accept > the | operator instead, or whether you hate the whole operator concept > regardless of how it is spelled. > > And to those who support this PEP, code examples where a dict merge > operator will help are most welcome! > > > > > ^1 Coming Real Soon Now™. > > > -- > Steven > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, 21 Mar 2019 at 17:27, Rémi Lapeyre wrote: > > Le 21 mars 2019 à 17:43:31, Steven D'Aprano > (st...@pearwood.info(mailto:st...@pearwood.info)) a écrit: > > > I'd like to make a plea to people: > > > > I get it, there is now significant opposition to using the + symbol for > > this proposed operator. At the time I wrote the first draft of the PEP, > > there was virtually no opposition to it, and the | operator had very > > little support. This has clearly changed. > > > > At this point I don't think it is productive to keep making subjective > > claims that + will be more confusing or surprising. You've made your > > point that you don't like it, and the next draft^1 of the PEP will make > > that clear. > > > > But if you have *concrete examples* of code that currently is easy to > > understand, but will be harder to understand if we add dict.__add__, > > then please do show me! > > > > For those who oppose the + operator, it will help me if you made it > > clear whether it is *just* the + symbol you dislike, and would accept > > the | operator instead, or whether you hate the whole operator concept > > regardless of how it is spelled. > > Thanks for the work you are doing on this PEP and for debunking my > misconceptions regarding types, I’m currently learning a lot about them. > > I don’t know if it matters but I’m in favor of the method > > > And to those who support this PEP, code examples where a dict merge > > operator will help are most welcome! > > Not matter the notation you end up choosing, I think this code: > https://github.com/jpadilla/pyjwt/blob/master/jwt/utils.py#L71-L81 > > which is part of a widely used library to validate JWTs would greatly > benefit from a new merge to merge dicts. (This package is 78 on > https://hugovk.github.io/top-pypi-packages/) It's already got a function that does the job. How much benefit is there *really* from being able to replace it with d1 + d2 once you drop support for Python < 3.8? But point taken that new code would have been able to avoid the function in the first place. ... or would it? def merge_dict(original, updates): if not updates: return original With the + operator. d1 + None will fail with an error. With your code, updates=None means "return the original unchanged". Does that matter with your current code? The point is that in many real world cases, you'd write a function *anyway*, to handle corner cases, and a new operator doesn't make much difference at that point. Having said all of that, I'm mostly indifferent to the idea of having a built in "dictionary merge" capability - I doubt I'd use it *much*, but if it were there I'm sure I'd find useful it on the odd occasion. I'm somewhat against an operator, I really don't see why this couldn't be a method (although the asymmetry in d1.merge(d2) makes me have a mild preference for a class method or standalone function). I can't form an opinion between + and |, I find | significantly uglier (I tend to avoid using it for sets, in favour of the union method) but I am mildly uncomfortable with more overloading of +. Serious suggestion - why not follow the lead of sets, and have *both* an operator and a method? And if you think that's a bad idea, it would be worth considering *why* it's a bad idea for dictionaries, when it's OK for sets (and "well, I didn't like it when sets did it" isn't sufficient ;-)) And having said that, I'll go back to lurking and not really caring one way or the other. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
Steven D'Aprano wrote:
> But if you have *concrete examples* of code that currently is easy to
> understand, but will be harder to understand if we add dict.__add__,
> then please do show me!
# What does this do?
>>> items. update(points)
# And what does this do?
>>> items += points
What did you get? Here's one possible context.
>>> Point = namedtuple('Point', ['x', 'y'])
>>> p, q, r = Point(1,2), Point(3, 4), Point(5, 6)
>>> points = set([p, q, r])
>>> points
{Point(x=1, y=2), Point(x=5, y=6), Point(x=3, y=4)}
>>> items = dict(a=4, b=8)
--
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
Le 21 mars 2019 à 17:43:31, Steven D'Aprano (st...@pearwood.info(mailto:st...@pearwood.info)) a écrit: > I'd like to make a plea to people: > > I get it, there is now significant opposition to using the + symbol for > this proposed operator. At the time I wrote the first draft of the PEP, > there was virtually no opposition to it, and the | operator had very > little support. This has clearly changed. > > At this point I don't think it is productive to keep making subjective > claims that + will be more confusing or surprising. You've made your > point that you don't like it, and the next draft^1 of the PEP will make > that clear. > > But if you have *concrete examples* of code that currently is easy to > understand, but will be harder to understand if we add dict.__add__, > then please do show me! > > For those who oppose the + operator, it will help me if you made it > clear whether it is *just* the + symbol you dislike, and would accept > the | operator instead, or whether you hate the whole operator concept > regardless of how it is spelled. Thanks for the work you are doing on this PEP and for debunking my misconceptions regarding types, I’m currently learning a lot about them. I don’t know if it matters but I’m in favor of the method > And to those who support this PEP, code examples where a dict merge > operator will help are most welcome! Not matter the notation you end up choosing, I think this code: https://github.com/jpadilla/pyjwt/blob/master/jwt/utils.py#L71-L81 which is part of a widely used library to validate JWTs would greatly benefit from a new merge to merge dicts. (This package is 78 on https://hugovk.github.io/top-pypi-packages/) Rémi > ^1 Coming Real Soon Now™. > > > -- > Steven > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Fri, 22 Mar 2019 03:42:00 +1100 Steven D'Aprano wrote: > > For those who oppose the + operator, it will help me if you made it > clear whether it is *just* the + symbol you dislike, and would accept > the | operator instead, or whether you hate the whole operator concept > regardless of how it is spelled. I'd rather see a method. Dict merging just doesn't occur often enough that an operator is desirable for it. > And to those who support this PEP, code examples where a dict merge > operator will help are most welcome! Yes, I still have no idea why this operator would supposedly be useful. How many dict merges do you write per month? Regards Antoine. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
I'd like to make a plea to people: I get it, there is now significant opposition to using the + symbol for this proposed operator. At the time I wrote the first draft of the PEP, there was virtually no opposition to it, and the | operator had very little support. This has clearly changed. At this point I don't think it is productive to keep making subjective claims that + will be more confusing or surprising. You've made your point that you don't like it, and the next draft^1 of the PEP will make that clear. But if you have *concrete examples* of code that currently is easy to understand, but will be harder to understand if we add dict.__add__, then please do show me! For those who oppose the + operator, it will help me if you made it clear whether it is *just* the + symbol you dislike, and would accept the | operator instead, or whether you hate the whole operator concept regardless of how it is spelled. And to those who support this PEP, code examples where a dict merge operator will help are most welcome! ^1 Coming Real Soon Now™. -- Steven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Thu, Mar 21, 2019 at 09:11:18AM -0700, Guido van Rossum wrote: > I don't find it easy to understand or remember that d1.update(d2) modifies > d1 in place, while d1.merge(d2) first copies d1. > > Maybe the name can indicate the copying stronger? Like we did with sorting: > l.sort() sorts in-place, while sorted(l) returns a sorted copy. How about dict.merged(*args, **kw)? Or dict.updated()? That would eliminate some of the difficulties with an operator, such as the difference between + which requires both operands to be a dict but += which can take any mapping or (key,value) iterable. -- Steven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Thu, Mar 21, 2019 at 09:11:18AM -0700, Guido van Rossum wrote: > I don't find it easy to understand or remember that d1.update(d2) modifies > d1 in place, while d1.merge(d2) first copies d1. > > Maybe the name can indicate the copying stronger? Like we did with sorting: > l.sort() sorts in-place, while sorted(l) returns a sorted copy. Then shouldn't it be a function (not a method)? dictutils.merge()? > --Guido van Rossum (python.org/~guido) Oleg. -- Oleg Broytmanhttps://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019 at 03:16:44PM +0200, Serhiy Storchaka wrote:
> 21.03.19 14:51, Chris Angelico пише:
> >... then, in the interests of productive discussion, could you please
> >explain? What is it about dict addition that makes it harder to
> >understand than other addition?
>
> Currently the + operator has 2 meanings for builtin types (both are
> widely used), after adding it for dicts it will have 3 meanings.
Just two meanings? I get at least eight among the builtins:
- int addition;
- float addition;
- complex addition;
- string concatenation;
- list concatenation;
- tuple concatenation;
- bytes concatenation;
- bytearray concatenation.
I suppose if you cover one eye and focus on the "big picture", ignoring
vital factors like "you can't add a list to a string" and "float
addition and int addition aren't precisely the same", we might pretend
that this is just two operations:
- numeric addition;
- sequence concatenation.
But in practice, when reading code, it's usually not enough to know that
some use of the + operator means "concatenation", you need to know
*what* is being concatenated. There's no point trying to add a tuple if
a bytearray is required.
> 3 > 2, is not?
Okay, but how does this make it harder to determine what a piece of code
using + does? Antoine insists that *if we allow dict addition*, then we
won't be able to tell what
spam + eggs # for example
does unless we know what spam and eggs are. This is very true. But it is
*equally true today*, right now, and its been equally true going back to
Python 1.0 or before.
This proposed change doesn't add any uncertainty that doesn't already
exist, nor will it make code that is clear today less clear tomorrow.^1
And don't forget that Python allows us to create non-builtin types that
overload operators. If you don't know what spam and eggs are, you can't
assume they are builtins. With operator overloading, any operator can
mean literally anything at all. In practice though, this rarely becomes
a serious problem.
Is there a significant increase in difficulty between the current
situation:
# Is this addition or concatenation or something else?
spam + eggs
versus the proposed:
# Is this addition or concatenation or merge or something else?
spam + eggs
Obviously there's *one more builtin* to consider, but I don't think that
changes the process of understanding the meaning of the operation.
I think that the problem you and Antoine fear ("dict.__add__ will make
it harder to read code") requires a process that goes something like
this:
1. Here's a mysterious "spam + eggs" operation we need to understand.
2. For each operation in ("numeric addition", "concatenation"):
3. assume + represents that operation;
4. if we understand the spam+eggs expression now, break
If that's how we read code, then adding one more operation would make it
harder to understand. We'd have to loop three times, not twice:
2. For each operation in ("numeric addition", "concatenation", "dict merging"):
Three is greater than two, so we may have to do more work to understand
the code. But I don't think that's how people actually read code. I
think they do this:
1. Here's a mysterious "spam + eggs" operation we need to understand.
2. Read the code to find out what spam and eggs are.
3. Knowing what they are (tuples, lists, floats, etc) immediately tells
you what the plus operator does; at worst, a programmer unfamiliar
with the type may need to read the docs.
Adding dict.__add__ doesn't make it any harder to work out what the
operands spam and eggs are. The process we go through to determine what
the operands are remains the same:
- if one of operands is a literal, that gives you a strong hint that
the other is the same type;
- the names or context may make it clear ("header + text" probably
isn't doing numeric addition);
- read back through the code looking for where the variables are
defined; etc.
That last bit isn't always easy. People can write obfuscated, complex
code using poor or misleading names. But allowing dict.__add__ doesn't
make it more obfuscated or more complex. Usually the naming and context
will make it clear. Most code is not terrible.
At worst, there will be a transition period where people have a
momentary surprise:
"Wait, what, these are dicts??? How can you add dicts???"
but then they will read the docs (or ask StackOverflow) and the second
time they see it, it shouldn't be a surprise.
^1 That's my assertion, but if anyone has a concrete example of actual
code which is self-evident today but will become ambiguous if this
proposal goes ahead, please show me!
--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On 2019-03-21 17:11, Guido van Rossum wrote: I don't find it easy to understand or remember that d1.update(d2) modifies d1 in place, while d1.merge(d2) first copies d1. That would be an advantage with + versus += (or | versus |=). ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Thu, Mar 21, 2019 at 7:45 AM Antoine Pitrou wrote: > On Tue, 5 Mar 2019 16:39:40 +0900 > INADA Naoki > wrote: > > I think some people in favor of PEP 584 just want > > single expression for merging dicts without in-place update. > > > > But I feel it's abuse of operator overload. I think functions > > and methods are better than operator unless the operator > > has good math metaphor, or very frequently used as concatenate > > strings. > > > > This is why function and methods are better: > > > > * Easy to search. > > * Name can describe it's behavior better than abused operator. > > * Simpler lookup behavior. (e.g. subclass and __iadd__) > > > > Then, I propose `dict.merge` method. It is outer-place version > > of `dict.update`, but accepts multiple dicts. (dict.update() > > can be updated to accept multiple dicts, but it's not out of scope). > > > > * d = d1.merge(d2) # d = d1.copy(); d.update(d2) > > One should also be able to write `d = dict.merge(d1, d2, ...)` > > If dict merging is important enough to get a new spelling, then I think > this proposal is the best: explicit, unambiguous, immediately > understandable and easy to remember. > I don't find it easy to understand or remember that d1.update(d2) modifies d1 in place, while d1.merge(d2) first copies d1. Maybe the name can indicate the copying stronger? Like we did with sorting: l.sort() sorts in-place, while sorted(l) returns a sorted copy. -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
21.03.19 15:24, Chris Angelico пише: On Fri, Mar 22, 2019 at 12:17 AM Serhiy Storchaka wrote: 21.03.19 14:51, Chris Angelico пише: ... then, in the interests of productive discussion, could you please explain? What is it about dict addition that makes it harder to understand than other addition? Currently the + operator has 2 meanings for builtin types (both are widely used), after adding it for dicts it will have 3 meanings. 3 > 2, is not? I suppose you could call it two (numeric addition and sequence concatenation), but there are subtleties to the way that lists concatenate that don't apply to strings (esp since lists are mutable), so I'd call it at least three already. I do not understand what are these subtleties that you treat list concatenation different from string concatenation. Could you please explain? In any case, it does not matter how you count meanings, n + 1 > n. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Tue, 5 Mar 2019 16:39:40 +0900 INADA Naoki wrote: > I think some people in favor of PEP 584 just want > single expression for merging dicts without in-place update. > > But I feel it's abuse of operator overload. I think functions > and methods are better than operator unless the operator > has good math metaphor, or very frequently used as concatenate > strings. > > This is why function and methods are better: > > * Easy to search. > * Name can describe it's behavior better than abused operator. > * Simpler lookup behavior. (e.g. subclass and __iadd__) > > Then, I propose `dict.merge` method. It is outer-place version > of `dict.update`, but accepts multiple dicts. (dict.update() > can be updated to accept multiple dicts, but it's not out of scope). > > * d = d1.merge(d2) # d = d1.copy(); d.update(d2) One should also be able to write `d = dict.merge(d1, d2, ...)` If dict merging is important enough to get a new spelling, then I think this proposal is the best: explicit, unambiguous, immediately understandable and easy to remember. Regards Antoine. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019 at 9:17 AM Serhiy Storchaka wrote: > 21.03.19 14:51, Chris Angelico пише: > > ... then, in the interests of productive discussion, could you please > > explain? What is it about dict addition that makes it harder to > > understand than other addition? > > Currently the + operator has 2 meanings for builtin types (both are > widely used), after adding it for dicts it will have 3 meanings. > > 3 > 2, is not? > It depends how abstractly you define the "meanings". If you define + as "arithmetic addition" and "sequence concatenation", then yes, there are 2. But novices have to learn that the same concatenation operator applies to strings as well as lists/tuples. And when reading x + y, it is probably relevant whether x and y are numbers, strings, or sequence containers like lists. The proposal would generalize "sequence concatenation" to something like "asymmetric sequence/collection combination". (Asymmetric because d1 + d2 may not equal d2 + d1.) It seems a natural extension to me, though the | alternative is also reasonable (interpreted as taking the OR of keys in the two dicts; but unlike unioning two sets, the dict-merge operator would be asymmetric). The third proposed alternative, <<, has no "baggage" from an existing use as a combination operator, but at the same time it is a more obscure choice. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Fri, Mar 22, 2019 at 12:17 AM Serhiy Storchaka wrote: > > 21.03.19 14:51, Chris Angelico пише: > > ... then, in the interests of productive discussion, could you please > > explain? What is it about dict addition that makes it harder to > > understand than other addition? > > Currently the + operator has 2 meanings for builtin types (both are > widely used), after adding it for dicts it will have 3 meanings. > > 3 > 2, is not? I suppose you could call it two (numeric addition and sequence concatenation), but there are subtleties to the way that lists concatenate that don't apply to strings (esp since lists are mutable), so I'd call it at least three already. And what about non-builtin types? You can add two numpy arrays and it does pairwise addition, quite different from how lists add together. But in every case, the + operator means "add these things together". It will be the same with dicts: you add the dicts together. Antoine has stated that the problem is NOT understanding what dict.__add__ does, so I am at a loss as to what the problem IS. We *already* have many different definitions of "add", according to the data types involved. That is exactly what polymorphism is for. Why is it such a bad thing for a dict? Now, my own opinion is that | would be a better operator than +, but it's only a weak preference, and I'd be happy with either. Also, to my understanding, the concerns about "what does addition mean" apply identically to "what does Or mean", but as we've already seen, my understanding doesn't currently extend as far as comprehending this issue. Hence asking. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On 21/03/2019 11:34, Antoine Pitrou wrote: On Wed, 20 Mar 2019 15:46:24 -1000 Christopher Barker wrote: This is precisely why I worded my question this way: what has changed in the last 20 years that make a "+" dict operator more compelling today than it was? Do we merge dicts much more frequently than we did? The analogy doesn't hold because @ was a new operator -- a MUCH bigger change than dimply defining the use of + (or | ) for dicts. But it's less disruptive when reading code, because "x @ y" is unambiguous: it's a matrix multiplication. "x + y" can be many different things, and now it can be one more thing. "x @ y" is unambiguous once you know what it means. Until then, it's just mysterious. I wouldn't mind the new operator if its meaning was clear-cut. But here we have potential for confusion, both for writers and readers of code. but it's NOT a new operator, it is making use of an existing one, and sure you could guess at a couple meanings, but the merge one is probably one of the most obvious to guess, and one quick test and you know -- I really can't see it being a ongoing source of confusion. Did you actually read what I said? The problem is not to understand what dict.__add__ does. It's to understand what code using the + operator does, without knowing upfront whether the inputs are dicts. Welcome to polymorphism. -- Rhodri James *-* Kynesim Ltd ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
21.03.19 14:51, Chris Angelico пише: ... then, in the interests of productive discussion, could you please explain? What is it about dict addition that makes it harder to understand than other addition? Currently the + operator has 2 meanings for builtin types (both are widely used), after adding it for dicts it will have 3 meanings. 3 > 2, is not? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, 21 Mar 2019 23:51:12 +1100 Chris Angelico wrote: > On Thu, Mar 21, 2019 at 11:45 PM Antoine Pitrou wrote: > > > > On Thu, 21 Mar 2019 23:35:36 +1100 > > Chris Angelico wrote: > > > On Thu, Mar 21, 2019 at 10:35 PM Antoine Pitrou > > > wrote: > > > > > but it's NOT a new operator, it is making use of an existing one, and > > > > > sure > > > > > you could guess at a couple meanings, but the merge one is probably > > > > > one of > > > > > the most obvious to guess, and one quick test and you know -- I really > > > > > can't see it being a ongoing source of confusion. > > > > > > > > Did you actually read what I said? The problem is not to understand > > > > what dict.__add__ does. It's to understand what code using the + > > > > operator does, without knowing upfront whether the inputs are dicts. > > > > > > The + operator adds two things together. I don't understand the issue > > > here. > > > > I'm not expecting you to understand, either. > > > > ... then, in the interests of productive discussion, could you please > explain? What is it about dict addition that makes it harder to > understand than other addition? "Productive discussion" is something that requires mutual implication. Asking me to repeat exactly what I spelled out above (and that you even quoted) is not productive. Regards Antoine. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019 at 11:45 PM Antoine Pitrou wrote: > > On Thu, 21 Mar 2019 23:35:36 +1100 > Chris Angelico wrote: > > On Thu, Mar 21, 2019 at 10:35 PM Antoine Pitrou wrote: > > > > but it's NOT a new operator, it is making use of an existing one, and > > > > sure > > > > you could guess at a couple meanings, but the merge one is probably one > > > > of > > > > the most obvious to guess, and one quick test and you know -- I really > > > > can't see it being a ongoing source of confusion. > > > > > > Did you actually read what I said? The problem is not to understand > > > what dict.__add__ does. It's to understand what code using the + > > > operator does, without knowing upfront whether the inputs are dicts. > > > > The + operator adds two things together. I don't understand the issue here. > > I'm not expecting you to understand, either. > ... then, in the interests of productive discussion, could you please explain? What is it about dict addition that makes it harder to understand than other addition? ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, 21 Mar 2019 23:35:36 +1100 Chris Angelico wrote: > On Thu, Mar 21, 2019 at 10:35 PM Antoine Pitrou wrote: > > > but it's NOT a new operator, it is making use of an existing one, and sure > > > you could guess at a couple meanings, but the merge one is probably one of > > > the most obvious to guess, and one quick test and you know -- I really > > > can't see it being a ongoing source of confusion. > > > > Did you actually read what I said? The problem is not to understand > > what dict.__add__ does. It's to understand what code using the + > > operator does, without knowing upfront whether the inputs are dicts. > > The + operator adds two things together. I don't understand the issue here. I'm not expecting you to understand, either. Regards Antoine. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Thu, Mar 21, 2019 at 10:35 PM Antoine Pitrou wrote: > > but it's NOT a new operator, it is making use of an existing one, and sure > > you could guess at a couple meanings, but the merge one is probably one of > > the most obvious to guess, and one quick test and you know -- I really > > can't see it being a ongoing source of confusion. > > Did you actually read what I said? The problem is not to understand > what dict.__add__ does. It's to understand what code using the + > operator does, without knowing upfront whether the inputs are dicts. The + operator adds two things together. I don't understand the issue here. You can add integers: 1 + 2 == 3 You can add floats: 0.5 + 1.25 == 1.75 You can add lists: [1,2] + [3,4] == [1,2,3,4] You can add strings: "a" + "b" == "ab" And soon you'll be able to add dictionaries. The exact semantics need to be defined, but it's not fundamentally changing how you interpret the + operator. I don't understand the panic here - or rather, I don't understand why it's happening NOW, not back when lists got the ability to be added (if that wasn't in the very first release). Conversely, if it's the | operator, it's a matter of merging, and the same is true. You can merge integers, treating them as bit sets. You can merge sets. And now you'll be able to merge dictionaries. Same same. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Wed, 20 Mar 2019 15:46:24 -1000 Christopher Barker wrote: > > > This is precisely why I worded my question this way: what has changed > > in the last 20 years that make a "+" dict operator more compelling > > today than it was? Do we merge dicts much more frequently than we > > did? > > The analogy doesn't hold because @ was a new operator -- a MUCH bigger > change than dimply defining the use of + (or | ) for dicts. But it's less disruptive when reading code, because "x @ y" is unambiguous: it's a matrix multiplication. "x + y" can be many different things, and now it can be one more thing. > I wouldn't mind the new operator if its meaning was clear-cut. But > > here we have potential for confusion, both for writers and readers of > > code. > > > > but it's NOT a new operator, it is making use of an existing one, and sure > you could guess at a couple meanings, but the merge one is probably one of > the most obvious to guess, and one quick test and you know -- I really > can't see it being a ongoing source of confusion. Did you actually read what I said? The problem is not to understand what dict.__add__ does. It's to understand what code using the + operator does, without knowing upfront whether the inputs are dicts. Regards Antoine. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
