Re: [Python-ideas] Allow creation of polymorph function (async function executable syncronously)
Is this what syncer does with a sync() function? Why isn't this a built-in? https://github.com/miyakogi/syncer On Wednesday, March 6, 2019, Nathaniel Smith wrote: > Defining a single polymorphic function is easy at the library level. > For example, with asyncio: > > > > def maybe_async(fn): > @functools.wraps(fn) > def wrapper(*args, **kwargs): > coro = fn(*args, **kwargs) > if asyncio.get_running_loop() is not None: > return coro > else: > return await coro > > @maybe_async > async def my_func(...): > ... use asyncio freely in here ... > > > > You can't do it at the language level though (e.g. with your proposed > 'polymorph' keyword), because the language doesn't know whether an > event loop is running or not. > > Extending this from a single function to a whole library API is > substantially more complex, because you have to wrap every function > and method, deal with __iter__ versus __aiter__, etc. > > -n > > On Tue, Mar 5, 2019 at 8:02 PM Jorropo . wrote: > > > > I was doing some async networking and I wondered, why I have to use 2 > different api for making the same things in async or sync regime. > > Even if we make 2 perfectly identical api (except function will be sync > and async), it will still 2 different code). > > > > So I first thinked to allow await in syncronous function but that create > some problems (ex: an async function calling async.create_task) so if we > allow that we have to asume to allways be in async regime (like js). > > > > Or we can differentiate async function wich can be awaited in syncronous > regime, maybe with a new keyword (here I will use polymorph due to a lack > of imagination but I find that one too long) ? > > > > So a polymorph function can be awaited in a syncronous function, and a > polymorph function can only await polymorph functions. > > > > Polymorph function work exacly like async function BUT they assure of > the ability to execute syncronously. > > And in a syncronous regime if an await need to wait (like async.sleep or > network operation), just wait (like the equivalent of this function in > syncronous way). > > > > So why made that ? > > To provide the same api for async and sync regime when its possible, > example http api. > > This allow to code less librairy. > > Syncronous users can just use the librairy like any other sync lib (with > the keyword await for executing but, personally, I think that is worth). > > And asyncronous users can run multiples tasks using the same lib. > > Moving from a regime to an other is simpler, source code size is reduced > (doesn't need to create 2 api for the same lib), gain of time for the same > reason. > > > > Also why it need to await in syncronous function, why not just execute > polymorph function like any sync function while called in a sync function ? > > Because we need to create runnable objects for async.run, ... > > > > So I would have your though about that, what can be improved, a better > name for polymorph ? > > ___ > > Python-ideas mailing list > > Python-ideas@python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > Nathaniel J. Smith -- https://vorpus.org > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Suggestions: dict.flow_update and dict.__add__
dicttoolz has functions for working with these objects; including dicttoolz.merge (which returns a reference to the merged dicts but does not mutate the arguments passed). https://toolz.readthedocs.io/en/latest/api.html#dicttoolz https://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoolz.merge pyrsistent has a PRecord class with invariants and type checking that precedes dataclasses. pyrsistent also has 'freeze' and 'thaw' functions for immutability. PRecord extends PMap, which implements __add__ as self.update(arg) (which does not mutate self) https://github.com/tobgu/pyrsistent/blob/master/README.rst#precord https://github.com/tobgu/pyrsistent/blob/master/pyrsistent/_pmap.py On Tuesday, March 5, 2019, Guido van Rossum wrote: > If you have to tell such a long and convoluted story to explain a name > that you've picked out of the blue and that has no equivalent in other > Python data types, it's probably a bad idea. If you're proposing that other > mutating methods also gain a flow_XXX variant, please, no! That's like the > theory of supersymmetry (SUSY) in particle physics, where ever known > particle from the Standard Model would have to have a much heavier > "superpartner" just to make some esoteric idea work. > > On Tue, Mar 5, 2019 at 12:54 AM Jonathan Fine wrote: > >> SUMMARY >> Instead of using dict + dict, perhaps use dict.flow_update. Here, >> flow_update is just like update, except that it returns self. >> >> BACKGROUND >> There's a difference between a sorted copy of a list, and sorting the >> list in place. >> >> >>> items = [2, 0, 1, 9] >> >>> sorted(items), items >> ([0, 1, 2, 9], [2, 0, 1, 9]) >> >>> items.sort(), items >>(None, [0, 1, 2, 9]) >> >> In Python, mutating methods generally return None. Here, this prevents >> beginners thinking their code has produced a sorted copy of a list, >> when in fact it has done an in-place sort on the list. If they write >> >>> aaa = my_list.sort() >> they'll get a None error when they use aaa. >> >> The same goes for dict.update. This is a useful feature, particularly >> for beginners. It helps them think clearly, and express themselves >> clearly. >> >> THE PROBLEM >> This returning None can be a nuisance, sometimes. Suppose we have a >> dictionary of default values, and a dictionary of use supplied >> options. We wish to combine the two dictionaries, say into a new >> combined dictionary. >> >> One way to do this is: >> >>combined = defaults.copy() >>combined.update(options) >> >> But this is awkward when you're in the middle of calling a function: >> >> call_big_method( >> # lots of arguments, one to a line, with comments >> arg = combined, # Look up to see what combined is. >> # more arguments >> ) >> >> USING + >> There's a suggestion, that instead one extends Python so that this works: >> arg = defaults + options # What does '+' mean here? >> >> USING flow_update >> Here's another suggestion. Instead write: >> dict_arg = defaults.copy().flow_update(options) # Is this >> clearer? >> >> IMPLEMENTATION >> Here's an implementation, as a subclass of dict. >> >> class mydict(dict): >> >> def flow_update(self, *argv, **kwargs): >> self.update(*argv, **kwargs) >> return self >> >> def copy(self): >> return self.__class__(self) >> >> A DIRTY HACK >> Not tested, using an assignment expression. >>dict_arg = (tmp := defaults.copy(), tmp.update(options))[0] >> Not recommend. >> >> -- >> Jonathan >> ___ >> Python-ideas mailing list >> Python-ideas@python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > --Guido van Rossum (python.org/~guido) > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Allow creation of polymorph function (async function executable syncronously)
I was doing some async networking and I wondered, why I have to use 2 different api for making the same things in async or sync regime. Even if we make 2 perfectly identical api (except function will be sync and async), it will still 2 different code). So I first thinked to allow await in syncronous function but that create some problems (ex: an async function calling async.create_task) so if we allow that we have to asume to allways be in async regime (like js). Or we can differentiate async function wich can be awaited in syncronous regime, maybe with a new keyword (here I will use polymorph due to a lack of imagination but I find that one too long) ? So a polymorph function can be awaited in a syncronous function, and a polymorph function can only await polymorph functions. Polymorph function work exacly like async function BUT they assure of the ability to execute syncronously. And in a syncronous regime if an await need to wait (like async.sleep or network operation), just wait (like the equivalent of this function in syncronous way). So why made that ? To provide the same api for async and sync regime when its possible, example http api. This allow to code less librairy. Syncronous users can just use the librairy like any other sync lib (with the keyword await for executing but, personally, I think that is worth). And asyncronous users can run multiples tasks using the same lib. Moving from a regime to an other is simpler, source code size is reduced (doesn't need to create 2 api for the same lib), gain of time for the same reason. Also why it need to await in syncronous function, why not just execute polymorph function like any sync function while called in a sync function ? Because we need to create runnable objects for async.run, ... So I would have your though about that, what can be improved, a better name for polymorph ? ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
On Tue, Mar 5, 2019 at 6:07 PM Raymond Hettinger < raymond.hettin...@gmail.com> wrote: > > > On Mar 5, 2019, at 2:13 PM, Greg Ewing > wrote: > > > > Rhodri James wrote: > >> I have to go and look in the documentation because I expect the union > operator to be '+'. > > > > Anyone raised on Pascal is likely to find + and * more > > natural. Pascal doesn't have bitwise operators, so it > > re-uses + and * for set operations. I like the economy > > of this arrangement -- it's not as if there's any > > other obvious meaning that + and * could have for sets. > > The language SETL (the language of sets) also uses + and * for set > operations.¹ > So the secret is out: Python inherits a lot from SETL, through ABC -- ABC was heavily influenced by SETL. > ¹ https://www.linuxjournal.com/article/6805 > ² https://www.python.org/dev/peps/pep-0218/ > -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
> On Mar 5, 2019, at 2:13 PM, Greg Ewing wrote: > > Rhodri James wrote: >> I have to go and look in the documentation because I expect the union >> operator to be '+'. > > Anyone raised on Pascal is likely to find + and * more > natural. Pascal doesn't have bitwise operators, so it > re-uses + and * for set operations. I like the economy > of this arrangement -- it's not as if there's any > other obvious meaning that + and * could have for sets. The language SETL (the language of sets) also uses + and * for set operations.¹ For us though, the decision to use | and & are set in stone. The time for debating the decision was 19 years ago.² Raymond ¹ https://www.linuxjournal.com/article/6805 ² https://www.python.org/dev/peps/pep-0218/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Wed, Mar 6, 2019 at 12:08 AM Guido van Rossum wrote:
> On Tue, Mar 5, 2019 at 3:50 PM Josh Rosenberg <
> shadowranger+pythonid...@gmail.com> wrote:
>
>>
>> On Tue, Mar 5, 2019 at 11:16 PM Steven D'Aprano
>> wrote:
>>
>>> On Sun, Mar 03, 2019 at 09:28:30PM -0500, James Lu wrote:
>>>
>>> > I propose that the + sign merge two python dictionaries such that if
>>> > there are conflicting keys, a KeyError is thrown.
>>>
>>> This proposal is for a simple, operator-based equivalent to
>>> dict.update() which returns a new dict. dict.update has existed since
>>> Python 1.5 (something like a quarter of a century!) and never grown a
>>> "unique keys" version.
>>>
>>> I don't recall even seeing a request for such a feature. If such a
>>> unique keys version is useful, I don't expect it will be useful often.
>>>
>>
>> I have one argument in favor of such a feature: It preserves
>> concatenation semantics. + means one of two things in all code I've ever
>> seen (Python or otherwise):
>>
>> 1. Numeric addition (including element-wise numeric addition as in
>> Counter and numpy arrays)
>> 2. Concatenation (where the result preserves all elements, in order,
>> including, among other guarantees, that len(seq1) + len(seq2) == len(seq1 +
>> seq2))
>>
>> dict addition that didn't reject non-unique keys wouldn't fit *either*
>> pattern; the main proposal (making it equivalent to left.copy(), followed
>> by .update(right)) would have the left hand side would win on ordering, the
>> right hand side on values, and wouldn't preserve the length invariant of
>> concatenation. At least when repeated keys are rejected, most concatenation
>> invariants are preserved; order is all of the left elements followed by all
>> of the right, and no elements are lost.
>>
>
> I must by now have seen dozens of post complaining about this aspect of
> the proposal. I think this is just making up rules (e.g. "+ never loses
> information") to deal with an aspect of the design where a *choice* must be
> made. This may reflect the Zen of Python's "In the face of ambiguity,
> refuse the temptation to guess." But really, that's a pretty silly rule
> (truly, they aren't all winners). Good interface design constantly makes
> choices in ambiguous situations, because the alternative is constantly
> asking, and that's just annoying.
>
> We have a plethora of examples (in fact, almost all alternatives
> considered) of situations related to dict merging where a choice is made
> between conflicting values for a key, and it's always the value further to
> the right that wins: from d[k] = v (which overrides the value when k is
> already in the dict) to d1.update(d2) (which lets the values in d2 win),
> including the much lauded {**d1, **d2} and even plain {'a': 1, 'a': 2} has
> a well-defined meaning where the latter value wins.
>
> Yeah. And I'm fine with the behavior for update because the name itself is
descriptive; we're spelling out, in English, that we're update-ing the
thing it's called on, so it makes sense to have the thing we're sourcing
for updates take precedence.
Similarly, for dict literals (and by extension, unpacking), it's following
an existing Python convention which doesn't contradict anything else.
Overloading + lacks the clear descriptive aspect of update that describes
the goal of the operation, and contradicts conventions (in Python and
elsewhere) about how + works (addition or concatenation, and a lot of
people don't even like it doing the latter, though I'm not that pedantic).
A couple "rules" from C++ on overloading are "*Whenever the meaning of an
operator is not obviously clear and undisputed, it should not be
overloaded.* *Instead, provide a function with a well-chosen name.*"
and "*Always
stick to the operator’s well-known semantics".* (Source:
https://stackoverflow.com/a/4421708/364696 , though the principle is
restated in many other places). Obviously the C++ community isn't perfect
on this (see iostream and <> operators), but they're otherwise pretty
consistent. + means addition, and in many languages including C++ strings,
concatenation, but I don't know of any languages outside the "esoteric"
category that use it for things that are neither addition nor
concatenation. You've said you don't want the whole plethora of set-like
behaviors on dicts, but dicts are syntactically and semantically much more
like sets than sequences, and if you add + (with semantics differing from
both sets and sequences), the language becomes less consistent.
I'm not against making it easier to merge dictionaries. But people seem to
be arguing that {**d1, **d2} is bad because of magic punctuation that
obscures meaning, when IMO:
d3 = d1 + d2
is obscuring meaning by adding yet a third rule for what + means,
inconsistent with both existing rules (from both Python and the majority of
languages I've had cause to use). A named method (class or instance) or
top-level function (a la sorted) is more explicit, easier to look up (after
all, the
Re: [Python-ideas] PEP: Dict addition and subtraction
On Tue, Mar 5, 2019 at 3:50 PM Josh Rosenberg <
shadowranger+pythonid...@gmail.com> wrote:
>
> On Tue, Mar 5, 2019 at 11:16 PM Steven D'Aprano
> wrote:
>
>> On Sun, Mar 03, 2019 at 09:28:30PM -0500, James Lu wrote:
>>
>> > I propose that the + sign merge two python dictionaries such that if
>> > there are conflicting keys, a KeyError is thrown.
>>
>> This proposal is for a simple, operator-based equivalent to
>> dict.update() which returns a new dict. dict.update has existed since
>> Python 1.5 (something like a quarter of a century!) and never grown a
>> "unique keys" version.
>>
>> I don't recall even seeing a request for such a feature. If such a
>> unique keys version is useful, I don't expect it will be useful often.
>>
>
> I have one argument in favor of such a feature: It preserves concatenation
> semantics. + means one of two things in all code I've ever seen (Python or
> otherwise):
>
> 1. Numeric addition (including element-wise numeric addition as in Counter
> and numpy arrays)
> 2. Concatenation (where the result preserves all elements, in order,
> including, among other guarantees, that len(seq1) + len(seq2) == len(seq1 +
> seq2))
>
> dict addition that didn't reject non-unique keys wouldn't fit *either*
> pattern; the main proposal (making it equivalent to left.copy(), followed
> by .update(right)) would have the left hand side would win on ordering, the
> right hand side on values, and wouldn't preserve the length invariant of
> concatenation. At least when repeated keys are rejected, most concatenation
> invariants are preserved; order is all of the left elements followed by all
> of the right, and no elements are lost.
>
I must by now have seen dozens of post complaining about this aspect of the
proposal. I think this is just making up rules (e.g. "+ never loses
information") to deal with an aspect of the design where a *choice* must be
made. This may reflect the Zen of Python's "In the face of ambiguity,
refuse the temptation to guess." But really, that's a pretty silly rule
(truly, they aren't all winners). Good interface design constantly makes
choices in ambiguous situations, because the alternative is constantly
asking, and that's just annoying.
We have a plethora of examples (in fact, almost all alternatives
considered) of situations related to dict merging where a choice is made
between conflicting values for a key, and it's always the value further to
the right that wins: from d[k] = v (which overrides the value when k is
already in the dict) to d1.update(d2) (which lets the values in d2 win),
including the much lauded {**d1, **d2} and even plain {'a': 1, 'a': 2} has
a well-defined meaning where the latter value wins.
As to why raising is worse: First, none of the other situations I listed
above raises for conflicts. Second, there's the experience of str+unicode
in Python 2, which raises if the str argument contains any non-ASCII bytes.
In fact, we disliked it so much that we changed the language incompatibly
to deal with it.
--
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Tue, Mar 5, 2019 at 11:16 PM Steven D'Aprano wrote:
> On Sun, Mar 03, 2019 at 09:28:30PM -0500, James Lu wrote:
>
> > I propose that the + sign merge two python dictionaries such that if
> > there are conflicting keys, a KeyError is thrown.
>
> This proposal is for a simple, operator-based equivalent to
> dict.update() which returns a new dict. dict.update has existed since
> Python 1.5 (something like a quarter of a century!) and never grown a
> "unique keys" version.
>
> I don't recall even seeing a request for such a feature. If such a
> unique keys version is useful, I don't expect it will be useful often.
>
>
I have one argument in favor of such a feature: It preserves concatenation
semantics. + means one of two things in all code I've ever seen (Python or
otherwise):
1. Numeric addition (including element-wise numeric addition as in Counter
and numpy arrays)
2. Concatenation (where the result preserves all elements, in order,
including, among other guarantees, that len(seq1) + len(seq2) == len(seq1 +
seq2))
dict addition that didn't reject non-unique keys wouldn't fit *either*
pattern; the main proposal (making it equivalent to left.copy(), followed
by .update(right)) would have the left hand side would win on ordering, the
right hand side on values, and wouldn't preserve the length invariant of
concatenation. At least when repeated keys are rejected, most concatenation
invariants are preserved; order is all of the left elements followed by all
of the right, and no elements are lost.
>
> > This way, d1 + d2 isn’t just another obvious way to do {**d1, **d2}.
>
> One of the reasons for preferring + is that it is an obvious way to do
> something very common, while {**d1, **d2} is as far from obvious as you
> can get without becoming APL or Perl :-)
>
>
>From the moment PEP 448 published, I've been using unpacking as a more
composable/efficient form of concatenation, merging, etc. I'm sorry you
don't find it obvious, but a couple e-mails back you said:
"The Zen's prohibition against guessing in the face of ambiguity does not
mean that we must not add a feature to the language that requires the
user to learn what it does first."
Learning to use the unpacking syntax in the case of function calls is
necessary for tons of stuff (writing general function decorators, handling
initialization in class hierarchies, etc.), and as PEP 448 is titled, this
is just a generalization combining the features of unpacking arguments with
collection literals.
> The second syntax makes it clear that a new dictionary is being
> > constructed and that d2 overrides keys from d1.
>
> Only because you have learned the rule that {**d, **e) means to
> construct a new dict by merging, with the rule that in the event of
> duplicate keys, the last key seen wins. If you hadn't learned that rule,
> there is nothing in the syntax which would tell you the behaviour. We
> could have chosen any rule we liked:
>
>
No, because we learned the general rule for dict literals that {'a': 1,
'a': 2} produces {'a': 2}; the unpacking generalizations were very good
about adhering to the existing rules, so it was basically zero learning
curve if you already knew dict literal rules and less general unpacking
rules. The only part to "learn" is that when there is a conflict between
dict literal rules and function call rules, dict literal rules win.
To be clear: I'm not supporting + as raising error on non-unique keys. Even
if it makes dict + dict adhere to the rules of concatenation, I don't think
it's a common or useful functionality. My order of preferences is roughly:
1. Do nothing (even if you don't like {**d1, **d2}, .copy() followed by
.update() is obvious, and we don't need more than one way to do it)
2. Add a new method to dict, e.g. dict.merge (whether it's a class method
or an instance method is irrelevant to me)
3. Use | (because dicts are *far* more like sets than they are like
sequences, and the semi-lossy rules of unioning make more sense there); it
would also make - make sense, since + is only matched by - in numeric
contexts; on collections, | and - are paired. And I consider the -
functionality the most useful part of this whole proposal (because I *have*
wanted to drop a collection of known blacklisted keys from a dict and while
it's obvious you can do it by looping, I always wanted to be able to do
something like d1.keys() -= badkeys, and remain disappointed nothing like
it is available)
-Josh Rosenberg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Mon, Mar 04, 2019 at 08:01:38PM -0500, James Lu wrote:
>
> > On Mar 4, 2019, at 11:25 AM, Steven D'Aprano wrote:
> > Another example would be when reading command line options, where the
> > most common convention is for "last option seen" to win:
> >
> > [steve@ando Lib]$ grep --color=always --color=never "zero" f*.py
[...]
> Indeed, in this case you would want to use {**, **} syntax.
No I would NOT want to use the {**, **} syntax, because it is ugly.
That's why people ask for + instead. (Or perhaps I should say "as well
as" since the double-star syntax is not going away.)
[...]
> > Unless someone can demonstrate that the design of dict.update() was a
> > mistake
>
> You’re making a logical mistake here. + isn’t supposed to have
> .update’s behavior and it never was supposed to.
James, I'm the author of the PEP, and for the purposes of the proposal,
the + operator is supposed to do what I say it is supposed to do.
You might be able to persuade me to change the PEP, if you have a
sufficiently good argument, or you can write your own counter PEP making
a different choice, but please don't tell me what I intended. I know
what I intended, and it is for + to have the same last-key-wins
behaviour as update. That's the behaviour which is most commonly
requested in the various times this comes up.
> > , and the "require unique keys" behaviour is more common,
>
> I just have.
No you haven't -- you have simply *declared* that it is more common,
without giving any evidence for it.
> 99% of the time you want to have keys from one dict override another,
> you’d be better off doing it in-place and so would be using .update()
> anyways.
I don't know if it is "99% of the time" or 50% of the time or 5%,
but this PEP is for the remaining times where we don't want in-place
updates but we want a new dict.
I use list.append or list.extend more often than list concatenation, but
when I want a new list, list concatenation is very useful. This proposal
is about those cases where we want a new dict.
--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Sun, Mar 03, 2019 at 09:28:30PM -0500, James Lu wrote:
> I propose that the + sign merge two python dictionaries such that if
> there are conflicting keys, a KeyError is thrown.
This proposal is for a simple, operator-based equivalent to
dict.update() which returns a new dict. dict.update has existed since
Python 1.5 (something like a quarter of a century!) and never grown a
"unique keys" version.
I don't recall even seeing a request for such a feature. If such a
unique keys version is useful, I don't expect it will be useful often.
> This way, d1 + d2 isn’t just another obvious way to do {**d1, **d2}.
One of the reasons for preferring + is that it is an obvious way to do
something very common, while {**d1, **d2} is as far from obvious as you
can get without becoming APL or Perl :-)
If I needed such a unique key version of update, I'd use a subclass:
class StrictDict(dict):
def __add__(self, other):
if isinstance(other, dict) and (self.keys() & other.keys()):
raise KeyError('non-unique keys')
return super().__add__(self, other)
# and similar for __radd__.
rather than burden the entire language, and every user of it, with
having to learn the subtle difference between the obvious + operator and
the error-prone and unobvious trick of {*d1, *d2}.
( Did you see what I did there? *wink* )
> The second syntax makes it clear that a new dictionary is being
> constructed and that d2 overrides keys from d1.
Only because you have learned the rule that {**d, **e) means to
construct a new dict by merging, with the rule that in the event of
duplicate keys, the last key seen wins. If you hadn't learned that rule,
there is nothing in the syntax which would tell you the behaviour. We
could have chosen any rule we liked:
- raise an exception, like you get a TypeError if you pass the
same keyword argument to a function twice: spam(foo=1, foo=2);
- first value seen wins;
- last value seen wins;
- random value wins;
- anything else we liked!
There is nothing "clear" about the syntax which makes it obvious which
behaviour is implemented. We have to learn it.
> One can reasonably expect or imagine a situation where a section of
> code that expects to merge two dictionaries with non-conflicting keys
> commits a semantic error if it merges two dictionaries with
> conflicting keys.
I can imagine it, but I don't think I've ever needed it, and I can't
imagine wanting it often enough to wish it was not just a built-in
function or method, but actual syntax.
Do you have some real examples of wanting an error when trying to update
a dict if keys match?
> To better explain, imagine a program where options is a global
> variable storing parsed values from the command line.
>
> def verbose_options():
> if options.quiet
> return {'verbose': True}
>
> def quiet_options():
> if options.quiet:
> return {'verbose': False}
That seems very artifical to me. Why not use a single function:
def verbose_options(): # There's more than one?
return {'verbose': not options.quiet}
The way you have written those functions seems weird to me. You already
have a nice options object, with named fields like "options.quiet", why
are you turning it into not one but *two* different dicts, both
reporting the same field?
And its buggy: if options.quiet is True, then the key 'quiet'
should be True, not the 'verbose' key.
Do you have *two* functions for every preference setting that takes a
true/false flag?
What do you do for preference settings that take multiple values? Create
a vast number of specialised functions, one for each possible value?
def A4_page_options():
if options.page_size == 'A4':
return {'page_size': 'A4'}
def US_Letter_page_options():
if options.page_size == 'US Letter':
return {'page_size': 'US Letter'}
page_size = (
A4_page_options() + A3_page_options() + A5_page_options()
+ Foolscape_page_options + Tabloid_page_options()
+ US_Letter_page_options() + US_Legal_page_options()
# and about a dozen more...
)
The point is, although I might be wrong, I don't think that this example
is a practical, realistic use-case for a unique keys version of update.
To me, your approach seems so complicated and artificial that it seems
like it was invented specifically to justify this "unique key" operator,
not something that we would want to write in real life.
But even if it real code, the question is not whether it is EVER useful
for a dict update to raise an exception on matching keys. The question
is whether this is so often useful that this is the behaviour we want to
make the default for dicts.
[...]
> Again, I propose that the + sign merge two python dictionaries such
> that if there are conflicting keys, a KeyError is thrown, because such
> “non-conflicting merge” behavior would be useful in Python.
I don't think it would be, at least not
Re: [Python-ideas] PEP: Dict addition and subtraction
Actually, this was made even more condition-y in 3.8. Now we check __iter__ too: D.update([E, ]**F) -> None. Update D from dict/iterable E and F. If E is present, has a .keys() method, is a subclass of dict, and hasn't overridden __iter__, then does: for k in E: D[k] = E[k] If E is present, has a .keys() method, and is not a subclass of dict or has overridden __iter__, then does: for k in E.keys(): D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k] Bleh. On Tue, Mar 5, 2019 at 2:54 PM Brandt Bucher wrote: > Should our __sub__ behavior be the same... > > > Sorry, our "__isub__" behavior. Long day... > > On Tue, Mar 5, 2019 at 2:47 PM Brandt Bucher > wrote: > >> These semantics are intended to match those of update as closely as >>> possible. For the dict built-in itself, calling keys is redundant as >>> iteration over a dict iterates over its keys; but for subclasses or other >>> mappings, update prefers to use the keys method. >>> >>> The above paragraph may be inaccurate. Although the dict docstring >>> states that keys will be called if it exists, this does not seem to be >>> the case for dict subclasses. Bug or feature? >>> >> >> >>> print(dict.update.__doc__) >> D.update([E, ]**F) -> None. Update D from dict/iterable E and F. >> If E is present and has a .keys() method, then does: for k in E: D[k] = >> E[k] >> If E is present and lacks a .keys() method, then does: for k, v in E: >> D[k] = v >> In either case, this is followed by: for k in F: D[k] = F[k] >> >> It's actually pretty interesting... and misleading/wrongish. It never >> says that keys is *called*... in reality, it just checks for the "keys" >> method before deciding whether to proceed with PyDict_Merge or PyDict >> _MergeFromSeq2. It should really read more like: >> >> D.update([E, ]**F) -> None. Update D from dict/iterable E and F. >> If E is present, has a .keys() method, and is a subclass of dict, then >> does: for k in E: D[k] = E[k] >> If E is present, has a .keys() method, and is not a subclass of dict, >> then does: for k in E.keys(): D[k] = E[k] >> If E is present and lacks a .keys() method, then does: for k, v in E: >> D[k] = v >> In either case, this is followed by: for k in F: D[k] = F[k] >> >> Should our __sub__ behavior be the same (i.e., iterate for dict >> subclasses and objects without "keys()", otherwise call "keys()" and >> iterate over that)? __iadd__ calls into this logic already. It seems to be >> the most "natural" solution here, if we desire behavior analogous to >> "update". >> >> Brandt >> >> On Fri, Mar 1, 2019 at 8:26 AM Steven D'Aprano >> wrote: >> >>> Attached is a draft PEP on adding + and - operators to dict for >>> discussion. >>> >>> This should probably go here: >>> >>> https://github.com/python/peps >>> >>> but due to technical difficulties at my end, I'm very limited in what I >>> can do on Github (at least for now). If there's anyone who would like to >>> co-author and/or help with the process, that will be appreciated. >>> >>> >>> -- >>> Steven >>> ___ >>> Python-ideas mailing list >>> Python-ideas@python.org >>> https://mail.python.org/mailman/listinfo/python-ideas >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
> > Should our __sub__ behavior be the same... Sorry, our "__isub__" behavior. Long day... On Tue, Mar 5, 2019 at 2:47 PM Brandt Bucher wrote: > These semantics are intended to match those of update as closely as >> possible. For the dict built-in itself, calling keys is redundant as >> iteration over a dict iterates over its keys; but for subclasses or other >> mappings, update prefers to use the keys method. >> >> The above paragraph may be inaccurate. Although the dict docstring states >> that keys will be called if it exists, this does not seem to be the case >> for dict subclasses. Bug or feature? >> > > >>> print(dict.update.__doc__) > D.update([E, ]**F) -> None. Update D from dict/iterable E and F. > If E is present and has a .keys() method, then does: for k in E: D[k] = > E[k] > If E is present and lacks a .keys() method, then does: for k, v in E: > D[k] = v > In either case, this is followed by: for k in F: D[k] = F[k] > > It's actually pretty interesting... and misleading/wrongish. It never says > that keys is *called*... in reality, it just checks for the "keys" method > before deciding whether to proceed with PyDict_Merge or PyDict > _MergeFromSeq2. It should really read more like: > > D.update([E, ]**F) -> None. Update D from dict/iterable E and F. > If E is present, has a .keys() method, and is a subclass of dict, then > does: for k in E: D[k] = E[k] > If E is present, has a .keys() method, and is not a subclass of dict, then > does: for k in E.keys(): D[k] = E[k] > If E is present and lacks a .keys() method, then does: for k, v in E: > D[k] = v > In either case, this is followed by: for k in F: D[k] = F[k] > > Should our __sub__ behavior be the same (i.e., iterate for dict subclasses > and objects without "keys()", otherwise call "keys()" and iterate over > that)? __iadd__ calls into this logic already. It seems to be the most > "natural" solution here, if we desire behavior analogous to "update". > > Brandt > > On Fri, Mar 1, 2019 at 8:26 AM Steven D'Aprano > wrote: > >> Attached is a draft PEP on adding + and - operators to dict for >> discussion. >> >> This should probably go here: >> >> https://github.com/python/peps >> >> but due to technical difficulties at my end, I'm very limited in what I >> can do on Github (at least for now). If there's anyone who would like to >> co-author and/or help with the process, that will be appreciated. >> >> >> -- >> Steven >> ___ >> Python-ideas mailing list >> Python-ideas@python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Suggestions: dict.flow_update and dict.__add__
Christopher Barker wrote: That violates an important convention in Python: mutating methods do not return self. We really want to preserve that convention. Smalltalk has an abbreviated way of writing a series of method calls to the same object: x doThis; doThatWith: y; doTheOther. is equivalent to x doThis. x doThatWith: y. x doTheOther. Something like this could no doubt be added to Python, but I'm not sure it would be worth the bother. Giving a short name to the recipient and then writing the calls out explicitly isn't much harder and is clearer to read, IMO. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
> > These semantics are intended to match those of update as closely as > possible. For the dict built-in itself, calling keys is redundant as > iteration over a dict iterates over its keys; but for subclasses or other > mappings, update prefers to use the keys method. > > The above paragraph may be inaccurate. Although the dict docstring states > that keys will be called if it exists, this does not seem to be the case > for dict subclasses. Bug or feature? > >>> print(dict.update.__doc__) D.update([E, ]**F) -> None. Update D from dict/iterable E and F. If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k] It's actually pretty interesting... and misleading/wrongish. It never says that keys is *called*... in reality, it just checks for the "keys" method before deciding whether to proceed with PyDict_Merge or PyDict_MergeFromSeq2. It should really read more like: D.update([E, ]**F) -> None. Update D from dict/iterable E and F. If E is present, has a .keys() method, and is a subclass of dict, then does: for k in E: D[k] = E[k] If E is present, has a .keys() method, and is not a subclass of dict, then does: for k in E.keys(): D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k] Should our __sub__ behavior be the same (i.e., iterate for dict subclasses and objects without "keys()", otherwise call "keys()" and iterate over that)? __iadd__ calls into this logic already. It seems to be the most "natural" solution here, if we desire behavior analogous to "update". Brandt On Fri, Mar 1, 2019 at 8:26 AM Steven D'Aprano wrote: > Attached is a draft PEP on adding + and - operators to dict for > discussion. > > This should probably go here: > > https://github.com/python/peps > > but due to technical difficulties at my end, I'm very limited in what I > can do on Github (at least for now). If there's anyone who would like to > co-author and/or help with the process, that will be appreciated. > > > -- > Steven > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
Steven D'Aprano wrote: The question is, is [recursive merge] behaviour useful enough and > common enough to be built into dict itself? I think not. It seems like just one possible way of merging values out of many. I think it would be better to provide a merge function or method that lets you specify a function for merging values. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
Rhodri James wrote: I have to go and look in the documentation because I expect the union operator to be '+'. Anyone raised on Pascal is likely to find + and * more natural. Pascal doesn't have bitwise operators, so it re-uses + and * for set operations. I like the economy of this arrangement -- it's not as if there's any other obvious meaning that + and * could have for sets. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
There was a thread for the topic. https://mail.python.org/pipermail/python-dev/2018-October/155435.html 2019年3月6日(水) 2:02 Anders Hovmöller : > > On a related note: **kwargs, should they support arbitrary strings as > keys? I depend on this behavior in production code and all python > implementations handle it. > > / Anders ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Tue, Mar 5, 2019 at 9:02 AM Anders Hovmöller wrote: > On a related note: **kwargs, should they support arbitrary strings as > keys? I depend on this behavior in production code and all python > implementations handle it. > The ice is much thinner there, but my position is that as long as they are *strings* such keys should be allowed. -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Suggestions: dict.flow_update and dict.__add__
I thank Guido and Christopher for their thoughtful comments. You certainly found some weak points. I chose the name 'flow' to match: https://en.wikipedia.org/wiki/Fluent_interface#Python Instead of my previous arg = defaults.copy().flow_update(options) one could instead from somewhere import flow, and then write arg = flow(defaults.copy()).update(options) This avoids a profusion of flow_ methods, and also the need to subclass dict. Once could of course use a different name. Perhaps 'follow' would be better. And it would work 'out the box' in other situations. Christopher might prefer the flow(obj).update approach, as it respects the convention "mutating methods do not return self." (Thank you for your clear statement, Christopher.) (Aside: For the non-experts, a word if I may about the implementation. The key point is that in Python the programmer 'owns the dot' and so the desired semantics can be implemented. We use a custom __getattribute__ .) Finally, please forgive my fictional use case. I think studying real-world use cases for dict + dict would be very helpful to the discussion. I don't recall seeing any, but I haven't looked hard. Instructive use cases should, of course, be placed in the PEP. -- Jonathan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
> I'd like to remove all doubt: {**d1} needs to work regardless of the key
> type, as long as it's hashable (d1 could be some mapping implemented without
> hashing, e.g. using a balanced tree, so that it could support unhashable
> keys).
>
> If there's doubt about this anywhere, we could add an example to the docs and
> to the PEP.
On a related note: **kwargs, should they support arbitrary strings as keys? I
depend on this behavior in production code and all python implementations
handle it.
/ Anders
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Tue, Mar 5, 2019 at 2:38 AM Inada Naoki wrote:
> On Tue, Mar 5, 2019 at 7:26 PM Steven D'Aprano
> wrote:
> >
> > On Sat, Mar 02, 2019 at 01:47:37AM +0900, INADA Naoki wrote:
> > > > If the keys are not strings, it currently works in CPython, but it
> may not work with other implementations, or future versions of CPython[2].
> > >
> > > I don't think so. https://bugs.python.org/issue35105 and
> > > https://mail.python.org/pipermail/python-dev/2018-October/155435.html
> > > are about kwargs. I think non string keys are allowed for {**d1,
> > > **d2} by language.
> >
> > Is this documented somewhere?
>
> It is not explicitly documented. But unlike keyword argument,
> dict display supported non-string keys from very old.
>
> I believe {3: 4} is supported by Python language, not CPython
> implementation behavior.
>
>
> https://docs.python.org/3/reference/expressions.html#grammar-token-dict-display
>
I'd like to remove all doubt: {**d1} needs to work regardless of the key
type, as long as it's hashable (d1 could be some mapping implemented
without hashing, e.g. using a balanced tree, so that it could support
unhashable keys).
If there's doubt about this anywhere, we could add an example to the docs
and to the PEP.
--
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Suggestions: dict.flow_update and dict.__add__
On Tue, Mar 5, 2019 at 12:53 AM Jonathan Fine wrote: > SUMMARY > Instead of using dict + dict, perhaps use dict.flow_update. Here, > flow_update is just like update, except that it returns self. That violates an important convention in Python: mutating methods do not return self. We really want to preserve that convention. On the other hand, as seen in other recent threads, there is a desire for chaining operations of many sorts, so a .flow_update() that returned a new dict would provide that feature. Though I would only recommend that if it was decided that we wanted to generally support that approach for all mutable containers — which would mean adding quite a few methods. And we could then use the same naming convention for them all. I’m not sure I like “flow_” though, it’s not very commonly known jargon. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Tue, Mar 05, 2019 at 08:11:29AM -0500, David Shawley wrote:
> "Putting Metaclasses to Work" (ISBN-13 978-0201433050) presents a more
> mathematical view of programming language types that includes two
> distinct operations for combining dictionaries -- merge and recursive
> merge.
>
> For two input dictionaries D1 & D2 and the output dictionary O
>
> D1 merge D2
> O is D1 with the of those keys of D2 that do not have keys in D1
>
> D1 recursive-merge D2
> For all keys k, O[k] = D1[k] recursive merge D2[k] if both D1[k]
> and D2[k] are dictionaries, otherwise O[k] = (D1 merge D2)[k].
I'm afraid I cannot understand either of those algorithms as written. I
suspect that you've left at least one word out of the first.
Fortunately your example below is extremely clear, thank you.
[...]
> The following example uses dictionaries from "Putting
> Metaclasses to Work":
>
> >>> d1 = {
> ... 'title': 'Structured Programming',
> ... 'authors': 'Dahl, Dijkstra, and Hoare',
> ... 'locations': {
> ... 'Dahl': 'University of Oslo',
> ... 'Dijkstra': 'University of Texas',
> ... 'Hoare': 'Oxford University',
> ... },
> ... }
> >>>
> >>> d2 = {
> ... 'publisher': 'Academic Press',
> ... 'locations': {
> ... 'North America': 'New York',
> ... 'Europe': 'London',
> ... },
> ... }
> >>>
> >>> o = d1.copy()
> >>> o.update(d2)
> >>> o
> {'publisher': 'Academic Press',
> 'title': 'Structured Programming',
> 'locations': {'North America': 'New York', 'Europe': 'London'},
> 'authors': 'Dahl, Dijkstra, and Hoare'}
Yes, that's the classic "update with last seen wins". That's what the
PEP proposes as that seems to be the most frequently requested
behaviour. It is also the only behaviour which has been deemed useful
enough in nearly 30 years of Python's history to be added to dict as a
method.
> >>> merge(d1, d2)
> {'publisher': 'Academic Press',
> 'title': 'Structured Programming',
> 'locations': {'Dijkstra': 'University of Texas',
>'Hoare': 'Oxford University',
>'Dahl': 'University of Oslo'},
> 'authors': 'Dahl, Dijkstra, and Hoare'}
That seems to be "update with first seen wins", which is easily done
using ChainMap or the proposed dict difference operator:
dict( ChainMap(d1, d2) )
# or
d1 + (d2 - d1)
or simply by swapping the order of the operands:
d2 + d1
(These are not *identical* in effect, there are small differences with
respect to key:value identity, and order of keys. But they ought to give
*equal* results.)
Personally, I don't think that behaviour is as useful as the first, but
it is certainly a legitimate kind of merge.
As far as I know, this has never been requested before. Perhaps it is
too niche?
> >>> recursive_merge(d1, d2)
> {'publisher': 'Academic Press',
> 'title': 'Structured Programming',
> 'locations': {'North America': 'New York',
>'Europe': 'London',
>'Dijkstra': 'University of Texas',
>'Hoare': 'Oxford University',
>'Dahl': 'University of Oslo'},
> 'authors': 'Dahl, Dijkstra, and Hoare'}
That's an interesting one. I'd write it something like this:
def merge(a, b):
new = a.copy()
for key, value in b:
if key not in a:
# Add new keys.
new[key] = value
else:
v = new[key]
if isinstance(value, dict) and isinstance(v, dict):
# If both values are dicts, merge them.
new[key] = merge(v, value)
else:
# What to do if only one is a dict?
# Or if neither is a dict?
return new
I've seen variants of this where duplicate keys are handled by building
a list of the values:
def merge(a, b):
new = a.copy()
for key, value in b:
if key in a:
v = new[key]
if isinstance(v, list):
v.append(value)
else:
new[key] = [v, value]
...
or by concatenating values, or adding them (as Counter does), etc. We
have subclasses and operator overloading, so you can implement whatever
behaviour you like.
The question is, is this behaviour useful enough and common enough to be
built into dict itself?
> IMO, having more than one obvious outcome means that we should refuse
> the temptation to guess.
We're not *guessing*. We're *chosing* which behaviour we want.
Nobody says:
When I print some strings, I can seperate them with spaces, or
dots, or newlines, and print a newline at the end, or suppress
the newline. Since all of these behaviours might be useful for
somebody, we should not "guess" what the user wants. Therefore
we should not have a print() function at all.
The behaviour of print() is not a guess as to what the user wants. We
offer a specific behaviour, and if the user is happy with that, then
they can use print(), and if not, they can
Re: [Python-ideas] Suggestions: dict.flow_update and dict.__add__
If you have to tell such a long and convoluted story to explain a name that you've picked out of the blue and that has no equivalent in other Python data types, it's probably a bad idea. If you're proposing that other mutating methods also gain a flow_XXX variant, please, no! That's like the theory of supersymmetry (SUSY) in particle physics, where ever known particle from the Standard Model would have to have a much heavier "superpartner" just to make some esoteric idea work. On Tue, Mar 5, 2019 at 12:54 AM Jonathan Fine wrote: > SUMMARY > Instead of using dict + dict, perhaps use dict.flow_update. Here, > flow_update is just like update, except that it returns self. > > BACKGROUND > There's a difference between a sorted copy of a list, and sorting the > list in place. > > >>> items = [2, 0, 1, 9] > >>> sorted(items), items > ([0, 1, 2, 9], [2, 0, 1, 9]) > >>> items.sort(), items >(None, [0, 1, 2, 9]) > > In Python, mutating methods generally return None. Here, this prevents > beginners thinking their code has produced a sorted copy of a list, > when in fact it has done an in-place sort on the list. If they write > >>> aaa = my_list.sort() > they'll get a None error when they use aaa. > > The same goes for dict.update. This is a useful feature, particularly > for beginners. It helps them think clearly, and express themselves > clearly. > > THE PROBLEM > This returning None can be a nuisance, sometimes. Suppose we have a > dictionary of default values, and a dictionary of use supplied > options. We wish to combine the two dictionaries, say into a new > combined dictionary. > > One way to do this is: > >combined = defaults.copy() >combined.update(options) > > But this is awkward when you're in the middle of calling a function: > > call_big_method( > # lots of arguments, one to a line, with comments > arg = combined, # Look up to see what combined is. > # more arguments > ) > > USING + > There's a suggestion, that instead one extends Python so that this works: > arg = defaults + options # What does '+' mean here? > > USING flow_update > Here's another suggestion. Instead write: > dict_arg = defaults.copy().flow_update(options) # Is this clearer? > > IMPLEMENTATION > Here's an implementation, as a subclass of dict. > > class mydict(dict): > > def flow_update(self, *argv, **kwargs): > self.update(*argv, **kwargs) > return self > > def copy(self): > return self.__class__(self) > > A DIRTY HACK > Not tested, using an assignment expression. >dict_arg = (tmp := defaults.copy(), tmp.update(options))[0] > Not recommend. > > -- > Jonathan > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add a "week" function or attribute to datetime.date
On Mon, Mar 4, 2019 at 10:00 PM Steve Barnes wrote: > If anybody is looking for such components then wx.DateTime > There has got to be a stand alone python library for that! Anyone know the status of the venerable mxDateTime? -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
> Does anyone have an example of another programming language that
> allows for addition of dictionaries/mappings?
>
kotlin does that (`to` means `:`) :
fun main() {
var a = mutableMapOf("a" to 1, "b" to 2)
var b = mutableMapOf("c" to 1, "b" to 3)
println(a)
println(b)
println(a + b)
println(b + a)
}
{a=1, b=2}
{c=1, b=3}
{a=1, b=3, c=1}
{c=1, b=2, a=1}
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Mar 4, 2019, at 4:51 AM, Stefan Behnel wrote:
>
> I think the main intentions is to close a gap in the language.
>
>[1,2,3] + [4,5,6]
>
> works for lists and tuples,
>
>{1,2,3} | {4,5,6}
>
> works for sets, but joining two dicts isn't simply
>
>{1:2, 3:4} + {5:6}
>
> but requires either some obscure syntax or a statement instead of a simple
> expression.
>
> The proposal is to enable the obvious syntax for something that should be
> obvious.
I would challenge that this dictionary merging is something that is
obvious. The existing sequences are simple collections of values
where a dictionary is a mapping of values. The difference between
the two is akin to the difference between a mathematical array or
set and a unary mapping function. There is a clear and obvious way to
combine arrays and sets -- concatenation for arrays and union for sets.
Combining mapping functions is less than obvious.
"Putting Metaclasses to Work" (ISBN-13 978-0201433050) presents a more
mathematical view of programming language types that includes two
distinct operations for combining dictionaries -- merge and recursive
merge.
For two input dictionaries D1 & D2 and the output dictionary O
D1 merge D2
O is D1 with the of those keys of D2 that do not have keys in D1
D1 recursive-merge D2
For all keys k, O[k] = D1[k] recursive merge D2[k] if both D1[k]
and D2[k] are dictionaries, otherwise O[k] = (D1 merge D2)[k].
Note that neither of the cases is the same as:
>>> O = D1.copy()
>>> O.update(D2)
So that gives us three different ways to combine dictionaries that are
each sensible. The following example uses dictionaries from "Putting
Metaclasses to Work":
>>> d1 = {
... 'title': 'Structured Programming',
... 'authors': 'Dahl, Dijkstra, and Hoare',
... 'locations': {
... 'Dahl': 'University of Oslo',
... 'Dijkstra': 'University of Texas',
... 'Hoare': 'Oxford University',
... },
... }
>>>
>>> d2 = {
... 'publisher': 'Academic Press',
... 'locations': {
... 'North America': 'New York',
... 'Europe': 'London',
... },
... }
>>>
>>> o = d1.copy()
>>> o.update(d2)
>>> o
{'publisher': 'Academic Press',
'title': 'Structured Programming',
'locations': {'North America': 'New York', 'Europe': 'London'},
'authors': 'Dahl, Dijkstra, and Hoare'}
>>>
>>> merge(d1, d2)
{'publisher': 'Academic Press',
'title': 'Structured Programming',
'locations': {'Dijkstra': 'University of Texas',
'Hoare': 'Oxford University',
'Dahl': 'University of Oslo'},
'authors': 'Dahl, Dijkstra, and Hoare'}
>>>
>>> recursive_merge(d1, d2)
{'publisher': 'Academic Press',
'title': 'Structured Programming',
'locations': {'North America': 'New York',
'Europe': 'London',
'Dijkstra': 'University of Texas',
'Hoare': 'Oxford University',
'Dahl': 'University of Oslo'},
'authors': 'Dahl, Dijkstra, and Hoare'}
>>>
https://repl.it/@dave_shawley/PuttingMetaclassesToWork
IMO, having more than one obvious outcome means that we should refuse
the temptation to guess. If we do, then the result is only obvious
to a subset of users and will be a surprise to the others.
It's also useful to note that I am having trouble coming up with
another programming language that supports a "+" operator for map types.
Does anyone have an example of another programming language that
allows for addition of dictionaries/mappings?
If so, what is the behavior there?
- dave
--
Any linter or project that treats PEP 8 as mandatory has *already*
failed, as PEP 8 itself states that the rules can be broken as needed. - Paul
Moore.___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
On 05/03/2019 09:42, Jimmy Girardet wrote: Indeed the "obscure" argument should be thrown away. The `|` operator in sets seems to be evident for every one on this list but I would be curious to know how many people first got a TypeError doing set1 + set2 and then found set1 | set2 in the doc. Every. Single. Time. I don't use sets a lot (purely by happenstance rather than choice), and every time I do I have to go and look in the documentation because I expect the union operator to be '+'. Except for math geek the `|` is always something obscure. Two thirds of my degree is in maths, and '|' is still something I don't associate with sets. It would be unreasonable to expect '∩' and '∪' as the operators, but reasoning from '-' for set difference I always expect '+' and '*' as the union and intersection operators. Alas my hopes are always cruelly crushed :-) -- Rhodri James *-* Kynesim Ltd ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
I just wanted to mention this since it hasn't been brought up, but neither of these work a.keys() + b.keys() a.values() + b.values() a.items() + b.items() However, the following do work: a.keys() | b.keys() a.items() | b.items() Perhaps they work by coincidence (being set types), but I think it's worth bringing up, since a naive/natural Python implementation of dict addition/union would possibly involve the |-operator. Pål ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Tue, Mar 5, 2019 at 7:59 PM Steven D'Aprano wrote: > > On Tue, Mar 05, 2019 at 06:04:40PM +0900, INADA Naoki wrote: > [...] > > One obvious merit of d.merge(...) is it returns same type of d. > > `type(d1)(d1, d2)` looks ugly. > > > > But people just want dict instead of some subtype of dict. > > This merit is not so important. > > Not to me! It *is* important to me. I'm sorry, I missed "most". > > I want builtins to honour their subclasses. It is probably too late to > change existing behaviour, but my proposal specifies that subclasses are > honoured. > Then my proposal `d1.merge(d2)` is much better than alternative dict(d1, d2) for you. -- Inada Naoki ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Tue, Mar 05, 2019 at 06:04:40PM +0900, INADA Naoki wrote: [...] > One obvious merit of d.merge(...) is it returns same type of d. > `type(d1)(d1, d2)` looks ugly. > > But people just want dict instead of some subtype of dict. > This merit is not so important. Not to me! It *is* important to me. I want builtins to honour their subclasses. It is probably too late to change existing behaviour, but my proposal specifies that subclasses are honoured. -- Steven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
04.03.19 15:29, Serhiy Storchaka пише: Using "|" looks more natural to me than using "+". We should look at discussions for using the "|" operator for sets, if the alternative of using "+" was considered, I think the same arguments for preferring "|" for sets are applicable now for dicts. See the Python-Dev thread with the subject "Re: Re: PEP 218 (sets); moving set.py to Lib" starting from https://mail.python.org/pipermail/python-dev/2002-August/028104.html ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
Python C API has PyDict_Merge (https://docs.python.org/3/c-api/dict.html#c.PyDict_Merge) function which has different behavior than the proposed Python level method (doesn't copy but merge in-place). This is a red flag for me. On Tue, Mar 5, 2019 at 12:24 PM fhsxfhsx wrote: > > I agree so much on your opinion that I was just to create a topic about this > if you didn't. > I also propose here a small modification to make it more general which adds > an argument `how` (name to be discussed), telling how to merge the dicts, as > many have pointed out that there could be different ways to merge dicts. > So things would be like > > def addition_merge(key, values, exists): > """ > :param key: the key to merge > :param values: values of dicts to merge indexed at `key` > :param exists: whether each dict contains `key` > """ > if any(exists): > return True, sum([value for exist, value in zip(exists, values) > if exist]) > else: > return False > d1.merge(d2, d3, ..., how=addition_merge) > > We could even have > > def discard(key, values, exists): > return not any(exists[1:]), values[0] > d1.merge(d2, how=discard) > > which does the same thing as proposed `d1-d2`. > > This would make things like > d = d1.merge(iter_of_pairs) > d = d1.merge(key=value) > not working, but people could easily wrap a `dict()` over the iterator or > key-value stuff and attach no complication. > > > At 2019-03-05 15:39:40, "INADA Naoki" wrote: > >I think some people in favor of PEP 584 just want > >single expression for merging dicts without in-place update. > > > >But I feel it's abuse of operator overload. I think functions > >and methods are better than operator unless the operator > >has good math metaphor, or very frequently used as concatenate > >strings. > > > >This is why function and methods are better: > > > >* Easy to search. > >* Name can describe it's behavior better than abused operator. > >* Simpler lookup behavior. (e.g. subclass and __iadd__) > > > >Then, I propose `dict.merge` method. It is outer-place version > >of `dict.update`, but accepts multiple dicts. (dict.update() > >can be updated to accept multiple dicts, but it's not out of scope). > > > >* d = d1.merge(d2) # d = d1.copy(); d.update(d2) > >* d = d1.merge(d2, d3) # d = d1.copy(); d.update(d2); d2.update(d3) > >* d = d1.merge(iter_of_pairs) > >* d = d1.merge(key=value) > > > > > >## Merits of dict.merge() over operator + > > > >* Easy to Google (e.g. "python dict merge"). > >* Easy to help(dict.merge). (or dict.merge? in IPython) > >* No inefficiency of d1+d2+d3+...+dN, or sum(list_of_many_dicts) > >* Type of returned value is always same to d1.copy(). No issubclass, > >no __iadd__. > > > >## Why not dict.updated()? > > > >sorted() is a function so it looks different from L.sort() > >But d.updated() is very similar to d.update() for human eyes. > > > >## How about d1 - d2? > > > >If it is really useful, it can be implemented as method too. > > > >dict.discard(sequence_of_keys) > > > >Regards, > >-- > >INADA Naoki > >___ > >Python-ideas mailing list > >Python-ideas@python.org > >https://mail.python.org/mailman/listinfo/python-ideas > >Code of Conduct: http://python.org/psf/codeofconduct/ > > > > > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- Thanks, Andrew Svetlov ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Tue, Mar 5, 2019 at 7:26 PM Steven D'Aprano wrote:
>
> On Sat, Mar 02, 2019 at 01:47:37AM +0900, INADA Naoki wrote:
> > > If the keys are not strings, it currently works in CPython, but it may
> > > not work with other implementations, or future versions of CPython[2].
> >
> > I don't think so. https://bugs.python.org/issue35105 and
> > https://mail.python.org/pipermail/python-dev/2018-October/155435.html
> > are about kwargs. I think non string keys are allowed for {**d1,
> > **d2} by language.
>
> Is this documented somewhere?
>
It is not explicitly documented. But unlike keyword argument,
dict display supported non-string keys from very old.
I believe {3: 4} is supported by Python language, not CPython
implementation behavior.
https://docs.python.org/3/reference/expressions.html#grammar-token-dict-display
> Or is there a pronouncement somewhere that it is definitely expected to
> work in any language calling itself Python?
>
>
> Thanks,
>
>
>
> --
> Steven
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
--
Inada Naoki
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
On Sat, Mar 02, 2019 at 01:47:37AM +0900, INADA Naoki wrote:
> > If the keys are not strings, it currently works in CPython, but it may not
> > work with other implementations, or future versions of CPython[2].
>
> I don't think so. https://bugs.python.org/issue35105 and
> https://mail.python.org/pipermail/python-dev/2018-October/155435.html
> are about kwargs. I think non string keys are allowed for {**d1,
> **d2} by language.
Is this documented somewhere?
Or is there a pronouncement somewhere that it is definitely expected to
work in any language calling itself Python?
Thanks,
--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
I agree so much on your opinion that I was just to create a topic about this if you didn't. I also propose here a small modification to make it more general which adds an argument `how` (name to be discussed), telling how to merge the dicts, as many have pointed out that there could be different ways to merge dicts. So things would be like def addition_merge(key, values, exists): """ :param key: the key to merge :param values: values of dicts to merge indexed at `key` :param exists: whether each dict contains `key` """ if any(exists): return True, sum([value for exist, value in zip(exists, values) if exist]) else: return False d1.merge(d2, d3, ..., how=addition_merge) We could even have def discard(key, values, exists): return not any(exists[1:]), values[0] d1.merge(d2, how=discard) which does the same thing as proposed `d1-d2`. This would make things like d = d1.merge(iter_of_pairs) d = d1.merge(key=value) not working, but people could easily wrap a `dict()` over the iterator or key-value stuff and attach no complication. At 2019-03-05 15:39:40, "INADA Naoki" wrote: >I think some people in favor of PEP 584 just want >single expression for merging dicts without in-place update. > >But I feel it's abuse of operator overload. I think functions >and methods are better than operator unless the operator >has good math metaphor, or very frequently used as concatenate >strings. > >This is why function and methods are better: > >* Easy to search. >* Name can describe it's behavior better than abused operator. >* Simpler lookup behavior. (e.g. subclass and __iadd__) > >Then, I propose `dict.merge` method. It is outer-place version >of `dict.update`, but accepts multiple dicts. (dict.update() >can be updated to accept multiple dicts, but it's not out of scope). > >* d = d1.merge(d2) # d = d1.copy(); d.update(d2) >* d = d1.merge(d2, d3) # d = d1.copy(); d.update(d2); d2.update(d3) >* d = d1.merge(iter_of_pairs) >* d = d1.merge(key=value) > > >## Merits of dict.merge() over operator + > >* Easy to Google (e.g. "python dict merge"). >* Easy to help(dict.merge). (or dict.merge? in IPython) >* No inefficiency of d1+d2+d3+...+dN, or sum(list_of_many_dicts) >* Type of returned value is always same to d1.copy(). No issubclass, >no __iadd__. > >## Why not dict.updated()? > >sorted() is a function so it looks different from L.sort() >But d.updated() is very similar to d.update() for human eyes. > >## How about d1 - d2? > >If it is really useful, it can be implemented as method too. > >dict.discard(sequence_of_keys) > >Regards, >-- >INADA Naoki >___ >Python-ideas mailing list >Python-ideas@python.org >https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
On Mon, Mar 04, 2019 at 10:18:13PM -0800, Amber Yust wrote: > Adding the + operator for dictionaries feels like it would be a mistake in > that it offers at most sugar-y benefits, but introduces the significant > drawback of making it easier to introduced unintended errors. What sort of errors? I know that some (mis-)features are "bug magnets" that encourage people to write buggy code, but I don't see how this proposal is worse than dict.update(). In one way it is better, since D + E returns a new dict, instead of over-writing the data in D. Ask any functional programmer, and they'll tell you that we should avoid side-effects. > This would be > the first instance of "addition" where the result can potentially > lose/overwrite data (lists and strings both preserve the full extent of > each operand; Counters include the full value from each operand, etc). I don't see why this is relevant to addition. It doesn't even apply to numeric addition! If I give you the result of an addition: 101 say, you can't tell what the operands were. And that's not even getting into the intricicies of floating point addition, which can violate associativity ``(a + b) + c`` is not necessarily equal to ``a + (b + c)`` and distributivity: ``x*(a + b)`` is not necessarily equal to ``x*a + x*b`` even for well-behaved, numeric floats (not NANs or INFs). > Combining dictionaries is fundamentally an operation that requires more > than one piece of information, because there's no single well-defined way > to combine a pair of them. Indeed, But some ways are more useful than others. > Off the top of my head, I can think of at least > 2 different common options (replacement aka .update(), combination of > values a la Counter). Neither of these is really a more valid "addition" of > dictionaries. That's why we have subclasses and operator overloading :-) By far the most commonly requested behaviour for this is copy-and- update (or merge, if you prefer). But subclasses are free to define it as they will, including: - add values, as Counter already does; - raise an exception if there is a duplicate key; - "first seen wins" or anything else. -- Steven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
On Mon, Mar 04, 2019 at 03:33:36PM -0500, Neil Girdhar wrote:
> Maybe, but reading through the various replies, it seems that if you
> are adding "-" to be analogous to set difference, then the combination
> operator should be analogous to set union "|".
That's the purpose of this discussion, to decide whether dict merging is
more like addition/concatenation or union :-)
> And it also opens an
> opportunity to add set intersection "&".
What should intersection do in the case of matching keys?
I see the merge + operator as a kind of update, whether it makes a copy
or does it in place, so to me it is obvious that "last seen wins" should
apply just as it does for the update method.
But dict *intersection* is a more abstract operation than merge/update.
And that leads to the problem, what do you do with the values?
{key: "spam"} & {key: "eggs"}
# could result in any of:
{key: "spam"}
{key: "eggs"}
{key: ("spam", "eggs")}
{key: "spameggs"}
an exception
something else?
Unlike "update", I don't have any good use-cases to prefer any one of
those over the others.
> After all, how do you filter a dictionary to a set of keys?
>
> >> d = {'some': 5, 'extra': 10, 'things': 55}
> >> d &= {'some', 'allowed', 'options'}
> >> d
> {'some': 5}
new = d - (d - allowed)
{k:v for (k,v) in d if k in allowed}
> >> > * Regarding how to construct the new set in __add__, I now think this
> >> > should be done like this:
> >> >
> >> > class dict:
> >> >
> >> > def __add__(self, other):
> >> >
> >> > new = self.copy() # A subclass may or may not choose to override
> >> > new.update(other)
> >> > return new
> >>
> >> I like that, but it would be inefficient to do that for __sub__ since
> >> it would create elements that it might later delete.
> >>
> >> def __sub__(self, other):
> >> new = self.copy()
> >> for k in other:
> >> del new[k]
> >> return new
> >>
> >> is less efficient than
> >>
> >> def __sub__(self, other):
> >> return type(self)({k: v for k, v in self.items() if k not in other})
I don't think you should be claiming what is more or less efficient
unless you've actually profiled them for speed and memory use. Often,
but not always, the two are in opposition: we make things faster by
using more memory, and save memory at the cost of speed.
Your version of __sub__ creates a temporary dict, which then has to be
copied in order to preserve the type. Its not obvious to me that that's
faster or more memory efficient than building a dict then deleting keys.
(Remember that dicts aren't lists, and deleting keys is an O(1)
operation.)
--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
On Tue, Mar 5, 2019 at 6:42 PM Jimmy Girardet wrote: > > Indeed the "obscure" argument should be thrown away. > > The `|` operator in sets seems to be evident for every one on this list > but I would be curious to know how many people first got a TypeError > doing set1 + set2 and then found set1 | set2 in the doc. > > Except for math geek the `|` is always something obscure. > Interesting point. In Japan, we learn set in high school, not in university. And I think it's good idea that people using `set` type learn about `set` in math. So I don't think "union" is not only for math geeks. But we use "A ∪ B" in math. `|` is borrowed from "bitwise OR" in C. And "bitwise" operators are for "geeks". Although I'm not in favor of adding `+` to set, it will be worth enough to add `+` to set too if it is added to dict for consistency. FWIW, Scala uses `++` for join all containers. Kotlin uses `+` for join all containers. (ref https://discuss.python.org/t/pep-584-survey-of-other-languages-operator-overload/977) Regards, -- Inada Naoki ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] PEP: Dict addition and subtraction
This is mainly for Steve, as the author of PEP 584. I'm grateful to Steve for preparing the current draft. Thank you. It's strong on implementation, but I find it weak on motivation. I hope that when time is available you (and the other contributors) could transfer some motivating material into the PEP, from python-ideas. According to PEP 001, the PEP "should clearly explain why the existing language specification is inadequate to address the problem that the PEP solves". So it is important. -- Jonathan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
Indeed the "obscure" argument should be thrown away. The `|` operator in sets seems to be evident for every one on this list but I would be curious to know how many people first got a TypeError doing set1 + set2 and then found set1 | set2 in the doc. Except for math geek the `|` is always something obscure. >> Even if dict1 - dict2 were >> added to the language, I think I'd steer clear of it as being too >> obscure. > Everything is obscure until people learn it and get used to it. > > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Dict joining using + and +=
On Mon, Mar 04, 2019 at 09:34:34PM +, Paul Moore wrote:
> On Mon, 4 Mar 2019 at 20:42, Guido van Rossum wrote:
> >
> > Honestly I would rather withdraw the subtraction operators than
> > reopen the discussion about making dict more like set.
As some people have repeatedly pointed out, we already have four ways
to spell dict merging:
- in-place dict.update;
- copy, followed by update;
- use a ChainMap;
- the obscure new (**d1, ...} syntax.
But if there's a good way to get dict difference apart from a manual
loop or comprehension, I don't know it.
So from my perspective, even though most of the attention has been on
the merge operator, I'd rather keep the difference operator.
As far as making dicts "more like set", I'm certainly not proposing
that. The furthest I'd go is bow to the consensus if it happened to
decide that | is a better choice than + (but that seems unlikely).
> I'm neutral on dict addition, but dict subtraction seemed an odd
> extension to the proposal. Using b in a - b solely for its keys, and
> ignoring its values, seems weird to me.
The PEP current says that dict subtraction requires the right-hand
operand to be a dict. That's the conservative choice that follows the
example of list addition (it requires a list, not just any iterable) and
avoids breaking changes to code that uses operator-overloading:
mydict - some_object
works if some_object overloads __rsub__. If dict.__sub__ was greedy in
what it accepted, it could break such code. Better (in my opinion) to be
less greedy by only allowing dicts.
dict -= on the other hand can take any iterable of keys, as the
right-hand operand isn't called.
Oh, another thing the PEP should gain... a use-case for dict
subtraction. Here's a few:
(1) You have a pair of dicts D and E, and you want to update D with only
the new keys from E:
D.update(E - D)
which I think is nicer than writing a manual loop:
D.update({k:E[k] for k in (E.keys() - D.keys())})
# or
D.update({k:v for k,v in E.items() if k not in D})
(This is a form of update with "first seen wins" instead of the usual
"last seen wins".)
(2) You have a dict D, and you want to unconditionally remove keys from
a blacklist, e.g.:
all_users = {'username': user, ...}
allowed_users = all_users - banned_users
(3) You have a dict, and you want to ensure there's no keys that
you didn't expect:
if (d := actual-expected):
print('unexpected key:value pairs', d)
> Even if dict1 - dict2 were
> added to the language, I think I'd steer clear of it as being too
> obscure.
Everything is obscure until people learn it and get used to it.
--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
> * Type of returned value is always same to d1.copy(). No issubclass, > no __iadd__. I'm sorry, I meant __radd__, not __iadd__. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Tue, Mar 5, 2019 at 5:50 PM Nathaniel Smith wrote: > > On Mon, Mar 4, 2019 at 11:41 PM INADA Naoki wrote: > > Then, I propose `dict.merge` method. It is outer-place version > > of `dict.update`, but accepts multiple dicts. (dict.update() > > can be updated to accept multiple dicts, but it's not out of scope). > > > > * d = d1.merge(d2) # d = d1.copy(); d.update(d2) > > * d = d1.merge(d2, d3) # d = d1.copy(); d.update(d2); d2.update(d3) > > * d = d1.merge(iter_of_pairs) > > * d = d1.merge(key=value) > > Another similar option would be to extend the dict constructor to > allow: d = dict(d1, d2, d3, ...) > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org Yes, it's an option too. One obvious merit of d.merge(...) is it returns same type of d. `type(d1)(d1, d2)` looks ugly. But people just want dict instead of some subtype of dict. This merit is not so important. I'm bit nervous about adding much overload to constructor. That's main reason why I proposed method instead of constructor. Regards, -- INADA Naoki ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Suggestions: dict.flow_update and dict.__add__
SUMMARY Instead of using dict + dict, perhaps use dict.flow_update. Here, flow_update is just like update, except that it returns self. BACKGROUND There's a difference between a sorted copy of a list, and sorting the list in place. >>> items = [2, 0, 1, 9] >>> sorted(items), items ([0, 1, 2, 9], [2, 0, 1, 9]) >>> items.sort(), items (None, [0, 1, 2, 9]) In Python, mutating methods generally return None. Here, this prevents beginners thinking their code has produced a sorted copy of a list, when in fact it has done an in-place sort on the list. If they write >>> aaa = my_list.sort() they'll get a None error when they use aaa. The same goes for dict.update. This is a useful feature, particularly for beginners. It helps them think clearly, and express themselves clearly. THE PROBLEM This returning None can be a nuisance, sometimes. Suppose we have a dictionary of default values, and a dictionary of use supplied options. We wish to combine the two dictionaries, say into a new combined dictionary. One way to do this is: combined = defaults.copy() combined.update(options) But this is awkward when you're in the middle of calling a function: call_big_method( # lots of arguments, one to a line, with comments arg = combined, # Look up to see what combined is. # more arguments ) USING + There's a suggestion, that instead one extends Python so that this works: arg = defaults + options # What does '+' mean here? USING flow_update Here's another suggestion. Instead write: dict_arg = defaults.copy().flow_update(options) # Is this clearer? IMPLEMENTATION Here's an implementation, as a subclass of dict. class mydict(dict): def flow_update(self, *argv, **kwargs): self.update(*argv, **kwargs) return self def copy(self): return self.__class__(self) A DIRTY HACK Not tested, using an assignment expression. dict_arg = (tmp := defaults.copy(), tmp.update(options))[0] Not recommend. -- Jonathan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Mon, Mar 4, 2019 at 11:41 PM INADA Naoki wrote: > Then, I propose `dict.merge` method. It is outer-place version > of `dict.update`, but accepts multiple dicts. (dict.update() > can be updated to accept multiple dicts, but it's not out of scope). > > * d = d1.merge(d2) # d = d1.copy(); d.update(d2) > * d = d1.merge(d2, d3) # d = d1.copy(); d.update(d2); d2.update(d3) > * d = d1.merge(iter_of_pairs) > * d = d1.merge(key=value) Another similar option would be to extend the dict constructor to allow: d = dict(d1, d2, d3, ...) -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Tue, Mar 5, 2019 at 5:23 PM Chris Angelico wrote:
>
> On Tue, Mar 5, 2019 at 6:40 PM INADA Naoki wrote:
> > This is why function and methods are better:
> >
> > * Easy to search.
> >
> > ## Merits of dict.merge() over operator +
> >
> > * Easy to Google (e.g. "python dict merge").
>
> This keeps getting thrown around. It's simply not true.
>
> https://www.google.com/search?q=%7B**d1%2C+**d2%7D
>
> First hit when I do that search is Stack Overflow:
>
> https://stackoverflow.com/questions/2255878/what-does-mean-in-the-expression-dictd1-d2
>
> which, while it's not specifically about that exact syntax, does
> mention it in the comments on the question. Symbols ARE searchable. In
> fact, adding the word "python" to the beginning of that search
> produces a number of very useful hits, including a Reddit thread on
> combining dictionaries, and PEP 584 itself.
>
> Please can people actually test these lines of argument before reiterating
> them?
>
> ChrisA
I'm surprised {**d1, **d2} is searchable. But in my proposal, I compared with
one character operator `+`.
I switched my browser as English and Googled "python str +"
https://www.google.com/search?q=python+str+%2B=python+str+%2B
As far as I can see, top result is https://docs.python.org/2/library/string.html
When I search "+" in the page, it's difficult to find concat string.
I tried Google "python set union" and "python set |" too.
"union" is much easier to reach the answer.
So I don't think "name is easier to Google than symbol" is a fake or FUD.
Regards,
--
INADA Naoki
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)
On Tue, Mar 5, 2019 at 6:40 PM INADA Naoki wrote: > This is why function and methods are better: > > * Easy to search. > > ## Merits of dict.merge() over operator + > > * Easy to Google (e.g. "python dict merge"). This keeps getting thrown around. It's simply not true. https://www.google.com/search?q=%7B**d1%2C+**d2%7D First hit when I do that search is Stack Overflow: https://stackoverflow.com/questions/2255878/what-does-mean-in-the-expression-dictd1-d2 which, while it's not specifically about that exact syntax, does mention it in the comments on the question. Symbols ARE searchable. In fact, adding the word "python" to the beginning of that search produces a number of very useful hits, including a Reddit thread on combining dictionaries, and PEP 584 itself. Please can people actually test these lines of argument before reiterating them? ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
