subject:"\[Python\-ideas\] Re\: Unpack operator \"\*\*\" and Mapping"


On 2020-12-29 16:18, Steven D'Aprano wrote:

On Tue, Dec 29, 2020 at 02:20:05PM -0800, Brendan Barnwell wrote:


So, to give an example, the iterator protocol should be documented
right where the `for` statement is documented, and it should be
explicitly framed as "this is the definition of what the `for` statement
does", not something separate.


Hmmm, that's going to be a bit confusing for people who want to
understand iter() and next() but don't care about for loops directly.


	With regard to this and your other comments, I can see that that could 
be useful in some cases.  But the important points are: 1) if the for 
statement uses the iterator protocol (which it does), the part of the 
documentation where the for statement is defined absolutely must link to 
the docs on the iterator protocol; and 2) the details of how `for` uses 
the protocol must be explained in full in the docs for `for`.



I totally agree with you that the connections between features, and
detailed information about protocols and dunders can sometimes be hard
to find, but I think that the solution is not a complete redesign of the
docs, but more hyperlinks and See Alsos.


	I would go slightly further than that though.  It's not just "see 
alsos".  It can be hyperlinks, but what needs to be made clear is that 
the hyperlinks are to DEFINITIONAL information where that is the case. 
In other words, the `for` statement docs need to say that the operation 
of the `for` statement is DEFINED by the iterator protocol.  Now yes, if 
that definiton has many nooks and crannies we can factor those out into 
a separate page.  But the way that `for` uses the iterator should indeed 
be spelled out.  For instance, right now it says:


An iterator is created for the result of the expression_list. The suite 
is then executed once for each item provided by the iterator, in the 
order returned by the iterator.  Each item in turn is assigned to the 
target list using the standard rules for assignments (see Assignment 
statements), and then the suite is executed.


What I'm saying is it should say something like:

An iterator (link to iterator protocol here) is created by calling 
iter() on the result of the expression_list.  For each loop iteration, 
next() is called on the iterator, then the resulting value is assigned 
to the target list using the standard rules for assignment (link to 
assignment here), and then the suite is executed.


	Now, you're right that's a bit different from it maybe sounded like in 
my message before.  I'm not necessarily saying that the entire 
definition of next and iter needs to be in there.  But what I am saying 
is that the nuts and bolts of how `for` uses the protocol DO need to be 
in there.  Things like "returned by the iterator" (which is not clear in 
the terminology of the iterator protocol) should be replaced by things 
like "returned by calling next() on the iterator" or "returned by the 
iterator's __next__ method".


	My point here is really more about the docs on other parts of Python 
rather than the documentation of the iterator protocol itself (or other 
such protocols).  You're right that they need to be integrated, but it's 
not solely a matter of providing links; I think there needs to be 
considerably more detail about the precise way in which syntactic 
features make use of these protocols.  (Or in some cases less detail may 
be needed; for instance, right now the `for` docs include a little note 
about "modifying the sequence" while it is being iterated over; this 
doesn't belong there because the for statement doesn't care about 
sequences, but only about the iterator protocol.)


	For the same reason, I'm less concerned about people who want to know 
about iter() and next() themselves.  That information is nice to have, 
and it should be there in an easily-findable section of the docs for 
sure.  But `for` is a basic language construct, and it's in the 
documentation of those basic language constructs where we need that 
clarity about how they interact with the protocols.


	I don't think this would require a complete redesign of the docs, 
because sure, a lot of it could stay the same, but what I meant by 
"overhaul" is that some restructuring (not just changing of wording) is 
necessary.  For instance, protocols like iterator, descriptor, etc., 
which, as you say, may be used in various ways, really need their own 
section if we're going to be referring to them via hyperlinks.  But more 
than that, I think a lot of the documentation on basic stuff like loops, 
attribute access, even mathematical operators, needs to be 
reconceptualized so that it couches the entire description in terms of 
the protocols that are actually used.  It's not just a few links here 
and there.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Greg Ewing


On 30/12/20 11:20 am, Brendan Barnwell wrote:
 So, to give an example, the iterator protocol should be documented 
right where the `for` statement is documented, and it should be 
explicitly framed as "this is the definition of what the `for` statement 
does",


Not sure about that -- there are other syntactic situations that use
the iterator protocol, e.g. unpacking assignments. But there should
at least be links to it from all the places it's used.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EY4XFLEY3U2ZB5L6IRVX7RSCZBARE6OH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Greg Ewing

On 30/12/20 9:44 am, Christopher Barker wrote:
So there are dunders, and protocols, and ABCs, and 
they all overlap a bit in purpose. And "protocols" seem to be the least 
clearly specified.

> there are various protocols described in the C API docs, including
> the Mapping protocol:
>
> https://docs.python.org/3/c-api/mapping.html
> 
>
> But I *think* that's really about the C API -- not what we are talking
> about here.

The library reference doesn't seem to use the terms "sequence protocol" 
and "mapping protocol". It talks about "sequence types" and "mapping

types", but doesn't use the word "protocol" in relation to them.

Nor is there any clear list of methods that a type needs to provide
in order to be considered a sequence or mapping. There are lists of
operations supported by the built-in container types, but those are
fairly extensive, and it's not obvious which ones are vital.

So are the "sequence protocol" and "mapping protocol" really mythical
beasts that don't really exist? Are they more akin to the infamous
"file-like object" which is whatever it needs to be for the purpose
at hand?

Guido has since said that the ABCs are intended to be definitive,
but the docs don't really make that clear either. (And the ABC doc page
talks about "APIs", not "protocols"!)

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SU57ZIL3ESMT4QABP7I7HVG445EOALZP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Steven D'Aprano

On Tue, Dec 29, 2020 at 02:20:05PM -0800, Brendan Barnwell wrote:

>   So, to give an example, the iterator protocol should be documented 
> right where the `for` statement is documented, and it should be 
> explicitly framed as "this is the definition of what the `for` statement 
> does", not something separate.

Hmmm, that's going to be a bit confusing for people who want to 
understand iter() and next() but don't care about for loops directly.

> The unpacking "protocol" (or whatever we 
> call it) should be documented right where function call syntax is 
> documented.

That's going to be a bit obscure for people who want to understand 
unpacking:

a, b, *extras, z = values

>  __getitem__ should be documented right where the [] syntax 
> is documented. 

That's a bit confusing for people who know the method name but don't 
know what syntax uses it. And its a lot confusing for people who just 
want to know what subscripting means but don't care about the dunder 
methods used to implement it.

> The descriptor protocol should be documented right where 
> attribute access syntax is documented.

And that's quite hostile to those who want a simple high-level overview 
of what attribute access does without having a low-level and very 
advanced feature, descriptors, shoved down their throat.

Brendan, I applaud your zeal in wanting to fix the Python docs, but 
things which are no-brainer improvements to *you* will be no-brainer 
regressions to others.

In particular, the Python docs aren't a dead-tree book, they are a 
hypertext document. There's no need to have features documented "right 
where" related features are documented, because that leads to needing 
everything documented in one place:

- subscripting `[]` needs to have mappings and sequences documented 
  right there;

- mappings need to have hash documented right there;

- sequences need to have integers and the `__index__` dunder documented 
  right there

- integers need to have the numeric tower documented right there

- which needs ABCs to be documented right there

and because everything in Python is connected to everything else, the 
entire language and std lib needs to be on one page.

I totally agree with you that the connections between features, and 
detailed information about protocols and dunders can sometimes be hard 
to find, but I think that the solution is not a complete redesign of the 
docs, but more hyperlinks and See Alsos.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6J54T2BBDASJQQEHVN7JTAHRUVETKBLB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Steven D'Aprano

On Tue, Dec 29, 2020 at 10:30:16AM -0800, Guido van Rossum wrote:

[Christopher]
> > Does there need to be a single defined "protocol" for a mapping (other
> > than the ABC)? -- that is, would **unpacking be able to use .items() and
> > keys() be used in other contexts?

Yes, I think there should be one protocol, or interface if you prefer, 
for something to be considered dict-like. If some operations expect 
objects to quack like a dict, then it would be annoying if other related 
operations expect them to swim like a dict.

The status quo right now is:

(1) The dict constructor, dict.update, and dict union assignment `|=` 
all support the double-barrelled interface:

- try keys() and `__getitem__`

- otherwise fall back onto direct iteration.

(2) Dict unpacking `**` only supports the keys/getitem interface.

My guess is that's an oversight, not a deliberate difference in 
behaviour.

I think that those four operations should operate in a consistant 
manner.

Otherwise, we would be okay if we can write a object that quacks like a 
dict via keys/getitem, since that is supported by all four, but suppose 
you can't and have to use the fallback operation.

So you dutifully add the direct iteration API, and now your object works 
in the dict constructor etc but not dict unpacking. So you have to 
*also* add the items() method you suggest, an otherwise unnecessary 
second way of doing the same thing. This is annoying and error-prone: 
being two methods, there is a risk that they will diverge in behaviour, 
or that documentation will diverge, and now users of your object need to 
worry about whether to use direct iteration or items(), and you know 
that people will keep asking what's the difference between them.

It wouldn't be a disaster if we went this way, but it would add 
unnecessary friction.

There is however a third possibility: extend the dict interface by 
adding a second fallback. I think we're stuck with keeping the existing 
order as first and second attempts, but we can tack items() at the end:

- try keys() and `__getitem__`

- try direct iteration

- otherwise, try items()

I don't hate this, but honestly I think it is YAGNI.

[Guido] 
> I don't understand why LBYL is considered such an anti-pattern. It helps
> produce much clearer error messages in this case for users who are
> exploring this feature, and distinguishing *early* between sequences and
> mappings is important for that. Long ago we decided that the distinctive
> feature is that mappings have a `keys()` method whereas sequences don't
> (and users who add a `keys()` method to a sequence are just asking for
> trouble). So that's what we use.

I remember learning that EAPF was preferred over LBYL back when I 
started in Python 1.5 days. I think the main reasons were to encourage 
people to duck-type, and to avoid Time Of Check To Time Of Use errors, 
e.g. when opening files. But although EAFP was preferred, there are 
definitely times when LBYL is better, and I think that calling LBYL 
"unPythonic" is a hyper-correction.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OTC5HL2XC2SRGEQLOKMIOUM5L5IDGTFW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

On 2020-12-29 13:54, Chris Angelico wrote:

Documentation bugs are bugs to be fixed, just like any other. Rather
than complaining that this should have prevented the feature from
being released, just propose an improvement. Where should this be
documented? How should it be worded?

Well, I did propose it, for this particular case, in a part of the
message you didn't quote.

But I think ultimately to do it right the documentation needs a
structural overhaul. The way it is now tries to separate "language
reference" and "library reference", where the former seems to mainly be
describing syntax, with other stuff shunted off into opaquely-titled
sections like "data model". But I think that is confusing

So, to give an example, the iterator protocol should be documented
right where the `for` statement is documented, and it should be
explicitly framed as "this is the definition of what the `for` statement
does", not something separate. The unpacking "protocol" (or whatever we
call it) should be documented right where function call syntax is
documented. __getitem__ should be documented right where the [] syntax
is documented. The descriptor protocol should be documented right where
attribute access syntax is documented. And so on. In other words, the
"language reference" should be more or less a catalog of syntactic
features with in-situ definitions of their semantics as the invocation
of a particular protocol.

I think the way Python handles these things as a language is one of its
greatest features. Basically Python is a sort of syntactic framework on
which you can hang various implementations by overriding dunders, and
that's very powerful. So it's unfortunate that that isn't used as the
main organizing principle of the documentation.

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."

--author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/HRS4BXHPGP27I32AVWRUZPYKW327IHCN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Christopher Barker

On Tue, Dec 29, 2020 at 1:51 PM Brendan Barnwell 
wrote:

> > they all overlap a bit in purpose. And "protocols" seem to be the least
> > clearly specified. Indeed, the "iteration protocol" is well known and
> > talked about, but not well documented in the standard docs:
>
> No offense to anyone here, but I don't really see that as a valid
> excuse.

It was an explanation, not an excuse -- this is an open source project --
you have better ideas for documentation, submit a PR -- or process, there
are places to contribute there as well.

>   Documentation can evolve too.  It's fine if the protocols and
> so on overlap, but the nature of that overlap needs to be documented.
>

And I made some suggestions for that -- but the Duck Typing is tricky to
document, it's by definition not clearly defined.

 Is there
> any ongoing project to bring the official documentation up to date with
> regard to the kinds of issues you describe?
>

not that I know of -- great place to volunteer, though.

> > Maybe something along the lines of:
> >
> > "When a Mapping type is required in python (e.g. dict.update, **
> > unpacking),  .keys() and __getitem__ are the minimum required to meet
> > the Mapping protocol."
>
> Even that isn't sufficiently explicit, to my mind.  It needs to be
> something more like:
>
> When an object is unpacked with ** unpacking, its .keys() method is
> called to get the available keys, and its __getitem__ is called to
> obtain the value for each key.
>

Ah, but that's different -- I was attempting to document the "mapping
protocol" such as it is, not ** behavior. And where THAT should be
doucmented isn't clear to me, but maybe here:

https://docs.python.org/3/reference/expressions.html?highlight=unpacking

where it says:

"A double asterisk ** denotes dictionary unpacking. Its operand must be a
mapping."

With "mapping" being a lnk to the Glossary. So either there, or somewhere
linked to there could be a defintion of what a minimal"mapping" is.

 If dict.update internally uses .keys() and that's relevant
> for stuff like subclassing dict, that should be in the dict documentation.
>

That's actually the one place I could find where it IS documented.

In other words, every operation that uses .keys() needs to be
> defined
> to use .keys() where that operation it is defined.  Or, if there is a
> "dict protocol" (like maybe the Mapping ABC) then there should be a
> separate doc page about that protocol, which gives a COMPLETE list of
> all methods that are part of the protocol

This is what I'm getting at: there is an informal protocol for "reading" a
mapping. and it should be documented. but the COMPLETE list is simply
.keys() and .__getitem__ :-)

along with a COMPLETE list of
> the language features that make use of that protocol.
>

That is pretty much impossible -- that's kind of the point of a protocol --
it can be used in arbitrary places in arbitrary code.

would you expect a COMPLETE list of all the language features that use the
iteration protocol?

> Otherwise, the Mapping ABC is clearly defined, but not, in fact,
> > required in most of the contexts that expect a Mapping.
>
>  > you have to actually give a complete list of
> situations when it's going to be implicitly called, and update that list
> if needed as the language evolves.
>

I don't "have" to do anything, nor does anyone else contributing to
the development or documentation of Python.

But tone aside, I think you're pushing pretty hard for something that
doesn't really fit Python -- again, Duck typing is not a set of hard and
fast rules.

If you have specific ideas for how the documentation can be improved, by
all means make them.

-CHB

-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OKENWBMJLD67UTYS4JVVZ4ZE3QWF5EY2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Chris Angelico

On Wed, Dec 30, 2020 at 8:49 AM Brendan Barnwell  wrote:
> To my mind, every time a change is made to Python behavior there must
> be a corresponding change to the main documentation describing the
> change.  I would go so far as to say that the lack of such documentation
> updates should be a blocker on the release of the feature.  Features
> without complete documentation in a straightforward place on
> python.org/docs (not the PEPs!) should not be shipped.
>

It's not that simple. As one example, consider zipapp:

https://docs.python.org/3/whatsnew/3.5.html#whatsnew-zipapp

It existed for a long time before that, and technically WAS
documented, but nobody knew about it. Would you say that Python would
have been better off leaving this feature languishing on the bug
tracker for lack of adequate documentation? What does "adequate" even
mean? (Or to use your term, "complete"? Perhaps even harder to
define.)

Documentation bugs are bugs to be fixed, just like any other. Rather
than complaining that this should have prevented the feature from
being released, just propose an improvement. Where should this be
documented? How should it be worded?

(I'd contribute actual wording myself, but I don't know the details of
the feature well enough. TBH, I only ever use ** with actual
dictionaries.)

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MHU4TI66RSYHYAJCEF5O54PIFNEIU42H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

On 2020-12-29 12:44, Christopher Barker wrote:

On Tue, Dec 29, 2020 at 11:18 AM Brendan Barnwell mailto:brenb...@brenbarn.net>> wrote:

On 2020-12-29 10:30, Guido van Rossum wrote:
> Long ago we decided that the distinctive
> feature is that mappings have a `keys()` method whereas sequences
don't

It seems the real issue here is one of documentation.

Exactly. The challenge here is that Python is dynamic and has evolved (a
lot!) over the years. So there are dunders, and protocols, and ABCs, and
they all overlap a bit in purpose. And "protocols" seem to be the least
clearly specified. Indeed, the "iteration protocol" is well known and
talked about, but not well documented in the standard docs:

No offense to anyone here, but I don't really see that as a valid
excuse. Documentation can evolve too. It's fine if the protocols and
so on overlap, but the nature of that overlap needs to be documented.

To my mind, every time a change is made to Python behavior there must
be a corresponding change to the main documentation describing the
change. I would go so far as to say that the lack of such documentation
updates should be a blocker on the release of the feature. Features
without complete documentation in a straightforward place on
python.org/docs (not the PEPs!) should not be shipped.

I can understand really old stuff like dict having undocumented
innards, when it was just Guido tinkering on his own, but stuff that
dates from the era of a wider development team and widespread use of
Python (like the iterator protocol) faces a higher standard. Is there
any ongoing project to bring the official documentation up to date with
regard to the kinds of issues you describe?

Maybe something along the lines of:

"When a Mapping type is required in python (e.g. dict.update, **
unpacking), .keys() and __getitem__ are the minimum required to meet
the Mapping protocol."

Even that isn't sufficiently explicit, to my mind. It needs to be
something more like:

When an object is unpacked with ** unpacking, its .keys() method is
called to get the available keys, and its __getitem__ is called to
obtain the value for each key.

That should be somewhere where it describes the unpacking syntax and
semantics. If dict.update internally uses .keys() and that's relevant
for stuff like subclassing dict, that should be in the dict documentation.

In other words, every operation that uses .keys() needs to be defined
to use .keys() where that operation it is defined. Or, if there is a
"dict protocol" (like maybe the Mapping ABC) then there should be a
separate doc page about that protocol, which gives a COMPLETE list of
all methods that are part of the protocol along with a COMPLETE list of
the language features that make use of that protocol.

Otherwise, the Mapping ABC is clearly defined, but not, in fact,
required in most of the contexts that expect a Mapping.

Again, to my mind, the Mapping ABC (and most of the other ABCs there)
are not clearly defined, because the semantics of the methods and when
they are invoked by other parts of Python's machinery (e.g., syntax) are
not defined. In other words, it's not enough to say that .keys() "is
part of the Mapping ABC"; you have to actually give a complete list of
situations when it's going to be implicitly called, and update that list
if needed as the language evolves.

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."

--author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/GSHJAXE7WVGSO4WGMY332WYVS7UB7WWY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Christopher Barker

On Tue, Dec 29, 2020 at 10:30 AM Guido van Rossum  wrote:

> Interesting -- thanks for taking up the challenge. I still suspect that if
> we ran the corresponding benchmark at the C level, the first form would win,
>

I was thinking that might be the case -- but either way, there is little
difference, and at least for ** unpacking, it's probably mostly used for
small dicts anyway.

> Does there need to be a single defined "protocol" for a mapping (other
>> than the ABC)? -- that is, would **unpacking be able to use .items() and
>> keys() be used in other contexts?
>>
>> And why does ** unpacking need to check at all (LBYL) couldn't it simply
>> do something like:
>>
>> {k: d[k] for k in d}
>>
>> I don't understand why LBYL is considered such an anti-pattern. It helps
> produce much clearer error messages in this case for users who are
> exploring this feature, and distinguishing *early* between sequences and
> mappings is important for that.
>

Fair enough, though in this case, it's producing a not quite clear error
message, whereas simply trying to call keys() would reflect the actual
error. I was thinking about this a bit more, and realized that this pattern
is used (at least) in the dict constructor and dict.update() -- but in both
of those cases, they can take either a mapping or an iterable of (key,
value) pairs. So it is required to make a distinction, and looking for
keys() is as good a way as any (maybe the best, certainly well established)

(and if you pass a object with a keys() method, but no __getitem__ into
dict(), you get: "TypeError: 'MinMap' object is not subscriptable" -- not
anything it about it needing to be a Mapping)

But for **, which only supports Mappings, maybe there is no need to check
for keys() -- it is clearly defined that iter(a_mapping) iterates over the
keys, so that part should work, and if the __getitem__ doesn't work
appropriately, then that's not really different than passing a
iterable that doesn't produce valid (key, value) pairs to dict().

But this is all theoretical -- it's established, and a bit better docs
should clear up the confusion.

One more note on the docstrings:

dict.update() says:

"... If E is present and has a .keys() method..."

Which nicely defines what is actually required.

Whereas dict() says:

"...dict(mapping) -> new dictionary initialized from a mapping
object's (key, value) pairs..."

Without saying anything about how it determines what whether it's a
mapping. So maybe that could be made a bit more clear as well.

I also just noticed something else in the docs -- in typing, there is a
Protocol type -- maybe we could/should pre-define a mapping protocol type?

Or maybe a MinimialMapping ABC, analogous to the Iterable ABC -- though no
idea what to call it that would be clear :-)

-CHB

-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IXPZJSCT2C2Z4MFWQUZVZY4MGBGBJ457/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Christopher Barker

On Tue, Dec 29, 2020 at 11:18 AM Brendan Barnwell
wrote:

> On 2020-12-29 10:30, Guido van Rossum wrote:
> > Long ago we decided that the distinctive
> > feature is that mappings have a `keys()` method whereas sequences don't
>

It seems the real issue here is one of documentation.

The only actual reference to it I could find is in the C API docs.

__iter__ and __next__ are referred to a lot, but I don't see the protocol
described anywhere.

And there are various protocols described in the C API docs, including
the Mapping protocol:

https://docs.python.org/3/c-api/mapping.html

But I *think* that's really about the C API -- not what we are talking
about here. Also interestingly, it says:
"Note that it returns 1 for Python classes with a __getitem__() method" --
so it's not checking for keys() here.

There IS a discussion of the iterator protocol (without using that word)
here:

https://docs.python.org/3/library/stdtypes.html#iterator-types

Under "Iterator Types" -- which is a bit odd, as strictly speaking, it's
not a type, but where else would it go?

So maybe we could add some text to the Mapping Type section on that page:

https://docs.python.org/3/library/stdtypes.html#mapping-types-dict

Maybe something along the lines of:

"When a Mapping type is required in python (e.g. dict.update, **
unpacking), .keys() and __getitem__ are the minimum required to meet the
Mapping protocol."

Otherwise, the Mapping ABC is clearly defined, but not, in fact, required
in most of the contexts that expect a Mapping.

The other place we could perhaps improve the docs is in the error message
for **:

"TypeError: 'str' object is not a mapping"

perhaps it could say something like:

"TypeError: 'str' object is not a mapping. An object must have a .keys()
method to be used in this context"

or some such.

As far as I can
> see, on python.org/docs all we have is that the Mapping abc lists keys()
> as a mixin method, but nowhere actually says that those methods are also
> what is used to implement syntactic things like **-unpacking.

Indeed, and it also lists .items() among others, but it would be good to
specify that it is keys() that is used to determine if an object "is* a
Mapping. I see this as a fundamental clash between the concept of an "ABC"
and duck typing -- an ABC defines everything that *could* be needed,
whereas duck typing requires only what is actually needed in a given
context. But if that's the case, it should be documented.

-CHB

--
Christopher Barker, PhD

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/IDNWMMZTBPUSU6GZXYNCQ4XAF2RTLYU3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

On 2020-12-29 10:30, Guido van Rossum wrote:

I don't understand why LBYL is considered such an anti-pattern. It helps
produce much clearer error messages in this case for users who are
exploring this feature, and distinguishing *early* between sequences and
mappings is important for that. Long ago we decided that the distinctive
feature is that mappings have a `keys()` method whereas sequences don't
(and users who add a `keys()` method to a sequence are just asking for
trouble). So that's what we use.

I think what is confusing to me that is that it was not ever clear to
me that such a decision was ever made. I can't find anything on
python.org/docs that explicitly says that that is how a mapping defines
its mapping behavior, let alone that that specifically defines unpacking
behavior.

It seems the real issue here is one of documentation. My view tends to
me that python.org/docs is the ultimate authority on the language's
behavior. Unfortunately it sometimes doesn't work that way, and I've
been meaning to make a post on this list about it, which hopefully I
will do at some point. But like, important stuff like "** unpacking is
implemented with this method" should not be buried in a PEP, nor a
docstring, and certainly not a source-code comment. As far as I can
see, on python.org/docs all we have is that the Mapping abc lists keys()
as a mixin method, but nowhere actually says that those methods are also
what is used to implement syntactic things like **-unpacking. I see
this is a major oversight and I think the docs should be updated more
comprehensively when PEPs are approved or decisions like the one you
describe are made, so that the actual, real documentation --- not the
PEPs! --- is always reflective and in fact definitional of the
language's behavior. In other words, what the language is accountable
to is the documentation on python.org/docs and nothing else.

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."

--author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/KYYJ6V2DWPWCVTICSF6SCLIUHMZEVB5E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Guido van Rossum

On Mon, Dec 28, 2020 at 11:36 PM Christopher Barker 
wrote:

> On Mon, Dec 28, 2020 at 12:33 PM Guido van Rossum 
> wrote:
>
>> On Mon, Dec 28, 2020 at 12:15 PM Christopher Barker 
>> wrote:
>>
>>> Though frankly, I would rather have had it use .items() -- seems more
>>> efficient to me, and you do need both the keys and the values, and items()
>>> is just as much part of the Mapping API as keys.
>>>
>>
>> There may be a (small) performance issue with that -- items() requires
>> creating a tuple object for each key/value pair.
>>
>
> it does look like items() is a tad faster (dict with 1000 items), but not
> enough to matter:
>
> In [61]: %timeit {k: d[k] for k in d.keys()}
> 112 µs ± 1.02 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)
>
> In [62]: %timeit {k: v for k, v in d.items()}
> 92.6 µs ± 1.9 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)
>

Interesting -- thanks for taking up the challenge. I still suspect that if
we ran the corresponding benchmark at the C level, the first form would
win, but it's a matter of hashing twice vs. creating a tuple -- both of
which have the wazoo optimized out of them so it could go either way. There
are many surprises possible here (e.g. long ago someone found that
`s.startswith('x')` is slower than `s[:1] == 'x'` and the reason is the
name lookup for `startswith`!).

> Anyway, of course it's too late to change. And there are probably other
>> "protocols" that check for the presence of keys and __getitem__(). Also, in
>> a sense keys() is more fundamental -- deriving keys() from items() would be
>> backwards (throwing away the values -- imagine a data type that stores the
>> values on disk).
>>
>
> Does there need to be a single defined "protocol" for a mapping (other
> than the ABC)? -- that is, would **unpacking be able to use .items() and
> keys() be used in other contexts?
>
> And why does ** unpacking need to check at all (LBYL) couldn't it simply
> do something like:
>
> {k: d[k] for k in d}
>
> sure, there could occasionally be a Sequence for which that would happen
> to work (like a range object for instance), but then it would be
> unlikely to result in the expected result anyway -- just like many other
> uses of Duck typing. Or not, and it could still be correct.
>

I don't understand why LBYL is considered such an anti-pattern. It helps
produce much clearer error messages in this case for users who are
exploring this feature, and distinguishing *early* between sequences and
mappings is important for that. Long ago we decided that the distinctive
feature is that mappings have a `keys()` method whereas sequences don't
(and users who add a `keys()` method to a sequence are just asking for
trouble). So that's what we use.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OQOGANKVMEBWTSPMNMOT7ZKCMLAMRBBU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Anton Abrosimov

So, I’m ready to admit that I was mistaken in considering `**` as an operator. 
Therefore, my further reasoning and suggestions were not correct.
If I had the right to vote, I would vote for `dict.update()` like behaviour.
Thanks for your attention and clarification.

I think it is worth creating a separate thread about the correct conversion of 
`dataclass -> dict`.
I will create it soon.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YUILM4N5TGRGUDCU5XVYLR5YRK6U7QH7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Guido van Rossum

On Tue, Dec 29, 2020 at 8:11 AM Steven D'Aprano  wrote:

> On Mon, Dec 28, 2020 at 11:36:55PM -0800, Christopher Barker wrote:
>
> > Side Question: when should one use __dict__ vs vars() vs getattr() ???
> all
> > three work in this case, but I'm never quite sure which is prefered, and
> > why.
>
> `vars(obj)` is defined as returning `obj.__dict__` so technically it
> probably doesn't matter, but I feel that as a matter of aesthetics and
> future-proofing, we should avoid direct use of dunders whenever
> practical.
>

To the contrary, vars() is something I added to the language for the
benefit of REPL users (like dir()), and other usages look suspect to me. I
find that using `__dict__` is more direct about the purpose, and also it is
the prevailing style.


> Do you prefer to write `mylist.__len__()` over `len(mylist)`? Then you
> will probably prefer `obj.__dict__` over `vars(obj)` too :-)
>

Not a valid analogy.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YBACDV7JURIR7YUS3JCC6QQYJLBJXQYX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-29 Thread Steven D'Aprano

On Mon, Dec 28, 2020 at 11:36:55PM -0800, Christopher Barker wrote:

> Side Question: when should one use __dict__ vs vars() vs getattr() ??? all
> three work in this case, but I'm never quite sure which is prefered, and
> why.

`vars(obj)` is defined as returning `obj.__dict__` so technically it 
probably doesn't matter, but I feel that as a matter of aesthetics and 
future-proofing, we should avoid direct use of dunders whenever 
practical.

Do you prefer to write `mylist.__len__()` over `len(mylist)`? Then you 
will probably prefer `obj.__dict__` over `vars(obj)` too :-)

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GNYDVF6VSXREPQJMQOG3CQARXAMT6MEQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-28 Thread Christopher Barker

On Mon, Dec 28, 2020 at 12:33 PM Guido van Rossum  wrote:

> On Mon, Dec 28, 2020 at 12:15 PM Christopher Barker 
> wrote:
>
>> Though frankly, I would rather have had it use .items() -- seems more
>> efficient to me, and you do need both the keys and the values, and items()
>> is just as much part of the Mapping API as keys.
>>
>
> There may be a (small) performance issue with that -- items() requires
> creating a tuple object for each key/value pair.
>

it does look like items() is a tad faster (dict with 1000 items), but not
enough to matter:

In [61]: %timeit {k: d[k] for k in d.keys()}
112 µs ± 1.02 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)

In [62]: %timeit {k: v for k, v in d.items()}
92.6 µs ± 1.9 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)

Anyway, of course it's too late to change. And there are probably other
> "protocols" that check for the presence of keys and __getitem__(). Also, in
> a sense keys() is more fundamental -- deriving keys() from items() would be
> backwards (throwing away the values -- imagine a data type that stores the
> values on disk).
>

Does there need to be a single defined "protocol" for a mapping (other than
the ABC)? -- that is, would **unpacking be able to use .items() and keys()
be used in other contexts?

And why does ** unpacking need to check at all (LBYL) couldn't it simply do
something like:

{k: d[k] for k in d}

sure, there could occasionally be a Sequence for which that would happen to
work (like a range object for instance), but then it would be unlikely to
result in the expected result anyway -- just like many other uses of Duck
typing. Or not, and it could still be correct.

But as you say -- too late the change now anyway.

To the OP: you suggested that you had, I think, four ways to make a
dataclass "unpackable", but none were satisfactory. How about this
decorator:

def make_mapping(cls):
def __getitem__(self, key):
if key in self.__dataclass_fields__:
return self.__dict__[key]
else:
raise KeyError(key)

def keys(self):
return self.__dataclass_fields__.keys()

cls.__getitem__ = __getitem__
cls.keys = keys

return cls

@make_mapping
@dataclasses.dataclass
class Point:
x: int
y: int

p = Point(1, 2)

print(p)

print({**p})
print(dict(p))

-CHB

Side Question: when should one use __dict__ vs vars() vs getattr() ??? all
three work in this case, but I'm never quite sure which is prefered, and
why.








>
>
>> But there is an argument that the ** operator should be able to be
>> supported only with dunder methods -- which could be done if it used the
>> iterator protocol to get the keys, rather than the keys() method, which
>> does not appear to work now. though to be fair, all you need to do to get
>> that is add a __len__ and derive from Mapping.
>>
>
> If we had to do it all over from scratch we would probably design mappings
> and sequences to be differentiably using dunders only. But it's about 31
> years too late for that. And looking at the mess JavaScript made of this
> (sequences are mappings with string keys "0", "1" and so on), I'm pretty
> happy with how Python did this.
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZPDXCZSYVPXMV54T54BUH233SYYFQ7FA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-28 Thread Jan Kaliszewski

2020-12-28 Christopher Barker  dixit:

> I don't know about the OP, but all I wanted was a clear definition of
> the part of the API needed to support **, and apparently it's a
> keys() method that returns an iterator of the keys, and a __getitem__
[...]

To be more precise: an *iterable* of the keys -- not necessarily an
*iterator*; it can be, for example, a list or string:

>>> def fun(**kwargs):print(kwargs)
... 
>>> class C:
... def keys(self): return list('abc')
... def __getitem__(self, key): return 42
... 
>>> c = C()
>>> fun(**c)
{'a': 42, 'c': 42, 'b': 42}

And, even, the `keys()` method does not need to be defined on the
class level -- it can be a callable attribute of an *instance*:

>>> class D:
... def __getitem__(self, key): return 42
... 
>>> d = D()
>>> d.keys = lambda: 'abc'
>>> fun(**d)
{'a': 42, 'c': 42, 'b': 42}

Cheers,
*j
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F3QHN73PMFEM3W67YYS4O3YYSJ6B7VX3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-28 Thread Guido van Rossum

On Mon, Dec 28, 2020 at 12:15 PM Christopher Barker 
wrote:

>
> I don't know about the OP, but all I wanted was a clear definition of the
> part of the API needed to support **, and apparently it's a keys() method
> that returns an iterator of the keys, and a __getitem__ that then returns
> the values associated with those keys. Which is fine.
>
> Though frankly, I would rather have had it use .items() -- seems more
> efficient to me, and you do need both the keys and the values, and items()
> is just as much part of the Mapping API as keys.
>

There may be a (small) performance issue with that -- items() requires
creating a tuple object for each key/value pair.

Anyway, of course it's too late to change. And there are probably other
"protocols" that check for the presence of keys and __getitem__(). Also, in
a sense keys() is more fundamental -- deriving keys() from items() would be
backwards (throwing away the values -- imagine a data type that stores the
values on disk).

> But there is an argument that the ** operator should be able to be
> supported only with dunder methods -- which could be done if it used the
> iterator protocol to get the keys, rather than the keys() method, which
> does not appear to work now. though to be fair, all you need to do to get
> that is add a __len__ and derive from Mapping.
>

If we had to do it all over from scratch we would probably design mappings
and sequences to be differentiably using dunders only. But it's about 31
years too late for that. And looking at the mess JavaScript made of this
(sequences are mappings with string keys "0", "1" and so on), I'm pretty
happy with how Python did this.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3DLPC2DLDTT4Y3NBQHLBIWCMS76CRC56/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-28 Thread Christopher Barker

On Mon, Dec 28, 2020 at 3:54 AM Steven D'Aprano  wrote:

> > Steven D'Aprano wrote:
> > > Why do you want something that isn't a mapping to be usable with
> mapping
> > > unpacking?
> >
> > I think mapping is not `abc.Mapping` class only.
>
> You don't have to inherit from Mapping for this to work. Double-star
> unpacking already supports duck-typing:
>
> >>> class MyMapping:
> ... def keys(self):
> ... return iter('abc')
> ... def __getitem__(self, key):
> ... if key in ('a', 'b', 'c'):
> ... return key.upper()
> ... raise KeyError
> ...
> >>> def demo(**kwargs):
> ... print(kwargs)
>

Thanks! this would have shortcutted this conversation if it come earlier.
The OP, and me, took the error message at its word, and also, in
experimenting, didn't happen on the right part of the Mapping API that
needed to be supported.

I don't know about the OP, but all I wanted was a clear definition of the
part of the API needed to support **, and apparently it's a keys() method
that returns an iterator of the keys, and a __getitem__ that then returns
the values associated with those keys. Which is fine.

Though frankly, I would rather have had it use .items() -- seems more
efficient to me, and you do need both the keys and the values, and items()
is just as much part of the Mapping API as keys.

But there is an argument that the ** operator should be able to be
supported only with dunder methods -- which could be done if it used the
iterator protocol to get the keys, rather than the keys() method, which
does not appear to work now. though to be fair, all you need to do to get
that is add a __len__ and derive from Mapping.

and to the OP's question a decorator that makes a Mapping from a dataclass
would be pretty easy to write.

-CHB


> >>> import collections.abc
> >>> issubclass(MyMapping, collections.abc.Mapping)
> False
> >>>
> >>> demo(**MyMapping())
> {'a': 'A', 'b': 'B', 'c': 'C'}
>
>
> So we already support duck-typing here.
>
> We can't use the fallback iteration interface:
>
>
> >>> class MyOtherMapping:
> ... def __iter__(self):
> ... return zip('xyz', 'XYZ')
> ...
> >>> demo(**MyOtherMapping())
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: __main__.demo() argument after ** must be a mapping, not
> MyOtherMapping
>
>
> so that's a possible enhancement: dict unpacking should(?) fall back on
> iteration of (key, value) pairs just as dict.update does. I would
> cautiously support adding that as an enhancement.
>
>
> > What about:
> > `Iterator[Tuple[str, int]]`
>
> I said I would *cautiously* support that, because there are some subtle
> issues to do with exceptions, but we can worry about that later.
>
>
> > ```
> > @dataclass
> > class MyMap:
> > x: int
> > y: int
> > ```
> >
> > Is this "mapping"?
>
> No:
>
> >>> issubclass(MyMap, collections.abc.Mapping)
> False
>
> It doesn't even duck-type as a mapping. It does not support len, keys or
> subscripting:
>
>
> >>> obj = MyMap(2, 3)
> >>> len(obj)
> TypeError: object of type 'MyMap' has no len()
> >>> obj.keys()
> AttributeError: 'MyMap' object has no attribute 'keys'
> >>> obj['x']
> TypeError: 'MyMap' object is not subscriptable
>
>
> so it is certainly not a mapping.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UYDIPMY2HXGL4OLEEFXBTZ2T4CK6TSVU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F2EV7VISEQWKLNLCG4T35LD6RDNZPHRE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-28 Thread Steven D'Aprano

On Mon, Dec 28, 2020 at 08:44:07AM -, Anton Abrosimov wrote:
> Steven D'Aprano wrote:
> > Why do you want something that isn't a mapping to be usable with mapping 
> > unpacking?
> 
> I think mapping is not `abc.Mapping` class only.

You don't have to inherit from Mapping for this to work. Double-star 
unpacking already supports duck-typing:


>>> class MyMapping:
... def keys(self):
... return iter('abc')
... def __getitem__(self, key):
... if key in ('a', 'b', 'c'):
... return key.upper()
... raise KeyError
... 
>>> def demo(**kwargs):
... print(kwargs)
... 
>>> import collections.abc
>>> issubclass(MyMapping, collections.abc.Mapping)
False
>>>
>>> demo(**MyMapping())
{'a': 'A', 'b': 'B', 'c': 'C'}


So we already support duck-typing here.

We can't use the fallback iteration interface:


>>> class MyOtherMapping:
... def __iter__(self):
... return zip('xyz', 'XYZ')
... 
>>> demo(**MyOtherMapping())
Traceback (most recent call last):
  File "", line 1, in 
TypeError: __main__.demo() argument after ** must be a mapping, not 
MyOtherMapping


so that's a possible enhancement: dict unpacking should(?) fall back on 
iteration of (key, value) pairs just as dict.update does. I would 
cautiously support adding that as an enhancement.


> What about:
> `Iterator[Tuple[str, int]]`

I said I would *cautiously* support that, because there are some subtle 
issues to do with exceptions, but we can worry about that later.


> ```
> @dataclass
> class MyMap:
> x: int
> y: int
> ```
> 
> Is this "mapping"?

No:

>>> issubclass(MyMap, collections.abc.Mapping)
False

It doesn't even duck-type as a mapping. It does not support len, keys or 
subscripting:


>>> obj = MyMap(2, 3)
>>> len(obj)
TypeError: object of type 'MyMap' has no len()
>>> obj.keys()
AttributeError: 'MyMap' object has no attribute 'keys'
>>> obj['x']
TypeError: 'MyMap' object is not subscriptable


so it is certainly not a mapping.



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UYDIPMY2HXGL4OLEEFXBTZ2T4CK6TSVU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

Steven D'Aprano wrote:
> On Mon, Dec 28, 2020 at 09:06:40AM -, Anton Abrosimov wrote:
> > Steven D'Aprano wrote:
> > You contradict yourself:
> > "I can implement any behaviour"
> > "I can't realize any other behaviour ..."
> > Which is correct?
> > I apologize for my english, I meant that I cannot implement the 
> > following behavior inside the class:
> > class MyClass:
> > def __iter__(self):
> > return self.items_for_iteration
> > def __unpack__(self):
> > return self.items_for_unpack
> > 
> > I have to make a separate method and have to rely on the user of the class.
> > Ah, now I understand what you mean: you want iteration and iterator 
> unpacking to do different things:
> obj = MyClass()
> list(obj)  # iteration
> # --> returns a b c d
> print(*obj)  # iterator unpacking
> # --> returns x y z
> 
> You can't do that, just like you can't make these different:
> items = list(obj)  # iteration
> 
> items = [item for item in obj]  # iteration in a comprehension
> 
> items = []
> for item in obj:  # iteration in a for-loop
> items.append(item)
> 
> And that is a good thing because it would be confusing and horrible 
> if iteration over an object was different depending on how you iterate 
> over it.
> We're not going to invent new dunder methods:
> def __iter__(self):
> 
> def __forloop__(self):
> 
> def __comprehension__(self):
> 
> so that they can be different, and I don't think we should invent a new 
> dunder method __unpack__ so it can be different from iteration.
> Iterator unpacking is just a form of iteration.

I agree with that. List unpacking is not a problem for me.
The only thought: If `*` is an operator as PEP 448 say then there must be a 
method for it.

The `**` behavior makes me sad.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ASX4TJJKXN6ORP3I64LUAUYFEQKCM6US/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-28 Thread Steven D'Aprano

On Mon, Dec 28, 2020 at 09:06:40AM -, Anton Abrosimov wrote:

> Steven D'Aprano wrote:
> > You contradict yourself:
> > "I can implement any behaviour"
> > "I can't realize any other behaviour ..."
> > Which is correct?
> 
> I apologize for my english, I meant that I cannot implement the 
> following behavior inside the class:
> 
> ```
> class MyClass:
> def __iter__(self):
> return self.items_for_iteration
> def __unpack__(self):
> return self.items_for_unpack
> ```
> I have to make a separate method and have to rely on the user of the class.

Ah, now I understand what you mean: you want iteration and iterator 
unpacking to do different things:

obj = MyClass()
list(obj)  # iteration
# --> returns a b c d
print(*obj)  # iterator unpacking
# --> returns x y z


You can't do that, just like you can't make these different:

items = list(obj)  # iteration

items = [item for item in obj]  # iteration in a comprehension

items = []
for item in obj:  # iteration in a for-loop
items.append(item)

And that is a **good thing** because it would be confusing and horrible 
if iteration over an object was different depending on how you iterate 
over it.

We're not going to invent new dunder methods:

def __iter__(self):

def __forloop__(self):

def __comprehension__(self):

so that they can be different, and I don't think we should invent a new 
dunder method `__unpack__` so it can be different from iteration.

Iterator unpacking is just a form of iteration.



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C5EUHU2OXRU5XSKN4I6L5SBNF3GUZFDE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

Chris Angelico wrote:
> Allow me to rephrase what I think you're arguing here, and you can
> tell me if I'm close to the mark.

You close to the mark. :)

Chris Angelico wrote:
> Given an object of a custom class C, you can make it usable as "x, y,
> z = C()" or "f(*C())" or anything else by defining __iter__, and in
> all ways that object will be iterable, unpackable, etc.

Yes, the only small problem:
```
class MyClass:
def __iter__(self):
return self.items_for_iteration
def __unpack__(self):
return self.items_for_unpack
```
This is not hurt.

Chris Angelico wrote:
> Given the same object, how can you ensure that it can be used as
> "f(**C())"? What about in "{}.update(C())"? Or "dict(C())"? Is there a
> single well-defined protocol that allows you to make your object
> usable in all mapping-like contexts?

Look at the example:

```
seq = zip('abc', 'xyz')
d = {}
d.update(seq)
print(d)
# {'a': 'x', 'b': 'y', 'c': 'z'}

def f(**kwargs):
  print(kwargs)

seq = zip('abc', 'xyz')
f(**seq)
# TypeError: f() argument after ** must be a mapping, not zip
```

`dict.update` does the job.
I think a single protocol is:

`x, y, z = C()`, `[i for i in C()]`, ... call `C.__iter__()`
`f(C())`: get C instance.
`f(*C())`: call `C.__unpack_args__(self) -> Iterator[Any]:`
`f(**C())`: call `C.__unpack_kwargs__(self) -> Iterator[Tuple[str, Any]]:`
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AFBMUM2R2PCR42LEIESBFU2C25APQVJI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

Steven D'Aprano wrote:
> On Sun, Dec 27, 2020 at 02:05:38PM -, Anton Abrosimov wrote:
> > 
> > *, ** are operators, but behaviorally they are methods or 
> > functions. I think this is the main problem.
> > 
> > No they aren't operators. They aren't in the operator precedence table, 
> and they don't have dunders associated with them:
> https://docs.python.org/3/reference/expressions.html#operator-precedence
> Nor can you use them in places you can use arbitrary expressions:
> >>> a, b, c  = *(1, 2, 3)
>   File "", line 1
> SyntaxError: can't use starred expression here

Hmm... PEP 448 -- Additional Unpacking Generalizations:
> This PEP proposes extended usages of the * iterable unpacking operator and ** 
> dictionary unpacking operators to allow unpacking in more positions, an 
> arbitrary number of times, and in additional circumstances. Specifically, in 
> function calls, in comprehensions and generator expressions, and in displays.

Steven D'Aprano wrote:
> You contradict yourself:
> "I can implement any behaviour"
> "I can't realize any other behaviour ..."
> Which is correct?

I apologize for my english, I meant that I cannot implement the following 
behavior inside the class:

```
class MyClass:
def __iter__(self):
return self.items_for_iteration
def __unpack__(self):
return self.items_for_unpack
```
I have to make a separate method and have to rely on the user of the class.

Steven D'Aprano wrote:
> What are you trying to do that you want to use iterator unpacking on 
> something but not make it an iterator?

How can I implement an unpackable dataclass without littering it with 
non-dunder methods?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6FGSOQE4AQPJLVOWFXR23GVC2HJB7VBY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-28 Thread Chris Angelico

On Mon, Dec 28, 2020 at 7:45 PM Anton Abrosimov  wrote:
>
> Steven D'Aprano wrote:
> > Why do you want something that isn't a mapping to be usable with mapping
> > unpacking?
>
> I think mapping is not `abc.Mapping` class only.
>
> What about:
> `Iterator[Tuple[str, int]]`
>
> ```
> @dataclass
> class MyMap:
> x: int
> y: int
> ```
>
> Is this "mapping"?
> In Python I can use `/` as path separator: `pathlib.Path.cwd() / 'my_dir'`. I 
> can control the behavior of my class.
> But I only have one way to unpack the object. Not a perfect way. `dict.update 
> ()` gives more freedom.
>
> Steven D'Aprano wrote:
> > Does it really hurt you to provide mapping methods when you get them
> > for free? Just inherit from Mapping.
>
> ```
> # first.py:
> @dataclass
> class Point2D(Mapping):
>
> # second.py
> @dataclass
> class Point3D(Point2D):
> ```
>
> Now I have to think about unnecessary public methods.

Allow me to rephrase what I *think* you're arguing here, and you can
tell me if I'm close to the mark.

Given an object of a custom class C, you can make it usable as "x, y,
z = C()" or "f(*C())" or anything else by defining __iter__, and in
all ways that object will be iterable, unpackable, etc.

Given the same object, how can you ensure that it can be used as
"f(**C())"? What about in "{}.update(C())"? Or "dict(C())"? Is there a
single well-defined protocol that allows you to make your object
usable in all mapping-like contexts?

If that's not what your point is, ignore this post :)

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C75H7LYS3ZLF44ION2K5JYIII7PGQH2M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

Steven D'Aprano wrote:
> Why do you want something that isn't a mapping to be usable with mapping 
> unpacking?

I think mapping is not `abc.Mapping` class only.

What about:
`Iterator[Tuple[str, int]]`

```
@dataclass
class MyMap:
x: int
y: int
```

Is this "mapping"?
In Python I can use `/` as path separator: `pathlib.Path.cwd() / 'my_dir'`. I 
can control the behavior of my class.
But I only have one way to unpack the object. Not a perfect way. `dict.update 
()` gives more freedom.

Steven D'Aprano wrote:
> Does it really hurt you to provide mapping methods when you get them 
> for free? Just inherit from Mapping.

```
# first.py:
@dataclass
class Point2D(Mapping):

# second.py
@dataclass
class Point3D(Point2D):
```

Now I have to think about unnecessary public methods.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NWWJI4UO44A3AJXDGWRKMUQJVLQRGIXU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-28 Thread Steven D'Aprano

On Sun, Dec 27, 2020 at 02:05:38PM -, Anton Abrosimov wrote:

> 1. `*`, `**` are operators, but behaviorally they are methods or 
> functions. I think this is the main problem.

No they aren't operators. They aren't in the operator precedence table, 
and they don't have dunders associated with them:

https://docs.python.org/3/reference/expressions.html#operator-precedence

Nor can you use them in places you can use arbitrary expressions:

>>> a, b, c  = *(1, 2, 3)
  File "", line 1
SyntaxError: can't use starred expression here

> 2. `*` nailed to `Iterable`, not so bad.
> 2.1. It uses the `__iter__` method. I can implement any behaviour.
> 2.2. I only see one problem. I can't realize any other behavior for iterating 
> and unpacking inside a custom class.

You contradict yourself:

"I can implement any behaviour"

"I can't realize any other behaviour ..."

Which is correct?

> 2.3. A new special method for unpacking is a better idea. By default, 
> this method should return `self.__iter__`. This will give control and 
> not break existing code.

You already have control. If you want to use iterator unpacking on an 
object, make it an iterator.

What are you trying to do that you want to use iterator unpacking on 
something but not make it an iterator?

Without a concrete example of why you need:

- to use iterator unpacking on something that isn't an iterator;

- and mapping unpacking on something that isn't a mapping;

all I can say is that his is needless over-generalization. As far as I 
can see, everything in this thread is YAGNI.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C4M54KBFFEVOOMP2C4XVDT2H3BUX4BEN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-27 Thread Steven D'Aprano

On Sat, Dec 26, 2020 at 11:23:16AM -, Anton Abrosimov wrote:

> I am trying to release comfortable dataclass unpacking using `**` 
> operator. Now I have 5 different ways to do it. But not a single good 
> one. Confused by the implementation of the unpacking operator.
> 
> So when I try to unpack any custom class, I get the error:
> 
> `type object argument after ** must be a mapping, not MyClass`
> 
> Ok, nothing special. I need to use `collections.abc.Mapping` right?
> Now I need to implement: `__getitem__`, `__iter__`, `__len__`. Not a problem.
> But additionally I get: `keys`, `items`, `values`.
> Hey, I don't need them. I don't need the full mapping functionality. I 
> only need the double asterisk to work.

Why do you want something that isn't a mapping to be usable with mapping 
unpacking?

Does it *really* hurt you to provide mapping methods when you get them 
for free? Just inherit from Mapping.



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/76XZD4JTDNPHDTJBDJWYXUWWQDUUKRDY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-27 Thread Steven D'Aprano

On Sat, Dec 26, 2020 at 09:18:09PM -0800, Brendan Barnwell wrote:

> >Yes it is documented:
> >
> > help(dict.update)
> >
> >and it was intentionally the inspiration for the behaviour of dict
> >augmented assignment.
> 
>   I see.  It's rather disturbing that that isn't mentioned in the docs 
>   on python.org.

"Disturbing"? It's an oversight, not a conspiracy :-)

Just submit a PR and I'm sure it will be accepted.


> >If you want to argue it's not a protocol, just an interface, okay, it's
> >an interface. That's a difference that makes no difference.
> 
>   No, it does make a difference.  What you're describing is the 
>   interface to a single existing type.

It's part of the MutableMapping ABC, although you need to read the 
source code of that (or the doctsring) to see it:

https://github.com/python/cpython/blob/3.9/Lib/_collections_abc.py


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BIUZTVKNPG74ADCLTFQWQVBGWLN77VUQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-27 Thread Anton Abrosimov

Christopher Barker wrote:
> My first thought is that for dataclasses, you can use the asdict() method,
> and you're done.

I thought in a similar way. I'll ask another (7K wtf  in 2 years, 3 month) 
question about converting a dataclass to dict.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Y3VLMQY3E7LPGIHN7P4WUJPMBZW37DLA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-27 Thread Anton Abrosimov

0. I believe that the `dict` behavior needs to be frozen. The change will break 
a lot of existing code, it's too much damage.
0.1. Yes, `keys` is not a good name for internal use, but that's okay.
0.2. If I want to make a class look like a `dict`, I understand that I will get 
`keys`, `items`... This is what I expect.
0.3. When I work with dicts in my code, I have a choice, I can use the default 
`dict`, or I can create my own dict-like class and implement different behavior.
0.4. `other[k] for k in other.keys()` default behaviour in `dict.update(other: 
Dict)` is a different big question about the degree of damage. Basically I can 
use `dict.update(dict.items())`.

Back to the stars:

1. `*`, `**` are operators, but behaviorally they are methods or functions. I 
think this is the main problem.
1.1. Python operators (mostly?) use their dunder methods to control their 
behavior.
1.2. Unpack operators are nailed to specific objects and their behavior, like 
an function or method. As a result, we lose control over them.

2. `*` nailed to `Iterable`, not so bad.
2.1. It uses the `__iter__` method. I can implement any behaviour.
2.2. I only see one problem. I can't realize any other behavior for iterating 
and unpacking inside a custom class.
2.3. A new special method for unpacking is a better idea. By default, this 
method should return `self.__iter__`. This will give control and not break 
existing code.

3. `**` nailed to `dict`. I think this is the fundamental problem.
3.1. `dict` is a good choice for the DEFAULT `kwargs` container. But `dict` is 
too excess for `**`. One method that returns an iterator is enough.
3.2. `**` use a `kwargs[k] for k in kwargs.keys()` like implementation. I have 
no control over this behavior.
3.3. I am forced to implement excessive functionality.
3.4. I must take userspace named `keys()`.
3.5. I cannot implement `keys` and `__getitem__` independent unpacking inside 
the class.

4. Which I think we can do.
4.1. Make `*` and `**` operators with their own methods.
4.2. Implement `return self .__ iter __ ()` as the default behavior of `*`.
4.3. Create a new implementation of the `**` operator expecting: `Iterator 
[Tuple [key, val]]`.
4.4. Implement `return ((k, self[k]) for k in self.keys())` as the specific 
behaviour of `dict`.
4.5. Create a `collections.abc` layout with an abstract two star unpacking 
method.
4.6. Update PEP 448.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XTNNF2ZW52KCT4MJW5POJWZLIXXZZJ4G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-26 Thread Serhiy Storchaka

26.12.20 13:23, Anton Abrosimov пише:
> I am trying to release comfortable dataclass unpacking using `**` operator. 
> Now I have 5 different ways to do it.
> But not a single good one. Confused by the implementation of the unpacking 
> operator.
> 
> So when I try to unpack any custom class, I get the error:
> 
> `type object argument after ** must be a mapping, not MyClass`
> 
> Ok, nothing special. I need to use `collections.abc.Mapping` right?
> Now I need to implement: `__getitem__`, `__iter__`, `__len__`. Not a problem.
> But additionally I get: `keys`, `items`, `values`.
> Hey, I don't need them. I don't need the full mapping functionality. I only 
> need the double asterisk to work.
> 
> Right, we have a duck typing!
> We throw out `abc.Mapping`. What do we need to implement?
> It's `__getitem__` and `keys`. Wtf `keys`?
> 
> I am looking at Python Data model: 
> https://docs.python.org/3/reference/datamodel.html
> There many operators, and they depend on special double underscore methods.
> Hmm, I don't see unpack operators there, it's strange.
> But why it's `keys`? Because the historical is `dict`?
> I think a dependency on `__iter__` is more preferable and expectable over a 
> userspace named `keys`.
> Actually, `items()` is more predictable.

See thread "Add __keys__ or __items__ protocol".
https://mail.python.org/archives/list/python-ideas@python.org/thread/A3CK7Y2ICIIVYKMAXSCUVYLK7IC7CH6G/#LCWMA6WMFLLSWXPKYC6FEUDHZBR3MYPE

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3AJOVRCP5J2YXS4GH66D6YQS52JHXF2T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

On Sun, Dec 27, 2020 at 4:30 PM Brendan Barnwell  wrote:
> That said. . . I'm starting to wonder why not just create a new dunder
> called __items__ and have dict alias that to .items().  Then the
> **-unpacking protocol could use that and everything would be fine, right?
>

+0.95.

If we could borrow the time machine and create this protocol way back
in Python's past, I think it'd be the best. The semantics of
maintaining backward compatibility MAY complicate things... but then
again, iterator protocol is a bit complicated too, and everyone's fine
with just defining __iter__.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/M5XJF4SI2FRTOEUIXVXJXD6IXWWBISR7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping


On 2020-12-26 21:02, Steven D'Aprano wrote:

On Sat, Dec 26, 2020 at 06:52:46PM -0800, Brendan Barnwell wrote:

On 2020-12-26 18:44, Steven D'Aprano wrote:
 >I think if we were designing mapping protocols now, that would be an
 >excellent idea, but we aren't, we have decades of history from `dict`
 >behind us. And protocols from dict use `keys()` and getitem. E.g.
 >update.
>>>
>>>What do you mean by "protocols from dict"?  What are these protocols?
>"And protocols from dict use `keys()` and getitem. E.g. update."

If I understand you right, that's not a protocol, that's just the
behavior of the dict type specifically.  As far as I can tell, it's not
even documented behavior, so it doesn't constrain anything.


Yes it is documented:

 help(dict.update)

and it was intentionally the inspiration for the behaviour of dict
augmented assignment.


	I see.  It's rather disturbing that that isn't mentioned in the docs on 
python.org.



If you want to argue it's not a protocol, just an interface, okay, it's
an interface. That's a difference that makes no difference.


	No, it does make a difference.  What you're describing is the interface 
to a single existing type.  A protocol is a framework that defines 
behavior for USER-DEFINED types to hook into, as the descriptor protocol 
lets you define __get__ or the iterator protocol lets you define 
__iter__.  The fact that dict uses a method with a particular name to do 
this or that should not constrain the creation of future protocols that 
define behavior for methods to be defined in user-created classes.


	That said. . . I'm starting to wonder why not just create a new dunder 
called __items__ and have dict alias that to .items().  Then the 
**-unpacking protocol could use that and everything would be fine, right?


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5UA5OOOCQIGAUKVLPQQI3Y2U2HV5FYAZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-26 Thread Steven D'Aprano

On Sat, Dec 26, 2020 at 06:52:46PM -0800, Brendan Barnwell wrote:
> On 2020-12-26 18:44, Steven D'Aprano wrote:
>  >I think if we were designing mapping protocols now, that would be an
>  >excellent idea, but we aren't, we have decades of history from `dict`
>  >behind us. And protocols from dict use `keys()` and getitem. E.g.
>  >update.
> >>>
> >>>   What do you mean by "protocols from dict"?  What are these protocols?
> >"And protocols from dict use `keys()` and getitem. E.g. update."
> 
>   If I understand you right, that's not a protocol, that's just the 
> behavior of the dict type specifically.  As far as I can tell, it's not 
> even documented behavior, so it doesn't constrain anything.

Yes it is documented:

help(dict.update)

and it was intentionally the inspiration for the behaviour of dict 
augmented assignment.

If you want to argue it's not a protocol, just an interface, okay, it's 
an interface. That's a difference that makes no difference.



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2RBFMYAZ4QA2W4ZITAZVECBOF6WZST2E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

On Sun, Dec 27, 2020 at 1:45 PM Greg Ewing  wrote:
>
> On 27/12/20 3:03 pm, Chris Angelico wrote:
> > But that would mean that a lot of iterables would look like mappings
> > when they're not.
>
> In the context of ** you're expecting a mapping, not a sequence.
>

Exactly; and under the proposal I was replying to, ANY object would
look like a mapping if you can iterate over it and subscript it -
which means all sequences look like mappings, just broken ones.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SRQFBK65CGYETKVHZPFCY6NNA4N73NBW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping


On 2020-12-26 18:44, Steven D'Aprano wrote:

> >I think if we were designing mapping protocols now, that would be an
> >excellent idea, but we aren't, we have decades of history from `dict`
> >behind us. And protocols from dict use `keys()` and getitem. E.g.
> >update.

>
>What do you mean by "protocols from dict"?  What are these protocols?

"And protocols from dict use `keys()` and getitem. E.g. update."


	If I understand you right, that's not a protocol, that's just the 
behavior of the dict type specifically.  As far as I can tell, it's not 
even documented behavior, so it doesn't constrain anything.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KBUBD4DE4D4EFRMRJDRYLQJXFXTYXY2V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-26 Thread Greg Ewing


On 27/12/20 3:03 pm, Chris Angelico wrote:

But that would mean that a lot of iterables would look like mappings
when they're not.


In the context of ** you're expecting a mapping, not a sequence.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7HSMEYDRGANI2XX6QBAMGP6IDNFK4JXK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

2020-12-26 Thread Steven D'Aprano

On Sat, Dec 26, 2020 at 06:09:42PM -0800, Brendan Barnwell wrote:
> On 2020-12-26 18:00, Steven D'Aprano wrote:

> >I think if we were designing mapping protocols now, that would be an
> >excellent idea, but we aren't, we have decades of history from `dict`
> >behind us. And protocols from dict use `keys()` and getitem. E.g.
> >update.
> 
>   What do you mean by "protocols from dict"?  What are these protocols?

"And protocols from dict use `keys()` and getitem. E.g. update."

The dict in-place union assignment operator also uses the same protocol:

>>> class A:
... def keys(self):
... return iter('abc')
... def __getitem__(self, key):
... return key.upper()
... 
>>> d = {}
>>> d |= A()
>>> d
{'a': 'A', 'b': 'B', 'c': 'C'}


(Regular union operator does not, it requires an actual dict.)


There may be others. I know I have written code that followed the same 
interface as update, although I don't have it easily at hand.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PO3EOV3Z6XADC4WIHO2XHQL4COOBLEVQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

On Sun, Dec 27, 2020 at 1:15 PM Brendan Barnwell  wrote:
>
> On 2020-12-26 18:03, Chris Angelico wrote:
> > On Sun, Dec 27, 2020 at 11:36 AM Greg Ewing  
> > wrote:
> >>
> >> On 27/12/20 10:15 am, Christopher Barker wrote:
> >> > It does seem like ** could be usable with any iterable that returns
> >> > pairs of objects. However the trick is that when you iterate a dict, you
> >> > get the keys, not the items, which makes me think that the only thing
> >> > you should *need* is an items() method that returns an iterable (pf
> >> > pairs of objects).
> >>
> >> It seems to me it would be more fundamental to use iteration to get
> >> the keys and indexing to get the corresponding values. You're only
> >> relying on dunder methods then.
> >>
> >
> > But that would mean that a lot of iterables would look like mappings
> > when they're not. Consider:
> >
>  def naive_items(x):
> > ... return [(key, x[key]) for key in x]
> > ...
>  naive_items(range(9, -1, -1))
> > [(9, 0), (8, 1), (7, 2), (6, 3), (5, 4), (4, 5), (3, 6), (2, 7), (1, 8), 
> > (0, 9)]
>
> I don't see that as a major problem.  It is no more "surprising" than
> doing something like list('abc') and getting ['a', 'b', 'c'].  If you do
> {**range(9, -1, -1)} you may get a result that looks strange or isn't
> useful, but as long as the result is consistent with the protocol,
> that's fine.  Just don't use **-unpacking with ranges if you don't want to.
>

Perhaps, but that means you can't raise TypeError for anything
iterable. Instead, you'd have to raise ValueError, because it could
potentially be valid. Are mappings really just iterables with indexing
(which most iterables support), or are they distinctly different?
Remember, most things iterate over their *values*, but a dict iterates
over its *keys*.

On the plus side, you COULD convert some objects into sparse lists by
basically just dictifying them:

>>> {**range(5)} # with this proposal
{0: 0, 1: 1, 2: 2, 3: 3, 4: 4}

But on the downside, you could only do this if they iterate the 'right
way', which most iterables won't. It would be MUCH more useful if
there were a dedicated way to request the valid keys or items, and use
that instead. Then you could convert *any* iterable that way.

At the moment, you can create an actual dict from any iterable by
enumerating, and that will give the correct items:

>>> def smarter_items(x):
... return list(dict(enumerate(x)).items())
...
>>> smarter_items(range(9, -1, -1))
[(0, 9), (1, 8), (2, 7), (3, 6), (4, 5), (5, 4), (6, 3), (7, 2), (8, 1), (9, 0)]
>>> smarter_items(range(10, 20))
[(0, 10), (1, 11), (2, 12), (3, 13), (4, 14), (5, 15), (6, 16), (7,
17), (8, 18), (9, 19)]

If you want to dictify something, that'd be the more normal way to do
it, IMO. Instead of something with lots of false positives, wouldn't
it be cleaner to have a protocol that specifically returns the
equivalent of dict.items()?

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/S24DU6DYBNFHMWIIDQU23YW5MI5OUU7U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping


On 2020-12-26 16:34, Greg Ewing wrote:

On 27/12/20 10:15 am, Christopher Barker wrote:

It does seem like ** could be usable with any iterable that returns
pairs of objects. However the trick is that when you iterate a dict, you
get the keys, not the items, which makes me think that the only thing
you should *need* is an items() method that returns an iterable (pf
pairs of objects).


It seems to me it would be more fundamental to use iteration to get
the keys and indexing to get the corresponding values. You're only
relying on dunder methods then.


	This is what I was thinking as well.  I don't like the idea of relying 
on a non-dunder method like .keys() to implement syntax like ** unpacking.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DTPGJ4HOZSHIEFKBINHMNW3XXXSD3OZS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping


On 2020-12-26 18:03, Chris Angelico wrote:

On Sun, Dec 27, 2020 at 11:36 AM Greg Ewing  wrote:


On 27/12/20 10:15 am, Christopher Barker wrote:
> It does seem like ** could be usable with any iterable that returns
> pairs of objects. However the trick is that when you iterate a dict, you
> get the keys, not the items, which makes me think that the only thing
> you should *need* is an items() method that returns an iterable (pf
> pairs of objects).

It seems to me it would be more fundamental to use iteration to get
the keys and indexing to get the corresponding values. You're only
relying on dunder methods then.



But that would mean that a lot of iterables would look like mappings
when they're not. Consider:


def naive_items(x):

... return [(key, x[key]) for key in x]
...

naive_items(range(9, -1, -1))

[(9, 0), (8, 1), (7, 2), (6, 3), (5, 4), (4, 5), (3, 6), (2, 7), (1, 8), (0, 9)]


	I don't see that as a major problem.  It is no more "surprising" than 
doing something like list('abc') and getting ['a', 'b', 'c'].  If you do 
{**range(9, -1, -1)} you may get a result that looks strange or isn't 
useful, but as long as the result is consistent with the protocol, 
that's fine.  Just don't use **-unpacking with ranges if you don't want to.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B6DRJ5YVCMXTY3E4R2RYJD6BSYV33PZL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping

On 2020-12-26 18:00, Steven D'Aprano wrote:

On Sun, Dec 27, 2020 at 01:34:02PM +1300, Greg Ewing wrote:

On 27/12/20 10:15 am, Christopher Barker wrote:
>It does seem like ** could be usable with any iterable that returns
>pairs of objects. However the trick is that when you iterate a dict, you
>get the keys, not the items, which makes me think that the only thing
>you should *need* is an items() method that returns an iterable (pf
>pairs of objects).

It seems to me it would be more fundamental to use iteration to get
the keys and indexing to get the corresponding values. You're only
relying on dunder methods then.

I think if we were designing mapping protocols now, that would be an
excellent idea, but we aren't, we have decades of history from `dict`
behind us. And protocols from dict use `keys()` and getitem. E.g.
update.

What do you mean by "protocols from dict"?  What are these protocols?

--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4HOWZPELGXVBP7UKKNUUZUMH7C7SA6ZZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpack operator "**" and Mapping