[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[MRAB]

On 2021-10-16 20:31, Erik Demaine wrote:

On Sun, 17 Oct 2021, Steven D'Aprano wrote:


On Sat, Oct 16, 2021 at 11:42:49AM -0400, Erik Demaine wrote:


I guess the question is whether to define `(*it for it in its)` to mean
tuple or generator comprehension or nothing at all.


I don't see why that is even a question. We don't have tuple
comprehensions and `(expr for x in items)` is always a generator, never
a tuple. There's no ambiguity there. Why would allowing unpacking turn
it into a tuple?


Agreed.  I got confused by the symmetry.


The only tricky corner case is that generator comprehensions can forgo
the surrounding brackets in the case of a function call:

   func( (expr for x in items) )
   func( expr for x in items )  # we can leave out the brackets

But with the unpacking operator, it is unclear whether the unpacking
star applies to the entire generator or the inner expression:

   func(*expr for x in items)

That could be read as either:

   it = (expr for x in items)
   func(*it)

or this:

   it = (*expr for x in items)
   func(it)

Of course we can disambiguate it with precedence rules, [...]


I'd be inclined to go that way, as the latter seems like the only reasonable
(to me) parse for that syntax.  Indeed, that's how the current parser
interprets this:

```
  func(*expr for x in items)
   ^
SyntaxError: iterable unpacking cannot be used in comprehension
```

To get the former meaning, which is possible today, you already need
parentheses, as in


   func(*(expr for x in items))




But it would be quite surprising for this minor issue to lead to the
major inconsistency of prohibiting unpacking inside generator comps when
it is allowed in list, dict and set comps.


Good point.  Now I'm much more inclined to define the generator expression
`(*expr for x in items)`.  Thanks for your input!


As we can already have:

func(*a, *b)

it's clear that * has a high precedence, so having:

func(*expr for x in items)

be equivalent to:

func((*expr for x in items))

does makes sense. It would be passing a generator.

On the other hand, wouldn't that make:

[*s for s in ["abc", "def", "ghi"]]

be equivalent to:

[(*s for s in ["abc", "def", "ghi"])]

? It would be making a list that contains a generator.

If:

[*s for s in ["abc", "def", "ghi"]]

should unpack, then shouldn't:

func(*expr for x in items)

also unpack?


On Sat, 16 Oct 2021, Serhiy Storchaka wrote:


It was considered and rejected in PEP 448. What was changed since? What
new facts or arguments have emerged?


I need to read the original discussion more (e.g.
https://mail.python.org/pipermail/python-dev/2015-February/138564.html), but
you can see the summary of why it was removed here:
https://www.python.org/dev/peps/pep-0448/#variations

In particular, there was "limited support" before (and the generator ambiguity
issue discussed above).  I expect now that we've gotten to enjoy PEP 448 for 5
years, it's more "obvious" that this functionality is missing and useful.  So
far that seems true (all responses have been at least +0), but if anyone
disagree, please say so.

Erik



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BWRSUO7AL4KFNWDHDR5SIWBGSFGGTUXH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Abdulla Al Kathiri]

Even though I am just a regular user of Python, +10 for me. It makes sense and 
I think I can easily teach it to people. However, (*expr for expr in its) 
should be generator expression and should be allowed to have nice mirror to all 
the un-packing comprehensions. It would be equivalent to: 
def gen(its):
for expr in its:
for x in expr:
yield x 
Abdulla 

Sent from my iPhone

> On 17 Oct 2021, at 12:28 AM, Guido van Rossum  wrote:
> 
> 
>> On Sat, Oct 16, 2021 at 12:21 PM David Mertz, Ph.D.  
>> wrote:
>> I'm not making any claims about tuple creation speed vs. list creation on 
>> microbenchmarks. It might we'll be 10% faster to create a million item tuple 
>> than a million item list. Or maybe the opposite, I don't know.
> 
> The thing to know about this (for everyone) is that creating a new tuple of 
> known size requires a single allocation while creating a new list always 
> requires two allocations. Where this really makes a difference is not when 
> you have a million items but when you create thousands or millions of objects 
> of modest size -- for lists it takes twice as many allocations.
> 
> If the number of items is not known there are different strategies depending 
> on what API you use; interestingly, the strategy deployed by 
> PySequence_Tuple() has a comment claiming it "can grow a bit faster than for 
> lists because unlike lists the over-allocation isn't permanent."
> 
> Finally, the bytecode generated for (*a, *b) creates a list first and then 
> turns that into a tuple (which will be allocated with the right size since 
> it's known at that point).
>  
> -- 
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him (why is my pronoun here?)
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/MNQBV6ORMV6DHUV44M524TCJHCLOBKI5/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RL6ZNWIFRL57OVO7ADAGKRN3X2OIBAGV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Guido van Rossum]

On Sat, Oct 16, 2021 at 12:21 PM David Mertz, Ph.D. 
wrote:

> I'm not making any claims about tuple creation speed vs. list creation on
> microbenchmarks. It might we'll be 10% faster to create a million item
> tuple than a million item list. Or maybe the opposite, I don't know.
>

The thing to know about this (for everyone) is that creating a new tuple of
known size requires a single allocation while creating a new list always
requires two allocations. Where this really makes a difference is not when
you have a million items but when you create thousands or millions of
objects of modest size -- for lists it takes twice as many allocations.

If the number of items is not known there are different strategies
depending on what API you use; interestingly, the strategy deployed by
PySequence_Tuple() has a comment claiming it "can grow a bit faster than
for lists because unlike lists the over-allocation isn't permanent."

Finally, the bytecode generated for (*a, *b) creates a list first and then
turns that into a tuple (which will be allocated with the right size since
it's known at that point).

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MNQBV6ORMV6DHUV44M524TCJHCLOBKI5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Erik Demaine]

On Sun, 17 Oct 2021, Steven D'Aprano wrote:


On Sat, Oct 16, 2021 at 11:42:49AM -0400, Erik Demaine wrote:


I guess the question is whether to define `(*it for it in its)` to mean
tuple or generator comprehension or nothing at all.


I don't see why that is even a question. We don't have tuple
comprehensions and `(expr for x in items)` is always a generator, never
a tuple. There's no ambiguity there. Why would allowing unpacking turn
it into a tuple?


Agreed.  I got confused by the symmetry.


The only tricky corner case is that generator comprehensions can forgo
the surrounding brackets in the case of a function call:

   func( (expr for x in items) )
   func( expr for x in items )  # we can leave out the brackets

But with the unpacking operator, it is unclear whether the unpacking
star applies to the entire generator or the inner expression:

   func(*expr for x in items)

That could be read as either:

   it = (expr for x in items)
   func(*it)

or this:

   it = (*expr for x in items)
   func(it)

Of course we can disambiguate it with precedence rules, [...]


I'd be inclined to go that way, as the latter seems like the only reasonable 
(to me) parse for that syntax.  Indeed, that's how the current parser 
interprets this:


```
func(*expr for x in items)
 ^
SyntaxError: iterable unpacking cannot be used in comprehension
```

To get the former meaning, which is possible today, you already need 
parentheses, as in



   func(*(expr for x in items))




But it would be quite surprising for this minor issue to lead to the
major inconsistency of prohibiting unpacking inside generator comps when
it is allowed in list, dict and set comps.


Good point.  Now I'm much more inclined to define the generator expression 
`(*expr for x in items)`.  Thanks for your input!



On Sat, 16 Oct 2021, Serhiy Storchaka wrote:


It was considered and rejected in PEP 448. What was changed since? What
new facts or arguments have emerged?


I need to read the original discussion more (e.g. 
https://mail.python.org/pipermail/python-dev/2015-February/138564.html), but 
you can see the summary of why it was removed here: 
https://www.python.org/dev/peps/pep-0448/#variations


In particular, there was "limited support" before (and the generator ambiguity 
issue discussed above).  I expect now that we've gotten to enjoy PEP 448 for 5 
years, it's more "obvious" that this functionality is missing and useful.  So 
far that seems true (all responses have been at least +0), but if anyone 
disagree, please say so.


Erik
--
Erik Demaine  |  edema...@mit.edu  |  http://erikdemaine.org/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DGPZMQXAZG55J4HLACIXMBZFCTEM6FPG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[David Mertz, Ph.D.]

On Sat, Oct 16, 2021, 12:08 PM Steven D'Aprano

> unpacking in *comprehensions*.
>
> # not currently permitted
> [*s for s in ["abc", "def", "ghi"]]
>
> > Moreover, it is an anti-pattern to create large and indefinite
> sized tuples,
>
> Is it? In what way?
>

As mentioned, I clipped the wrong part.

That said, even `(*it1, *it2, *it3)` feels like an anti-pattern in most
cases,  although syntactically defined. A hypothetical "tuple
comprehension" seems that much worse.

I'm not making any claims about tuple creation speed vs. list creation on
microbenchmarks. It might we'll be 10% faster to create a million item
tuple than a million item list. Or maybe the opposite, I don't know.

Rather, I'm concerned with readability and programmer expectations. Tuples
are best used as "records" of heterogeneous but structured data. This is
what makes namedtuples such an elegant extension. When I see a collection
of tuples, I generally expect each to have the same "shape", such as a
string at index 0, a float at index 1, and an int at index 2. If those
positions have attribute names, so much the better.

In contrast, lists (and iterators) I expect to contain many things that are
"the same" in a duck-type way. I usually want to loop through them and
perform the same operation on each (maybe with some switches, but in the
same block).

Having a million such similar items is commonplace. Having a million
*fields* is non-existent.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/M534EOJIH4GOSQMTMNQAFSCRJGOCZCKT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?

[Paul Moore]

On Sat, 16 Oct 2021 at 18:01, Steven D'Aprano  wrote:
>
> On Sat, Oct 16, 2021 at 02:49:42PM +0100, Paul Moore wrote:
>
> > I'd be more likely to just remove the types (the type checking
> > equivalent of `#noqa` when you don't agree with what your style
> > checker says).
>
> Type annotations are still useful to the human reader, even if the type
> checker is absent or wrong.
>
> I presume that mypy does support some "skip this" directive? If not, it
> should.

My impression (from the projects I've worked on using typing) was that
it didn't. But I checked and I was wrong: `# type: ignore` does
exactly that.
Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YLGIGKYKNMCKWFHGLONT7L4R67DDH7MX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Serhiy Storchaka]

16.10.21 17:07, Erik Demaine пише:
> Extended unpacking notation (* and **) from PEP 448 gives us great ways
> to concatenate a few iterables or dicts:
> 
> ```
> (*it1, *it2, *it3)  # tuple with the concatenation of three iterables
> [*it1, *it2, *it3]  # list with the concatenation of three iterables
> {*it1, *it2, *it3}  # set with the union of three iterables
> {**dict1, **dict2, **dict3}  # dict with the combination of three dicts
> # roughly equivalent to dict1 | dict2 | dict3 thanks to PEP 584
> ```
> 
> I propose (not for the first time) that similarly concatenating an
> unknown number of iterables or dicts should be possible via comprehensions:
> 
> ```
> (*it for it in its)  # tuple with the concatenation of iterables in 'its'
> [*it for it in its]  # list with the concatenation of iterables in 'its'
> {*it for it in its}  # set with the union of iterables in 'its'
> {**d for d in dicts} # dict with the combination of dicts in 'dicts'
> ```

It was considered and rejected in PEP 448. What was changed since? What
new facts or arguments have emerged?

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WLX7JPMEF3JGLMB4MS3LQKGIDMIL5KID/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?

[Steven D'Aprano]

On Sat, Oct 16, 2021 at 02:49:42PM +0100, Paul Moore wrote:

> I'd be more likely to just remove the types (the type checking
> equivalent of `#noqa` when you don't agree with what your style
> checker says).

Type annotations are still useful to the human reader, even if the type 
checker is absent or wrong.

I presume that mypy does support some "skip this" directive? If not, it 
should.

> But this is probably off-topic. I'm not 100% sure what the OP's
> proposal was, but as far as I can tell it seems to me that "dict[str,
> Number] is the correct usage for static typing" is the answer,
> regardless of mypy's ability to process it.

That's what it looks like to me too.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IP6OLOIFT46Y6DTBC6LKLPWHD2YEIVCE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Steven D'Aprano]

On Sat, Oct 16, 2021 at 11:42:49AM -0400, Erik Demaine wrote:

> I guess the question is whether to define `(*it for it in its)` to mean 
> tuple or generator comprehension or nothing at all.

I don't see why that is even a question. We don't have tuple 
comprehensions and `(expr for x in items)` is always a generator, never 
a tuple. There's no ambiguity there. Why would allowing unpacking turn 
it into a tuple?

It would be extremely surprising for `(*expr for x in items)` to return 
a tuple instead instead of a generator, or for it to be forbidden when 
unpacking versions of list/set/dict comprehensions are allowed.

Remember that it is *commas*, not the brackets, that make a tuple (the 
empty tuple excepted). The brackets around `(expr for x in items)` 
doesn't make it a tuple any more than the brackets in f(arg) makes a 
tuple.

[...]
> I'd be inclined to not define `(*it for it in its)`, given the ambiguity.

The only tricky corner case is that generator comprehensions can forgo 
the surrounding brackets in the case of a function call:

func( (expr for x in items) )
func( expr for x in items )  # we can leave out the brackets

But with the unpacking operator, it is unclear whether the unpacking 
star applies to the entire generator or the inner expression:

func(*expr for x in items)

That could be read as either:

it = (expr for x in items)
func(*it)

or this:

it = (*expr for x in items)
func(it)

Of course we can disambiguate it with precedence rules, in the same way 
that there is not really any ambiguity in `-3**2` (for example). Is it 
minus (3 squared) or (minus 3) squared? The precedence rules are clear 
that it absolutely is the first, but people do tend to read it wrong.

In this case, even if the precedence rules make the "unpacking generator 
comprehension inside a function call" case unambiguous, it *might* be 
clearer to require the extra brackets:

func(*(expr for x in items))
func((*expr for x in items))

are then clearly different. (The non-unpacking case of course can remain 
as it is today.)

But it would be quite surprising for this minor issue to lead to the 
major inconsistency of prohibiting unpacking inside generator comps when 
it is allowed in list, dict and set comps.



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DWMV4CVTRNVKRTGOB7L2XYDFAJHRUAPK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Steven D'Aprano]

On Sat, Oct 16, 2021 at 10:56:07AM -0400, David Mertz, Ph.D. wrote:
> POn Sat, Oct 16, 2021, 10:10 AM Erik Demaine
> 
> > (*it1, *it2, *it3)  # tuple with the concatenation of three iterables
> > [*it1, *it2, *it3]  # list with the concatenation of three iterables
> > {*it1, *it2, *it3}  # set with the union of three iterables
> > {**dict1, **dict2, **dict3}  # dict with the combination of three dicts
> 
> 
> I'm +0 on the last three of these.
> 
> But the first one is much more suggestive of a generator comprehension. I
> would want/expect it to be equivalent to itertools.chain(), not create a
> tuple.

Too late.

>>> (*"abc", *"def", *"ghi")
('a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i')

I think you may have missed that those examples of unpacking are 
existing functionality, not the proposal. The proposal is to allow 
unpacking in *comprehensions*.

# not currently permitted
[*s for s in ["abc", "def", "ghi"]]


> Moreover, it is an anti-pattern to create large and indefinite sized
> tuples,

Is it? In what way?

As far as I understand it, a large tuple is more memory efficient than a 
large list (it has no over-allocated space). The only issue that I know 
of is that if the length of the tuple is not known ahead of time, the 
interpreter may have to grow, or shrink, the underlying array before 
completing the tuple construction.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NY32M7DHURO75YXAFQYSYUEM6ENNAV4D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[David Mertz, Ph.D.]

On Sat, Oct 16, 2021, 11:42 AM Erik Demaine

> > But the first one is much more suggestive of a generator comprehension. I
> > would want/expect it to be equivalent to itertools.chain(), not create a
> > tuple.
>
> I guess you were referring to `(*it for it in its)` (proposed notation)
> rather
> than `(*it1, *it2, *it3)` (which already exists and builds a tuple).
>
> Very good point!  This is confusing.  I could also read `(*it for it in
> its)`
> as wanting to build the following generator (or something like it):
>

Oops. Yes. I trimmed the wrong part from my phone. What you write!

>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2DVYZFEIF3H7DJJD36XQRKQCSAFATB7C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Accessing target name at runtime

[Steven D'Aprano]

On Sat, Oct 16, 2021 at 09:19:26AM -0400, Erik Demaine wrote:

> To me (a mathematician), the existence of this magic in def, class, import, 
> etc. is a sign that this is indeed useful functionality.  As a fan of 
> first-class language features, it definitely makes me wonder whether it 
> could be generalized.

Obviously it is useful, because we have four major uses for it: imports, 
classes, functions and decorators. And I can think of at least two minor 
uses: the three argument form of type(), and namedtuple.

The question is, are any additional uses worth the effort and potential 
ugliness of generalising it?

Not every generalisation is worth it. Sometimes you can overgeneralise. 

https://www.joelonsoftware.com/2001/04/21/dont-let-architecture-astronauts-scare-you/

Until we have some compelling use-cases beyond the Big Four, I think 
this is a case of YAGNI.

https://www.martinfowler.com/bliki/Yagni.html

I've been thinking about use-cases for this feature for literally 
*years*, I've even got a handful of rough notes for a proto-PEP written 
down. To my embarrassment, today was the first time I realised that 
imports are also an example of this feature. So it is possible that 
there are many great use-cases for this and I just can't see them.


> But I'm not sure what the best mechanism is.

The mechanism is probably easy, if built into the compiler. When the 
compiler sees something that looks like a binding operation:

target = expression

it knows what the target is. If the expression contains some magic 
token, the compiler can substitute the target as a string for the magic 
token. (There is at least one other possible solution.)

So the implementation is probably easy. It is the design that is hard, 
and the justification lacking.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XVKKXFEI6ACCSECTTUYPVZ6KFU4ZACGE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Erik Demaine]

On Sat, 16 Oct 2021, David Mertz, Ph.D. wrote:


On Sat, Oct 16, 2021, 10:10 AM Erik Demaine
  (*it1, *it2, *it3)  # tuple with the concatenation of three
  iterables
  [*it1, *it2, *it3]  # list with the concatenation of three
  iterables
  {*it1, *it2, *it3}  # set with the union of three iterables
  {**dict1, **dict2, **dict3}  # dict with the combination of
  three dicts 

I'm +0 on the last three of these. 


But the first one is much more suggestive of a generator comprehension. I
would want/expect it to be equivalent to itertools.chain(), not create a
tuple.


I guess you were referring to `(*it for it in its)` (proposed notation) rather 
than `(*it1, *it2, *it3)` (which already exists and builds a tuple).


Very good point!  This is confusing.  I could also read `(*it for it in its)` 
as wanting to build the following generator (or something like it):


```
def generate():
for it in its:
yield from it
```

I guess the question is whether to define `(*it for it in its)` to mean tuple 
or generator comprehension or nothing at all.  Tuples are nice because they 
mirror `(*it1, *it2, *it3)` but bad for the reasons you raise:



Moreover, it is an anti-pattern to create large and indefinite sized tuples,
whereas such large collections as lists, sets, and dicts are common and
useful.


I'd be inclined to not define `(*it for it in its)`, given the ambiguity.

Assuming the support remains relatively unanimous for [*...], {*...}, and 
{**...} (thanks for all the quick replies!), I'll put together a PEP.


On Sat, 16 Oct 2021, Guido van Rossum wrote:


Seems sensible to me. I’d write the equivalency as

for x in y: answer.extend([…x…])


Oh, nice!  That indeed works in all cases.

Erik
--
Erik Demaine  |  edema...@mit.edu  |  http://erikdemaine.org/___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2AZBMZGKL56PERIJRCPTIJ6BRITTWHGM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Unpacking in tuple/list/set/dict comprehensions

[Stephen J. Turnbull]

Erik Demaine writes:


 > I propose (not for the first time) that similarly concatenating an unknown 
 > number of iterables or dicts should be possible via comprehensions:
 > 
 > ```
 > (*it for it in its)  # tuple with the concatenation of iterables in 'its'
 > [*it for it in its]  # list with the concatenation of iterables in 'its'
 > {*it for it in its}  # set with the union of iterables in 'its'
 > {**d for d in dicts} # dict with the combination of dicts in 'dicts'
 > ```

I don't have a problem with this, although that's moot if Guido doesn't.

 > There are other ways to do this, of course:
 > 
 > ```
 > [x for it in its for x in it]
 > itertools.chain(*its)
 > sum(it for it in its, [])
 > functools.reduce(operator.concat, its, [])
 > ```

The last three are more or less obscure, and the sum version looks
potentially very inefficient.  Nested comprehensions are quite awkward
because of the way both the occurances of 'x' and the occurances of
'it' are separated (and have to be).  Despite the redundancy the
suggestion seems plausible.

There was a thread during the summer (sorry, no reference, it's
bedtime) about something like container[*it] which uncovered some
subtleties about implementing unpacking notation.  I don't recall the
details, except that it had to do with the fact that in indexing the
indicies are passed as a tuple (even though there are no stdlib types
that take multiple indicies).  I think that means that it's not
relevant to your proposal, but you might want to check, especially for
tuple comprehensions like (*it for it in its).

 > 2. Maybe `list1.append(a, b)` should be equivalent to
 >`list1.extend([a, b])`?

[*it for it in its] has much greater appeal than multiargument
.append, which looks much like .extend, and .extend already exists.  I
think equivalencing *it and .extend(it) makes much more sense.


Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RITV6KM5DRGZ7F2F62DGKT3XY7AOHPJR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[David Mertz, Ph.D.]

POn Sat, Oct 16, 2021, 10:10 AM Erik Demaine

> (*it1, *it2, *it3)  # tuple with the concatenation of three iterables
> [*it1, *it2, *it3]  # list with the concatenation of three iterables
> {*it1, *it2, *it3}  # set with the union of three iterables
> {**dict1, **dict2, **dict3}  # dict with the combination of three dicts


I'm +0 on the last three of these.

But the first one is much more suggestive of a generator comprehension. I
would want/expect it to be equivalent to itertools.chain(), not create a
tuple.

Moreover, it is an anti-pattern to create large and indefinite sized
tuples, whereas such large collections as lists, sets, and dicts are common
and useful.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OZ3VBETDU2PSR7ZZ4UGYZKHCOIEEKK3Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Valentin Berlier]

+1

I think this is a very sensible proposal and I encountered the use-cases you 
mentioned several times.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5ILYEOUJY7U2RZ6CFVVFHUZD2UFUNUCE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Unpacking in tuple/list/set/dict comprehensions

[Guido van Rossum]

Seems sensible to me. I’d write the equivalency as

for x in y: answer.extend([…x…])

On Sat, Oct 16, 2021 at 07:11 Erik Demaine  wrote:

> Extended unpacking notation (* and **) from PEP 448 gives us great ways to
> concatenate a few iterables or dicts:
>
> ```
> (*it1, *it2, *it3)  # tuple with the concatenation of three iterables
> [*it1, *it2, *it3]  # list with the concatenation of three iterables
> {*it1, *it2, *it3}  # set with the union of three iterables
> {**dict1, **dict2, **dict3}  # dict with the combination of three dicts
> # roughly equivalent to dict1 | dict2 | dict3 thanks to PEP 584
> ```
>
> I propose (not for the first time) that similarly concatenating an unknown
> number of iterables or dicts should be possible via comprehensions:
>
> ```
> (*it for it in its)  # tuple with the concatenation of iterables in 'its'
> [*it for it in its]  # list with the concatenation of iterables in 'its'
> {*it for it in its}  # set with the union of iterables in 'its'
> {**d for d in dicts} # dict with the combination of dicts in 'dicts'
> ```
>
> The above is my attempt to argue that the proposed notation is natural:
> `[*it for it in its]` is exactly analogous to `[*its[0], *its[1], ...,
> *its[len(its)-1]]`.
>
> There are other ways to do this, of course:
>
> ```
> [x for it in its for x in it]
> itertools.chain(*its)
> sum(it for it in its, [])
> functools.reduce(operator.concat, its, [])
> ```
>
> But none are as concise and (to me, and hopefully others who understand *
> notation) as intuitive.  For example, I recently wanted to write a
> recursion
> like so, which accumulated a set of results from within a tree structure:
>
> ```
> def recurse(node):
># base case omitted
>return {*recurse(child) for child in node.children}
> ```
>
> In fact, I am teaching a class and just asked a question on a written exam
> for
> which several students wrote this exact code in their solution (which
> inspired
> writing this message).  So I do think it's quite intuitive, even to those
> relatively new to Python.
>
> Now, on to previous proposals.  I found this thread from 2016 (!); please
> let
> me know if there are others.
>
>
> https://mail.python.org/archives/list/python-ideas@python.org/thread/SBM3LYESPJMI3FMTMP3VQ6JKKRDHYP7A/#DE4PCVNXBQJIGFBYRB2X7JUFZT75KYFR
>
> There are several arguments for and against this feature in that thread.
> I'll
> try to summarize:
>
> Arguments for:
>
> * Natural extension to PEP 448 (it's mentioned as a variant within PEP 448)
>
> * Easy to implement: all that's needed in CPython is to *remove* some code
> blocking this.
>
> Arguments against:
>
> * Counterintuitive (to some)
>
> * Hard to teach
>
> * `[...x... for x in y]` is no longer morally equivalent to
> `answer = []; for x in y: answer.append(...x...)`
> (unless `list1.append(a, b)` were equivalent to `list1.extend([a, b])`)
>
> Above I've tried to counter the first two "against" arguments.
> Some counters to the third "against" argument:
>
> 1. `[*...x... for x in y]` is equivalent to
> `answer = []; for x in y: answer.extend(...x...)`
> (about as easy to teach, I'd say)
>
> 2. Maybe `list1.append(a, b)` should be equivalent to `list1.extend([a,
> b])`?
> It is in JavaScript (`Array.push`).  And I don't see why one would expect
> it to append a tuple `(a, b)`; that's what `list1.append((a, b))` is for.
> I think the main argument against this is to avoid programming errors,
> which is fine, but I don't see why it should hold back an advanced feature
> involving both unpacking and comprehensions.
>
> Erik
> --
> Erik Demaine  |  edema...@mit.edu  |  http://erikdemaine.org/
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/7G732VMDWCRMWM4PKRG6ZMUKH7SUC7SH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CQPULNM6PM623PLXF5Z63BIUZGOSQEKW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Unpacking in tuple/list/set/dict comprehensions

[Erik Demaine]

Extended unpacking notation (* and **) from PEP 448 gives us great ways to 
concatenate a few iterables or dicts:


```
(*it1, *it2, *it3)  # tuple with the concatenation of three iterables
[*it1, *it2, *it3]  # list with the concatenation of three iterables
{*it1, *it2, *it3}  # set with the union of three iterables
{**dict1, **dict2, **dict3}  # dict with the combination of three dicts
# roughly equivalent to dict1 | dict2 | dict3 thanks to PEP 584
```

I propose (not for the first time) that similarly concatenating an unknown 
number of iterables or dicts should be possible via comprehensions:


```
(*it for it in its)  # tuple with the concatenation of iterables in 'its'
[*it for it in its]  # list with the concatenation of iterables in 'its'
{*it for it in its}  # set with the union of iterables in 'its'
{**d for d in dicts} # dict with the combination of dicts in 'dicts'
```

The above is my attempt to argue that the proposed notation is natural:
`[*it for it in its]` is exactly analogous to `[*its[0], *its[1], ..., 
*its[len(its)-1]]`.


There are other ways to do this, of course:

```
[x for it in its for x in it]
itertools.chain(*its)
sum(it for it in its, [])
functools.reduce(operator.concat, its, [])
```

But none are as concise and (to me, and hopefully others who understand * 
notation) as intuitive.  For example, I recently wanted to write a recursion 
like so, which accumulated a set of results from within a tree structure:


```
def recurse(node):
  # base case omitted
  return {*recurse(child) for child in node.children}
```

In fact, I am teaching a class and just asked a question on a written exam for 
which several students wrote this exact code in their solution (which inspired 
writing this message).  So I do think it's quite intuitive, even to those 
relatively new to Python.


Now, on to previous proposals.  I found this thread from 2016 (!); please let 
me know if there are others.


https://mail.python.org/archives/list/python-ideas@python.org/thread/SBM3LYESPJMI3FMTMP3VQ6JKKRDHYP7A/#DE4PCVNXBQJIGFBYRB2X7JUFZT75KYFR

There are several arguments for and against this feature in that thread.  I'll 
try to summarize:


Arguments for:

* Natural extension to PEP 448 (it's mentioned as a variant within PEP 448)

* Easy to implement: all that's needed in CPython is to *remove* some code 
blocking this.


Arguments against:

* Counterintuitive (to some)

* Hard to teach

* `[...x... for x in y]` is no longer morally equivalent to
`answer = []; for x in y: answer.append(...x...)`
(unless `list1.append(a, b)` were equivalent to `list1.extend([a, b])`)

Above I've tried to counter the first two "against" arguments.
Some counters to the third "against" argument:

1. `[*...x... for x in y]` is equivalent to
`answer = []; for x in y: answer.extend(...x...)`
(about as easy to teach, I'd say)

2. Maybe `list1.append(a, b)` should be equivalent to `list1.extend([a, b])`?
It is in JavaScript (`Array.push`).  And I don't see why one would expect
it to append a tuple `(a, b)`; that's what `list1.append((a, b))` is for.
I think the main argument against this is to avoid programming errors,
which is fine, but I don't see why it should hold back an advanced feature
involving both unpacking and comprehensions.

Erik
--
Erik Demaine  |  edema...@mit.edu  |  http://erikdemaine.org/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7G732VMDWCRMWM4PKRG6ZMUKH7SUC7SH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?

[Paul Moore]

On Sat, 16 Oct 2021 at 14:07, Alex Waygood  wrote:
>
> Indeed — we essentially lie to mypy about the method resolution order for 
> list, dict, etc (mypy thinks that list directly inherits from 
> collections.abc.MutableSequence — see the typeshed stub here: 
> https://github.com/python/typeshed/blob/32bc2161a107db20c2ebc85aad31c29730db3e38/stdlib/builtins.pyi#L746),
>  but numeric ABCs are not special-cased by typeshed in the same way as 
> collections ABCs. Fundamentally, mypy has no knowledge about virtual 
> subclassing.

IMO, that's a bug in mypy. Maybe the fix is "too hard", maybe it
isn't. But if we added runtime isinstance checks, they would pass, so
arguing that the type is wrong or that the variable "needs" to be
typed differently is incorrect.

I guess my argument here is that flagging an error when you're not
100% sure the code is wrong is a problem - "in the case of ambiguity,
refuse the temptation to guess" seems relevant here. Personally, I'd
be unwilling to mess around with the typing in a situation like this -
I'd be more likely to just remove the types (the type checking
equivalent of `#noqa` when you don't agree with what your style
checker says). I'd much prefer mypy to miss the odd problem rather
than flagging usages that aren't actually errors, precisely because
you don't (as far as I know) have a mechanism for telling it you know
what you're doing...

But this is probably off-topic. I'm not 100% sure what the OP's
proposal was, but as far as I can tell it seems to me that "dict[str,
Number] is the correct usage for static typing" is the answer,
regardless of mypy's ability to process it.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WNR3A6YSC4PPLMZOTAYRFO5DYZ6PXQYX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Accessing target name at runtime

[Erik Demaine]

On Sat, 16 Oct 2021, Steven D'Aprano wrote:


The token should preferably be:

* self-explanatory, not line-noise;

* shorter rather than longer, otherwise it is easier to just
 type the target name as a string: 'x' is easier to type than
 NAME_OF_ASSIGNMENT_TARGET;

* backwards compatible, which means it can't be anything that
 is already a legal name or expression;

* doesn't look like an error or typo.


A possible soft keyword: __lhs__ (short for 'left-hand side'):


REGION = os.getenv(__lhs__)
db_url = config[REGION][__lhs__]


It's not especially short, and it's not backward-compatible,
but at least there's a history of adding double-underscore things.
Perhaps, for backward compatibility, the feature could be disabled in any 
scope (or file?) where __lhs__ is assigned, in which case it's treated like a 
variable as usual.  The magic version only applies when it's used in a 
read-only fashion.  It's kind of like a builtin variable, but its value 
changes on every line (and it's valid only in an assignment line).


One thing I wonder: what happens if you write the following?


foo[1] = __lhs__  # or <<< or whatever


Maybe you get 'foo[1]', or maybe this is invalid syntax, in the same way that 
the following is.



def foo[1]: pass



Classes, functions, decorators and imports already satisfy the "low
hanging fruit" for this functionality. My estimate is that well over 99%
of the use-cases for this fall into just four examples, which are
already satisfied by the interpreter:
[...]
# like func = decorator(func)
# similarly for classes
@decorator
def func(): ...


This did get me wondering about how you could simulate this feature with 
decorators.  Probably obvious, but here's the best version I came up with:


```
def env_var(x):
return os.getenv(x.__name__)

@env_var
def REGION(): pass
```

It's definitely ugly to avoid repetition...  Using a class, I guess we could 
at least get several such variables at once.



If we didn't already have interpreter support for these four cases, it
would definitely be worth coming up with a solution. But the use-cases
that remain are, I think, quite niche and uncommon.


To me (a mathematician), the existence of this magic in def, class, import, 
etc. is a sign that this is indeed useful functionality.  As a fan of 
first-class language features, it definitely makes me wonder whether it could 
be generalized.


But I'm not sure what the best mechanism is.  (From the description in the 
original post, I gather that variable assignment decorators didn't work out 
well.)  I wonder about some generalized mechanism for automatically setting 
the __name__ of an assigned object (like def and class), but I'm not sure what 
it would look like...


Erik
--
Erik Demaine  |  edema...@mit.edu  |  http://erikdemaine.org/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BHGDRTX3BBYB66NINSTOPROTCIRKZNRU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?

[Alex Waygood]

Indeed — we essentially lie to mypy about the method resolution order for 
list, dict, etc (mypy thinks that list directly inherits from 
collections.abc.MutableSequence — see the typeshed stub here: 
https://github.com/python/typeshed/blob/32bc2161a107db20c2ebc85aad31c29730db3e38/stdlib/builtins.pyi#L746),
 but numeric ABCs are not special-cased by typeshed in the same way as 
collections ABCs. Fundamentally, mypy has no knowledge about virtual 
subclassing.

Again, I covered this very comprehensively on StackOverflow ;) 
https://stackoverflow.com/questions/69334475/how-to-hint-at-number-types-i-e-subclasses-of-number-not-numbers-themselv/69383462#69383462

Best,
Alex 

> On 16 Oct 2021, at 13:54, Steven D'Aprano  wrote:
> 
> On Sat, Oct 16, 2021 at 09:54:13AM +0100, Alex Waygood wrote:
 On 16 Oct 2021, at 06:13, Steven D'Aprano  wrote:
>>> Be careful about believing what you are told.
>> Indeed, MyPy will correctly raise errors if you assign {None: []} to a
>> variable annotated with dict[str, Number]. However, you'll find that
>> MyPy also raises an error if you assign {'foo': 4} to a variable
>> annotated with dict[str, Number]:
> 
> Hah, serves me right for not testing it for both positive and negative 
> cases.
> 
> In my very limited testing now, I see that the problem appears to be 
> with the Number type. mypy correctly accepts this:
> 
>   Data = Dict[str, int]
>   a: Data = {'spam': 42}
> 
> (no errors), but if you change the annotation to use Number instead of 
> int, it wrongly flags that as a type error.
> 
> Possibly mypy doesn't know that ints and floats are instances of Number?
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/FJY3QKAUJOJYTODMZH72X2ZS63GKT6AN/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JFS65SMVETZ7W4KARJYJUNP6KA5XOFPL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?

[Steven D'Aprano]

On Sat, Oct 16, 2021 at 09:54:13AM +0100, Alex Waygood wrote:
> > On 16 Oct 2021, at 06:13, Steven D'Aprano  wrote:
> > Be careful about believing what you are told.
> 
> Indeed, MyPy will correctly raise errors if you assign {None: []} to a 
> variable annotated with dict[str, Number]. However, you'll find that 
> MyPy also raises an error if you assign {'foo': 4} to a variable 
> annotated with dict[str, Number]:

Hah, serves me right for not testing it for both positive and negative 
cases.

In my very limited testing now, I see that the problem appears to be 
with the Number type. mypy correctly accepts this:

Data = Dict[str, int]
a: Data = {'spam': 42}

(no errors), but if you change the annotation to use Number instead of 
int, it wrongly flags that as a type error.

Possibly mypy doesn't know that ints and floats are instances of Number?

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FJY3QKAUJOJYTODMZH72X2ZS63GKT6AN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Antichecked exceptions

[Stephen J. Turnbull]

Soni L. writes:

 > Can we call this "antichecked exceptions"?

I don't see why not.  Personally I'd prefer calling it "exception
filtering" since it's not a property of the exception, but a pattern
in the calling code.  But I'm only -0 on "antichecked" and +0 on
"filtered".

 > (Is this the wrong list for this?

It's not wrong.

But unless you can get a more general PEP than 479, folks who feel
like you do are going to have to watch All The Lists for different
terminology, and propose "antichecked" in any threads where it comes
up.  A different term such as "checked exceptions" could establish
itself while you're not looking.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VIHD76N4NO5RWNAHXEMVDXYL5K2TTZ7B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?

[Alex Waygood]

> On 16 Oct 2021, at 06:13, Steven D'Aprano  wrote:
> Be careful about believing what you are told.

Indeed, MyPy will correctly raise errors if you assign {None: []} to a variable 
annotated with dict[str, Number]. However, you'll find that MyPy also raises an 
error if you assign {'foo': 4} to a variable annotated with dict[str, Number]:

https://mypy-play.net/?mypy=latest&python=3.10&gist=de7faea3b4d6a0ffefc64acf3bfa7b77

The issue isn't a matter of false negatives from Mypy; it's a matter of false 
positives. For more, see the (rather long — sorry!) answer I gave Sebastian to 
a previous question of his on Stack Overflow:

https://stackoverflow.com/questions/69334475/how-to-hint-at-number-types-i-e-subclasses-of-number-not-numbers-themselv/69383462#69383462

Having said that, I absolutely agree that type-hinting doesn't seem to have 
much to do with this specific problem, and that a UserDict subclass is probably 
the way to go here.

Best,
Alex 

> On 16 Oct 2021, at 06:13, Steven D'Aprano  wrote:
> 
> On Fri, Oct 15, 2021 at 06:17:12PM +0200, Sebastian M. Ernst wrote:
> 
>> Data = Dict[str, Number]
>> 
>> @typechecked
>> def foo(bar: Data):
>>print(bar)
>> ```
>> 
>> Yes, this is using run-time checks (typeguard), which works just fine.
>> Only strings as keys and Number objects as values are going through. (I
>> was told that MyPy does not get this at the moment.)
> 
> Be careful about believing what you are told.
> 
> 
>[steve ~]$ cat test.py 
>from numbers import Number
>from typing import Dict
>Data = Dict[str, Number]
>a: Data = {None: []}
> 
>[steve ~]$ mypy test.py 
>test.py:4: error: Dict entry 0 has incompatible type "None": 
>"List[]"; expected "str": "Number"
>Found 1 error in 1 file (checked 1 source file)
> 
> 
>> The issue is that `bar` itself still allows "everything" to go in (and out):
>> 
>> ```python
>> @typechecked
>> def foo2(bar: Data):
>>bar[1.0] = b'should not be allowed'
>> ```
> 
> The Python interpreter intentionally doesn't know or care about 
> typing annotations. The interpreter should absolutely allow that.
> 
> However mypy correctly flags that assignment as a type error.
> 
> How about typeguard? That's for the typeguard developers to answer, but 
> my guess is that they will say that the typechecked decorator can 
> enforce type checking of the input parameters and perhaps even the 
> return result, but there is nothing they can do about what happens 
> inside the body of the function.
> 
> 
>> PEP 589 introduces typed dictionaries, but for a fixed set of predefined
>> keys (similar to struct-like constructs in other languages). In
>> contrast, I am looking for an arbitrary number of typed keys/value pairs.
> 
> Something like this?
> 
>from numbers import Number
>from collections import UserDict
> 
>class RestrictedDict(UserDict):
>def __setitem__(self, key, value):
>if not isinstance(key, str):
>raise TypeError('key must be a string')
>if not isinstance(value, Number):
>raise TypeError('value must be a number')
>super().__setitem__(key, value)
> 
> 
> (I think that is the only method that needs to be overloaded.)
> 
> 
> -- 
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/HLGDIWGHSIG3LDSBN7RDARK6E7VYTXQT/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NJQD6TKI5EIWBIAFKOL5L7SNHJX2KK7A/
Code of Conduct: http://python.org/psf/codeofconduct/