[Python-ideas] Re: Type checking for **kwargs based on its use
Hi Mauricio, I'd advise starting a discussion on typing-sig ( https://mail.python.org/archives/list/typing-...@python.org/) or creating an issue on the python/typing repo (https://github.com/python/typing). The folks interested in typing generally hang out in those places, so those are usually the best places to discuss new typing features. Best wishes, Alex On Fri, Jun 17, 2022 at 2:18 PM Mauricio Villegas via Python-ideas < python-ideas@python.org> wrote: > Look at the following example: > > ``` > def some_function(f1: int): > assert isinstance(f1, int) > > def other_function(f0: str, **kwargs): > assert isinstance(f0, str) > some_function(**kwargs) > > other_function(f0='a', f1='b') > ``` > > I would expect a static type checker to warn that f1='b' is wrong because > the type should be int. There shouldn't be a need to add a type to **kwargs > since what is accepted can be deduced from how it is used. Note that > some_function could be defined in another module and be used in multiple > places, thus the "don't repeat yourself" and "separation of concerns" > principles apply. Better to have the type for f1 be only in the definition > of some_function and not in the **kwargs. > > I created a github issue in pyright ( > https://github.com/microsoft/pyright/issues/3583) and the response was > that PEP 484 forbids this. I have seen that there are discussions about > TypedDict for more precise typing of **kwargs. However, requiring to use > TypedDict for the cases in which it is possible to derive what **kwargs > accepts based on its use, seems over-complicating things, which is yet > another principle "keep it simple, stupid". > > I would think that it wouldn't be too problematic to have a PEP stating > that static type checkers when **kwargs has not type annotation are allowed > to analyze the source code to identify what **kwargs accepts. I would even > say that when **kwargs has a type, checkers are allowed to analyze how it > is used. Also that the type annotation for **kwargs be not necessary if it > is possible to deduce what it accepts. Anyway, I am just giving out this > idea to see what happens. > > As a side note, I have been developing jsonargparse which can be used for > automatically creating complex parsers based on signatures ( > https://jsonargparse.readthedocs.io/en/stable/#classes-methods-and-functions). > I do plan to support identifying what **kwargs accepts based on its use. > See all the cases currently supported > https://jsonargparse.readthedocs.io/en/latest/#ast-resolver. > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/TEDDC7FU7TR6AESZT2NLHUYCKIJYFZTD/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NI4DXOYCTA3TICFUT2MI4R7Y7MBDWV3E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Adding pep8-casing-compliant aliases for the entire stdlib
I'm a -1 on this proposal, as I don't see any way of doing it that wouldn't cause a huge amount of disruption. Yes, the situation — especially with regard to unittest and logging — is far from ideal. But, it's what we've got.However I'm -100 on doing something like this while there already exist a large number of camelcase typing-module aliases for classes found elsewhere in the stdlib. It's already confusing enough to have `builtins.list` and `typing.List` be different objects. Under this proposal, we'd have `builtins.list`, `builtins.List`, and `typing.List`. Two of these objects would have the same identity, but it wouldn't be the two that are spelt the same. To my mind, this would be appalling.Yes, the aliases in the typing module are all deprecated, but they're still very widely used.Best, Alex Original message From: Matt del Valle Date: 11/11/2021 13:44 (GMT+00:00) To: Python-Ideas Subject: [Python-ideas] Adding pep8-casing-compliant aliases for the entire stdlib So I was reading the docs for the `threading` module and I stumbled upon this little note: Note: In the Python 2.x series, this module contained camelCase names for some methods and functions. These are deprecated as of Python 3.10, but they are still supported for compatibility with Python 2.5 and lower.And it got me thinking.Given that there is some precedent, would it be feasible to make a concerted effort to add aliases across the board for all public-facing stdlib types and functions that don't follow pep8-recommended casing?I realize that large chunks of the stdlib predates pep8 and therefore use various non-uniform conventions. For example, the logging module is fully camelCased, and many core types like `str` and `list` don't use PascalCase as pep8 recommends. The `collections` module is a veritable mosaic of casing conventions, with some types like `deque` and `namedtuple` being fully lowercased while others like `Counter` and `ChainMap` are PascalCased.My motivation for this twofold:1) I'll confess that this has just always been a wart that has bothered me way more than it has any right to. I just hate it. Somewhere deep inside my lizard-brain it makes me unhappy to have disparate naming conventions in my code. I realize this isn't a good reason in and of itself but I wonder if this might not be the case for others as well. While I've come to accept it because that's just how it is, maybe it doesn't have to be this way?2) It's always been an extra thing to explain when teaching python to someone. I always try to cover pep8 very early to discourage people I'm training from internalizing bad habits, and it means you have to explain that the very standard library itself contains style violations that would get flagged in most modern code reviews, and that they just have to keep in mind that despite the fact that the core language does it, they should not.So the scope of my suggestion is as follows:- lowercase types become PascalCase (e.g., `str` -> `Str`, `collections.defaultdict` -> `collections.DefaultDict`)- lowercase attributes/functions/methods become snake_case (no changes for names that only contain a single word, so `str.lower()` would be unaffected, but `str.removeprefix()` would get the alias `str.remove_prefix()`)- pep8 and the python docs are updated to state that the pep8-compliant forms of stdlib names should be strongly preferred over the legacy names, and that IDEs and linters should include (configurable?) weak warnings to discourage the use of legacy-cased stdlib names- `help()` would be special-cased for builtin types to no longer display any current non-pep8-compliant names, and the python docs would also no longer show them, instead only making a note at the top of the page as with the `threading` module.Given the horrors of the python 2.7 schism I don't think there's any rush to officially deprecate or remove the current non-pep8 names at all. I think that's the sort of thing that can happily and fully be kicked down the road. If we add aliases and they see widespread adoption to the point where the non-pep8 forms are barely ever even seen out in the wild then maybe in 10 or 20 years time when the steering council is deliberating on a new major python version they can consider rolling the removal of legacy badly-cased names into it. And if not then no big deal.So yeah, thoughts? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/6667GUFX36JTGD433KPPOPWVQV444GNW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?
Indeed — we essentially lie to mypy about the method resolution order for
list, dict, etc (mypy thinks that list directly inherits from
collections.abc.MutableSequence — see the typeshed stub here:
https://github.com/python/typeshed/blob/32bc2161a107db20c2ebc85aad31c29730db3e38/stdlib/builtins.pyi#L746),
but numeric ABCs are not special-cased by typeshed in the same way as
collections ABCs. Fundamentally, mypy has no knowledge about virtual
subclassing.
Again, I covered this very comprehensively on StackOverflow ;)
https://stackoverflow.com/questions/69334475/how-to-hint-at-number-types-i-e-subclasses-of-number-not-numbers-themselv/69383462#69383462
Best,
Alex
> On 16 Oct 2021, at 13:54, Steven D'Aprano wrote:
>
> On Sat, Oct 16, 2021 at 09:54:13AM +0100, Alex Waygood wrote:
>>>> On 16 Oct 2021, at 06:13, Steven D'Aprano wrote:
>>> Be careful about believing what you are told.
>> Indeed, MyPy will correctly raise errors if you assign {None: []} to a
>> variable annotated with dict[str, Number]. However, you'll find that
>> MyPy also raises an error if you assign {'foo': 4} to a variable
>> annotated with dict[str, Number]:
>
> Hah, serves me right for not testing it for both positive and negative
> cases.
>
> In my very limited testing now, I see that the problem appears to be
> with the Number type. mypy correctly accepts this:
>
> Data = Dict[str, int]
> a: Data = {'spam': 42}
>
> (no errors), but if you change the annotation to use Number instead of
> int, it wrongly flags that as a type error.
>
> Possibly mypy doesn't know that ints and floats are instances of Number?
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/FJY3QKAUJOJYTODMZH72X2ZS63GKT6AN/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/JFS65SMVETZ7W4KARJYJUNP6KA5XOFPL/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?
> On 16 Oct 2021, at 06:13, Steven D'Aprano wrote:
> Be careful about believing what you are told.
Indeed, MyPy will correctly raise errors if you assign {None: []} to a variable
annotated with dict[str, Number]. However, you'll find that MyPy also raises an
error if you assign {'foo': 4} to a variable annotated with dict[str, Number]:
https://mypy-play.net/?mypy=latest=3.10=de7faea3b4d6a0ffefc64acf3bfa7b77
The issue isn't a matter of false negatives from Mypy; it's a matter of false
positives. For more, see the (rather long — sorry!) answer I gave Sebastian to
a previous question of his on Stack Overflow:
https://stackoverflow.com/questions/69334475/how-to-hint-at-number-types-i-e-subclasses-of-number-not-numbers-themselv/69383462#69383462
Having said that, I absolutely agree that type-hinting doesn't seem to have
much to do with this specific problem, and that a UserDict subclass is probably
the way to go here.
Best,
Alex
> On 16 Oct 2021, at 06:13, Steven D'Aprano wrote:
>
> On Fri, Oct 15, 2021 at 06:17:12PM +0200, Sebastian M. Ernst wrote:
>
>> Data = Dict[str, Number]
>>
>> @typechecked
>> def foo(bar: Data):
>>print(bar)
>> ```
>>
>> Yes, this is using run-time checks (typeguard), which works just fine.
>> Only strings as keys and Number objects as values are going through. (I
>> was told that MyPy does not get this at the moment.)
>
> Be careful about believing what you are told.
>
>
>[steve ~]$ cat test.py
>from numbers import Number
>from typing import Dict
>Data = Dict[str, Number]
>a: Data = {None: []}
>
>[steve ~]$ mypy test.py
>test.py:4: error: Dict entry 0 has incompatible type "None":
>"List[]"; expected "str": "Number"
>Found 1 error in 1 file (checked 1 source file)
>
>
>> The issue is that `bar` itself still allows "everything" to go in (and out):
>>
>> ```python
>> @typechecked
>> def foo2(bar: Data):
>>bar[1.0] = b'should not be allowed'
>> ```
>
> The Python interpreter intentionally doesn't know or care about
> typing annotations. The interpreter should absolutely allow that.
>
> However mypy correctly flags that assignment as a type error.
>
> How about typeguard? That's for the typeguard developers to answer, but
> my guess is that they will say that the typechecked decorator can
> enforce type checking of the input parameters and perhaps even the
> return result, but there is nothing they can do about what happens
> inside the body of the function.
>
>
>> PEP 589 introduces typed dictionaries, but for a fixed set of predefined
>> keys (similar to struct-like constructs in other languages). In
>> contrast, I am looking for an arbitrary number of typed keys/value pairs.
>
> Something like this?
>
>from numbers import Number
>from collections import UserDict
>
>class RestrictedDict(UserDict):
>def __setitem__(self, key, value):
>if not isinstance(key, str):
>raise TypeError('key must be a string')
>if not isinstance(value, Number):
>raise TypeError('value must be a number')
>super().__setitem__(key, value)
>
>
> (I think that is the only method that needs to be overloaded.)
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/HLGDIWGHSIG3LDSBN7RDARK6E7VYTXQT/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/NJQD6TKI5EIWBIAFKOL5L7SNHJX2KK7A/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Structure Pattern for annotation
I agree with Steven. I very much like Abdulla's proposed syntax for dicts,
TypedDicts and sets. But I'm not sure that the idea for `Annotated` is
workable, and the proposal for lists seems too prone to ambiguity, given how
extensively square brackets are already used in typing syntax.
One question about the proposals, however: how would we represent other
mappings apart from dicts? Would `collections.abc.Mapping` and
`types.MappingProxyType` still be spelled as `Mapping[str, int]` and
`MappingProxyType[str, int]`? If so, that would be a slightly annoying
inconsistency.
I do also quite like the idea of an improved syntax for tuples specifically.
Tuples are already very different types to lists/strings/etc in the context of
typing, essentially representing heterogeneous structs of fixed length rather
than homogenous sequences of unspecified length. As such, I think it "makes
sense" to special-case tuples without special-casing other sequences such as
list, `collections.abc.Sequence` or `collections.deque`.
def foo() -> tuple[list[list[str]], list[list[str]]]
would become
def foo() -> (list[list[str]], list[list[str]])
That feels quite readable and natural, to me.
Best,
Alex
> On 14 Oct 2021, at 09:25, Steven D'Aprano wrote:
> On Thu, Oct 14, 2021 at 12:32:57AM +0400, Abdulla Al Kathiri wrote:
>
>> Today I found myself write a function that returns a tuple of list of
>> list of strings (tuple[list[list[str]], list[list[str]]]). Wouldn’t it
>> easier to read to write it like the following:
>> ([[str]], [[str]])?
>
> Not really. Your first example is explicit and I can get the meaning by
> just reading it out loud:
>
>tuple[list[list[str]], list[list[str]]]
>
>"tuple (of) list (of) list (of) str, list (of) list (of) str
>
> Your abbreviated version:
>
>([[str]], [[str]])
>
> is too terse. I have to stop and think about what it means, not just
> read it out loud. Without the hint of named types (tuple and list), my
> first reaction to seeing [str] is "is this an optional string?".
>
> And then I wonder why it's not written:
>
>([[""]], [[""]])
>
> Why abbreviate list and tuple but not string?
>
> Code is read more than it is written, and can be too terse as well as
> too verbose.
>
> On the other hand:
>
>> Similarly for TypedDict, replace the following..
>> class Movie(TypedDict):
>>name: str
>>year: int
>> with
>> {‘name’: str, ‘year’: int}
>
> To my eye, that one does work. As far as I know, curly brackets {}
> aren't otherwise used in annotations (unlike square brackets), and they
> don't look like "optional" to me. They look like a dict.
>
> So on first glance at least, I think that:
>
>{'name': str, 'year': int}
>
> is better than the class syntax we already have.
>
>
> Likewise:
>
>> dict[str, int] will be {str: int}
>> set[int] will be {int}.
>
> work for me too.
>
>
>> Also, Annotated[float, “seconds”] can be replaced with something like
>> float #seconds indicating whatever comes after the hashtag is just
>> extra information similar to a normal comment in the code.
>
> No, because the # indicates that the rest of the line is a comment. This
> is already legal:
>
>def func(d: {str # this is an actual comment
> : int}) -> Any: ...
>
> so this would be ambiguous between a real comment and an annotation.
>
> Even if we agreed to change the behaviour of comments, you suggested:
>
>func(d: {str # product label: [float] # prices from 2000 to 2015})
>
> How is the interpreter to know that the first annotation is just
>
>"product label"
>
> rather than this?
>
>"product label: [float] # prices from 2000 to 2015"
>
> So I don't think this works.
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/T7E54DZW6EENRT373ZMCTU5WWRQ5M2TN/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/YUPJNZVGYDANDHFD5VI54GRRNLGUAFBP/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: dict_items.__getitem__?
> > If one DID write a first() function, it maybe or maybe not should raise a > different exception, but it should certainly provide a better error message For reference, the more-itertools package on PyPI has `first()` and `last()` functions: https://more-itertools.readthedocs.io/en/stable/_modules/more_itertools/more.html#first. (Not endorsing details of their implementations -- just providing a reference to a piece of prior art.) On Tue, Oct 12, 2021 at 4:41 PM Christopher Barker wrote: > On Tue, Oct 12, 2021 at 4:51 AM Chris Angelico wrote: > >> > Exactly: simple usage of next is often a bug. We need to be careful >> about this every time someone suggests that it's straight-forward to do >> next(iter(obj)). >> > > >> Please give a real example of where calling first() and getting >> ValueError is safer than calling next(iter(x)) and getting >> StopIteration. So far, I am undeterred in believing that the two >> exceptions have equivalent effect if the caller isn't expecting them. >> > > I don't know about safer, but it is a clear example of why using > next(iter(obj)) requires a pretty complete knowledge of the iteration > protocol. > > I can guarantee you I'd get some questions from my students when they got > a StopIterationError! > > If one DID write a first() function, it maybe or maybe not should raise a > different exception, but it should certainly provide a better error message: > > >>> next(iter([])) > Traceback (most recent call last): > File "", line 1, in > StopIteration > > Is not very helpful. > > -CHB > > > > > >> ChrisA >> ___ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/2AL5FE3KZI4EBTRMJ7O5EL6MBVN7RUYF/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > Christopher Barker, PhD (Chris) > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/VQFAQMAKKQ6BG3VIP6BLZ6Q6RBB4V4CR/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ODGZXXYBBXJOJRQIP6TRIFXY6A7VRSMO/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: dict_items.__getitem__?
> > here is the thread from the last time that this was brought up: > > https://mail.python.org/archives/list/python-ideas@python.org/thread/S7UMTWK65X6BJDYZ3SSU7I7HOIASDMMJ/#S7UMTWK65X6BJDYZ3SSU7I7HOIASDMMJ Thanks, that's very helpful. Sounds like Guido is right, and the short answer is "while there would be ways of making it fast (and faster than the status quo) for small dicts, there is probably no way of making it O(1), as indexing is for lists and tuples". For me, that's a deal-breaker. I think if you're adding in a way to get the *n*th key of a dict by sequence-style indexing, it's going to be very surprising for newbies if they find out that getting the 8743rd key of the dict takes a lot longer than getting the 1st key of the dict, given that that's not at all the behaviour of lists and tuples. Best, Alex On Sun, Oct 10, 2021 at 7:26 PM Christopher Barker wrote: > here is the thread from the last time that this was brought up: > > > https://mail.python.org/archives/list/python-ideas@python.org/thread/S7UMTWK65X6BJDYZ3SSU7I7HOIASDMMJ/#S7UMTWK65X6BJDYZ3SSU7I7HOIASDMMJ > > It was very thoroughly discussed then. > > -CHB > > > On Sun, Oct 10, 2021 at 8:33 AM Guido van Rossum wrote: > >> You have to check the C code to be sure, but IIRC the latest dict >> implementation has a dense array of the values in insert order, and the >> hash table (which has gaps) contains indexes into the values array. So you >> could easily index into the values array (which I believe also has the >> keys) in O(1) time. >> >> Though what happens to the dense array when a key is deleted? It must >> leave a gap there too. So, never mind, you’d have to walk through the array >> counting items but not gaps, and that’s O(n). Which explains why we don’t >> have such an API. But please check the C code! >> >> —Guido >> >> On Sun, Oct 10, 2021 at 07:18 Alex Waygood >> wrote: >> >>> Should `dict.items()` be indexable now that dicts are ordered? I say >>> yes. Why shouldn't it? >>> >>> >>> Would there be a way to ensure that this had the same time complexity as >>> indexing of sequences? If "yes", I would support this — I think it would be >>> useful in some situations, and it would be more efficient than existing >>> mechanisms to obtain the *n*th key from a dictionary. If (as I >>> presume), the answer is "no", then I would not support this — I think it >>> would give the misleading impression that obtaining the *n*th >>> key/value from a dictionary is just as efficient as obtaining the *n*th >>> item from a list or tuple. >>> >>> Best, >>> Alex >>> >>> On 10 Oct 2021, at 05:05, Finn Mason wrote: >>> >>> >>> >>> On Sat, Oct 9, 2021, 9:56 PM Steven D'Aprano >>> wrote: >>> >>>> [Snip...] >>> >>> >>> Newbies won't know first() lives in itertools, and those experienced >>>> enough to know it is there probably won't bother to use it. >>>> >>> >>> A very good point. >>> >>> Let's get back to the original topic. Should `dict.items()` be indexable >>> now that dicts are ordered? I say yes. Why shouldn't it? >>> >>> >>> -- >>> Finn Mason >>> >>>> ___ >>> Python-ideas mailing list -- python-ideas@python.org >>> To unsubscribe send an email to python-ideas-le...@python.org >>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-ideas@python.org/message/OOR2AUMA7UMHHVW7XLLUXHTNKGRXTPU4/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >>> ___ >>> Python-ideas mailing list -- python-ideas@python.org >>> To unsubscribe send an email to python-ideas-le...@python.org >>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-ideas@python.org/message/RHNNAZR2ZBYZFQ75VHR3FUMVY6GWDDB6/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> -- >> --Guido (mobile) >> ___ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archi
[Python-ideas] Re: dict_items.__getitem__?
> Should `dict.items()` be indexable now that dicts are ordered? I say yes. Why > shouldn't it? Would there be a way to ensure that this had the same time complexity as indexing of sequences? If "yes", I would support this — I think it would be useful in some situations, and it would be more efficient than existing mechanisms to obtain the nth key from a dictionary. If (as I presume), the answer is "no", then I would not support this — I think it would give the misleading impression that obtaining the nth key/value from a dictionary is just as efficient as obtaining the nth item from a list or tuple. Best, Alex > On 10 Oct 2021, at 05:05, Finn Mason wrote: > > >> On Sat, Oct 9, 2021, 9:56 PM Steven D'Aprano wrote: > >> [Snip...] > > > >> Newbies won't know first() lives in itertools, and those experienced >> enough to know it is there probably won't bother to use it. > > > A very good point. > > Let's get back to the original topic. Should `dict.items()` be indexable now > that dicts are ordered? I say yes. Why shouldn't it? > > > -- > Finn Mason > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/OOR2AUMA7UMHHVW7XLLUXHTNKGRXTPU4/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RHNNAZR2ZBYZFQ75VHR3FUMVY6GWDDB6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: dict_items.__getitem__?
> The problem is that first() does not return the first item of the
> iterator, but the *next item still available*.
>
>a = list('12345')
>b = iter('12345')
>
>first(a) == first(a) # True
>first(b) == first(b) # False
This is an excellent point, and something I hadn't considered. Unless someone
can think of a good workaround that doesn't make the implementation hideously
complex, I retract my support for adding `first()` and `last()` to itertools.
> On 10 Oct 2021, at 05:09, Steven D'Aprano wrote:
>
> On Wed, Oct 06, 2021 at 03:42:28PM +0100, Alex Waygood wrote:
>
>>> Whether they are added to dict or itertools, there are still nine of
>>> them
>>
>> No, the suggestion was to add two functions to itertools (first() and
>> last(), which would work with any iterable, not just dicts), rather
>> than adding nine methods to the dict interface. This was precisely why
>> I was saying that I liked the itertools solution more.
>
> Okay.
>
> In another post, I've explained why I don't think that putting first()
> in itertools would actually be useful (TL;DR: newbies wouldn't know it
> was there, and to people experienced enough to know it was there, it's
> likely easier to just use next() directly).
>
> When I started writing this post, I was going to argue that we should
> put first() and last() in itertools as recipes, for their pedagogical
> value:
>
>def first(iterable):
>return next(iter(iterable))
>
>def last(iterable):
>return next(reversed(iterable))
>
> I had got so far as to open bugs.python.org to create a ticket.
>
> But I've convinced myself that they aren't even worthwhile as recipes.
> The problem is that if we are being precise and accurate, and we should
> be, the names are simply *wrong* and the behaviour will be confusing to
> those we're supposedly trying to help.
>
> The problem is that first() does not return the first item of the
> iterator, but the *next item still available*.
>
>a = list('12345')
>b = iter('12345')
>
>first(a) == first(a) # True
>first(b) == first(b) # False
>
> If that behaviour makes sense to you, and is what you expected, then
> congratulations, you're probably a very sophisticated Pythonista who
> understands that iterators mutate after each item is retrieved, in
> which case you probably aren't going to get any benefit at all from a
> named first() function.
>
> But if you are one of those newbies who (we're told) need a named
> function, then you will probably be totally gobsmacked that sometimes
> first() will return the first item, and always the first item, and
> sometimes it will return the first, second, third... items in sequence.
>
> If we are to be precise and accurate, *sequences* have a first item, but
> iterators only have a next item.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/F6TG5EW3SKKMN6D6F7UBRRUNG2KYMMK5/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/K3UUIZOW7NDSSR5ZJMHWTMQBDHKFWZX6/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: dict_items.__getitem__?
> Whether they are added to dict or itertools, there are still nine of > them No, the suggestion was to add two functions to itertools (first() and last(), which would work with any iterable, not just dicts), rather than adding nine methods to the dict interface. This was precisely why I was saying that I liked the itertools solution more. > On 6 Oct 2021, at 15:01, Steven D'Aprano wrote: > > On Wed, Oct 06, 2021 at 11:11:09AM +0100, Alex Waygood wrote: >>> The temptation to insist "see, YAGNI!" at this point I shall resist. >> >> *You* might not need it, but I've seen it come up a lot on Stack >> Overflow, and all too often people end up going for the much less >> efficient solution. I personally have also written code with practical >> applications using `next(iter(mydict))`. > > Under what circumstances do you care what the first key in a dict is, > without going on to care about the second, third, fourth etc? > > They are surely extremely niche, or artificial, or both, e.g. the > Stackoverflow problem you link to: "find the first non-repeating > character in a string -- using only one loop". Why the *first* rather > than any, or all? > > In any case, the presence of one or two uses for a piece of > functionality doesn't mandate that we make this a builtin. Your solution > with next() is perfectly adequate. > > The other suggested methods are even more obscure. Why have a method > for returning the first value, without knowing the key? > > "I don't know what the first key is, and I don't care, but I know that > whatever it is, it maps to the value 17." > > Now what are you going to do with that knowledge? This seems like a > method in desperate need of a use-case. > > > [...] >> I agree that it's a lot of methods to add. That's precisely why I >> prefer Inada Naoki's suggestion of additions to itertools > > Whether they are added to dict or itertools, there are still nine of > them, and they are pretty much near clones of each other: > ># first_ and last_ whatsits >next([iter|reversed](obj.[keys|values|items]())) > > if you will excuse the misuse of hybrid Python/BNF syntax :-) > > > > -- > Steve > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/T3TOFAFBPGY44LOVKSMVZJGBNQ7MUNEL/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OCVRRJPZQCQ6UKVKR4GQRJFWAUJNDFK6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: dict_items.__getitem__?
> The temptation to insist "see, YAGNI!" at this point I shall resist. *You* might not need it, but I've seen it come up a lot on Stack Overflow, and all too often people end up going for the much less efficient solution. I personally have also written code with practical applications using `next(iter(mydict))`. > I was +0 sympathetic to that until you posted this obvious extension > of the idea I don't see these dict methods as an "obvious extension" of the idea; I see them as unneeded/irrelevant *if* a more general way of doing this is added to itertools. > That's an awful lot of new methods to take advantage of what for many > applications of dict (in fact, *all* of my applications ever[1]) is an > irrelevant ordering. I agree that it's a lot of methods to add. That's precisely why I prefer Inada Naoki's suggestion of additions to itertools, and why I've never bothered starting a thread on the subject myself. But since the topic was being discussed after Erik started this thread, I thought I might mention the alternative solution that's been in the back of my head for a while. > [A]nyone who wants it can do it themselves: As I said in my previous message, the existing idiom (`next(iter(mydict))`) appears to be extremely non-obvious for beginners in Python, causing them to often go for much more inefficient options. No one is claiming "there is no way to do this" at present; the argument is that the best way to do this is, at present, *not sufficiently obvious*. > These defs do something undefined on unordered mappings (ie, not based > on dict), and may be dangerous in that sense. Excellent point (though note that my suggestion was to extend the dict interface, not the interface for all mappings). > In the current implementation, positional indexing is time-expensive Yes, that's exactly why I don't support Erik's original suggestion of positional indexing for dict.keys(), dict.values() and dict.items(), and why I *only* support easy ways of fetching the first/last object from the view. > I wonder if any code is being written obscurely to ensure that > keys get added to dicts in the "right" order I personally have used dictionaries as ordered sets in the past (skip to the last code snippet in this answer: https://codereview.stackexchange.com/a/264437/24517), and would object to this being called an "obscure" use of the data structure. It is, by now, both well-known and well-advertised in the documentation that dictionaries are guaranteed to maintain insertion order. It's obviously not how the data structure has historically been used, but I don't see how that makes it an invalid use of the data structure *now*. Why shouldn't users be expected to exploit an advertised feature of the language? > On 6 Oct 2021, at 10:29, Stephen J. Turnbull > wrote: > > Alex Waygood writes: > >> Whereas obviously, > > The temptation to insist "see, YAGNI!" at this point I shall resist. > >> a much better way (especially if it's a very large dictionary) is >> to do: >> first_key = next(iter(mydict)) > >> [Inada Naoki] >>> I think we can add `itertools.first()` for this idiom, and >>> `itertools.last()` for `next(iter(reversed(x)))` idiom. > > I was +0 sympathetic to that until you posted this obvious extension > of the idea: > >> I like this idea, a lot. Another possibility I've been wondering >> about was whether several methods should be added to the dict >> interface: >> dict.first_key = lambda self: next(iter(self)) >> dict.first_val = lambda self: next(iter(self.values())) >> dict.first_item = lambda self: next(iter(self.items())) >> dict.last_key = lambda self: next(reversed(self)) >> dict.last_val = lambda self: next(reversed(self.values())) >> dict.last_item = lambda self: next(reversed(self.items())) > > That's an awful lot of new methods to take advantage of what for many > applications of dict (in fact, *all* of my applications ever[1]) is an > irrelevant ordering. > > And anyone who wants it can do it themselves: > > def first_key(dct): return next(iter(dct)) > def first_val(dct): return next(iter(dct.values())) > def first_item(dct): return next(iter(dct.items())) > def last_key(dct): return next(reversed(iter(dct))) > def last_val(dct): return next(reversed(iter(dct.values( > def last_item(dct): return next(reversed(iter(dct.items( > > These defs do something undefined on unordered mappings (ie, not based > on dict), and may be dangerous in that sense. OTOH, I suspect the > methods will do the wrong thing with many ordered mappings based on > dict that support orders other than insertion order. > >> But I think I like a lot more the idea of addi
[Python-ideas] Re: dict_items.__getitem__?
I think there definitely should be a more obvious way to do this (specifically the first and last keys/values/items of a dictionary — I'm ambivalent about the others, since they won't always be fast, as discussed). An anti-pattern you see quite often on Stack Overflow to get the first key of a dictionary is something like the following: first_key = list(mydict.keys())[0] Whereas obviously, a much better way (especially if it's a very large dictionary) is to do: first_key = next(iter(mydict)) [Christopher Barker] > I'll leave it exercise for the reader to find that thead For reference, the (very long) previous thread is here: https://mail.python.org/archives/list/python-ideas@python.org/thread/S7UMTWK65X6BJDYZ3SSU7I7HOIASDMMJ/. [Inada Naoki] > I think we can add `itertools.first()` for this idiom, and > `itertools.last()` for `next(iter(reversed(x)))` idiom. I like this idea, a lot. Another possibility I've been wondering about was whether several methods should be added to the dict interface: dict.first_key = lambda self: next(iter(self)) dict.first_val = lambda self: next(iter(self.values())) dict.first_item = lambda self: next(iter(self.items())) dict.last_key = lambda self: next(reversed(self)) dict.last_val = lambda self: next(reversed(self.values())) dict.last_item = lambda self: next(reversed(self.items())) But I think I like a lot more the idea of adding general ways of doing these things to itertools. Best, Alex > On 5 Oct 2021, at 05:30, Christopher Barker wrote: > > On Mon, Oct 4, 2021 at 5:46 PM Erik Demaine wrote: >> Have folks thought about allowing indexing dictionary views as in the >> following code, where d is a dict object? >> >> d.keys()[0] >> d.keys()[-1] >> d.values()[0] >> d.values()[-1] >> d.items()[0] >> d.items()[-1] # item that would be returned by d.popitem() > > since dicts were made order-preserving, indexing the keys, items, etc does > make some sense. > > > > I've also often wanted to get an arbitrary item/key from a dictionary, and > > This is indeed one of the use cases identified. > >> I found some related discussion in >> https://mail.python.org/archives/list/python-ideas@python.org/thread/QVTGZD6USSC34D4IJG76UPKZRXBBB4MM/ >> but not this exact idea. > > That's a pretty different idea but this exact idea has been discussed on this > list relatively recently. I still like it, but there wan't much general > support. > > I'll leave it exercise for the read to find that thead, but it is there, and > I suggest you look for it if you want to further pursue this idea. > > -CHB > > > -- > Christopher Barker, PhD (Chris) > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/RAEDZPUTNABJLX3ESU32PZQBJ25DDOPK/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DY6JFKMZESHOXEHNO5I6A5GMEKFBUUSN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Syntax Sugar for __name__ == "__main__" boilerplate?
I disagree that "it teaches a lot about how Python works" is a good reason to keep things the way they are. If you applied this principle more broadly, it would seem to be an argument in favour of complexity in most situations, that would imply we should keep syntactic sugar to a bare minimum at all times. Learning about how Python works under the hood is extremely valuable for becoming a more advanced programmer. However, not everybody needs to be a mechanic to drive a car. We should surely make it as easy as possible for beginners to write fully functional scripts that conform to best practices. I love this proposal. "if__name__ == '__main__'" has always felt, to me, as though it flies in the face of the simplicity and elegance Python generally prizes in other parts of the language. Best wishes, Alex > On 2 Oct 2021, at 11:00, Abdur-Rahmaan Janhangeer > wrote: > > > Greetings list, > > I am -1 on this proposition. > > I can relate that if name == main is very confusing to teach. > However, it teaches a lot about how Python works. > If you know Python it is very clear. So if it's confusing, you > wrongly taught Python (I myself was in this situation) > > The simplification idea is to coerce Python to use patterns forged elsewhere. > The tools are here if you wish to use it. > > But, specifically pointing a pattern for it and adding additional layers to > make it > work is an enforcement of the main function. Something python is not bound to > and does not need to run. > > Kind Regards, > > Abdur-Rahmaan Janhangeer > about | blog > github > Mauritius > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/VHZK76RSXE3WIL2ZJ75WHCZ5OWQPE2QL/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/H52A65WBSOPGLJ4YAJBRLKFIFW5VED46/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: os.workdir() context manager
I also really like this idea. Though I wonder if it might be a better fit for `contextlib` rather than `os`? Can see arguments both ways (and it's a minor detail, anyway). Alex > On 14 Sep 2021, at 20:55, Paul Moore wrote: > > On Tue, 14 Sept 2021 at 20:44, Marc-Andre Lemburg wrote: >> >> I sometimes write Python scripts which need to work in specific >> work directories. >> >> When putting such code into functions, the outer function typically >> does not expect the current work dir (CWD) to be changed, so wrap the >> code which need the (possibly) modified CWD using a simply context >> manager along the lines of: >> >> class workdir: >>def __init__(self, dir): >>self.dir = dir >>def __enter__(self): >>self.curdir = os.getcwd() >>os.chdir(self.dir) >>def __exit__(self, *exc): >>os.chdir(self.curdir) >>return False >> >> Would there be interest in adding something like this to the os module >> as os.workdir() ? > > I've needed (and implemented my own version of) this often enough that > I'd be in favour of it. Of course, as you show, it's not hard to write > it yourself, so I can live with it not getting implemented, too :-) > > Paul > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/I6YFWYNUAV7G6DNPZTBJ6T3SGJDPGBJL/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/B2DICGY5Q65IFWPSBVFYCYNJBGJ7KNGY/ Code of Conduct: http://python.org/psf/codeofconduct/
