[Python-ideas] Re: PEP 637 and keyword only subscripts

[Guido van Rossum]

But this requires introspection.

On Fri, Sep 25, 2020 at 22:46 Ricky Teachey  wrote:

> On Fri, Sep 25, 2020 at 11:50 PM Steven D'Aprano 
> wrote:
>
>> TL;DR:
>>
>>
>>
>>
>>
>> 1. We have to pass a sentinel to the setitem dunder if there is no
>>
>>
>> positional index passed. What should that sentinel be?
>>
>
> Isn't there a problem with this starting assumption?
>
> Say that I have item dunder code of the signature that is common today,
> and I have no interest in providing direct support for kwd args in my item
> dunders. Say also, kwd unpacking is supported.
>
> If a person unpacks an empty dictionary (something that surely will happen
> occasionally), and a SENTIAL is provided, this poses a pretty good
> possibility to lead to unintended behavior in many cases:
>
> class C:
> def __getitem__(self, index):
> ...
>
> d = {}
> c = C()
> c[**d]
>
> The above will call:
>
> c.__getitem__(SENTINEL)
>
> Who knows what the effect of c[SENTINEL] will be for so much existing code
> out there? Bugs are surely to be created, aren't they? So a lot of people
> will have to make changes to existing code to handle this with a sentinel.
> This seems like a big ugly problem to me that needs to be avoided.
>
> There's a third option: just prohibit keyword-only subscripts. I think
>>
>>
>> that's harsh, throwing the baby out with the bathwater. I personally
>>
>>
>> have use-cases where I would use keyword-only subscripts so I would
>>
>>
>> prefer options 1 or 2.
>>
>
> Maybe there's a middle way option that could avoid the particular problem
> outlined above:
>
> # Option 4: do use a sentinel at all, user provides own (if desired)
>
> obj[spam=1, eggs=2]
> # => calls type(obj).__getitem__(USER_PROVIDED_SENTINEL, spam=1,
> eggs=2)
>
> del obj[spam=1, eggs=2]
> # => calls type(obj).__delitem__(USER_PROVIDED_SENTINEL, spam=1,
> eggs=2)
>
> obj[spam=1, eggs=2] = value
> # => calls type(obj).__setitem__(USER_PROVIDED_SENTINEL, value,
> spam=1, eggs=2)
>
> How do we write the dunder item to allow for this? Simple: just require
> that if it is desired to support kwd-only item setting, the writer of the
> code has to provide a default-- ANY default-- to the value argument in
> setitem (and this default will never be used).
>
> I suggest the ellipses object as the standard convention:
>
> MyPreferredSentinel = object()
>
> class C:
> def __setitem__(self, index=MyPreferredSentinel, value=..., **kwargs):
> ...
>
> c=C()
> c[x=1] = "foo"
> # calls => c.__setitem__(MyPreferredSentinel, "foo", x=1)
>
> This option provides an additional benefit: if I don't want to provide
> support for the kwd-only case, I don't have to handle it explicitly--
> errors occur without any effort on my part, just as they do today (although
> the error itself is different):
>
> class C:
> def __setitem__(self, index, value, **kwargs):
> ...
>
> c = C()
> c[x=1] = "foo"  # yay, this produces an error with no effort
>
> Providing a default sentinel nullifies this possibility. In that case,
> have to handle a kwd-only indexing myself:
>
> class C:
> def __setitem__(self, index, value, **kwargs):
> if index is SENTINEL:
> handle_kwd_args_only()
>
> ---
> Ricky.
>
> "I've never met a Kentucky man who wasn't either thinking about going home
> or actually going home." - Happy Chandler
>
>
>
> ___
>
> Python-ideas mailing list -- python-ideas@python.org
>
> To unsubscribe send an email to python-ideas-le...@python.org
>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/G2ZGET57PC3QQ4AJGHZAJIWCDFXKS5QN/
>
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DZRGBYNTABBBF4F2GQ5UU32FEDYKREML/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 and keyword only subscripts

[Ricky Teachey]

On Fri, Sep 25, 2020 at 11:50 PM Steven D'Aprano 
wrote:

> TL;DR:
>
> 1. We have to pass a sentinel to the setitem dunder if there is no
> positional index passed. What should that sentinel be?
>

Isn't there a problem with this starting assumption?

Say that I have item dunder code of the signature that is common today, and
I have no interest in providing direct support for kwd args in my item
dunders. Say also, kwd unpacking is supported.

If a person unpacks an empty dictionary (something that surely will happen
occasionally), and a SENTIAL is provided, this poses a pretty good
possibility to lead to unintended behavior in many cases:

class C:
def __getitem__(self, index):
...

d = {}
c = C()
c[**d]

The above will call:

c.__getitem__(SENTINEL)

Who knows what the effect of c[SENTINEL] will be for so much existing code
out there? Bugs are surely to be created, aren't they? So a lot of people
will have to make changes to existing code to handle this with a sentinel.
This seems like a big ugly problem to me that needs to be avoided.

There's a third option: just prohibit keyword-only subscripts. I think
> that's harsh, throwing the baby out with the bathwater. I personally
> have use-cases where I would use keyword-only subscripts so I would
> prefer options 1 or 2.
>

Maybe there's a middle way option that could avoid the particular problem
outlined above:

# Option 4: do use a sentinel at all, user provides own (if desired)

obj[spam=1, eggs=2]
# => calls type(obj).__getitem__(USER_PROVIDED_SENTINEL, spam=1, eggs=2)

del obj[spam=1, eggs=2]
# => calls type(obj).__delitem__(USER_PROVIDED_SENTINEL, spam=1, eggs=2)

obj[spam=1, eggs=2] = value
# => calls type(obj).__setitem__(USER_PROVIDED_SENTINEL, value, spam=1,
eggs=2)

How do we write the dunder item to allow for this? Simple: just require
that if it is desired to support kwd-only item setting, the writer of the
code has to provide a default-- ANY default-- to the value argument in
setitem (and this default will never be used).

I suggest the ellipses object as the standard convention:

MyPreferredSentinel = object()

class C:
def __setitem__(self, index=MyPreferredSentinel, value=..., **kwargs):
...

c=C()
c[x=1] = "foo"
# calls => c.__setitem__(MyPreferredSentinel, "foo", x=1)

This option provides an additional benefit: if I don't want to provide
support for the kwd-only case, I don't have to handle it explicitly--
errors occur without any effort on my part, just as they do today (although
the error itself is different):

class C:
def __setitem__(self, index, value, **kwargs):
...

c = C()
c[x=1] = "foo"  # yay, this produces an error with no effort

Providing a default sentinel nullifies this possibility. In that case, have
to handle a kwd-only indexing myself:

class C:
def __setitem__(self, index, value, **kwargs):
if index is SENTINEL:
handle_kwd_args_only()

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/G2ZGET57PC3QQ4AJGHZAJIWCDFXKS5QN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Ben Rudiak-Gould]

On Fri, Sep 25, 2020 at 8:26 PM David Mertz  wrote:

> E.g. we would have:
>
> newthing[(), foo=1, bar=4:5] == newthing[foo=1, bar=4:5]
>

Right, but we also have

newthing[(2, 3), foo=1, bar=4:5] == newthing[2, 3, foo=1, bar=4:5]

which seems exactly analogous. A disambiguation scheme that worked for
every n might be worth it, but one that only works for n=0 and makes that
case less consistent with n>=2 doesn't seem worth it to me.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RDI6OOSSNCNIMBSVWLE52NRWHIR52IOT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Greg Ewing]

On 26/09/20 7:26 am, Oscar Benjamin wrote:

The suggestion ... was not that it would be a
type-checking error to call the inverse function without catching the
exception. It only becomes a type-checking error if a function calls
the inverse function and claims not to raise the exception.


This isn't very coherent. We've agreed that the declared exceptions
is not an exhaustive list of the exceptions that may be raised. So
not declaring an exception is *not* claiming not to raise it!

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6GJ5ZW3K52SYUT3RDDMYAW6WQUSQ5SAM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 9999 (provisional): support for indexing with keyword arguments

[Steven D'Aprano]

Thanks Eric,

On Fri, Sep 25, 2020 at 09:23:56AM +0100, Eric Wieser wrote:
> Thanks for the PEP, and for pinging the numpy list about it. Some comments:
> 
> Sequence unpacking remains a syntax error inside subscripts:
> Reason: unpacking items would result it being immediately repacked into a
> tuple
> 
> A simple counter-example is [:, *args, :], which could be treated as
> [(slice(None), *args, slice(None))].
> When there are index objects to the left or right of *args, it enables :
> syntax in places that would otherwise be forbidden.

This use-case is persuasive to me.

I recommend that the PEP be changed to allow `*args` inside subscripts.


> Keyword-only subscripts are permitted. The positional index will be the
> empty tuple
> 
> As discussed on the numpy mailing list, a better approach might be to not
> pass an argument at all, so that obj[spam=1, eggs=2] calls
> type(obj).__getitem__(obj, spam=1, eggs=2).
> Objects which want to support keyword-only indexing can provide a default
> of their own (and are free to choose between () and None, while objects
> which do not would just raise TypeError due to the missing positional
> argument

That's fine for getitem and delitem, but doesn't work with setitem.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AKMR6WOE6HN5SQQAIBHVML6GPJTEY7TF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 and keyword only subscripts

[Steven D'Aprano]

Change of subject line as I wish to focus on a single critical point of 
the PEP: keyword-only subscripts.

TL;DR:

1. We have to pass a sentinel to the setitem dunder if there is no 
positional index passed. What should that sentinel be?


*  None
*  the empty tuple ()
*  NotImplemented
*  something else


2. Even though we don't have to pass the same sentinel to getitem and 
delitem dunders, we could. Should we?


*  No, getitem and delitem should use no sentinel.
*  Yes, all three dunders should use the same rules.
*  Just prohibit keyword-only subscripts.


(Voting is non-binding. It's feedback, not a democracy :-)

Please read the details below before voting. Comments welcome.


--


For all three dunders, there is no difficulty in retrofitting keyword 
subscripts to the dunder signature if there is there is a positional 
index:

obj[index, spam=1, eggs=2]
# => calls type(obj).__getitem__(index, spam=1, eggs=2)

del obj[index, spam=1, eggs=2]
# => calls type(obj).__delitem__(index, spam=1, eggs=2)

obj[index, spam=1, eggs=2] = value
# => calls type(obj).__setitem__(index, value, spam=1, eggs=2)


If there is no positional index, the getitem and delitem calls are 
easy:

obj[spam=1, eggs=2]
# => calls type(obj).__getitem__(spam=1, eggs=2)

del obj[spam=1, eggs=2]
# => calls type(obj).__delitem__(spam=1, eggs=2)

If the dunders are defined with a default value for the index, the call 
will succeed; if there is no default, you will get a TypeError. This is 
what we expect to happen.

But setitem is hard:


obj[spam=1, eggs=2] = value
# => calls type(obj).__setitem__(???, value, spam=1, eggs=2)


Python doesn't easily give us a way to call a method and skip over 
positional arguments. So it seems that setitem needs to fill in a fake 
placeholder. Three obvious choices are None, the empty tuple () or 
NotImplemented.

All three are hashable, so they could be used as legitimate keys in a 
mapping; but in practice, I expect that only None and () would be. I 
can't see very many objects actually using NotImplemented as a key.

numpy also uses None to force creation of a new axis. I don't think that 
*quite* rules out None: numpy could distinguish the meaning of None as a 
subscript depending on whether or not there are keyword args.

But NotImplemented is special:

- I don't expect anyone is using NotImplemented as a key or index.

- NotImplemented is already used a sentinel for operators to say "I 
don't know how to handle this"; it's not far from that to interpret it 
as saying "I don't know what value to put in this positional argument".

- Starting in Python 3.9 or 3.10, NotImplemented is even more special: 
it no longer ducktypes as truthy or falsey. This will encourage people 
to explicitly check for it:

if index is NotImplemented: ...

rather than `if index: ...`.

So I think that NotImplemented is a better choice than None or an empty 
tuple.

Whatever sentinel we use, that implies that setitem cannot distingish 
these two cases:


obj[SENTINEL, spam=1, eggs=2] = value
obj[spam=1, eggs=2] = value


Since both None and () are likely to be legitimate indexes, and 
NotImplemented is less likely to be such, I think this supports using 
NotImplemented.

But whichever sentinel we choose, that brings us to the second part of 
the problem.

What should getitem and delitem do?

setitem must provide a sentinel for the first positional argument, but 
getitem and delitem don't have to. So we could have this:


# Option 1: only setitem is passed a sentinel

obj[spam=1, eggs=2]
# => calls type(obj).__getitem__(spam=1, eggs=2)

del obj[spam=1, eggs=2]
# => calls type(obj).__delitem__(spam=1, eggs=2)

obj[spam=1, eggs=2] = value
# => calls type(obj).__setitem__(SENTINEL, value, spam=1, eggs=2)


Advantages:

- The simple getitem and delitem cases stay simple; it is only the
  complicated setitem case that is complicated.

- getitem and delitem can distinguish the "no positional index at all" 
  case from the case where the caller explicitly passes the sentinel
  as a positional index; only setitem cannot distinguish them. If your
  class doesn't support setitem, this might be useful to you.

Disadvantages:

- Inconsistency: the rules for one dunder are different from the other
  two dunders.

- If your class does distinguish between no positional index, and the 
  sentinel, that means that there is a case that getitem and delitem can 
  handle but setitem cannot.


Or we could go with an alternative:

# Option 2: all three dunders are passed a sentinel

obj[spam=1, eggs=2]
# => calls type(obj).__getitem__(SENTINEL, spam=1, eggs=2)

del obj[spam=1, eggs=2]
# => calls type(obj).__delitem__(SENTINEL, spam=1, eggs=2)

obj[spam=1, eggs=2] = value
# => calls type(obj).__setitem__(SENTINEL, value, spam=1, eggs=2)


Even though the getitem 

[Python-ideas] Re: Suggestion: annotated exceptions

[MRAB]

On 2020-09-26 01:23, Greg Ewing wrote:

On 26/09/20 4:32 am, Oscar Benjamin wrote:

annotations could be used to document in a statically
analysable way what the "expected" exceptions are. A type checker
could use those to check whether a caller is handling the *expected*
exceptions


But that would be inappropriate. Even if an exception is
"expected" (assuming for a moment we agree on what that means),
the immediate caller is *not* obliged to handle it. Just as
with any other exception, it's perfectly fine to let it propagate
up to a level where something sensible can be done with it.

Treating this as an error would be more annoying than helpful in
most situations, I think.

There are a few cases where it *might* make sense, such as
StopIteration, which is pretty much part of the function's API
and letting it escape is probably a mistake. But such cases
are very rare and probably not worth adding a new language
mechanism for.

An "expected" exception would be one that could be regarded as part of 
the interface/API, e.g. json.load raising JSONDecodeError on malformed 
JSON. Other exceptions could also be raised, if you, for example, you 
didn't pass in a file, or there's an I/O error, or you run out of memory.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TQBQMSHAZDM5YVI4HQGLVJUQXZ47QQS7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Greg Ewing]

On 26/09/20 5:27 am, Wes Turner wrote:

Safe coding styles (in other languages) do specify that *there may not 
be any unhandled exceptions*.


I don't think a blind rule like that actually makes a program any safer.
It encourages a coding style that covers up problems and carries on to
produce garbage, rather than bringing the problem to someone's
attention.

The Ariane V incident is sometimes cited as an example of what can
happen if you fail to handle exceptions. But "handling" the exception
in that case wouldn't have helped -- the program was being fed
invalid inputs, and whatever it did, some kind of disaster would have
occurred.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZT6F7RMTENC7KXOH4NUOK6K3N5MLO7M2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[David Mertz]

On Fri, Sep 25, 2020 at 4:36 PM Steven D'Aprano  wrote:

> > I.e. consider:
> > >>> d = dict()
> > >>> d[()] = "foo"
> > >>> d
> > {(): 'foo'}
>
> I agree with Ricky that the choice of empty tuple should be justified
> better by the PEP, and alternatives (None, NotImplemented) discussed.
> But I don't think this is a breaking change. Can you explain further
> what you think will break?
>

I think empty tuple is the best choice, and the discussion on the NumPy
list mostly seems to agree.  I agree that it's basically just a question of
making the PEP more explicit in explaining the choice.

In a direct sense, no matter what `newthing[foo=1, bar=4:5]` does
internally, it CANNOT be a breaking change, since that is a SyntaxError
now.  However, it's also the case that if any sentinel is used for
"keywords only", whether None or () or EmptyIndexSentinel, it "steps on"
using that value as an actual index.

E.g. we would have:

newthing[(), foo=1, bar=4:5] == newthing[foo=1, bar=4:5]

... well, we have that if subscript access is not mutation, that is.  Which
it hopefully will not be, but anything is possible in code.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YTMZI36OHKQIXY4XWHYG635A4IHF7X25/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Steven D'Aprano]

On Fri, Sep 25, 2020 at 09:53:41PM +0100, Stefano Borini wrote:
> On Fri, 25 Sep 2020 at 05:55, David Mertz  wrote:

> > Smuggling in a generic function calls in square brackets is 
> > undesirable (but obviously not preventable at a language level).  
> > However, I feel like keywords might very often "alter the behavior."
> >
> > For example, imagine we have a distributed array within an engine 
> > that is intended to be "eventually consistent."  I would find code 
> > like this, in some hypothetical library, to be clear and useful, and 
> > not to violate the spirit of indexing.
> >
> >> snapshot1 = remote_array[300:310, 50:60, 30:35, source=worker1]
> >> snapshot2 = remote_array[300:310, 50:60, 30:35, source=worker2]
> 
> I would personally be very uneasy with this. A better solution would
> be to have a proxy object that handles that:
> 
> >> snapshot1 = remote_array.source(worker1)[300:310, 50:60, 30:35]
>
> Of course people can (and will) abuse the feature, but I would
> personally consider it poor design.

Did you just completely undermine the rationale for your own PEP?

Isn't the entire purpose of this PEP to allow subscripts to include 
keyword arguments? And now you are describing it as "poor design"?

For what it's worth, I think David's example is perfectly clear, within 
the spirit of subscripting, and much more understandable to the reader 
than a `.source` method that returns a proxy.

I'm not really sure why this hypothetical call:

snapshot1 = remote_array[300:310, 50:60, 30:35, source=worker1]

is "abuse" or should make us more uneasy that this hypothetical call:

snapshot1 = remote_array[300:310, 50:60, 30:35, axis=1]

say. Both cases would change the behaviour in some way. David's example 
specifies the distributed source to use, the second specifies the axis 
to use. Why is the first abuse and the second not?


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L4NQ2DXASLELKZUPCLOZKOCMRMEHHPLW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Steven D'Aprano]

On Fri, Sep 25, 2020 at 05:05:52PM -0400, Ricky Teachey wrote:

> * sheepishly, whispering mostly to myself *
> 
> All of that could be solved with new dunders.

What would be the signature of the new dunders? How will the new dunders 
be backwards compatible?

Previous discussions on this required runtime introspection to decide 
which set of dunders is called. That's going to be slow.

There's also the confusion of having two sets of very similar dunders. 
We got rid of `__getslice__` and friends back in the early 2.x days. Do 
we want to bring back another set of dunders with very similar names?

What would they be called? I've seen proposals `__get_item__` and 
`__getindex__` which are begging to be confused with `__getitem__`.


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AT33VZRZEA37DEYFE33B7HYLBCZWXYEE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Steven D'Aprano]

Quoting levels are a bit messed up in David's post, I've tried to fix 
them bu apologies if I'm attributing words to David that he didn't 
write.


On Thu, Sep 24, 2020 at 07:04:31PM -1000, David Mertz wrote:

> Is this a breaking change? It feels borderline.
> 
> > Keyword-only subscripts are permitted. The positional index will be 
> > the empty tuple:
> >
> >obj[spam=1, eggs=2]
> ># calls type(obj).__getitem__(obj, (), spam=1, eggs=2)
> 
> 
> I.e. consider:
> 
> >>> d = dict()
> >>> d[()] = "foo"
> >>> d
> {(): 'foo'}
> 
> 
> I don't really object to this fact, and one could argue it's not a breaking
> change since a built-in dict will simply raise an exception with keyword
> arguments.  However, it does make the empty tuple the "default key" for new
> objects that will accept keyword indices.

I agree with Ricky that the choice of empty tuple should be justified 
better by the PEP, and alternatives (None, NotImplemented) discussed. 
But I don't think this is a breaking change. Can you explain further 
what you think will break?



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZQZZPSBUJWHQW7QGJL4677SDT77UV2AL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Steven D'Aprano]

On Fri, Sep 25, 2020 at 04:58:00PM -0400, Wes Turner wrote:
> On Fri, Sep 25, 2020 at 1:43 PM Steven D'Aprano  wrote:

> > Of course rare or unusual exceptions probably won't be discovered
> > without a lot of testing, including stress testing, or not until the
> > code goes out into production. That's okay.
> >
> 
> While there are plenty of ways to debug in production,
> debugging in production is a bad idea and is not allowed (because: __, __,
> __) :
> log the exception with necessary details (traceback, exception attrs,  stack frame>) but exclude sensitive information that shouldn't be leaking
> into the logging system.

I hear you about sensitive information leaking into the logging system. 
But production code is not necessarily used by a third party where such 
leaks are a concern.

In a perfect world, our code would be 100% bug-free when we release it 
into production. And we have some strategies aimed to help approach 
that: TDD, fuzzing, test coverage metrics, etc.

If we are writing software that could kill people if we get it wrong:

https://hackaday.com/2015/10/26/killed-by-a-machine-the-therac-25/

this level of care is justified.

But under normal circumstances, don't let the perfect get in the way 
of the good. "Good enough" depends on what the code does and the 
consequences of a failure.

https://en.wikipedia.org/wiki/Perfect_is_the_enemy_of_good

In practice, we have a number of competing desires. We want the software 
for a reason, and we might not want to wait for it to be absolutely 100% 
bug free before actually using it in production. (Especially since we 
can never be sure that it is bug-free.)

So we have to expect some bugs will be reported from production. This is 
fine. We shouldn't let purists bully or shame us into thinking that this 
is necessarily the wrong thing. Sometimes "good enough, right now" is 
better than "perfect, in three years".



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T6A7EIBC4RTUPE4V33JELUBT5HKYNPUT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Steven D'Aprano]

On Fri, Sep 25, 2020 at 08:01:39PM +0100, Paul Moore wrote:
> On Fri, 25 Sep 2020 at 18:42, Steven D'Aprano  wrote:
> > There may be practical difficulties in sticking exceptions into
> > annotations. Annotations already can be pretty long and bulky. But if
> > you are okay with functions documenting that they might raise a certain
> > exception, then *in principle* you should be okay with moving that into
> > an annotation rather than the docstring.
> >
> > Annotations are just a form of documentation in a consistent standard
> > format to make it easy for IDEs to read them.
> 
> Expressed that way, it's a reasonable argument. And to the extent that
> we're talking about a way for functions to note what exceptions the
> developer feels are worth mentioning, having a language construct to
> do this would be OK.

Indeed. But the practical difficulties aren't trivial.

There's the issue of mere ackwardness: there's already a tonne of 
information in a complex function signature, adding annotations pretty 
much doubles that, and adding exceptions could easily increase that 
again by fifty percent.

More importantly, attempts in other languages to declare exceptions have 
been unsuccessful, if not counter-productive, for reasons that Chris 
explains.

On the other hand, for many years before mypy came on the 
scene, IDEs parsed docstrings for parameter information, with at least 
four different standards for declaring parameters:

- Epydoc
- ReST format
- Google's format
- Numpydoc format

https://stackoverflow.com/questions/3898572/what-is-the-standard-python-docstring-format

There may be others, and all of these seem to have support for declaring 
the raised exception types, so presumably IDEs have been able to make 
use of this information for a decade or more and the world hasn't ended.


> But in reality, people would treat the annotation as more definitive
> than it is. And people would feel pressure to "stick to the contract"
> when writing the annotated code. And style guides (which like it or
> not are sometimes written by people who are *not* experienced
> developers) will appear which state rules rather than allowing for
> nuanced interpretation. All of which might not be how the feature was
> intended to be used. I know, why assume the worst? But in contrast,
> what's the *best* outcome?

Yes, Chris does a good job of explaning the problems, and I'm not going 
to repeat them here.

And the benefit seems small: so your IDE will autocomplete the 
exceptions for you.

I think this may come down to this:

- people who use IDEs think that autocompletion in general is a
  great feature and may want want to extend that to exceptions;

- people who don't use IDEs think autocomplete is a very weak
  reason for adding a feature to the language.

My own personal feeling is "so I don't have to look up the docs" is a 
weak feature, especially given that an abbreviated version of the docs 
are available from the interactive interpreter:

help(some_function)


The bottom line is that I think any impetus for this feature would have 
to come from the mypy and other static tools community. Can they use it? 
I don't think it's far wrong to say that if Guido and the mypy devs want 
this, they will probably get it, and if they don't, you probably won't 
convince them to add it just for the sake of autocomplete.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7N5VR66P5AH4LHEMETVNHC5LI4D7CDZN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Wes Turner]

Search upwards for what (if anything) is going to catch an Exception would
be a mighty useful static/dynamic analysis thing. Is there anything that
can do this?

I'd hazard to guess that *most* apps will just crash and expect the process
spawner (e.g. systemd, supervisord, not sysV init) to re-spawn the crashed
process after the unhandled fault?

https://github.com/analysis-tools-dev/static-analysis#python
https://github.com/analysis-tools-dev/dynamic-analysis#python
https://github.com/vinta/awesome-python#code-analysis

On Fri, Sep 25, 2020 at 8:35 PM Greg Ewing 
wrote:

> On 26/09/20 4:32 am, Oscar Benjamin wrote:
> > annotations could be used to document in a statically
> > analysable way what the "expected" exceptions are. A type checker
> > could use those to check whether a caller is handling the *expected*
> > exceptions
>
> But that would be inappropriate. Even if an exception is
> "expected" (assuming for a moment we agree on what that means),
> the immediate caller is *not* obliged to handle it. Just as
> with any other exception, it's perfectly fine to let it propagate
> up to a level where something sensible can be done with it.
>
> Treating this as an error would be more annoying than helpful in
> most situations, I think.
>
> There are a few cases where it *might* make sense, such as
> StopIteration, which is pretty much part of the function's API
> and letting it escape is probably a mistake. But such cases
> are very rare and probably not worth adding a new language
> mechanism for.
>
> --
> Greg
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/DYD26ANQUYTIAO4ABCJJGQIXN7LX77ID/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H3TIFOAVG536N2QU4QTZ7F3V4UYC3LDN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Greg Ewing]

On 26/09/20 4:32 am, Oscar Benjamin wrote:

annotations could be used to document in a statically
analysable way what the "expected" exceptions are. A type checker
could use those to check whether a caller is handling the *expected*
exceptions


But that would be inappropriate. Even if an exception is
"expected" (assuming for a moment we agree on what that means),
the immediate caller is *not* obliged to handle it. Just as
with any other exception, it's perfectly fine to let it propagate
up to a level where something sensible can be done with it.

Treating this as an error would be more annoying than helpful in
most situations, I think.

There are a few cases where it *might* make sense, such as
StopIteration, which is pretty much part of the function's API
and letting it escape is probably a mistake. But such cases
are very rare and probably not worth adding a new language
mechanism for.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DYD26ANQUYTIAO4ABCJJGQIXN7LX77ID/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Wes Turner]

Depth-first-search for every callable, [property] descriptor, operator,
__getitem__

On Fri, Sep 25, 2020 at 4:36 PM Wes Turner  wrote:

>
> LSP servers and clients: https://langserver.org/
> - https://github.com/microsoft/python-language-server
>   -
> https://code.visualstudio.com/api/language-extensions/programmatic-language-features
>

What static and/or dynamic analysis tools can find all handled and
unhandled exceptions in every transitive {callable, [property] descriptor,
operator, __getitem__} given a callable?

def find_all_exceptions(callable):
"""
Args:
callable (callable): callable to traverse from in search of handled
and unhandled exceptions
Returns:
 list: nested lists of items [exception, [exception_locations],
[exception_handler_locations]] (?)
"""

What subset of these exceptions would a code annotation and/or docstring
happen to helpfully contain?

- AFAIU, there's not yet anything in any LSP Language Extensions for
> statically or dynamically discovering exceptions from the DFS of all
> references to types and callables of a callable
>
> (Type) Annotation type inference tools:
>
> pytype (Google) [1], PyAnnotate (Dropbox) [2], and MonkeyType (Instagram)
>> [3] all do dynamic / runtime PEP-484 type annotation type inference [4]
>> [1] https://github.com/google/pytype
>> [2] https://github.com/dropbox/pyannotate
>> [3] https://github.com/Instagram/MonkeyType
>> [4] https://news.ycombinator.com/item?id=19454411
>
>
I haven't realized that I had need for a tool that does this either; but I
guess I've effectively just manually searched for `raise` and `except` (and
failed to sufficiently fuzz with every possible combination of
user-supplied inputs)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GG7C246C2YGOBI4OMWBGO7DZYC5H7IYN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 10:19 AM Wes Turner  wrote:
>
>
>
> On Fri, Sep 25, 2020 at 8:09 PM Chris Angelico  wrote:
>>
>> On Sat, Sep 26, 2020 at 8:17 AM Wes Turner  wrote:
>> >
>> > What a worthless semantic distinction.
>> >
>> > You don't want to be executing code to determine why an Exception occurred 
>> > because you do not trust support devs to access all of the data in the 
>> > production system.
>> >
>> > "Exceptions happen" is true, but that's not satisfactory in an 
>> > organization focused on quality.
>> >
>>
>>  wut?
>>
>> I don't understand what you mean here. Sometimes a traceback isn't all
>> the information, and you need to add code to something to figure out
>> the cause. Are you implying that, because I actually put effort into
>> tracking bugs down, I am clearly not focused on quality??
>>
>> I'll just go ahead and repeat myself: "what?"
>
>
> - The context of this tangent was ~"[Exceptions happen in production] and 
> that's okay".
>   - Looseness with exception handling is entirely unacceptable for 
> safety-critical and high availability applications.
> - *Interactive debugging* is fine for many non-production applications.
> - Yup, the process spawner is going to reap and restart when the process 
> fails due to an unhandled Exception,
>but data loss due to unhandled exceptions (and non-transactional data 
> persistence) is a real problem
>- "It's just going to raise whatever sometimes and you can just pdb it 
> every time"
> - You can add information to exceptions and preserve the traceback with 
> `raise _ from _`
>
> I have nothing further to add about this non-personal tangent.

Okay, I can see what's happening. You're assuming that "debugging"
means "interactive debugging in a specialized harness". I consider
"debugging" to be "anything done with a view to removing a bug". So by
my definition, debugging in production is normal, and by yours, it's
impossible/unwise. I think we're just arguing terminology here, and
either way, checked exceptions won't really help.

There's nothing further to debate - we just use the word "debugging"
differently. :)

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/A7W3CISZRXMAZTA23UCPS5GIE4FBQZXP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Wes Turner]

On Fri, Sep 25, 2020 at 8:09 PM Chris Angelico  wrote:

> On Sat, Sep 26, 2020 at 8:17 AM Wes Turner  wrote:
> >
> > What a worthless semantic distinction.
> >
> > You don't want to be executing code to determine why an Exception
> occurred because you do not trust support devs to access all of the data in
> the production system.
> >
> > "Exceptions happen" is true, but that's not satisfactory in an
> organization focused on quality.
> >
>
>  wut?
>
> I don't understand what you mean here. Sometimes a traceback isn't all
> the information, and you need to add code to something to figure out
> the cause. Are you implying that, because I actually put effort into
> tracking bugs down, I am clearly not focused on quality??
>
> I'll just go ahead and repeat myself: "what?"
>

- The context of this tangent was ~"[Exceptions happen in production] and
that's okay".
  - Looseness with exception handling is entirely unacceptable for
safety-critical and high availability applications.
- *Interactive debugging* is fine for many non-production applications.
- Yup, the process spawner is going to reap and restart when the
process fails due to an unhandled Exception,
   but data loss due to unhandled exceptions (and non-transactional
data persistence) is a real problem
   - "It's just going to raise whatever sometimes and you can just pdb
it every time"
- You can add information to exceptions and preserve the traceback with
`raise _ from _`

I have nothing further to add about this non-personal tangent.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JSUBOGJLQTIIJWYMLP2DZFSKMM4AQYDG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 8:48 AM Oscar Benjamin
 wrote:
>
> On Fri, 25 Sep 2020 at 22:05, Chris Angelico  wrote:
> >
> > On Sat, Sep 26, 2020 at 6:56 AM Oscar Benjamin
> >  wrote:
> > >
> > > There are many common bugs that arise as a result of exceptions that
> > > are overlooked. There are also often simple ways to rewrite the code
> > > so that it becomes exception free. For example in sympy:
> > >
> > > if x.is_positive:  # never raises (excepting bugs) - False if 
> > > indeterminate
> > > if x > 0: # raises if indeterminate
> >
> > The novice believes that his ultimate goal is to stop the program from
> > crashing. If "exception free" is your goal, then sure, this feature
> > will certainly help you. I don't want Python to have language support
> > for something that is primarily aimed at helping novices to remain
> > forever novices.
>
> Yeah, because real professionals write applications that spew out
> tracebacks to their users for perfectly valid inputs and real users
> love that :)
>
> There are good reasons for writing your code in such a way that it
> handles all cases rather than blowing up. Of course if you can't
> handle all cases then that's one good reason for raising an exception.

That's something you do at a single location, usually - an error
boundary. There are points in the code where it's correct to absorb
and log all errors, but what that really means is that you're handling
Exception (or BaseException) - you still don't need to enumerate every
possible exception and you don't want to.

I still don't know of ANY situation in Python where I have wanted a
full list of every exception something might raise.

> The way that I would propose is this:
>
> Yes, g can possibly raise any exception class. As you say it's not
> generally possible to verify that arbitrary Python code can not raise
> any exception outside of a nontrivial set (BaseException).
>
> However g can not raise any exception from a function that was
> explicitly annotated as raising that exception. If some function h is
> annotated as raising FooError then a FooError from h will not arise
> from calling g.
>

I'm not sure I follow. Maybe some example code? Are you saying that,
if g calls h and h is declared as raising FooError, g must be declared
as raising FooError or must catch it, but any other exception is fair
game?

Very confused and still don't see any benefit here.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PCLEVTHFBKQEDNSPXDTIHAKITD6UVF6H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 8:17 AM Wes Turner  wrote:
>
> What a worthless semantic distinction.
>
> You don't want to be executing code to determine why an Exception occurred 
> because you do not trust support devs to access all of the data in the 
> production system.
>
> "Exceptions happen" is true, but that's not satisfactory in an organization 
> focused on quality.
>

 wut?

I don't understand what you mean here. Sometimes a traceback isn't all
the information, and you need to add code to something to figure out
the cause. Are you implying that, because I actually put effort into
tracking bugs down, I am clearly not focused on quality??

I'll just go ahead and repeat myself: "what?"

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XVNFVANNIJUBIHTYD7LJRRGH72X5OKAV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Oscar Benjamin]

On Fri, 25 Sep 2020 at 22:05, Chris Angelico  wrote:
>
> On Sat, Sep 26, 2020 at 6:56 AM Oscar Benjamin
>  wrote:
> >
> > There are many common bugs that arise as a result of exceptions that
> > are overlooked. There are also often simple ways to rewrite the code
> > so that it becomes exception free. For example in sympy:
> >
> > if x.is_positive:  # never raises (excepting bugs) - False if 
> > indeterminate
> > if x > 0: # raises if indeterminate
>
> The novice believes that his ultimate goal is to stop the program from
> crashing. If "exception free" is your goal, then sure, this feature
> will certainly help you. I don't want Python to have language support
> for something that is primarily aimed at helping novices to remain
> forever novices.

Yeah, because real professionals write applications that spew out
tracebacks to their users for perfectly valid inputs and real users
love that :)

There are good reasons for writing your code in such a way that it
handles all cases rather than blowing up. Of course if you can't
handle all cases then that's one good reason for raising an exception.
Raising an exception just because you wrote your code in slightly the
wrong way is obviously something to avoid though. A tool that helps
you to see non-obvious ways that exceptions can be raised can be very
useful.

> > That is *not* what I am suggesting. Rather only an explicit raises
> > hint could give an error so the above is fine. The type checker knows
> > that g might raise FooError but there's nothing wrong with raising an
> > error without any declaration. However the type checker *would*
> > complain about this:
> >
> > def f(): raises FooError
> > ...
> >
> > def g(): raises None
> > f()   # <--- typecheck error because g claims that it does not raise
> >
>
> Okay. So what exceptions can a "raises None" function raise? Or if it
> "raises FooError", then what, other than FooError, can it raise? If
> this is to have the blessing of the language, there needs to be an
> absolutely 100% consistent answer to that question. And there's only
> one valid answer that will always be correct, and it's the useless
> one: BaseException.

The way that I would propose is this:

Yes, g can possibly raise any exception class. As you say it's not
generally possible to verify that arbitrary Python code can not raise
any exception outside of a nontrivial set (BaseException).

However g can not raise any exception from a function that was
explicitly annotated as raising that exception. If some function h is
annotated as raising FooError then a FooError from h will not arise
from calling g.


Oscar
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7O5ITX23PIFDM3L3CUKB657224GULXPA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Wes Turner]

What a worthless semantic distinction.

You don't want to be executing code to determine why an Exception
occurred because you do not trust support devs to access all of the data in
the production system.

"Exceptions happen" is true, but that's not satisfactory in an organization
focused on quality.

On Fri, Sep 25, 2020 at 5:38 PM Chris Angelico  wrote:

> On Sat, Sep 26, 2020 at 7:25 AM Wes Turner  wrote:
> >
> > (Executing arbitrary code on a production server is debugging in
> production.
> >
> > Logging (additional information added to exceptions with 'raise _ from
> _`) may assist with root cause analysis and debugging on a different
> instance but IMHO logging is not debugging.)
> >
>
> Uhh... okay sure. What if you add instrumentation to the live
> server specifically so that it can log useful exceptions as you're
> trying to probe a bug that shows up only in prod and only once every
> two weeks? Is that debugging in prod, or is that logging?
>
> Or... does the distinction really even matter, and "debugging in prod"
> is just part of life?
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/THD4Y4RKERBSFUTU36EQHBEPDGTLIVQM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/G3K55L2GFHP2R3PSTK5POM4E4BPZZH6P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Stefano Borini]

Ricky, I'd love new dunders too but it would explode in complexity.
The likelihood of being approved is pretty much non-existent as far as
I understand.

If you want, we can write together a competing PEP with that solution,
so that the Steering Council can have two ideas. I would not mind
exploring that option a bit more, but it's likely to become an
exercise.

On Fri, 25 Sep 2020 at 22:06, Ricky Teachey  wrote:
>
> On Fri, Sep 25, 2020 at 4:57 PM Stefano Borini  
> wrote:
>>
>> On Fri, 25 Sep 2020 at 14:07, Ricky Teachey  wrote:
>> >
>> > I'd like to hear more about why the empty tuple has been selected as the 
>> > default index.
>>
>> It's still not settled. Steven proposed None, I propose empty tuple
>> for affinity with *args behavior, others proposed no positional arg at
>> all, but then there's the problem of the setter.
>> I sent a mail to the numpy mailing list yesterday, and all people who
>> replied seemed to prefer empty tuple.
>>
>> Here is the thread
>>
>> http://numpy-discussion.10968.n7.nabble.com/Request-for-comments-on-PEP-637-Support-for-indexing-with-keyword-arguments-td48489.html
>>
>>
>>
>> --
>> Kind regards,
>>
>> Stefano Borini
>
>
> * sheepishly, whispering mostly to myself *
>
> All of that could be solved with new dunders.
>
> ---
> Ricky.
>
> "I've never met a Kentucky man who wasn't either thinking about going home or 
> actually going home." - Happy Chandler
>
>



-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TVDJHXCDPL4POOFMF4WKKDAPIRHJ4LUO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 7:25 AM Wes Turner  wrote:
>
> (Executing arbitrary code on a production server is debugging in production.
>
> Logging (additional information added to exceptions with 'raise _ from _`) 
> may assist with root cause analysis and debugging on a different instance but 
> IMHO logging is not debugging.)
>

Uhh... okay sure. What if you add instrumentation to the live
server specifically so that it can log useful exceptions as you're
trying to probe a bug that shows up only in prod and only once every
two weeks? Is that debugging in prod, or is that logging?

Or... does the distinction really even matter, and "debugging in prod"
is just part of life?

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/THD4Y4RKERBSFUTU36EQHBEPDGTLIVQM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Wes Turner]

(Executing arbitrary code on a production server is debugging in production.

Logging (additional information added to exceptions with 'raise _ from _`)
may assist with root cause analysis and debugging on a different instance
but IMHO logging is not debugging.)

On Fri, Sep 25, 2020, 5:18 PM Chris Angelico  wrote:

> On Sat, Sep 26, 2020 at 7:05 AM Wes Turner  wrote:
> >> "Well, if I get an import error, I can add some more directories to
> >> sys.path and try adding the values again, that might fix it..."
> >>
> >> Who does that? Not me. And I bet you don't either.
> >
> >
> > "Defensive programming" / "Offensive programming"
> > https://en.wikipedia.org/wiki/Defensive_programming
>
> Do you plan for every possible exception from every possible line of
> code? Really? Okay, show me something that catches ImportError from
> "import math" then. What's in your except clause?
>
> > While there are plenty of ways to debug in production,
> > debugging in production is a bad idea and is not allowed (because: __,
> __, __) :
> > log the exception with necessary details (traceback, exception attrs,
> ) but exclude sensitive information that shouldn't be
> leaking into the logging system.
> >
>
> That IS debugging in production. I don't understand why you say that
> that's a bad thing.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/PYQ6KWUGIVRH4BSDGAA2JLBCVBSTLKSJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Y46CYIII6YMMHNLGCV4MTO3Q7OANMTVG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 7:05 AM Wes Turner  wrote:
>> "Well, if I get an import error, I can add some more directories to
>> sys.path and try adding the values again, that might fix it..."
>>
>> Who does that? Not me. And I bet you don't either.
>
>
> "Defensive programming" / "Offensive programming"
> https://en.wikipedia.org/wiki/Defensive_programming

Do you plan for every possible exception from every possible line of
code? Really? Okay, show me something that catches ImportError from
"import math" then. What's in your except clause?

> While there are plenty of ways to debug in production,
> debugging in production is a bad idea and is not allowed (because: __, __, 
> __) :
> log the exception with necessary details (traceback, exception attrs,  stack frame>) but exclude sensitive information that shouldn't be leaking 
> into the logging system.
>

That IS debugging in production. I don't understand why you say that
that's a bad thing.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PYQ6KWUGIVRH4BSDGAA2JLBCVBSTLKSJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 6:56 AM Oscar Benjamin
 wrote:
>
> There are many common bugs that arise as a result of exceptions that
> are overlooked. There are also often simple ways to rewrite the code
> so that it becomes exception free. For example in sympy:
>
> if x.is_positive:  # never raises (excepting bugs) - False if 
> indeterminate
> if x > 0: # raises if indeterminate

The novice believes that his ultimate goal is to stop the program from
crashing. If "exception free" is your goal, then sure, this feature
will certainly help you. I don't want Python to have language support
for something that is primarily aimed at helping novices to remain
forever novices.

> That is *not* what I am suggesting. Rather only an explicit raises
> hint could give an error so the above is fine. The type checker knows
> that g might raise FooError but there's nothing wrong with raising an
> error without any declaration. However the type checker *would*
> complain about this:
>
> def f(): raises FooError
> ...
>
> def g(): raises None
> f()   # <--- typecheck error because g claims that it does not raise
>

Okay. So what exceptions can a "raises None" function raise? Or if it
"raises FooError", then what, other than FooError, can it raise? If
this is to have the blessing of the language, there needs to be an
absolutely 100% consistent answer to that question. And there's only
one valid answer that will always be correct, and it's the useless
one: BaseException.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RYCEOPKWRLRGSWAE5BB5FRC7PMFBSF4K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Ricky Teachey]

On Fri, Sep 25, 2020 at 4:57 PM Stefano Borini 
wrote:

> On Fri, 25 Sep 2020 at 14:07, Ricky Teachey  wrote:
> >
> > I'd like to hear more about why the empty tuple has been selected as the
> default index.
>
> It's still not settled. Steven proposed None, I propose empty tuple
> for affinity with *args behavior, others proposed no positional arg at
> all, but then there's the problem of the setter.
> I sent a mail to the numpy mailing list yesterday, and all people who
> replied seemed to prefer empty tuple.
>
> Here is the thread
>
>
> http://numpy-discussion.10968.n7.nabble.com/Request-for-comments-on-PEP-637-Support-for-indexing-with-keyword-arguments-td48489.html
>
>
>
> --
> Kind regards,
>
> Stefano Borini


* sheepishly, whispering mostly to myself *

All of that could be solved with new dunders.

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BSCNXKAAOB5V72AJULUXP5MDKAY7CGZQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Wes Turner]

On Fri, Sep 25, 2020 at 1:43 PM Steven D'Aprano  wrote:

> On Fri, Sep 25, 2020 at 11:14:01PM +1000, Chris Angelico wrote:
> > On Fri, Sep 25, 2020 at 7:59 PM Sergio Fenoll  wrote:
>
> > > Surely there has to be a better way of programming than running stuff,
> > > watching it fail, and then keeping track of how it fails so you can
> > > later handle that failure?
> >
> > Why? Do you really think you can enumerate EVERY possible way that
> > something might fail?
>
> Nobody is demanding that "EVERY possible way" is handled -- but if you
> need to, then Python lets you do so:
>
> # Please don't do this.
> try:
> something()
> except:
> # handle EVERY error
>
> Of course this is nearly always the wrong choice. "What can I cope with"
> is easy: it's always *everything*, if you define "cope with" as just
> suppressing the error.
>
> > Think, instead, about all the possible problems that you can actually
> > cope with. That way, you have a finite - and usually small - set of
> > things to deal with, instead of an infinite field of "well this could
> > go wrong, but we can't do anything about that".
>
> The problem here is that you can't decide what you can deal with in
> isolation. I can deal with UnicodeEncodeError easily: try again with a
> different encoding, or with a different error handler, easy-peasy.
>
> But if I'm doing `y = x + 1` and it somehow raised UnicodeEncodeError,
> what do I do? I'm stuck.
>
> In order to tell what you can deal with, you need to know the
> circumstances of the error, and why it occurred. In other words, you
> need to understand the operation being called, in particular, what
> exceptions it might raise under normal circumstances.
>
> I'm pretty confident that most people, unless they are TDD zealots,
> start by using either their pre-existing knowledge of the operation, or
> reading the documentation, to find out what exceptions are likely under
> normal circumstances, and *only then* start to think about how to deal
> with such exceptions.
>
> The alternative is to waste time and mental energy thinking about how to
> deal with exceptions that you will probably never get in real life:
>
> "Well, if I get an import error, I can add some more directories to
> sys.path and try adding the values again, that might fix it..."
>
> Who does that? Not me. And I bet you don't either.
>

"Defensive programming" / "Offensive programming"
https://en.wikipedia.org/wiki/Defensive_programming

... Tacking additional information onto the exception and re-raising may be
the helpful thing to do; though that's still not handling the situation.

Recently I learned about the `raise _ from _` syntax (when writing an
example implementation for "[Python-ideas] f-strings as assignment targets":

"""
def cast_match_groupdict(matchobj, typemap):
matchdict = matchobj.groupdict()
if not typemap:
return matchdict
for attr, castfunc in typemap.items():
try:
matchdict[attr] = castfunc(matchdict[attr])
except ValueError as e:
raise ValueError(("attr", attr), ("rgx", matchobj.re)) from e
return matchdict
"""


>
> So I think that most of us:
>
> - start with documented or well-known exceptions;
>
> - and only then decide whether or not we can deal with them.
>
> Of course rare or unusual exceptions probably won't be discovered
> without a lot of testing, including stress testing, or not until the
> code goes out into production. That's okay.
>

While there are plenty of ways to debug in production,
debugging in production is a bad idea and is not allowed (because: __, __,
__) :
log the exception with necessary details (traceback, exception attrs, ) but exclude sensitive information that shouldn't be leaking
into the logging system.

Catching exceptions early is easier when:
- TDD / test coverage are emphasized
- fuzzing is incorporated into the release process (fuzzing is easier with
parameterized test cases)
- unit/functional/integration testing in a copy of production (sufficient
DevOps/DevSecOps)

- the coding safety guide says that all exceptions must be handled


> And I think that is Sergio's point: it would be good to have a standard,
> consistent place for functions to document which exceptions they are
> likely to raise under normal circumstances, and one which is available
> to IDEs and runtime inspection. Annotations.
>

Annotation: (type)
Docstring: (type, docstr)


>
> Of course we can inspect the docstring of the function, but it's hard
> for an automated tool to distinguish:
>
> This will raise WidgetExplosionError if the widget explodes.
>
> from:
>
> This is guaranteed to never raise WidgetExplosionError even
> if the widget explodes.
>
>
> There may be practical difficulties in sticking exceptions into
> annotations. Annotations already can be pretty long and bulky. But if
> you are okay with functions documenting that they might raise a certain
> exception, then *in principle* you should 

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Stefano Borini]

On Fri, 25 Sep 2020 at 14:07, Ricky Teachey  wrote:
>
> I'd like to hear more about why the empty tuple has been selected as the 
> default index.

It's still not settled. Steven proposed None, I propose empty tuple
for affinity with *args behavior, others proposed no positional arg at
all, but then there's the problem of the setter.
I sent a mail to the numpy mailing list yesterday, and all people who
replied seemed to prefer empty tuple.

Here is the thread

http://numpy-discussion.10968.n7.nabble.com/Request-for-comments-on-PEP-637-Support-for-indexing-with-keyword-arguments-td48489.html



-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5CGHSL5G3ILP5FLX4PY4VDGSRAXKOL3S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Oscar Benjamin]

On Fri, 25 Sep 2020 at 20:36, Chris Angelico  wrote:
>
> On Sat, Sep 26, 2020 at 5:27 AM Oscar Benjamin
>  wrote:
> >
> > On Fri, 25 Sep 2020 at 18:59, Chris Angelico  wrote:
> > >
> > > On Sat, Sep 26, 2020 at 2:32 AM Oscar Benjamin
> > >  wrote:
> > > > I do agree but maybe that suggests a different role for annotated
> > > > exceptions in Python. Rather than attempting to enumerate all possible
> > > > exceptions annotations could be used to document in a statically
> > > > analysable way what the "expected" exceptions are. A type checker
> > > > could use those to check whether a caller is handling the *expected*
> > > > exceptions rather than to verify that the list of *all* exceptions
> > > > possibly raised is exhaustive.
> > > >
> > > > Consider an example:
> > > >
> > > > def inverse(M: Matrix) -> Matrix: raises(NotInvertibleError)
> > > > if determinant(M) == 0:
> > > > raise NotInvertibleError
> > > > rows, cols = M.shape
> > > > for i in range(rows):
> > > > for j in range(cols):
> > > > ...
> > > >
> > > > Here the function is expected to raise NotInvertibleError for some
> > > > inputs.
> > >
> > > Please no. Once again, this will encourage "handling" of errors by
> > > suppressing them. Don't have anything that forces people to handle
> > > exceptions they can't handle.
> >
> > Why would anyone be forced to "handle" the exceptions?
> >
> > The suggestion (which you've snipped away) was not that it would be a
> > type-checking error to call the inverse function without catching the
> > exception. It only becomes a type-checking error if a function calls
> > the inverse function and claims not to raise the exception. The
> > checker just verifies the consistency of the different claims about
> > what is raised.
> >
>
> Forcing people to declare it is exactly the same as forcing them to
> handle it every other way. There are many MANY bad ways this can go,
> most of which have been mentioned already:

I wouldn't suggest that a declaration be forced but rather that a
false declaration be disallowed. More below...

> 1) Catching exceptions and doing nothing, just so your linter shuts up
> 2) Catching exceptions and raising a generic "something went wrong"
> error so that's the only thing you have to declare
> 3) Declaring that you could raise BaseException, thus adding useless
> and meaningless boilerplate to your functions
> 4) Redeclaring every exception all the way up the chain, and having to
> edit them every time something changes
> 5) Copying and pasting a gigantic list of exceptions onto every
> function's signature

I'm sure those things would happen sometimes. At least all of these
things are very visible in the diff so a reviewer can at least see
what's going on. The tricky thing with exceptions is the action at a
distance effect so that it's not always obvious what the effect of a
change is so when you see a diff like:

   -   x = f()
   +  x = g()

you would potentially need to trawl a lot of code to see how that
affects what exceptions might be raised.

> And what are the possible *good* ways to use this? What do you
> actually gain? So far, the only use-case I've heard is "IDEs might
> help you tab-complete an except clause". That's only going to be
> useful if the exception tracking is accurate all the way up and down
> the tree, and that's only going to happen if it's done by analysis,
> NOT by forcing people to declare them.

I can tell you what I would use a feature like this for (if it is
implemented in a useful way) which is cleaning up the exception
handling in the sympy codebase. It's a large codebase with thousands
of raises and excepts:

$ git grep raise | wc -l
8186
$ git grep except | wc -l
1657

There are many common bugs that arise as a result of exceptions that
are overlooked. There are also often simple ways to rewrite the code
so that it becomes exception free. For example in sympy:

if x.is_positive:  # never raises (excepting bugs) - False if indeterminate
if x > 0: # raises if indeterminate

That distinction is not understood by many contributors to sympy but
is easy to explain during code review. That's an easy example but
others are not so easy.

> I've worked with Java's checked exceptions, which is what you're
> getting at here. They're terrible. The best way to handle them is to
> ensure that all your exceptions are RuntimeErrors so they don't get
> checked by the compiler.

My understanding of the situation in Java although I don't have much
experience of it as that checked exceptions must *always* be declared
so e.g. (in hypothetical Python syntax):

def f(): raises FooError
...

def g():
f()   # <--- typecheck error because g does not declare FooError

That is *not* what I am suggesting. Rather only an explicit raises
hint could give an error so the above is fine. The type checker knows
that g might raise FooError but there's nothing wrong with raising an
error without any 

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Stefano Borini]

On Fri, 25 Sep 2020 at 05:55, David Mertz  wrote:
> Smuggling in a generic function calls in square brackets is undesirable (but 
> obviously not preventable at a language level).  However, I feel like 
> keywords might very often "alter the behavior."
>
> For example, imagine we have a distributed array within an engine that is 
> intended to be "eventually consistent."  I would find code like this, in some 
> hypothetical library, to be clear and useful, and not to violate the spirit 
> of indexing.
>
>> snapshot1 = remote_array[300:310, 50:60, 30:35, source=worker1]
>> snapshot2 = remote_array[300:310, 50:60, 30:35, source=worker2]

I would personally be very uneasy with this. A better solution would
be to have a proxy object that handles that:

>> snapshot1 = remote_array.source(worker1)[300:310, 50:60, 30:35]

Of course people can (and will) abuse the feature, but I would
personally consider it poor design. These tricks were discussed in
PEP-472 (e.g. specify the unit to be returned), but I always felt
uncomfortable with them.

-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZJHMFBANR3FZLYMEAM4XX5LYLGTTMZ3Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Christopher Barker]

On Fri, Sep 25, 2020 at 1:36 PM MRAB  wrote:

> It currently doesn't support multiple indexes, so there's no distinction
> between one index that's a 2-tuple and 2 indexes: d[(1, 2)] == d[1, 2].
>

yeah, but one index isn't in a 1-tuple (much discussed on this thread), so
now we have the someone awkward (if consistent with the language):

obj[i] -->  __getitem__(i)
obj[i, j] -->  __getitem__((i, j))

if make a default an empty tuple, then we'll have:

obj[] -->  __getitem__(())
obj[i] -->  __getitem__(i)
obj[i, j] -->  __getitem__((i, j))

Or would the default only be used if there were one or more keyword
arguments? if so, we'd still have:

obj[keyword=k] -->  __getitem__((), keyword=k)
obj[i, keyword=k] -->  __getitem__(i, keyword=k)
obj[i, j, keyword=k] -->  __getitem__((i, j), keyword=k)

Which is, shall we say, not ideal.

-CHB







> Using an empty tuple as the default index isn't that bad, assuming
> you're going to allow a default.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/MN7MOV4EL5K4MWGGSAKWZYLFZQFTI3Y2/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LAKGUS36CJ6M75QPANMX4RIWIYIC6NMH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[MRAB]

On 2020-09-25 20:36, Christopher Barker wrote:
On Fri, Sep 25, 2020 at 6:05 AM Ricky Teachey > wrote:


I'd like to hear more about why the empty tuple has been selected as
the default index. 



It makes sense to me: if more than one index is passed, they are passed 
as a tuple. so many classes need to handle tuples anyway.


What other options are there? I suppose None is a possibility, but None 
is a valid dict key, so probably not a great idea. Hmm, so is an empty 
tuple. Darn.


I think having no default is a better option, as someone pointed out 
already in this thread.


It currently doesn't support multiple indexes, so there's no distinction 
between one index that's a 2-tuple and 2 indexes: d[(1, 2)] == d[1, 2].


Using an empty tuple as the default index isn't that bad, assuming 
you're going to allow a default.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MN7MOV4EL5K4MWGGSAKWZYLFZQFTI3Y2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Wes Turner]

On Fri, Sep 25, 2020 at 3:03 PM Paul Moore  wrote:

> On Fri, 25 Sep 2020 at 18:42, Steven D'Aprano  wrote:
> > There may be practical difficulties in sticking exceptions into
> > annotations. Annotations already can be pretty long and bulky. But if
> > you are okay with functions documenting that they might raise a certain
> > exception, then *in principle* you should be okay with moving that into
> > an annotation rather than the docstring.
> >
> > Annotations are just a form of documentation in a consistent standard
> > format to make it easy for IDEs to read them.
>
> Expressed that way, it's a reasonable argument. And to the extent that
> we're talking about a way for functions to note what exceptions the
> developer feels are worth mentioning, having a language construct to
> do this would be OK.
>
> But in reality, people would treat the annotation as more definitive
> than it is. And people would feel pressure to "stick to the contract"
> when writing the annotated code. And style guides (which like it or
> not are sometimes written by people who are *not* experienced
> developers) will appear which state rules rather than allowing for
> nuanced interpretation. All of which might not be how the feature was
> intended to be used. I know, why assume the worst? But in contrast,
> what's the *best* outcome?
>

Exception docstrings contain the Exception type and the explanation str:
(type, docstr)

Exception annotations as proposed contain the Exception type: (type)
and presumably then docstrings would also be expected to contain (type,
docstr);
which isn't DRY.

But annotations are parseable and more commonly linted.


>
> My problem with the proposal is that it's about adding a mechanism,
> without any clear statement of how that mechanism is intended to be
> used. And there's so much scope for misguided use, and so few clear
> benefits, that it feels like a bad trade-off at the moment. (For the
> record, I'm not convinced by the IDE argument. I use Vim, which is
> relentlessly primitive in its basic form, and wouldn't use anything
> like this, and VS Code, which is so clever that it seems like magic
> most of the time, and I'm pretty certain that if people wanted it, VS
> Code could include "exception autocompletion" even without explicit
> annotation like this).
>

LSP servers and clients: https://langserver.org/
- https://github.com/microsoft/python-language-server
  -
https://code.visualstudio.com/api/language-extensions/programmatic-language-features
- AFAIU, there's not yet anything in any LSP Language Extensions for
statically or dynamically discovering exceptions from the DFS of all
references to types and callables of a callable

(Type) Annotation type inference tools:

pytype (Google) [1], PyAnnotate (Dropbox) [2], and MonkeyType (Instagram)
> [3] all do dynamic / runtime PEP-484 type annotation type inference [4]
> [1] https://github.com/google/pytype
> [2] https://github.com/dropbox/pyannotate
> [3] https://github.com/Instagram/MonkeyType
> [4] https://news.ycombinator.com/item?id=19454411
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DBYCXFI3ZQDNHZJGQ6BV7QKX7STHWUDQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Ricky Teachey]

On Fri, Sep 25, 2020 at 3:36 PM Christopher Barker 
wrote:

> On Fri, Sep 25, 2020 at 6:05 AM Ricky Teachey  wrote:
>
>> I'd like to hear more about why the empty tuple has been selected as the
>> default index.
>>
>
> It makes sense to me: if more than one index is passed, they are passed as
> a tuple. so many classes need to handle tuples anyway.
>
> What other options are there? I suppose None is a possibility, but None is
> a valid dict key, so probably not a great idea. Hmm, so is an empty tuple.
> Darn.
>
> I think having no default is a better option, as someone pointed out
> already in this thread.
>
> -CHB
>
>
That is where my thinking went as well, but I probably haven't thought
through all the implications.

Essentially, you'd be letting the write of the __XXXitem__ method(s) choose
the default, rather than making the decision for them.

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HPWEBGP3L6MGA6GILDVAKZHUEQNGMHXO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Christopher Barker]

On Fri, Sep 25, 2020 at 6:05 AM Ricky Teachey  wrote:

> I'd like to hear more about why the empty tuple has been selected as the
> default index.
>

It makes sense to me: if more than one index is passed, they are passed as
a tuple. so many classes need to handle tuples anyway.

What other options are there? I suppose None is a possibility, but None is
a valid dict key, so probably not a great idea. Hmm, so is an empty tuple.
Darn.

I think having no default is a better option, as someone pointed out
already in this thread.

-CHB






>
> On Fri, Sep 25, 2020, 1:07 AM David Mertz  wrote:
>
>> Is this a breaking change? It feels borderline.
>>
>> Keyword-only subscripts are permitted. The positional index will be the
>>> empty tuple:
>>> obj[spam=1, eggs=2]
>>> # calls type(obj).__getitem__(obj, (), spam=1, eggs=2)
>>
>>
>> I.e. consider:
>>
>> >>> d = dict()
>> >>> d[()] = "foo"
>> >>> d
>> {(): 'foo'}
>>
>>
>> I don't really object to this fact, and one could argue it's not a
>> breaking change since a built-in dict will simply raise an exception with
>> keyword arguments.  However, it does make the empty tuple the "default key"
>> for new objects that will accept keyword indices.
>>
>> --
>> The dead increasingly dominate and strangle both the living and the
>> not-yet born.  Vampiric capital and undead corporate persons abuse
>> the lives and control the thoughts of homo faber. Ideas, once born,
>> become abortifacients against new conceptions.
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/65KCDFYRUWH6K4HHMZT4UJTPDLE7JOPK/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/C4JVKD2ZUXU2Z6FDJOMHD5YYTACIHEIO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KFAUT77CH2IOIFR2WZXAIZ5DSJDEKCES/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 5:27 AM Oscar Benjamin
 wrote:
>
> On Fri, 25 Sep 2020 at 18:59, Chris Angelico  wrote:
> >
> > On Sat, Sep 26, 2020 at 2:32 AM Oscar Benjamin
> >  wrote:
> > > I do agree but maybe that suggests a different role for annotated
> > > exceptions in Python. Rather than attempting to enumerate all possible
> > > exceptions annotations could be used to document in a statically
> > > analysable way what the "expected" exceptions are. A type checker
> > > could use those to check whether a caller is handling the *expected*
> > > exceptions rather than to verify that the list of *all* exceptions
> > > possibly raised is exhaustive.
> > >
> > > Consider an example:
> > >
> > > def inverse(M: Matrix) -> Matrix: raises(NotInvertibleError)
> > > if determinant(M) == 0:
> > > raise NotInvertibleError
> > > rows, cols = M.shape
> > > for i in range(rows):
> > > for j in range(cols):
> > > ...
> > >
> > > Here the function is expected to raise NotInvertibleError for some
> > > inputs.
> >
> > Please no. Once again, this will encourage "handling" of errors by
> > suppressing them. Don't have anything that forces people to handle
> > exceptions they can't handle.
>
> Why would anyone be forced to "handle" the exceptions?
>
> The suggestion (which you've snipped away) was not that it would be a
> type-checking error to call the inverse function without catching the
> exception. It only becomes a type-checking error if a function calls
> the inverse function and claims not to raise the exception. The
> checker just verifies the consistency of the different claims about
> what is raised.
>

Forcing people to declare it is exactly the same as forcing them to
handle it every other way. There are many MANY bad ways this can go,
most of which have been mentioned already:

1) Catching exceptions and doing nothing, just so your linter shuts up
2) Catching exceptions and raising a generic "something went wrong"
error so that's the only thing you have to declare
3) Declaring that you could raise BaseException, thus adding useless
and meaningless boilerplate to your functions
4) Redeclaring every exception all the way up the chain, and having to
edit them every time something changes
5) Copying and pasting a gigantic list of exceptions onto every
function's signature

And what are the possible *good* ways to use this? What do you
actually gain? So far, the only use-case I've heard is "IDEs might
help you tab-complete an except clause". That's only going to be
useful if the exception tracking is accurate all the way up and down
the tree, and that's only going to happen if it's done by analysis,
NOT by forcing people to declare them.

I've worked with Java's checked exceptions, which is what you're
getting at here. They're terrible. The best way to handle them is to
ensure that all your exceptions are RuntimeErrors so they don't get
checked by the compiler.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SVTC3HU5NXX62FII37WI5BVGPHQUOO32/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Oscar Benjamin]

On Fri, 25 Sep 2020 at 18:59, Chris Angelico  wrote:
>
> On Sat, Sep 26, 2020 at 2:32 AM Oscar Benjamin
>  wrote:
> > I do agree but maybe that suggests a different role for annotated
> > exceptions in Python. Rather than attempting to enumerate all possible
> > exceptions annotations could be used to document in a statically
> > analysable way what the "expected" exceptions are. A type checker
> > could use those to check whether a caller is handling the *expected*
> > exceptions rather than to verify that the list of *all* exceptions
> > possibly raised is exhaustive.
> >
> > Consider an example:
> >
> > def inverse(M: Matrix) -> Matrix: raises(NotInvertibleError)
> > if determinant(M) == 0:
> > raise NotInvertibleError
> > rows, cols = M.shape
> > for i in range(rows):
> > for j in range(cols):
> > ...
> >
> > Here the function is expected to raise NotInvertibleError for some
> > inputs.
>
> Please no. Once again, this will encourage "handling" of errors by
> suppressing them. Don't have anything that forces people to handle
> exceptions they can't handle.

Why would anyone be forced to "handle" the exceptions?

The suggestion (which you've snipped away) was not that it would be a
type-checking error to call the inverse function without catching the
exception. It only becomes a type-checking error if a function calls
the inverse function and claims not to raise the exception. The
checker just verifies the consistency of the different claims about
what is raised.


Oscar
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TPGC6DJUSOHJN32UFYYRJRS53CZFXFPV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Paul Moore]

On Fri, 25 Sep 2020 at 18:42, Steven D'Aprano  wrote:
> There may be practical difficulties in sticking exceptions into
> annotations. Annotations already can be pretty long and bulky. But if
> you are okay with functions documenting that they might raise a certain
> exception, then *in principle* you should be okay with moving that into
> an annotation rather than the docstring.
>
> Annotations are just a form of documentation in a consistent standard
> format to make it easy for IDEs to read them.

Expressed that way, it's a reasonable argument. And to the extent that
we're talking about a way for functions to note what exceptions the
developer feels are worth mentioning, having a language construct to
do this would be OK.

But in reality, people would treat the annotation as more definitive
than it is. And people would feel pressure to "stick to the contract"
when writing the annotated code. And style guides (which like it or
not are sometimes written by people who are *not* experienced
developers) will appear which state rules rather than allowing for
nuanced interpretation. All of which might not be how the feature was
intended to be used. I know, why assume the worst? But in contrast,
what's the *best* outcome?

My problem with the proposal is that it's about adding a mechanism,
without any clear statement of how that mechanism is intended to be
used. And there's so much scope for misguided use, and so few clear
benefits, that it feels like a bad trade-off at the moment. (For the
record, I'm not convinced by the IDE argument. I use Vim, which is
relentlessly primitive in its basic form, and wouldn't use anything
like this, and VS Code, which is so clever that it seems like magic
most of the time, and I'm pretty certain that if people wanted it, VS
Code could include "exception autocompletion" even without explicit
annotation like this).

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LT6LFEHTUZY7SMFHEQU3K25ZQJIEUBIM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Stefano Borini]

On Thu, 24 Sep 2020 at 22:22, Guido van Rossum  wrote:
> - I recommend that you carefully look over the PEP as rendered on python.org 
> (link above) and try to fix any markup oddities. E.g. some comments are 
> line-wrapped, which looks ugly, and some bulleted lists have an extra blank 
> line above.
>
> - Looking at the generated ToC, you have two top-level sections labeled 
> "Syntax and Semantics". That seems a bit odd. I'm not sure what you meant to 
> happen here, but I recommend renaming at least one of these. (Another 
> recommendation: don't mix description of the status quo with the 
> specification.)

No matter how many times one re-reads something, there's always going
to be a typo :)
Just as a question, the generated html using make pep-0637.html are
not supposed to get the css style. I get no stylisation, but I do see
a .css file. Code seems to support that css is only there to be pushed
to remote deployment, but maybe there's a trick I haven't spotted.

> - While I can understand the desire to keep C function names short, I don't 
> see we should continue the tradition of using the meaningless 'Ex' suffix for 
> API variations that take an additional dict of keywords. Looking through the 
> existing APIs, I recommend PyObject_{Get,Set,Del}ItemWithKeywords instead. 
> (Note you have a typo here, "DetItem". Also I recommend adding markup (e.g. 
> bullets) so each signature is on a line by itself.

Will do. Thanks.
In fact I'll focus tonight on fixing the visual aspect and typos. I
also have feedback from the numpy mailing list about the argument to
pass in case no positional index is specified), but I'll add that
later, as well as the rest of this ml feedback.

I also tried to start implementing the feature as a PoC... but the
parser grammar is beyond my skills I fear. I understand it, but unsure
(at the moment) how to modify it. Nevertheless I am willing to spend
some days and do some experiments. I'll keep you all posted if I can
achieve something that seems to work.


-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FZT3EY3TRKVRWBD4F3QBG6NNI32BBO5F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Paul Moore]

On Fri, 25 Sep 2020 at 18:28, Samuel Colvin  wrote:
>> There's also the problem that you've explicitly acknowledged, that
>> exception hints are *always* going to be inaccurate, in stark contrast
>> to type hints which are expected to be correct when used.
>
> We should draw a distinction between two kinds of "inaccurate":
> * inaccuracies because data about what errors could be raised is
>   missing some exceptions
> * errors that could happen at almost any time:
>   MemoryError, OverflowError, SystemError, SystemExit
>
> The first set of errors is not in stark contrast to type hints,
> in fact they're very similar to errors in type hints which come up
> from time to time.

I get what you are saying, but I think you're being optimistic in
assuming there's a clean distinction. It may be that you;'re thinking
mostly about application code, where you are in control of what types
exist in the system. But in library code, you simply cannot assume
that. Consider a library that manipulates strings. There have been
many attempts to write libraries that represent filesystem paths using
a subclass of the string type. Suppose your library was passed one of
those, and concatenated some text onto it. Possible exceptions that
could in theory be raised include:

* UnicodeError (if the path class checks that the value can be encoded
in the filesystem encoding)
* OSError (if the class enforces maximum path length or allowed
character restrictions)
* ValueError (if the programmer was lazy and used these rather than
one of the above)

as well as pretty much anything else, if the code that implemented
__add__ for the path class had a bug.

And of course, if libraries don't declare "expected" exceptions,
application code using those libraries don't have anything to work
with. Maybe you could get away with just declaring use of exception
types that "you" created. But that's going to be very messy to get
right.

> Granted the "exceptions raised" database would be much more
> immature than the type hints database to start with. But that's
> not a reason not to start the error.

Agreed. What *is* a reason to not start is if we can't even agree what
we're building.

> The second set of exceptions are a different matter, I think
> in general they should not be included in the
> "exceptions raised" database. If that makes exceptions
> raised data invalid, then virtually all python logic in the world
> that catches some exceptions but doesn't catch those
> exceptions is also invalid!

Precisely. So it's critical to clearly define exactly what exceptions
can happen "at any time", and given that operations like indexing,
basic operators, and even attribute access can call arbitrary Python
code, essentially any exception can happen at any time. Unless you
start mandating exception behaviour on basic operations, and then you
preclude creative but unusual designs such as the path class I
described above. Maybe that's not unreasonable (over-clever path
classes never really became that popular) but Python has a tradition
of *not* arbitrarily constraining how features can be used.

> In other words: right now when we write code and catch
> exceptions we generally (but not always) catch
> exceptions that are likely to happen in the general
> course of calling the code in the try, except block. But
> not all the strange things that could conceivably happen.
>
> I think we should maintain that convention but provide a
> way to remind developers when they forget to catch an exception.

How do you distinguish between a developer "forgetting" to catch an
exception and deliberately letting it bubble up. I'm a strong -1 on
any proposal that makes people explicitly say "I meant to let this
through" (it's essentially leading to checked exceptions, as well as
violating the principle I stated above about not constraining how
people use features - in this case the feature of exceptions bubbling
up). And it's obviously impossible to make people declare when they
forgot to catch an exception.

> I guess overall, the question is: would it be useful to have a
> way to check what exceptions a block of code might raise?
>
> Yes, massively.

You state that without any justification, and it's the key point. My
view is that it's (barely) possible that it might be occasionally
convenient, but no-one has really demonstrated a benefit that I care
about at this point. And to be honest, claims like "tools might be
able to warn about certain types of error" are too vague to carry much
weight. I've been developing Python for many years, and work on pip,
which is a major codebase that's full of legacy weirdness - and I
don't think I've *ever* hit an issue where I've felt that having a
checker warn me that I haven't caught a type of exception would be
useful. (Many, many times I've had to fix issues where uncaught
exceptions were raised, but they have basically always been logic
errors where fixing the underlying issue is the correct action, not
catching the exception.

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 3:42 AM Steven D'Aprano  wrote:
> > In the list of all possible failures, will you include MemoryError?
>
> Sure, if I'm writing some sort of ultra-high availability server
> application that is expected to run 24/7 for months at a time.
>
> while memory_in_reserve():
> try:
> something()
> break
> except MemoryError:
> release_rainy_day_fund()  # Frees some emergency memory.

But if you DON'T have code like that, will you declare that your
function might raise MemoryError?

Just because it might be raised doesn't mean it's part of what needs
to be documented. Nor because it might be caught.

The problem with a proposal like this is that it is impossible to
determine which exceptions make sense to document. My alarm script
attempts to query upcoming events, and it catches ssl.SSLError,
OSError, IOError, socket.error, and googleapiclient.http.HttpError.
Should my upcoming_events function (a call to which is the only line
in the 'try' block) be declared as raising all of those? Are there any
other exceptions that I should declare? How would I know what set of
exceptions is relevant to my caller? Only my caller knows what's
relevant, so that's where the list of exceptions belongs: in the
try/except block itself.

Oh, I just discovered that that function can raise OverflowError if
the script is run with --days=9 - should I
add that to the annotated list of exceptions? Is it relevant? How
would I know?

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MSI5KHXWBLMMG5BUSM2JIHJIJUJIN7O7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Sat, Sep 26, 2020 at 2:32 AM Oscar Benjamin
 wrote:
> I do agree but maybe that suggests a different role for annotated
> exceptions in Python. Rather than attempting to enumerate all possible
> exceptions annotations could be used to document in a statically
> analysable way what the "expected" exceptions are. A type checker
> could use those to check whether a caller is handling the *expected*
> exceptions rather than to verify that the list of *all* exceptions
> possibly raised is exhaustive.
>
> Consider an example:
>
> def inverse(M: Matrix) -> Matrix: raises(NotInvertibleError)
> if determinant(M) == 0:
> raise NotInvertibleError
> rows, cols = M.shape
> for i in range(rows):
> for j in range(cols):
> ...
>
> Here the function is expected to raise NotInvertibleError for some
> inputs.

Please no. Once again, this will encourage "handling" of errors by
suppressing them. Don't have anything that forces people to handle
exceptions they can't handle.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2CHBAJGWGO7S6P4CKWYG7ZZLTBZB2W35/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Trash bin

[Paul Bryan]

On Fri, 2020-09-25 at 18:19 +0200, Marco Sulla wrote:
> That I hope it's not the place where this proposal will be sent.
> 
> My idea is apparently simple: what if, anytime we create an object,
> instead of deleting it, we send it in a trash bin? If the object is,
> for some reason, recreated, we can take it from the trash bin. If
> there is no more memory, the trash bin will be empty.
> 
> The problem is that, probably, args and kwargs used by object creation
> must be stored. Maybe also copied? This could slow down the object
> creation instead of speed it up? Could this be done by a separate
> thread?

This would assume that said objects: a) are immutable and b) initialize
to identical values for a given *args and **kwargs.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/62CXAQ5EFHODHNUD2V6YZS5ECBC75YKL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Steven D'Aprano]

On Fri, Sep 25, 2020 at 11:14:01PM +1000, Chris Angelico wrote:
> On Fri, Sep 25, 2020 at 7:59 PM Sergio Fenoll  wrote:

> > Surely there has to be a better way of programming than running stuff,
> > watching it fail, and then keeping track of how it fails so you can
> > later handle that failure?
> 
> Why? Do you really think you can enumerate EVERY possible way that
> something might fail?

Nobody is demanding that "EVERY possible way" is handled -- but if you 
need to, then Python lets you do so:

# Please don't do this.
try:
something()
except:
# handle EVERY error

Of course this is nearly always the wrong choice. "What can I cope with" 
is easy: it's always *everything*, if you define "cope with" as just 
suppressing the error.

> Think, instead, about all the possible problems that you can actually
> cope with. That way, you have a finite - and usually small - set of
> things to deal with, instead of an infinite field of "well this could
> go wrong, but we can't do anything about that".

The problem here is that you can't decide what you can deal with in 
isolation. I can deal with UnicodeEncodeError easily: try again with a 
different encoding, or with a different error handler, easy-peasy.

But if I'm doing `y = x + 1` and it somehow raised UnicodeEncodeError, 
what do I do? I'm stuck.

In order to tell what you can deal with, you need to know the 
circumstances of the error, and why it occurred. In other words, you 
need to understand the operation being called, in particular, what 
exceptions it might raise under normal circumstances.

I'm pretty confident that most people, unless they are TDD zealots, 
start by using either their pre-existing knowledge of the operation, or 
reading the documentation, to find out what exceptions are likely under 
normal circumstances, and *only then* start to think about how to deal 
with such exceptions.

The alternative is to waste time and mental energy thinking about how to 
deal with exceptions that you will probably never get in real life:

"Well, if I get an import error, I can add some more directories to 
sys.path and try adding the values again, that might fix it..."

Who does that? Not me. And I bet you don't either.

So I think that most of us:

- start with documented or well-known exceptions;

- and only then decide whether or not we can deal with them.

Of course rare or unusual exceptions probably won't be discovered 
without a lot of testing, including stress testing, or not until the 
code goes out into production. That's okay.

And I think that is Sergio's point: it would be good to have a standard, 
consistent place for functions to document which exceptions they are 
likely to raise under normal circumstances, and one which is available 
to IDEs and runtime inspection. Annotations.

Of course we can inspect the docstring of the function, but it's hard 
for an automated tool to distinguish:

This will raise WidgetExplosionError if the widget explodes.

from:

This is guaranteed to never raise WidgetExplosionError even
if the widget explodes.


There may be practical difficulties in sticking exceptions into 
annotations. Annotations already can be pretty long and bulky. But if 
you are okay with functions documenting that they might raise a certain 
exception, then *in principle* you should be okay with moving that into 
an annotation rather than the docstring.

Annotations are just a form of documentation in a consistent standard 
format to make it easy for IDEs to read them.


> In the list of all possible failures, will you include MemoryError?

Sure, if I'm writing some sort of ultra-high availability server 
application that is expected to run 24/7 for months at a time.

while memory_in_reserve():
try:
something()
break
except MemoryError:
release_rainy_day_fund()  # Frees some emergency memory.

Or perhaps I'll catch the exception and try to bail out safely after 
writing out data files etc. Or maybe:

try:
fast_but_fat()
except MemoryError:
slow_but_lean()


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CBS5XGTLH7567TDLFFNBPCI7R2IWLGKT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Wes Turner]

I strongly disagree that it's useless to document which Exceptions a
function could raise; even in Python (which, for a few reasons, is not a
language that's considered for safety-critical application).

In Python, it is common practice to - at a high level in the call stack -
trap Exceptions that can occur anywhere like KeyboardInterrupt and
MemoryError (and separately specify signal handler callbacks).

A high-level catchall (except for KeyboardInterrupt) and restart may be the
best way to handle exceptions in Python.

Safe coding styles (in other languages) do specify that *there may not be
any unhandled exceptions*.
Other languages made the specific decision to omit exceptions entirely:
developers should return `retval, err := func(arg)` and handle every value
of err.
Python has Exceptions and it's helpful to document what exceptions a
function might `raise` (even though it is possible to parse the AST to find
the `raise` statements within a callable and any callables it may or may
not handle). There are a few useful ideas for checking Exception
annotations at compile-time in this thread.

https://en.wikipedia.org/wiki/Exception_handling#Static_checking_of_exceptions
https://en.wikipedia.org/wiki/Exception_handling#Dynamic_checking_of_exceptions

We could pick one or more of the software safety standards listed here and
quote and cite our favs:
https://awesome-safety-critical.readthedocs.io/en/latest/#software-safety-standards

## Exception docstrings
You can specify Exceptions in all formats of sphinx docstrings:

### Sphinx-style docstrings:

```python
"""
:raises: AttributeError: The ``Raises`` section is a list of all exceptions
that are relevant to the interface.
:raises: ValueError: If `param2` is equal to `param1`.
"""
```

### Google-style docstrings:
```python
"""
Raises:
AttributeError: The ``Raises`` section is a list of all exceptions
that are relevant to the interface.
ValueError: If `param2` is equal to `param1`.
"""
```

###Numpy-style docstrings:
```python
"""
Raises
--
AttributeError
The ``Raises`` section is a list of all exceptions
that are relevant to the interface.
ValueError
If `param2` is equal to `param1`.
"""
```

https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html#example-google
https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html#example-numpy

## Design-by-contracts
FWICS, neither e.g. icontract nor zope.interface support Exception
contracts. How could that work.


## Awesome-safety-critical
https://awesome-safety-critical.readthedocs.io/en/latest/#software-safety-standards


On Fri, Sep 25, 2020 at 12:34 PM Oscar Benjamin 
wrote:

> On Fri, 25 Sep 2020 at 15:57, Paul Moore  wrote:
> >
> > On Fri, 25 Sep 2020 at 14:15, Chris Angelico  wrote:
> >
> > > Why? Do you really think you can enumerate EVERY possible way that
> > > something might fail?
> >
> > Rust does a surprisingly good job of that, actually. But the point is
> > that Python is not Rust, and the infrastructure Rust uses to allow it
> > to manage code safety is baked into the language core at a very
> > fundamental level.
> >
> > Enumerating the exceptions that a piece of code can raise is
> > impractical and unhelpful in Python. But that may not be immediately
> > obvious to someone coming from a different language. That's why it's
> > important to understand Python properly before proposing new features
> > that work in other languages. (I don't think that's what the OP is
> > doing here, to be clear, but the discussion is drifting in that
> > direction, with Rust's Result type having been mentioned).
> >
> > **In Python**, writing code from the perspective of "what can I handle
> > at this point" is the right approach. Deferring unexpected exceptions
> > to your caller is the default behaviour, and results in a clean,
> > natural style *for Python*. The proposal here is basically in direct
> > contradiction to that style.
>
> I do agree but maybe that suggests a different role for annotated
> exceptions in Python. Rather than attempting to enumerate all possible
> exceptions annotations could be used to document in a statically
> analysable way what the "expected" exceptions are. A type checker
> could use those to check whether a caller is handling the *expected*
> exceptions rather than to verify that the list of *all* exceptions
> possibly raised is exhaustive.
>
> Consider an example:
>
> def inverse(M: Matrix) -> Matrix: raises(NotInvertibleError)
> if determinant(M) == 0:
> raise NotInvertibleError
> rows, cols = M.shape
> for i in range(rows):
> for j in range(cols):
> ...
>
> Here the function is expected to raise NotInvertibleError for some
> inputs. It is also possible that the subsequent code could raise an
> exception e.g. AttributeError, TypeError etc and it's not necessarily
> possible to enumerate or exhaustively rule out 

[Python-ideas] Re: Trash bin

[Chris Angelico]

On Sat, Sep 26, 2020 at 2:19 AM Marco Sulla
 wrote:
>
> That I hope it's not the place where this proposal will be sent.
>
> My idea is apparently simple: what if, anytime we create an object,
> instead of deleting it, we send it in a trash bin? If the object is,
> for some reason, recreated, we can take it from the trash bin. If
> there is no more memory, the trash bin will be empty.
>
> The problem is that, probably, args and kwargs used by object creation
> must be stored. Maybe also copied? This could slow down the object
> creation instead of speed it up? Could this be done by a separate
> thread?

This is actually the way certain objects ARE handled, but since memory
is allocated in chunks (not all objects are the same size), there are
in fact several "trash bins". They're called "free lists" and you can
explore the CPython source code if you're curious about how they work!

For info on free lists in general, Wikipedia might be of value:
https://en.wikipedia.org/wiki/Free_list

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C6FF72U7PERWAOLYLXE2Q5LY557RGTKI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Oscar Benjamin]

On Fri, 25 Sep 2020 at 15:57, Paul Moore  wrote:
>
> On Fri, 25 Sep 2020 at 14:15, Chris Angelico  wrote:
>
> > Why? Do you really think you can enumerate EVERY possible way that
> > something might fail?
>
> Rust does a surprisingly good job of that, actually. But the point is
> that Python is not Rust, and the infrastructure Rust uses to allow it
> to manage code safety is baked into the language core at a very
> fundamental level.
>
> Enumerating the exceptions that a piece of code can raise is
> impractical and unhelpful in Python. But that may not be immediately
> obvious to someone coming from a different language. That's why it's
> important to understand Python properly before proposing new features
> that work in other languages. (I don't think that's what the OP is
> doing here, to be clear, but the discussion is drifting in that
> direction, with Rust's Result type having been mentioned).
>
> **In Python**, writing code from the perspective of "what can I handle
> at this point" is the right approach. Deferring unexpected exceptions
> to your caller is the default behaviour, and results in a clean,
> natural style *for Python*. The proposal here is basically in direct
> contradiction to that style.

I do agree but maybe that suggests a different role for annotated
exceptions in Python. Rather than attempting to enumerate all possible
exceptions annotations could be used to document in a statically
analysable way what the "expected" exceptions are. A type checker
could use those to check whether a caller is handling the *expected*
exceptions rather than to verify that the list of *all* exceptions
possibly raised is exhaustive.

Consider an example:

def inverse(M: Matrix) -> Matrix: raises(NotInvertibleError)
if determinant(M) == 0:
raise NotInvertibleError
rows, cols = M.shape
for i in range(rows):
for j in range(cols):
...

Here the function is expected to raise NotInvertibleError for some
inputs. It is also possible that the subsequent code could raise an
exception e.g. AttributeError, TypeError etc and it's not necessarily
possible to enumerate or exhaustively rule out what those
possibilities might be. If we wanted to annotate this with
raises(NotInvertibleError) then it would be very hard or perhaps
entirely impossible for a type checker to verify that no other
exception can be raised. Or maybe even the type checker could easily
come up with a large list of possibilities that you would never want
to annotate your code with. Maybe that's not what the purpose of the
annotation is though.

What the type checker can do is check whether a caller of this
function handles NotInvertibleError after seeing the *explicit* type
hint. A function that calls inverse without catching the exception can
also be considered as raises(NotInvertibleError). You might want to
enforce in your codebase that the caller should catch and suppress the
expected exception or should itself have a compatible raises
annotation indicating that it can also be expected to raise the same
exception e.g. either of these is fine:

def some_calc(M: Matrix): raises(NotInvertibleError)
A = inverse(M)
...

def some_calc(M: Matrix):
try:
A = inverse(M)
except NotInvertibleError
# do something else
...

Perhaps rather than requiring all exceptions to be annotated
everywhere you could allow the raises type hints to propagate
implicitly and only verify them where there is another explicit type
hint:

def some_calc(M):
# no hint but checker infers this raises NotInvertibleError
A = inverse(M)

def other_func(M): raises(ZeroError)
# checker gives an error for this
# because the raises should include NotInvertibleError
B = some_calc(M)

You could then have an explicit hint for the type checker to say that
a function is not expected to raise any exceptions maybe like this:

def main(args): raises(None)
...

The intent of this would be that the type checker could then follow
the chain of all functions called by main to verify that any
exceptions that were expected to raise had been handled somewhere.
This wouldn't verify all of the exceptions that could possibly be
raised by any line of code. It could verify that for those exceptions
that have been explicitly annotated.


Oscar
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L2YK75C7XSFWOJLM6ROAI3ZVAY2WE5GZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Trash bin

[Marco Sulla]

That I hope it's not the place where this proposal will be sent.

My idea is apparently simple: what if, anytime we create an object,
instead of deleting it, we send it in a trash bin? If the object is,
for some reason, recreated, we can take it from the trash bin. If
there is no more memory, the trash bin will be empty.

The problem is that, probably, args and kwargs used by object creation
must be stored. Maybe also copied? This could slow down the object
creation instead of speed it up? Could this be done by a separate
thread?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AI5ALV7D64DJCVSJPARZQEI2G6BPKTX6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Marco Sulla]

On Fri, 25 Sep 2020 at 14:44, Samuel Colvin  wrote:
>
> Sorry I probably wasn't clear enough in what I was suggesting.
>
>>
>> The main question here is why using a hint or a decorator should be
>> better than a simple documentation.
>
>
> For the same reason type hints are better than documentation

Type hints help an IDE to check if you're potentially passing a bad
parameter to your function.
What does an "exception hint" will do in an IDE? Alerts you that you
are not catching that exception, and, if you really want it to bubble
up, silencing that warning? Again, no thanks :-)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CFVT7ICR7A5C5AFT2U4JEO2XSCMEHS2M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Paul Moore]

On Fri, 25 Sep 2020 at 14:15, Chris Angelico  wrote:

> Why? Do you really think you can enumerate EVERY possible way that
> something might fail?

Rust does a surprisingly good job of that, actually. But the point is
that Python is not Rust, and the infrastructure Rust uses to allow it
to manage code safety is baked into the language core at a very
fundamental level.

Enumerating the exceptions that a piece of code can raise is
impractical and unhelpful in Python. But that may not be immediately
obvious to someone coming from a different language. That's why it's
important to understand Python properly before proposing new features
that work in other languages. (I don't think that's what the OP is
doing here, to be clear, but the discussion is drifting in that
direction, with Rust's Result type having been mentioned).

**In Python**, writing code from the perspective of "what can I handle
at this point" is the right approach. Deferring unexpected exceptions
to your caller is the default behaviour, and results in a clean,
natural style *for Python*. The proposal here is basically in direct
contradiction to that style.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OERGYDEBVCBA67RBZN6EJYMWPPJ4JMVN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Paul Moore]

On Fri, 25 Sep 2020 at 13:46, Samuel Colvin  wrote:
>
> Sorry I probably wasn't clear enough in what I was suggesting.
>
>>
>> The main question here is why using a hint or a decorator should be
>> better than a simple documentation.
>
>
> For the same reason type hints are better than documentation -
>
> 1. static analysis can catch a multitude of potential errors that humans 
> often miss.
> 2. type hints stay with code in a way documentation often doesn't
> 3. developers are often forced to keep type hints up to date to get tests to 
> pass
>   the same is very rarely true of documentation, this would apply to
>   "exceptions raised" too.

Type hints were largely driven by developers of type checking
software. Is there interest from the mypy developers in this sort of
exception annotation? From any other static checking tool? If there's
no-one proposing to actually use the proposed feature in a tool, it's
not clear how the cost of implementing it is justified. You;ve
discounted runtime checking (and everyone hates that anyway), and no
type checkers have spoken up saying they want it.

You mention IDEs, but if you look at the history of suggestions
intended to support IDEs in this list, you'll find that they very
rarely go anywhere. A significant proportion of Python users don't use
an IDE. And generally, IDE developers don't seem interested in pushing
new language features that they can use - I don't recall ever seeing
an IDE developer speak up to say how much a proposed feature will help
them. In practice, I think IDEs tend to work with heuristics or what
they can do with the language as it is, and aren't really interested
in shaping the language to be "IDE-friendly". But that's just my
impression.

There's also the problem that you've explicitly acknowledged, that
exception hints are *always* going to be inaccurate, in stark contrast
to type hints which are expected to be correct when used. Enumerating
all possible exceptions is impossible, and (as Java's checked
exceptions shows) typically does more harm than good. No exception
hints is where we are now. You're suggesting a feature that lets
people list "some" exceptions - somewhere in the spectrum between
"all" and "none" - but it's completely unspecified what exceptions
need to be included and what don't. "Whatever the developer feels is
appropriate" is basically useless - there's no way any tool or user
can usefully interpret what an exception hint means if things are this
wide open. I can't see how anything could make any practical use of
the data the proposal offers...

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ERXDU7SOOILCVS4N7CD2TBSZ2WR6HKXW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Fri, Sep 25, 2020 at 7:59 PM Sergio Fenoll  wrote:
>
> O 25/09/20 ás 11:52, Chris Angelico escribiu:
> > But requests.get() doesn't have a single raise statement anywhere in
> > it. And if you dig through the entire source code for the requests
> > package, you'll still only find a small number of the exceptions that
> > might be raised. Errors come from anywhere, and if you're not handling
> > them, you will simply let them bubble; is it your responsibility to
> > document that? No. It's just the normal thing to do.
> >
> > When you want to catch an exception, don't look at the function to see
> > what it's documented as raising. Look at its behaviour and see what it
> > does that you can cope with. You're looking at things backwards and
> > that's why you're wanting a list of possible things to catch. Instead,
> > look at your program WITHOUT any exception handling, and see what
> > exceptions are happening. Those are the ones to look at.
> >
> > ChrisA
>
>
> Surely there has to be a better way of programming than running stuff,
> watching it fail, and then keeping track of how it fails so you can
> later handle that failure?

Why? Do you really think you can enumerate EVERY possible way that
something might fail?

Think, instead, about all the possible problems that you can actually
cope with. That way, you have a finite - and usually small - set of
things to deal with, instead of an infinite field of "well this could
go wrong, but we can't do anything about that".

In the list of all possible failures, will you include MemoryError?
What would you do if one got raised? If the answer is "nothing, just
let it propagate", then it doesn't need to be in the list. Same goes
for every other exception ever invented.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TZ5AOOEBFMD7HTP6A2JFLP25HFUBRT2Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[2QdxY4RzWzUUiLuE]

On 2020-09-25 at 13:44:36 +0100,
Samuel Colvin  wrote:

> 2. We should have some easy way to say "let this error propagate",
> rust uses very nice question mark at the end of a line syntax, python
> could use something similar one day, until then a magic comment,
> wrapper function or context function could be used.

In Python, the way to say "let this error propagate" is not to say
anything at all.  y = f(x) says (and very concisely, I might add) "if f
raises an exception, or propagates some exception from some other place,
then let that exception propagate."
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/25E6NBXYXZF5A5UGCXOOG6QZZ5NXH365/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

[Ricky Teachey]

I'd like to hear more about why the empty tuple has been selected as the
default index.

On Fri, Sep 25, 2020, 1:07 AM David Mertz  wrote:

> Is this a breaking change? It feels borderline.
>
> Keyword-only subscripts are permitted. The positional index will be the
>> empty tuple:
>> obj[spam=1, eggs=2]
>> # calls type(obj).__getitem__(obj, (), spam=1, eggs=2)
>
>
> I.e. consider:
>
> >>> d = dict()
> >>> d[()] = "foo"
> >>> d
> {(): 'foo'}
>
>
> I don't really object to this fact, and one could argue it's not a
> breaking change since a built-in dict will simply raise an exception with
> keyword arguments.  However, it does make the empty tuple the "default key"
> for new objects that will accept keyword indices.
>
> --
> The dead increasingly dominate and strangle both the living and the
> not-yet born.  Vampiric capital and undead corporate persons abuse
> the lives and control the thoughts of homo faber. Ideas, once born,
> become abortifacients against new conceptions.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/65KCDFYRUWH6K4HHMZT4UJTPDLE7JOPK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C4JVKD2ZUXU2Z6FDJOMHD5YYTACIHEIO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[David Kolovratník]

On Fri, Sep 25, 2020 at 01:23:01PM +0900, Stephen J. Turnbull wrote:
> Sergio Fenoll writes:
> 
>  > In the same vein as adding type annotations to code, I think it'd
>  > be very useful to have exception "raises" annotations, i.e. a way
>  > to annotate what exceptions a function raises.
> 
> I think you need to explain the use cases in more detail.  You mention
> IDEs, but they can already grovel through the source code and work out
> exactly what exceptions each function explicitly raises, and keep a
> database for builtins and the stdlib, which could easily be updated by
> the user by running the groveler on Python itself.  3rd party imports,
> ditto.  This would allow far more accurate inference of possible
> exceptions than an optional 'raises' annotation would.
pyntch (currently ready for Python 2) can already tell us about exceptions.

Do you know about other tools?

A tiny example:

l=[0,1,2]
with open('f') as fh:
a = int(fh.read())
print(1 / l[a])

command (after fixing option hangling in tchecker.py):
tchecker.py -C show_all_exceptions=True -C raise_uncertain=True p.py

output:
loading: 'p.py' as 'p'
processing: 322 161
processing: 324 6
processing: 328 8
processing: 328 4
processing: 329 2
processing: 332 2
processing: 332 1
processing: 333 2
processing: 333 2
total files=1, lines=4 in 0.05sec
[p (p.py)]
  a = 
  fh = 
  l = []
  raises IOError: cannot open a file at p:2
  raises EOFError: end of file at p:3
  raises IndexError: index out of range at p:4

ZeroDivisionError is missing there, however, it couldbe seen as a space for
improvement.

David Kolovratník
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GZW7NXKJMUTK54KTGBCAXMFVIWDKTTFN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Samuel Colvin]

Sorry I probably wasn't clear enough in what I was suggesting.


> The main question here is why using a hint or a decorator should be
> better than a simple documentation.


For the same reason type hints are better than documentation -

1. static analysis can catch a multitude of potential errors that humans
often miss.
2. type hints stay with code in a way documentation often doesn't
3. developers are often forced to keep type hints up to date to get tests
to pass
  the same is very rarely true of documentation, this would apply to
  "exceptions raised" too.

If the goal is to force people to manage the exception, no, thanks.


Total agree. Here we can take some lessons from rust, but not try
to make python into something it's not (a system language).

I would suggest:

1. "possible exceptions" information should be for static analysis only -
so it would only come into play if you used a static analysis tool anyway,
like type hints.

2. We should have some easy way to say "let this error propagate",
rust uses very nice question mark at the end of a line syntax, python
could use something similar one day, until then a magic comment,
wrapper function or context function could be used.

--

Samuel Colvin
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RF27BCESZR26N5CJ4G6EKHH7IBYNCDTG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Marco Sulla]

On Fri, 25 Sep 2020 at 11:58, Samuel Colvin  wrote:
> I first found myself wanting this when I came back to python
> having been writing rust. The Result type in rust is somewhat
> similar to what's being suggested here. See
> https://doc.rust-lang.org/std/result/

I do not know Rust and I'm not sure I understood 100% the code. But,
if I'm not wrong, Rust does not use try-catch, but pattern matching.
It seems to me that Rust has a good exception handling system, but
Python does not (yet) have pattern matching.

The main question here is why using a hint or a decorator should be
better than a simple documentation.

If the goal is to force people to manage the exception, no, thanks. As
Serhiy Storchaka already said, it was historically proven as bad. And
even if I've not a great knowledge in Java (3 years), I can assure you
checked exceptions are a really bad idea.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BRU5WUAFQWMKIJ5I6GWGKTRPSYAGTLRS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Sergio Fenoll]

O 25/09/20 ás 11:52, Chris Angelico escribiu:

But requests.get() doesn't have a single raise statement anywhere in
it. And if you dig through the entire source code for the requests
package, you'll still only find a small number of the exceptions that
might be raised. Errors come from anywhere, and if you're not handling
them, you will simply let them bubble; is it your responsibility to
document that? No. It's just the normal thing to do.

When you want to catch an exception, don't look at the function to see
what it's documented as raising. Look at its behaviour and see what it
does that you can cope with. You're looking at things backwards and
that's why you're wanting a list of possible things to catch. Instead,
look at your program WITHOUT any exception handling, and see what
exceptions are happening. Those are the ones to look at.

ChrisA



Surely there has to be a better way of programming than running stuff, 
watching it fail, and then keeping track of how it fails so you can 
later handle that failure?


Don't get me wrong, it's what I've been doing with Python up until now 
because there's no way around it (other than hoping the library 
documents how and when common exceptions are raised), but I can't say it 
I've really *enjoyed* it.


I hope I'm not the only person who sees this as a suboptimal approach? 
Of course, I could be the weird one out but I frankly doubt it.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/W7GVIYVDWK63TYHTYVADR44MFG734MDG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Samuel Colvin]

>
> I think it'd be very useful to have exception "raises"

annotations, i.e. a way to annotate what exceptions

a function raises.


I personally think this would be wonderful, I've been meaning
to suggest it here for some time, but haven't got around to it.

I first found myself wanting this when I came back to python
having been writing rust. The Result type in rust is somewhat
similar to what's being suggested here. See
https://doc.rust-lang.org/std/result/

This could be initially implemented as a decorator, without
any special syntax required, in fact to avoid excessive line
length, decorators might be the best way to describe
what exceptions can be raised by a function in the long term.

Using a decorator, this could be implemented as a separate
project (and/or extension to mypy) without any changes to the
standard library. it would obviously be a massive piece of work
to build a database of all the exceptions which could be raised
by all standard library functions, let alone 3rd party libraries.

However you could aid 90% of code by covering
10% of methods (or insert your own 80/20, 99/1 ratio). The
point is that the library could be useful long before it was complete
or 100% correct.

In this regard I see a lot of similarities with type hints and
typeshed.

Anyway, thank you Sergio for suggesting this. I really hope
it comes to fruition somehow.

--

Samuel Colvin
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Y6WJENUAKLHVSAXJOYPMIFKWXCPCYEI7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Fri, Sep 25, 2020 at 7:46 PM Sergio Fenoll  wrote:
> I'm really failing to see what part of my logic is faulty. I know that
> any exception that isn't handled can be raised. I know that is the
> expected behaviour. I don't expect every function to exhaustively
> document every single exception it may raise (because as you and others
> have said, that would include things like MemoryErrors and
> KeyboardInterrupts). All I'm saying is that *some* exceptions are
> "expected" by whomever implemented the function. To bring it back to my
> earlier example, requests.get() raises a ConnectionError if it can't
> connect to the url you pass it. It would be super useful as a caller of
> that function to have this information *inside your IDE*.
>
> I really don't understand why it's faulty logic to want to have at least
> *some* information about the exceptions a function *may* raise.

But requests.get() doesn't have a single raise statement anywhere in
it. And if you dig through the entire source code for the requests
package, you'll still only find a small number of the exceptions that
might be raised. Errors come from anywhere, and if you're not handling
them, you will simply let them bubble; is it your responsibility to
document that? No. It's just the normal thing to do.

When you want to catch an exception, don't look at the function to see
what it's documented as raising. Look at its behaviour and see what it
does that you can cope with. You're looking at things backwards and
that's why you're wanting a list of possible things to catch. Instead,
look at your program WITHOUT any exception handling, and see what
exceptions are happening. Those are the ones to look at.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2IN57NIPEGYLA4VBR5ODATC746BZMVBN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Sergio Fenoll]

O 25/09/20 ás 11:34, Chris Angelico escribiu:

On Fri, Sep 25, 2020 at 7:19 PM Sergio Fenoll  wrote:

O 25/09/20 ás 10:56, Chris Angelico escribiu:


There's the shallow "exceptions that I expect to raise", which is
those explicitly listed within the function as raise statements; but
that's not the whole story, since exceptions can be raised by anything
that the function calls. So, no, I don't think the callee should have
a notion of "exceptions that I or anyone I call might raise".

ChrisA

I don't understand the need of talking about "exceptions that something
I call might raise" (from the POV of the callee) though, to me they are
irrelevant in this discussion. When you implement a function, you are
handling all/some/none exceptions and letting all/some/nonebubble up and
be handled by whatever called you, right? Somewhere, be it in a
docstring, some manual, external documentation, ... you write down what
exceptions you expect to raise, be it explicit or implicit (e.g. by
letting a known exception bubble up without explicitly raising it
yourself).

Look at your own code. Do you know what EVERY function you call could
raise? If not, then your code could raise those exceptions. It's not
about "letting a known exception bubble up". It's the *normal
behaviour* of letting all exceptions bubble up unless explicitly
caught. Don't think in terms of knowing every exception that could be
raised. That simply doesn't work, and it leads to faulty logic like
this.

ChrisA



I'm really failing to see what part of my logic is faulty. I know that 
any exception that isn't handled can be raised. I know that is the 
expected behaviour. I don't expect every function to exhaustively 
document every single exception it may raise (because as you and others 
have said, that would include things like MemoryErrors and 
KeyboardInterrupts). All I'm saying is that *some* exceptions are 
"expected" by whomever implemented the function. To bring it back to my 
earlier example, requests.get() raises a ConnectionError if it can't 
connect to the url you pass it. It would be super useful as a caller of 
that function to have this information *inside your IDE*.


I really don't understand why it's faulty logic to want to have at least 
*some* information about the exceptions a function *may* raise.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WIBFHB2MBQ2DFINRWCAL72TJJRXGAF52/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Fri, Sep 25, 2020 at 7:19 PM Sergio Fenoll  wrote:
>
> O 25/09/20 ás 10:56, Chris Angelico escribiu:
>
> > There's the shallow "exceptions that I expect to raise", which is
> > those explicitly listed within the function as raise statements; but
> > that's not the whole story, since exceptions can be raised by anything
> > that the function calls. So, no, I don't think the callee should have
> > a notion of "exceptions that I or anyone I call might raise".
> >
> > ChrisA
>
> I don't understand the need of talking about "exceptions that something
> I call might raise" (from the POV of the callee) though, to me they are
> irrelevant in this discussion. When you implement a function, you are
> handling all/some/none exceptions and letting all/some/nonebubble up and
> be handled by whatever called you, right? Somewhere, be it in a
> docstring, some manual, external documentation, ... you write down what
> exceptions you expect to raise, be it explicit or implicit (e.g. by
> letting a known exception bubble up without explicitly raising it
> yourself).

Look at your own code. Do you know what EVERY function you call could
raise? If not, then your code could raise those exceptions. It's not
about "letting a known exception bubble up". It's the *normal
behaviour* of letting all exceptions bubble up unless explicitly
caught. Don't think in terms of knowing every exception that could be
raised. That simply doesn't work, and it leads to faulty logic like
this.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EYYRYYOUDWDMXSXH2MKKJ45XL7HG4G2F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Sergio Fenoll]

O 25/09/20 ás 06:23, Stephen J. Turnbull escribiu:


Sergio Fenoll writes:

  > In the same vein as adding type annotations to code, I think it'd
  > be very useful to have exception "raises" annotations, i.e. a way
  > to annotate what exceptions a function raises.

I think you need to explain the use cases in more detail.  You mention
IDEs, but they can already grovel through the source code and work out
exactly what exceptions each function explicitly raises, and keep a
database for builtins and the stdlib, which could easily be updated by
the user by running the groveler on Python itself.  3rd party imports,
ditto.  This would allow far more accurate inference of possible
exceptions than an optional 'raises' annotation would.

Even a closed-source vendor removing the .py wouldn't stop a truly
dedicated groveler, I believe.  Because of the way exceptions are
created, there'd be a trace in a namespace in the .pyc or even in a
DLL.


I'm no expert so I'm not sure how IDEs currently handle typing 
annotations, I assume they make use of the built-in language features 
for that as opposed to some proprietary solution that every IDE would 
have to re-implement? If that is the case, i.e. they use a built-in 
language feature, I think it'd be beneficial to have exception 
annotations as part of the language as well. Also, if you had a dunder 
like __raises__ for functions (like you have __annotations__) you could 
write some cool (and possibly useful?) introspective code.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/A3BPJNCFTEEYKOPZJ6RVHGEJROLTYHPJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Sergio Fenoll]

O 25/09/20 ás 10:56, Chris Angelico escribiu:


There's the shallow "exceptions that I expect to raise", which is
those explicitly listed within the function as raise statements; but
that's not the whole story, since exceptions can be raised by anything
that the function calls. So, no, I don't think the callee should have
a notion of "exceptions that I or anyone I call might raise".

ChrisA


I don't understand the need of talking about "exceptions that something 
I call might raise" (from the POV of the callee) though, to me they are 
irrelevant in this discussion. When you implement a function, you are 
handling all/some/none exceptions and letting all/some/nonebubble up and 
be handled by whatever called you, right? Somewhere, be it in a 
docstring, some manual, external documentation, ... you write down what 
exceptions you expect to raise, be it explicit or implicit (e.g. by 
letting a known exception bubble up without explicitly raising it 
yourself). I'm basically only talking about those kinds of exceptions 
here, because in my experience those are the exceptions I'm most often 
having to try-except. Sure, as a caller you may want to handle more 
exceptions than what the callee sees as "expected exceptions", but 
nothing will stop you from doing that just as you do it now. I don't see 
how having these exceptions be annotated in the code makes anything 
worse. From my perspective it just makes the most common try-except 
you'll write easier to implement without having to leave the IDE and 
check external documentation.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AUESIFXSGGBAZIJBRBAHJ3UFOODR233I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Fri, Sep 25, 2020 at 6:14 PM Sergio Fenoll  wrote:
> I see what you mean now, and I agree. But surely there's a fairly big
> overlap between
> "exceptions I expect to raise" (from the POV of the callee) and
> "exceptions I can handle"
> (from the POV of the caller). Wouldn't it still be useful to have the
> first part (exceptions
> I expect to raise) defined in the code (i.e., via some kind of
> annotation) in such a way that
> makes it easier for the caller to handle the exceptions they want to catch?
>
> Or, in your opinion, should the callee not even have a notion of
> "exceptions I expect to raise"?
>

There's the shallow "exceptions that I expect to raise", which is
those explicitly listed within the function as raise statements; but
that's not the whole story, since exceptions can be raised by anything
that the function calls. So, no, I don't think the callee should have
a notion of "exceptions that I or anyone I call might raise".

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2IUTNJBADJDHCR7J7UNDRYL32PFY7YIC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 9999 (provisional): support for indexing with keyword arguments

[Eric Wieser]

Thanks for the PEP, and for pinging the numpy list about it. Some comments:

Sequence unpacking remains a syntax error inside subscripts:
Reason: unpacking items would result it being immediately repacked into a
tuple

A simple counter-example is [:, *args, :], which could be treated as
[(slice(None),
*args, slice(None))].
When there are index objects to the left or right of *args, it enables :
syntax in places that would otherwise be forbidden.
I’d argue this should be folded into the scope of the PEP - if the parser
for indexing is going to be overhauled, it would be nice to cross this one
off.
I think I requested this change independently at some point

Keyword arguments must allow Ellipsis

This is sort of uninteresting now, Ellipsis is not special-cased in the
parser, all places where Ellipsis is allowed also allow ... - even
__class__!

a similar case occurs with setter notation

It would perhaps be a good idea to recommend users declare their setter
arguments as positional-only too,def __setitem__(self, index, value, /,
**kwargs):, to avoid this problem

Keyword-only subscripts are permitted. The positional index will be the
empty tuple

As discussed on the numpy mailing list, a better approach might be to not
pass an argument at all, so that obj[spam=1, eggs=2] calls
type(obj).__getitem__(obj,
spam=1, eggs=2).
Objects which want to support keyword-only indexing can provide a default
of their own (and are free to choose between () and None, while objects
which do not would just raise TypeError due to the missing positional
argument

the solution relies on the assumption that all keyword indices necessarily
map into positional indices, or that they must have a name

This seems like a non-issue to me. Keyword-only arguments already exist,
and could be used in the relevant magic method def __getitem_ex__(self, *,
x, y):. I would be inclined to remove this argument.

Eric
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZQ4CHT37YUHYEFYFBQ423QNZMFCRULAW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Sergio Fenoll]

O 25/09/20 ás 10:02, Chris Angelico escribiu:

On Fri, Sep 25, 2020 at 5:09 PM Sergio Fenoll  wrote:

O 25/09/20 ás 08:41, Chris Angelico escribiu:


It would have to not only look at get(), but everything that it calls.
Either that, or you're back to the Java thing of "catch it or declare
it", and we've seen from Java that that's a bad idea.

ChrisA

I don't really understand why those are the only options. The way I see it,
whomever implements get() can just have a list of expected Exceptions and as
a caller you just handle those (now with the added ease that you don't
need to
look them up out-of-bounds, they're already in your IDE!) and if an
unexpected
exception happens, well you handle it the same way you do it now (i.e.
either not
at all, with a bare except, or something along those lines).

Don't think in terms of "expected exceptions". That's the mentality
that leads to coding styles where you try to catch every exception the
function can raise, which is the wrong way to think about it.

Instead, think in terms of "exceptions that I can handle".

ChrisA
I see what you mean now, and I agree. But surely there's a fairly big 
overlap between
"exceptions I expect to raise" (from the POV of the callee) and 
"exceptions I can handle"
(from the POV of the caller). Wouldn't it still be useful to have the 
first part (exceptions
I expect to raise) defined in the code (i.e., via some kind of 
annotation) in such a way that

makes it easier for the caller to handle the exceptions they want to catch?

Or, in your opinion, should the callee not even have a notion of 
"exceptions I expect to raise"?


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/V36GLZXCT63ODWMTRQY23VT7WUVMO6W2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Fri, Sep 25, 2020 at 5:09 PM Sergio Fenoll  wrote:
>
> O 25/09/20 ás 08:41, Chris Angelico escribiu:
>
> > It would have to not only look at get(), but everything that it calls.
> > Either that, or you're back to the Java thing of "catch it or declare
> > it", and we've seen from Java that that's a bad idea.
> >
> > ChrisA
> I don't really understand why those are the only options. The way I see it,
> whomever implements get() can just have a list of expected Exceptions and as
> a caller you just handle those (now with the added ease that you don't
> need to
> look them up out-of-bounds, they're already in your IDE!) and if an
> unexpected
> exception happens, well you handle it the same way you do it now (i.e.
> either not
> at all, with a bare except, or something along those lines).

Don't think in terms of "expected exceptions". That's the mentality
that leads to coding styles where you try to catch every exception the
function can raise, which is the wrong way to think about it.

Instead, think in terms of "exceptions that I can handle".

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CGQESM3SQA3BYOEVG4OBLABNIGKXEFR5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Sergio Fenoll]

O 25/09/20 ás 09:12, Serhiy Storchaka escribiu:

Did not the experience of C++ show that exceptions declaration was a bad
idea? In C++ it was optional and is abandoned now. Java developers still
suffer from necessary to declare all raised exception, or just declare
the most general exception class, that makes the feature useless.

Could you elaborate as to why it was bad in C++? I looked it up and see that
it was deprecated before I learned C++ so I never used it myself, so I'm
interested to hear what was wrong with it.
I do agree that the way Java does it is too cumbersome, but I'm not 
proposing
that Python gets Java-like checked exceptions, so I don't think it's a 
good comparison.

This may be especially bad for Python where virtually every line of code
can raise arbitrary exception. KeybordInterrupt and MemoryError are most
common examples, but in rare cases which usually are not covered by
tests it can be also UnicodeError, NameError, AttributeError, TypeError,
BufferError, etc.
Agreed! This is why I think this feature would be useful to document 
_expected_ Exceptions.
Something that already happens anyway (in actual documentation) but 
would be (IMO)
far more useful if it were to happen in the code itself (much like type 
annotations).

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OWYWMYFKDOLWNQX2R4P4LGUXJIZR3NSW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Serhiy Storchaka]

24.09.20 11:47, Sergio Fenoll пише:
> In the same vein as adding type annotations to code, I think it'd be
> very useful to have exception "raises" annotations, i.e. a way to
> annotate what exceptions a function raises. Again, like type
> annotations, it shouldn't be mandatory nor actually be enforced at
> runtime. It would purely serve as a feature that IDEs can make use of.
> 
> An example of how it may look:
> 
> def divide(numerator: float, denominator: float) raises [ZeroDivisionError] 
> -> float:
>     return numerator / denominator
> 
> I'd love to know if this is an idea you'd be interested in having added
> to the language.

Did not the experience of C++ show that exceptions declaration was a bad
idea? In C++ it was optional and is abandoned now. Java developers still
suffer from necessary to declare all raised exception, or just declare
the most general exception class, that makes the feature useless.

This may be especially bad for Python where virtually every line of code
can raise arbitrary exception. KeybordInterrupt and MemoryError are most
common examples, but in rare cases which usually are not covered by
tests it can be also UnicodeError, NameError, AttributeError, TypeError,
BufferError, etc.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F244P5HTWU666LZSE3XLULFMNGPZQ27M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Sergio Fenoll]

O 25/09/20 ás 08:41, Chris Angelico escribiu:


On Fri, Sep 25, 2020 at 4:25 PM Sergio Fenoll  wrote:

What I had in mind was that an IDE could use this information to show
autocomplete options when writing except blocks. The other day I was
writing some code like this:

import requests

try:

  requests.get('https://python.org')

except WhateverExceptionTheRequestsLibraryRaises:

  pass


It would have to not only look at get(), but everything that it calls.
Either that, or you're back to the Java thing of "catch it or declare
it", and we've seen from Java that that's a bad idea.

ChrisA

I don't really understand why those are the only options. The way I see it,
whomever implements get() can just have a list of expected Exceptions and as
a caller you just handle those (now with the added ease that you don't 
need to
look them up out-of-bounds, they're already in your IDE!) and if an 
unexpected
exception happens, well you handle it the same way you do it now (i.e. 
either not

at all, with a bare except, or something along those lines).
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AZ6FLPAYPWBYBI65EGCPCO3B2KBOZGBM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Chris Angelico]

On Fri, Sep 25, 2020 at 4:25 PM Sergio Fenoll  wrote:
> What I had in mind was that an IDE could use this information to show
> autocomplete options when writing except blocks. The other day I was
> writing some code like this:
>
> import requests
>
> try:
>
>  requests.get('https://python.org')
>
> except WhateverExceptionTheRequestsLibraryRaises:
>
>  pass
>

It would have to not only look at get(), but everything that it calls.
Either that, or you're back to the Java thing of "catch it or declare
it", and we've seen from Java that that's a bad idea.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L6M7WROB7FF7SPMPY7DPRW2ZQP4ISR4F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Suggestion: annotated exceptions

[Sergio Fenoll]

O 25/09/20 ás 07:37, Steven D'Aprano escribiu:


I think that it's a truism that any function written in Python could
raise any exception :-)

In practice, though, I think it is reasonable to say that functions will
*typically* but not necessarily exclusively raise a certain set of
exceptions.

What you seem to be describing is similar to Java's "Checked
Exceptions", which are widely agreed to be an failure. Can you explain
why this would be more successful?

The last thing I want to see is people being encouraged to write code
like this:

 def demo(arg)->Something:
 # Raises ValueError
 try:
 processing...
 except ValueError:
 raise
 except:
 # Any other unexpected error.
 raise ValueError('something unexpected')


just to ensure that the declared exception is correct.
I haven't extensively used Java, but the way I understand it in Java 
you're meant to either handle every single exception that a function can 
throw, or explicitly throw anything you haven't handled.
I do agree that such an approach wouldn't be great for Python. I don't 
think (but don't quote me on that) that introducing this idea of 
annotating exceptions would lead people to writing code like the above 
(in the same way introducing typing annotations didn't lead to people 
checking all function argument types in every function :-P ). After all, 
there wouldn't be any runtime checks to make sure you handled any 
raisable exceptions.



In the same vein as adding type annotations to code, I think it'd be
very useful to have exception "raises" annotations, i.e. a way to
annotate what exceptions a function raises. Again, like type
annotations, it shouldn't be mandatory nor actually be enforced at
runtime. It would purely serve as a feature that IDEs can make use of.

How would an IDE make use of this?
What I had in mind was that an IDE could use this information to show 
autocomplete options when writing except blocks. The other day I was 
writing some code like this:


import requests

try:

    requests.get('https://python.org')

except WhateverExceptionTheRequestsLibraryRaises:

    pass

And I remember having to dive into the documentation of the library to 
figure out what exception was actually raised by this method, and what 
the base class of said exception was.
Whereas, IMO, it would be a lot nicer to have a built-in way to know 
what exceptions may be raised by a function call.



An example of how it may look:

def divide(numerator: float, denominator: float) raises [ZeroDivisionError] -> 
float:
     return numerator / denominator

If either argument is a subclass of float, this could raise any
exception.
I see! Well, the way I had it in mind, you'd only really annotate the 
exceptions you as a function implementer think are meaningful. It 
doesn't have to be an exhaustive list, just exceptions the callers of 
the function will (probably) want to handle.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LWUWC4RXI5WOWAIYHWSTJK23VWNHBDOG/
Code of Conduct: http://python.org/psf/codeofconduct/