Re: [Python-ideas] Improving Catching Exceptions
Hi Andy, What you propose is essentially the "try .. catch .. in" construct as described for Standard ML in: https://pdfs.semanticscholar.org/b24a/60f84b296482769bb6752feeb3d93ba6aee8.pdf Something similar for Clojure is at: https://github.com/rufoa/try-let So clearly this is something more people have struggled with. The paper above goes into deep detail on the practical and (proof-)theoretical advantages of such a construct. Stephan 2017-06-23 1:47 GMT+02:00 Andy Dirnberger : > > >> On Jun 22, 2017, at 7:29 PM, Cameron Simpson wrote: >> >>> On 23Jun2017 06:55, Steven D'Aprano wrote: On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote: We usually teach our newbies to catch exceptions as narrowly as possible, i.e. MyModel.DoesNotExist instead of a plain Exception. This works out quite well for now but the number of examples continue to grow where it's not enough. >>> >>> (1) Under what circumstances is it not enough? >> >> I believe that he means that it isn't precise enough. In particular, "nested >> exceptions" to me, from his use cases, means exceptions thrown from within >> functions called at the top level. I want this control too sometimes. >> >> Consider: >> >> try: >> foo(bah[5]) >> except IndexError as e: >> ... infer that there is no bah[5] ... >> >> Of course, it is possible that bah[5] existed and that foo() raised an >> IndexError of its own. One might intend some sane handling of a missing >> bah[5] but instead silently conceal the IndexError from foo() by mishandling >> it as a missing bah[5]. >> >> Naturally one can rearrange this code to call foo() outside that try/except, >> but that degree of control often leads to quite fiddly looking code with the >> core flow obscured by many tiny try/excepts. >> >> One can easily want, instead, some kind of "shallow except", which would >> catch exceptions only if they were directly raised from the surface code; >> such a construct would catch the IndexError from a missing bah[5] in the >> example above, but _not_ catch an IndexError raised from deeper code such >> within the foo() function. >> >> Something equivalent to: >> >> try: >> foo(bah[5]) >> except IndexError as e: >> if e.__traceback__ not directly from the try..except lines: >> raise >> ... infer that there is no bah[5] ... >> >> There doesn't seem to be a concise way to write that. It might not even be >> feasible at all, as one doesn't have a way to identify the line(s) within >> the try/except in a form that one can recognise in a traceback. > > How about something like this? > > try: > val = bah[5] > except IndexError: > # handle your expected exception here > else: > foo(val) >> >> >> Cheers, >> Cameron Simpson >> ___ >> Python-ideas mailing list >> Python-ideas@python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
On 23.06.2017 03:02, Cameron Simpson wrote: How about something like this? try: val = bah[5] except IndexError: # handle your expected exception here else: foo(val) That is the kind of refactor to which I alluded in the paragraph above. Doing that a lot tends to obscure the core logic of the code, hence the desire for something more succinct requiring less internal code fiddling. And depending on how complex bha.__getitem__ is, it can raise IndexError unintentionally as well. So, rewriting the outer code doesn't even help then. :-( Regards, Sven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
On 23 June 2017 at 15:20, Sven R. Kunze wrote: > On 23.06.2017 03:02, Cameron Simpson wrote: > > > How about something like this? > >try: >val = bah[5] >except IndexError: ># handle your expected exception here >else: >foo(val) > > > That is the kind of refactor to which I alluded in the paragraph above. > Doing that a lot tends to obscure the core logic of the code, hence the > desire for something more succinct requiring less internal code fiddling. > > > And depending on how complex bha.__getitem__ is, it can raise IndexError > unintentionally as well. So, rewriting the outer code doesn't even help > then. :-( At this point, it becomes unclear to me what constitutes an "intentional" IndexError, as opposed to an "unintentional" one, at least in any sense that can actually be implemented. I appreciate that you want IndexError to mean "there is no 5th element in bah". But if bah has a __getitem__ that raises IndexError for any reason other than that, then the __getitem__ implementation has a bug. And while it might be nice to be able to continue working properly even when the code you're executing has bugs, I think it's a bit optimistic to hope for :-) On the other hand, I do see the point that insisting on finer and finer grained exception handling ultimately ends up with unreadable code. But it's not a problem I'd expect to see much in real life code (where code is either not written that defensively, because either there's context that allows the coder to make assumptions that objects will behave reasonably sanely, or the code gets refactored to put the exception handling in a function, or something like that). Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
Hi Stephan, On Fri, Jun 23, 2017 at 6:23 AM, Stephan Houben wrote: > Hi Andy, > > What you propose is essentially the "try .. catch .. in" construct as > described for Standard ML in: > It's not really a proposal. It's existing syntax. I was suggesting a way to implement the example that would catch an IndexError raised by accessing elements in bah but not those raised by foo. > > https://pdfs.semanticscholar.org/b24a/60f84b296482769bb6752feeb3d93b > a6aee8.pdf > > Something similar for Clojure is at: > https://github.com/rufoa/try-let > > So clearly this is something more people have struggled with. > The paper above goes into deep detail on the practical and > (proof-)theoretical > advantages of such a construct. > > Stephan > Andy > > 2017-06-23 1:47 GMT+02:00 Andy Dirnberger : > > > > > >> On Jun 22, 2017, at 7:29 PM, Cameron Simpson wrote: > >> > >>> On 23Jun2017 06:55, Steven D'Aprano wrote: > On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote: > We usually teach our newbies to catch exceptions as narrowly as > possible, i.e. MyModel.DoesNotExist instead of a plain Exception. This > works out quite well for now but the number of examples continue to > grow > where it's not enough. > >>> > >>> (1) Under what circumstances is it not enough? > >> > >> I believe that he means that it isn't precise enough. In particular, > "nested exceptions" to me, from his use cases, means exceptions thrown from > within functions called at the top level. I want this control too sometimes. > >> > >> Consider: > >> > >> try: > >> foo(bah[5]) > >> except IndexError as e: > >> ... infer that there is no bah[5] ... > >> > >> Of course, it is possible that bah[5] existed and that foo() raised an > IndexError of its own. One might intend some sane handling of a missing > bah[5] but instead silently conceal the IndexError from foo() by > mishandling it as a missing bah[5]. > >> > >> Naturally one can rearrange this code to call foo() outside that > try/except, but that degree of control often leads to quite fiddly looking > code with the core flow obscured by many tiny try/excepts. > >> > >> One can easily want, instead, some kind of "shallow except", which > would catch exceptions only if they were directly raised from the surface > code; such a construct would catch the IndexError from a missing bah[5] in > the example above, but _not_ catch an IndexError raised from deeper code > such within the foo() function. > >> > >> Something equivalent to: > >> > >> try: > >> foo(bah[5]) > >> except IndexError as e: > >> if e.__traceback__ not directly from the try..except lines: > >> raise > >> ... infer that there is no bah[5] ... > >> > >> There doesn't seem to be a concise way to write that. It might not even > be feasible at all, as one doesn't have a way to identify the line(s) > within the try/except in a form that one can recognise in a traceback. > > > > How about something like this? > > > > try: > > val = bah[5] > > except IndexError: > > # handle your expected exception here > > else: > > foo(val) > >> > >> > >> Cheers, > >> Cameron Simpson > >> ___ > >> Python-ideas mailing list > >> Python-ideas@python.org > >> https://mail.python.org/mailman/listinfo/python-ideas > >> Code of Conduct: http://python.org/psf/codeofconduct/ > > ___ > > Python-ideas mailing list > > Python-ideas@python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] be upfront if you aren't willing to implement your own idea (was: socket module: plain stuples vs named tuples - Thank you)
Everyone, please be upfront when proposing any ideas if you refuse to
implement your own idea yourself. It's implicit that if you have an idea to
discuss here that you are serious enough about it to see it happen, so if
that's not the case then do say so in your first email (obviously if your
circumstances change during the discussion then that's understandable).
Otherwise people will spend what little spare time they have helping you
think through your idea, and then find out that the discussion will more
than likely end up leading to no change because the most motivated person
behind the discussion isn't motivated enough to actually enact the change.
And if you lack knowledge in how to implement the idea or a certain area of
expertise, please be upfront about that as well. We have had instances here
where ideas have gone as far as PEPs to only find out the OP didn't know C
which was a critical requirement to implementing the idea, and so the idea
just fell to the wayside and hasn't gone anywhere. It's totally reasonable
to ask for help, but once again, please be upfront that you will need it to
have any chance of seeing your idea come to fruition.
To be perfectly frank, I personally find it misleading to not be told
upfront that you know you will need help (if you learn later because you
didn't know e.g. C would be required, that's different, but once you do
learn then once again be upfront about it). Otherwise I personally feel
like I was tricked into a discussion under false pretenses that the OP was
motivated enough to put the effort in to see their idea come to be. Had I
known to begin with that no one was actually stepping forward to make this
change happen I would have skipped the thread and spent the time I put in
following the discussion into something more productive like reviewing a
pull request.
On Thu, 22 Jun 2017 at 08:26 Thomas Güttler
wrote:
> thank you! I am happy that Guido is open for a pull request ... There
> were +1 votes, too (and some concern about python
> startup time).
>
>
> I stopped coding in spare time, since my children are more important at
> the moment .. if some wants to try it ... go
> ahead and implement named tuples for the socket standard library - would
> be great.
>
> Just for the records, I came here because of this feature request:
>
> https://github.com/giampaolo/psutil/issues/928
>
> Regards,
>Thomas Güttler
>
> PS: For some strange reasons I received only some mails of this thread.
> But I could
> find the whole thread in the archive.
>
> Am 20.06.2017 um 04:05 schrieb INADA Naoki:
> > Namedtuple in Python make startup time slow.
> > So I'm very conservative to convert tuple to namedtuple in Python.
> > INADA Naoki
> >
> >
> > On Tue, Jun 20, 2017 at 7:27 AM, Victor Stinner
> > wrote:
> >> Oh, about the cost of writing C code, we started to enhance the socket
> >> module in socket.py but keep the C code unchanged. I am thinking to the
> >> support of enums. Some C functions are wrapped in Python.
> >>
> >> Victor
> >>
> >> Le 19 juin 2017 11:59 PM, "Guido van Rossum" a
> écrit :
> >>>
> >>> There are examples in timemodule.c which went through a similar
> conversion
> >>> from plain tuples to (sort-of) named tuples. I agree that upgrading the
> >>> tuples returned by the socket module to named tuples would be nice,
> but it's
> >>> a low priority project. Maybe someone interested can create a PR?
> (First
> >>> open an issue stating that you're interested; point to this email from
> me to
> >>> prevent that some other core dev just closes it again.)
> >>>
> >>> On Mon, Jun 19, 2017 at 2:24 PM, Victor Stinner <
> victor.stin...@gmail.com>
> >>> wrote:
>
> Hi,
>
> 2017-06-13 22:13 GMT+02:00 Thomas Güttler <
> guettl...@thomas-guettler.de>:
> > AFAIK the socket module returns plain tuples in Python3:
> >
> >https://docs.python.org/3/library/socket.html
> >
> > Why not use named tuples?
>
> For technical reasons: the socket module is mostly implemented in the
> C language, and define a "named tuple" in C requires to implement a
> "sequence" time which requires much more code than creating a tuple.
>
> In short, create a tuple is as simple as Py_BuildValue("OO", item1,
> item2).
>
> Creating a "sequence" type requires something like 50 lines of code,
> maybe more, I don't know exactly.
>
> Victor
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> --Guido van Rossum (python.org/~guido)
> >>
> >>
> >> ___
> >> Python-ideas mailing list
> >> Python-ideas@python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] be upfront if you aren't willing to implement your own idea
On 2017-06-23 09:49, Brett Cannon wrote: Everyone, please be upfront when proposing any ideas if you refuse to implement your own idea yourself. It's implicit that if you have an idea to discuss here that you are serious enough about it to see it happen, so if that's not the case then do say so in your first email (obviously if your circumstances change during the discussion then that's understandable). Otherwise people will spend what little spare time they have helping you think through your idea, and then find out that the discussion will more than likely end up leading to no change because the most motivated person behind the discussion isn't motivated enough to actually enact the change. And if you lack knowledge in how to implement the idea or a certain area of expertise, please be upfront about that as well. We have had instances here where ideas have gone as far as PEPs to only find out the OP didn't know C which was a critical requirement to implementing the idea, and so the idea just fell to the wayside and hasn't gone anywhere. It's totally reasonable to ask for help, but once again, please be upfront that you will need it to have any chance of seeing your idea come to fruition. To be perfectly frank, I personally find it misleading to not be told upfront that you know you will need help (if you learn later because you didn't know e.g. C would be required, that's different, but once you do learn then once again be upfront about it). Otherwise I personally feel like I was tricked into a discussion under false pretenses that the OP was motivated enough to put the effort in to see their idea come to be. Had I known to begin with that no one was actually stepping forward to make this change happen I would have skipped the thread and spent the time I put in following the discussion into something more productive like reviewing a pull request. That is a reasonable position, but I think if that's really how this list is supposed to work then it'd be good to state those requirements more explicitly in the list description. Right now the description (https://mail.python.org/mailman/listinfo/python-ideas) just says the list is for "discussion of speculative language ideas for Python". There is no hint that any particular technical qualifications are required other than having used Python enough to have an idea about how to improve it. I also don't think such a requirement is obvious even from reading the list traffic (since I've rarely seen anyone explicitly state their inability to implement, as you suggest, although it does sometimes come up later, as in this case). No doubt this leads to the occasional cockamamie proposal but I think it also allows discussion of useful ideas that otherwise might never be raised. Also, the description does mention that at some point ideas might get moved on to python-dev; although it's not explicit about how this works, I think that creates a vague impression that thinking about how or whether you can implement an idea might be something for a later stage. That said, I don't personally agree with your position here. My impression of discussion on this list is that a good deal of it doesn't really have to do with implementation at all. It has to do with the proposal itself in terms of how it would feel to use it, hashing out what its semantics would be, what the benefits would be for code readability, what confusion it might create etc. --- in short, discussion from the perspective of people who USE Python, not people who implement Python. I think that's good discussion to have even if the proposal eventually stalls because no one with the right skills has the time or inclination to implement it. It would be a shame for all such discussion to get nipped in the bud just because the person with the original proposal doesn't know C or whatever. Also, because, as you say, some people don't know what would be needed to implement their ideas, requiring this kind of disclosure might perversely muffle discussion from people who know enough to know they don't know how to implement their idea, while still allowing all the ideas from people who don't even know whether they know how to implement their idea --- and the latter are probably more likely to fall into the cockamamie category. I realize you're not proposing that all such discussion be stopped entirely, just that it be tagged as I-can't-implement-this-myself at the outset. However, your last paragraph suggests to me that the effect might be similar. You seem to be saying that (some of) those who do know how to implement stuff would like to be able to ignore discussion from anyone who doesn't know how to implement stuff. That's certainly anyone's prerogative, but I think it would be a shame if this resulted in a bifurcation of the list in which ideas can't reach the attention of people who could implement them unless they're proposed by someone who could do s
Re: [Python-ideas] Improving Catching Exceptions
2017-06-23 17:09 GMT+02:00 Andy Dirnberger : > It's not really a proposal. It's existing syntax. Wow! I have been using Python since 1.5.2 and I never knew this. This is not Guido's famous time machine in action, by any chance? Guess there's some code to refactor using this construct now... Stephan 2017-06-23 17:09 GMT+02:00 Andy Dirnberger : > Hi Stephan, > > On Fri, Jun 23, 2017 at 6:23 AM, Stephan Houben > wrote: >> >> Hi Andy, >> >> What you propose is essentially the "try .. catch .. in" construct as >> described for Standard ML in: > > > It's not really a proposal. It's existing syntax. I was suggesting a way to > implement the example that would catch an IndexError raised by accessing > elements in bah but not those raised by foo. > > >> >> >> >> https://pdfs.semanticscholar.org/b24a/60f84b296482769bb6752feeb3d93ba6aee8.pdf >> >> Something similar for Clojure is at: >> https://github.com/rufoa/try-let >> >> So clearly this is something more people have struggled with. >> The paper above goes into deep detail on the practical and >> (proof-)theoretical >> advantages of such a construct. >> >> Stephan > > > Andy > > >> >> >> 2017-06-23 1:47 GMT+02:00 Andy Dirnberger : >> > >> > >> >> On Jun 22, 2017, at 7:29 PM, Cameron Simpson wrote: >> >> >> >>> On 23Jun2017 06:55, Steven D'Aprano wrote: >> On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote: >> We usually teach our newbies to catch exceptions as narrowly as >> possible, i.e. MyModel.DoesNotExist instead of a plain Exception. >> This >> works out quite well for now but the number of examples continue to >> grow >> where it's not enough. >> >>> >> >>> (1) Under what circumstances is it not enough? >> >> >> >> I believe that he means that it isn't precise enough. In particular, >> >> "nested exceptions" to me, from his use cases, means exceptions thrown >> >> from >> >> within functions called at the top level. I want this control too >> >> sometimes. >> >> >> >> Consider: >> >> >> >> try: >> >> foo(bah[5]) >> >> except IndexError as e: >> >> ... infer that there is no bah[5] ... >> >> >> >> Of course, it is possible that bah[5] existed and that foo() raised an >> >> IndexError of its own. One might intend some sane handling of a missing >> >> bah[5] but instead silently conceal the IndexError from foo() by >> >> mishandling >> >> it as a missing bah[5]. >> >> >> >> Naturally one can rearrange this code to call foo() outside that >> >> try/except, but that degree of control often leads to quite fiddly looking >> >> code with the core flow obscured by many tiny try/excepts. >> >> >> >> One can easily want, instead, some kind of "shallow except", which >> >> would catch exceptions only if they were directly raised from the surface >> >> code; such a construct would catch the IndexError from a missing bah[5] in >> >> the example above, but _not_ catch an IndexError raised from deeper code >> >> such within the foo() function. >> >> >> >> Something equivalent to: >> >> >> >> try: >> >> foo(bah[5]) >> >> except IndexError as e: >> >> if e.__traceback__ not directly from the try..except lines: >> >> raise >> >> ... infer that there is no bah[5] ... >> >> >> >> There doesn't seem to be a concise way to write that. It might not even >> >> be feasible at all, as one doesn't have a way to identify the line(s) >> >> within >> >> the try/except in a form that one can recognise in a traceback. >> > >> > How about something like this? >> > >> > try: >> > val = bah[5] >> > except IndexError: >> > # handle your expected exception here >> > else: >> > foo(val) >> >> >> >> >> >> Cheers, >> >> Cameron Simpson >> >> ___ >> >> Python-ideas mailing list >> >> Python-ideas@python.org >> >> https://mail.python.org/mailman/listinfo/python-ideas >> >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > ___ >> > Python-ideas mailing list >> > Python-ideas@python.org >> > https://mail.python.org/mailman/listinfo/python-ideas >> > Code of Conduct: http://python.org/psf/codeofconduct/ > > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] be upfront if you aren't willing to implement your own idea
"to put it succinctly" -- IMO we shouldn't discuss features without giving thought to their implementation. On Fri, Jun 23, 2017 at 11:28 AM, Brendan Barnwell wrote: > On 2017-06-23 09:49, Brett Cannon wrote: > >> Everyone, please be upfront when proposing any ideas if you refuse to >> implement your own idea yourself. It's implicit that if you have an idea >> to discuss here that you are serious enough about it to see it happen, >> so if that's not the case then do say so in your first email (obviously >> if your circumstances change during the discussion then that's >> understandable). Otherwise people will spend what little spare time they >> have helping you think through your idea, and then find out that the >> discussion will more than likely end up leading to no change because the >> most motivated person behind the discussion isn't motivated enough to >> actually enact the change. >> >> And if you lack knowledge in how to implement the idea or a certain area >> of expertise, please be upfront about that as well. We have had >> instances here where ideas have gone as far as PEPs to only find out the >> OP didn't know C which was a critical requirement to implementing the >> idea, and so the idea just fell to the wayside and hasn't gone anywhere. >> It's totally reasonable to ask for help, but once again, please be >> upfront that you will need it to have any chance of seeing your idea >> come to fruition. >> >> To be perfectly frank, I personally find it misleading to not be told >> upfront that you know you will need help (if you learn later because you >> didn't know e.g. C would be required, that's different, but once you do >> learn then once again be upfront about it). Otherwise I personally feel >> like I was tricked into a discussion under false pretenses that the OP >> was motivated enough to put the effort in to see their idea come to be. >> Had I known to begin with that no one was actually stepping forward to >> make this change happen I would have skipped the thread and spent the >> time I put in following the discussion into something more productive >> like reviewing a pull request. >> > > That is a reasonable position, but I think if that's really how > this list is supposed to work then it'd be good to state those requirements > more explicitly in the list description. Right now the description ( > https://mail.python.org/mailman/listinfo/python-ideas) just says the list > is for "discussion of speculative language ideas for Python". There is no > hint that any particular technical qualifications are required other than > having used Python enough to have an idea about how to improve it. I also > don't think such a requirement is obvious even from reading the list > traffic (since I've rarely seen anyone explicitly state their inability to > implement, as you suggest, although it does sometimes come up later, as in > this case). No doubt this leads to the occasional cockamamie proposal but > I think it also allows discussion of useful ideas that otherwise might > never be raised. Also, the description does mention that at some point > ideas might get moved on to python-dev; although it's not explicit about > how this works, I think that creates a vague impression that thinking about > how or whether you can implement an idea might be something for a later > stage. > > That said, I don't personally agree with your position here. My > impression of discussion on this list is that a good deal of it doesn't > really have to do with implementation at all. It has to do with the > proposal itself in terms of how it would feel to use it, hashing out what > its semantics would be, what the benefits would be for code readability, > what confusion it might create etc. --- in short, discussion from the > perspective of people who USE Python, not people who implement Python. I > think that's good discussion to have even if the proposal eventually stalls > because no one with the right skills has the time or inclination to > implement it. It would be a shame for all such discussion to get nipped in > the bud just because the person with the original proposal doesn't know C > or whatever. Also, because, as you say, some people don't know what would > be needed to implement their ideas, requiring this kind of disclosure might > perversely muffle discussion from people who know enough to know they don't > know how to implement their idea, while still allowing all the ideas from > people who don't even know whether they know how to implement their idea > --- and the latter are probably more likely to fall into the cockamamie > category. > > I realize you're not proposing that all such discussion be stopped > entirely, just that it be tagged as I-can't-implement-this-myself at the > outset. However, your last paragraph suggests to me that the effect might > be similar. You seem to be saying that (some of) those who do know how to > implement stuff would like to be
Re: [Python-ideas] Improving Catching Exceptions
On Fri, Jun 23, 2017 at 09:29:23AM +1000, Cameron Simpson wrote:
> On 23Jun2017 06:55, Steven D'Aprano wrote:
> >On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote:
> >>We usually teach our newbies to catch exceptions as narrowly as
> >>possible, i.e. MyModel.DoesNotExist instead of a plain Exception. This
> >>works out quite well for now but the number of examples continue to grow
> >>where it's not enough.
> >
> >(1) Under what circumstances is it not enough?
>
> I believe that he means that it isn't precise enough. In particular,
> "nested exceptions" to me, from his use cases, means exceptions thrown from
> within functions called at the top level. I want this control too sometimes.
But why teach it to newbies? Sven explicitly mentions teaching
beginners. If we are talking about advanced features for experts, that's
one thing, but it's another if we're talking about Python 101 taught to
beginners and newbies.
Do we really need to be teaching beginners how to deal with circular
imports beyond "don't do it"?
> Consider:
>
>try:
>foo(bah[5])
>except IndexError as e:
>... infer that there is no bah[5] ...
>
> Of course, it is possible that bah[5] existed and that foo() raised an
> IndexError of its own. One might intend some sane handling of a missing
> bah[5] but instead silently conceal the IndexError from foo() by
> mishandling it as a missing bah[5].
Indeed -- if both foo and bah[5] can raise IndexError when the coder
believes that only bah[5] can, then the above code is simply buggy.
On the other hand, if the author is correct that foo cannot raise
IndexError, then the code as given is fine.
> Naturally one can rearrange this code to call foo() outside that
> try/except, but that degree of control often leads to quite fiddly looking
> code with the core flow obscured by many tiny try/excepts.
Sadly, that is often the nature of real code, as opposed to "toy" or
textbook code that demonstrates an algorithm as cleanly as possible.
It's been said that for every line of code in the textbook, the
function needs ten lines in production.
> One can easily want, instead, some kind of "shallow except", which would
> catch exceptions only if they were directly raised from the surface code;
> such a construct would catch the IndexError from a missing bah[5] in the
> example above, but _not_ catch an IndexError raised from deeper code such
> within the foo() function.
I think the concept of a "shallow exception" is ill-defined, and to the
degree that it is defined, it is *dangerous*: a bug magnet waiting to
strike.
What do you mean by "directly raised from the surface code"? Why is
bah[5] "surface code" but foo(x) is not? But call a function (or
method).
But worse, it seems that the idea of "shallow" or "deep" depends on
*implementation details* of where the exception comes from.
For example, changing from a recursive function to a while loop might
change the exception from "50 function calls deep" to "1 function deep".
What makes bah[5] "shallow"? For all you know, it calls a chain of a
dozen __getitem__ methods, due to inheritance or proxying, before the
exception is actually raised. Or it might call just a single __getitem__
method, but the method's implementation puts the error checking into a
helper method:
def __getitem__(self, n):
self._validate(n) # may raise IndexError
...
How many function calls are shallow, versus deep?
This concept of a shallow exception is, it seems to me, a bug magnet. It
is superficially attractive, but then you realise that:
try:
spam[5]
except shallow IndexError:
...
will behave differently depending on how spam is implemented, even if
the interface (raises IndexError) is identical.
It seems to me that this concept is trying to let us substitute some
sort of undefined but mechanical measurement of "shallowness" for
actually understanding what our code does. I don't think this can work.
It would be awesome if there was some way for our language to Do What We
Mean instead of What We Say. And then we can grow a money tree, and have
a magic plum-pudding that stays the same size no matter how many slices
we eat, and electricity so cheap the power company pays you to use it...
*wink*
> The nested exception issue actually bites me regularly, almost always with
> properties.
[...]
> However, more commonly I end up hiding coding errors with @property,
> particularly nasty when the coding error is deep in some nested call. Here
> is a nondeep example based on the above:
>
>@property
>def target(self):
>if len(self.targgets) == 1:
>return self.targets[0]
>raise AttributeError('only exists when this has exactly one target')
The obvious solution to this is to learn to spell correctly :-)
Actually, a linter probably would have picked up that typo. But I do
see that the issue if more than just typos.
[...]
>try:
Re: [Python-ideas] be upfront if you aren't willing to implement your own idea
On 23 June 2017 at 19:28, Brendan Barnwell wrote: > So to put it succinctly, as someone who's found discussion on this list > interesting and valuable, I think there is value in having discussion about > "what would Python be like if this idea were implemented" even if we never > get very far with "how would we implement this idea in Python". And I would > find it unfortunate if discussion of the former were prematurely restricted > by worries about the latter. No-one is proposing otherwise, just that people are open when starting a discussion as to whether they anticipate being able to follow through with an implementation if the idea meets with approval, or if they are simply making a suggestion that they hope someone else will take up. That's not too much to ask, nor does it in any way stifle reasonable discussion (it may discourage people who want to *deliberately* give the impression that they will do the work, but actually have no intention of doing so - but I hope there's no-one like that here and if there were, I'm happy with discouraging them). So I'm +1 on Brett's request. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] be upfront if you aren't willing to implement your own idea
+1 I'm quite active in the CoffeeScript community, but am also on a ton of medication that ultimately means I won't implement much of what I suggest doing, but the core devs understand the situation well enough to respond accordingly. It really does help when people know what they can reasonably expect from others, and it doesn't take much to let them know. None of this has ever prevented me from being involved. It just prevents me from wasting other people's time. -- Carl -- Carl Smith carl.in...@gmail.com On 23 June 2017 at 21:09, Paul Moore wrote: > On 23 June 2017 at 19:28, Brendan Barnwell wrote: > > So to put it succinctly, as someone who's found discussion on this list > > interesting and valuable, I think there is value in having discussion > about > > "what would Python be like if this idea were implemented" even if we > never > > get very far with "how would we implement this idea in Python". And I > would > > find it unfortunate if discussion of the former were prematurely > restricted > > by worries about the latter. > > No-one is proposing otherwise, just that people are open when starting > a discussion as to whether they anticipate being able to follow > through with an implementation if the idea meets with approval, or if > they are simply making a suggestion that they hope someone else will > take up. That's not too much to ask, nor does it in any way stifle > reasonable discussion (it may discourage people who want to > *deliberately* give the impression that they will do the work, but > actually have no intention of doing so - but I hope there's no-one > like that here and if there were, I'm happy with discouraging them). > > So I'm +1 on Brett's request. > Paul > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
On 24Jun2017 05:02, Steven D'Aprano wrote: On Fri, Jun 23, 2017 at 09:29:23AM +1000, Cameron Simpson wrote: On 23Jun2017 06:55, Steven D'Aprano wrote: >On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote: >>We usually teach our newbies to catch exceptions as narrowly as >>possible, i.e. MyModel.DoesNotExist instead of a plain Exception. This >>works out quite well for now but the number of examples continue to grow >>where it's not enough. > >(1) Under what circumstances is it not enough? I believe that he means that it isn't precise enough. In particular, "nested exceptions" to me, from his use cases, means exceptions thrown from within functions called at the top level. I want this control too sometimes. But why teach it to newbies? Sven explicitly mentions teaching beginners. If we are talking about advanced features for experts, that's one thing, but it's another if we're talking about Python 101 taught to beginners and newbies. It depends. Explaining that exceptions from called code can be mishandled by a naive try/except is something newbies need to learn to avoid common pitfalls with exceptions, and a real world situation that must be kept in mind when acting on caught exceptions. Do we really need to be teaching beginners how to deal with circular imports beyond "don't do it"? Sven's example is with import. The situation is more general. [... snip basic example of simple code where IndexError can arise from multiple causes ...] Naturally one can rearrange this code to call foo() outside that try/except, but that degree of control often leads to quite fiddly looking code with the core flow obscured by many tiny try/excepts. Sadly, that is often the nature of real code, as opposed to "toy" or textbook code that demonstrates an algorithm as cleanly as possible. It's been said that for every line of code in the textbook, the function needs ten lines in production. But not always so. And the reason for many language constructs and idioms is explicitly to combat what would otherwise need 10 lines of code (or 100 to do it "right" with corner cases), obscuring the core task and reducing readability/maintainability. So "Sadly, that is often the nature of real code" is not itself an argument against this idea. One can easily want, instead, some kind of "shallow except", which would catch exceptions only if they were directly raised from the surface code; such a construct would catch the IndexError from a missing bah[5] in the example above, but _not_ catch an IndexError raised from deeper code such within the foo() function. I think the concept of a "shallow exception" is ill-defined, and to the degree that it is defined, it is *dangerous*: a bug magnet waiting to strike. What do you mean by "directly raised from the surface code"? Why is bah[5] "surface code" but foo(x) is not? But call a function (or method). [...] I've replied to Paul Moore and suggested this definition as implementable and often useful: A shallow catch would effectively need to mean "the exception's uppermost traceback frame refers to one of the program lines in the try/except suite". Which would work well for lists and other builtin types. And might be insufficient for a duck-type with python-coded dunder methods. The target here is not perform magic but to have a useful tool to identify exceptions that arise fairly directly from the adjacent clode and not what it calls. Without writing cumbersome and fragile boilerplate to dig into an exception's traceback. Cheers, Cameron Simpson ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
On 23Jun2017 11:48, Nick Coghlan wrote:
On 23 June 2017 at 09:29, Cameron Simpson wrote:
This is so common that I actually keep around a special hack:
def prop(func):
''' The builtin @property decorator lets internal AttributeErrors
escape.
While that can support properties that appear to exist
conditionally,
in practice this is almost never what I want, and it masks deeper
errors.
Hence this wrapper for @property that transmutes internal
AttributeErrors
into RuntimeErrors.
'''
def wrapper(*a, **kw):
try:
return func(*a, **kw)
except AttributeError as e:
e2 = RuntimeError("inner function %s raised %s" % (func, e))
if sys.version_info[0] >= 3:
try:
eval('raise e2 from e', globals(), locals())
except:
# FIXME: why does this raise a SyntaxError?
raise e
else:
raise e2
return property(wrapper)
Slight tangent, but I do sometimes wonder if adding a decorator
factory like the following to functools might be useful:
def raise_when_returned(return_exc):
def decorator(f):
@wraps(f)
def wrapper(*args, **kwds):
try:
result = f(*args, **kwds)
except selective_exc as unexpected_exc:
msg = "inner function {} raised {}".format(f,
unexpected_exc)
raise RuntimeError(msg) from unexpected_exc
if isinstance(result, return_exc):
raise result
return result
It's essentially a generalisation of PEP 479 to arbitrary exception
types, since it lets you mark a particular exception type as being
communicated back to the wrapper via the return channel rather than as
a regular exception:
def with_traceback(exc):
try:
raise exc
except BaseException as caught_exc:
return caught_exc
@property
@raise_when_returned(AttributeError)
def target(self):
if len(self.targets) == 1:
return self.targets[0]
return with_traceback(AttributeError('only exists when this
has exactly one target'))
Funnily enough I have an @transmute decorator which serves just this purpose.
It doesn't see as much use as I might imagine, but that is partially because my
function predates "raise ... from", which meant that it loses the stack trace
from the transmuted exception, impeding debugging. I need to revisit it with
that in mind.
So yes, your proposed decorator has supporting real world use cases in my
world.
Cheers,
Cameron Simpson
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] be upfront if you aren't willing to implement your own idea (was: socket module: plain stuples vs named tuples - Thank you)
It has been brought to my attention that some people found this email as
sounding rather angry. I was frustrated (and there is more to this specific
issue than what everyone is seeing publicly), but I didn't meant for it to
come off as angry, and for that I apologize.
On Fri, 23 Jun 2017 at 09:49 Brett Cannon wrote:
> Everyone, please be upfront when proposing any ideas if you refuse to
> implement your own idea yourself. It's implicit that if you have an idea to
> discuss here that you are serious enough about it to see it happen, so if
> that's not the case then do say so in your first email (obviously if your
> circumstances change during the discussion then that's understandable).
> Otherwise people will spend what little spare time they have helping you
> think through your idea, and then find out that the discussion will more
> than likely end up leading to no change because the most motivated person
> behind the discussion isn't motivated enough to actually enact the change.
>
> And if you lack knowledge in how to implement the idea or a certain area
> of expertise, please be upfront about that as well. We have had instances
> here where ideas have gone as far as PEPs to only find out the OP didn't
> know C which was a critical requirement to implementing the idea, and so
> the idea just fell to the wayside and hasn't gone anywhere. It's totally
> reasonable to ask for help, but once again, please be upfront that you will
> need it to have any chance of seeing your idea come to fruition.
>
> To be perfectly frank, I personally find it misleading to not be told
> upfront that you know you will need help (if you learn later because you
> didn't know e.g. C would be required, that's different, but once you do
> learn then once again be upfront about it). Otherwise I personally feel
> like I was tricked into a discussion under false pretenses that the OP was
> motivated enough to put the effort in to see their idea come to be. Had I
> known to begin with that no one was actually stepping forward to make this
> change happen I would have skipped the thread and spent the time I put in
> following the discussion into something more productive like reviewing a
> pull request.
>
> On Thu, 22 Jun 2017 at 08:26 Thomas Güttler
> wrote:
>
>> thank you! I am happy that Guido is open for a pull request ... There
>> were +1 votes, too (and some concern about python
>> startup time).
>>
>>
>> I stopped coding in spare time, since my children are more important at
>> the moment .. if some wants to try it ... go
>> ahead and implement named tuples for the socket standard library - would
>> be great.
>>
>> Just for the records, I came here because of this feature request:
>>
>> https://github.com/giampaolo/psutil/issues/928
>>
>> Regards,
>>Thomas Güttler
>>
>> PS: For some strange reasons I received only some mails of this thread.
>> But I could
>> find the whole thread in the archive.
>>
>> Am 20.06.2017 um 04:05 schrieb INADA Naoki:
>> > Namedtuple in Python make startup time slow.
>> > So I'm very conservative to convert tuple to namedtuple in Python.
>> > INADA Naoki
>> >
>> >
>> > On Tue, Jun 20, 2017 at 7:27 AM, Victor Stinner
>> > wrote:
>> >> Oh, about the cost of writing C code, we started to enhance the socket
>> >> module in socket.py but keep the C code unchanged. I am thinking to the
>> >> support of enums. Some C functions are wrapped in Python.
>> >>
>> >> Victor
>> >>
>> >> Le 19 juin 2017 11:59 PM, "Guido van Rossum" a
>> écrit :
>> >>>
>> >>> There are examples in timemodule.c which went through a similar
>> conversion
>> >>> from plain tuples to (sort-of) named tuples. I agree that upgrading
>> the
>> >>> tuples returned by the socket module to named tuples would be nice,
>> but it's
>> >>> a low priority project. Maybe someone interested can create a PR?
>> (First
>> >>> open an issue stating that you're interested; point to this email
>> from me to
>> >>> prevent that some other core dev just closes it again.)
>> >>>
>> >>> On Mon, Jun 19, 2017 at 2:24 PM, Victor Stinner <
>> victor.stin...@gmail.com>
>> >>> wrote:
>>
>> Hi,
>>
>> 2017-06-13 22:13 GMT+02:00 Thomas Güttler <
>> guettl...@thomas-guettler.de>:
>> > AFAIK the socket module returns plain tuples in Python3:
>> >
>> >https://docs.python.org/3/library/socket.html
>> >
>> > Why not use named tuples?
>>
>> For technical reasons: the socket module is mostly implemented in the
>> C language, and define a "named tuple" in C requires to implement a
>> "sequence" time which requires much more code than creating a tuple.
>>
>> In short, create a tuple is as simple as Py_BuildValue("OO", item1,
>> item2).
>>
>> Creating a "sequence" type requires something like 50 lines of code,
>> maybe more, I don't know exactly.
>>
>> Victor
>> ___
>> Python-ideas mailing list
>> Pyth
Re: [Python-ideas] Improving Catching Exceptions
On 2017-06-23 23:56, Cameron Simpson wrote: On 24Jun2017 05:02, Steven D'Aprano wrote: [snip] I think the concept of a "shallow exception" is ill-defined, and to the degree that it is defined, it is *dangerous*: a bug magnet waiting to strike. What do you mean by "directly raised from the surface code"? Why is bah[5] "surface code" but foo(x) is not? But call a function (or method). [...] I've replied to Paul Moore and suggested this definition as implementable and often useful: A shallow catch would effectively need to mean "the exception's uppermost traceback frame refers to one of the program lines in the try/except suite". Which would work well for lists and other builtin types. And might be insufficient for a duck-type with python-coded dunder methods. The target here is not perform magic but to have a useful tool to identify exceptions that arise fairly directly from the adjacent clode and not what it calls. Without writing cumbersome and fragile boilerplate to dig into an exception's traceback. I think a "shallow exception" would be one that's part of a defined API, as distinct from one that is an artifact of the implementation, a leak in the abstraction. It's like when "raise ... from None" was introduced to help in those cases where you want to replace an exception that's a detail of the (current) internal implementation with one that's intended for the user. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
On 23Jun2017 15:59, Paul Moore wrote: On 23 June 2017 at 15:20, Sven R. Kunze wrote: On 23.06.2017 03:02, Cameron Simpson wrote: How about something like this? try: val = bah[5] except IndexError: # handle your expected exception here else: foo(val) That is the kind of refactor to which I alluded in the paragraph above. Doing that a lot tends to obscure the core logic of the code, hence the desire for something more succinct requiring less internal code fiddling. And depending on how complex bha.__getitem__ is, it can raise IndexError unintentionally as well. So, rewriting the outer code doesn't even help then. :-( At this point, it becomes unclear to me what constitutes an "intentional" IndexError, as opposed to an "unintentional" one, at least in any sense that can actually be implemented. While I agree that in object with its own __getitem__ would look "deep", what I was actually suggesting as a possibility was a "shallow" except catch, not some magic "intentional" semantic. A shallow catch would effectively need to mean "the exceptions uppermost traceback frame referers to one of the program lines in the try/except suite". Which would work well for lists and other builtin types. And might be insufficient for a duck-type with python-coded dunder methods. [...snip...] On the other hand, I do see the point that insisting on finer and finer grained exception handling ultimately ends up with unreadable code. But it's not a problem I'd expect to see much in real life code (where code is either not written that defensively, because either there's context that allows the coder to make assumptions that objects will behave reasonably sanely, or the code gets refactored to put the exception handling in a function, or something like that). Sure, there are many circumstances where a succinct "shallow catch" might not be useful. But there are also plenty of circumstances where one would like just this flavour of precision. Cheers, Cameron Simpson ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
On 23Jun2017 20:30, Stephan Houben wrote: 2017-06-23 17:09 GMT+02:00 Andy Dirnberger : It's not really a proposal. It's existing syntax. Wow! I have been using Python since 1.5.2 and I never knew this. This is not Guido's famous time machine in action, by any chance? Guess there's some code to refactor using this construct now... Alas, no. It is existing syntax in Standard ML, not in Python. Cheers, Cameron Simpson ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
Cameron Simpson wrote: Alas, no. It is existing syntax in Standard ML, not in Python. But Python doesn't need it, because try-except-else covers the same thing. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
Cameron Simpson wrote: try: foo(bah[5]) except IndexError as e: ... infer that there is no bah[5] ... One can easily want, instead, some kind of "shallow except", which would catch exceptions only if they were directly raised from the surface code; The problem I see with that is how to define what counts as "surface code". If the __getitem__ method of bah is written in Python, I don't see how you could tell that an IndexError raised by it should be caught, but one raised by foo() shouldn't. In any case, this doesn't address the issue raised by the OP, which in this example is that if the implementation of bah.__getitem__ calls something else that raises an IndexError, there's no easy way to distinguish that from one raised by bah.__getitem__ itself. I don't see any way to solve that by messing around with different try-except constructs. It can only be addressed from within bah.__getitem__ itself, by having it catch any incidental IndexErrors and turn them into a different exception. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improving Catching Exceptions
Cameron Simpson wrote: A shallow catch would effectively need to mean "the exceptions uppermost traceback frame referers to one of the program lines in the try/except suite". Which would work well for lists and other builtin types. And might be insufficient for a duck-type with python-coded dunder methods. I think it would be a very bad idea to have a language construct that only works for built-in types. It would be far too unpredictable and fragile. -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] be upfront if you aren't willing to implement your own idea
On Fri, 23 Jun 2017 at 13:10 Paul Moore wrote: > On 23 June 2017 at 19:28, Brendan Barnwell wrote: > > So to put it succinctly, as someone who's found discussion on this list > > interesting and valuable, I think there is value in having discussion > about > > "what would Python be like if this idea were implemented" even if we > never > > get very far with "how would we implement this idea in Python". And I > would > > find it unfortunate if discussion of the former were prematurely > restricted > > by worries about the latter. > > No-one is proposing otherwise, just that people are open when starting > a discussion as to whether they anticipate being able to follow > through with an implementation if the idea meets with approval, or if > they are simply making a suggestion that they hope someone else will > take up. That's not too much to ask, nor does it in any way stifle > reasonable discussion (it may discourage people who want to > *deliberately* give the impression that they will do the work, but > actually have no intention of doing so - but I hope there's no-one > like that here and if there were, I'm happy with discouraging them). > > So I'm +1 on Brett's request. +1 to what Paul and Guido said: people are welcome to have hypothetical discussions here as long as they are upfront that it is hypothetical, but I personally choose to ignore all discussions that don't involve discussing the implementation of said idea (hence why learning that at the end is frustrating for those of us who are trying to be pragmatic with our time). Sorry if that wasn't clear enough in my original email where the distinction between "Brett as list admin" and "Brett as list participant" started and ended. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
