Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
On 05.06.2018 0:54, Ivan Pozdeev wrote:
On 04.06.2018 23:52, Ivan Pozdeev wrote:
On 04.06.2018 20:11, Chris Angelico wrote:
On Tue, Jun 5, 2018 at 2:57 AM, Yury Selivanov
wrote:
On Mon, Jun 4, 2018 at 12:50 PM Chris Angelico
wrote:
On Tue, Jun 5, 2018 at 2:11 AM, Victor Stinner
wrote:
[..]
For me, it's fine to catch any exception using "except:" if the
block
contains "raise", typical pattern to cleanup a resource in case of
error. Otherwise, there is a risk of leaking open file or not
flushing
data on disk, for example.
Pardon the dumb question, but why is try/finally unsuitable?
Because try..finally isn't equivalent to try..except? Perhaps you
should look at the actual code:
https://github.com/python/cpython/blob/b609e687a076d77bdd687f5e4def85e29a044bfc/Lib/asyncio/base_events.py#L1117-L1123
In this particular code, it looks like just KeyboardInterrupt needs to
be handled in addition to Exception -- and even that's not certain
'cuz KeyboardInterrupt is an abnormal termination and specifically
designed to not be messed with by the code ("The exception inherits
from |BaseException|
so as to not be accidentally caught by code that catches |Exception|
and thus prevent the interpreter from exiting.").
It only makes sense to catch it in REPL interfaces where the user
clearly wants to terminale the current command rather than the entire
program.
Remembered `pip`, too -- there, it's justified by it working in
transactions.
If e.g. a warning is upgraded to exception, this means that some code
is broken from user's POV, but not from Python team's POV, so we can't
really be sure if we can handle this situation gracefully: our cleanup
code can fail just as well!
Oh. Duh. Yep, it was a dumb question. Sorry! The transport should ONLY
be closed on error.
I smell a big, big design violation here.
The whole point of Exception vs BaseException is that anything not
Exception is "not an error", has a completely different effect on the
program than an error, and thus is to be dealt with completely
differently. For example, warnings do not disrupt the control flow,
and GeneratorExit is normally handled by the `for` loop machinery.
That's the whole point why except: is strongly discouraged.
Be _very_ careful because when a system has matured, the risk of
making bad to disastrous design decisions skyrockets (because "the
big picture" grows ever larger, and it's ever more difficult to
account for all of it).
The best solution I know of is an independent sanity-check against
the project's core design principles: focus solely on them and say if
the suggestion is in harmony with the existing big picture. This
prevents the project from falling victim to
https://en.wikipedia.org/wiki/Design_by_committee in the long run.
This is easier to do for someone not intimately involved with the
change and the affected area 'cuz they are less biased in favor of
the change and less distracted by minute details.
Someone may take up this role to "provide a unified vision" (to
reduce the load on a single
http://meatballwiki.org/wiki/BenevolentDictator , different projects
have tried delegates (this can run afoul of
https://en.wikipedia.org/wiki/Conway%27s_law though) and a
round-robin approach (Apache)).
The best way, however, would probably be for anyone dealing with a
design change to remember to make this check.
This is even easier in Python, 'cuz the core values are officially
formulated as Python Zen, and any module has one or two governing
principles at its core, tops, that can be extracted by skimming
through its docs.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru
--
Regards,
Ivan
--
Regards,
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
On 04.06.2018 23:52, Ivan Pozdeev wrote:
On 04.06.2018 20:11, Chris Angelico wrote:
On Tue, Jun 5, 2018 at 2:57 AM, Yury Selivanov
wrote:
On Mon, Jun 4, 2018 at 12:50 PM Chris Angelico
wrote:
On Tue, Jun 5, 2018 at 2:11 AM, Victor Stinner
wrote:
[..]
For me, it's fine to catch any exception using "except:" if the block
contains "raise", typical pattern to cleanup a resource in case of
error. Otherwise, there is a risk of leaking open file or not
flushing
data on disk, for example.
Pardon the dumb question, but why is try/finally unsuitable?
Because try..finally isn't equivalent to try..except? Perhaps you
should look at the actual code:
https://github.com/python/cpython/blob/b609e687a076d77bdd687f5e4def85e29a044bfc/Lib/asyncio/base_events.py#L1117-L1123
In this particular code, it looks like just KeyboardInterrupt needs to
be handled in addition to Exception -- and even that's not certain 'cuz
KeyboardInterrupt is an abnormal termination and specifically designed
to not be messed with by the code ("The exception inherits from
|BaseException|
so as to not be accidentally caught by code that catches |Exception|
and thus prevent the interpreter from exiting."). It only makes sense to
catch it in REPL interfaces where the user clearly wants to terminale
the current command rather than the entire program.
If e.g. a warning is upgraded to exception, this means that some code is
broken from user's POV, but not from Python team's POV, so we can't
really be sure if we can handle this situation gracefully: our cleanup
code can fail just as well!
Oh. Duh. Yep, it was a dumb question. Sorry! The transport should ONLY
be closed on error.
I smell a big, big design violation here.
The whole point of Exception vs BaseException is that anything not
Exception is "not an error", has a completely different effect on the
program than an error, and thus is to be dealt with completely
differently. For example, warnings do not disrupt the control flow,
and GeneratorExit is normally handled by the `for` loop machinery.
That's the whole point why except: is strongly discouraged.
Be _very_ careful because when a system has matured, the risk of
making bad to disastrous design decisions skyrockets (because "the big
picture" grows ever larger, and it's ever more difficult to account
for all of it).
The best solution I know of is an independent sanity-check against the
project's core design principles: focus solely on them and say if the
suggestion is in harmony with the existing big picture. This prevents
the project from falling victim to
https://en.wikipedia.org/wiki/Design_by_committee in the long run.
This is easier to do for someone not intimately involved with the
change and the affected area 'cuz they are less biased in favor of the
change and less distracted by minute details.
Someone may take up this role to "provide a unified vision" (to reduce
the load on a single http://meatballwiki.org/wiki/BenevolentDictator ,
different projects have tried delegates (this can run afoul of
https://en.wikipedia.org/wiki/Conway%27s_law though) and a round-robin
approach (Apache)).
The best way, however, would probably be for anyone dealing with a
design change to remember to make this check.
This is even easier in Python, 'cuz the core values are officially
formulated as Python Zen, and any module has one or two governing
principles at its core, tops, that can be extracted by skimming
through its docs.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru
--
Regards,
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
On 04.06.2018 20:11, Chris Angelico wrote: On Tue, Jun 5, 2018 at 2:57 AM, Yury Selivanov wrote: On Mon, Jun 4, 2018 at 12:50 PM Chris Angelico wrote: On Tue, Jun 5, 2018 at 2:11 AM, Victor Stinner wrote: [..] For me, it's fine to catch any exception using "except:" if the block contains "raise", typical pattern to cleanup a resource in case of error. Otherwise, there is a risk of leaking open file or not flushing data on disk, for example. Pardon the dumb question, but why is try/finally unsuitable? Because try..finally isn't equivalent to try..except? Perhaps you should look at the actual code: https://github.com/python/cpython/blob/b609e687a076d77bdd687f5e4def85e29a044bfc/Lib/asyncio/base_events.py#L1117-L1123 Oh. Duh. Yep, it was a dumb question. Sorry! The transport should ONLY be closed on error. I smell a big, big design violation here. The whole point of Exception vs BaseException is that anything not Exception is "not an error", has a completely different effect on the program than an error, and thus is to be dealt with completely differently. For example, warnings do not disrupt the control flow, and GeneratorExit is normally handled by the `for` loop machinery. That's the whole point why except: is strongly discouraged. Be _very_ careful because when a system has matured, the risk of making bad to disastrous design decisions skyrockets (because "the big picture" grows ever larger, and it's ever more difficult to account for all of it). The best solution I know of is an independent sanity-check against the project's core design principles: focus solely on them and say if the suggestion is in harmony with the existing big picture. This prevents the project from falling victim to https://en.wikipedia.org/wiki/Design_by_committee in the long run. This is easier to do for someone not intimately involved with the change and the affected area 'cuz they are less biased in favor of the change and less distracted by minute details. Someone may take up this role to "provide a unified vision" (to reduce the load on a single http://meatballwiki.org/wiki/BenevolentDictator , different projects have tried delegates (this can run afoul of https://en.wikipedia.org/wiki/Conway%27s_law though) and a round-robin approach (Apache)). The best way, however, would probably be for anyone dealing with a design change to remember to make this check. This is even easier in Python, 'cuz the core values are officially formulated as Python Zen, and any module has one or two governing principles at its core, tops, that can be extracted by skimming through its docs. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru -- Regards, Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
On Mon, Jun 4, 2018 at 3:38 PM Victor Stinner wrote: > > 2018-06-04 18:45 GMT+02:00 Guido van Rossum : > > It is currently a general convention in asyncio to only catch Exception, not > > BaseException. I consider this a flaw and we should fix it, but it's > > unfortunately not so easy -- the tests will fail if you replace all > > occurrences of Exception with BaseException, and it is not always clear > > what's the right thing to do. E.g. catching KeyboardInterrupt may actually > > make it harder to stop a runaway asyncio app. > > I recall vaguely something about loop.run_until_complete() which > didn't behave "as expected" when interrupted by CTRL+c, like the > following call to loop.run_until_complete() didn't work as expected. > But this issue has been sorted out, no? No, the issue is still there. And it's not an easy fix. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
2018-06-04 18:45 GMT+02:00 Guido van Rossum : > It is currently a general convention in asyncio to only catch Exception, not > BaseException. I consider this a flaw and we should fix it, but it's > unfortunately not so easy -- the tests will fail if you replace all > occurrences of Exception with BaseException, and it is not always clear > what's the right thing to do. E.g. catching KeyboardInterrupt may actually > make it harder to stop a runaway asyncio app. I recall vaguely something about loop.run_until_complete() which didn't behave "as expected" when interrupted by CTRL+c, like the following call to loop.run_until_complete() didn't work as expected. But this issue has been sorted out, no? I mean that maybe asyncio uses "except Exception:" for "historical reasons"? Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
On Tue, Jun 5, 2018 at 2:57 AM, Yury Selivanov wrote: > On Mon, Jun 4, 2018 at 12:50 PM Chris Angelico wrote: >> >> On Tue, Jun 5, 2018 at 2:11 AM, Victor Stinner wrote: > [..] >> > For me, it's fine to catch any exception using "except:" if the block >> > contains "raise", typical pattern to cleanup a resource in case of >> > error. Otherwise, there is a risk of leaking open file or not flushing >> > data on disk, for example. >> >> Pardon the dumb question, but why is try/finally unsuitable? > > Because try..finally isn't equivalent to try..except? Perhaps you > should look at the actual code: > https://github.com/python/cpython/blob/b609e687a076d77bdd687f5e4def85e29a044bfc/Lib/asyncio/base_events.py#L1117-L1123 Oh. Duh. Yep, it was a dumb question. Sorry! The transport should ONLY be closed on error. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
On Mon, Jun 4, 2018 at 12:50 PM Chris Angelico wrote: > > On Tue, Jun 5, 2018 at 2:11 AM, Victor Stinner wrote: [..] > > For me, it's fine to catch any exception using "except:" if the block > > contains "raise", typical pattern to cleanup a resource in case of > > error. Otherwise, there is a risk of leaking open file or not flushing > > data on disk, for example. > > Pardon the dumb question, but why is try/finally unsuitable? Because try..finally isn't equivalent to try..except? Perhaps you should look at the actual code: https://github.com/python/cpython/blob/b609e687a076d77bdd687f5e4def85e29a044bfc/Lib/asyncio/base_events.py#L1117-L1123 Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
> It is currently a general convention in asyncio to only catch Exception, not > BaseException. I consider this a flaw and we should fix it, but it's > unfortunately not so easy -- the tests will fail if you replace all > occurrences of Exception with BaseException, and it is not always clear > what's the right thing to do. E.g. catching KeyboardInterrupt may actually > make it harder to stop a runaway asyncio app. Yes. Catching BaseExceptions or KeyboardInterrupts in start_tls() would be pointless. Currently asyncio's internal state isn't properly hardened to survive a BaseException in all other places it can occur. Fixing that is one of my goals for 3.8. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
On Tue, Jun 5, 2018 at 2:11 AM, Victor Stinner wrote: > Hi, > > I just read a recent bugfix in asyncio: > > https://github.com/python/cpython/commit/9602643120a509858d0bee4215d7f150e6125468 > > + try: > + await waiter > + except Exception: > + transport.close() > + raise > > Why only catching "except Exception:"? Why not also catching > KeyboardInterrupt or MemoryError? Is it a special rule for asyncio, or > a general policy in Python stdlib? > > For me, it's fine to catch any exception using "except:" if the block > contains "raise", typical pattern to cleanup a resource in case of > error. Otherwise, there is a risk of leaking open file or not flushing > data on disk, for example. Pardon the dumb question, but why is try/finally unsuitable? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?
It is currently a general convention in asyncio to only catch Exception, not BaseException. I consider this a flaw and we should fix it, but it's unfortunately not so easy -- the tests will fail if you replace all occurrences of Exception with BaseException, and it is not always clear what's the right thing to do. E.g. catching KeyboardInterrupt may actually make it harder to stop a runaway asyncio app. We should move this discussion to the issue tracker. On Mon, Jun 4, 2018 at 9:11 AM, Victor Stinner wrote: > Hi, > > I just read a recent bugfix in asyncio: > > https://github.com/python/cpython/commit/9602643120a509858d0bee4215d7f1 > 50e6125468 > > + try: > + await waiter > + except Exception: > + transport.close() > + raise > > Why only catching "except Exception:"? Why not also catching > KeyboardInterrupt or MemoryError? Is it a special rule for asyncio, or > a general policy in Python stdlib? > > For me, it's fine to catch any exception using "except:" if the block > contains "raise", typical pattern to cleanup a resource in case of > error. Otherwise, there is a risk of leaking open file or not flushing > data on disk, for example. > > Victor > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
