PEP 377 is a proposal to allow context manager __enter__() methods to
skip the body of the with statement by raising a specific (new) flow
control exception.
Since there is a working reference implementation now, I thought it was
time to open it up for broader discussion.
Full PEP attached, or you can find it in the usual place at
http://www.python.org/dev/peps/pep-0377
Cheers,
Nick.
P.S. I expect a rationale for the StatementSkipped value binding is
probably going to be pretty high on the list of questions that aren't
currently covered by the PEP. I hope to write more on that some time
this week.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
---------------------------------------------------------------
PEP: 377
Title: Allow __enter__() methods to skip the statement body
Version: $Revision: 70384 $
Last-Modified: $Date: 2009-03-15 22:48:49 +1000 (Sun, 15 Mar 2009) $
Author: Nick Coghlan <ncogh...@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 8-Mar-2009
Python-Version: 2.7, 3.1
Post-History: 8-Mar-2009
Abstract
========
This PEP proposes a backwards compatible mechanism that allows ``__enter__()``
methods to skip the body of the associated ``with`` statement. The lack of
this ability currently means the ``contextlib.contextmanager`` decorator
is unable to fulfil its specification of being able to turn arbitrary
code into a context manager by moving it into a generator function
with a yield in the appropriate location. One symptom of this is that
``contextlib.nested`` will currently raise ``RuntimeError`` in
situations where writing out the corresponding nested ``with``
statements would not [1].
The proposed change is to introduce a new flow control exception
``SkipStatement``, and skip the execution of the ``with``
statement body if ``__enter__()`` raises this exception.
Proposed Change
===============
The semantics of the ``with`` statement will be changed to include a
new ``try``/``except``/``else`` block around the call to ``__enter__()``.
If ``SkipStatement`` is raised by the ``__enter__()`` method, then
the main section of the ``with`` statement (now located in the ``else``
clause) will not be executed. To avoid leaving the names in any ``as``
clause unbound in this case, a new ``StatementSkipped`` singleton
(similar to the existing ``NotImplemented`` singleton) will be
assigned to all names that appear in the ``as`` clause.
The components of the ``with`` statement remain as described in PEP 343 [2]::
with EXPR as VAR:
BLOCK
After the modification, the ``with`` statement semantics would
be as follows::
mgr = (EXPR)
exit = mgr.__exit__ # Not calling it yet
try:
value = mgr.__enter__()
except SkipStatement:
VAR = StatementSkipped
# Only if "as VAR" is present and
# VAR is a single name
# If VAR is a tuple of names, then StatementSkipped
# will be assigned to each name in the tuple
else:
exc = True
try:
try:
VAR = value # Only if "as VAR" is present
BLOCK
except:
# The exceptional case is handled here
exc = False
if not exit(*sys.exc_info()):
raise
# The exception is swallowed if exit() returns true
finally:
# The normal and non-local-goto cases are handled here
if exc:
exit(None, None, None)
With the above change in place for the ``with`` statement semantics,
``contextlib.contextmanager()`` will then be modified to raise
``SkipStatement`` instead of ``RuntimeError`` when the underlying
generator doesn't yield.
Rationale for Change
====================
Currently, some apparently innocuous context managers may raise
``RuntimeError`` when executed. This occurs when the context
manager's ``__enter__()`` method encounters a situation where
the written out version of the code corresponding to the
context manager would skip the code that is now the body
of the ``with`` statement. Since the ``__enter__()`` method
has no mechanism available to signal this to the interpreter,
it is instead forced to raise an exception that not only
skips the body of the ``with`` statement, but also jumps over
all code until the nearest exception handler. This goes against
one of the design goals of the ``with`` statement, which was to
be able to factor out arbitrary common exception handling code
into a single context manager by putting into a generator
function and replacing the variant part of the code with a
``yield`` statement.
Specifically, the following examples behave differently if
``cmB().__enter__()`` raises an exception which ``cmA().__exit__()``
then handles and suppresses::
with cmA():
with cmB():
do_stuff()
# This will resume here without executing "do_stuff()"
@contextlib.contextmanager
def combined():
with cmA():
with cmB():
yield
with combined():
do_stuff()
# This will raise a RuntimeError complaining that the context
# manager's underlying generator didn't yield
with contextlib.nested(cmA(), cmB()):
do_stuff()
# This will raise the same RuntimeError as the contextmanager()
# example (unsurprising, given that the nested() implementation
# uses contextmanager())
# The following class based version shows that the issue isn't
# specific to contextlib.contextmanager() (it also shows how
# much simpler it is to write context managers as generators
# instead of as classes!)
class CM(object):
def __init__(self):
self.cmA = None
self.cmB = None
def __enter__(self):
if self.cmA is not None:
raise RuntimeError("Can't re-use this CM")
self.cmA = cmA()
self.cmA.__enter__()
try:
self.cmB = cmB()
self.cmB.__enter__()
except:
self.cmA.__exit__(*sys.exc_info())
# Can't suppress in __enter__(), so must raise
raise
def __exit__(self, *args):
suppress = False
try:
if self.cmB is not None:
suppress = self.cmB.__exit__(*args)
except:
suppress = self.cmA.__exit__(*sys.exc_info()):
if not suppress:
# Exception has changed, so reraise explicitly
raise
else:
if suppress:
# cmB already suppressed the exception,
# so don't pass it to cmA
suppress = self.cmA.__exit__(None, None, None):
else:
suppress = self.cmA.__exit__(*args):
return suppress
With the proposed semantic change in place, the contextlib based examples
above would then "just work", but the class based version would need
a small adjustment to take advantage of the new semantics::
class CM(object):
def __init__(self):
self.cmA = None
self.cmB = None
def __enter__(self):
if self.cmA is not None:
raise RuntimeError("Can't re-use this CM")
self.cmA = cmA()
self.cmA.__enter__()
try:
self.cmB = cmB()
self.cmB.__enter__()
except:
if self.cmA.__exit__(*sys.exc_info()):
# Suppress the exception, but don't run
# the body of the with statement either
raise SkipStatement
raise
def __exit__(self, *args):
suppress = False
try:
if self.cmB is not None:
suppress = self.cmB.__exit__(*args)
except:
suppress = self.cmA.__exit__(*sys.exc_info()):
if not suppress:
# Exception has changed, so reraise explicitly
raise
else:
if suppress:
# cmB already suppressed the exception,
# so don't pass it to cmA
suppress = self.cmA.__exit__(None, None, None):
else:
suppress = self.cmA.__exit__(*args):
return suppress
There is currently a tentative suggestion [3] to add import-style syntax to
the ``with`` statement to allow multiple context managers to be included in
a single ``with`` statement without needing to use ``contextlib.nested``. In
that case the compiler has the option of simply emitting multiple ``with``
statements at the AST level, thus allowing the semantics of actual nested
``with`` statements to be reproduced accurately. However, such a change
would highlight rather than alleviate the problem the current PEP aims to
address: it would not be possible to use ``contextlib.contextmanager`` to
reliably factor out such ``with`` statements, as they would exhibit exactly
the same semantic differences as are seen with the ``combined()`` context
manager in the above example.
Performance Impact
==================
Implementing the new semantics makes it necessary to store the references
to the ``__enter__`` and ``__exit__`` methods in temporary variables instead
of on the stack. This results in a slight regression in ``with`` statement
speed relative to Python 2.6/3.1. However, implementing a custom
``SETUP_WITH`` opcode would negate any differences between the two
approaches (as well as dramatically improving speed by eliminating more
than a dozen unnecessary trips around the eval loop).
Reference Implementation
========================
Patch attached to Issue 5251 [1]. That patch uses only existing opcodes
(i.e. no ``SETUP_WITH``).
Acknowledgements
================
James William Pye both raised the issue and suggested the basic outline of
the solution described in this PEP.
References
==========
.. [1] Issue 5251: contextlib.nested inconsistent with nested with statements
(http://bugs.python.org/issue5251)
.. [2] PEP 343: The "with" Statement
(http://www.python.org/dev/peps/pep-0343/)
.. [3] Import-style syntax to reduce indentation of nested with statements
(http://mail.python.org/pipermail/python-ideas/2009-March/003188.html)
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
