On Thu, Oct 17, 2013 at 3:51 PM, Oscar Benjamin <oscar.j.benja...@gmail.com>wrote:
> On 17 October 2013 20:01, Guido van Rossum <gu...@python.org> wrote: > > On Thu, Oct 17, 2013 at 11:55 AM, Oscar Benjamin > > <oscar.j.benja...@gmail.com> wrote: > >> > >> On 17 October 2013 19:40, Xavier Morel <python-...@masklinn.net> wrote: > >> > I think there's already a significant split between context managers > >> > which handle the lifecycle of a local resource (file, transaction) and > >> > those which purport to locally alter global-ish state (cwd, > >> > decimal.localcontext, logging.captureWarnings, redirect_stdout). > >> > > >> > And the latter worries me (much more than the very localized behavior > of > >> > suppress) because I don't see any way to implement them safely and > >> > correctly when mixing it with coroutines in today's Python (some of > them > >> > aren't even thread-safe), all of that while I expect coroutines will > see > >> > significantly more use in the very near future with yield from and > >> > tulip's promotion of coroutine-style async. > >> > >> I maybe misunderstanding how the coroutine-style async works but I > >> would have thought that it would be as simple as: don't use > >> global-state-restoring-context-managers around statements that yield > >> control (it would be simpler if there was a good term for describing > >> that kind of CM). That's simpler to implement and computationally > >> cheaper than e.g. the thread-local state used by the decimal module. > > > > Context managers that actually save and restore *global* state are > already > > not thread-safe, so concluding they are also not coroutine-safe (or > > task-safe?) seems a small step. > > > > I'd be more worried about context manager that use thread-local state -- > > there is no similar concept in Tulip. > > It's unnecessary in Tulip. The need for thread-local state in e.g. > decimal contexts is driven by the fact that multi-threaded execution > switches in an uncontrollable way. Tulip specifically makes it > possible to control the points at which a switch occurs making this > safe (even if localcontext() wasn't thread-safe): > > with decimal.localcontext() as ctx: > ctx.prec = 100 > c = a + b > # more synchronous decimal calculations > > # State is restored before allowing other code to execute > yield from save_in_database(c) > > So it's fine to use global/thread-local state modifying/restoring > context managers in Tulip as long as you don't yield control to other > code within the with block. (unless I misunderstand - I lost track of > Tulip some time ago). > You've got it exactly right. > The issue with decimal.localcontext() and yield arises when using > generators as much as coroutines e.g.: > > def exact_sum(nums): > start = Decimal(0) > with decimal.localcontext() as ctx: > ctx.traps[decimal.Inexact] = True > for num in nums: > try: > total += Decimal(num) > except decimal.Inexact: > ctx.prec *= 2 > return total > > The above is fine for computing the sum of a list of > Decimals/ints/floats. However it fails if you pass in a generator that > carelessly modifies the arithmetic context around yield calls: > > def compute(): > with decimal.localcontext() as ctx: > ctx.prec = 15 > ctx.traps[decimal.Inexact] = False > yield a + b > yield b - c > # etc. > > exact_sum(compute()) > > There needs to be a convention that either functions like exact_sum() > mustn't assume continuity of the context between iterations or a > function like compute() must restore before yielding. IMO the only > sane approach for async coroutines is to say that if you yield or > yield from then it is your responsibility to restore any temporarily > altered global/thread-local state first. > Right again. The simplest rule to remember seems to be "don't use yield or yield-from inside a with-statement". You can relax it by limiting it to context managers that manage any kind of shared resource, but that is probably already too subtle: e.g. yielding inside "with open(file) as f" seems fine, but yielding inside "with lock" is problematic, since the other side might try to acquire the same lock, and deadlock. -- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com