On Sun, Jun 4, 2017 at 12:23 AM, Lucas Wiman <lucas.wi...@gmail.com> wrote:
> I agree that the stated use cases are better handled with ExitStack. One
> area where `defer` might be useful is in lazy-evaluating global constants.
> For example in a genomics library used at my work, one module involves
> compiling a large number of regular expressions, and setting them as global
> contants in the module, like:
>
> FOO1_re = re.compile(r'...')
> FOO_TO_BAR_re = {foo: complicated_computation_of_regex(foo) for foo in
> LONG_LIST_OF_THINGS}
> ...
>
> This utility module is imported in a lot of places in the codebase, which
> meant that importing almost anything from our codebase involved precompiling
> all these regular expressions, which took around 500ms to to run the
> anything (the test runner, manually testing code in the shell, etc.) It
> would be ideal to only do these computations if/when they are needed. This
> is a more general issue than this specific example, e.g. for libraries which
> parse large data sources like pycountry (see my PR for one possible ugly
> solution using proxy objects; the author instead went with the simpler, less
> general solution of manually deciding when the data is needed). See also
> django.utils.functional.lazy, which is used extensively in the framework.
>
> A statement like: `defer FOO = lengthy_computation_of_foo()` which deferred
> the lengthy computation until it is used for something would be useful to
> allow easily fixing these issues without writing ugly hacks like proxy
> objects or refactoring code into high-overhead cached properties or the
> like.
I think in general I'd recommend making the API for accessing these
things be a function call interface, so that it's obvious to the
caller that some expensive computation might be going on. But if
you're stuck with an attribute-lookup based interface, then you can
use a __getattr__ hook to compute them the first time they're
accessed:
class LazyConstants:
def __getattr__(self, name):
value = compute_value_for(name)
setattr(self, name, value)
return value
__getattr__ is only called as a fallback, so by setting the computed
value on the object we make any future attribute lookups just as cheap
as they would be otherwise.
You can get this behavior onto a module object by doing
"sys.modules[__name__] = Constants()" inside the module body, or by
using a <del>hack</del> elegant bit of code like
https://github.com/njsmith/metamodule/ (mostly the latter would only
be preferred if you have a bunch of other attributes exported from
this same module and trying to move all of them onto the LazyConstants
object would be difficult).
-n
--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
