Function decorator that caches function results

After working through a fair number of the challenges at www.mathschallenge.net, I noticed that some long-running functions can be helped *a lot* by caching their function results and retrieving from cache instead of calculating again. This means that often I can use a natural recursive impleme

Re: Function decorator that caches function results

Lasse Vågsæther Karlsen wrote: > However, this: > > for index in range(1, 101): > print index, fibonacci(idx = index) > > this one uses the kwargs list of arguments, and I'm not sure how to > change my function to take that into account. > > Does anyone have any clues as to how to do that?

Re: Function decorator that caches function results

On Sat, Oct 08, 2005 at 01:53:28PM +, Duncan Booth wrote: > Unless the results stored in the cache are very large data structures, I > would suggest that you simply store (args,kwargs) as the cache key and > accept the hit that sometime you'll cache the same call multiple times. ... except t

Re: Function decorator that caches function results

What about not storing args at all? Something like this: def cache_function(func, args_list): cache = {} def cached_result(*args, **kwargs): kwargs.update(dict(zip(args_list, args))) if kwargs in cache: return cache[kwargs] result = func(**kwargs)

Re: Function decorator that caches function results

[EMAIL PROTECTED] wrote: > On Sat, Oct 08, 2005 at 01:53:28PM +, Duncan Booth wrote: > >>Unless the results stored in the cache are very large data structures, I >>would suggest that you simply store (args,kwargs) as the cache key and >>accept the hit that sometime you'll cache the same call

Re: Function decorator that caches function results

Sam Pointon wrote: > What about not storing args at all? Something like this: > > def cache_function(func, args_list): > cache = {} > def cached_result(*args, **kwargs): > kwargs.update(dict(zip(args_list, args))) > if kwargs in cache: > return cache[kwargs] >

Re: Function decorator that caches function results

Lasse Vågsæther Karlsen wrote: > Yeah, but as far as I can see it, this one too fails to recognize > situations where the function is called twice with essentially the same > values, except that in one call it uses named arguments: > > k1 = fibonacci(100) > k2 = fibonacci(idx = 100) > > this is es

Re: Function decorator that caches function results

Lasse Vågsæther Karlsen wrote: > Sam Pointon wrote: > >> What about not storing args at all? Something like this: Ok, here's my updated version: class cache(object): def __init__(self, timeout=0): self.timeout = timeout self.cache = {} def __call__(self, fn):

Re: Function decorator that caches function results

Fredrik Lundh wrote: >>k1 = fibonacci(100) >>k2 = fibonacci(idx = 100) > whoever writes code like that deserves to be punished. > > I'd say this thread points to a misunderstanding of what keyword arguments > are, and how they should be used. the basic rule is that you shouldn't mix > and > ma

Re: Function decorator that caches function results

"Lasse Vågsæther Karlsen" <[EMAIL PROTECTED]> wrote: > [snip] > > Ok, so I thought, how about creating a decorator that caches the > function results and retrieves them from cache if possible, otherwise it > calls the function and store the value in the cache for the next invokation. > > [snip] C

Re: Function decorator that caches function results

George Sakkis wrote: > Cool, you re-invented the memoization pattern: > http://en.wikipedia.org/wiki/Memoization > http://aspn.activestate.com/ASPN/search?query=memoize&x=0&y=0§ion=PYTHONCKBK&type=Subsection > > Yes, it's kinda discouraging that most interesting ideas have already been > conceiv

Re: Function decorator that caches function results

On Sat, 08 Oct 2005 15:20:12 +0200, Lasse Vågsæther Karlsen wrote: > Ok, so I thought, how about creating a decorator that caches the > function results and retrieves them from cache if possible, otherwise it > calls the function and store the value in the cache for the next invokation. > > Thi

Re: Function decorator that caches function results

Steven D'Aprano <[EMAIL PROTECTED]> writes: > > def cache_function(fn): > > cache = {} > > def cached_result(*args, **kwargs): > > if args in cache: > > return cache[args] > > result = fn(*args, **kwargs) > > cache[args] = result > > return

Re: Function decorator that caches function results

>>def cache_function(fn): >> cache = {} >> def cached_result(*args, **kwargs): >> if args in cache: >> return cache[args] >> result = fn(*args, **kwargs) >> cache[args] = result >> return result >> return cached_result > > > I'm curious... w

Re: Function decorator that caches function results

On Sun, 09 Oct 2005 02:49:38 -0700, Paul Rubin wrote: >> I'm curious... where does cache live after you use cache_function to >> memoize some function? It doesn't appear to be an attribute of the newly >> memoized function, nor does it look like a global variable. > > It's in the closure returned

Re: Function decorator that caches function results

Steven D'Aprano wrote: > I notice that type(some_closure) and type(ordinary_function) both return > the same result, . I also notice that both the closure > and ordinary functions have an attribute "func_closure". Is there a way to > tell which is a closure and which is not? In my tests, the ordin

Re: Function decorator that caches function results

On Sun, 09 Oct 2005 13:43:38 +0200, Fredrik Lundh wrote: > Steven D'Aprano wrote: > >> I notice that type(some_closure) and type(ordinary_function) both return >> the same result, . I also notice that both the closure >> and ordinary functions have an attribute "func_closure". Is there a way to >

Re: Function decorator that caches function results

> If you create a closure, using a memoization technique as per the original > post, and then call type() on that closure, Python reports . Because it is. The closure is only sort of an extension to the locals() available to that function. Not more, not less. > > If you use dir() on that closur

Re: Function decorator that caches function results

Steven D'Aprano wrote: > Yes I did. Did you read my post? given that the wikipedia page says "a closure is an abstraction representing a function, plus the lexical environment (see static scoping) in which the function was created." and you're still focussing on type(function), it s

Re: Function decorator that caches function results

Steven D'Aprano wrote: > On Sat, 08 Oct 2005 15:20:12 +0200, Lasse Vågsæther Karlsen wrote: > > >>Ok, so I thought, how about creating a decorator that caches the >>function results and retrieves them from cache if possible, otherwise it >>calls the function and store the value in the cache for

Re: Function decorator that caches function results

On Sun, 09 Oct 2005 14:27:32 +0200, Fredrik Lundh wrote: > Steven D'Aprano wrote: > >> Yes I did. Did you read my post? > > given that the wikipedia page says > > "a closure is an abstraction representing a function, plus the > lexical environment (see static scoping) in which the funct

Re: Function decorator that caches function results

Ron Adam wrote: > In effect, 'cache' and 'fn' are replaced by the objects they reference > before the cached_result function is returned. not really; accesses to "free variables" always go via special cell objects (rather than direct object references), and the compiler uses special byte codes fo

Re: Function decorator that caches function results

On Sun, 09 Oct 2005 13:43:38 +0200, Fredrik Lundh wrote: >> If I wanted to inspect the value of cache, where would I find it? > > nowhere. at least no officially; see "Rebinding names in enclosing > scopes" in the design document for a discussion: > > http://www.python.org/peps/pep-0227.htm

Re: Function decorator that caches function results

Steven D'Aprano wrote: > > let's take it again, with emphasis on some important words: > > > > "a closure is an ABSTRACTION representing a function, PLUS the > > lexical ENVIRONMENT (see static scoping) in which the function > > was created." > > > >> If you create a closure, using a m

Re: Function decorator that caches function results

> Clearly there is no DISTINCT closure object. If there were, I wouldn't > need to ask how one can tell them apart, because type() would just report > that one was a function and one was a closure. I don't have a problem with > that. But read on... > > >>function objects always con- >>tain all th

Re: Function decorator that caches function results

Steven D'Aprano wrote: > "Each def or lambda expression that is executed will create a closure if > the body of the function or any contained function has free variables." > > Presumably that means that if there are no free variables, no closure > is created. you're quoting selectively; the word

Re: Function decorator that caches function results

Fredrik Lundh wrote: > Ron Adam wrote: > > >>In effect, 'cache' and 'fn' are replaced by the objects they reference >>before the cached_result function is returned. > > > not really; accesses to "free variables" always go via special cell objects > (rather than direct object references), and th

Re: Function decorator that caches function results

On Sun, 09 Oct 2005 18:00:13 +0200, Fredrik Lundh wrote: > Steven D'Aprano wrote: > >> "Each def or lambda expression that is executed will create a closure if >> the body of the function or any contained function has free variables." >> >> Presumably that means that if there are no free variable

Re: Function decorator that caches function results

On Sun, 09 Oct 2005 17:55:16 +0200, Diez B. Roggisch wrote: >> If what you say is true, then all functions are closures, and closure is >> just a synonym for function, and there is no difference between a function >> and a closure. Then why bother to create the term? Clearly, whoever >> invented t

Re: Function decorator that caches function results

On Sun, 09 Oct 2005 17:39:23 +0200, Fredrik Lundh wrote: > only if you're obsessed with CPython implementation details. No. I'm obsessed with finding out what closures are, since nobody seems to have a good definition of them! However, I have learnt some things: closures are something which func

Re: Function decorator that caches function results

> Steven D'Aprano <[EMAIL PROTECTED]> (SD) wrote: >SD> [penny drops] Now we're getting somewhere... a closure is something >SD> _added_ to a function. So we should talk about functions-without-closures >SD> and functions-with-closures. Well, I have never heard that definition. For more than h

Re: Function decorator that caches function results

Steven D'Aprano <[EMAIL PROTECTED]> writes: > No. I'm obsessed with finding out what closures are, since nobody seems to > have a good definition of them! Why don't you read SICP: http://mitpress.mit.edu/sicp/full-text/book/book.html You will be a much wiser person for having done so. > Howe

Re: Function decorator that caches function results

Steven D'Aprano <[EMAIL PROTECTED]> writes: > Is it correct to say that Python *always* creates a closure whenever a def > or lambda is executed? Or is it only for *certain* defs/lambdas? The word closure is being used two different ways. First of all the computer-science term "closure" which mea

Re: Function decorator that caches function results

> > [penny drops] Now we're getting somewhere... a closure is something > _added_ to a function. So we should talk about functions-without-closures > and functions-with-closures. Yes. > > >>Speaking in >>terms of "is a" could be seen as some inheritance relation. > > > [penny rises again]

Re: Function decorator that caches function results

On Mon, 10 Oct 2005, Steven D'Aprano wrote: > On Sun, 09 Oct 2005 17:39:23 +0200, Fredrik Lundh wrote: > >> only if you're obsessed with CPython implementation details. > > No. I'm obsessed with finding out what closures are, since nobody seems > to have a good definition of them! On the contrar

Re: Function decorator that caches function results

Tom Anderson <[EMAIL PROTECTED]> writes: > Okay, a crack at a definition: a closure is a function in which some > of the variable names refer to variables outside the function. That's misleading, I'd say a closure is a combination of a function (executable code) and a lexical environment (the valu

Re: Function decorator that caches function results

> Paul Rubin (PR) wrote: >PR> Tom Anderson <[EMAIL PROTECTED]> writes: >PR> That's misleading, I'd say a closure is a combination of a function >PR> (executable code) and a lexical environment [snip] >PR> This is all described in SICP (mitpress.mit.edu/sicp). Wh

Re: Function decorator that caches function results

Tom Anderson wrote: > Okay, a crack at a definition: a closure is a function in which some of > the variable names refer to variables outside the function. And i don't > mean global variables - i mean somebody else's locals; call them 'remote > variables'. in Python, the global variables are some

Re: Function decorator that caches function results

On Tue, 10 Oct 2005, it was written: > Tom Anderson <[EMAIL PROTECTED]> writes: > >> Okay, a crack at a definition: a closure is a function in which some of >> the variable names refer to variables outside the function. > > That's misleading, You're right. I don't think it's wrong, but it depend