@functools.lru_cache on a nested function

2 points · submitted Mar 2022 by u/patrickbrianmooney

So I have, essentially, this:

import functools
def do_stuff(foo, bar):
    @functools.lru_cache(maxsize=None)
    def cmp_form(word):
        return word.strip().casefold()
    cmp_form.cache_clear()
[...]
[... main contents of do_stuff function ...]

The situation is more complex and the calculation being cached is more expensive, but that's the nutshell version. It's worth caching the result because it's expensive to calculate and the same values are used over and over in a nested loop. But multiple calls to do_stuff don't necessarily share any data and there's no point in letting the cache grow boundlessly over a long run (though there's a good argument to letting it grow boundlessly over a single call to do_stuff, because many many results might have to be cached, and this is both meaningful and helpful during that single run).

What I'm wondering is whether the call to cmp_form.cache_clear() is necessary: is the cache cleared already each time do_stuff is called?

My suspicion is yes, because I expect that def is executable code, not a compiler macro, and that the decorator will be rebound each time the outer function is run, creating a new cache.

But I'm not sure that I'm right. Can anyone confirm or deny?

9 comments · 100% upvoted · other discussions

comments

TangibleLight 2 points · Mar 2022

The cache is cleared when the garbage collector processes the cache object.

That happens at some point once there are no references to it (at function return, unless the cache object itself is returned).

You're guaranteed that it will happen, but not when. It will happen before you run out of memory, or before program exit. Usually it'll happen immediately but that's an implementation detail. For example it can be performant to not worry about memory allocation until a later point in the program; even though more memory is used overall, no expensive allocation operations occur during that period. AFAIK that level of optimization doesn't happen in CPython, but it might in PyPy or similar.

Your intuition about how @cache is "rebound" on each call is correct. For example:

def make():
    @functools.cache
    def func():
        pass

    return func

cache1 = make()
cache2 = make()

print(cache1 is cache2) # False

Technically you could create multiple caches using the same base function.

def func():
    pass

cache1 = functools.cache(func)
cache2 = functions.cache(func)

Which would be marginally more performant since each call wouldn't re-compile func. To be clear, that's not your bottleneck, so I wouldn't worry about that kind of micro-optimization. Do whatever's most readable.

permalink

patrickbrianmooney [OP] 1 points · Mar 2022

The cache is cleared when the garbage collector processes the cache object.

That happens at some point once there are no references to it (at function return, unless the cache object itself is returned).

That's really all I'm concerned about. I'm not trying to manage a tight memory ship -- I probably wouldn't be writing in Python if I were -- I just want to make sure the garbage collector is in fact getting a chance at disposing of the cache periodically when it disposes of other things. The function's called at irregular intervals over what might be a very long run, and a few million items in it for a time isn't a deal breaker. But if the outer function gets called hundreds of times over a run lasting a few days, and it contains a few billion items, that's a different story. I just wanted to check that my understanding is correct.

Thank you so much for the detailed answer!

permalink

TangibleLight 2 points · Mar 2022

You could also make use of the maxsize argument.

But either way you probably wouldn't need to call clear() unless you specifically want to invalidate the cache; for example if some state elsewhere changed and the entries no longer apply.

permalink