On Fri, Feb 22, 2013 at 8:58 AM, Brian Sutherland <br...@vanguardistas.net> wrote: > On Thu, Feb 21, 2013 at 08:05:54AM -0800, Simon King wrote: >> While reading the source, I noticed the documentation for >> pyramid.traversal.quote_path_segment which says: >> >> The return value for each segment passed to this >> function is cached in a module-scope dictionary for >> speed: the cached version is returned when possible >> rather than recomputing the quoted version. No cache >> emptying is ever done for the lifetime of an >> application, however. If you pass arbitrary >> user-supplied strings to this function (as opposed to >> some bounded set of values from a 'working set' known to >> your application), it may become a memory leak. >> >> >> https://github.com/Pylons/pyramid/blob/1.4/pyramid/traversal.py#L549 >> >> >> As far as I can tell, if you are generating URLs based on strings from a >> database, whether with traversal or url dispatch, that cache will end up >> containing every string from your database that ever gets used during URL >> generation. > > This is what it looks like on a server I manage: > > >>> len(pyramid.traversal._segment_cache) > 110943 > > Looks big, but the total memory used is modest, only around 19 MB > according to sys.getsizeof. You would need a pretty big database for it > to be a problem. >
I agree that for many applications this won't be a problem. The database I'm currently working with has a few million "resources", so if it ran for long enough, and enough pages were visited that most of those URLs were actually generated, I guess I'd be looking at at least an order of magnitude more memory. And I tend to use mod_wsgi with multiple daemon processes, so each process will have its own cache. In all likelihood most of those URLs will never be generated, but I think it's important to be aware of the possibility. (I'm actually updating an old TurboGears app that I currently restart every night because of memory leaks) My users are unlikely to notice the difference in page rendering time between using the cache and not using it, so I'm tempted to add a bit of code that runs on each request and clears the cache when it gets bigger than some threshold. Cheers, Simon -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To post to this group, send email to pylons-discuss@googlegroups.com. Visit this group at http://groups.google.com/group/pylons-discuss?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
