Hi Bertrand, One reason I see for the locmem cache to pickles the data is stores would be to prevent mutable objects from being shared between threads.
e.g. Thread one stores [1,2,3] as 'foo', keeps a reference to it to alter it. Thread two gets 'foo' and alters it. And to prevent memory leaks if a thread specific object is stored. e.g. Store [request, database_connection] as 'foo-%d'. In this case the pickle roundtripping asserts the Locmem backend behaves like others in regard to deferencing. Simon Le dimanche 25 octobre 2015 11:50:12 UTC-4, Bertrand Bordage a écrit : > > Hi everyone, > > Is there a valid reason why locmem is pickling data? > From what I understand, locmem is doing that just because its behaviour > was copied from other caches. > But since locmem is basically just a dictionary stored in the current > Python process without persistence, there is no need to serialize data. > By skipping the pickling step, we avoid creating a new object and we avoid > the CPU time needed for this serialization. So that’s better in terms of > memory and CPU. > > I created a gist containing the unpickling version of LocMemCache: > https://gist.github.com/BertrandBordage/7f33b3ccf2686e8da9ed > I tested it on the test suite of django-cachalot, everything works fine > including thread-safe tests. It also scores drastically better when caching > medium/large objects: > > Using the current locmem backend: > In [1]: %timeit cache.set('something', list(range(10000))) > 1000 loops, best of 3: 429 µs per loop > > In [2]: %timeit cache.get('something') > 1000 loops, best of 3: 377 µs per loop > > Using the unpickling version of locmem: > In [1]: %timeit cache.set('something', list(range(10000))) > 10000 loops, best of 3: 179 µs per loop > > In [2]: %timeit cache.get('something') > 10000 loops, best of 3: 23.8 µs per loop > > Regards, > Bertrand > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5d8ea38a-2d06-48b3-9247-a81ef8d8dea7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
