Hi, On Thu, Feb 18, 2010 at 03:17:32PM -0500, Alex Gaynor wrote: > On Thu, Feb 18, 2010 at 3:12 PM, Jeremy Dunck <jdu...@gmail.com> wrote: > > On Thu, Feb 18, 2010 at 1:34 PM, ramu...@gmail.com <ramu...@gmail.com> > > wrote: > >> Does anyone know why django can't keep a data bigger, than 1 mb in > >> memcached ? > >> > >> This is no big deal to cut big data to 1mb pieces before setting and > >> merge this pieces after getting from memcached. And this algorithm can > >> work transparently for users. > > > > I think Ramusus is asking for the low-level caching framework to split > > serialized data into 1MB chunks before storing in a bunch of keys if > > the data is larger than 1MB. > > > > He's not asking why there's a limit, he's asking for Django to work > > around the limit. > > > > I think the general reasoning for a 1MB limit is that it places bounds > > on access/storage times for a given key, and avoids an antipattern of > > storing really large objects in cache. I've wished for special-case > > side-stepping of the 1MB limit before myself, but I'd hate for it to > > be the default. > > I don't see how that would be possible, sure storing it would be easy, > but how do you retrieve it? Either you have to always try the > "part-2" key (unreasonable overhead fro 99% of the use cases) or you > have to store a tuple of (blocks, data) in the first slot, that's not > a huge runtime overhead, but it feels like a large code-complexity > overhead for a very rare usecase (I had no idea memcached even had > these limits, and I use it pretty extensively).
Such operations would also not be atomic. Concurrent attempts to update cache would be problematic. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org
signature.asc
Description: Digital signature
