It's not a matter of memcached being bad, it's a matter of pulling only X items off the big list in order to perform more efficiently. Sometimes we want the entire list .. but also sometimes we just want to look at the last 10 and see if there are new items vs. pulling the entire list across the network and looking at only the top few items.
It seems since memcached (at least for text serialization) is just storing serialized bytes I could say give me X byte chunks and my app would know how to do the deserialization. At least I THINK this sounds feasible. On Jan 19, 12:20 pm, Boris Partensky <boris.parten...@gmail.com> wrote: > Can you always invalidate memcached entry when a new item is inserted into > the list? This way you can assume that what's in memcached is always good. > > Boris > > > > On Tue, Jan 19, 2010 at 9:11 AM, jim <jim.basi...@gmail.com> wrote: > > Hi: > > > I've recently started on a project that is utilizing memcached to > > buffer lists of 'long's that represent key values for larger more > > detailed objects to load if needed. This list will have a maximum of > > 10,000 items. > > > ex: > > > List of longs: > > key : actList value : maximum 10,000 longs sorted chronologically > > based on their 'detailed' object timestamp. > > > Detailed entry: > > key : act_<id> value : text serialized object representing detailed > > activity object. <id> is contained on the 'List of longs' above. > > > My question is there are times where we would like to load only the > > first block of entries from the list of longs, say 10 records. We > > would then look at those 10 records and see if they are new based on > > what a currently displayed list shows and if so grab these new entries > > without pulling all 10,000 across from cache only to utilize the > > relatively small number of new entries on the top of the list. This > > kind of goes hand in hand with the prepend (or append) operations > > where when new activities arrive we push these new activities into the > > front of the 'List of longs' and it's these new entries that clients > > may be interested in if they do not yet have them. > > > My question is, is there a way to do this? Is there a way to grab > > only X bytes from a value in memcache? I read over the protocol > > document and it doesn't appear there is. Is there any interest or > > valid use case that this seems to fill for other users? > > > An alternate solution I can see if to store the 'list of longs' as a > > flat list of keys with a naming convention, such as actList_1, > > actList_2, etc. However, this will obviously lead to an extremely > > long key name in the multi-key 'get' as well as lots of churn in > > managing these objects .. so it appears far less than ideal. However, > > we have many (200,000) lists of these 10,000 item 'List of longs' > > which leads to pulling loads of data over the wire when a large update > > occurs that needs to be communicated with cache ... it would be much > > more efficient to only go after a certain number of bytes in cache vs. > > the entire cached value. > > > Any other thoughts? Ideas? > > > Thank you. > > Jim > > -- > --Boris
