Thanks, hadn't thought about the zipping yet. Any recommendations? The query cursor would work fine if there was no sorting involved - but since this is not the only list to be sorted, and since each sort requires a separate index, the number of indexes would get out of control quickly. We're already at some 100 indexes, and I'd rather not create new ones unless really needed.
Cheers, Per On Feb 7, 11:36 am, Sameer Lodha <sams...@gmail.com> wrote: > One tip is to "Zip" data before you put it into memcache. That way you > can easily store 4x to 10x more information. But again there would be > limit to how much you can save. > > Beyond that, I think the better approach would be to use Query Cursor > to get just the required data (to be displayed) at runtime. > > Regards, > Sam > > > > > > > > On Tue, Feb 7, 2012 at 3:39 PM, Per <per.fragem...@gmail.com> wrote: > > > Hi all, > > > say I have a few hundred users in one organisation, and I want to > > display them in a list. I want to be able to sort them by user name, > > or manager, or some other criteria. Right now I load them from the > > database all in one go, sort them, store them in memcache, and then > > during pagination, I load them from that memcache each time the user > > clicks forth or back. When the user changes the ordering, I load the > > list from memcache, re-order, and store it back. > > > This works fine for 1000 users (each of them has some 15 fields). Now > > if I had a few thousand users, I'd have to make sure the user objects > > are not too big, or the list will be more than 1 megabyte. I can > > denormalise into a slimmer user object, which makes database access > > faster too. But what if I want to display 10.000 or more users -- Each > > user object can't have more than 100 bytes then, because of the > > memcache limit of 1MB. But that's not even enough for even the most > > basic fields. > > > Is the solution to split the list into several memcache blocks? If so, > > are there any libraries or tools that support this? Or am I barking up > > the wrong tree, should I deal with this in an entirely different way? > > I did some googling, but the main questions (and answer) was always > > around speeding up the initial database call. This is working fine for > > us, and it doesn't matter if it takes a few seconds either. > > > Feedback from people who have dealt with similar issues would be > > greatly appreciated! :) > > > Cheers, > > Per > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
