Before reading or responding to anything else.. let me direct you to this guys experience.. part of which was his revelation with batch puts:
http://www.gregtracy.com/adventures-in-performance-tuning-on-google-ap <http://www.gregtracy.com/adventures-in-performance-tuning-on-google-ap>He was doing big time putting.. so he saw a more dramatic drop in cpu time (than you might see), but the lesson still applies. Here is the choice quote about batch putting from his writeup: "This had a dramatic impact on the overall performance. The time it took to loop through the route file went from 12,000 megacycles to 2,000 megacycles (!)." Now, I shall read the rest of your reply. On Wed, Sep 29, 2010 at 6:43 PM, Michael <michaelpb...@gmail.com> wrote: > For the request you describe here, what modules are you importing? > *Nothing extra that I know of besides what is necessary for Java/JDO to > work with App Engine. What's the best way to figure this out?* > > Are the Gets by keyname or are they filtered queries? > *The gets are done with keys.* > > Are you doing the puts all at once (for Python, that would mean putting all > the entities into a entityList and calling db.put(entityList)) or are you > doing separate puts for each entity? > *The puts are done separately, but I was going to look into batching them > later today. Doesn't this type of batching reduce realtime, and not cpu/api > time?* > > Have you looked into using memcache for the gets? (if you can't use > memcache, why not? etc) > *We haven't looked at memcache yet. What's the overhead like if you don't > hit the cache often? For our game, each user will have their own list of > games that they will be making moves in. So there isn't a lot of overlap in > data between users or between games, but they might access a couple of the > same objects in a matter of a few minutes. Does that seem like a data > access pattern that could benefit from memcache?* > > Are the CPU Times you are reporting the average for a cold instance of the > request or a hot instance (If you look in the logs for the request, it > should tell you "This request caused a new process to be started.." if it is > a cold instance)? > *Hot instance.* > > How many indexes are defined on the Model types that are being put to the > datastore? > *There are 3 different types of objects being saved. Two of them have 1 > index each and the other has 3 indices. I don't know if it really needs all > those anymore since I think some of them were auto-generated during > development.* > > Is there any way around using a Transaction? (not really worth exploring > unless you've tested the process without a transaction.. and it performs > drastically better) > *We can't really get rid of the transaction. App Engine threw an > exception once while writing the objects and one was saved while one wasn't, > and that screwed things up. So the transaction protects against that.* > > Have you looked into manually logging cpu time in the code? (e.g. > recording cpu time before the gets.. and then after.. to see what CPU they > use.. then doing the same for the Puts) > *We've mostly been relying on the timings that are already in the log and > in AppStats. But we can manually log the times too. What is the difference > between what that would tell us and the stats we're already looking at?** > * > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
