Wow, this is a great link. Posterous is really, really slow for me, though.
On Thu, Aug 12, 2010 at 1:34 PM, Robert Kluin <robert.kl...@gmail.com>wrote: > Hi Greg. Nice write-up. > > > Robert > > > > > > > On Thu, Aug 12, 2010 at 8:25 AM, Greg Tracy <gtr...@gmail.com> wrote: > > > > Circling back on this thread to report on a couple of outcomes. > > > > First, from a numbers standpoint... I saw massive improvement when I > > started doing batch puts. > > - On average, a batch put of 472 relatively small entities takes 2,200 > > megacycles > > - On average, a batch put of 363 entities takes 1,900 megacycles > > > > Second, total CPU cycles for this job (of parsing a text file, > > creating entities, and storing them) is a fraction of the API cycle > > time. 3,000 v. 98,000 megacycles > > > > Also, I did a write-up on my experience with this analysis process. If > > you're interested, check it out and share your thoughts and > > experiences. > > > > http://www.gregtracy.com/adventures-in-performance-tuning-on-google-ap > > > > Now... how do I manage the API time?! > > > > > > On Jul 21, 2:26 am, Robert Kluin <robert.kl...@gmail.com> wrote: > >> Hey Greg, > >> Yeah puts are relatively expensive. Doing a batch put is about the > only way to reduceCPUtime, but of course that does basically nothing > forAPItime. > >> > >> I always like to see benchmarks and related discussions. Look forward > to your results. > >> > >> Robert > >> > >> On Jul 20, 2010, at 13:51, Greg Tracy <gtr...@gmail.com> wrote: > >> > >> > >> > >> > >> > >> > Thanks for the input, Robert. > >> > >> > After more performance tuning, I've been able to eliminate the > >> > DeadlineExceeded errors. But I still have a hard time scaling the app > >> > since the put calls are so expensive. Over thousands of request > >> > handler calls, I quickly eat upCPUquota (free and paid). > >> > >> > Going back to the original question, I'm still unsure how the quota > >> > math is working. I'm in the process of creating an-to-end performance > >> > model of these calls and will post the results. > >> > >> > On Jul 15, 4:40 pm, Robert Kluin <robert.kl...@gmail.com> wrote: > >> >> Greg, > >> >> If you are having issues with long write times, there are two > common > >> >> options you may consider: smaller batches and transactional tasks. > >> > >> >> I have some code that does batch processing, I set it up to search > >> >> for the maximum batch size that is currently succeeding. Basically I > >> >> slowly increase my batch size until I am getting deadline exceeded > >> >> warnings then I drop back down adjust my 'growth factor' and repeat > >> >> the processes. This seems to keep my batches running very close to > >> >> the upper limit without too many deadline exceeded exceptions. > >> > >> >> The other approach I have used it to offload some of the writes to > >> >> other background tasks. I write a group of entities, then send > >> >> another handler another block of entities to write. This has been > >> >> working well for me too. > >> > >> >> Robert > >> > >> >> On Thu, Jul 15, 2010 at 5:22 PM, Greg Tracy <gtr...@gmail.com> > wrote: > >> > >> >>> I just started using AppStats for the first time - very nice > package! > >> >>> However, it didn't reveal any glaring problems. But it did help > >> >>> validate my belief that I was using the memcache effectively. > >> > >> >>> I've been using the 'quota' package to find costly operations, and I > >> >>> did discover a bug in my measurement. I was "starting the clock" in > >> >>> the wrong spot so I have found some new areas to optimize. Will work > >> >>> on that now... but I'm still not confident I can compensate for the > >> >>> long write times. > >> > >> >>> Thanks! > >> > >> >>> On Jul 14, 10:46 pm, Robert Kluin <robert.kl...@gmail.com> wrote: > >> >>>> If you have not already, take a look at AppStats. Perhaps you can > >> >>>> find a way to improve performance. You may also want to read > through > >> >>>> some of the datastore articles in the docs to be sure you are > >> >>>> minimizing resource usage. > >> > >> >>>> Robert > >> > >> >>>> On Wed, Jul 14, 2010 at 7:24 PM, Greg Tracy <gtr...@gmail.com> > wrote: > >> > >> >>>>> Not for me... I set a budget and immediately started paying. > >> > >> >>>>> Not thrilled about this and may need to start a new thread asking > for > >> >>>>> advice on datastore contention... > >> > >> >>>>> Thanks. > >> > >> >>>>> On Jul 14, 5:57 pm, Nate Bauernfeind <nate.bauernfe...@gmail.com> > >> >>>>> wrote: > >> >>>>>> I have been. > >> > >> >>>>>> On Wed, Jul 14, 2010 at 5:37 PM, Greg Tracy <gtr...@gmail.com> > wrote: > >> >>>>>>> Does this imply that I can go over theCPUTime quota and not pay > for > >> >>>>>>> it? > >> > >> >>>>>>> On Jul 14, 5:26 pm, Nate Bauernfeind < > nate.bauernfe...@gmail.com> > >> >>>>>>> wrote: > >> >>>>>>>> The DataStoreCPUtime is included in totalCPUtime. To take > advantage > >> >>>>>>> of > >> >>>>>>>> "free datastorecputime" you need to increase yourcpu-usage > quota. When > >> >>>>>>> I > >> >>>>>>>> loaded a bunch of initial test data into the datastore I got > tons of > >> >>>>>>>> deadline exceeded errors, though I assumed it was cause I was > trying to > >> >>>>>>>> stuff a lot of data in there all at once. > >> > >> >>>>>>>> Nate > >> > >> >>>>>>>> On Wed, Jul 14, 2010 at 5:24 PM, Greg Tracy <gtr...@gmail.com> > wrote: > >> > >> >>>>>>>>> I've been adding some new datastore-intensive features to an > app and > >> >>>>>>>>> am blowing through the "CPUTime" quota. While my understanding > is > >> >>>>>>>>> that the "DatastoreCPUTime" quotas have been lifted while the > App > >> >>>>>>>>> Engine team continues to work on the performance, I can't > figure out > >> >>>>>>>>> why these are mutually exclusive metrics in my app. > >> > >> >>>>>>>>> When I measure thecpucyclesbeing consumed in the new features, > it > >> >>>>>>>>> is almost exclusively in the db.put() call. In fact, 50% of > the time, > >> >>>>>>>>> I'm getting DeadlineExceeded errors before the put call even > returns. > >> > >> >>>>>>>>> Is the "DatastoreCPUTime" also counted in the "CPUTime"? If > so, are > >> >>>>>>>>> there plans to extend the quota limits there as well? > >> > >> >>>>>>>>> I'm interested in finding some resources that track > performance > >> >>>>>>>>> metrics on App Engine. I don't know what to expect in terms of > >> >>>>>>>>> performance when I store, for example, 400 entities in the > data store. > >> >>>>>>>>> Are there resources where folks are contributing their metrics > for > >> >>>>>>>>> others to see? > >> > >> >>>>>>>>> Thanks. > >> > >> >>>>>>>>> -- > >> >>>>>>>>> You received this message because you are subscribed to the > Google > >> >>>>>>> Groups > >> >>>>>>>>> "Google App Engine" group. > >> >>>>>>>>> To post to this group, send email to > google-appengine@googlegroups.com > >> >>>>>>> . > >> >>>>>>>>> To unsubscribe from this group, send email to > >> >>>>>>>>> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com><google-appengine%2Bunsubscrib > e...@googlegroups.com><google-appengine%2Bunsubscrib > >> >>>>>>> e...@googlegroups.com> > >> >>>>>>>>> . > >> >>>>>>>>> For more options, visit this group at > >> >>>>>>>>>http://groups.google.com/group/google-appengine?hl=en. > >> > >> >>>>>>> -- > >> >>>>>>> You received this message because you are subscribed to the > Google Groups > >> >>>>>>> "Google App Engine" group. > >> >>>>>>> To post to this group, send email to > google-appeng...@googlegroups.com. > >> >>>>>>> To unsubscribe from this group, send email to > >> >>>>>>> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com><google-appengine%2Bunsubscrib > e...@googlegroups.com> > >> >>>>>>> . > >> >>>>>>> For more options, visit this group at > >> >>>>>>>http://groups.google.com/group/google-appengine?hl=en. > >> > >> >>>>> -- > >> >>>>> You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > >> >>>>> To post to this group, send email to > google-appeng...@googlegroups.com. > >> >>>>> To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > >> >>>>> For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en. > >> > >> >>> -- > >> >>> You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > >> >>> To post to this group, send email to > google-appeng...@googlegroups.com. > >> >>> To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > >> >>> For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en. > >> > >> > -- > >> > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > >> > To post to this group, send email to > google-appeng...@googlegroups.com. > >> > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > >> > For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en. > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
