Just a couple of questions to clarify - how many indexes are there for the Account entity and are all the fields single dimensional properties, or are there lists in there too? And out of interest, is it the parent of an entity group?
I would agree that a single request that involves putting an entity with 24 flat properties and simple indexes should not put you over quota, and if this is what's happening, then perhaps it should be looked at further (perhaps you could come up with a small test case). The only other question would be whether you can split the Account into multiple entities so that fields that are updated frequently are separated out and the entire entity doesn't need to be written each time. On 24/09/2008, at 5:03 PM, mitnickcbc wrote: > > Hi Michael, > > Thanks for putting these thoughts and that eBay best practice article > is awesome! It really helps me understand more about distribution > system. It also convinced me that global transaction is not that > important. And I have refined my code according to this article, but I > didn't notice big improvements. And I have analyzed my request data > carefully and I'd like to share some findings. > > I have one Account entity which has 24 fields and this one seems to be > the biggest issue. 30% of my requests have similar complexity. They do > 2 - 3 Account puts, 4 small entity (4 fields) puts, and 2 URL calls to > Facebook. All these requests are consuming 6K - 9K mcycles. By > comparing different URL calls, I notice this data: > > 1 Account put uses about 1800 mcycles. > 3 small puts plus 1 url call uses about 2000 mcycles. > 1 small put uses about 600 mcycles (estimated). > > Your average CPU turns into orange when beyond 1200 mcycles, and turns > into red when beyond around 2000 mcycles. > > I don't know what you feel, but I think this "high amount" definition > is too low. If this is not too low, that's fine to warn us. But add > quota to this will kick out a lot of applications. > > > On Sep 24, 8:45 am, Michael Hart <[EMAIL PROTECTED]> wrote: >> Hi all, >> >> Having been through a number of shared hosting situations, I'm gonna >> have to say that I disagree with most of the complaints on this >> thread. >> >> I think one of the main things not to lose sight of is the fact that >> GAE is intended to be a scalable platform, not a generic hosting >> solution. If you want or need a dedicated amount of CPU or memory, >> then GAE, at least as it stands today, is probably not what you want. >> The limits that are imposed in GAE are there to ensure that your app >> will scale. There are many apps that don't need to scale, and I would >> suggest that it's probably going to be easier for these to be >> developed on a more traditional platform. >> >> Global transactions don't scale (if eBay doesn't need them, than >> mostly likely neither will >> youhttp://www.infoq.com/articles/ebay-scalability-best-practices) >> . Consistent CPU spikes don't scale. Long request times don't scale - >> at least, not particularly well. If each of your requests are taking, >> say 10 seconds to fetch a page from another URL and you get hammered >> with a few hundred requests a second, then I don't want my app on the >> same box as yours (and no doubt the site you're fetching from won't >> be >> happy either). Now if Google introduce pricing that guarantees you a >> certain amount of memory or CPU or I/O on a box ala EC2, then that's >> fine, but until then (or if that doesn't happen), it's all about >> being >> a good cloud citizen. >> >> Scalability isn't easy. It's made easier with tools like BigTable and >> Memcache, but you still have to use them correctly and they will >> impose certain limitations on the way you develop. If you develop an >> app from scratch on EC2 or whatever VPS/dedicated hosting solution >> you >> like and you have the need to scale, then you'll need to start >> thinking about these issues - issues that GAE force you to think >> about >> upfront. Maybe Google should make it mandatory to read a Scalability >> 101 paper before signing up - at least that way it's clear that this >> platform is different to most standard platforms and thus reduce >> apple/ >> orange comparisons. >> >> Note that I'm not adverse to Google introducing something similar to >> EC2 for processing services - you could purchase a particular slice >> that your GAE app could communicate with using Protocol Buffers or >> whatever - or maybe you could run a bunch of map-reduce tasks. But >> these require dedicated resources and should be kept well away from >> the GAE serving boxes. Similarly there could be an S3-like storage >> solution, as well as a message bus ala SQS. For many large systems, >> these facilities are essential - and in time, Google may introduce >> them. >> >> But until then, Amazon's services (or anyone else's) are fine. You >> want storage? Go S3. You want to run background tasks? Go EC2 or some >> other VPS host. Your GAE app will talk to these services almost the >> same as it would have to if they were hosted at Google anyway. Sure, >> there'd be less latency and for some apps that may make all the >> difference - but for most, the issues will lie elsewhere. >> >> So basically, push through the pain, rethink your designs, and you'll >> come out with a well-engineered app that can scale like nobody's >> business. (well, nobody's business but your own multi-million dollar >> generating Web 3.5 Überapp that is) >> >> Cheers, >> >> Michael > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
