[google-appengine] Appengine scalability ~200 get/sec and about 50 put/sec
Hi, I am estimating whether AppEngine is scalable for ~200 get/sec and about 50 put/sec RPC calls. I have ran the AppStat on my current app, which have currently very low traffic about ~20get and ~4put/sec. And the average time for my get rpc calls is about ~10ms while ~100ms for the put calls. Actually, I have a handler which calls a model on each call to get the user data. And I am expecting the load on this handler for about ~200hits/sec And some of these users may update some data. And I am expecting about ~50hits/sec on this update handler. I am just looking whether appengine will be scaleable for my above requirements? Thanks -- Best Regards, Atif Gulzar I Unicode, ɹɐzlnƃ ɟıʇɐ -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Appengine scalability ~200 get/sec and about 50 put/sec
Test it! http://curl-loader.sourceforge.net/ (many similar tools are available, just a nice looking one I found via a search) On 10 February 2011 16:08, Atif Gulzar wrote: > Hi, > I am estimating whether AppEngine is scalable for ~200 get/sec and about 50 > put/sec RPC calls. > I have ran the AppStat on my current app, which have currently very low > traffic about ~20get and ~4put/sec. And the average time for my get rpc > calls is about ~10ms while ~100ms for the put calls. > Actually, I have a handler which calls a model on each call to get the user > data. And I am expecting the load on this handler for about ~200hits/sec > And some of these users may update some data. And I am expecting about > ~50hits/sec on this update handler. > I am just looking whether appengine will be scaleable for my above > requirements? > Thanks > > > -- > Best Regards, > Atif Gulzar > > I Unicode, ɹɐzlnƃ ɟıʇɐ > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Appengine scalability ~200 get/sec and about 50 put/sec
Yes I can run the load runner tools. But I just want to evaluate the methods how Appengine scalability can be measured? even before writing any webservice. -- Best Regards, Atif Gulzar I Unicode, ɹɐzlnƃ ɟıʇɐ On Thu, Feb 10, 2011 at 9:30 PM, Barry Hunter wrote: > Test it! > > http://curl-loader.sourceforge.net/ > (many similar tools are available, just a nice looking one I found via > a search) > > > On 10 February 2011 16:08, Atif Gulzar wrote: > > Hi, > > I am estimating whether AppEngine is scalable for ~200 get/sec and about > 50 > > put/sec RPC calls. > > I have ran the AppStat on my current app, which have currently very low > > traffic about ~20get and ~4put/sec. And the average time for my get rpc > > calls is about ~10ms while ~100ms for the put calls. > > Actually, I have a handler which calls a model on each call to get the > user > > data. And I am expecting the load on this handler for about ~200hits/sec > > And some of these users may update some data. And I am expecting about > > ~50hits/sec on this update handler. > > I am just looking whether appengine will be scaleable for my above > > requirements? > > Thanks > > > > > > -- > > Best Regards, > > Atif Gulzar > > > > I Unicode, ɹɐzlnƃ ɟıʇɐ > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Appengine scalability ~200 get/sec and about 50 put/sec
GAE is optimized for datastore read operations. No problem in scaling for get operations. The put scaling depends on if you are updating a particular entity group more then 1 or 2 times a second. If not then there is no problem in scaling put operations. If updating one (1) entity group more then 1/sec but it happens in bursts you can use the task queue to perform the update, it will retry in case you have a transaction exception. 2011/2/10 Atif Gulzar : > Hi, > I am estimating whether AppEngine is scalable for ~200 get/sec and about 50 > put/sec RPC calls. > I have ran the AppStat on my current app, which have currently very low > traffic about ~20get and ~4put/sec. And the average time for my get rpc > calls is about ~10ms while ~100ms for the put calls. > Actually, I have a handler which calls a model on each call to get the user > data. And I am expecting the load on this handler for about ~200hits/sec > And some of these users may update some data. And I am expecting about > ~50hits/sec on this update handler. > I am just looking whether appengine will be scaleable for my above > requirements? > Thanks > > > -- > Best Regards, > Atif Gulzar > > I Unicode, ɹɐzlnƃ ɟıʇɐ > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Appengine scalability ~200 get/sec and about 50 put/sec
Scalability is more than benchmarking a specific operation. Are you trying to do 200 writes to a single row transactionally? App Engine (and honestly, very few other systems in the world) will not meet your needs. Are you distributing 200 writes across various keys, randomly spread out across the entire keyspace? You'll probably be fine. You'll need to provide a more useful description of what it is you plan on doing to get a useful answer. I find this blog post to be great in describing the differences between scalability and benchmarking. It's about node.js and non-blocking I/O, but the concepts apply in general: http://al3x.net/2010/07/27/node.html -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Thu, Feb 10, 2011 at 8:42 PM, Atif Gulzar wrote: > > Yes I can run the load runner tools. But I just want to evaluate the > methods how Appengine scalability can be measured? even before writing any > webservice. > > > -- > Best Regards, > Atif Gulzar > > I Unicode, ɹɐzlnƃ ɟıʇɐ > > > > On Thu, Feb 10, 2011 at 9:30 PM, Barry Hunter wrote: > >> Test it! >> >> http://curl-loader.sourceforge.net/ >> (many similar tools are available, just a nice looking one I found via >> a search) >> >> >> On 10 February 2011 16:08, Atif Gulzar wrote: >> > Hi, >> > I am estimating whether AppEngine is scalable for ~200 get/sec and >> about 50 >> > put/sec RPC calls. >> > I have ran the AppStat on my current app, which have currently very low >> > traffic about ~20get and ~4put/sec. And the average time for my get rpc >> > calls is about ~10ms while ~100ms for the put calls. >> > Actually, I have a handler which calls a model on each call to get the >> user >> > data. And I am expecting the load on this handler for about ~200hits/sec >> > And some of these users may update some data. And I am expecting about >> > ~50hits/sec on this update handler. >> > I am just looking whether appengine will be scaleable for my above >> > requirements? >> > Thanks >> > >> > >> > -- >> > Best Regards, >> > Atif Gulzar >> > >> > I Unicode, ɹɐzlnƃ ɟıʇɐ >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine?hl=en. >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Appengine scalability ~200 get/sec and about 50 put/sec
My app briefly seved 120k request in an hour when I ranked for a trending topic. That includes some static request. These were about 99% served from memcache data store. As Ikai Lan points out this is not a “scale” it is a “benchmark” App engine ran my app on 14 instances at that time, and the latency was about 280ms. I don’t have any doubt that if I had had twice that traffic that GAE would have spun up to 28 instances and kept running and if I had 10 fold the traffic, I have every confidence that I would have contacted Google and They’d have taken the training wheels off and I would have had 128 instances. It has been my experience that Appengine will serve as many people as you want, but if you write your code wrong it could cost you a fortune, or hit one of the “hey you tried to do something stupid so we stopped you” synthetic quotas. When you hit one of these quotas you can either fix your code, or tell Google that you don’t care that it will cost a fortune, let me do it anyway. Perhaps someone at Google can quote some numbers as to how many people ArtProject has served? A screen shot of the dash board with quota’s and latency for that project would be awesome. -brandon From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Ikai Lan (Google) Sent: Friday, February 11, 2011 4:22 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Appengine scalability ~200 get/sec and about 50 put/sec Scalability is more than benchmarking a specific operation. Are you trying to do 200 writes to a single row transactionally? App Engine (and honestly, very few other systems in the world) will not meet your needs. Are you distributing 200 writes across various keys, randomly spread out across the entire keyspace? You'll probably be fine. You'll need to provide a more useful description of what it is you plan on doing to get a useful answer. I find this blog post to be great in describing the differences between scalability and benchmarking. It's about node.js and non-blocking I/O, but the concepts apply in general: http://al3x.net/2010/07/27/node.html -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com <http://googleappengine.blogspot.com/> Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Thu, Feb 10, 2011 at 8:42 PM, Atif Gulzar wrote: Yes I can run the load runner tools. But I just want to evaluate the methods how Appengine scalability can be measured? even before writing any webservice. -- Best Regards, Atif Gulzar I Unicode, ɹɐzlnƃ ɟıʇɐ On Thu, Feb 10, 2011 at 9:30 PM, Barry Hunter wrote: Test it! http://curl-loader.sourceforge.net/ (many similar tools are available, just a nice looking one I found via a search) On 10 February 2011 16:08, Atif Gulzar wrote: > Hi, > I am estimating whether AppEngine is scalable for ~200 get/sec and about 50 > put/sec RPC calls. > I have ran the AppStat on my current app, which have currently very low > traffic about ~20get and ~4put/sec. And the average time for my get rpc > calls is about ~10ms while ~100ms for the put calls. > Actually, I have a handler which calls a model on each call to get the user > data. And I am expecting the load on this handler for about ~200hits/sec > And some of these users may update some data. And I am expecting about > ~50hits/sec on this update handler. > I am just looking whether appengine will be scaleable for my above > requirements? > Thanks > > > -- > Best Regards, > Atif Gulzar > > I Unicode, ɹɐzlnƃ ɟıʇɐ > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > <mailto:google-appengine%2bunsubscr...@googlegroups.com> . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com <mailto:google-appengine%2bunsubscr...@googlegroups.com> . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com <mail
