Hi,
Did you try ramping up your request ratea little more slowly? If
your app has a long startup time (and it liekly does since it is Java)
and services most requests very fast, then I've sometimes seen the
scheduler be a bit too resistent to spin up new instances. In that
situation I sometimes start seeing those nasty pending_ms times, which
causes increased latency, which then causes the app not to get more
instances, etc.... To me it appears to be a very nasty feedback loop.
Watch the instance info in your app's dashboard, paying particular
attention to the latency. You want to try keeping that number well
under 1000ms if possible. I find that by ramping up the traffic a bit
more slowly I get more instnances. Note that by slowly I'm not
talking about hours, but at least over a minute or two. Another
option is to 'prime' the app with a couple smaller bursts of traffic
to get a few instances spun up.
Robert
On Thu, May 26, 2011 at 07:36, Hamish <hmg...@gmail.com> wrote:
> Hi,
>
> We are developing a Java app for Google App Engine. We did some
> testing today using "ab" to hit a particular URL. We had 1000
> concurrent threads in ab hitting the URL 1 million times.
>
> We were surprised by the response times. A lot of the responses were
> taking well over a second to return with some even taking up to 4
> seconds. This is a lot higher than we would like. From looking at the
> logs it seems the majority of the time is in "pending_ms". At the peak
> of the traffic GAE had a total of 7 instances with 4 active
> instances.
>
> Could someone explain what "pending_ms" actually means? What can we do
> to improve it?
>
>
> Some application details:
> - <threadsafe>true</threadsafe> in the appengine-web.xml file.
> - the app uses the Restlet framework
> - the URL we hit returns JSON data less than 2kb in size
> - the app increments a counter in memcache for each request
> - the datastore is _not_ hit for a typical request
> - we have a cron that runs every couple minutes to persist the
> counters to the datastore
>
>
> Here is a typical log entry:
>
> 50.16.218.117 - - [26/May/2011:02:51:26 -0700] "GET /v1/{my_url} HTTP/
> 1.1" 200 370 - "ApacheBench/2.3,gzip(gfe)" "{appname}.appspot.com"
> ms=1262 cpu_ms=19 api_cpu_ms=0 cpm_usd=0.000604 pending_ms=1179
>
> note: I have removed the actual app name and URL.
>
>
> Thanks for your help.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
