Hello, We are experiencing an issue in the following setup. We have a module that processes tasks from a push queue. The module is using automatic scaling. Normally it runs around 8 instances and processes around 200 tasks per second. Recently I noticed that there are spikes in the App Engine Dashboard in the Latency graph. (Worths mentioning that the spikes seem to occur more or less exactly once an hour) So I checked the logs and found the following pattern:
1. All of a sudden all the instances start to report the following: "After handling this request, the process that handled this request reached the maximum number of requests that may be handled in a single process' lifetime, and exited normally." I mean one by one all of them will throw this message. 2. Next you see a bunch of the following messages: "This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application.". Each request reporting this lasts like a few seconds so it truly "takes longer" as the message says. Ultimately what seems to happen is that all our instances restart at the same time which means that for a few seconds there are no instances ready to serve. This generates huge latency spikes that we surely would like to eliminate if possible. Is there something we could do on this? Thank you, Iliya. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/20ffe46d-f443-43ae-b531-70a751b79f31%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
