I'm also not clear on why it would take on the order of 30 seconds to warm up an instance, but I assume it's an unavoidable property of GAE. There's nothing in my code that would make it take so long, unless it has to install the dependencies every time or something.
On Monday, August 3, 2020 at 1:26:37 PM UTC-4, Alan deLespinasse wrote: > > Python 3.7, Standard Environment. Here's my entire app.yaml: > > runtime: python37 > instance_class: F4 > automatic_scaling: > min_instances: 1 > max_instances: 10 > > inbound_services: > - warmup > > > > The problem: Several times a day, the instance (of which there's only one) > seems to get terminated randomly. This is fine; I understand that App > Engine doesn't guarantee a single instance will last forever. The problem > is that the new instance doesn't start warming up until after the old one > stops, which means that any real requests made during warmup end up having > very high latency, sometimes over 30 seconds. Shouldn't GAE wait to > terminate the old instance until the replacement is finished warming up? > > I'm pretty sure the process isn't crashing or running out of memory or > anything. Most of the time, this service just handles one request every > minute (it's a cron job), and the terminate signals seem to happen at > random times in between those requests. > > Maybe there's just something I don't understand about GAE configuration. > > I could increase min_instances to 2, of course, which I assume would make > this much less likely, but I'd rather not. This isn't a high-traffic or > (currently) mission critical server. On the other hand, having occasional > requests take 30+ seconds can be pretty annoying. > > Is there something else I can do? > > Yes, I've read How Instances Are Managed > <https://cloud.google.com/appengine/docs/standard/python3/how-instances-are-managed> > . > > Typical logs are below (stdout, stderr, and request_log, all versions). > You can see the cron job at the top at 22:39:00, 22:40:00, 22:41:00. > > At 22:41:47 is the terminate signal, and at 22:41:48 the warmup request > for the new instance. There's an exception right after that that I assume > is triggered by the process terminating (related to a Firestore connection, > I think). > > Then at 22:42:00, the next cron request happens, and I think it actually > triggers a second new instance because there's no instance currently ready > to go. If it just waited for the first one to finish warming up, this > request would have taken about 19 seconds. But since it instead starts a > second one, it takes 34 seconds, and increases my bill as well. > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/2d474331-bf78-4ac2-8f21-38a8960d888bo%40googlegroups.com.
