The scheduler is the same. This might have to do with `threadsafe` in app.yaml, which is complicated and confusing. I'd suggest removing `threadsafe` if it's present, then try setting the max_concurrent_requests to something higher (like 2x your number of workers): https://cloud.google.com/appengine/docs/standard/python/config/appref#max_concurrent_requests
On Fri, Nov 5, 2021 at 1:48 PM Joshua Smith <mrjoshuaesm...@gmail.com> wrote: > I figured out from the logs (since it prints a message for each worker it > starts) that 3.7 is starting 8 workers by default. So adding the entrypoint > with defaults has no effect. > > However, I did some experiments and I think I can characterize the > difference between 2.7 and 3.7 when it comes to scaling. > > When first landing on my app's page it shows a list of thumbnails. These > come from the datastore, with each thumbnail making a HTTP request (GET > /image?which=1, GET /image?which=2, etc.). If we: > > 1. Query for all of these at once so the server gets hammered with > simultaneous requests: > - 2.7 starts one extra instance, and while that's happening the images are > all served by the instance that already existed; the new instance never > actually serves any of those image requests. > - 3.7 starts a lot of instances; it dumps a bunch of requests to each of > them (one per worker thread, I presume), all of which are going to take 5 > seconds because that's how long it takes each instance to start. > > The net result is the user of the 2.7 app sees all the images load right > away, while the 3.7 app gets them over the course of 15 seconds or so, and > sometimes there are timeout errors. > > 2. Query for these with a delay that's long enough to avoid overlapping > requests: > - Neither starts extra instances. > > The user sees all the images load pretty quick, but not quite as quick as > 2.7 with the first approach. > > My conclusion is that the change is that the old scheduler wouldn't use an > instance until it was running, whereas the new scheduler queues up requests > for instances that are still starting up. The old scheduler doesn't start a > new instance until the one it started most recently is running, whereas the > new scheduler starts a new instance every time the request queue fills up > for existing instances. > > If you can serve a burst of requests in the time it take an instance to > start, the old scheduler will get all those served. But the new scheduler > will choke and perform very badly. > > I'm not saying that the old scheduler was better. It just handled this > particular case (cold start with a firehose) a lot better. Once the two > systems are running at scale, I wouldn't expect there to be much difference. > > -Joshua > > > On Nov 5, 2021, at 3:29 PM, Joshua Smith <mrjoshuaesm...@gmail.com> wrote: > > I think it's telling me that since my P3.7 app has a F4_1G instance, I > should be configuring it to run gunicorn with 8 workers. I don't see any > information about the default number of workers. I guess I can do an > experiment to see what explicitly setting that does... > > On Nov 5, 2021, at 3:17 PM, Jason Collins <jason.a.coll...@gmail.com> > wrote: > > It sounds like there might be some differences due to concurrency between > 2.7 and 3.7/8/9. There are some notes on how to tweak the number of worker > processes started for Python 3 here: > https://cloud.google.com/appengine/docs/standard/python3/runtime#entrypoint_best_practices > > > On Friday, 5 November 2021 at 11:50:52 UTC-7 George (Cloud Platform > Support) wrote: > >> There is no official description, somewhere, of "how the scaling has >> changed" or "how it has been implemented differently", as there is no >> change in scaling behavior, and no different implementation. This is an >> assumption while we investigate the situation described by Joshua above. I >> could not find a similar issue in the Public Issue Tracker >> <https://issuetracker.google.com/>, so the difference in scaling between >> Python 2 and 3 is likely a one-time occurrence, and caused by idiosyncratic >> settings in the project, or application-specific factors. Such issue are >> difficult to approach in a publicly-accessible thread as this one, as >> investigation needs particulars such as project ID, and access to other >> private data. To have this issue properly addressed, it's better to open >> a support case >> <https://cloud.google.com/support/docs/manage-cases#creating_cases> or >> an issue in the Public Issue Tracker <https://issuetracker.google.com/>. >> >> On Friday, 05 November 2021 at 05:50:42 UTC-4 Nicolas Fonrose (Teevity) >> wrote: >> >>> Hello David, >>> >>> >There’s not an official way of forcing the scaling to behave like it >>> used to be on python 2.7. >>> Is there an official description, somewhere, of "how the scaling has >>> changed" or "how it has been implemented differently" ? >>> Or this change just a side effect of technical choices that were made in >>> the new runtimes? >>> >>> Thanks, >>> >>> On Thursday, November 4, 2021 at 9:56:43 PM UTC+1 David (Cloud Platform >>> Support) wrote: >>> >>>> Hello, >>>> >>>> Other than applying changes within your own application so it will go >>>> easy at startup, you can try using warmup requests >>>> <https://cloud.google.com/appengine/docs/standard/python3/configuring-warmup-requests> >>>> along with tweaking the min_idle_instances element >>>> <https://cloud.google.com/appengine/docs/standard/python3/config/appref#scaling_elements> >>>> in your app.yaml, in order to reduce request and response latency during >>>> the time when your app's code is being loaded to a newly created instance. >>>> There’s not an official way of forcing the scaling to behave like it >>>> used to be on python 2.7. However, this type of feedback can be passed to >>>> the App Engine engineering team in the form of a feature request. You can >>>> create such a feature request here >>>> <https://issuetracker.google.com/issues/new?component=187191&template=1162953>. >>>> The App Engine engineering team would then evaluate it and decide whether >>>> it could be implemented or not. >>>> On Thursday, November 4, 2021 at 11:38:52 AM UTC-4 Joshua Smith wrote: >>>> >>>>> Thanks for sending out this update. I did, indeed, miss most of this >>>>> news. >>>>> >>>>> This item, in particular, is *awesome*: >>>>> >>>>> >>>>> >>>>> - >>>>> >>>>> Extending support for App Engine bundled services >>>>> >>>>> <https://cloud.google.com/blog/products/serverless/support-for-app-engine-services-in-second-generation-runtimes> >>>>> (Sep 2021) >>>>> >>>>> >>>>> This sounds like it will make it so much easier to migrate to Python >>>>> 3.7. >>>>> >>>>> One thing I've noticed is that my older apps on 2.7 seem to handle >>>>> peak scaling a lot better than my newer apps on 3.7. For example, if I >>>>> have >>>>> a web page that hits a 2.7 app with 100 REST calls at startup (bad design, >>>>> but it happens), the old app serves them all eventually. But if I do the >>>>> same thing in a 3.7 app, it's likely to choke and fail a bunch of those >>>>> requests with these: >>>>> >>>>> >>>>> The specific pattern is that after the first couple requests, it spins >>>>> up a new instance. That takes 5 seconds to serve its first request (simple >>>>> app, so I guess that's just GAE overhead). Then it spins up a couple more. >>>>> Then I start getting those errors on some of the requests, because 15s >>>>> have >>>>> passed. In this capture the blue ones are new instances spinning up, and >>>>> the orange ones are timeout errors (note the 15s time on those). >>>>> >>>>> >>>>> I've been designing around the issue by making sure my web apps go >>>>> easy on the server on startup. But it is yet another concern about >>>>> migrating. >>>>> >>>>> Is there a way to tune the auto-scaling for 3.7 to behave like the >>>>> 2.7? Here's a similar set of requests to my 2.7 app, which only spins up >>>>> one extra instance, and never throws timeout errors, ever: >>>>> >>>>> >>>>> -Joshua >>>>> >>>>> >>>>> > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/150109bb-18a4-40b1-9efb-8b86bc60eceen%40googlegroups.com > <https://groups.google.com/d/msgid/google-appengine/150109bb-18a4-40b1-9efb-8b86bc60eceen%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Google App Engine" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/google-appengine/pEi3hxX5YpE/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > google-appengine+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/5B04AE8B-06A4-43C3-9922-D9859BB4610E%40gmail.com > <https://groups.google.com/d/msgid/google-appengine/5B04AE8B-06A4-43C3-9922-D9859BB4610E%40gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAL_dBebOMgXrwsxX2U3VXD%3DTxPHCDHy84vouRveb22N20Bt_Xw%40mail.gmail.com.
