Hello everyone, To summarize this conversation, it is possible to set proactive limits by: - Maximum number of instance[1] in an app - Disable your app programmatically[2] for any resource consumption, etc.. - Cap API limits to prevent too many requests[3]
If I understood properly, Joshua's situation is about caching. So in that case, we are talking about resource limits related to CPU cycles, or maybe amount of requests... I suspect that whatever resource spiked by this situation, can probably make use of the second option above or even the third if it's API consumption. But it all depends on how fast this happened and I would agree that having a hard cap feature would prevent the delay. In terms of instantaneous hard caps (with no delays), a feature request[4] can be done. However, I think it would be advantageous to clarify how such a cap can be helpful without impacting scalability. Please see below. When submitting a feature request: - Try to clarify how the limit should differentiate a CPU cycle that is legit and versus one that isn't legit. Would you want that on all your instances? Would you mind if it stopped all your services as a false alarm? (The current solution for that is max instances in my first point above because scaling is a horizontal concept and it doesn't completely stop things since it's a delta quantity. If some of the other issues mentioned were due to excessive instances or API requests , then please refer to the above). I hope this message consolidates the research and saves time for any new person reading this post. Thank you in advance. [1] https://cloud.google.com/appengine/docs/managing-costs#specify_the_maximum_number_of_instances [2] https://cloud.google.com/appengine/docs/managing-costs#disable_your_app_programmatically [3] https://cloud.google.com/apis/docs/capping-api-usage [4] https://cloud.google.com/support/docs/issue-trackers?hl=id#trackers-list On Wednesday, September 2, 2020 at 11:18:59 AM UTC-4 vit....@gmail.com wrote: > Canceling a simple and straightforward option "daily spending limit" is a > very cruel action for beginners just starting to learn the platform. A > separate warning should be given in the starter guide: > > In order not to go bankrupt when using GCP, you first need to carefully > study the documentation section https://cloud.google.com/cost-management. > > My story. > > Ndb model with a lot of indexed fields (since the field property indexed = > True by default). The cost of writing one entry to such a model was very > high. > > The main program sent a message to the queue when a fairly rare event > occurred. The handler of such a message from the queue updated the > corresponding record in the model, and then an exception occurred due to an > error in the code, and the handler exited with an error code. > > The platform initiated several processing retries, which also failed. In > each attempt, there was a write to the model. > > The main program, not finding the results of processing, added a new > message to the queue, which caused a new series of attempts described > above. And so on. > > The daily budget was depleted within a few minutes and the program was > stopped by a fuse "daily spending limit". > > In the current reality, I would have received a bill for several thousand > dollars if I had reacted to this situation within 24 hours. > > Yes, then I changed the model, leaving indexed=True only for a couple of > fields where it was really necessary and changed the logic of both the main > program and the message handler from the queue. > > But all this happened later. > > вторник, 25 августа 2020 г. в 20:03:33 UTC+4, Joshua Smith: > >> Once again last night, my wallet was saved when a runaway bot chewed up >> my site’s whole daily spending limit. I got an email from a user, set up a >> firewall rule, and goosed my budget to get things going again. >> >> I’m *very* concerned about Google’s decision to remove this feature. >> Offering a cloud service that bills by usage without having a way to limit >> the spend shifts an unreasonable amount of risk onto the subscriber. >> >> I’ve set up budget alerts, as suggested, but I’m concerned that: >> >> - What if my bill shoots up really fast? How quickly is this alert going >> to go out? >> >> - What if I am away from the computer (remember when we used to be able >> to leave our houses? good times… good times…)? >> >> I run this particular site as a not-for-profit social good. (It’s a site >> that small town governments use to post their meetings.) I make *no* money >> on it. >> >> I’d be perfectly happy to handle this with self-set quotas on something >> other than dollars. For example, in my case the budget-buster is always >> “Cloud Datastore Read Operations.” If I could set a cap on that one thing, >> it’d give me the protection I need. >> >> -Joshua >> >> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f4e25aa3-2ee2-45d9-8329-723145c969bbn%40googlegroups.com.
