[google-appengine] Re: Spending LImits Going Away :(
Hello, Jukka I am hopeful that the steps described in the documentation[1][2] on how disable your App Engine application when costs exceed your set threshold are comprehensible as I cannot find an example at this time that shows the step. Should you certainly consider a need for the Documentation improvement, perhaps to have the example added, you can submit such Documentation improvement feature request using the issue link[3]. Our Documentation team would be glad to review the possibility. However, such requests do not have ETA. Thank you. [1]https://cloud.google.com/appengine/docs/managing-costs#disable_manally [2]https://cloud.google.com/appengine/docs/managing-costs#disable_your_app_programmatically [3]https://developers.google.com/issue-tracker/#public_users On Wednesday, September 2, 2020 at 4:39:40 PM UTC-4 jukka.ha...@gmail.com wrote: > Is there a simple example how to disable after budget alert, let's say php > google app engine application ? > > keskiviikko 2. syyskuuta 2020 klo 22.19.57 UTC+3 Alexis (Google Cloud > Platform Support) kirjoitti: > >> >> Hello everyone, >> >> To summarize this conversation, it is possible to set proactive limits by: >> - Maximum number of instance[1] in an app >> - Disable your app programmatically[2] for any resource consumption, etc.. >> - Cap API limits to prevent too many requests[3] >> >> If I understood properly, Joshua's situation is about caching. So in that >> case, we are talking about resource limits related to CPU cycles, or maybe >> amount of requests... I suspect that whatever resource spiked by this >> situation, can probably make use of the second option above or even the >> third if it's API consumption. But it all depends on how fast this happened >> and I would agree that having a hard cap feature would prevent the delay. >> >> In terms of instantaneous hard caps (with no delays), a feature >> request[4] can be done. However, I think it would be advantageous to >> clarify how such a cap can be helpful without impacting scalability. Please >> see below. >> >> When submitting a feature request: >> - Try to clarify how the limit should differentiate a CPU cycle that is >> legit and versus one that isn't legit. Would you want that on all your >> instances? Would you mind if it stopped all your services as a false alarm? >> (The current solution for that is max instances in my first point above >> because scaling is a horizontal concept and it doesn't completely stop >> things since it's a delta quantity. If some of the other issues mentioned >> were due to excessive instances or API requests , then please refer to the >> above). >> >> I hope this message consolidates the research and saves time for any new >> person reading this post. Thank you in advance. >> >> [1] >> https://cloud.google.com/appengine/docs/managing-costs#specify_the_maximum_number_of_instances >> [2] >> https://cloud.google.com/appengine/docs/managing-costs#disable_your_app_programmatically >> [3] https://cloud.google.com/apis/docs/capping-api-usage >> [4] >> https://cloud.google.com/support/docs/issue-trackers?hl=id#trackers-list >> >> On Wednesday, September 2, 2020 at 11:18:59 AM UTC-4 vit@gmail.com >> wrote: >> >>> Canceling a simple and straightforward option "daily spending limit" is >>> a very cruel action for beginners just starting to learn the platform. A >>> separate warning should be given in the starter guide: >>> >>> In order not to go bankrupt when using GCP, you first need to carefully >>> study the documentation section https://cloud.google.com/cost-management >>> . >>> >>> My story. >>> >>> Ndb model with a lot of indexed fields (since the field property indexed >>> = True by default). The cost of writing one entry to such a model was very >>> high. >>> >>> The main program sent a message to the queue when a fairly rare event >>> occurred. The handler of such a message from the queue updated the >>> corresponding record in the model, and then an exception occurred due to an >>> error in the code, and the handler exited with an error code. >>> >>> The platform initiated several processing retries, which also failed. In >>> each attempt, there was a write to the model. >>> >>> The main program, not finding the results of processing, added a new >>> message to the queue, which caused a new series of attempts described >>> above. And so on. >>> >>> The daily budget was depleted within a few minutes and the program was >>> stopped by a fuse "daily spending limit". >>> >>> In the current reality, I would have received a bill for several >>> thousand dollars if I had reacted to this situation within 24 hours. >>> >>> Yes, then I changed the model, leaving indexed=True only for a couple of >>> fields where it was really necessary and changed the logic of both the main >>> program and the message handler from the queue. >>> >>> But all this happened later. >>> >>> вторник, 25 августа 2020 г. в 20:03:33 UTC+4,
[google-appengine] Re: Spending LImits Going Away :(
Is there a simple example how to disable after budget alert, let's say php google app engine application ? keskiviikko 2. syyskuuta 2020 klo 22.19.57 UTC+3 Alexis (Google Cloud Platform Support) kirjoitti: > > Hello everyone, > > To summarize this conversation, it is possible to set proactive limits by: > - Maximum number of instance[1] in an app > - Disable your app programmatically[2] for any resource consumption, etc.. > - Cap API limits to prevent too many requests[3] > > If I understood properly, Joshua's situation is about caching. So in that > case, we are talking about resource limits related to CPU cycles, or maybe > amount of requests... I suspect that whatever resource spiked by this > situation, can probably make use of the second option above or even the > third if it's API consumption. But it all depends on how fast this happened > and I would agree that having a hard cap feature would prevent the delay. > > In terms of instantaneous hard caps (with no delays), a feature request[4] > can be done. However, I think it would be advantageous to clarify how such > a cap can be helpful without impacting scalability. Please see below. > > When submitting a feature request: > - Try to clarify how the limit should differentiate a CPU cycle that is > legit and versus one that isn't legit. Would you want that on all your > instances? Would you mind if it stopped all your services as a false alarm? > (The current solution for that is max instances in my first point above > because scaling is a horizontal concept and it doesn't completely stop > things since it's a delta quantity. If some of the other issues mentioned > were due to excessive instances or API requests , then please refer to the > above). > > I hope this message consolidates the research and saves time for any new > person reading this post. Thank you in advance. > > [1] > https://cloud.google.com/appengine/docs/managing-costs#specify_the_maximum_number_of_instances > [2] > https://cloud.google.com/appengine/docs/managing-costs#disable_your_app_programmatically > [3] https://cloud.google.com/apis/docs/capping-api-usage > [4] > https://cloud.google.com/support/docs/issue-trackers?hl=id#trackers-list > > On Wednesday, September 2, 2020 at 11:18:59 AM UTC-4 vit@gmail.com > wrote: > >> Canceling a simple and straightforward option "daily spending limit" is a >> very cruel action for beginners just starting to learn the platform. A >> separate warning should be given in the starter guide: >> >> In order not to go bankrupt when using GCP, you first need to carefully >> study the documentation section https://cloud.google.com/cost-management. >> >> My story. >> >> Ndb model with a lot of indexed fields (since the field property indexed >> = True by default). The cost of writing one entry to such a model was very >> high. >> >> The main program sent a message to the queue when a fairly rare event >> occurred. The handler of such a message from the queue updated the >> corresponding record in the model, and then an exception occurred due to an >> error in the code, and the handler exited with an error code. >> >> The platform initiated several processing retries, which also failed. In >> each attempt, there was a write to the model. >> >> The main program, not finding the results of processing, added a new >> message to the queue, which caused a new series of attempts described >> above. And so on. >> >> The daily budget was depleted within a few minutes and the program was >> stopped by a fuse "daily spending limit". >> >> In the current reality, I would have received a bill for several thousand >> dollars if I had reacted to this situation within 24 hours. >> >> Yes, then I changed the model, leaving indexed=True only for a couple of >> fields where it was really necessary and changed the logic of both the main >> program and the message handler from the queue. >> >> But all this happened later. >> >> вторник, 25 августа 2020 г. в 20:03:33 UTC+4, Joshua Smith: >> >>> Once again last night, my wallet was saved when a runaway bot chewed up >>> my site’s whole daily spending limit. I got an email from a user, set up a >>> firewall rule, and goosed my budget to get things going again. >>> >>> I’m *very* concerned about Google’s decision to remove this feature. >>> Offering a cloud service that bills by usage without having a way to limit >>> the spend shifts an unreasonable amount of risk onto the subscriber. >>> >>> I’ve set up budget alerts, as suggested, but I’m concerned that: >>> >>> - What if my bill shoots up really fast? How quickly is this alert going >>> to go out? >>> >>> - What if I am away from the computer (remember when we used to be able >>> to leave our houses? good times… good times…)? >>> >>> I run this particular site as a not-for-profit social good. (It’s a site >>> that small town governments use to post their meetings.) I make *no* money >>> on it. >>> >>> I’d be perfectly
[google-appengine] Re: Spending LImits Going Away :(
Hello everyone, To summarize this conversation, it is possible to set proactive limits by: - Maximum number of instance[1] in an app - Disable your app programmatically[2] for any resource consumption, etc.. - Cap API limits to prevent too many requests[3] If I understood properly, Joshua's situation is about caching. So in that case, we are talking about resource limits related to CPU cycles, or maybe amount of requests... I suspect that whatever resource spiked by this situation, can probably make use of the second option above or even the third if it's API consumption. But it all depends on how fast this happened and I would agree that having a hard cap feature would prevent the delay. In terms of instantaneous hard caps (with no delays), a feature request[4] can be done. However, I think it would be advantageous to clarify how such a cap can be helpful without impacting scalability. Please see below. When submitting a feature request: - Try to clarify how the limit should differentiate a CPU cycle that is legit and versus one that isn't legit. Would you want that on all your instances? Would you mind if it stopped all your services as a false alarm? (The current solution for that is max instances in my first point above because scaling is a horizontal concept and it doesn't completely stop things since it's a delta quantity. If some of the other issues mentioned were due to excessive instances or API requests , then please refer to the above). I hope this message consolidates the research and saves time for any new person reading this post. Thank you in advance. [1] https://cloud.google.com/appengine/docs/managing-costs#specify_the_maximum_number_of_instances [2] https://cloud.google.com/appengine/docs/managing-costs#disable_your_app_programmatically [3] https://cloud.google.com/apis/docs/capping-api-usage [4] https://cloud.google.com/support/docs/issue-trackers?hl=id#trackers-list On Wednesday, September 2, 2020 at 11:18:59 AM UTC-4 vit@gmail.com wrote: > Canceling a simple and straightforward option "daily spending limit" is a > very cruel action for beginners just starting to learn the platform. A > separate warning should be given in the starter guide: > > In order not to go bankrupt when using GCP, you first need to carefully > study the documentation section https://cloud.google.com/cost-management. > > My story. > > Ndb model with a lot of indexed fields (since the field property indexed = > True by default). The cost of writing one entry to such a model was very > high. > > The main program sent a message to the queue when a fairly rare event > occurred. The handler of such a message from the queue updated the > corresponding record in the model, and then an exception occurred due to an > error in the code, and the handler exited with an error code. > > The platform initiated several processing retries, which also failed. In > each attempt, there was a write to the model. > > The main program, not finding the results of processing, added a new > message to the queue, which caused a new series of attempts described > above. And so on. > > The daily budget was depleted within a few minutes and the program was > stopped by a fuse "daily spending limit". > > In the current reality, I would have received a bill for several thousand > dollars if I had reacted to this situation within 24 hours. > > Yes, then I changed the model, leaving indexed=True only for a couple of > fields where it was really necessary and changed the logic of both the main > program and the message handler from the queue. > > But all this happened later. > > вторник, 25 августа 2020 г. в 20:03:33 UTC+4, Joshua Smith: > >> Once again last night, my wallet was saved when a runaway bot chewed up >> my site’s whole daily spending limit. I got an email from a user, set up a >> firewall rule, and goosed my budget to get things going again. >> >> I’m *very* concerned about Google’s decision to remove this feature. >> Offering a cloud service that bills by usage without having a way to limit >> the spend shifts an unreasonable amount of risk onto the subscriber. >> >> I’ve set up budget alerts, as suggested, but I’m concerned that: >> >> - What if my bill shoots up really fast? How quickly is this alert going >> to go out? >> >> - What if I am away from the computer (remember when we used to be able >> to leave our houses? good times… good times…)? >> >> I run this particular site as a not-for-profit social good. (It’s a site >> that small town governments use to post their meetings.) I make *no* money >> on it. >> >> I’d be perfectly happy to handle this with self-set quotas on something >> other than dollars. For example, in my case the budget-buster is always >> “Cloud Datastore Read Operations.” If I could set a cap on that one thing, >> it’d give me the protection I need. >> >> -Joshua >> >> -- You received this message because you are subscribed to the Google Groups
[google-appengine] Re: Spending LImits Going Away :(
Canceling a simple and straightforward option "daily spending limit" is a very cruel action for beginners just starting to learn the platform. A separate warning should be given in the starter guide: In order not to go bankrupt when using GCP, you first need to carefully study the documentation section https://cloud.google.com/cost-management. My story. Ndb model with a lot of indexed fields (since the field property indexed = True by default). The cost of writing one entry to such a model was very high. The main program sent a message to the queue when a fairly rare event occurred. The handler of such a message from the queue updated the corresponding record in the model, and then an exception occurred due to an error in the code, and the handler exited with an error code. The platform initiated several processing retries, which also failed. In each attempt, there was a write to the model. The main program, not finding the results of processing, added a new message to the queue, which caused a new series of attempts described above. And so on. The daily budget was depleted within a few minutes and the program was stopped by a fuse "daily spending limit". In the current reality, I would have received a bill for several thousand dollars if I had reacted to this situation within 24 hours. Yes, then I changed the model, leaving indexed=True only for a couple of fields where it was really necessary and changed the logic of both the main program and the message handler from the queue. But all this happened later. вторник, 25 августа 2020 г. в 20:03:33 UTC+4, Joshua Smith: > Once again last night, my wallet was saved when a runaway bot chewed up my > site’s whole daily spending limit. I got an email from a user, set up a > firewall rule, and goosed my budget to get things going again. > > I’m *very* concerned about Google’s decision to remove this feature. > Offering a cloud service that bills by usage without having a way to limit > the spend shifts an unreasonable amount of risk onto the subscriber. > > I’ve set up budget alerts, as suggested, but I’m concerned that: > > - What if my bill shoots up really fast? How quickly is this alert going > to go out? > > - What if I am away from the computer (remember when we used to be able to > leave our houses? good times… good times…)? > > I run this particular site as a not-for-profit social good. (It’s a site > that small town governments use to post their meetings.) I make *no* money > on it. > > I’d be perfectly happy to handle this with self-set quotas on something > other than dollars. For example, in my case the budget-buster is always > “Cloud Datastore Read Operations.” If I could set a cap on that one thing, > it’d give me the protection I need. > > -Joshua > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/aabd10e6-e3f9-41a7-83ef-ebbc2f5d1ef4n%40googlegroups.com.
[google-appengine] Re: Spending LImits Going Away :(
Hello Joshua, I'll try to help the best I can. I've added a question below, along with some answers too. Questions: - Which feature did Google remove? What was this feature called and where was it shown in the GUI? I'm asking so that we can try to maybe do a feature request or see why it was removed. To answer your question, the alerts should be prompt unless there is an outage or some other exceptional circumstance. However, keep in mind that we do not have control over public communication channel delays such as SMS, email, etc... Double-checking in the GUI could tell you if it was already sent out or not. Delays can also happen if alerts have multiple conditions and one of them hasn't been met yet. See full article here[1] for more details about latency possibilities. If you're away from the computer, you have notification options here[2], called "channels". [1] https://cloud.google.com/monitoring/alerts/concepts-indepth#notification-latency [2] https://cloud.google.com/monitoring/support/notification-options#creating_channels On Tuesday, August 25, 2020 at 12:03:33 PM UTC-4 Joshua Smith wrote: > Once again last night, my wallet was saved when a runaway bot chewed up my > site’s whole daily spending limit. I got an email from a user, set up a > firewall rule, and goosed my budget to get things going again. > > I’m *very* concerned about Google’s decision to remove this feature. > Offering a cloud service that bills by usage without having a way to limit > the spend shifts an unreasonable amount of risk onto the subscriber. > > I’ve set up budget alerts, as suggested, but I’m concerned that: > > - What if my bill shoots up really fast? How quickly is this alert going > to go out? > > - What if I am away from the computer (remember when we used to be able to > leave our houses? good times… good times…)? > > I run this particular site as a not-for-profit social good. (It’s a site > that small town governments use to post their meetings.) I make *no* money > on it. > > I’d be perfectly happy to handle this with self-set quotas on something > other than dollars. For example, in my case the budget-buster is always > “Cloud Datastore Read Operations.” If I could set a cap on that one thing, > it’d give me the protection I need. > > -Joshua > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f77ae31b-61bd-4251-bb5d-4809076b70fan%40googlegroups.com.
