[google-appengine] Re: High Amount CPU Quota should be removed
Hi, > Basically your understanding is correct. I conclude my experience > about tuning against the high CPU quota here: my own app gives the "high amount of CPU" error when it hits the root of the app. I can think of this: 1. py files are probably getting compiled after the app is uploaded 2. a django template is rendered 3. memcache is queried and/or written to Best regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Hi Peter, Basically your understanding is correct. I conclude my experience about tuning against the high CPU quota here: 1. Don't care about the Admin Console warning too much. It said you will exceed quota soon for your red requests, but it never happen. So just look at your warning logs. I spent huge amount of time during the last few days to figure out this... 2. Since you have the log, you know the request causing you issues and you can put debug info there to help you figure out what the issue is. Here are some of the cases I met that showing me the high CPU warning: 1) Loading too much data. For example, I used to load 50 records and it takes quite some time to process and render them to generate the response. Then I break it into pages and everything is fine. Almost all of my warnings are caused by this root cause. 2) Memcache usage seems to be counted here. Generally you should not meet much issue here. But if you put big objects like list to memcache, it may cause problems. The Admin Console warning is useful to help you figure out that which request is doing heavy datastore operations or URL calls. So you may want to tune your datastore operation and URL calls against this. On Sep 27, 9:48 am, Peter Recore <[EMAIL PROTECTED]> wrote: > mitnick, > > Can you explain more fully what the problem was, and how you were able > to debug the problem once you knew to look at the logs vs the > dashboard? After all the back and forth on the high cpu quota issue, > I am curious about the particulars. Right now, this is how I > understand it: > > Warnings in the logs are based solely on the CPU used in Non API calls > - so if I had a request that did nothing but try to factor a really > large prime, I would probably get a lot of these warnings, and they > would also trigger dashboard warnings? > > Warnings in the dashboard include CPU used in API calls, so if I had a > request that did nothing but complicated datastore calls, it would get > a dashboard warning, but *not* a log warning? > > If that's the case, I am guessing you (mitnick) had a request that was > using a lot of runtime CPU but not necessarily a lot of API time? > > On Sep 26, 8:55 pm, mitnickcbc <[EMAIL PROTECTED]> wrote: > > > > Request denials for high CPU are based on the runtime CPU usage (which > > > does not include the CPU used in API calls). Since the runtime CPU > > > cost are the ones present in the logs, it is more useful to rely on > > > the log information when trying to determine how best to reduce CPU > > > load for specific requests. > > > Thanks for the clarification! Now I'm able to find the bottleneck > > and... it scales! > > > On Sep 27, 2:00 am, Jeff S <[EMAIL PROTECTED]> wrote: > > > > On Sep 26, 1:48 am, mitnickcbc <[EMAIL PROTECTED]> wrote: > > > > > Some update about my latest findings. And these are my guesses, please > > > > do correct me if I'm wrong. > > > > > The quota denial for high amount CPU and the warning message "Many of > > > > the requests to your application are taking a very long time. Please > > > > optimize these requests." is calculated based on the number of warning > > > > you get in your log saying that "This request used a high amount of > > > > CPU, and was roughly 1.0 times over the average request CPU limit. > > > > High CPU requests have a small quota, and if you exceed this quota, > > > > your app will be temporarily disabled." > > > > > Please note, the average CPU usage and related warning are very > > > > misleading. The mcycles value displayed in log is also very different > > > > from the x.x times value showed in warning message. I used to think > > > > that request using beyond 1800 mcycles CPU will be considered as a > > > > high CPU request since it shows red in Average CPU column. But the > > > > latest finding is that it's not the case. One kind of my requests use > > > > about 6K-7K mcycles in average CPU, but I seldom see a high CPU > > > > warning in log for them. Again, this mcycles value seems to mean > > > > nothing and the algorithm used to calculate the x.x times high CPU > > > > warning really matters! > > > > As you have noticed, there are differences in the CPU usage > > > measurements presented in the logs and in the dashboard. This thread > > > contains clarification which I've copied below: > > > >http://groups.google.com/group/google-appengine/browse_frm/thread/f21... > > > "Concerning CPU accounting overall - The warning threshold for the > > > logs > > > and for the dashboard actually differ. In the logs, warnings will > > > only > > > occur for high runtime CPU. The warnings on the dashboard are > > > triggered for > > > high overall CPU, including CPU used in the runtime and per API, most > > > notably being the datastore API." > > > > Request denials for high CPU are based on the runtime CPU usage (which > > > does not include the CPU used in API calls). Since the runtime CPU > > > cost are the ones present in the logs, it is more useful to rely on
[google-appengine] Re: High Amount CPU Quota should be removed
mitnick, Can you explain more fully what the problem was, and how you were able to debug the problem once you knew to look at the logs vs the dashboard? After all the back and forth on the high cpu quota issue, I am curious about the particulars. Right now, this is how I understand it: Warnings in the logs are based solely on the CPU used in Non API calls - so if I had a request that did nothing but try to factor a really large prime, I would probably get a lot of these warnings, and they would also trigger dashboard warnings? Warnings in the dashboard include CPU used in API calls, so if I had a request that did nothing but complicated datastore calls, it would get a dashboard warning, but *not* a log warning? If that's the case, I am guessing you (mitnick) had a request that was using a lot of runtime CPU but not necessarily a lot of API time? On Sep 26, 8:55 pm, mitnickcbc <[EMAIL PROTECTED]> wrote: > > Request denials for high CPU are based on the runtime CPU usage (which > > does not include the CPU used in API calls). Since the runtime CPU > > cost are the ones present in the logs, it is more useful to rely on > > the log information when trying to determine how best to reduce CPU > > load for specific requests. > > Thanks for the clarification! Now I'm able to find the bottleneck > and... it scales! > > On Sep 27, 2:00 am, Jeff S <[EMAIL PROTECTED]> wrote: > > > On Sep 26, 1:48 am, mitnickcbc <[EMAIL PROTECTED]> wrote: > > > > Some update about my latest findings. And these are my guesses, please > > > do correct me if I'm wrong. > > > > The quota denial for high amount CPU and the warning message "Many of > > > the requests to your application are taking a very long time. Please > > > optimize these requests." is calculated based on the number of warning > > > you get in your log saying that "This request used a high amount of > > > CPU, and was roughly 1.0 times over the average request CPU limit. > > > High CPU requests have a small quota, and if you exceed this quota, > > > your app will be temporarily disabled." > > > > Please note, the average CPU usage and related warning are very > > > misleading. The mcycles value displayed in log is also very different > > > from the x.x times value showed in warning message. I used to think > > > that request using beyond 1800 mcycles CPU will be considered as a > > > high CPU request since it shows red in Average CPU column. But the > > > latest finding is that it's not the case. One kind of my requests use > > > about 6K-7K mcycles in average CPU, but I seldom see a high CPU > > > warning in log for them. Again, this mcycles value seems to mean > > > nothing and the algorithm used to calculate the x.x times high CPU > > > warning really matters! > > > As you have noticed, there are differences in the CPU usage > > measurements presented in the logs and in the dashboard. This thread > > contains clarification which I've copied below: > > >http://groups.google.com/group/google-appengine/browse_frm/thread/f21... > > "Concerning CPU accounting overall - The warning threshold for the > > logs > > and for the dashboard actually differ. In the logs, warnings will > > only > > occur for high runtime CPU. The warnings on the dashboard are > > triggered for > > high overall CPU, including CPU used in the runtime and per API, most > > notably being the datastore API." > > > Request denials for high CPU are based on the runtime CPU usage (which > > does not include the CPU used in API calls). Since the runtime CPU > > cost are the ones present in the logs, it is more useful to rely on > > the log information when trying to determine how best to reduce CPU > > load for specific requests. > > > Happy coding, > > > Jeff > > > > With this in mind, the focus is back to the warning message itself to > > > figure out what's the algorithm. So there is an "average request CPU > > > limit" in this message. This value seems to be quite big and only my > > > requests that are doing some work which should be done by a background > > > scheduler exceeds it. The question is, is this a fixed value? Or a > > > dynamic value calculated based on all apps? Or a dynamic value > > > calculated based on the app itself? > > > > And the number of such warnings has a fixed quota that doesn't scale > > > with other quotas. If all my assumption is right here, the best option > > > to solve this issue in my mind is to make the high CPU quota scale as > > > other quotas. > > > > On Sep 26, 1:22 am, mitnickcbc <[EMAIL PROTECTED]> wrote: > > > > > Anyone has any idea how this warning message is calculated? "Many of > > > > the requests to your application are taking a very long time. Please > > > > optimize these requests." > > > > > It seems I meet quota denials every time this one shows up. I'm unable > > > > to tune anymore if I don't know how it is calculated. > > > > > There are several possible algorithms for this in my mind and I need > > > > to know which one is used to be able
[google-appengine] Re: High Amount CPU Quota should be removed
> Request denials for high CPU are based on the runtime CPU usage (which > does not include the CPU used in API calls). Since the runtime CPU > cost are the ones present in the logs, it is more useful to rely on > the log information when trying to determine how best to reduce CPU > load for specific requests. Thanks for the clarification! Now I'm able to find the bottleneck and... it scales! On Sep 27, 2:00 am, Jeff S <[EMAIL PROTECTED]> wrote: > On Sep 26, 1:48 am, mitnickcbc <[EMAIL PROTECTED]> wrote: > > > > > Some update about my latest findings. And these are my guesses, please > > do correct me if I'm wrong. > > > The quota denial for high amount CPU and the warning message "Many of > > the requests to your application are taking a very long time. Please > > optimize these requests." is calculated based on the number of warning > > you get in your log saying that "This request used a high amount of > > CPU, and was roughly 1.0 times over the average request CPU limit. > > High CPU requests have a small quota, and if you exceed this quota, > > your app will be temporarily disabled." > > > Please note, the average CPU usage and related warning are very > > misleading. The mcycles value displayed in log is also very different > > from the x.x times value showed in warning message. I used to think > > that request using beyond 1800 mcycles CPU will be considered as a > > high CPU request since it shows red in Average CPU column. But the > > latest finding is that it's not the case. One kind of my requests use > > about 6K-7K mcycles in average CPU, but I seldom see a high CPU > > warning in log for them. Again, this mcycles value seems to mean > > nothing and the algorithm used to calculate the x.x times high CPU > > warning really matters! > > As you have noticed, there are differences in the CPU usage > measurements presented in the logs and in the dashboard. This thread > contains clarification which I've copied below: > > http://groups.google.com/group/google-appengine/browse_frm/thread/f21... > "Concerning CPU accounting overall - The warning threshold for the > logs > and for the dashboard actually differ. In the logs, warnings will > only > occur for high runtime CPU. The warnings on the dashboard are > triggered for > high overall CPU, including CPU used in the runtime and per API, most > notably being the datastore API." > > Request denials for high CPU are based on the runtime CPU usage (which > does not include the CPU used in API calls). Since the runtime CPU > cost are the ones present in the logs, it is more useful to rely on > the log information when trying to determine how best to reduce CPU > load for specific requests. > > Happy coding, > > Jeff > > > > > With this in mind, the focus is back to the warning message itself to > > figure out what's the algorithm. So there is an "average request CPU > > limit" in this message. This value seems to be quite big and only my > > requests that are doing some work which should be done by a background > > scheduler exceeds it. The question is, is this a fixed value? Or a > > dynamic value calculated based on all apps? Or a dynamic value > > calculated based on the app itself? > > > And the number of such warnings has a fixed quota that doesn't scale > > with other quotas. If all my assumption is right here, the best option > > to solve this issue in my mind is to make the high CPU quota scale as > > other quotas. > > > On Sep 26, 1:22 am, mitnickcbc <[EMAIL PROTECTED]> wrote: > > > > Anyone has any idea how this warning message is calculated? "Many of > > > the requests to your application are taking a very long time. Please > > > optimize these requests." > > > > It seems I meet quota denials every time this one shows up. I'm unable > > > to tune anymore if I don't know how it is calculated. > > > > There are several possible algorithms for this in my mind and I need > > > to know which one is used to be able to move forward: > > > 1. Based on the total number of high CPU requests in a certain period > > > of time. And what's the CPU usage value to make a request counted in? > > > 1200 mcycles? 1800, or anything else? > > > 2. Based on total CPU consumed by these high CPU. > > > 3. Based on response time. > > > > However, no matter how much I tuned, it's just about time for me to > > > reach the load limit. Since 30% of my write requests will never > > > possible to fit the current high CPU bar. It also seems that the high > > > CPU quota is not increased with the other quota. So say assume 10% of > > > your requests are high CPU ones, and GAE only allows 1 high CPU > > > request per second, then you will not be able to scale beyond 10 > > > requests per second, no matter how much CPU quota you have. > > > > There are 3 solutions in my mind: > > > 1. Raise the high CPU bar to fit real world needs like raising it to > > > 10K mcycles. > > > 2. Make the high amount CPU quota scalable and configurable instead of > > > a fixed number. Such as
[google-appengine] Re: High Amount CPU Quota should be removed
Only one question, I wonder how much CPU of google is spent in calculating high amount coutas. Pherahaps google should assign our application that cpu time. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
On Sep 26, 1:48 am, mitnickcbc <[EMAIL PROTECTED]> wrote: > Some update about my latest findings. And these are my guesses, please > do correct me if I'm wrong. > > The quota denial for high amount CPU and the warning message "Many of > the requests to your application are taking a very long time. Please > optimize these requests." is calculated based on the number of warning > you get in your log saying that "This request used a high amount of > CPU, and was roughly 1.0 times over the average request CPU limit. > High CPU requests have a small quota, and if you exceed this quota, > your app will be temporarily disabled." > > Please note, the average CPU usage and related warning are very > misleading. The mcycles value displayed in log is also very different > from the x.x times value showed in warning message. I used to think > that request using beyond 1800 mcycles CPU will be considered as a > high CPU request since it shows red in Average CPU column. But the > latest finding is that it's not the case. One kind of my requests use > about 6K-7K mcycles in average CPU, but I seldom see a high CPU > warning in log for them. Again, this mcycles value seems to mean > nothing and the algorithm used to calculate the x.x times high CPU > warning really matters! As you have noticed, there are differences in the CPU usage measurements presented in the logs and in the dashboard. This thread contains clarification which I've copied below: http://groups.google.com/group/google-appengine/browse_frm/thread/f210131b359b8fe6 "Concerning CPU accounting overall - The warning threshold for the logs and for the dashboard actually differ. In the logs, warnings will only occur for high runtime CPU. The warnings on the dashboard are triggered for high overall CPU, including CPU used in the runtime and per API, most notably being the datastore API." Request denials for high CPU are based on the runtime CPU usage (which does not include the CPU used in API calls). Since the runtime CPU cost are the ones present in the logs, it is more useful to rely on the log information when trying to determine how best to reduce CPU load for specific requests. Happy coding, Jeff > > With this in mind, the focus is back to the warning message itself to > figure out what's the algorithm. So there is an "average request CPU > limit" in this message. This value seems to be quite big and only my > requests that are doing some work which should be done by a background > scheduler exceeds it. The question is, is this a fixed value? Or a > dynamic value calculated based on all apps? Or a dynamic value > calculated based on the app itself? > > And the number of such warnings has a fixed quota that doesn't scale > with other quotas. If all my assumption is right here, the best option > to solve this issue in my mind is to make the high CPU quota scale as > other quotas. > > On Sep 26, 1:22 am, mitnickcbc <[EMAIL PROTECTED]> wrote: > > > > > Anyone has any idea how this warning message is calculated? "Many of > > the requests to your application are taking a very long time. Please > > optimize these requests." > > > It seems I meet quota denials every time this one shows up. I'm unable > > to tune anymore if I don't know how it is calculated. > > > There are several possible algorithms for this in my mind and I need > > to know which one is used to be able to move forward: > > 1. Based on the total number of high CPU requests in a certain period > > of time. And what's the CPU usage value to make a request counted in? > > 1200 mcycles? 1800, or anything else? > > 2. Based on total CPU consumed by these high CPU. > > 3. Based on response time. > > > However, no matter how much I tuned, it's just about time for me to > > reach the load limit. Since 30% of my write requests will never > > possible to fit the current high CPU bar. It also seems that the high > > CPU quota is not increased with the other quota. So say assume 10% of > > your requests are high CPU ones, and GAE only allows 1 high CPU > > request per second, then you will not be able to scale beyond 10 > > requests per second, no matter how much CPU quota you have. > > > There are 3 solutions in my mind: > > 1. Raise the high CPU bar to fit real world needs like raising it to > > 10K mcycles. > > 2. Make the high amount CPU quota scalable and configurable instead of > > a fixed number. Such as making it 30% of normal CPU quota. However, I > > would still suggest to not even fix the percentage since every > > application is different. Better to make it configurable by developer > > themselves. > > 3. Make high amount CPU quota same as other quotas like CPU and > > bandwidth. So developer can apply for it if they are running out of > > it. > > > I'm not sure which one is more doable to GAE, but there must be some > > solution to be happen otherwise it will not scale. > > > On Sep 21, 7:27 pm,mitnickcbc<[EMAIL PROTECTED]> wrote: > > > > I have filed a feature request for this issue, if
[google-appengine] Re: High Amount CPU Quota should be removed
Agree with you, man, CPU quota error is frequently for me, but just some query action for less than 100 records, for a python environment, we need more CPU power than static processing ability, so I think the way of the limitation should be adjusted. On Sep 21, 7:27 pm, mitnickcbc <[EMAIL PROTECTED]> wrote: > I have filed a feature request for this issue, if you feel so please > star it. > > http://code.google.com/p/googleappengine/issues/detail?id=720 > > Here is the description: > > Well, I don't get the purpose for having a High Amount CPU Quota since > there is already a general CPU Quota. And there are quite a lot of > problems caused by this quota which restrict my application from > affording more load. And my application is far away from a 5 million > month PV app. > 1. Requests can become a high amount CPU one very easily. A little > complex request will consume more than 3K mcycles. And what do I mean > a little > complex? If you do more than 2 put, or you do a url fetch, or you do a > query based on an order to select more than 50 model, or you do a put > on a > model more than 20 fields, it is that complex. In my case, 80% of the > requests are that complex and I don't think it can be optimized > anymore > since I do need to do put and url fetch. > 2. Bad visibility to this quota. There is no way we can know when we > will run out of quota for this. All we got is the warning in admin > console that > says our requests are using high amount CPU quota and will soon run > out of it. > 3. Doesn't back to normal quickly. I'm using a script to continue > fetching my data from production and do some offline process. > Unfortunately I fetch > too much in one call that I try to get 100 models at a time and which > makes the request a high CPU one. Soon, I meet quota deny. Then I stop > my script, > but after 1.5 hours, I still see quota deny for requests from my > users. The "Many of the requests to your application are taking a very > long time. > Please optimize these requests." warning keeps there even I have > stopped the script for so long time. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Some update about my latest findings. And these are my guesses, please do correct me if I'm wrong. The quota denial for high amount CPU and the warning message "Many of the requests to your application are taking a very long time. Please optimize these requests." is calculated based on the number of warning you get in your log saying that "This request used a high amount of CPU, and was roughly 1.0 times over the average request CPU limit. High CPU requests have a small quota, and if you exceed this quota, your app will be temporarily disabled." Please note, the average CPU usage and related warning are very misleading. The mcycles value displayed in log is also very different from the x.x times value showed in warning message. I used to think that request using beyond 1800 mcycles CPU will be considered as a high CPU request since it shows red in Average CPU column. But the latest finding is that it's not the case. One kind of my requests use about 6K-7K mcycles in average CPU, but I seldom see a high CPU warning in log for them. Again, this mcycles value seems to mean nothing and the algorithm used to calculate the x.x times high CPU warning really matters! With this in mind, the focus is back to the warning message itself to figure out what's the algorithm. So there is an "average request CPU limit" in this message. This value seems to be quite big and only my requests that are doing some work which should be done by a background scheduler exceeds it. The question is, is this a fixed value? Or a dynamic value calculated based on all apps? Or a dynamic value calculated based on the app itself? And the number of such warnings has a fixed quota that doesn't scale with other quotas. If all my assumption is right here, the best option to solve this issue in my mind is to make the high CPU quota scale as other quotas. On Sep 26, 1:22 am, mitnickcbc <[EMAIL PROTECTED]> wrote: > Anyone has any idea how this warning message is calculated? "Many of > the requests to your application are taking a very long time. Please > optimize these requests." > > It seems I meet quota denials every time this one shows up. I'm unable > to tune anymore if I don't know how it is calculated. > > There are several possible algorithms for this in my mind and I need > to know which one is used to be able to move forward: > 1. Based on the total number of high CPU requests in a certain period > of time. And what's the CPU usage value to make a request counted in? > 1200 mcycles? 1800, or anything else? > 2. Based on total CPU consumed by these high CPU. > 3. Based on response time. > > However, no matter how much I tuned, it's just about time for me to > reach the load limit. Since 30% of my write requests will never > possible to fit the current high CPU bar. It also seems that the high > CPU quota is not increased with the other quota. So say assume 10% of > your requests are high CPU ones, and GAE only allows 1 high CPU > request per second, then you will not be able to scale beyond 10 > requests per second, no matter how much CPU quota you have. > > There are 3 solutions in my mind: > 1. Raise the high CPU bar to fit real world needs like raising it to > 10K mcycles. > 2. Make the high amount CPU quota scalable and configurable instead of > a fixed number. Such as making it 30% of normal CPU quota. However, I > would still suggest to not even fix the percentage since every > application is different. Better to make it configurable by developer > themselves. > 3. Make high amount CPU quota same as other quotas like CPU and > bandwidth. So developer can apply for it if they are running out of > it. > > I'm not sure which one is more doable to GAE, but there must be some > solution to be happen otherwise it will not scale. > > On Sep 21, 7:27 pm,mitnickcbc<[EMAIL PROTECTED]> wrote: > > > I have filed a feature request for this issue, if you feel so please > > star it. > > >http://code.google.com/p/googleappengine/issues/detail?id=720 > > > Here is the description: > > > Well, I don't get the purpose for having a High Amount CPU Quota since > > there is already a general CPU Quota. And there are quite a lot of > > problems caused by this quota which restrict my application from > > affording more load. And my application is far away from a 5 million > > month PV app. > > 1. Requests can become a high amount CPU one very easily. A little > > complex request will consume more than 3K mcycles. And what do I mean > > a little > > complex? If you do more than 2 put, or you do a url fetch, or you do a > > query based on an order to select more than 50 model, or you do a put > > on a > > model more than 20 fields, it is that complex. In my case, 80% of the > > requests are that complex and I don't think it can be optimized > > anymore > > since I do need to do put and url fetch. > > 2. Bad visibility to this quota. There is no way we can know when we > > will run out of quota for this. All we got is the warning in admin > > console th
[google-appengine] Re: High Amount CPU Quota should be removed
Anyone has any idea how this warning message is calculated? "Many of the requests to your application are taking a very long time. Please optimize these requests." It seems I meet quota denials every time this one shows up. I'm unable to tune anymore if I don't know how it is calculated. There are several possible algorithms for this in my mind and I need to know which one is used to be able to move forward: 1. Based on the total number of high CPU requests in a certain period of time. And what's the CPU usage value to make a request counted in? 1200 mcycles? 1800, or anything else? 2. Based on total CPU consumed by these high CPU. 3. Based on response time. However, no matter how much I tuned, it's just about time for me to reach the load limit. Since 30% of my write requests will never possible to fit the current high CPU bar. It also seems that the high CPU quota is not increased with the other quota. So say assume 10% of your requests are high CPU ones, and GAE only allows 1 high CPU request per second, then you will not be able to scale beyond 10 requests per second, no matter how much CPU quota you have. There are 3 solutions in my mind: 1. Raise the high CPU bar to fit real world needs like raising it to 10K mcycles. 2. Make the high amount CPU quota scalable and configurable instead of a fixed number. Such as making it 30% of normal CPU quota. However, I would still suggest to not even fix the percentage since every application is different. Better to make it configurable by developer themselves. 3. Make high amount CPU quota same as other quotas like CPU and bandwidth. So developer can apply for it if they are running out of it. I'm not sure which one is more doable to GAE, but there must be some solution to be happen otherwise it will not scale. On Sep 21, 7:27 pm, mitnickcbc <[EMAIL PROTECTED]> wrote: > I have filed a feature request for this issue, if you feel so please > star it. > > http://code.google.com/p/googleappengine/issues/detail?id=720 > > Here is the description: > > Well, I don't get the purpose for having a High Amount CPU Quota since > there is already a general CPU Quota. And there are quite a lot of > problems caused by this quota which restrict my application from > affording more load. And my application is far away from a 5 million > month PV app. > 1. Requests can become a high amount CPU one very easily. A little > complex request will consume more than 3K mcycles. And what do I mean > a little > complex? If you do more than 2 put, or you do a url fetch, or you do a > query based on an order to select more than 50 model, or you do a put > on a > model more than 20 fields, it is that complex. In my case, 80% of the > requests are that complex and I don't think it can be optimized > anymore > since I do need to do put and url fetch. > 2. Bad visibility to this quota. There is no way we can know when we > will run out of quota for this. All we got is the warning in admin > console that > says our requests are using high amount CPU quota and will soon run > out of it. > 3. Doesn't back to normal quickly. I'm using a script to continue > fetching my data from production and do some offline process. > Unfortunately I fetch > too much in one call that I try to get 100 models at a time and which > makes the request a high CPU one. Soon, I meet quota deny. Then I stop > my script, > but after 1.5 hours, I still see quota deny for requests from my > users. The "Many of the requests to your application are taking a very > long time. > Please optimize these requests." warning keeps there even I have > stopped the script for so long time. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
In another post someone was exceeding the 5 second timeout with urlfetch trying to get an RSS feed for a Google Docs account. I am planning on using the Google Charts API and I wonder if I won't hit the same issue. However, this got me thinking that writing a service like the Charting API on GAE is quite improbable given what we presently know about CPU thresholds, however we have to presume that the charting API that google offers is scalable. I know the service is new and I don't mind the limits since I haven't actually been locked out yet and I assume things will get better. I would encourage Google to look at the CPU profiles for services like the charting API when thinking about what limits to set on GAE. On Tue, Sep 23, 2008 at 7:45 PM, Michael Hart <[EMAIL PROTECTED]>wrote: > > Hi all, > > Having been through a number of shared hosting situations, I'm gonna > have to say that I disagree with most of the complaints on this thread. > > I think one of the main things not to lose sight of is the fact that > GAE is intended to be a scalable platform, not a generic hosting > solution. If you want or need a dedicated amount of CPU or memory, > then GAE, at least as it stands today, is probably not what you want. > The limits that are imposed in GAE are there to ensure that your app > will scale. There are many apps that don't need to scale, and I would > suggest that it's probably going to be easier for these to be > developed on a more traditional platform. > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
I agree 100%. The high Amount CPU Quota for applications is to low except if you code something trivial like a string manipulation it is ok. But if you want to code a real end user application google app is not a reliable platform at this point because it's not possible to scale your needs. suggestion: Less campfire and more REAL quotas. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Thank you for digging into this issue, Michael. There are 8 indexes on Account to support different kind of query. 3 options of order, 2 options of filter generates 6 indexes then plus 2 other ones. Only one field is list and it contains very few items and almost never change. It's parent of another payment entity, and there are very few number of this entity. I do have a small test case which does 1 Account put, 1 small entity put, 1 url call, and it costs 2600 mcycles. Spliting Account is the thing I have not tested. And I'm not going to do so since it is quite risky. I need to handle the data migration, also may need to introduce duplicate fields. I also don't know even if I put those efforts to make this happen, whether I will really get much benefit out of it. But I may try this out in my new applications to strictly limit the size of entity. Best practice from GAE or any experienced expert is well welcomed here. On Sep 24, 3:29 pm, Michael Hart <[EMAIL PROTECTED]> wrote: > Just a couple of questions to clarify - how many indexes are there for > the Account entity and are all the fields single dimensional > properties, or are there lists in there too? And out of interest, is > it the parent of an entity group? > > I would agree that a single request that involves putting an entity > with 24 flat properties and simple indexes should not put you over > quota, and if this is what's happening, then perhaps it should be > looked at further (perhaps you could come up with a small test case). > > The only other question would be whether you can split the Account > into multiple entities so that fields that are updated frequently are > separated out and the entire entity doesn't need to be written each > time. > > On 24/09/2008, at 5:03 PM, mitnickcbc wrote: > > > > > Hi Michael, > > > Thanks for putting these thoughts and that eBay best practice article > > is awesome! It really helps me understand more about distribution > > system. It also convinced me that global transaction is not that > > important. And I have refined my code according to this article, but I > > didn't notice big improvements. And I have analyzed my request data > > carefully and I'd like to share some findings. > > > I have one Account entity which has 24 fields and this one seems to be > > the biggest issue. 30% of my requests have similar complexity. They do > > 2 - 3 Account puts, 4 small entity (4 fields) puts, and 2 URL calls to > > Facebook. All these requests are consuming 6K - 9K mcycles. By > > comparing different URL calls, I notice this data: > > > 1 Account put uses about 1800 mcycles. > > 3 small puts plus 1 url call uses about 2000 mcycles. > > 1 small put uses about 600 mcycles (estimated). > > > Your average CPU turns into orange when beyond 1200 mcycles, and turns > > into red when beyond around 2000 mcycles. > > > I don't know what you feel, but I think this "high amount" definition > > is too low. If this is not too low, that's fine to warn us. But add > > quota to this will kick out a lot of applications. > > > On Sep 24, 8:45 am, Michael Hart <[EMAIL PROTECTED]> wrote: > >> Hi all, > > >> Having been through a number of shared hosting situations, I'm gonna > >> have to say that I disagree with most of the complaints on this > >> thread. > > >> I think one of the main things not to lose sight of is the fact that > >> GAE is intended to be a scalable platform, not a generic hosting > >> solution. If you want or need a dedicated amount of CPU or memory, > >> then GAE, at least as it stands today, is probably not what you want. > >> The limits that are imposed in GAE are there to ensure that your app > >> will scale. There are many apps that don't need to scale, and I would > >> suggest that it's probably going to be easier for these to be > >> developed on a more traditional platform. > > >> Global transactions don't scale (if eBay doesn't need them, than > >> mostly likely neither will > >> youhttp://www.infoq.com/articles/ebay-scalability-best-practices) > >> . Consistent CPU spikes don't scale. Long request times don't scale - > >> at least, not particularly well. If each of your requests are taking, > >> say 10 seconds to fetch a page from another URL and you get hammered > >> with a few hundred requests a second, then I don't want my app on the > >> same box as yours (and no doubt the site you're fetching from won't > >> be > >> happy either). Now if Google introduce pricing that guarantees you a > >> certain amount of memory or CPU or I/O on a box ala EC2, then that's > >> fine, but until then (or if that doesn't happen), it's all about > >> being > >> a good cloud citizen. > > >> Scalability isn't easy. It's made easier with tools like BigTable and > >> Memcache, but you still have to use them correctly and they will > >> impose certain limitations on the way you develop. If you develop an > >> app from scratch on EC2 or whatever VPS/dedicated hosting solution > >> you > >> like
[google-appengine] Re: High Amount CPU Quota should be removed
Just a couple of questions to clarify - how many indexes are there for the Account entity and are all the fields single dimensional properties, or are there lists in there too? And out of interest, is it the parent of an entity group? I would agree that a single request that involves putting an entity with 24 flat properties and simple indexes should not put you over quota, and if this is what's happening, then perhaps it should be looked at further (perhaps you could come up with a small test case). The only other question would be whether you can split the Account into multiple entities so that fields that are updated frequently are separated out and the entire entity doesn't need to be written each time. On 24/09/2008, at 5:03 PM, mitnickcbc wrote: > > Hi Michael, > > Thanks for putting these thoughts and that eBay best practice article > is awesome! It really helps me understand more about distribution > system. It also convinced me that global transaction is not that > important. And I have refined my code according to this article, but I > didn't notice big improvements. And I have analyzed my request data > carefully and I'd like to share some findings. > > I have one Account entity which has 24 fields and this one seems to be > the biggest issue. 30% of my requests have similar complexity. They do > 2 - 3 Account puts, 4 small entity (4 fields) puts, and 2 URL calls to > Facebook. All these requests are consuming 6K - 9K mcycles. By > comparing different URL calls, I notice this data: > > 1 Account put uses about 1800 mcycles. > 3 small puts plus 1 url call uses about 2000 mcycles. > 1 small put uses about 600 mcycles (estimated). > > Your average CPU turns into orange when beyond 1200 mcycles, and turns > into red when beyond around 2000 mcycles. > > I don't know what you feel, but I think this "high amount" definition > is too low. If this is not too low, that's fine to warn us. But add > quota to this will kick out a lot of applications. > > > On Sep 24, 8:45 am, Michael Hart <[EMAIL PROTECTED]> wrote: >> Hi all, >> >> Having been through a number of shared hosting situations, I'm gonna >> have to say that I disagree with most of the complaints on this >> thread. >> >> I think one of the main things not to lose sight of is the fact that >> GAE is intended to be a scalable platform, not a generic hosting >> solution. If you want or need a dedicated amount of CPU or memory, >> then GAE, at least as it stands today, is probably not what you want. >> The limits that are imposed in GAE are there to ensure that your app >> will scale. There are many apps that don't need to scale, and I would >> suggest that it's probably going to be easier for these to be >> developed on a more traditional platform. >> >> Global transactions don't scale (if eBay doesn't need them, than >> mostly likely neither will >> youhttp://www.infoq.com/articles/ebay-scalability-best-practices) >> . Consistent CPU spikes don't scale. Long request times don't scale - >> at least, not particularly well. If each of your requests are taking, >> say 10 seconds to fetch a page from another URL and you get hammered >> with a few hundred requests a second, then I don't want my app on the >> same box as yours (and no doubt the site you're fetching from won't >> be >> happy either). Now if Google introduce pricing that guarantees you a >> certain amount of memory or CPU or I/O on a box ala EC2, then that's >> fine, but until then (or if that doesn't happen), it's all about >> being >> a good cloud citizen. >> >> Scalability isn't easy. It's made easier with tools like BigTable and >> Memcache, but you still have to use them correctly and they will >> impose certain limitations on the way you develop. If you develop an >> app from scratch on EC2 or whatever VPS/dedicated hosting solution >> you >> like and you have the need to scale, then you'll need to start >> thinking about these issues - issues that GAE force you to think >> about >> upfront. Maybe Google should make it mandatory to read a Scalability >> 101 paper before signing up - at least that way it's clear that this >> platform is different to most standard platforms and thus reduce >> apple/ >> orange comparisons. >> >> Note that I'm not adverse to Google introducing something similar to >> EC2 for processing services - you could purchase a particular slice >> that your GAE app could communicate with using Protocol Buffers or >> whatever - or maybe you could run a bunch of map-reduce tasks. But >> these require dedicated resources and should be kept well away from >> the GAE serving boxes. Similarly there could be an S3-like storage >> solution, as well as a message bus ala SQS. For many large systems, >> these facilities are essential - and in time, Google may introduce >> them. >> >> But until then, Amazon's services (or anyone else's) are fine. You >> want storage? Go S3. You want to run background tasks? Go EC2 or some >> other VPS host.
[google-appengine] Re: High Amount CPU Quota should be removed
Hi Michael, Thanks for putting these thoughts and that eBay best practice article is awesome! It really helps me understand more about distribution system. It also convinced me that global transaction is not that important. And I have refined my code according to this article, but I didn't notice big improvements. And I have analyzed my request data carefully and I'd like to share some findings. I have one Account entity which has 24 fields and this one seems to be the biggest issue. 30% of my requests have similar complexity. They do 2 - 3 Account puts, 4 small entity (4 fields) puts, and 2 URL calls to Facebook. All these requests are consuming 6K - 9K mcycles. By comparing different URL calls, I notice this data: 1 Account put uses about 1800 mcycles. 3 small puts plus 1 url call uses about 2000 mcycles. 1 small put uses about 600 mcycles (estimated). Your average CPU turns into orange when beyond 1200 mcycles, and turns into red when beyond around 2000 mcycles. I don't know what you feel, but I think this "high amount" definition is too low. If this is not too low, that's fine to warn us. But add quota to this will kick out a lot of applications. On Sep 24, 8:45 am, Michael Hart <[EMAIL PROTECTED]> wrote: > Hi all, > > Having been through a number of shared hosting situations, I'm gonna > have to say that I disagree with most of the complaints on this thread. > > I think one of the main things not to lose sight of is the fact that > GAE is intended to be a scalable platform, not a generic hosting > solution. If you want or need a dedicated amount of CPU or memory, > then GAE, at least as it stands today, is probably not what you want. > The limits that are imposed in GAE are there to ensure that your app > will scale. There are many apps that don't need to scale, and I would > suggest that it's probably going to be easier for these to be > developed on a more traditional platform. > > Global transactions don't scale (if eBay doesn't need them, than > mostly likely neither will > youhttp://www.infoq.com/articles/ebay-scalability-best-practices) > . Consistent CPU spikes don't scale. Long request times don't scale - > at least, not particularly well. If each of your requests are taking, > say 10 seconds to fetch a page from another URL and you get hammered > with a few hundred requests a second, then I don't want my app on the > same box as yours (and no doubt the site you're fetching from won't be > happy either). Now if Google introduce pricing that guarantees you a > certain amount of memory or CPU or I/O on a box ala EC2, then that's > fine, but until then (or if that doesn't happen), it's all about being > a good cloud citizen. > > Scalability isn't easy. It's made easier with tools like BigTable and > Memcache, but you still have to use them correctly and they will > impose certain limitations on the way you develop. If you develop an > app from scratch on EC2 or whatever VPS/dedicated hosting solution you > like and you have the need to scale, then you'll need to start > thinking about these issues - issues that GAE force you to think about > upfront. Maybe Google should make it mandatory to read a Scalability > 101 paper before signing up - at least that way it's clear that this > platform is different to most standard platforms and thus reduce apple/ > orange comparisons. > > Note that I'm not adverse to Google introducing something similar to > EC2 for processing services - you could purchase a particular slice > that your GAE app could communicate with using Protocol Buffers or > whatever - or maybe you could run a bunch of map-reduce tasks. But > these require dedicated resources and should be kept well away from > the GAE serving boxes. Similarly there could be an S3-like storage > solution, as well as a message bus ala SQS. For many large systems, > these facilities are essential - and in time, Google may introduce them. > > But until then, Amazon's services (or anyone else's) are fine. You > want storage? Go S3. You want to run background tasks? Go EC2 or some > other VPS host. Your GAE app will talk to these services almost the > same as it would have to if they were hosted at Google anyway. Sure, > there'd be less latency and for some apps that may make all the > difference - but for most, the issues will lie elsewhere. > > So basically, push through the pain, rethink your designs, and you'll > come out with a well-engineered app that can scale like nobody's > business. (well, nobody's business but your own multi-million dollar > generating Web 3.5 Überapp that is) > > Cheers, > > Michael --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit th
[google-appengine] Re: High Amount CPU Quota should be removed
Hi all, Having been through a number of shared hosting situations, I'm gonna have to say that I disagree with most of the complaints on this thread. I think one of the main things not to lose sight of is the fact that GAE is intended to be a scalable platform, not a generic hosting solution. If you want or need a dedicated amount of CPU or memory, then GAE, at least as it stands today, is probably not what you want. The limits that are imposed in GAE are there to ensure that your app will scale. There are many apps that don't need to scale, and I would suggest that it's probably going to be easier for these to be developed on a more traditional platform. Global transactions don't scale (if eBay doesn't need them, than mostly likely neither will you http://www.infoq.com/articles/ebay-scalability-best-practices) . Consistent CPU spikes don't scale. Long request times don't scale - at least, not particularly well. If each of your requests are taking, say 10 seconds to fetch a page from another URL and you get hammered with a few hundred requests a second, then I don't want my app on the same box as yours (and no doubt the site you're fetching from won't be happy either). Now if Google introduce pricing that guarantees you a certain amount of memory or CPU or I/O on a box ala EC2, then that's fine, but until then (or if that doesn't happen), it's all about being a good cloud citizen. Scalability isn't easy. It's made easier with tools like BigTable and Memcache, but you still have to use them correctly and they will impose certain limitations on the way you develop. If you develop an app from scratch on EC2 or whatever VPS/dedicated hosting solution you like and you have the need to scale, then you'll need to start thinking about these issues - issues that GAE force you to think about upfront. Maybe Google should make it mandatory to read a Scalability 101 paper before signing up - at least that way it's clear that this platform is different to most standard platforms and thus reduce apple/ orange comparisons. Note that I'm not adverse to Google introducing something similar to EC2 for processing services - you could purchase a particular slice that your GAE app could communicate with using Protocol Buffers or whatever - or maybe you could run a bunch of map-reduce tasks. But these require dedicated resources and should be kept well away from the GAE serving boxes. Similarly there could be an S3-like storage solution, as well as a message bus ala SQS. For many large systems, these facilities are essential - and in time, Google may introduce them. But until then, Amazon's services (or anyone else's) are fine. You want storage? Go S3. You want to run background tasks? Go EC2 or some other VPS host. Your GAE app will talk to these services almost the same as it would have to if they were hosted at Google anyway. Sure, there'd be less latency and for some apps that may make all the difference - but for most, the issues will lie elsewhere. So basically, push through the pain, rethink your designs, and you'll come out with a well-engineered app that can scale like nobody's business. (well, nobody's business but your own multi-million dollar generating Web 3.5 Überapp that is) Cheers, Michael --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
I apologize if people don't feel good by the tone in my posts. My whole point is to let GAE folks know that where is the pain from me with GAE and that can help them improve GAE better. I believe I'm one the the eagerest developers here who want to see a great success for GAE. To see unhappy comments earlier is to prevent more and worse tone comments later after release. GAE is in preview status. So I will never ask Google to be responsible for any unreliability or unstability. They can ignore my comments and requests and it's also fine I don't get any response from them. I take my own bet and risk. I do the bet because I saw Google start using GAE for their live product like Google Code Jam Site. Everyone has a restriction in resource and budget, and GAE is the preference for me even if it is in preview status. Keep telling GAE that how happy I'm will not improve the platform. Most people will not meet the issues mentioned in this thread if they are just leveraging this platform. And most people will not do load test here. And most people don't face real users using GAE. But I'm sure the goal is to keep live product developers happy with GAE. And I would like to reemphasize the point of this thread so people don't lose focus: the unvisiable and unpredictable high amount CPU quota is an issue. My suggestion is just remove it since there is already a CPU quota. Update: I didn't see quota deny for 36 hours even my load is getting higher than before, which is great! For anyone has done anything for me, I really appreciate your help! On Sep 24, 2:42 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > I think there are some things people need to keep in mind. > > GAE is currently in a preview release with no announced date for when > it will be ready for production use. Google has committed to try to > get it up by year end, based on things said at conferences. That's it. > You really should not build anything that's "live" production right > now on it. > > There is a big difference between criticism and constructive > criticism, and there is more of the former in this thread. It may be > true that the quotas are too strict, but bashing them for making them > too strict is going to increase the change that they'll loosen them > up. Personally, if I was working on releasing something to the scale > of appengine to the public, my initial release would have strict > quotas and then leave me in a position to loosen them in order to meet > demand, rather than being too easy with the quotas and being forced to > make them stricter in order to support load. > > As far as the memcache and whether developers will use it or not, I'd > suggest that you should use the tools available to you. Maybe I come > from different era of development and am jaded by over a decade of > working a sysadmin for organizations with tight IT budgets. I have the > belief that a developer should always optimize their code in order to > have it run as cleanly and optimal as possible in order to cut > hardware costs, rather than taking the shortest route to meet a > deadline which in the long run increases operational costs when it > scales beyond current hardware. Caching is always a good thing, rather > than repeatedly doing the same computation over and over. > > Not to toot my own whistle, but if you don't have the time or > inclination to write your own cache handler to take advantage of > memcache, then take a look at the one I wrote > at:http://gaeutilities.appspot.com/cache > > It can be used like any other dictionary object and uses both memcache > and the datastore to store your results for both reliability and > performance. There's also several others out there as well, some using > just memcache, and others just using the datastore. > > This is a preview release and you are not currently paying for the > product. I think you should consider that when you consider the tone > in which you write your messages. Hopefully the developers and > decision makers realize that there is a vocal minority and there and > many more of us happily developing on the platform and taking > advantage of what it has to offer. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
I think there are some things people need to keep in mind. GAE is currently in a preview release with no announced date for when it will be ready for production use. Google has committed to try to get it up by year end, based on things said at conferences. That's it. You really should not build anything that's "live" production right now on it. There is a big difference between criticism and constructive criticism, and there is more of the former in this thread. It may be true that the quotas are too strict, but bashing them for making them too strict is going to increase the change that they'll loosen them up. Personally, if I was working on releasing something to the scale of appengine to the public, my initial release would have strict quotas and then leave me in a position to loosen them in order to meet demand, rather than being too easy with the quotas and being forced to make them stricter in order to support load. As far as the memcache and whether developers will use it or not, I'd suggest that you should use the tools available to you. Maybe I come from different era of development and am jaded by over a decade of working a sysadmin for organizations with tight IT budgets. I have the belief that a developer should always optimize their code in order to have it run as cleanly and optimal as possible in order to cut hardware costs, rather than taking the shortest route to meet a deadline which in the long run increases operational costs when it scales beyond current hardware. Caching is always a good thing, rather than repeatedly doing the same computation over and over. Not to toot my own whistle, but if you don't have the time or inclination to write your own cache handler to take advantage of memcache, then take a look at the one I wrote at: http://gaeutilities.appspot.com/cache It can be used like any other dictionary object and uses both memcache and the datastore to store your results for both reliability and performance. There's also several others out there as well, some using just memcache, and others just using the datastore. This is a preview release and you are not currently paying for the product. I think you should consider that when you consider the tone in which you write your messages. Hopefully the developers and decision makers realize that there is a vocal minority and there and many more of us happily developing on the platform and taking advantage of what it has to offer. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
One of the concerns is that very simple actions, a post to a small model with a single put is throwing warnings. So the only think you can do is reduce the amount of data in the post or break the post into multiple smaller posts. You should end up using more cpu with the latter but that might not throw any warnings. On Tue, Sep 23, 2008 at 12:31 PM, Bill <[EMAIL PROTECTED]> wrote: > > > Imagine the potential consequences if any app could consume as much > > resources as it sees fit at any time from that daily quota it has. > > Peak load could (and would) go up unpredictably, and that would > > potentially cause problems even to apps that are more optimized than > > the CPU hogs. Remeber that this is a shared system, so these barriers > > are necessary. > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
> Imagine the potential consequences if any app could consume as much > resources as it sees fit at any time from that daily quota it has. > Peak load could (and would) go up unpredictably, and that would > potentially cause problems even to apps that are more optimized than > the CPU hogs. Remeber that this is a shared system, so these barriers > are necessary. A main point of App Engine is to let any app potentially consume as much resources as it needs at any time whether it's from a slashdotting or a lower volume of high CPU requests. I look at the High Amount CPU quota warnings as an information service to help tune your app. After pay-as-you-go is released, these warnings should be purely informational IMHO. High CPU requests will cause slow responses, which is a reason to tune those requests, but you shouldn't get an error. I'm hitting the High Amount CPU warnings on a site that's not even close to daily quotas. It's been helpful to see the flags for specific URLs so I can tune them. For example, I've got an AJAX client that loads in JSON data for a specific "workshop" model. My current model can produce, in one request, the JSON for all submissions in all meetings for that workshop, so it goes three models deep in queries. Now that I see the bottleneck, I'm making the data transfer less chunky. So my AJAX client will issue separate GETs for the models, which is one way around High CPU requests -- make the transactions more granular and use AJAX on the client-side to piece together the supplied data. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Hopefully the initial complaint is not lost in the emotion of some of the posts. An application which performs 200 read requests per day per user, with 1000 users at 20-50K per read would likely get a ton of warnings and may even be shut down. The assumption being made by Google appears to be with the intent of building everything so that it can scale to tens of thousands of users/requests but some applications will just never be that large. My idea would be to increase the thresholds significantly for classes or urls' that require user login. There is almost no chance an application would get slashdotted if the costly requests required login. This would give google time to see that there was significant interest growing in an application that might have trouble scaling and notify the owner. I think this would achieve a greater level of flexibility. On Tue, Sep 23, 2008 at 9:33 AM, Wooble <[EMAIL PROTECTED]> wrote: > > > > On Sep 23, 12:15 am, gg <[EMAIL PROTECTED]> wrote: > > Agreed. The problem at Google is very simple and has two components. > > First, the engineers run the show and there is no rational voice at > > Google to point out to them when they do something that does not meet > > a real world demand (they make something stupid). And the second > > component is they still have a lot of money to blow so they can get > > away making things that do not meet market demand (writing stuff that > > does not work and is stupid). > > If they're so stupid, you're free to start a competing service and > steal all of their business. That's how the free market works. > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
On Sep 23, 12:15 am, gg <[EMAIL PROTECTED]> wrote: > Agreed. The problem at Google is very simple and has two components. > First, the engineers run the show and there is no rational voice at > Google to point out to them when they do something that does not meet > a real world demand (they make something stupid). And the second > component is they still have a lot of money to blow so they can get > away making things that do not meet market demand (writing stuff that > does not work and is stupid). If they're so stupid, you're free to start a competing service and steal all of their business. That's how the free market works. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Google should start charging for the service and let customers set their own quotas. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Thank you for addressing these questions. Please check my comments. > I understand that this is bad for your business, but as stated by > Google, this is a preview release off App Engine. Why would one run a > production system on it? The thing is I didn't have resources to maintain scalable servers on AWS and with GAE available, it gives me the possibility to run business. And GAE is so attractive regarding of features and it does save me lots of effort. I didn't expect my application becomes a real business so fast. I thought 5 million PV per month will take me several months to reach it and actually I'm very far from this number today. My application is not complex, and I initially just wanted to test out the GAE platform. And since I have no idea whether my application will become popular or not, free start with GAE is also preferred. And it didn't make sense to not release it to face the real users after everything is ready and works so well at first since it is a new application. Since there are many rumors that GAE will be released by end of this year, there is no point for me to build totally another system by the meantime. Even if now I want to switch to other platform, I need to build up a totally different system, learn all the server and load balance configures. Not mention how to get the data out of GAE. This really doesn't buy me anything. > Imagine the potential consequences if any app could consume as much > resources as it sees fit at any time from that daily quota it has. > Peak load could (and would) go up unpredictably, and that would > potentially cause problems even to apps that are more optimized than > the CPU hogs. Remeber that this is a shared system, so these barriers > are necessary. I cannot comment much on this since I don't know the details about how GAE is implemented and how much impact it would be if GAE remove the high amount CPU quota. I would love to see some person from GAE Team to tell me about the difficulties about removing this quota or the reasons they keep it. And the high amount CPU is nothing related with the peak load. We can use up to the daily quota if all our requests are not high CPU consuming. So with or without high CPU quota, it is the same unpredictably as you described. For the normal app be affected by other CPU hogs, I think it depends on GAE request dispatching algorithm. There should be some way to monitor how much CPU each app consumed for each request and GAE can send less requests to each machine hosting this app while hosting it on more machines. By doing this way, each app consumes the same CPU on one machine. It also costs the same since user quota and charge are based on CPU usage instead of request number. But if high amount CPU barrier is existing for preventing normal app be affected by CPU hogs, maybe there is a design issue in GAE since real-world applications do need to do high amount CPU computations. Without removing this barrier, it's hard to meet the market needs. > > Need Five: Don't expect developers will use memcache more than the > > simplest pattern. > > Isn't that exactly what it's meant for? It is supposed to. But I will never get my request out of the high CPU category if I don't do complex algorithms with memcache. Simply cache the query result is not enough. And I'm not going to do this algorithm since it is error-prone. On Sep 23, 6:41 pm, Jonk <[EMAIL PROTECTED]> wrote: > On 23 syys, 12:08, mitnickcbc <[EMAIL PROTECTED]> wrote: > > Howdy. > > > Because I'm a full time developer working on GAE application and > > running a serious LIVE business application on GAE, and my users > > already started to pay money for this service! > > I understand that this is bad for your business, but as stated by > Google, this is a preview release off App Engine. Why would one run a > production system on it? > > > Need Four: High amount CPU Quota issue needs to be solved ASAP! > > Suggestion from me is just remove it and let user uses the 100% normal > > CPU Quota as the way they want. > > Imagine the potential consequences if any app could consume as much > resources as it sees fit at any time from that daily quota it has. > Peak load could (and would) go up unpredictably, and that would > potentially cause problems even to apps that are more optimized than > the CPU hogs. Remeber that this is a shared system, so these barriers > are necessary. > > > Need Five: Don't expect developers will use memcache more than the > > simplest pattern. > > Isn't that exactly what it's meant for? > > j --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
On 23 syys, 12:08, mitnickcbc <[EMAIL PROTECTED]> wrote: Howdy. > Because I'm a full time developer working on GAE application and > running a serious LIVE business application on GAE, and my users > already started to pay money for this service! I understand that this is bad for your business, but as stated by Google, this is a preview release off App Engine. Why would one run a production system on it? > Need Four: High amount CPU Quota issue needs to be solved ASAP! > Suggestion from me is just remove it and let user uses the 100% normal > CPU Quota as the way they want. Imagine the potential consequences if any app could consume as much resources as it sees fit at any time from that daily quota it has. Peak load could (and would) go up unpredictably, and that would potentially cause problems even to apps that are more optimized than the CPU hogs. Remeber that this is a shared system, so these barriers are necessary. > Need Five: Don't expect developers will use memcache more than the > simplest pattern. Isn't that exactly what it's meant for? j --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
So I would like to list some of the needs that real-world business developer expected from GAE. Why I have the right to say like this? Because I'm a full time developer working on GAE application and running a serious LIVE business application on GAE, and my users already started to pay money for this service! Story started when I first noticed GAE's quotations: Built on Google's infrastructure, Easy scale, Easy Deploy and Free to start. Cool! It looks great! That's all I needed! Then I started to learn Python, read every article in GAE Docs and some samples, and started coding! Coding with Python and web framework is fun, I really feel improvements in productivity with GAE SDK and Python since I used to work on Java. Soon I started coding, then I realize some functionality cannot be implemented on GAE, like background scheduler and global transaction for user to user trading. For background scheduler, I try to make a workaround using user requests to update data (which causes quite big overhead for CPU usage and lead to the biggest issue about CPU Quota I will describe later). For user to user trading, I do the "add" work first and "minus" work second to make sure user will not complain that they spend money and get nothing! So who meet the transaction issue, who is the lucky guy to get my free bonus... So here comes Need One and Need Two: Need One: Background scheduler support to be able to perform high CPU tasks for long time. Need Two: Global transaction support to be able to keep data integrity. Soon the coding finished, deployment is really easy! Within 10 minutes I can get my application live! Then I did little marketing and some users started to use my application and I am happy to see it works smoothly. So far so good, everything is great. Every morning I felt excited and got up early and cannot wait to check how much more users have come. I also got some feature requests, and I can manage them quickly since there is nearly no effort for deployment. Thanks GAE for letting me have a such wonderful life! After several days, my business logic is getting more complex due to user requests. Then I started to notice increasing error rate for datastore. Most errors are saying "operation took too long" or "Timeout". This is caused by saving a 30 fields entity. The error rate is still acceptable but I decided that I am not going to have entity with more than 20 fields in future. Need Three: We are expecting lower error rate on datastore for big entities or at least suggest some best practice here. Since this application is getting popular, the nightmare is coming. Quota Deny! It's not from any known quota described by GAE like CPU, URL Fetch, Datastore api call, bandwith, etc. It's called High amount CPU Quota! And unfortunately, 80% of my requests are high CPU ones. Every time my users are playing high on my application, the quota deny put them down. No access for 10 minutes but just a 403 error page. While the CPU usage on Admin Console is just showing 0% - 2% usage. I will skip comments about high amount CPU here since this thread already has a lot of inputs about this issue. It makes me hard to sleep well since I'm very worried that how much quota denies I have met during the night... Need Four: High amount CPU Quota issue needs to be solved ASAP! Suggestion from me is just remove it and let user uses the 100% normal CPU Quota as the way they want. And last weekend, I'm trying to reduce the amount of high CPU request by using memcache. It helps some since I cached some URL fetch and common queries. But 75-80% of my requests are still in high CPU category. It is possible to write some complex algorithm to optimize my requests further, but that will cause huge issues for readability and very error-prone. Need Five: Don't expect developers will use memcache more than the simplest pattern. Well, sounds like I have big problems, then why I have so much time to write such a long post? Because I can do absolutely nothing when I met quota denies during weekend except waiting to see a response about quota increase! Need Six: Give us some control over quota like quota configuration in Admin Console! Further thinking: Good visibility into all quotas are very important to let us apply quota increase in time to keep our application reliable and available. There is no doubt GAE can scale, and I also believe it will start a new age of cloud computing. But without scale easily and quickly (Currently mainly because of process instead of technical), GAE has no advantage over other competitors. Summarized here for the needs I mentioned, and please star the issues you feel important: Need One: Background scheduler support to be able to perform high CPU tasks for long time. http://code.google.com/p/googleappengine/issues/detail?id=6 Need Two: Global transaction support to be able to keep data integrity. http://code.google.com/p/googleappengine/issues/detail?id=313 Need Three: We are expecting lower error
[google-appengine] Re: High Amount CPU Quota should be removed
On Sep 21, 10:29 am, रवींदर ठाकुर (ravinder thakur) <[EMAIL PROTECTED]> wrote: > I think this is the biggest roadblock that will be preventing anyone > to develop and deploy any real world apps based on appengine.The > calculation of these warning limits is so unpredictable so how can one > be supposed to optimize the queries for these limits ? Agreed. The problem at Google is very simple and has two components. First, the engineers run the show and there is no rational voice at Google to point out to them when they do something that does not meet a real world demand (they make something stupid). And the second component is they still have a lot of money to blow so they can get away making things that do not meet market demand (writing stuff that does not work and is stupid). These two problems are endemic at Google. And the only thing to cure it is some red ink on the books. An example of the problem as lined out above that comes to mind is Google's Carrier Calculated Shipping API in Google Checkout which was released exactly one year ago. Carrier Calculated Shipping as described here: http://googlecheckoutapi.blogspot.com/2007/09/carrier-calculated-shipping-is-here.html " is to provide buyers with shipping rates for the major shipping carriers." This allows merchants to provide shipping rates to buyers when they provide information about the shipments for Google Checkout Transactions. It does provide rates. Rates that are 30% or more off the actual real rates from UPS, FedEx or USPS. It seems that the Google Engineers have designed an Algorithm to compute the rates instead of using the published web services from the carriers which give accurate results. I am sure the Algorithm required many hours of time from the best of Google Engineers. And I am sure the engineers are sure they have coded the best shipping cost predictive algorithm known to man, with only a +/- 30% margin of error! Don't question the fact that a merchant may lose $ on every transaction with low shipping costs or scare away customers with overly high shipping cost. The Google Engineers are smarter than you! They have designed the best Algorithm in the world to compute shipping costs. +/- 30% is acceptable to them and no one in the world could do better. It is the same with Google App Engine. The Google engineers have designed the ultimate computing platform. Who are we to question the fact that it can not repeatedly return a reasonably sized result set without displaying an "Over Quote" error page. You just do not know how to use it! There is a demand for free/low cost reliable scalable cloud computing services. Even those that run only python web frameworks. There is not a demand for a cloud computing service that can not repeatedly return a reasonably sized result set without undo amounts of caching or other machinations to get it to happen. Nor is there a demand for a checkout system that returns wildly inaccurate shipping quotes! I am sure the Engineers will all be very puzzled when they try to charge for Google App Engine and no one is willing to pay. Just remember that they are smarter than you and their solution is the best. Even if you would never pay for it. For more info on the Joke that is Carrier Calculated Shipping checkot the Google Checkout forum: http://groups.google.com/group/google-checkout-api-troubleshooting/search?group=google-checkout-api-troubleshooting&q=carrier+calculated+shipping&qt_g=Search+this+category --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Agree 100%. All I am doing is posting 10-20K every now and then, say every 15 minutes and I get warnings with every post, only 1 or 2 puts. I have tested my app for a few days and all my daily quotas stay at near 0%. On Sun, Sep 21, 2008 at 6:27 AM, mitnickcbc <[EMAIL PROTECTED]> wrote: > > I have filed a feature request for this issue, if you feel so please > star it. > > http://code.google.com/p/googleappengine/issues/detail?id=720 > > Here is the description: > > Well, I don't get the purpose for having a High Amount CPU Quota since > there is already a general CPU Quota. And there are quite a lot of > problems caused by this quota which restrict my application from > affording more load. And my application is far away from a 5 million > month PV app. > 1. Requests can become a high amount CPU one very easily. A little > complex request will consume more than 3K mcycles. And what do I mean > a little > complex? If you do more than 2 put, or you do a url fetch, or you do a > query based on an order to select more than 50 model, or you do a put > on a > model more than 20 fields, it is that complex. In my case, 80% of the > requests are that complex and I don't think it can be optimized > anymore > since I do need to do put and url fetch. > 2. Bad visibility to this quota. There is no way we can know when we > will run out of quota for this. All we got is the warning in admin > console that > says our requests are using high amount CPU quota and will soon run > out of it. > 3. Doesn't back to normal quickly. I'm using a script to continue > fetching my data from production and do some offline process. > Unfortunately I fetch > too much in one call that I try to get 100 models at a time and which > makes the request a high CPU one. Soon, I meet quota deny. Then I stop > my script, > but after 1.5 hours, I still see quota deny for requests from my > users. The "Many of the requests to your application are taking a very > long time. > Please optimize these requests." warning keeps there even I have > stopped the script for so long time. > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
I think this is the biggest roadblock that will be preventing anyone to develop and deploy any real world apps based on appengine.The calculation of these warning limits is so unpredictable so how can one be supposed to optimize the queries for these limits ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Starred it too. Big issue here. - fetch an url + a put() -> 2000 mcycles and more - fetch(100) (+order, filter...) -> 2000 mcycles and more - use import.zip (level = 6 ) -> 2000 mcycles and more - GQL restriction then you need to order by python -> 2000 mcycles and more - etc,... This quota is really too low : Free Quota HTTP Requests per Day = 650,000 CPU Megacycles per Day = 200,000,000 So the average mcycle/request = (200 000 000 / 650 000) = 300 mcycles. 300 mcycles/req is really low compared to the cost of a basic operation (put, fetch,...). Another things : - I really wonder how it is calculated. As I have always written in another post, it seems that there is no clear relationship between the profiler results and the number of mcycles. You can have a low profiler score and have a warning (CPU quota) and a big profiler score and no warning. So how to optimize ? - It seems that this quota is calculated twice but differently : there is no relationship between the number of mcycles and the Warning (This request used a high amount of CPU). Example from my log : * 1159mcycles = This request used a high amount of CPU, and was roughly 1.0 times over... * 2030mcycles = no warning ? strange ? no ? If there is no change, my app will be soon offline. Thank you for your answer. Regards On 21 sep, 16:37, Aral Balkan <[EMAIL PROTECTED]> wrote: > Starred it. I agree that this is a big problem. > > The quota system in general is the thorn in Google App Engine's side > and, from a public relations perspective, quite a nightmare. > > Given that most people will check out an application when it's getting > a lot of publicity (and hits), the first experience many people have > with a Google App Engine app is the "Over Quota" message. Not exactly > the first impression you want to make for your shiny new "Cloud" > solution. > > Aral > > On Sep 21, 12:27 pm, mitnickcbc <[EMAIL PROTECTED]> wrote: > > > I have filed a feature request for this issue, if you feel so please > > star it. > > >http://code.google.com/p/googleappengine/issues/detail?id=720 > > > Here is the description: > > > Well, I don't get the purpose for having a High Amount CPU Quota since > > there is already a general CPU Quota. And there are quite a lot of > > problems caused by this quota which restrict my application from > > affording more load. And my application is far away from a 5 million > > month PV app. > > 1. Requests can become a high amount CPU one very easily. A little > > complex request will consume more than 3K mcycles. And what do I mean > > a little > > complex? If you do more than 2 put, or you do a url fetch, or you do a > > query based on an order to select more than 50 model, or you do a put > > on a > > model more than 20 fields, it is that complex. In my case, 80% of the > > requests are that complex and I don't think it can be optimized > > anymore > > since I do need to do put and url fetch. > > 2. Bad visibility to this quota. There is no way we can know when we > > will run out of quota for this. All we got is the warning in admin > > console that > > says our requests are using high amount CPU quota and will soon run > > out of it. > > 3. Doesn't back to normal quickly. I'm using a script to continue > > fetching my data from production and do some offline process. > > Unfortunately I fetch > > too much in one call that I try to get 100 models at a time and which > > makes the request a high CPU one. Soon, I meet quota deny. Then I stop > > my script, > > but after 1.5 hours, I still see quota deny for requests from my > > users. The "Many of the requests to your application are taking a very > > long time. > > Please optimize these requests." warning keeps there even I have > > stopped the script for so long time. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High Amount CPU Quota should be removed
Starred it. I agree that this is a big problem. The quota system in general is the thorn in Google App Engine's side and, from a public relations perspective, quite a nightmare. Given that most people will check out an application when it's getting a lot of publicity (and hits), the first experience many people have with a Google App Engine app is the "Over Quota" message. Not exactly the first impression you want to make for your shiny new "Cloud" solution. Aral On Sep 21, 12:27 pm, mitnickcbc <[EMAIL PROTECTED]> wrote: > I have filed a feature request for this issue, if you feel so please > star it. > > http://code.google.com/p/googleappengine/issues/detail?id=720 > > Here is the description: > > Well, I don't get the purpose for having a High Amount CPU Quota since > there is already a general CPU Quota. And there are quite a lot of > problems caused by this quota which restrict my application from > affording more load. And my application is far away from a 5 million > month PV app. > 1. Requests can become a high amount CPU one very easily. A little > complex request will consume more than 3K mcycles. And what do I mean > a little > complex? If you do more than 2 put, or you do a url fetch, or you do a > query based on an order to select more than 50 model, or you do a put > on a > model more than 20 fields, it is that complex. In my case, 80% of the > requests are that complex and I don't think it can be optimized > anymore > since I do need to do put and url fetch. > 2. Bad visibility to this quota. There is no way we can know when we > will run out of quota for this. All we got is the warning in admin > console that > says our requests are using high amount CPU quota and will soon run > out of it. > 3. Doesn't back to normal quickly. I'm using a script to continue > fetching my data from production and do some offline process. > Unfortunately I fetch > too much in one call that I try to get 100 models at a time and which > makes the request a high CPU one. Soon, I meet quota deny. Then I stop > my script, > but after 1.5 hours, I still see quota deny for requests from my > users. The "Many of the requests to your application are taking a very > long time. > Please optimize these requests." warning keeps there even I have > stopped the script for so long time. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
