[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
On Tue, Dec 2, 2008 at 3:08 AM, Amir Michail <[EMAIL PROTECTED]> wrote: > On Tue, Dec 2, 2008 at 3:07 AM, Greg <[EMAIL PROTECTED]> wrote: >> >>> When doing a urlfetch, it may take quite a long time. As this is not >>> a CPU usage issue, what's the point of limiting the request time >>> anyway? Why not allow the request to go on for a minute or so? >> Because it ties up memory for the duration of the request. Imagine if >> they allowed hour long requests - it would be easy to swamp machines >> with requests waiting for a response. Or just handle it with some sort of memory quota. Amir > > Ok, but what about one minute requests? What would the impact be? > > Amir > >> >> >> >> > > > > -- > http://b4utweet.com > http://chatbotgame.com > http://numbrosia.com > http://twitter.com/amichail > -- http://b4utweet.com http://chatbotgame.com http://numbrosia.com http://twitter.com/amichail --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
On Tue, Dec 2, 2008 at 3:07 AM, Greg <[EMAIL PROTECTED]> wrote: > >> When doing a urlfetch, it may take quite a long time. As this is not >> a CPU usage issue, what's the point of limiting the request time >> anyway? Why not allow the request to go on for a minute or so? > Because it ties up memory for the duration of the request. Imagine if > they allowed hour long requests - it would be easy to swamp machines > with requests waiting for a response. Ok, but what about one minute requests? What would the impact be? Amir > > > > -- http://b4utweet.com http://chatbotgame.com http://numbrosia.com http://twitter.com/amichail --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
> When doing a urlfetch, it may take quite a long time. As this is not > a CPU usage issue, what's the point of limiting the request time > anyway? Why not allow the request to go on for a minute or so? Because it ties up memory for the duration of the request. Imagine if they allowed hour long requests - it would be easy to swamp machines with requests waiting for a response. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
On Tue, Dec 2, 2008 at 3:08 PM, Amir Michail <[EMAIL PROTECTED]> wrote: > > Hi, > > When doing a urlfetch, it may take quite a long time. As this is not > a CPU usage issue, what's the point of limiting the request time > anyway? Why not allow the request to go on for a minute or so? As I understand it, the thread doing the fetch will be blocked and cannot service any other requests until it finishes the fetch. Now, if a lot of threads are blocked waiting for that resource to be fetched, you will not be able to scale -- as it would be effectively an inadvertent denial of service for your app (and the whole infrastructure). -- JM Ibanez -- The worst government is often the most moral. One composed of cynics is often very tolerant and humane. But when fanatics are on top there is no limit to oppression. -- H. L. Mencken - http://www.livejournal.com/~jmibanez/ http://www.mycgiserver.com/~butiki/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
On Tue, Dec 2, 2008 at 4:20 PM, Amir Michail <[EMAIL PROTECTED]> wrote: > > On Tue, Dec 2, 2008 at 3:08 AM, Amir Michail <[EMAIL PROTECTED]> wrote: >> On Tue, Dec 2, 2008 at 3:07 AM, Greg <[EMAIL PROTECTED]> wrote: >>> When doing a urlfetch, it may take quite a long time. As this is not a CPU usage issue, what's the point of limiting the request time anyway? Why not allow the request to go on for a minute or so? >>> Because it ties up memory for the duration of the request. Imagine if >>> they allowed hour long requests - it would be easy to swamp machines >>> with requests waiting for a response. > > Or just handle it with some sort of memory quota. It's not just memory, it's also the fact that threads are also waiting. And there are only a limited number of threads that can be created and can be waiting at any time. If all of your threads are waiting, even for a single minute, then no other requests can be served in the meantime. Then, at peak, more requests will come in which will have to be queued. Ad infinitum: you'll end up with HTTP requests which have to timeout because there aren't any threads available to handle them. -- JM Ibanez -- The worst government is often the most moral. One composed of cynics is often very tolerant and humane. But when fanatics are on top there is no limit to oppression. -- H. L. Mencken - http://www.livejournal.com/~jmibanez/ httitp://www.mycgiserver.com/~butiki/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
On Tue, Dec 2, 2008 at 3:35 AM, JM Ibanez <[EMAIL PROTECTED]> wrote: > > On Tue, Dec 2, 2008 at 4:20 PM, Amir Michail <[EMAIL PROTECTED]> wrote: >> >> On Tue, Dec 2, 2008 at 3:08 AM, Amir Michail <[EMAIL PROTECTED]> wrote: >>> On Tue, Dec 2, 2008 at 3:07 AM, Greg <[EMAIL PROTECTED]> wrote: > When doing a urlfetch, it may take quite a long time. As this is not > a CPU usage issue, what's the point of limiting the request time > anyway? Why not allow the request to go on for a minute or so? Because it ties up memory for the duration of the request. Imagine if they allowed hour long requests - it would be easy to swamp machines with requests waiting for a response. >> >> Or just handle it with some sort of memory quota. > > It's not just memory, it's also the fact that threads are also > waiting. And there are only a limited number of threads that can be > created and can be waiting at any time. > > If all of your threads are waiting, even for a single minute, then no > other requests can be served in the meantime. Then, at peak, more > requests will come in which will have to be queued. Ad infinitum: > you'll end up with HTTP requests which have to timeout because there > aren't any threads available to handle them. What's so special about the current time limit in that respect? As for scalability, sometimes you need to do things in an undesirable manner when you don't have much data yet (e.g., heavy reliance on getting data via an API). But as you get more users and more data, then you could do things differently. Amir > > > -- > JM Ibanez -- > > The worst government is often the most moral. One composed of cynics > is often very tolerant and humane. But when fanatics are on top there > is no limit to oppression. >-- H. L. Mencken > > - > http://www.livejournal.com/~jmibanez/ > httitp://www.mycgiserver.com/~butiki/ > > > > -- http://b4utweet.com http://chatbotgame.com http://numbrosia.com http://twitter.com/amichail --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
Apparently GAE's infrastructure will just continue to spawn more processes as needed, so your app actually should continue to scale even if it waits 1 minute instead of 10 seconds for URLFetches, but operating your app will be more costly as memory usage time will be increased by a factor of 6. Currently google isn't tracking memory usage time though (at least that we know of), so they couldn't actually charge for it at present. On Dec 1, 11:14 pm, "JM Ibanez" <[EMAIL PROTECTED]> wrote: > On Tue, Dec 2, 2008 at 3:08 PM, Amir Michail <[EMAIL PROTECTED]> wrote: > > > Hi, > > > When doing a urlfetch, it may take quite a long time. As this is not > > a CPU usage issue, what's the point of limiting the request time > > anyway? Why not allow the request to go on for a minute or so? > > As I understand it, the thread doing the fetch will be blocked and > cannot service any other requests until it finishes the fetch. Now, if > a lot of threads are blocked waiting for that resource to be fetched, > you will not be able to scale -- as it would be effectively an > inadvertent denial of service for your app (and the whole > infrastructure). > > -- > JM Ibanez -- > > The worst government is often the most moral. One composed of cynics > is often very tolerant and humane. But when fanatics are on top there > is no limit to oppression. > -- H. L. Mencken > > -http://www.livejournal.com/~jmibanez/http://www.mycgiserver.com/~butiki/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
On Dec 2, 3:09 pm, Josh Heitzman <[EMAIL PROTECTED]> wrote: > Apparently GAE's infrastructure will just continue to spawn more > processes as needed, so your app actually should continue to scale > even if it waits 1 minute instead of 10 seconds for URLFetches, but > operating your app will be more costly as memory usage time will be > increased by a factor of 6. Currently google isn't tracking memory > usage time though (at least that we know of), so they couldn't > actually charge for it at present. More likely, they're not tracking CPU usage time - just overall time spend handling a request (even if that's waiting for the datastore or url fetch systems). This deals with memory usage as well in a neat and simple way. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
