[google-appengine] Re: 3000 mcycles and This request used a amount of CPU and may soon exceed its quota

2008-10-14 Thread Josh Heitzman

Jon said:

"Josh, to address your question, what you are really looking for is
improved support for offline processing (there are many issues for
this listed in the Issue Tracker). With that infrastructure in place
we may be able to adjust the limits. Offline processing is orthogonal
to billing."

Actually it isn't.  When a user issues a command in my strategy game,
it needs to be processed ASAP not eventually by offline processing.

Per this thread
http://groups.google.com/group/google-appengine/browse_thread/thread/2901a4ddd1f671d9/2695f308f3cb3fc8#2695f308f3cb3fc8
I'm currently seeing it take 500 mcycles of overhead and 1000 mcycles
per entity (i.e. 1500 mcycles to update one entity, 2500 to update
two, etc.) to update entities via a single db.put() at the end of a
transaction.

For commands that need to update entities in multiple entity groups,
I've implemented my own reliable update sequence mechanism, but this
requires a bunch of extra datastore operations for booking entities,
so processing one command that actually updates very little game state
data ends up taking upwards of 15000 mcycles due to the datastore
overhead.  From the users perspective it is taking less then 2 seconds
(according to my Firefox timer), which is reasonable compared to other
web based strategy games.

I can get around the CPU quotas by just partitioning up all of the
datastore operations into a chain of separate requests that just keep
redirecting on down the line (including using so no op request whose
purpose is just to bring the average down), but then from the users
perspective it will take a lot longer then two seconds not to mention
it will actually take a lot more total mcycles which goes counter to
your other statement "The best way to think about it is that if we
took it to the extreme, and let the app spend as much time on each
request as they liked, then we would only be able to serve a small
number of applications at the same time."  Regarding that, I
understand there will be charges for CPU time, so I don't see how apps
that consume and thus pay for lots of CPU time won't be paying there
own way (i.e. you'll be able to afford more machines because you'll
have more revenue coming in).

Josh

On Oct 14, 9:29 pm, Jon McAlister <[EMAIL PROTECTED]> wrote:
> Matija, you said:
>
> """or that app-engine quota permits this CPU
> usage only few times let's say per minute and that they will stop my
> application for several hours if I continue with this usage."""
>
> Bingo, that is exactly what we do. We don't absolutely forbid such
> high CPU requests but we do highly discourage them. The best way to
> think about it is that if we took it to the extreme, and let the app
> spend as much time on each request as they liked, then we would only
> be able to serve a small number of applications at the same time. It
> is only by enforcing request deadlines and cpu-consumption limits that
> we are able to make this service open to as many applications as
> possible while also guaranteeing isolation between them.
>
> Josh, to address your question, what you are really looking for is
> improved support for offline processing (there are many issues for
> this listed in the Issue Tracker). With that infrastructure in place
> we may be able to adjust the limits. Offline processing is orthogonal
> to billing.
>
> Jon
>
> On Oct 13, 9:32 am, Josh Heitzman <[EMAIL PROTECTED]> wrote:
>
> > I've been wondering the same thing.  When they start letting us pay
> > for this, will our apps still have to stay under some arbitrary
> > average number of CPU cycles?
>
> > On Oct 13, 3:21 am, Matija <[EMAIL PROTECTED]> wrote:
>
> > > Hi,
> > > I have some init stage imports to app-engine application and
> > > combining  with some ajax calls I was able to use maximum of 3000
> > > mcycles per request. When there is request that consumes more the 1000
> > > mcycles I have warning 'This request used a amount of CPU and may soon
> > > exceed its quota.' in my application log.
>
> > > Should I be worried because of that or not. Is this general warning
> > > for me to inform me that with this average CPU use i will use all my
> > > CPU quota eventually soon, or that app-engine quota permits this CPU
> > > usage only few times let's say per minute and that they will stop my
> > > application for several hours if I continue with this usage.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: 3000 mcycles and This request used a amount of CPU and may soon exceed its quota

2008-10-14 Thread Jon McAlister

Matija, you said:

"""or that app-engine quota permits this CPU
usage only few times let's say per minute and that they will stop my
application for several hours if I continue with this usage."""

Bingo, that is exactly what we do. We don't absolutely forbid such
high CPU requests but we do highly discourage them. The best way to
think about it is that if we took it to the extreme, and let the app
spend as much time on each request as they liked, then we would only
be able to serve a small number of applications at the same time. It
is only by enforcing request deadlines and cpu-consumption limits that
we are able to make this service open to as many applications as
possible while also guaranteeing isolation between them.

Josh, to address your question, what you are really looking for is
improved support for offline processing (there are many issues for
this listed in the Issue Tracker). With that infrastructure in place
we may be able to adjust the limits. Offline processing is orthogonal
to billing.

Jon

On Oct 13, 9:32 am, Josh Heitzman <[EMAIL PROTECTED]> wrote:
> I've been wondering the same thing.  When they start letting us pay
> for this, will our apps still have to stay under some arbitrary
> average number of CPU cycles?
>
> On Oct 13, 3:21 am, Matija <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> > I have some init stage imports to app-engine application and
> > combining  with some ajax calls I was able to use maximum of 3000
> > mcycles per request. When there is request that consumes more the 1000
> > mcycles I have warning 'This request used a amount of CPU and may soon
> > exceed its quota.' in my application log.
>
> > Should I be worried because of that or not. Is this general warning
> > for me to inform me that with this average CPU use i will use all my
> > CPU quota eventually soon, or that app-engine quota permits this CPU
> > usage only few times let's say per minute and that they will stop my
> > application for several hours if I continue with this usage.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: 3000 mcycles and This request used a amount of CPU and may soon exceed its quota

2008-10-13 Thread Josh Heitzman

I've been wondering the same thing.  When they start letting us pay
for this, will our apps still have to stay under some arbitrary
average number of CPU cycles?

On Oct 13, 3:21 am, Matija <[EMAIL PROTECTED]> wrote:
> Hi,
> I have some init stage imports to app-engine application and
> combining  with some ajax calls I was able to use maximum of 3000
> mcycles per request. When there is request that consumes more the 1000
> mcycles I have warning 'This request used a amount of CPU and may soon
> exceed its quota.' in my application log.
>
> Should I be worried because of that or not. Is this general warning
> for me to inform me that with this average CPU use i will use all my
> CPU quota eventually soon, or that app-engine quota permits this CPU
> usage only few times let's say per minute and that they will stop my
> application for several hours if I continue with this usage.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---