So I would like to list some of the needs that real-world business
developer expected from GAE. Why I have the right to say like this?
Because I'm a full time developer working on GAE application and
running a serious LIVE business application on GAE, and my users
already started to pay money for this service!
Story started when I first noticed GAE's quotations: Built on Google's
infrastructure, Easy scale, Easy Deploy and Free to start. Cool! It
looks great! That's all I needed! Then I started to learn Python, read
every article in GAE Docs and some samples, and started coding! Coding
with Python and web framework is fun, I really feel improvements in
productivity with GAE SDK and Python since I used to work on Java.
Soon I started coding, then I realize some functionality cannot be
implemented on GAE, like background scheduler and global transaction
for user to user trading. For background scheduler, I try to make a
workaround using user requests to update data (which causes quite big
overhead for CPU usage and lead to the biggest issue about CPU Quota I
will describe later). For user to user trading, I do the "add" work
first and "minus" work second to make sure user will not complain that
they spend money and get nothing! So who meet the transaction issue,
who is the lucky guy to get my free bonus... So here comes Need One
and Need Two:
Need One: Background scheduler support to be able to perform high CPU
tasks for long time.
Need Two: Global transaction support to be able to keep data
integrity.
Soon the coding finished, deployment is really easy! Within 10 minutes
I can get my application live! Then I did little marketing and some
users started to use my application and I am happy to see it works
smoothly. So far so good, everything is great. Every morning I felt
excited and got up early and cannot wait to check how much more users
have come. I also got some feature requests, and I can manage them
quickly since there is nearly no effort for deployment. Thanks GAE for
letting me have a such wonderful life!
After several days, my business logic is getting more complex due to
user requests. Then I started to notice increasing error rate for
datastore. Most errors are saying "operation took too long" or
"Timeout". This is caused by saving a 30 fields entity. The error rate
is still acceptable but I decided that I am not going to have entity
with more than 20 fields in future.
Need Three: We are expecting lower error rate on datastore for big
entities or at least suggest some best practice here.
Since this application is getting popular, the nightmare is coming.
Quota Deny! It's not from any known quota described by GAE like CPU,
URL Fetch, Datastore api call, bandwith, etc. It's called High amount
CPU Quota! And unfortunately, 80% of my requests are high CPU ones.
Every time my users are playing high on my application, the quota deny
put them down. No access for 10 minutes but just a 403 error page.
While the CPU usage on Admin Console is just showing 0% - 2% usage. I
will skip comments about high amount CPU here since this thread
already has a lot of inputs about this issue. It makes me hard to
sleep well since I'm very worried that how much quota denies I have
met during the night...
Need Four: High amount CPU Quota issue needs to be solved ASAP!
Suggestion from me is just remove it and let user uses the 100% normal
CPU Quota as the way they want.
And last weekend, I'm trying to reduce the amount of high CPU request
by using memcache. It helps some since I cached some URL fetch and
common queries. But 75-80% of my requests are still in high CPU
category. It is possible to write some complex algorithm to optimize
my requests further, but that will cause huge issues for readability
and very error-prone.
Need Five: Don't expect developers will use memcache more than the
simplest pattern.
Well, sounds like I have big problems, then why I have so much time to
write such a long post? Because I can do absolutely nothing when I met
quota denies during weekend except waiting to see a response about
quota increase!
Need Six: Give us some control over quota like quota configuration in
Admin Console!
Further thinking: Good visibility into all quotas are very important
to let us apply quota increase in time to keep our application
reliable and available. There is no doubt GAE can scale, and I also
believe it will start a new age of cloud computing. But without scale
easily and quickly (Currently mainly because of process instead of
technical), GAE has no advantage over other competitors.
Summarized here for the needs I mentioned, and please star the issues
you feel important:
Need One: Background scheduler support to be able to perform high CPU
tasks for long time.
http://code.google.com/p/googleappengine/issues/detail?id=6
Need Two: Global transaction support to be able to keep data
integrity.
http://code.google.com/p/googleappengine/issues/detail?id=313
Need Three: We are expecting lower error rate on datastore for big
entities or at least suggest some best practice here.
Need Four: High amount CPU Quota issue needs to be solved ASAP!
Suggestion from me is just remove it and let user uses the 100% normal
CPU Quota as the way they want.
http://code.google.com/p/googleappengine/issues/detail?id=720
Need Five: Don't expect developers will use memcache more than the
simplest pattern.
Need Six: Give us some control over quota like quota configuration in
Admin Console!
http://code.google.com/p/googleappengine/issues/detail?id=722
On Sep 23, 12:15 pm, gg <[EMAIL PROTECTED]> wrote:
> On Sep 21, 10:29 am, रवींदर ठाकुर (ravinder thakur)
>
> <[EMAIL PROTECTED]> wrote:
> > I think this is the biggest roadblock that will be preventing anyone
> > to develop and deploy any real world apps based on appengine.The
> > calculation of these warning limits is so unpredictable so how can one
> > be supposed to optimize the queries for these limits ?
>
> Agreed. The problem at Google is very simple and has two components.
> First, the engineers run the show and there is no rational voice at
> Google to point out to them when they do something that does not meet
> a real world demand (they make something stupid). And the second
> component is they still have a lot of money to blow so they can get
> away making things that do not meet market demand (writing stuff that
> does not work and is stupid).
>
> These two problems are endemic at Google. And the only thing to cure
> it is some red ink on the books.
>
> An example of the problem as lined out above that comes to mind is
> Google's Carrier Calculated Shipping API in Google Checkout which was
> released exactly one year ago. Carrier Calculated Shipping as
> described here:
>
> http://googlecheckoutapi.blogspot.com/2007/09/carrier-calculated-ship...
>
> " is to provide buyers with shipping rates for the major shipping
> carriers." This allows merchants to provide shipping rates to buyers
> when they provide information about the shipments for Google Checkout
> Transactions.
>
> It does provide rates. Rates that are 30% or more off the actual real
> rates from UPS, FedEx or USPS.
>
> It seems that the Google Engineers have designed an Algorithm to
> compute the rates instead of using the published web services from the
> carriers which give accurate results. I am sure the Algorithm required
> many hours of time from the best of Google Engineers. And I am sure
> the engineers are sure they have coded the best shipping cost
> predictive algorithm known to man, with only a +/- 30% margin of
> error!
>
> Don't question the fact that a merchant may lose $ on every
> transaction with low shipping costs or scare away customers with
> overly high shipping cost. The Google Engineers are smarter than you!
> They have designed the best Algorithm in the world to compute shipping
> costs. +/- 30% is acceptable to them and no one in the world could do
> better.
>
> It is the same with Google App Engine. The Google engineers have
> designed the ultimate computing platform. Who are we to question the
> fact that it can not repeatedly return a reasonably sized result set
> without displaying an "Over Quote" error page. You just do not know
> how to use it!
>
> There is a demand for free/low cost reliable scalable cloud computing
> services. Even those that run only python web frameworks.
>
> There is not a demand for a cloud computing service that can not
> repeatedly return a reasonably sized result set without undo amounts
> of caching or other machinations to get it to happen. Nor is there a
> demand for a checkout system that returns wildly inaccurate shipping
> quotes!
>
> I am sure the Engineers will all be very puzzled when they try to
> charge for Google App Engine and no one is willing to pay..... Just
> remember that they are smarter than you and their solution is the
> best..... Even if you would never pay for it.
>
> For more info on the Joke that is Carrier Calculated Shipping checkot
> the Google Checkout forum:
>
> http://groups.google.com/group/google-checkout-api-troubleshooting/se...
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
