Ray, Very interesting data, indeed. Although, it's tough to make any suggestions as to what was going on without any other context. If you'd like, we can take a look at your app to see if there is something out of the ordinary that would lead to the latency numbers that you're reporting.
-- Chris On Fri, Apr 6, 2012 at 7:49 PM, Ray <windz...@gmail.com> wrote: > The images are here: (Google Groups had problem on file attachment > apparently) > > http://dl.dropbox.com/u/182081/photo1.png > http://dl.dropbox.com/u/182081/photo2.png > http://dl.dropbox.com/u/182081/photo3.png > > > On Saturday, April 7, 2012 10:47:52 AM UTC+8, Ray wrote: >> >> AppEngine has had some pretty bad days recently. I was lucky enough to >> capture two dashboard with just a few hours apart which shows how bad the >> HR datastore could be: >> >> The first two image show the hourly "Avg Latency" of the requests, the >> first one was the data around 8 hours ago, the other one was a few hours >> back. Normally the requests finish in a few hundreds ms, which are mostly >> get by key datastore access. When HR datastore misbehave, there are many >> deadline exceeded error right on db access and even if they complete, they >> become very slow. (from you chart you can see 3s to 9s latency in average). >> The two time periods are having similar traffic and no special code was >> running to cause the problem. >> >> There were no code change, no abnormal status on the status page. They >> are typical (although this one is worst than normal in terms of error >> percentage) HR performance for me, HR datastore has constant errors, just >> not worst than M/S. >> >> The third image shows how unstable HR datastore could be. Those errors >> are mostly caused by timeout on db access. >> >> >> On Saturday, April 7, 2012 5:53:04 AM UTC+8, stevep wrote: >>> >>> Ray wrote: " I am still having around 1% percentage of datastore >>> timeout from any kind of datastore accesses like a simple >>> get_wih_id()." >>> >>> I have not seen this. If you and others are seeing 1% get_by_id()s >>> failing due to timeouts, it needs to be diagnosed by G. stat. Be sure >>> to report it. >>> >>> If it is a common issue, and remains so with G. acknowledging its part >>> and parcel of their PaaS, then it really is time to move to AWS for >>> me. >>> >>> -stevep >>> >>> On Apr 5, 6:26 pm, Ray <windz...@gmail.com> wrote: >>> > No one regret moving to HRD not because its great, instead its the M/S >>> > sucked too bad lately. 30 minutes download evey week is a disaster for >>> > any kind of app hosting on it. >>> > >>> > At the mean time, HRD still isn't flawless. I am still having around >>> 1% >>> > percentage of datastore timeout from any kind of datastore accesses >>> like a >>> > simple get_wih_id(). It is slower than M/S and cost more (the /G cost >>> is >>> > now the same since Google raised the price for M/S too, but the slower >>> > respond still cost more on instance number) >>> > >>> > The most stinky part is how Google advertise it as a 100% up time >>> service >>> > when it is generating more exceptions than most shared MySQL hosting >>> > outside. I know I know, its about scalability. But how many apps are >>> > having hundreds or thousands of DB access every second that can >>> benefit >>> > from its scalability which can compensate the constant flaw? >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > On Thursday, April 5, 2012 4:46:57 AM UTC+8, Robert Kluin wrote: >>> > >>> > > I think this is a really good idea. It will encourage people with >>> > > apps to move, and help further persuade new apps to not use >>> > > master-slave. I have not spoken to anyone who regretted the move to >>> > > high-replication. >>> > >>> > > Robert >>> > >>> > > On Wed, Apr 4, 2012 at 15:42, Chris Ramsdale <cramsd...@google.com> >>> wrote: >>> > > > Hey App Engine Users, >>> > >>> > > > Almost 4 years after launch, we’ve seen fantastic growth both in >>> Google >>> > > App >>> > > > Engine and the apps that run on it. And although the Master/Slave >>> > > Datastore >>> > > > was a big part of our early success, it's time to announce the >>> > > deprecation >>> > > > of the Master/Slave Datastore in favor of the High-Replication >>> Datastore >>> > > > (HRD). HRD has provided us with higher availability and better, >>> more >>> > > > predictable performance. Many upcoming features will be HRD-only >>> and we >>> > > > strongly encourage you to migrate all your applications as soon as >>> > > possible >>> > > > using the migration tools found in the Application Settings tab of >>> the >>> > > > Administration Console. The deprecation period will follow the >>> guidelines >>> > > > set in our terms of service. If you have any questions or >>> concerns, >>> > > please >>> > > > contact us at ms-datastore-deprecat...@**google.com. >>> > >>> > > > Chris >>> > >>> > > > Product Manager, Google App Engine >>> > >>> > > > -- >>> > > > You received this message because you are subscribed to the Google >>> Groups >>> > > > "Google App Engine" group. >>> > > > To post to this group, send email to >>> google-appengine@googlegroups.**com <google-appengine@googlegroups.com>. >>> >>> > > > To unsubscribe from this group, send email to >>> > > > google-appengine+unsubscribe@**googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>. >>> >>> > > > For more options, visit this group at >>> > > >http://groups.google.com/**group/google-appengine?hl=en<http://groups.google.com/group/google-appengine?hl=en> >>> . >> >> -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/BwVDunKB5hsJ. > > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
