[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

So you're saying stay on MS since it will become super stable when everyone has migrated to HR? :-) Look at Aug 29, http://code.google.com/status/appengine/detail/hr-datastore/2011/08/29#ae-trust-detail-hr-datastore-query-latency Bringing this back to billing, there's no sla on the datastore,

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

No, actually I migrated to HR earlier (was an awful experience with 30+hrs downtime. trial run took 8hrs only with the same data). I am just trying to give an signal to everyone before something bad could hit us again. Maybe GAE team could do sth before it happen. -- You received this messag

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

I would be interested in what the gae team has to say about this. It seems exactly the same as the evolution of the master slave data store. Its latency creeped up over the first few years and became pretty unstable. Definitely concerning. Mike On Sep 1, 9:30 pm, "Raymond C." wrote: > http://

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Hi Actually I would disagree on the M/S front, all of my apps are on M/S and even despite the recent issues M/S is way more stable than it was in 2008 and 2009, I mean WAY MORE STABLE ;-) Thats not to say more growing pains won't be felt. My guess is with a supposed imminent exodus of apps, w

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Yes, I agree that it won't (can't) get as bad as m/s did get because HRD is awesome. My concern is mostly over how they provision and that the latency is that much higher than it was. Why does the load have such a big impact on latency of datastore query? We keep very detailed statistics about t

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

This is really scary. I'm not so obsessed about the price increase as this is something you can "control", but if the HRD starts becoming slower and slower, there's absolutely nothing you can do. According to these official graphs, it's literally 2X slower than 1 year ago. I'm OK with training some

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

I will disagree with this post. We were on MS earlier and now on HRD. While on MS, we used to have datastore timeouts all the time and it was driving me insane looking at 20-30k error emails come in. After the move to HRD, everything is stable and there are no datastore timeouts anymore. I am happy

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

I agree on both fronts. Yes, M/S was a JOKE. Once moving to HRD I have not seen a datastore timeout at all. Until earlier this week I saw a small series of them. I really hope we are not going back to what we are all used to on M/S. The old M/S problem days were always written off as "well, it'

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Our app has been on HRD since the beginning and indeed, the past few days for the time ever, I saw some timeouts. To be clear, not timeouts due to contention which we get as expected now and then on transactions, but timeout on the db itself. Our usage patterns haven't changed though, so it would

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Will datastore latency be covered by the SLA? All I'm seeing online is an old draft SLA: http://code.google.com/appengine/sla.html -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.g

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Reviving this thread... GAE team: can we get an official comment on this really important issue? Thx On Sep 2, 1:14 pm, Pol wrote: > Our app has been on HRD since the beginning and indeed, the past few > days for the time ever, I saw some timeouts. To be clear, not timeouts > due to contention wh

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Especially the datastore is getting quite a number of timeout these two days: http://code.google.com/status/appengine/detail/hr-datastore/2011/09/04#ae-trust-detail-hr-datastore-get-latency -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. T

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Thanks for putting those together. It very clearly shows that recently things have changed. It would be nice to hear from Greg or someone about this. -Mike On Sep 6, 7:11 am, Simon Knott wrote: > I've attached graphs showing the daily stats for each of the HR operations > for the last year, dr

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Those trends are concerning. We're putting a large effort in to convert our app over to HR. Performance consistency is the only reason we're doing this. j On Sep 6, 7:57 am, Mike Wesner wrote: > Thanks for putting those together.  It very clearly shows that > recently things have changed. > > It

[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Great to see the problem acknowledged, so why is the HDR line still all green at http://code.google.com/status/appengine? The HRD is having problems now (yes, not as bad pas a MS problems, but that's beside the point as the expectancies have been officially raised quite a bit for HRD), non-negligi

Re: [google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

I think the point of this post is the trend. At one time master-slave also performed quite well, then we'd see trends where latency increased until a blowup. These graphs show a similar pattern to that: increasing latencies over time. We would like to know that: 1) the trend won't continue, 2) w

Re: [google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

On Friday, 2 September 2011 23:10:30 UTC+7, Robert Kluin wrote: > > As for the SLA, see the exclusions section. SLAs are fairly worthless > in my honest opinion anyway, so this probably isn't really something > to worry about though. > http://code.google.com/appengine/sla.html > The SLA does

Re: [google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

I've attached graphs showing the daily stats for each of the HR operations for the last year, driven from the GAE Status stats. I'll grab the M/S datastore statistics, when I get the time, to give a comparison.

Re: [google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Hello All, This is an issue right now related to higher latency on HRD, particularly affecting queries. We are working on potential fixes to mitigate the issue. Once we have determined the full scope of it, we will provide further information. This is not expected behavior for HRD and is indeed

Re: [google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Glad to hear Googler speak about it. Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/R3HOpbexI54J. To post to this group, send email to go

Re: [google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.

Hi Pol, that is unfortunately a bug in the status dashboard that we are working on... Greg On Wed, Sep 7, 2011 at 8:19 AM, Pol wrote: > Great to see the problem acknowledged, so why is the HDR line still > all green at http://code.google.com/status/appengine? > > The HRD is having problems now