Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Ikai Lan (Google)
You can also set the consistency guarantees to EVENTUALLY_CONSISTENT. Even on master/slave datastore, this fails over to a read-slave. The tradeoff is you might have consistency issues. Here's a pretty good read about consistency vs. availability tradeoffs: http://codahale.com/you-cant-sacrifice-

Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Rishi Arora
Ikai, thank you for a comprehensive reply. I had no idea you could set deadlines for datastore reads, even for Master/Slave db reads. This would be my temporary solution until I eventually migrate to HRD. I have already started looking into it, with the toughest part being appropriately managing

Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Ikai Lan (Google)
I just thought of something kind of cool: a real time knob that allows you to change the deadlines! -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Fri, Dec 2, 2011 at 11:37 AM, Ikai Lan (Google) wrote: > To answer the original question: > > -

Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Ikai Lan (Google)
To answer the original question: - As a general best practice, you almost always want to set a timeout for any call that might be frequent: http://code.google.com/appengine/docs/python/datastore/queries.html#Setting_the_Read_Policy_and_Datastore_Call_Deadline This is true for App Engine as well

Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Ikai Lan (Google)
As far as I know, there are no plans to shut down M/S within the next 12 months. This is NOT an official announcement, so if this does change later (I don't think it will), just know that it's a gut feeling. When there is a plan to official deprecate master/slave, we will announce it. Many new fea

Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Rishi Arora
Based on this: http://code.google.com/appengine/docs/python/datastore/hr/ I don't see any strong language indicating imminent deprecation. Although the fact that python2.7 does not support M/S might hint at that. If Google does indeed want to deprecate M/S, why wouldn't they just say it, and giv

Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Joshua Smith
I believe so. Google has made it pretty clear that M/S is deprecated and life on M/S will continue to get worse and worse. On Dec 2, 2011, at 10:54 AM, Rishi Arora wrote: > :) So I'm SOL, without HR? > > On Fri, Dec 2, 2011 at 9:50 AM, Joshua Smith wrote: > You can't use 2.7 unless you migra

Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Rishi Arora
:) So I'm SOL, without HR? On Fri, Dec 2, 2011 at 9:50 AM, Joshua Smith wrote: > You can't use 2.7 unless you migrate to HR. > > Once you migrate to HR, the datastore read timeouts go away. > > And then you don't need to migrate to 2.7. > > -Joshua > > On Dec 2, 2011, at 10:37 AM, Rishi Arora wr

Re: [google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Joshua Smith
You can't use 2.7 unless you migrate to HR. Once you migrate to HR, the datastore read timeouts go away. And then you don't need to migrate to 2.7. -Joshua On Dec 2, 2011, at 10:37 AM, Rishi Arora wrote: > Earlier this morning I had a situation where datastore reads were timing out. > That's

[google-appengine] How to gracefully handle datastore timeouts

2011-12-02 Thread Rishi Arora
Earlier this morning I had a situation where datastore reads were timing out. That's okay, and expected, given that I use a M/S datastore. However, the timeouts were of the order of 50 seconds, causing nearly 30 front-end instances to be spawned. My usual number of active front-end instances at