You can also set the consistency guarantees to EVENTUALLY_CONSISTENT. Even on master/slave datastore, this fails over to a read-slave. The tradeoff is you might have consistency issues.
Here's a pretty good read about consistency vs. availability tradeoffs: http://codahale.com/you-cant-sacrifice-partition-tolerance/ It's good to know this stuff so you have it in your back pocket. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Fri, Dec 2, 2011 at 11:49 AM, Rishi Arora <rishi.ar...@ship-rack.com>wrote: > Ikai, thank you for a comprehensive reply. I had no idea you could set > deadlines for datastore reads, even for Master/Slave db reads. This would > be my temporary solution until I eventually migrate to HRD. I have already > started looking into it, with the toughest part being appropriately > managing user experience during the downtime and during the changeover to > the new app-id. My users use an Android app and Google Accounts API in > conjunction with Android AccountManager, to interface with my GAE system, > and they could be running multiple versions of my app at any given time. > > Thanks again > - Rishi > > > On Fri, Dec 2, 2011 at 1:38 PM, Ikai Lan (Google) <ika...@google.com>wrote: > >> I just thought of something kind of cool: a real time knob that allows >> you to change the deadlines! >> >> -- >> Ikai Lan >> Developer Programs Engineer, Google App Engine >> plus.ikailan.com | twitter.com/ikai >> >> >> >> On Fri, Dec 2, 2011 at 11:37 AM, Ikai Lan (Google) <ika...@google.com>wrote: >> >>> To answer the original question: >>> >>> - As a general best practice, you almost always want to set a timeout >>> for any call that might be frequent: >>> >>> >>> http://code.google.com/appengine/docs/python/datastore/queries.html#Setting_the_Read_Policy_and_Datastore_Call_Deadline >>> >>> This is true for App Engine as well as any application outside of App >>> Engine. You degrade gracefully by telling the user to retry or showing only >>> a subset of data. It sucks, but at this point you have to decide between >>> making the user wait a long time or showing partial results and doing an >>> exponential backoff. >>> >>> - Use exponential backoffs if you tell your client to retry. You might >>> show something to the user that says "loading, please wait!", but you're >>> going to be in hot water if you just say "retry after 1 second, retry after >>> 1 second, retry after 1 second" because these will eventually pile up. You >>> want to do something like "retry after 2 seconds + some fuzz factor, retry >>> after 4 seconds plus some fuzz factor, and so forth". >>> >>> - I've been part of a project where there were many administrative >>> "knobs" that we could flip to gracefully degrade and turn off expensive >>> features. This is particularly useful when you push a new version because >>> you can narrow in on the component that seems to be causing your stack to >>> break. It's also nice because you can turn something off, gather with your >>> team and figure out what the plan of attack is without a full rollback. The >>> tradeoff is that you have to build it, and it's non-trivial to have this >>> all over your code, so you need to figure out which parts tend to be more >>> complex than others. In an ideal world, you always know what component does >>> what in your app, but in reality, you usually don't know that there is a >>> weird code path that might invoke the most expensive method call over and >>> over. >>> >>> >>> -- >>> Ikai Lan >>> Developer Programs Engineer, Google App Engine >>> plus.ikailan.com | twitter.com/ikai >>> >>> >>> >>> On Fri, Dec 2, 2011 at 11:31 AM, Ikai Lan (Google) <ika...@google.com>wrote: >>> >>>> As far as I know, there are no plans to shut down M/S within the next >>>> 12 months. This is NOT an official announcement, so if this does change >>>> later (I don't think it will), just know that it's a gut feeling. When >>>> there is a plan to official deprecate master/slave, we will announce it. >>>> >>>> Many new features will be HRD only, however. Python 2.7 is one of >>>> them. Full text search *may* be one of them. There are no plans to support >>>> Python 2.7 for master/slave applications, and I don't see this changing. >>>> >>>> I strongly recommend migrating to HRD or at least starting to >>>> investigate the possibility. >>>> >>>> -- >>>> Ikai Lan >>>> Developer Programs Engineer, Google App Engine >>>> plus.ikailan.com | twitter.com/ikai >>>> >>>> >>>> >>>> On Fri, Dec 2, 2011 at 9:03 AM, Rishi Arora >>>> <rishi.ar...@ship-rack.com>wrote: >>>> >>>>> Based on this: >>>>> http://code.google.com/appengine/docs/python/datastore/hr/ >>>>> >>>>> I don't see any strong language indicating imminent deprecation. >>>>> Although the fact that python2.7 does not support M/S might hint at that. >>>>> If Google does indeed want to deprecate M/S, why wouldn't they just say >>>>> it, and give some fixed timeline - 1 month, 6 months, 1 year, whatever. >>>>> But why hint it, why not just say it? Since I don't see a strong reason >>>>> for Google to drop subtle hints, I assumed there's no plan to deprecate >>>>> M/S, and I also assumed that eventually when python2.5 is phased out, >>>>> python2.7 will support M/S. I don't see any technical reason why >>>>> python2.7 >>>>> can't support M/S either. I can achieve nearly the same level of >>>>> concurrency with Python2.5 (albeit at a higher cost) that I can with >>>>> Python2.7, with regards to datastore operations. >>>>> >>>>> Can someone at Google provide some direction on long-term support (at >>>>> least a year or so) for M/S? >>>>> >>>>> >>>>> On Fri, Dec 2, 2011 at 10:07 AM, Joshua Smith < >>>>> joshuaesm...@charter.net> wrote: >>>>> >>>>>> I believe so. >>>>>> >>>>>> Google has made it pretty clear that M/S is deprecated and life on >>>>>> M/S will continue to get worse and worse. >>>>>> >>>>>> On Dec 2, 2011, at 10:54 AM, Rishi Arora wrote: >>>>>> >>>>>> :) So I'm SOL, without HR? >>>>>> >>>>>> On Fri, Dec 2, 2011 at 9:50 AM, Joshua Smith < >>>>>> joshuaesm...@charter.net> wrote: >>>>>> >>>>>>> You can't use 2.7 unless you migrate to HR. >>>>>>> >>>>>>> Once you migrate to HR, the datastore read timeouts go away. >>>>>>> >>>>>>> And then you don't need to migrate to 2.7. >>>>>>> >>>>>>> -Joshua >>>>>>> >>>>>>> On Dec 2, 2011, at 10:37 AM, Rishi Arora wrote: >>>>>>> >>>>>>> > Earlier this morning I had a situation where datastore reads were >>>>>>> timing out. That's okay, and expected, given that I use a M/S >>>>>>> datastore. >>>>>>> However, the timeouts were of the order of 50 seconds, causing nearly >>>>>>> 30 >>>>>>> front-end instances to be spawned. My usual number of active front-end >>>>>>> instances at that time of the day is about 5, peaking at 15 >>>>>>> occasionally. >>>>>>> This condition lasted only 3 minutes or so, and so, the cost impact was >>>>>>> minimal. However, I can imagine that if this lasted an hour or more, I >>>>>>> would incur a lot of costs while the downtime persists. I'm okay with >>>>>>> such >>>>>>> downtimes, as long as it only leads to my customers not being able to >>>>>>> access my site. But if it also leads to unnecessary increases in costs, >>>>>>> then it calls for further optimization. So, my loaded question is - how >>>>>>> can I handle this with python2.5? Is python 2.7 the only answer? I >>>>>>> imagine python2.7 will help because while a front-end is waiting for >>>>>>> data >>>>>>> store ops to complete, it can process other requests. But are there >>>>>>> other >>>>>>> ways of setting specific timeouts to datastore operations? So, if these >>>>>>> operations are taking too long, I'd rather just return an error to the >>>>>>> user, instead of letting my front-end run indefinitely. >>>>>>> > >>>>>>> > >>>>>>> > -- >>>>>>> > You received this message because you are subscribed to the Google >>>>>>> Groups "Google App Engine" group. >>>>>>> > To post to this group, send email to >>>>>>> google-appengine@googlegroups.com. >>>>>>> > To unsubscribe from this group, send email to >>>>>>> google-appengine+unsubscr...@googlegroups.com. >>>>>>> > For more options, visit this group at >>>>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Google App Engine" group. >>>>>>> To post to this group, send email to >>>>>>> google-appengine@googlegroups.com. >>>>>>> To unsubscribe from this group, send email to >>>>>>> google-appengine+unsubscr...@googlegroups.com. >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Google App Engine" group. >>>>>> To post to this group, send email to >>>>>> google-appengine@googlegroups.com. >>>>>> To unsubscribe from this group, send email to >>>>>> google-appengine+unsubscr...@googlegroups.com. >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Google App Engine" group. >>>>>> To post to this group, send email to >>>>>> google-appengine@googlegroups.com. >>>>>> To unsubscribe from this group, send email to >>>>>> google-appengine+unsubscr...@googlegroups.com. >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To post to this group, send email to google-appengine@googlegroups.com >>>>> . >>>>> To unsubscribe from this group, send email to >>>>> google-appengine+unsubscr...@googlegroups.com. >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>> >>>> >>>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
