PS: what does google do internally (wrt coding to avoid datastore timeouts)?
On Jun 18, 6:58 pm, Dennis <dennisf...@gmail.com> wrote: > > I presume by 'data structure', you mean multiple entities? > > Yes, multiple entity structures need to be integrity checked each time > they are used. > Thanks for the confirmation of how we should code. > > > only partially written. In general, though, the 'roll forward' approach > > works quite well. > > What is the 'roll forward' approach? > > On Jun 18, 6:49 pm, "Nick Johnson (Google)" <nick.john...@google.com> > wrote: > > > > > On Thu, Jun 18, 2009 at 11:45 AM, Dennis <dennisf...@gmail.com> wrote: > > > > I asked about this (datastore timeout exceptions) in yesterday's app > > > engine chat and just wanted to follow up. > > > > I'm trying to figure out an architecture to deal with the timeout > > > exceptions. > > > The "solution" of pushing the exception back to the end user is > > > actually ok for my app. > > > This works for read timeouts. > > > > For write timeouts, my code can be in the middle of storing a data > > > structure when the timeout happens. > > > Thus, I could have a corrupt data structure instance in my datastore. > > > I presume by 'data structure', you mean multiple entities? Because > > individual entities (and entity groups inside transactions) will always be > > written atomically - all or nothing - and never 'corrupted' or partially > > written. > > > > So, for fault-tolerant production code, any (later) access to any > > > datastore data structure should check > > > the integrity data structure before using it. > > > This is what Ken Ashcraft's google i/o 2008 talk recommended (without > > > referring specifically to timeout issues): > > > >http://sites.google.com/site/io/best-practices---building-a-productio... > > > > A bit troublesome but at least we know what to do: > > > every datastore read needs to do data structure integrity checking. > > > > I just want to get feedback on whether this line of thinking and the > > > resulting coding style is what the appengine team recommends. > > > Also, it would be very helpful to know what google engineers do to > > > avoid this issue (do they use the same style or do they have internal > > > tools or libraries that hide this issue from them). > > > More or less. If your write order is predictable, some of your queries will > > probably not care about incomplete data, or that they may see data that's > > only partially written. In general, though, the 'roll forward' approach > > works quite well. > > > -Nick Johnson > > > > Dennis > > > > search term: DatastoreTimeoutException > > > > On Jun 18, 6:01 pm, Iap <iap...@gmail.com> wrote: > > > > 2009/6/18 Nick Johnson (Google) nick.john...@google.com > > > > > > For datastore puts and deletes, we currently automatically retry when > > > > > timeouts occur, and for datastore transactions, your user code is > > > > > rerun > > > if a > > > > > timeout occurs. Gets and queries are currently not retried. > > > > > It's exciting that the GAE inherites the "I'm feeling lucky" tradition > > > > of > > > > Google. >_* > > > > > Why not retry the Get and Queires ? > > > > If the retry will eventually succeed, > > > > How about provide a decrator for convenience to do the things right. > > > > > Iap > > > -- > > Nick Johnson, App Engine Developer Programs Engineer > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > > 368047- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
