Speaking only for my app, I can tell this is not an application issue. My datastore is made up of only root entities and in most cases works well quickly. My app doesn't see large volumes of requests per second and it's more like requests per minute. There is no possible way the errors I'm seeing are from contention given the minutes between requests. And, the same exact process is handled a few hours before within 600ms. My app is soon to increase in volume and needs to be stable. I love using app engine, but I've never faced errors like this on any other platform from IIS Server and SQL Server to PHP and MY SQL or even Unix and Oracle. Don't get me wrong, I understand the difference in the platforms, If there is anything I can do to prevent this I would love to know. I'm not moving my app, but need to find a solution. My company will be on the local NBC news (I will be mentioning app engine) and may see a huge volume hitting the site next week.
On Apr 17, 4:54 am, Sylvain <sylvain.viv...@gmail.com> wrote: > Datastore timeout is one of the biggest (and oldest) issue with GAE > and mostly because it is random. > You can get it on get, put, fetch,... even with few entities. > > I hope that soon, datastore timeout will be negligible. Currently, it > is not the case. > > Regards > > On 17 avr, 09:50, Paul Kinlan <paul.kin...@gmail.com> wrote: > > > Hi, > > > I would just like to add myself into this, my app twitterautofollow (twollo) > > regularly gets DataStore time outs on puts (in the most part). All my > > entities are root entities. > > > I did have a thread open on this only a few days ago. > > > Paul > > > 2009/4/17RayMalone <rayish...@gmail.com> > > > > I'm seeing random timeouts where there are many seconds or minutes > > > between requests and hours before app engine handled the same request > > > within 400ms. I hit another one today that will actually cost my > > > business, not much but some. My app needs to catch these and I am on > > > many pain points. However, I think the timeouts needs addressed. > > > > On Apr 16, 7:10 pm, DarkCoiote <darkcoi...@gmail.com> wrote: > > > > Yes... I've read that... > > > > > The problem with using all entities as root is that I'm unable to use > > > > transactions as it is... I would have to code a lit bit... although > > > > I just found a paper describing a project that seems really good. > > > > >http://danielwilkerson.com/dist-trans-gae.html > > > > > I think it will be presented this weekend, and I really hope that > > > > it goes "public" or even yet plugged to app engine! > > > > > Just double-checked my GAE logs, and it seems that I'm getting > > > > timeouts in "get" operations as well... multiple requests could cause > > > > that too (one get 'over' a put for example)... > > > > > Have to check my code.... > > > > > Thank you > > > > > On Apr 16, 2:10 pm, "Jeff S (Google)" <j...@google.com> wrote: > > > > > > On Thu, Apr 16, 2009 at 4:16 AM, DarkCoiote <darkcoi...@gmail.com> > > > wrote: > > > > > > > On Apr 15, 3:26 pm, "Jeff S (Google)" <j...@google.com> wrote: > > > > > > > HiRay, > > > > > > > > Which operation was is that timed out (get, query, put)? Also, how > > > > > > > consistently are you seeing these timeouts? > > > > > > > > I generally recommend catching datastore timeouts and handling > > > > > > > them > > > in a > > > > > > way > > > > > > > that makes sense for your app. There are currently occasional > > > (quite rare > > > > > > as > > > > > > > a percentage) timeouts for queries and gets, and timeouts > > > > > > > > on a put is often > > > > > > > an indicator of contention on that entity or entity group. > > > > > > > well... almost (if not all) of the timeouts I'm seeing are on 'put' > > > > > > operations... > > > > > > but contention would need, like, 2 or more operations on the same > > > > > > entity > > > > > > (all my entities are roots... stupid, I know ), right? > > > > > > Actually, making most of your entities roots is often better than deep > > > > > ancestor trees in terms of overall write throughput. In a > > > > > transactional > > > > > write a entity is updated which has a parent entity, the ancestors are > > > > > locked. So if nearly concurrent requests update different entities > > > which all > > > > > share a common ancestor (in other words the entities are in the same > > > entity > > > > > group), some of the child entity updates could fail due to contention > > > on > > > > > writes. There are more details here in the documentation: > > > > > >http://code.google.com/appengine/docs/python/datastore/keysandentityg. > > > .. > > > > > > """ > > > > > Tips for using entity groups: > > > > > > - Only use entity groups when they are needed for transactions. For > > > other > > > > > relationships between entities, use ReferenceProperty properties and > > > Key > > > > > values, which can be used in queries. > > > > > > - The more entity groups your application has—that is, the more root > > > > > entities there are—the more efficiently the datastore can distribute > > > the > > > > > entity groups across datastore nodes. Better distribution improves the > > > > > performance of creating and updating data. Also, multiple users > > > attempting > > > > > to update entities in the same entity group at the same time will > > > > > cause > > > some > > > > > users to retry their transactions, possibly causing some to fail to > > > commit > > > > > changes. Do not put all of the application's entities under one root. > > > > > > - A good rule of thumb for entity groups is that they should be about > > > the > > > > > size of a single user's worth of data or smaller. > > > > > > - Entity groups do not have a significant impact on the speed of > > > queries. > > > > > """ > > > > > > If your entities are roots, then yes contention could occur if the > > > > > same > > > > > entity were updated by overlapping requests. > > > > > > Thank you, > > > > > > Jeff > > > > > > > I'll check for bugs that could cause multiples requests and stuff > > > like > > > > > > that.... > > > > > > > Thank you > > > > > > > > Happy coding, > > > > > > > > Jeff > > > > > > > > On Mon, Apr 13, 2009 at 4:11 PM, DarkCoiote <darkcoi...@gmail.com> > > > > > > wrote: > > > > > > > > > Looks like my and others problem... posted a few days ago: > > > >http://groups.google.com/group/google-appengine/browse_thread/thread/. > > > > > > .. > > > > > > > > > Random datastore timeouts in totally unexpected places... > > > > > > > > > On Apr 13, 2:36 pm,RayMalone <rayish...@gmail.com> wrote: > > > > > > > > > I'm not sure why this occurs all of a sudden. The same job > > > runs in > > > > > > > > > 600ms. Then once in a while this runs over 6800 MS and > > > > > > > > > times > > > out. > > > > > > > > > I can't have jobs just timeout for no reason. According to > > > my logs > > > > > > > > > during this time the only other request was a second prior. > > > This > > > > > > > > > occurred at 04-13 09:13AM 29.627 today and other times at > > > random in > > > > > > > > > the last few days. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
