This is a very interesting thread. And yes, indeed, cluster stability and uptime and transparency are all of the utmost importance to us here on the App Engine team.
I also agree that a fundamental increase in transparency would serve everyone's interests. As far as I can tell, transparency through a crisis occurs in one of three ways: (a) direct communication, (b) broadcasts of pr-approved messages, and (c) raw data. As an engineer, I like (c) more than (a) or (b). It's objective, and it can be distributed more efficiently. It's more efficient than (b) because we don't have to wordsmith it or get it reviewed and approved, it just goes out immediately. It's more efficient than (a) because it happens automatically; you don't take up the time of an engineer who would otherwise be busy trying to solve the problem. This is why our team has made it a priority to be very transparent about real time and detailed performance data with our Status Site: http://code.google.com/status/appengine. I encourage everyone who cares about their application to spend a bit of time learning that site. Make sure to click through to the detailed graphs as well. Further, if there is more data you would like us to export, then let us know. It is in everyone's best interests for us to export more raw data. It is my personal belief that by continuing to iterate and improve upon our Status Site that we will achieve the fundamental increase in transparency, as opposed to Daniel's proposal of dedicated hardware, direct communication, and promises as to when problems will be fixed. Although, Daniel does bring up the excellent point that there is not a particularly efficient way for developers to report live production problems other than this group. Any thoughts on systems that we should look into here to make this more efficient? Jon On Fri, Mar 6, 2009 at 9:25 AM, johnP <j...@thinkwave.com> wrote: > > My experience has shown that customers are generally OK with a > response: There is a problem. We are aware of it and people are > working on it. It is estimated to be resolved in X time. > > So if we (as vendors) are able to act as a positive PR army for > AppEngine, we need to be quickly armed with the data necessary to > accurately answer the above questions. > > If vendors (on the front lines of PR Triage) don't have this data, a > blame-game will start. No, we have no %$#$% idea what's wrong. No - > those #$$#&^%$ are not saying anything. Etc. > > So far, I'd say the following. Google has been almost good about > keeping the information flowing. There have been periods of 'radio > silence'; and the official status report usually reports that "There > has been an anomaly but we have determined that nobody was affected > (am I a nobody?)". But overall, they have responded. And have > resolved issues. And have upgraded the product. And in the end, have > built something really special. > > And I agree with Greg about sleeping at night. > > johnP > > > > On Mar 6, 5:42 am, peterk <peter.ke...@gmail.com> wrote: >> It's an interesting issue...I think we're all happy for things to >> behave pretty much like a black box when stuff is working as it's >> meant to, but are we so happy with that when things aren't working? >> >> Personally I don't need to know who specifically is responsible for >> the machines my apps run on, to be able to contact them directly etc. >> And I don't think that's really a viable option. >> >> However, I agree more feedback on causes and fix schedules when things >> go south would be good, along with frequent updates. I think 'naming >> and shaming' Google on infrastructure let downs, on your app, would be >> an option for you too, to lay the responsibility with Google. >> Significant issues, particularly on popular apps with decent >> visibility, are surely embarassing for Google given their reputation. >> Serving them a dose of that embarassment by telling your visitors 1) >> we're Google's infrastructure, 2) they're having problems right now, >> would probably give Google more incentive to avoid repetition of these >> issues.. :) If a number of popular apps were frequently reporting >> issues with Google, word about that would get around. >> >> From listening to talks from GAE folk though, I get the impression >> stability is of paramount importance..so I'm hopeful issues like those >> recently will be exceptions. >> >> On Mar 6, 2:24 am, dsw <daniel.wilker...@gmail.com> wrote: >> >> > I have long wanted to build apps on Google App Engine and have learned >> > a lot about it in preparation for doing so. However the one problem >> > is if I have a customer and their app goes down for an hour and they >> > call me and say "what happened?" and "how can we prevent that in the >> > future?" my only response will be "I don't know" and "we can't." >> > These are unacceptable answers. >> >> > If you want App Engine to "cross the chasm" and become really for real >> > then at the very least what you need to do is provide the kind of >> > depth of sight into your infrastructure that you the Google engineers >> > have. Further you need some kind of locality in the cloud: it would >> > help if there were some way of ensuring reliability by knowing that >> > (1) I have bought space on some particular cluster of machines (2) >> > which is now stable and more apps are not being added to it; I should >> > know (3) who is maintaining that cluster and (4) be able to send them >> > a trouble ticket and (5) have some idea of what is wrong and how long >> > it is going to take them to fix it. >> >> > This opaque cloud utility of compute stuff is a fantasy: some locality >> > and transparency will be needed or App Engine will never be really for >> > real. >> >> > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
