On Wednesday, August 17, 2011 5:24:48 AM UTC-7, Joshua Smith wrote: > > I certainly hope they don't deprecate M/S, unless/until the following > issues are addressed: > > 1) Ability to migrate without changing app ID. This is fundamental, and > we've never seen any *reason* why this is impossible. I suspect that this > is very hard, but not impossible. > We are able to create an alias for migrated apps, so that traffic sent to oldappid.appspot.com will be transparently routed to your new application. So while you won't be able to keep the same app id, you can keep using the same appspot.com domain, which is probably what you care about most.
> Changing the app ID causes problems because: > - External references to entity keys (not IDs) are pervasive in many apps > (such as google search links to deep pages, user bookmarks, or RSS feed > URLs) because the documentation encourages the use of these keys in URLs > This remains an issues, but can be easily addressed in a couple of lines of application code in the appropriate request handlers. The string encoded keys that are referenced externally have encoded within them the old app id. If you take the decoded key and extract the parent, kind and id/name to create a new key (again, this is 1-2 lines of code), then your app can use that new key to locate the migrated entity within the new datastore. As far as reference (list) properties: those reference keys are automatically updated as part of the the datastore migration and should not cause you any issues. > - Many users have keys wrapped up in properties that the migration tool > cannot recognize [this doesn't impact my app, but it's a common issue on > this list] > This does affect some apps. As I mentioned above, a couple of lines of code in the right place should take care of this. > As an alternative, the infrastructure could be fixed to recognize the case > of using the old app ID instead of the new app ID in a key, and quietly make > the substitution. > That's a neat idea. > 2) Migration of large data sets takes an extremely long time. Given that > we regularly see M/S downtimes in the couple of hours range, a couple of > hours is probably a reasonable amount of time to migrate. A couple of days > is much too much. [Note that I have not personally gotten far enough into a > migration to see this duration; I'm basing this on many user complaints I've > read on this list.] > Applications with small amounts of data can easily self migrate in a short period of time using datastore_admin. Applications with more data will benefit from the new migration tool Ikai mentioned. We created the tool specifically to simplify and speed up the migration for larger apps, and to reduce the read-only period required to make the final switch. > 3) The problem of transactional consistency needs to be addressed. It is > not acceptable for transactions to end in a "maybe this worked, maybe it > didn't" state. They should either succeed or fail. (Although the docs not > not say whether this problem is exclusive to HR, it seems to be, and > certainly it matters much more in HR since every put is implicitly > transactional.) > There are subtle differences between the consistency guarantees offered by M/S and HRD. In some cases, apps need to make a few tweaks in order to work correctly in the new environment. The benefit is much more predictable/consistency latency and increased reliability. A good article which reviews the consistency guarantees can be found here: http://code.google.com/appengine/articles/transaction_isolation.html 4) As Ikai said, there are some unresolved issues in HR that need to be > fixed, such as the current mystery in the 'Serious Problem: Rollback of data > on HRD' thread (which is eerily familiar: I think somebody else reported a > similar issue a while back). > I haven't looked at that thread yet, but I suspect that reviewing the above article (http://code.google.com/appengine/articles/transaction_isolation.html) would probably be beneficial in the context of that thread. Even if all these are addressed, it would certainly be nice if M/S remained > an option. It makes different trade-offs than HR, which may be better > trade-offs for some applications. I will be very disappointed if they > decide to make HR a requirement for multi-threading Python, since the two > are completely orthogonal choices. From my reading, M/S seems to have > better throughput than HR, at the expense of more frequent downtime, and > spurious datastore timeouts. I'd like to have that option in a > multi-threaded app. > Thanks for the input. I hope the above response addresses some of your concerns. > -Joshua > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/sLkBsAfFc84J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
