For this particular case I would consider doing a two pass process. Add the encrypted password to the datastore on the first pass. Then switch the code to use the encrypted passwords, and finally remove the old passwords. The advantages are that you can do your final testing on live data, it's easy to roll back, and there's no need to code a special transition case.
On Sep 23, 10:03 am, andy stevko <andy.ste...@gmail.com> wrote: > Not being very familiar with the namespace apis, is it possible with the > approach below to convert only a portion of the datastore object model and > not duplicate the entire datastore? > > For example - convert Users and Passwords but leave the weakly related > Accounts in place. > > > > On Thu, Sep 23, 2010 at 8:27 AM, Eli Jones <eli.jo...@gmail.com> wrote: > > Yes, you'll need to duplicate your data. > > > This is the cleanest way to do this.. in my mind. You would have > > two separate versions of your app.. and you could easily switch back to your > > old version of you realized some horrible mistake was made after switching > > to the new version. > > > The main work you would need to do for the changeover from Old Version to > > New Version would be: > > > 1. Write code that converts old datastore data to the New Version > > namespace data. > > 2. Test New Version datastore with converted data on New App version. > > 3. Schedule a downtime late at night where the app is brought offline > > (just upload a dummy app.yaml for the Old Version that points /.* to some > > message page.. or you could be official and have a "Maintenance Period" > > version of your app that you make live.. so that you make no changes > > whatsoever to the "Old Version".), and the datastore conversion code is run > > after you are comfortable that no users are using the site and are all > > viewing the "maintenance period" messages. > > 4. Once complete, make New Version app the live version. > > > To me, it seems like a massive headache to try to convert the data in > > place.. in the same datastore.. you have to have new model names or at least > > some new property for "version" on your models.. and then your key_names > > have to be different.. etc.. If you screw something up, that's a potential > > headache for your live app.. with the New Version namespace.. you can > > freely move ahead with experimenting with your coding.. and even make plenty > > of changes to your models (if you think they are beneficial) without > > thinking "I don't know.. it's going to be a pain in the ass to figure out > > how to keep track of the old version and new version entities and now I have > > to add a new model or property definition to my schema code.." > > > With a namespace, you just have to add the correct mapping, conversion code > > to your datastore converter code, and make schema changes to your New > > Version app code.. and just leave the Old Version datastore and code alone. > > > You can also get fancy with your converter.. (depending on how the > > conversion actually works) and you can have your converter code get a cursor > > with keys_only for each model, serialize that cursor and pass it off to a > > task that then iterates through the cursor and breaks up the keys into > > batches to be converted and fires off tasks to do the batches in parallel. > > And, to make the process simpler, you could just used the deferred api.. > > (so you don't have to deal with setting up task handler urls or anything > > like that). > > > However big your datastore is.. and however much money it might cost to > > have your data duplicated (it costs $0.01 a day for 2 GB of data above the > > free limit).. I think it would be worth it.. it would be valuable to learn > > to do it this way, since a large scale site would (or should) do something > > more like this. > > > On Wed, Sep 22, 2010 at 9:39 PM, Kyle Baley <k...@baley.org> wrote: > > >> That's an interesting idea. But if you do that, wouldn't you need to > >> first copy all the data from the v1 namespace to the v2 namespace? > > >> On Sep 22, 7:07 pm, Eli Jones <eli.jo...@gmail.com> wrote: > >> > It might be useful for you to use a namespace for the new version of the > >> > datastore. > > >> > Thus, you could have the "new version" of the app deployed as a non-live > >> > version of the app.. and code that "new version" to use the "new version > >> > datastore" namespace. > > >> > Then, when you are ready.. just change the live version of your app to > >> the > >> > "new version". > > >> > Here's a link: > > >> >http://code.google.com/appengine/docs/python/multitenancy/multitenanc. > >> .. > > >> > < > >>http://code.google.com/appengine/docs/python/multitenancy/multitenanc.. > >> .>So.. > >> > you could just think of your version 1 datastore as "that old customer > >> who > >> > we're going to dump just as soon as our new version 2 datastore customer > >> is > >> > ready"... or something like that. > > >> > It's better (I think) than adding a "version" property to all of your > >> models > >> > or trying to maintain model consistency between app versions. > > >> > On Mon, Sep 20, 2010 at 2:29 PM, Kyle Baley <k...@baley.org> wrote: > >> > > We've just released the first version of our application and are now > >> > > looking at a problem we've been avoiding until now. Namely, what is > >> > > the best way to upgrade the application to a new version that requires > >> > > changes to the datastore. We're looking at two options: > > >> > > 1) Big Bang Upgrade > >> > > We take the application down and run an upgrade process to update all > >> > > entities from version 1 to version 2. > > >> > > Pros: Easy to maintain; intuitive > >> > > Cons: App has to be taken down for a period of time, which will > >> > > increase as time passes and more data is added to the datastore > >> > > (potentially hitting the limit for long-running processes eventually) > >> > > Question: What's a good way to take the app offline? > > >> > > 2) Version Entities Individually > >> > > Each entity has a version number and we have a series of commands, > >> > > each one responsible for upgrading an entity from one version to the > >> > > next. As we request entities, we check to see if it's the latest > >> > > version. If not, we run each necessary upgrade command in sequence > >> > > until it is the latest version. > > >> > > Pros: No need to take the app offline; provides flexibility on whether > >> > > to upgrade everything at once or piecemeal > >> > > Cons: Not as intuitive; entities with different versions in the > >> > > datastore (if that matters) > > >> > > What do other people do to upgrade their datastore for a live > >> > > application? > > >> > > -- > >> > > You received this message because you are subscribed to the Google > >> Groups > >> > > "Google App Engine" group. > >> > > To post to this group, send email to > >> google-appeng...@googlegroups.com. > >> > > To unsubscribe from this group, send email to > >> > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2Bunsubscrib > >> > > e...@googlegroups.com><google-appengine%2Bunsubscrib > >> e...@googlegroups.com> > >> > > . > >> > > For more options, visit this group at > >> > >http://groups.google.com/group/google-appengine?hl=en. > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine" group. > >> To post to this group, send email to google-appeng...@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com<google-appengine%2Bunsubscrib > >> e...@googlegroups.com> > >> . > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2Bunsubscrib > > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
