Scott: Nacho is the author of SimpleDS.
Schema migration is something that Hibernate and RDBMSes actually do
rather poorly. The typical process is to prepare a series of scripts
(ALTER TABLE and then any relevant data transmogrification), shut down
the application, run the scripts, then bring up new code that
understands the new schema. Hibernate may help provide the scripts,
but it won't prevent your downtime.
With large data volumes and significant changes, this process could
take a very very long time.
The schemaless nature of AppEngine makes migration both easier and
harder. Any entity can have any shape, but there are no bulk
operations. Furthermore, iterating through your dataset and resaving
each entity is a *very slow* operation. Batch jobs on my entities
(which have very few indexes) convert less than 100 instances per
second. Want to convert 1 million entities? Nearly 3 hours. 10
million? 100 million? The math is easy.
We designed the Objectify schema migration tools with the assumptions:
1) You have vast quantities of data
2) The data is changing rapidly
3) Any amount of downtime is unacceptable
All transformations within a single entity are pretty easy and
straightforward using @AlsoLoad (on a field, lets you load the old
name as well as the new) or on a method parameter (lets you obtain the
raw data as whatever format you need, munge it, and save it to
whatever set of fields you care).
Condensing multiple entities into a single entity is also fairly
straightforward using @PostLoad and @PrePersist. I'll provide an
example if you want.
Splitting apart one entity into multiple is by far the hardest
transformation, and there isn't any one solution that works for
everyone. However, I have done it. Here is one strategy:
---- Starting Entity ----
class Person {
@Id Long id;
String addressStreet;
String addressCity;
}
Goal: Create an Address entity and add a foreign key reference in Person.
---- Translating Entity ----
class Address {
@Id Long id;
String street;
String city;
}
class Person {
@Id Long id;
@LoadOnly String addressStreet;
@LoadOnly String addressCity;
Key<Address> address;
@PrePersist void onSave() {
if (addressStreet != null || addressCity != null) {
Address addy = new Address(addressStreet, addressCity);
address = ObjectifyService.begin().put(addy);
}
}
}
You might also need to implement the getAddressStreet() and
getAddresCity() methods with a check to the address Key and a load of
the Address object.
It's somewhat convoluted, but it does work. Another alternative,
which gets rid of the complicated getAddressStreet() implementation is
to perform the conversion onLoad():
class Person {
@Id Long id;
@LoadOnly String addressStreet;
@LoadOnly String addressCity;
Key<Address> address;
@PostLoad void onLoad) {
if (addressStreet != null || addressCity != null) {
Address addy = new Address(addressStreet, addressCity);
address = ObjectifyService.begin().put(addy);
ObjectifyService.begin().put(this);
}
}
}
It really depends on the shape of your data and your query profile -
if your app reads a *lot* of entities, you might find the performance
profile of convert-on-save to be better. Keep in mind that this type
of conversion is a "worst case scenario". Most other schema
migrations are fairly simple.
Jeff
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
