I once wrote a pretty decent schema migration tool that fits most if not all of these requirements. It was built for the Kohana PHP framework, but a lot of it is pretty independent of that. If someone ends up working on this I'd love to help and maybe share some code and ideas.
-Andrew Otto http://ottomata.org http://www.flickr.com/photos/OttomatonA http://www.couchsurfing.org/people/otto On Apr 25, 2012, at 12:58 PM, Asher Feldman wrote: > I am generally in favor of all of this and in the meeting that proceeded > Rob's email, proposed that we develop a new schema migration tool for > mediawiki along similar lines. Such a beast would have to work in all > deployment cases without modifications (stock single wiki installs and at > wmf with many wikis across multiple masters with tiered replication), be > idempotent when run across many databases, track version and state per > migration, and include up/down steps in every migration. > > There are opensource php migration tools modeled along those used by the > popular ruby and python frameworks. I deployed > https://github.com/davejkiger/mysql-php-migrations at kiva.org a couple > years ago where it worked well and is still in use. Nothing will meet our > needs off the shelf though. A good project could at best be forked into > mediawiki with modifications if the license allows it, or more likely serve > as a model for our own development. > > On Tue, Apr 24, 2012 at 11:27 PM, Faidon Liambotis > <fai...@wikimedia.org>wrote: > >> >> In other systems I've worked before, such problems have been solved by >> each schema-breaking version providing schema *and data* migrations for >> both forward *and backward* steps. > > >> This means that the upgrade transition mechanism knew how to add or >> remove columns or tables *and* how to fill them with data (say by >> concatenating two columns of the old schema). The same program would >> also take care to do the exact opposite steps in a the migration's >> backward method, in case a rollback was needed. >> > > Down migrations aid development; I find them most useful as documentation > of prior state, making a migration readable as a diff. They generally > aren't useful in production environments at scale though, which developers > removed from the workings of production need to be aware of. Even with > transparent execution of migrations, the time it takes to apply changes > will nearly always be far outside of the acceptable bounds of an emergency > response necessitating a code rollback. So except in obvious cases such as > adding new tables, care is needed to keep forward migration backwards > compatible with code as much as possible. > > The migrations themselves can be kept in the source tree, perhaps even >> versioned and with the schema version kept in the database, so that both >> us and external users can at any time forward their database to any >> later version, automagically. > > > Yep. That we have to pull in migrations from both core and many extensions > (many projects, one migration system) while also running different sets of > extensions across different wikis intermingling on the same database > servers adds some complexity but we should get there. > > -Asher > _______________________________________________ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
