On Fri, 15 Jun 2012 12:34:12 +0200, Vincent Ladeuil <vila+...@canonical.com> wrote: > > It's not magic. It's moving from a database that's not designed for > > concurrent use to one that is designed for concurrent use. > > Despite not being designed for concurrent use, it *is* used this way and > lock contentions have been encountered leading me to believe that the > actual *design* needs to be fixed. The fact that changing the db is > triggering more contentions is a symptom of a deeper issue.
Changing the db access layer is triggering that, we were still running on the same (non-multi-user db). I agree that the design needs to be fixed, and that's exactly what we're taking about, fixing it by moving a db that is designed for multi-user use. > Well, when the correctness and safety is demonstrated, the context (and > hence my own answer) will probably be different but until then I just > can't say. > > > And I'm very reluctant to fork without an actual plan for merging > > back: how to know when it's safe & how to actually achieve it. > > And I have no idea (nor time right now) to debug the fallouts of such a > change that the actual package importer doesn't need. Hence my tendency > to consider that demonstrating the validity of this change should be > achieved first. But you just said above that you *do* think it needs to be fixed? How can we demonstrate the validity of the change? We can only demonstrate that it doesn't break production by running the changes in production. What would satisfy you that it was unlikely to break production? > Would there be a script to migrate from sqlite to PG ? Yes. > Can the package importer be re-started with empty dbs and catch up (how > long will it take ? Days ? Weeks ?). Can this trigger bugs because the > importer don't remember what it pushed to lp ? It can be restarted with the dbs from whenever the transition starts and it will catch up in roughly the between starting the transition and rolling back. There may be a few bugs due to replaying things, but we do it all the time (e.g. removing revids and re-importing when someone does push --overwrite) > Or do you expect us to see another peek like > http://webnumbr.com/ubuntu-package-import-failures.from%282012-01-24%29 ? Hopefully not. > Yes, that's why we're not in a position to safely accept such a change ! > > And all the time spent on integrating these changes is not spent on > allowing them to be accepted in good conditions. Sorry, I don't understand this, could you explain? > > but it demonstrated the locking problem and is how he came up with > > those options. > > Then the test improvements are certainly valuable to backport to lp:udd > or is there nothing to reuse from the EC2 experiment ? It wasn't a set of unit tests, it was a set of tests of a live system, so nothing to backport. > > We have had a lot of experience recently working with Canonical IS > > to get new servers and new staging servers deployed. If you want a > > staging server, we'd be happy to help you and would gladly > > advocate for you in their priority queue. > > Great to hear :) I should come back soon on this topic, just a quick > question though: are the new servers running lucid or precise ? New servers are being installed with precise. If an existing server is used it may be lucid, but there is a concerted effort to upgrade all machines to precise currently underway. Thanks, James -- ubuntu-distributed-devel mailing list ubuntu-distributed-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel
