On Wed, Apr 8, 2015 at 6:09 PM, Joshua Randall <josh.rand...@genomicsltd.com > wrote:
> > I'd want to hear from others on this matter, as my experience with > repairing upgrades is limited. > > I recommend you don't take any action on the following immediately, > until others have weighed in. > I will hold off doing anything else for now, as I don’t want to risk > losing our running environment (everything that Juju set up still seems to > be running fine at the moment, we just can’t make any changes). > > > John figured that the error you've got could be related to having > containers in the environment. > > Unfortunately it appears that the upgrade step doesn't work if there are > any containers, as it > > mistakenly thinks they're MAAS machines (and is rightly told that they > don't exist in MAAS). It'd > > be great to get confirmation that you do have containers in your > environment. > Yes! We are using containers, and they are present on machine 0. Actually, > one of the reasons we are upgrading Juju is in the hopes to resolve another > issue which seems related to having containers on machine 0, which was that > the cloud-init user-data that Juju has started sending to MAAS was using > the IP address of the LXC bridge for the state server instead of the real > IP address, so a machine we were trying to enroll into Juju hasn’t been > able to finish because it can’t contact the Juju state server. I saw some > bug fix changes that went into the 1.22 version that I thought might fix > that problem, which is why I was trying the upgrade. > > Anyway, it sounds like what you are saying is that the upgrade state > database migration step is iterating through a list that contains all of > the MAAS machines as well as the containers when it should just be > iterating through the MAAS machines themselves? Correct. > > Now, this could be repaired without *too* much trouble I think. That > upgrade step is the last one > > to upgrade to 1.22 completely. So, options are: > > 1. Hand-edit the agent's config file so it thinks it's finished > upgrading. > > 2. Fix Mongo so the upgrade step thinks it has nothing to do. > > 3. Wait for a fix to Juju. I'm not sure how long that'll take, as we're > all about to travel. I've just filed: > > https://bugs.launchpad.net/juju-core/+bug/1441478 > Thanks for filing the bug! I’ve subscribed to it and am happy to move this > discussion there. I’ll monitor both places. > > > #1 will work, but will mean that you'll have no availability-zone > information in your instances. > > I'm not sure if that'd cause any immediate problems (Eric?), but it will > mean that charms won't > > be able to take advantage of that information. > > > > #2 would be preferable; this would involve opening the Juju state > database with the mongo > > shell, and updating all documents in the "instancedata" collection so > that they have "availzone" > > fields. > I’ll take a look at the Juju state mongodb, and take a backup in > preparation for making changes, but I’ll hold off on making them for now. > > Should there be a default value entered for “availzone” or is an empty > value fine? My reading of the code (I think the relevant bit is: > https://github.com/juju/juju/blob/1.22/state/upgrades.go#L559-582) is > that just having the availzone key should be fine to convince the upgrade > it is done, but I’m not sure whether the availzone value might be used > during operations (will it safely be ignored for containers and/or is the > empty string a safe value to set it to?). > This is the bit I'm unsure of. I *think* it is only used for (a) FYI (in status), and (b) for the information of charms. Charms are able to identify the availability zone their machine is allocated to, and then organise appropriately (e.g. to minimise cross-zone traffic, or to maximise redundancy). Since you've said there's only one zone, I'd guess you could just set the value to that for all of the MAAS machines, and "" for the containers. Eric, who is currently at PyCon, will hopefully be able to answer on this definitively after the conference. Also, it seems likely that the upgrade will have bailed out as soon as the > interator hit the first container, meaning it may not have properly set the > availzone for all of our MAAS machines. I suppose I could hope that it at > least did machine 0 and use that value for the other MAAS machines (in any > case, we only have one availability zone). > > Cheers, > > Josh. > > > -- > This email is sent on behalf of Genomics Limited, a limited company > registered in England and Wales with registered number 8839972, VAT > registered number 189 2635 65 and registered office at 24 Cornhill, London, > EC3V 3ND, United Kingdom. > The contents of this e-mail and any attachments are confidential to the > intended recipient. If you are not the intended recipient please do not use > or publish its contents, contact Genomics Limited immediately at > i...@genomicsltd.com then delete. You may not copy, forward, use or > disclose the contents of this email to anybody else if you are not the > intended recipient. Emails are not secure and may contain viruses. >
-- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
