Yeah 915 runs with 0 failures is really impressive. Especially knowing how many random issues we had with upgrades. Commit "Add docker and lxc to ISO" is dated April, 7th in fuel-main repo. There was a long journey I would say to get such a stability after first commits, with many engineers participating.
Keep Fuel rock stable! On Thu, Sep 4, 2014 at 2:23 AM, Roman Alekseenkov <ralekseen...@mirantis.com > wrote: > Evgeniy, > > This is great. The patch set looks excellent and thanks for stress testing > Docker. Very well done! > > Thanks, > Roman > > > On Wed, Sep 3, 2014 at 12:41 PM, Dmitry Borodaenko < > dborodae...@mirantis.com> wrote: > >> Impressive work Evgeniy! And special kudos for the detailed commit >> message, an example to be followed! >> >> On Wed, Sep 3, 2014 at 10:37 AM, Evgeniy L <e...@mirantis.com> wrote: >> > Hi, >> > >> > I would like to provide status for master node upgrade feature. >> > We merged a patch [1] which increased stability of upgrade. >> > >> > It fixes a lot of problems here is the list of some of them: >> > * fixed known and unknown raise conditions, e.g. keystone >> > db migration interruption >> > * now we won't have problem with ip duplication, I haven't >> > seen the problem, so I cannot say how often it happened, >> > but can say that the patch solves the problem in case of upgrade >> > * the patch twice reduces probability of docker's death during >> > the upgrade >> > >> > In the last 24 hours I tested the patch on 4 virtual machines, >> > there were 915 upgrade runs (to reduce the time of upgrade >> > I enabled only docker engine, which is the most problematic >> > part of master node upgrade), at the end of the day there >> > were 0 failed upgrades. >> > >> > Thanks, >> > >> > [1] https://review.openstack.org/#/c/118387/ >> > >> > >> > On Wed, Aug 27, 2014 at 4:05 PM, Matthew Mosesohn < >> mmoses...@mirantis.com> >> > wrote: >> >> >> >> Regarding paste.openstack.org, we should look to another pastebin >> >> provider. It has been giving 500 errors quite nearly consistently for >> >> me lately and really interferes with my work. We could use pastie.org, >> >> for example. >> >> >> >> On Wed, Aug 27, 2014 at 12:53 PM, Mike Scherbakov >> >> <mscherba...@mirantis.com> wrote: >> >> > OpenStack uses paste.openstack.org all the time, and I've heard >> issues >> >> > with >> >> > how long content is stored there. >> >> > >> >> > >> >> > On Wed, Aug 27, 2014 at 12:45 PM, Aleksandra Fedorova >> >> > <afedor...@mirantis.com> wrote: >> >> >> >> >> >> >> >> >> I can not find the policy about how long data is stored there, but I >> >> >> doubt >> >> >> that pastebin service can be used as a longterm storage. If we don't >> >> >> want to >> >> >> lose logs and scripts data, the use of paste.o.o links for bug >> reports >> >> >> should be forbidden. >> >> >> >> >> >> Launchpad attachments are much more reliable even though less >> >> >> comfortable >> >> >> to use. >> >> >> >> >> >> On Aug 27, 2014 8:49 AM, "Mike Scherbakov" < >> mscherba...@mirantis.com> >> >> >> wrote: >> >> >>> >> >> >>> +1 on updating bug descriptions (not comments) about probability of >> >> >>> failure, and using paste.openstack.org more often. >> >> >>> >> >> >>> >> >> >>> On Wed, Aug 27, 2014 at 3:41 AM, Dmitry Borodaenko >> >> >>> <dborodae...@mirantis.com> wrote: >> >> >>>> >> >> >>>> On Tue, Aug 26, 2014 at 12:11 AM, Mike Scherbakov >> >> >>>> <mscherba...@mirantis.com> wrote: >> >> >>>> > "confusing versioning in OpenStack patching" - if we didn't >> change >> >> >>>> > puppet >> >> >>>> > manifests and Fuel/OpenStack reference architecture in next Fuel >> >> >>>> > versions, >> >> >>>> > then it would be as simple as patching from 5.0 to 5.1. But it >> >> >>>> > appeared to >> >> >>>> > be more complicated system than you would initially think of, >> so in >> >> >>>> > general >> >> >>>> > 5.0.2 may not be equal to 5.1, that's where all things come up. >> If >> >> >>>> > we >> >> >>>> > had >> >> >>>> > OpenStack upgrades, then we could just say 5.0 -> 6.0 - easy. >> >> >>>> >> >> >>>> We may have had technical reasons to make this decision, but it >> still >> >> >>>> is confusing and negatively impacts UX. I agree that having an >> >> >>>> incomplete feature early is better than not having it at all until >> >> >>>> much later, as long as we don't stop working on it until it's >> >> >>>> complete >> >> >>>> and these small but annoying deficiencies are addressed. Our >> >> >>>> experience with technical debt so far is not very reassuring. >> >> >>>> >> >> >>>> > "issues with containers" - we have same issues with everything. >> >> >>>> > Let's >> >> >>>> > take >> >> >>>> > Galera, for example. It's just issues. We can question maturity >> of >> >> >>>> > tools we >> >> >>>> > use, and here I'd agree - we spent too much fixing issues around >> >> >>>> > Docker. At >> >> >>>> > the same time, if we were about taking our own journey with >> LXC, we >> >> >>>> > would >> >> >>>> > likely spend even more time inventing our own bicycle. >> >> >>>> >> >> >>>> You're assuming that it was just Docker as a piece of software >> that >> >> >>>> is >> >> >>>> the primary cause of all our troubles with Fuel upgrades. Docker >> is >> >> >>>> only a small part of the a much large and much more intrusive >> design >> >> >>>> decision to use containers for upgrading Fuel (and also the design >> >> >>>> decision to use a different mechanism based on Puppet for patching >> >> >>>> OpenStack). I think we should question high-level design decisions >> >> >>>> like these more often, even after they are implemented. >> >> >>>> >> >> >>>> > Also, I'd like to ask everyone to provide >> >> >>>> > such information in every bug you report if possible (or if get >> >> >>>> > this >> >> >>>> > info >> >> >>>> > later, put comments): in many bug reports it is unclear to >> >> >>>> > understand >> >> >>>> > how >> >> >>>> > severe issue is. >> >> >>>> >> >> >>>> I think we should start updating bug description more often, so >> that >> >> >>>> you can find a summary of current state of the bug and of all >> >> >>>> relevant >> >> >>>> information from the description, without having to scroll through >> >> >>>> dozens of comments. We should also use paste.openstack.org more >> >> >>>> heavily and avoid pasting more than 1-2 lines of logs into bug >> >> >>>> description and comments, also to make it easier to find important >> >> >>>> bits in bugs history. >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> Mike Scherbakov >> >> >>> #mihgen >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> Mailing list: https://launchpad.net/~fuel-dev >> >> >>> Post to : fuel-dev@lists.launchpad.net >> >> >>> Unsubscribe : https://launchpad.net/~fuel-dev >> >> >>> More help : https://help.launchpad.net/ListHelp >> >> >>> >> >> > >> >> > >> >> > >> >> > -- >> >> > Mike Scherbakov >> >> > #mihgen >> >> > >> >> > >> >> > -- >> >> > Mailing list: https://launchpad.net/~fuel-dev >> >> > Post to : fuel-dev@lists.launchpad.net >> >> > Unsubscribe : https://launchpad.net/~fuel-dev >> >> > More help : https://help.launchpad.net/ListHelp >> >> > >> > >> > >> > >> > -- >> > Mailing list: https://launchpad.net/~fuel-dev >> > Post to : fuel-dev@lists.launchpad.net >> > Unsubscribe : https://launchpad.net/~fuel-dev >> > More help : https://help.launchpad.net/ListHelp >> > >> >> >> >> -- >> Dmitry Borodaenko >> >> -- >> Mailing list: https://launchpad.net/~fuel-dev >> Post to : fuel-dev@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~fuel-dev >> More help : https://help.launchpad.net/ListHelp >> > > > -- > Mailing list: https://launchpad.net/~fuel-dev > Post to : fuel-dev@lists.launchpad.net > Unsubscribe : https://launchpad.net/~fuel-dev > More help : https://help.launchpad.net/ListHelp > > -- Mike Scherbakov #mihgen
-- Mailing list: https://launchpad.net/~fuel-dev Post to : fuel-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~fuel-dev More help : https://help.launchpad.net/ListHelp