Re: [ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs
Thank you all for helping me in this issue. Finally vdsm.gluster modules are moved python site-packages. Regards, Ramesh - Original Message - > From: "Dan Kenigsberg" > To: "Ramesh Nachimuthu" > Cc: "Aravinda" , igoih...@redhat.com, "Nir Soffer" > , "devel" > , "Piotr Kliczewski" , "Yaniv Bronheim" > , "Sahina Bose" > > Sent: Thursday, December 1, 2016 5:16:34 PM > Subject: Re: python path conflicts issues while running python scripts from > vdsm verbs > > On Thu, Dec 01, 2016 at 03:38:36AM -0500, Ramesh Nachimuthu wrote: > > > > > > > > > > - Original Message - > > > From: "Dan Kenigsberg" > > > To: "Ramesh Nachimuthu" > > > Cc: "Aravinda" , igoih...@redhat.com, "Nir Soffer" > > > , "devel" > > > , "Piotr Kliczewski" , "Yaniv > > > Bronheim" , "Sahina Bose" > > > > > > Sent: Wednesday, November 30, 2016 4:54:59 PM > > > Subject: Re: python path conflicts issues while running python scripts > > > from vdsm verbs > > > > > > On Wed, Nov 30, 2016 at 03:49:02AM -0500, Ramesh Nachimuthu wrote: > > > > > > > > > > > > > > > > > > > > - Original Message - > > > > > From: "Dan Kenigsberg" > > > > > To: "Aravinda" , igoih...@redhat.com > > > > > Cc: "Ramesh Nachimuthu" , "Nir Soffer" > > > > > , "devel" , "Piotr > > > > > Kliczewski" , "Yaniv Bronheim" > > > > > , "Sahina Bose" > > > > > Sent: Wednesday, November 30, 2016 2:02:43 PM > > > > > Subject: Re: python path conflicts issues while running python > > > > > scripts > > > > > from vdsm verbs > > > > > > > > > > > > > > > Regardless of the solution you plan to take, let it serve as a > > > > > reminder > > > > > about moving gluster bits out of /usr/share/vdsm/ and into > > > > > site-packages. I am willing to pay the price of an ugly > > > > > exception-handling in xmlrpc, if this is the cost. > > > > > > > > Yes. I think this is the right time to revive the patch > > > > https://gerrit.ovirt.org/#/c/57530/ and > > > > https://gerrit.ovirt.org/#/c/57524/ to move all the gluster code to > > > > site-packages. Let me re-post these packages for review. > > > > > > Please do. > > > > > > IIRC https://gerrit.ovirt.org/#/c/57530/9 is broken in xmlrpc. > > > > > > Since xmlrpc is going to be used only by the command line in 4.1, I'd be > > > pleased to see the problem solved via a simplistic hack. > > > > > > > > > I have updated the patch https://gerrit.ovirt.org/#/c/57530/ for moving > > vdsm/gluster into site-packages. I took the patch > > https://gerrit.ovirt.org/#/c/57524/ as an workaround to solve the ClientIF > > issue. > > > > Dan: Please let me know if you have any simple fix for clientIF issue. > > Thanks. I think that the approach of your > https://gerrit.ovirt.org/#/c/67660/ is much better. > ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] Deprecating Defaults for to_all_distros Setting in repoman
> > > I guess the most impacted will be ovirt release engineering while working > on 4.0 repos but being 4.1 coming in 2 months we can survive. Just keep at > hand an old el7 rpm, just in case. > But it will be possible to have the previous behavior just by updating repoman configuration file with the list of packages. So it will not be hardcoded inside the source code, but instead stored in local configuration. Just let me know whom to contact so we can update and test the repoman configuration before the change is merged to make sure it won't cause any issues. Anton. -- Anton Marchukov Senior Software Engineer - RHEV CI - Red Hat ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] Deprecating Defaults for to_all_distros Setting in repoman
Il 01/Dic/2016 08:07 PM, "Anton Marchukov" ha scritto: > > Hello All. > > If you use repoman please be aware that it currently has hardcoded list of packages that defaults to being distroless. This is controlled by "to_all_distros" configuration setting, but if you do not set it than the defaults are used. > > For ovirt experimental we decided that we better have all packages to have distro specified in spec. This also inline with most distro packaging guidelines. But then with current defaults repoman will consider those package to be distroless even when they are not and specifying an empty list in configuration is a silly way to override this. > > This change provides an incompatibility, but you can still use this feature by specifying to_all_distros package list explicitly in the config. So it is just the change of default values at this point: > > https://gerrit.ovirt.org/#/c/67662/ > > If somebody is not ok with defaults change and need time to update configuration please let me know. Also if you are not sure we can go another way and have intermediate repoman version that does not do the change, but issue deprecation warning if defaults are being used, so everybody can test their setups. However I think that it is might not be actually needed in this case as it is a configuration thing. > > Please feel free to comment on change request. I guess the most impacted will be ovirt release engineering while working on 4.0 repos but being 4.1 coming in 2 months we can survive. Just keep at hand an old el7 rpm, just in case. > > Anton. > > -- > Anton Marchukov > Senior Software Engineer - RHEV CI - Red Hat > > > ___ > Infra mailing list > in...@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] Gerrit parallel patch handling and CI (Or, why did my code fail post-merge)
On 1 December 2016 at 19:19, Vojtech Szocs wrote: > > Thanks for sharing those links. > > I didn't know you're already working on adopting Zuul, my bad =) It ok, I guess oVirt Jira is not anyone's favorite reading material ;) > So the maintainer can simply express his/her intent to merge the given > patch, and CI infra takes care of the rest (run heavy tests and submit > changes if successful). Yep that is where we want to be. > I'd be cautious with this feature, since our heavy CI tests involve > GWT compilation, so Zuul trying to run more tests (on different patch > combinations) = more time spent. Its not something that Zuul allows you to heavily customize - and I am hoping we will manage to get more efficient with the builds. Besides - looking at a common scenario now - where the maintainer merges a set of patches before leaving the office for the night, having the CI system crunch all night and bring results in the morning is not such a bad thing. Way better then finding out which patch broke the experimental flow 3 days later. > We cannot rule out issues that might happen in future. There will be > flaky/broken tests or CI infra issues, we need to decide how to deal > with those, I think. Hopefully we get to the point where they are rare. > It was just an idea =) Zuul sounds more like a proper solution. Just to make things clear - Zuul does not do any magic with Gerrit's "submit" button - one needs to use a flag instead. -- Barak Korren bkor...@redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/ ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
[ovirt-devel] Deprecating Defaults for to_all_distros Setting in repoman
Hello All. If you use repoman please be aware that it currently has hardcoded list of packages that defaults to being distroless. This is controlled by "to_all_distros" configuration setting, but if you do not set it than the defaults are used. For ovirt experimental we decided that we better have all packages to have distro specified in spec. This also inline with most distro packaging guidelines. But then with current defaults repoman will consider those package to be distroless even when they are not and specifying an empty list in configuration is a silly way to override this. This change provides an incompatibility, but you can still use this feature by specifying to_all_distros package list explicitly in the config. So it is just the change of default values at this point: https://gerrit.ovirt.org/#/c/67662/ If somebody is not ok with defaults change and need time to update configuration please let me know. Also if you are not sure we can go another way and have intermediate repoman version that does not do the change, but issue deprecation warning if defaults are being used, so everybody can test their setups. However I think that it is might not be actually needed in this case as it is a configuration thing. Please feel free to comment on change request. Anton. -- Anton Marchukov Senior Software Engineer - RHEV CI - Red Hat ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] Gerrit parallel patch handling and CI (Or, why did my code fail post-merge)
- Original Message - > From: "Barak Korren" > To: "Vojtech Szocs" > Cc: "devel" > Sent: Thursday, December 1, 2016 9:10:19 AM > Subject: Re: [ovirt-devel] Gerrit parallel patch handling and CI (Or, why did > my code fail post-merge) > > On 30 November 2016 at 19:12, Vojtech Szocs wrote: > > > > So before we adopt auto-submit-after-ci-pass (e.g. using Zuul, as Eyal > > mentioned in that thread), we should be able to manually trigger heavy > > CI (check-merged) job from Gerrit web interface, is that correct? > > We could make it a two-step operation where you indicate you want to > submit, wait for the CI output and then click submit - but why not > just let the system submit for you? > > One road I don't think we should go down is to have 3 CI stages: > "light" check_patch, "heavy" check_patch and "pre-merge", this looks > like a design smell to me. > > WRT Zuul: > > Unless you are using a strict policy like ff-only (which so far seems > no to scale, because it is essentially a GIL on merging), Zuul is > essential in order to test the right code in CI (even just for basic > check_patch testing). So we are working to bring it in (See OVIRT-734 > [1] and OVIRT-882 [2]). Thanks for sharing those links. I didn't know you're already working on adopting Zuul, my bad =) So the maintainer can simply express his/her intent to merge the given patch, and CI infra takes care of the rest (run heavy tests and submit changes if successful). > > WRT merge gating - Zuul tries to make the process "fire and forget" as > far as maintainers are concerned - if failures are found, it actually > tries out different combinations of patches to see if, from a set of > patches that were asked to be merged, some could be merged even when > others could not. I'd be cautious with this feature, since our heavy CI tests involve GWT compilation, so Zuul trying to run more tests (on different patch combinations) = more time spent. > > > If so, it would be the patch owner's responsibility to submit (merge) > > only after heavy CI (check-merged) pass? > > The question hiding here is wither a maintainer could submit _without_ > passing the "heavy" CI. I'm guessing some maintainers will demand > that, but I'm hoping in the long run this will not be needed and you > will know that once submitted, a patch will be eventually merged > unless it has real issues in it. We cannot rule out issues that might happen in future. There will be flaky/broken tests or CI infra issues, we need to decide how to deal with those, I think. > > > So we could actually write Gerrit plugin using MergeValidationListener > > that would operate in the following way: > > ...snip... > > Just because something could be written doesn't mean it should - we > are already stretched too thinly over too much code in the CI system, > we do not want to maintain any more of it, certainly not in a core > component like Gerrit. It was just an idea =) Zuul sounds more like a proper solution. > > Then again, this is a prioritization question - it not up to me to decide. > > [1]: https://ovirt-jira.atlassian.net/browse/OVIRT-734 > [2]: https://ovirt-jira.atlassian.net/browse/OVIRT-882 > > -- > Barak Korren > bkor...@redhat.com > RHCE, RHCi, RHV-DevOps Team > https://ifireball.wordpress.com/ > ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] performance improvements and gwt-rpc switch
On Thu, Dec 1, 2016 at 5:35 PM, Moran Goldboim wrote: > > > On Thu, Dec 1, 2016 at 6:08 PM, Sven Kieske wrote: > >> On 01/12/16 16:27, Greg Sheremeta wrote: >> > I'm working on 3.6 now. My expectation is that we can speed it up a bit, >> > but it'll never be as fast as 4.0.6/4.1. Unless we decide to commit >> time to >> > it, in which case we've proven that it can be done, it's just a matter >> of >> > resources. >> >> Sorry for getting a little offtopic, but I'm really confused: >> >> It was stated somewhere that 3.6.7 will be the last release, and there >> has not been an official 3.6. release since then, but otoh there seems >> to be ongoing work on 3.6, see: >> >> https://gerrit.ovirt.org/#/admin/projects/ovirt-engine,branches >> >> there are branches for 3.6.8 and 3.6.9 already. >> >> Will there be future 3.6 releases, or not? >> > > we aren't planning to issue any upstream 3.6 based release > > >> >> Or is this just work for downstream RHEV which will get support >> much longer? >> >> > that's right - it's part of the add value for our downstream/product > customers. > > Of course all changes done for downstream RHEV 3.6 are merged into upstream oVirt 3.6 repositories, so you can always build up-to-date oVirt 3.6 from source codes. > > >> It would be nice if someone could clarify this. >> > > hope it helps. > > Regards, > Moran. > > > Personally I really would like to see additional 3.6.z releases, >> especially with these performance improvements around the webui. >> >> Thanks and sorry for hijacking this thread. >> > >> -- >> Mit freundlichen Grüßen / Regards >> >> Sven Kieske >> >> Systemadministrator >> Mittwald CM Service GmbH & Co. KG >> Königsberger Straße 6 >> 32339 Espelkamp >> T: +495772 293100 >> F: +495772 29 >> https://www.mittwald.de >> Geschäftsführer: Robert Meyer >> St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen >> Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad >> Oeynhausen >> >> >> ___ >> Devel mailing list >> Devel@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > ___ > Devel mailing list > Devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] performance improvements and gwt-rpc switch
On Thu, Dec 1, 2016 at 6:08 PM, Sven Kieske wrote: > On 01/12/16 16:27, Greg Sheremeta wrote: > > I'm working on 3.6 now. My expectation is that we can speed it up a bit, > > but it'll never be as fast as 4.0.6/4.1. Unless we decide to commit time > to > > it, in which case we've proven that it can be done, it's just a matter of > > resources. > > Sorry for getting a little offtopic, but I'm really confused: > > It was stated somewhere that 3.6.7 will be the last release, and there > has not been an official 3.6. release since then, but otoh there seems > to be ongoing work on 3.6, see: > > https://gerrit.ovirt.org/#/admin/projects/ovirt-engine,branches > > there are branches for 3.6.8 and 3.6.9 already. > > Will there be future 3.6 releases, or not? > we aren't planning to issue any upstream 3.6 based release > > Or is this just work for downstream RHEV which will get support > much longer? > > that's right - it's part of the add value for our downstream/product customers. > It would be nice if someone could clarify this. > hope it helps. Regards, Moran. Personally I really would like to see additional 3.6.z releases, > especially with these performance improvements around the webui. > > Thanks and sorry for hijacking this thread. > > -- > Mit freundlichen Grüßen / Regards > > Sven Kieske > > Systemadministrator > Mittwald CM Service GmbH & Co. KG > Königsberger Straße 6 > 32339 Espelkamp > T: +495772 293100 > F: +495772 29 > https://www.mittwald.de > Geschäftsführer: Robert Meyer > St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen > Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen > > > ___ > Devel mailing list > Devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] performance improvements and gwt-rpc switch
On 01/12/16 16:27, Greg Sheremeta wrote: > I'm working on 3.6 now. My expectation is that we can speed it up a bit, > but it'll never be as fast as 4.0.6/4.1. Unless we decide to commit time to > it, in which case we've proven that it can be done, it's just a matter of > resources. Sorry for getting a little offtopic, but I'm really confused: It was stated somewhere that 3.6.7 will be the last release, and there has not been an official 3.6. release since then, but otoh there seems to be ongoing work on 3.6, see: https://gerrit.ovirt.org/#/admin/projects/ovirt-engine,branches there are branches for 3.6.8 and 3.6.9 already. Will there be future 3.6 releases, or not? Or is this just work for downstream RHEV which will get support much longer? It would be nice if someone could clarify this. Personally I really would like to see additional 3.6.z releases, especially with these performance improvements around the webui. Thanks and sorry for hijacking this thread. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +495772 293100 F: +495772 29 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen signature.asc Description: OpenPGP digital signature ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] performance improvements and gwt-rpc switch
I'm working on 3.6 now. My expectation is that we can speed it up a bit, but it'll never be as fast as 4.0.6/4.1. Unless we decide to commit time to it, in which case we've proven that it can be done, it's just a matter of resources. Greg On Thu, Dec 1, 2016 at 4:30 AM, Moran Goldboim wrote: > looks like great work has been done here. > following QE validation and internal customers input assuming feedback is > very positive and regressions are covered , i would like to take this > information to our customers. > would probably worth a post on rhev-tech. what are the exceptions for the > performance boost due to the changes on 3.6 branch? > > > On Wed, Nov 30, 2016 at 7:46 PM, Vojtech Szocs wrote: > >> Thanks Greg for the excellent write-up! >> >> oVirt WebAdmin UI is quite complex, with lots of abstractions (the GWT >> technology being a Java-on-top-of-Web-APIs abstraction itself) as well >> as many features (and lots of dialogs..) so this task wasn't easy, but >> I guess we managed to get UI into a stabilized state now. >> >> Dialogs (non-singleton by design) were a major source of memory leaks, >> so we added UI infra to ensure automatic dialog cleanup. >> >> In 4.2, we'd like to upgrade GWT SDK version to latest stable (2.8.x) >> which will hopefully improve GWT compiler performance and, importantly, >> allow us to use Java 8 language features in our GWT UI code. >> >> There's still potential for improvement, we're tracking that through: >> >> [tracker] oVirt UI performance improvements >> https://bugzilla.redhat.com/1378935 >> >> (some IE-specific improvements are already submitted as part of ^^) >> >> Regards, >> Vojtech >> >> >> - Original Message - >> > From: "Greg Sheremeta" >> > To: "devel" >> > Sent: Monday, November 28, 2016 7:43:27 PM >> > Subject: [ovirt-devel] performance improvements and gwt-rpc switch >> > >> > Hi everyone, >> > >> > For the upcoming oVirt 4.1, the UX team has been focused hard on >> webadmin >> > performance improvements. There have been some reports [1] [2] of UI >> > sluggishness in both 3.6 and 4.0, usually after the browser had been >> open >> > some time, and usually in scale environments. >> > >> > After some research, we determined that the primary cause of this >> > sluggishness was memory leaks. >> > >> > We embarked on several weeks of hunting down memory leak bugs and >> squashing >> > them. Alexander Wels and Vojtech Szocs led this work, and I helped test >> the >> > performance of each patch as they created them. As they created patches >> to >> > squash leaks, performance kept getting better and better. Today we've >> merged >> > the last of our patches [*], and I'm happy to announce that we are now >> > seeing much better UI performance on 4.1-master and 4.0.6. >> > >> > Over the course of several hours with the browser window open, users >> should >> > see no sluggishness at all. >> > >> > [*] This last patch switches oVirt from using de-rpc to gwt-rpc in the >> > frontend. This improves performance, but it also allows us to upgrade >> to GWT >> > 2.8. We'd been previously blocked on that. >> > >> > If you're interested in UI performance testing, continue reading. If >> not, you >> > can stop here :) >> > >> > . >> > >> > To verify our performance improvements, we took some simple measurements >> > using selenium webdriver. The tests were unscientific, but very >> helpful. We >> > ran a webdriver flow through oVirt that clicked some buttons and tabs >> and >> > refreshed some grids. We did it a few hundred or thousand times. The >> tests >> > were run using stubbed hosts (ovirt-vdsmfake) so that only the engine >> and UI >> > were under test. >> > >> > Below are the important takeaways. The x axis is time, and each point >> on a >> > graph is a loop through the same webdriver flow. The (ms) y axes are >> > response times, and memory is in MB. >> > >> > In this graph, we compare oVirt 4.1 with and without our most impactful >> patch >> > applied. As you can see, with the patch applied, response time stays >> flat >> > for 200 loops of my test script over the course of 18 and 43 minutes. >> > Without the patch applied, response time quickly degraded such that 200 >> > loops of my test script took 1 hr 2 minutes vs. 18 minutes with the >> patch >> > applied -- a 66% improvement! >> > >> > In this graph [ignore the spike], we tested oVirt hard for 6 hours 25 >> minutes >> > (2000 loops). As you can see, the response times stay relatively flat >> over 6 >> > hours! This is a great improvement. Do note that the memory is still >> > growing, albeit much more slowly now. You can see towards the end of >> this >> > run, maybe around hour 5, that the deviation starts to go up (the line >> > thickens). Takeaway: maybe refresh your browser after many hours of >> having >> > webadmin open. But, this is a stress test -- I'm betting users won't >> notice >> > this slowdown after even 6 hours of regular webadmin use or idling. >> > >> > >> > Last,
Re: [ovirt-devel] ovirt-guest-agent job not building rpms
On Thu, Dec 1, 2016 at 4:32 PM, Yedidyah Bar David wrote: > On Thu, Dec 1, 2016 at 1:59 PM, Sandro Bonazzola wrote: >> Hi, please have a look at >> >> http://jenkins.ovirt.org/job/ovirt-guest-agent_master_build-artifacts-el7-x86_64/25/ >> >> ovirt-guest-agent job not building rpms > > IIRC that's as planned. Check build-artifacts.sh. We build only on fedora. And IIRC Vinzenz builds it on fedora copr too. Note that usually the guest agent is not really part of oVirt - it's installed in guests, which are not supposed to have oVirt repos. > >> Thanks, >> -- >> Sandro Bonazzola >> Better technology. Faster innovation. Powered by community collaboration. >> See how it works at redhat.com > > > > -- > Didi -- Didi ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] ovirt-guest-agent job not building rpms
On Thu, Dec 1, 2016 at 1:59 PM, Sandro Bonazzola wrote: > Hi, please have a look at > > http://jenkins.ovirt.org/job/ovirt-guest-agent_master_build-artifacts-el7-x86_64/25/ > > ovirt-guest-agent job not building rpms IIRC that's as planned. Check build-artifacts.sh. We build only on fedora. > Thanks, > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com -- Didi ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
[ovirt-devel] ovirt-guest-agent job not building rpms
Hi, please have a look at http://jenkins.ovirt.org/job/ovirt-guest-agent_master_build-artifacts-el7-x86_64/25/ ovirt-guest-agent job not building rpms Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs
On Thu, Dec 01, 2016 at 03:38:36AM -0500, Ramesh Nachimuthu wrote: > > > > > - Original Message - > > From: "Dan Kenigsberg" > > To: "Ramesh Nachimuthu" > > Cc: "Aravinda" , igoih...@redhat.com, "Nir Soffer" > > , "devel" > > , "Piotr Kliczewski" , "Yaniv > > Bronheim" , "Sahina Bose" > > > > Sent: Wednesday, November 30, 2016 4:54:59 PM > > Subject: Re: python path conflicts issues while running python scripts from > > vdsm verbs > > > > On Wed, Nov 30, 2016 at 03:49:02AM -0500, Ramesh Nachimuthu wrote: > > > > > > > > > > > > > > > - Original Message - > > > > From: "Dan Kenigsberg" > > > > To: "Aravinda" , igoih...@redhat.com > > > > Cc: "Ramesh Nachimuthu" , "Nir Soffer" > > > > , "devel" , "Piotr > > > > Kliczewski" , "Yaniv Bronheim" > > > > , "Sahina Bose" > > > > Sent: Wednesday, November 30, 2016 2:02:43 PM > > > > Subject: Re: python path conflicts issues while running python scripts > > > > from vdsm verbs > > > > > > > > > > > > Regardless of the solution you plan to take, let it serve as a reminder > > > > about moving gluster bits out of /usr/share/vdsm/ and into > > > > site-packages. I am willing to pay the price of an ugly > > > > exception-handling in xmlrpc, if this is the cost. > > > > > > Yes. I think this is the right time to revive the patch > > > https://gerrit.ovirt.org/#/c/57530/ and > > > https://gerrit.ovirt.org/#/c/57524/ to move all the gluster code to > > > site-packages. Let me re-post these packages for review. > > > > Please do. > > > > IIRC https://gerrit.ovirt.org/#/c/57530/9 is broken in xmlrpc. > > > > Since xmlrpc is going to be used only by the command line in 4.1, I'd be > > pleased to see the problem solved via a simplistic hack. > > > > > I have updated the patch https://gerrit.ovirt.org/#/c/57530/ for moving > vdsm/gluster into site-packages. I took the patch > https://gerrit.ovirt.org/#/c/57524/ as an workaround to solve the ClientIF > issue. > > Dan: Please let me know if you have any simple fix for clientIF issue. Thanks. I think that the approach of your https://gerrit.ovirt.org/#/c/67660/ is much better. ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] Gerrit parallel patch handling and CI (Or, why did my code fail post-merge)
On 1 December 2016 at 10:44, Eyal Edri wrote: ...snip... >> We could make it a two-step operation where you indicate you want to >> submit, wait for the CI output and then click submit - but why not >> just let the system submit for you? ...snip... > Another option is not to do it from Gerrit, but via a 'developer' job that > will be able > to run "heavy jobs" on demand, basically we should be able to utilize the > same > flow Zuul will run, only instead of basic sanity, run tier1/2 of testing and > not publish RPMs at the end. > > This is more complicated than it sounds ( at least to what we knew about > developer jobs in the past), > So we'll learn more about this as we move forward with the gating project. This is just like the "3 CI stages" I described above, just without placing the button in Gerrit - I still don't think we should have this (Better to aim for CI to be fast and reliable enough). -- Barak Korren bkor...@redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/ ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] performance improvements and gwt-rpc switch
looks like great work has been done here. following QE validation and internal customers input assuming feedback is very positive and regressions are covered , i would like to take this information to our customers. would probably worth a post on rhev-tech. what are the exceptions for the performance boost due to the changes on 3.6 branch? On Wed, Nov 30, 2016 at 7:46 PM, Vojtech Szocs wrote: > Thanks Greg for the excellent write-up! > > oVirt WebAdmin UI is quite complex, with lots of abstractions (the GWT > technology being a Java-on-top-of-Web-APIs abstraction itself) as well > as many features (and lots of dialogs..) so this task wasn't easy, but > I guess we managed to get UI into a stabilized state now. > > Dialogs (non-singleton by design) were a major source of memory leaks, > so we added UI infra to ensure automatic dialog cleanup. > > In 4.2, we'd like to upgrade GWT SDK version to latest stable (2.8.x) > which will hopefully improve GWT compiler performance and, importantly, > allow us to use Java 8 language features in our GWT UI code. > > There's still potential for improvement, we're tracking that through: > > [tracker] oVirt UI performance improvements > https://bugzilla.redhat.com/1378935 > > (some IE-specific improvements are already submitted as part of ^^) > > Regards, > Vojtech > > > - Original Message - > > From: "Greg Sheremeta" > > To: "devel" > > Sent: Monday, November 28, 2016 7:43:27 PM > > Subject: [ovirt-devel] performance improvements and gwt-rpc switch > > > > Hi everyone, > > > > For the upcoming oVirt 4.1, the UX team has been focused hard on webadmin > > performance improvements. There have been some reports [1] [2] of UI > > sluggishness in both 3.6 and 4.0, usually after the browser had been open > > some time, and usually in scale environments. > > > > After some research, we determined that the primary cause of this > > sluggishness was memory leaks. > > > > We embarked on several weeks of hunting down memory leak bugs and > squashing > > them. Alexander Wels and Vojtech Szocs led this work, and I helped test > the > > performance of each patch as they created them. As they created patches > to > > squash leaks, performance kept getting better and better. Today we've > merged > > the last of our patches [*], and I'm happy to announce that we are now > > seeing much better UI performance on 4.1-master and 4.0.6. > > > > Over the course of several hours with the browser window open, users > should > > see no sluggishness at all. > > > > [*] This last patch switches oVirt from using de-rpc to gwt-rpc in the > > frontend. This improves performance, but it also allows us to upgrade to > GWT > > 2.8. We'd been previously blocked on that. > > > > If you're interested in UI performance testing, continue reading. If > not, you > > can stop here :) > > > > . > > > > To verify our performance improvements, we took some simple measurements > > using selenium webdriver. The tests were unscientific, but very helpful. > We > > ran a webdriver flow through oVirt that clicked some buttons and tabs and > > refreshed some grids. We did it a few hundred or thousand times. The > tests > > were run using stubbed hosts (ovirt-vdsmfake) so that only the engine > and UI > > were under test. > > > > Below are the important takeaways. The x axis is time, and each point on > a > > graph is a loop through the same webdriver flow. The (ms) y axes are > > response times, and memory is in MB. > > > > In this graph, we compare oVirt 4.1 with and without our most impactful > patch > > applied. As you can see, with the patch applied, response time stays flat > > for 200 loops of my test script over the course of 18 and 43 minutes. > > Without the patch applied, response time quickly degraded such that 200 > > loops of my test script took 1 hr 2 minutes vs. 18 minutes with the patch > > applied -- a 66% improvement! > > > > In this graph [ignore the spike], we tested oVirt hard for 6 hours 25 > minutes > > (2000 loops). As you can see, the response times stay relatively flat > over 6 > > hours! This is a great improvement. Do note that the memory is still > > growing, albeit much more slowly now. You can see towards the end of this > > run, maybe around hour 5, that the deviation starts to go up (the line > > thickens). Takeaway: maybe refresh your browser after many hours of > having > > webadmin open. But, this is a stress test -- I'm betting users won't > notice > > this slowdown after even 6 hours of regular webadmin use or idling. > > > > > > Last, here is a graph that shows gwt-rpc performing slightly better than > > de-rpc. Memory consumption is about the same -- gwt-rpc is just a faster > rpc > > implementation. > > > > > > Reply with any questions or concerns. Thanks! > > > > Best wishes, > > Greg > > > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1368101 > > [2] https://bugzilla.redhat.com/show_bug.cgi?id=1388462 > > > > -- > > Greg Sheremeta, MBA > > Red Hat, Inc.
Re: [ovirt-devel] Gerrit parallel patch handling and CI (Or, why did my code fail post-merge)
On Thu, Dec 1, 2016 at 10:10 AM, Barak Korren wrote: > On 30 November 2016 at 19:12, Vojtech Szocs wrote: > > > > So before we adopt auto-submit-after-ci-pass (e.g. using Zuul, as Eyal > > mentioned in that thread), we should be able to manually trigger heavy > > CI (check-merged) job from Gerrit web interface, is that correct? > > We could make it a two-step operation where you indicate you want to > submit, wait for the CI output and then click submit - but why not > just let the system submit for you? > > One road I don't think we should go down is to have 3 CI stages: > "light" check_patch, "heavy" check_patch and "pre-merge", this looks > like a design smell to me. > > WRT Zuul: > > Unless you are using a strict policy like ff-only (which so far seems > no to scale, because it is essentially a GIL on merging), Zuul is > essential in order to test the right code in CI (even just for basic > check_patch testing). So we are working to bring it in (See OVIRT-734 > [1] and OVIRT-882 [2]). > > WRT merge gating - Zuul tries to make the process "fire and forget" as > far as maintainers are concerned - if failures are found, it actually > tries out different combinations of patches to see if, from a set of > patches that were asked to be merged, some could be merged even when > others could not. > > > If so, it would be the patch owner's responsibility to submit (merge) > > only after heavy CI (check-merged) pass? > > The question hiding here is wither a maintainer could submit _without_ > passing the "heavy" CI. I'm guessing some maintainers will demand > that, but I'm hoping in the long run this will not be needed and you > will know that once submitted, a patch will be eventually merged > unless it has real issues in it. > Another option is not to do it from Gerrit, but via a 'developer' job that will be able to run "heavy jobs" on demand, basically we should be able to utilize the same flow Zuul will run, only instead of basic sanity, run tier1/2 of testing and not publish RPMs at the end. This is more complicated than it sounds ( at least to what we knew about developer jobs in the past), So we'll learn more about this as we move forward with the gating project. > > > So we could actually write Gerrit plugin using MergeValidationListener > > that would operate in the following way: > > ...snip... > > Just because something could be written doesn't mean it should - we > are already stretched too thinly over too much code in the CI system, > we do not want to maintain any more of it, certainly not in a core > component like Gerrit. > > Then again, this is a prioritization question - it not up to me to decide. > > [1]: https://ovirt-jira.atlassian.net/browse/OVIRT-734 > [2]: https://ovirt-jira.atlassian.net/browse/OVIRT-882 > > -- > Barak Korren > bkor...@redhat.com > RHCE, RHCi, RHV-DevOps Team > https://ifireball.wordpress.com/ > ___ > Devel mailing list > Devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs
- Original Message - > From: "Dan Kenigsberg" > To: "Ramesh Nachimuthu" > Cc: "Aravinda" , igoih...@redhat.com, "Nir Soffer" > , "devel" > , "Piotr Kliczewski" , "Yaniv Bronheim" > , "Sahina Bose" > > Sent: Wednesday, November 30, 2016 4:54:59 PM > Subject: Re: python path conflicts issues while running python scripts from > vdsm verbs > > On Wed, Nov 30, 2016 at 03:49:02AM -0500, Ramesh Nachimuthu wrote: > > > > > > > > > > - Original Message - > > > From: "Dan Kenigsberg" > > > To: "Aravinda" , igoih...@redhat.com > > > Cc: "Ramesh Nachimuthu" , "Nir Soffer" > > > , "devel" , "Piotr > > > Kliczewski" , "Yaniv Bronheim" > > > , "Sahina Bose" > > > Sent: Wednesday, November 30, 2016 2:02:43 PM > > > Subject: Re: python path conflicts issues while running python scripts > > > from vdsm verbs > > > > > > > > > Regardless of the solution you plan to take, let it serve as a reminder > > > about moving gluster bits out of /usr/share/vdsm/ and into > > > site-packages. I am willing to pay the price of an ugly > > > exception-handling in xmlrpc, if this is the cost. > > > > Yes. I think this is the right time to revive the patch > > https://gerrit.ovirt.org/#/c/57530/ and > > https://gerrit.ovirt.org/#/c/57524/ to move all the gluster code to > > site-packages. Let me re-post these packages for review. > > Please do. > > IIRC https://gerrit.ovirt.org/#/c/57530/9 is broken in xmlrpc. > > Since xmlrpc is going to be used only by the command line in 4.1, I'd be > pleased to see the problem solved via a simplistic hack. > I have updated the patch https://gerrit.ovirt.org/#/c/57530/ for moving vdsm/gluster into site-packages. I took the patch https://gerrit.ovirt.org/#/c/57524/ as an workaround to solve the ClientIF issue. Dan: Please let me know if you have any simple fix for clientIF issue. Regards, Ramesh ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] Gerrit parallel patch handling and CI (Or, why did my code fail post-merge)
On 30 November 2016 at 19:12, Vojtech Szocs wrote: > > So before we adopt auto-submit-after-ci-pass (e.g. using Zuul, as Eyal > mentioned in that thread), we should be able to manually trigger heavy > CI (check-merged) job from Gerrit web interface, is that correct? We could make it a two-step operation where you indicate you want to submit, wait for the CI output and then click submit - but why not just let the system submit for you? One road I don't think we should go down is to have 3 CI stages: "light" check_patch, "heavy" check_patch and "pre-merge", this looks like a design smell to me. WRT Zuul: Unless you are using a strict policy like ff-only (which so far seems no to scale, because it is essentially a GIL on merging), Zuul is essential in order to test the right code in CI (even just for basic check_patch testing). So we are working to bring it in (See OVIRT-734 [1] and OVIRT-882 [2]). WRT merge gating - Zuul tries to make the process "fire and forget" as far as maintainers are concerned - if failures are found, it actually tries out different combinations of patches to see if, from a set of patches that were asked to be merged, some could be merged even when others could not. > If so, it would be the patch owner's responsibility to submit (merge) > only after heavy CI (check-merged) pass? The question hiding here is wither a maintainer could submit _without_ passing the "heavy" CI. I'm guessing some maintainers will demand that, but I'm hoping in the long run this will not be needed and you will know that once submitted, a patch will be eventually merged unless it has real issues in it. > So we could actually write Gerrit plugin using MergeValidationListener > that would operate in the following way: > ...snip... Just because something could be written doesn't mean it should - we are already stretched too thinly over too much code in the CI system, we do not want to maintain any more of it, certainly not in a core component like Gerrit. Then again, this is a prioritization question - it not up to me to decide. [1]: https://ovirt-jira.atlassian.net/browse/OVIRT-734 [2]: https://ovirt-jira.atlassian.net/browse/OVIRT-882 -- Barak Korren bkor...@redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/ ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
