On Wed, Feb 25, 2015 at 11:48 PM, Deepak Shetty <dpkshe...@gmail.com> wrote:
> > > On Wed, Feb 25, 2015 at 8:42 PM, Deepak Shetty <dpkshe...@gmail.com> > wrote: > >> >> >> On Wed, Feb 25, 2015 at 6:34 PM, Jeremy Stanley <fu...@yuggoth.org> >> wrote: >> >>> On 2015-02-25 17:02:34 +0530 (+0530), Deepak Shetty wrote: >>> [...] >>> > Run 2) We removed glusterfs backend, so Cinder was configured with >>> > the default storage backend i.e. LVM. We re-created the OOM here >>> > too >>> > >>> > So that proves that glusterfs doesn't cause it, as its happening >>> > without glusterfs too. >>> >>> Well, if you re-ran the job on the same VM then the second result is >>> potentially contaminated. Luckily this hypothesis can be confirmed >>> by running the second test on a fresh VM in Rackspace. >>> >> >> Maybe true, but we did the same on hpcloud provider VM too and both time >> it ran successfully with glusterfs as the cinder backend. Also before >> starting >> the 2nd run, we did unstack and saw that free memory did go back to 5G+ >> and then re-invoked your script, I believe the contamination could result >> in some >> additional testcase failures (which we did see) but shouldn't be related >> to >> whether system can OOM or not, since thats a runtime thing. >> >> I see that the VM is up again. We will execute the 2nd run afresh now and >> update >> here. >> > > Ran tempest with configured with default backend i.e. LVM and was able to > recreate > the OOM issue, so running tempest without gluster against a fresh VM > reliably > recreates the OOM issue, snip below from syslog. > > Feb 25 16:58:37 devstack-centos7-rax-dfw-979654 kernel: glance-api invoked > oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 > > Had a discussion with clarkb on IRC and given that F20 is discontinued, > F21 has issues with tempest (under debug by ianw) > and centos7 also has issues on rax (as evident from this thread), the only > option left is to go with ubuntu based CI job, which > BharatK is working on now. > Quick Update: Cinder-GlusterFS CI job on ubuntu was added ( https://review.openstack.org/159217) We ran it 3 times against our stackforge repo patch @ https://review.openstack.org/159711 and it works fine (2 testcase failures, which are expected and we're working towards fixing them) For the logs of the 3 experimental runs, look @ http://logs.openstack.org/11/159711/1/experimental/gate-tempest-dsvm-full-glusterfs/ Of the 3 jobs, 1 was schedued on rax and 2 on hpcloud, so its working nicely across the different cloud providers. thanx, deepak
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev