This sounds like a good candidate for a "git bisect" operation [1] since we already have a pretty tight window where things changed.
Carl [1] http://git-scm.com/docs/git-bisect On Thu, Sep 3, 2015 at 7:07 AM, Assaf Muller <[email protected]> wrote: > > > On Thu, Sep 3, 2015 at 8:43 AM, Andrey Pavlov <[email protected]> wrote: >> >> Hello, >> >> We have rally job with fake virt driver. And we run it periodically. >> This job runs 200 servers and measures 'show' operations. >> >> On 18.08 it was run well[1]. But on 21.08 it was failed by timeout[2]. >> I tried to understand what happens. >> I tried to check this job with 20 servers only[3]. It passed but I see >> that >> operations with neutron take more time now (list subnets, list network >> interfaces). >> and as result start and show instances take more time also. >> >> Maybe anyone knows what happens? > > > Looking at the merged Neutron patches between the 18th and 21st, there's a > lot of > candidates, including QoS and work around quotas. > > I think the best way to find out would be to run a profiler against Neutron > from the 18th, > and Neutron from the 21st while running the Rally tests, and finding out if > the major > bottlenecks moved. Last time I profiled Neutron I used GeventProfiler: > https://pypi.python.org/pypi/GreenletProfiler > > Ironically I was having issued with the profiler that comes with Eventlet. > >> >> >> >> [1] >> http://logs.openstack.org/13/211613/6/experimental/ec2-api-rally-dsvm-fakevirt/fac263e/ >> [2] >> http://logs.openstack.org/74/213074/7/experimental/ec2-api-rally-dsvm-fakevirt/91d0675/ >> [3] >> http://logs.openstack.org/46/219846/1/experimental/ec2-api-rally-dsvm-fakevirt/dad98f0/ >> >> -- >> Kind regards, >> Andrey Pavlov. >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
