Thanks: TIL. The filter invocation per host is the bit I was forgetting.
I'm assuming that the facts about the hosts don't change several times a second so if you held the facts in RAM and then asserted the rules against those facts allowing for age-out/invalidation based on incoming updates then the whole system will run faster. I remember a thread on using dogpile/memoization for this kind of thing. # Shawn Hartsock ----- Original Message ----- > From: "Yunhong Jiang" <yunhong.ji...@intel.com> > To: "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > Sent: Friday, November 1, 2013 1:42:03 PM > Subject: Re: [openstack-dev] [nova][scheduler]The database access in > the scheduler filters > > Shawn, yes, there is 56 VM access every second, and for each VM access, the > scheduler will invoke filter for each host, that means, for each VM access, > the filter function will be invoked 10k times. > So 56 * 10k = 560k, yes, half of 1M, but still big number. > > --jyh > > > -----Original Message----- > > From: Shawn Hartsock [mailto:hartso...@vmware.com] > > Sent: Friday, November 01, 2013 8:20 AM > > To: OpenStack Development Mailing List (not for usage questions) > > Subject: Re: [openstack-dev] [nova][scheduler]The database access in the > > scheduler filters > > > > > > > > ----- Original Message ----- > > > From: "Yunhong Jiang" <yunhong.ji...@intel.com> > > > To: openstack-dev@lists.openstack.org > > > Sent: Thursday, October 31, 2013 6:39:29 PM > > > Subject: [openstack-dev] [nova][scheduler]The database access in the > > scheduler filters > > > > > > I noticed several filters (AggregateMultiTenancyIsoaltion, ram_filter, > > > type_filter, AggregateInstanceExtraSpecsFilter) have DB access in the > > > host_passes(). Some will even access for each invocation. > > > > > > Just curios if this is considered a performance issue? With a 10k nodes, > > 60 > > > VM per node, and 3 hours VM life cycle cloud, it will have more than 1 > > > million DB access per second. Not a small number IMHO. > > > > > > Thanks > > > --jyh > > > > > > _______________________________________________ > > > OpenStack-dev mailing list > > > OpenStack-dev@lists.openstack.org > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > Sorry if I'm dumb, but please try to explain things to me. I don't think I > > follow... > > > > 10k nodes, 60 VM per node... is 600k VM in the whole cloud. A 3 hour life > > cycle for a VM means every hour 1/3 the nodes turn over so 200k VM > > are created/deleted per hour ... divide by 60 for ... 3,333.333 per minute > > or ... divide by 60 for ... 55.5 VM creations/deletions per second ... > > > > ... did I do that math right? So where's the million DB accesses per second > > come from? Are the rules fired for every VM on every access so that 600k > > VM + 1 new VM means the rules fire 600k + 1 times? What? Sorry... really > > confused. > > > > # Shawn Hartsock > > > > _______________________________________________ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev