Hey Vish, I'm still seeing the behavior happen when scheduling at least 52 instances nearly simultaneously. I am running with the patch mentioned above. We are running a single instance of the scheduler.
The scheduler placed the instances in the following manner: compute-node07 : 4 compute-node09 : 3 compute-node10 : 3 compute-node11 : 3 compute-node22 : 18 compute-node26 : 13 compute-node36 : 4 compute-node37 : 4 all nodes have 96GB of RAM. nova.conf: compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler scheduler_available_filters=nova.scheduler.filters.standard_filters # which filter class names to use for filtering hosts when not specified in the request. #scheduler_default_filters=AvailabilityZoneFilter,RamFilter,ComputeFilter,CoreFilter,SameHostFilter,DifferentHostFilter,RetryFilter scheduler_default_filters=RamFilter,ComputeFilter,RetryFilter node_availability_zone=nova default_schedule_zone=nova compute_fill_first_cost_fn_weight=-1.0 scheduler_max_attempts=5 Here is a snippet from the nova-scheduler.log 2012-11-15 22:28:16 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:16 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:16 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:16 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:17 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:18 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:18 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:18 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:18 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:18 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:19 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:19 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:19 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:20 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node22... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:21 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:22 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:22 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:22 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:22 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:22 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:23 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:23 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 2012-11-15 22:28:23 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous cast on compute.compute-node26... from (pid=10327) cast /usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py:376 Let me know if you need any more information. I can repeat it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1073956 Title: Scheduler Race Condition at high volume To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1073956/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
