Re: [Openstack] Which nova scheduler for different hardware sizes?
I know I'm reviving an older thread here but we implemented our own simple
"hardware aware" scheduler for Cactus. Maybe it will help you?
https://github.com/canarie/dair/blob/master/OpenStack/misc/hardware.py
Everett
On Thu, Nov 10, 2011 at 11:35 AM, Christian Wittwer wrote:
> Thanks a lot for your answers. Unfortunately I can't follow the trunk
> and I have to use the Diablo release. Is it possible to backport that
> new scheduler to Diablo?
> Anyway I gave the least cost scheduler a try, it loads but never
> schedules a vm correctly.
>
> --compute_scheduler_driver=nova.scheduler.least_cost.LeastCostScheduler
>
> But then log fires that message when I try to launch an instance
>
> 2011-11-10 19:31:15,588 DEBUG nova.scheduler.least_cost [-] Weighted
> Costs => [] from (pid=1697) weigh_hosts
> /usr/lib/pymodules/python2.7/nova/scheduler/least_cost.py:170
> 2011-11-10 19:31:15,591 ERROR nova.rpc [-] Exception during message
> handling
> (nova.rpc): TRACE: Traceback (most recent call last):
> (nova.rpc): TRACE: File
> "/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 620, in
> _process_data
> (nova.rpc): TRACE: rval = node_func(context=ctxt, **node_args)
> (nova.rpc): TRACE: File
> "/usr/lib/pymodules/python2.7/nova/scheduler/manager.py", line 103, in
> _schedule
> (nova.rpc): TRACE: host = real_meth(*args, **kwargs)
> (nova.rpc): TRACE: File
> "/usr/lib/pymodules/python2.7/nova/scheduler/abstract_scheduler.py",
> line 231, in schedule_run_instance
> (nova.rpc): TRACE: raise driver.NoValidHost(_('No hosts were
> available'))
> (nova.rpc): TRACE: NoValidHost: No hosts were available
> (nova.rpc): TRACE:
>
> My compute hosts are up and running, and there are no other instances
> running.
>
> root@unic-dev-os-controller:~# nova-manage service list
> Binary Host Zone
> Status State Updated_At
> nova-compute unic-dev-os-controller nova
> enabled:-) 2011-11-10 18:35:06
> nova-scheduler unic-dev-os-controller nova
> enabled:-) 2011-11-10 18:35:06
> nova-network unic-dev-os-controller nova
> enabled:-) 2011-11-10 18:35:06
> nova-compute unic-dev-os-compute1 nova
> enabled:-) 2011-11-10 18:35:06
>
> Any ideas why the scheduler does not find a valid host?
>
> Christian
>
> 2011/11/1 Lorin Hochstein :
> > Christian:
> >
> > Sandy's branch just landed in the repository. You should be able to use
> the distributed scheduler with the least cost functionality by specifying
> the following flag in nova.conf for the nova-scheduler service:
> >
> >
> --compute_scheduler_driver=nova.scheduler.distributed_scheduler.DistributedScheduler
> >
> > By default, this uses the
> nova.scheduler.least_cost.compute_fill_first_cost_fn weighting function.
> >
> > Note, however, that this function will favor scheduling instances to
> nodes that have the smallest amount of RAM available that can still fit the
> instance. If you're looking for the opposite effect (deploy to the node
> that has the most amount of RAM free), then you'll have to write your own
> cost function. One way would be to add the following method to
> least_cost.py:
> >
> >
> > def compute_least_loaded_cost_fn(host_info):
> >return -compute_fill_first_cost_fn(host_info)
> >
> >
> > Then add the following flag to your nova.conf
> >
> >
> --least_cost_functions=nova.scheduler.least_cost.compute_least_loaded_cost_fn
> >
> >
> > Lorin
> > --
> > Lorin Hochstein, Computer Scientist
> > USC Information Sciences Institute
> > 703.812.3710
> > http://www.east.isi.edu/~lorin
> >
> >
> >
> >
> > On Nov 1, 2011, at 11:37 AM, Sandy Walsh wrote:
> >
> >> I'm hoping to land this branch asap.
> >> https://review.openstack.org/#change,1192
> >>
> >> It replaces all the "kind of alike" schedulers with a single
> DistributedScheduler.
> >>
> >> -S
> >>
> >>
> >> From:
> >> openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net[openstack-bounces+sandy.walsh=
> rackspace@lists.launchpad.net] on behalf of Christian Wittwer [
> wittwe...@gmail.com]
> >> Sent: Tuesday, November 01, 2011 5:38 AM
> >> To: Lorin Hochstein
> >> Cc: openstack@lists.launchpad.net
> >> Subject: Re: [Openstack] Which nova scheduler for different hardware
> sizes?
> >>
> >>
Re: [Openstack] Which nova scheduler for different hardware sizes?
Thanks a lot for your answers. Unfortunately I can't follow the trunk
and I have to use the Diablo release. Is it possible to backport that
new scheduler to Diablo?
Anyway I gave the least cost scheduler a try, it loads but never
schedules a vm correctly.
--compute_scheduler_driver=nova.scheduler.least_cost.LeastCostScheduler
But then log fires that message when I try to launch an instance
2011-11-10 19:31:15,588 DEBUG nova.scheduler.least_cost [-] Weighted
Costs => [] from (pid=1697) weigh_hosts
/usr/lib/pymodules/python2.7/nova/scheduler/least_cost.py:170
2011-11-10 19:31:15,591 ERROR nova.rpc [-] Exception during message handling
(nova.rpc): TRACE: Traceback (most recent call last):
(nova.rpc): TRACE: File
"/usr/lib/pymodules/python2.7/nova/rpc/impl_kombu.py", line 620, in
_process_data
(nova.rpc): TRACE: rval = node_func(context=ctxt, **node_args)
(nova.rpc): TRACE: File
"/usr/lib/pymodules/python2.7/nova/scheduler/manager.py", line 103, in
_schedule
(nova.rpc): TRACE: host = real_meth(*args, **kwargs)
(nova.rpc): TRACE: File
"/usr/lib/pymodules/python2.7/nova/scheduler/abstract_scheduler.py",
line 231, in schedule_run_instance
(nova.rpc): TRACE: raise driver.NoValidHost(_('No hosts were available'))
(nova.rpc): TRACE: NoValidHost: No hosts were available
(nova.rpc): TRACE:
My compute hosts are up and running, and there are no other instances running.
root@unic-dev-os-controller:~# nova-manage service list
Binary Host Zone
Status State Updated_At
nova-compute unic-dev-os-controller nova
enabled:-) 2011-11-10 18:35:06
nova-scheduler unic-dev-os-controller nova
enabled:-) 2011-11-10 18:35:06
nova-network unic-dev-os-controller nova
enabled:-) 2011-11-10 18:35:06
nova-compute unic-dev-os-compute1 nova
enabled:-) 2011-11-10 18:35:06
Any ideas why the scheduler does not find a valid host?
Christian
2011/11/1 Lorin Hochstein :
> Christian:
>
> Sandy's branch just landed in the repository. You should be able to use the
> distributed scheduler with the least cost functionality by specifying the
> following flag in nova.conf for the nova-scheduler service:
>
> --compute_scheduler_driver=nova.scheduler.distributed_scheduler.DistributedScheduler
>
> By default, this uses the
> nova.scheduler.least_cost.compute_fill_first_cost_fn weighting function.
>
> Note, however, that this function will favor scheduling instances to nodes
> that have the smallest amount of RAM available that can still fit the
> instance. If you're looking for the opposite effect (deploy to the node that
> has the most amount of RAM free), then you'll have to write your own cost
> function. One way would be to add the following method to least_cost.py:
>
>
> def compute_least_loaded_cost_fn(host_info):
> return -compute_fill_first_cost_fn(host_info)
>
>
> Then add the following flag to your nova.conf
>
> --least_cost_functions=nova.scheduler.least_cost.compute_least_loaded_cost_fn
>
>
> Lorin
> --
> Lorin Hochstein, Computer Scientist
> USC Information Sciences Institute
> 703.812.3710
> http://www.east.isi.edu/~lorin
>
>
>
>
> On Nov 1, 2011, at 11:37 AM, Sandy Walsh wrote:
>
>> I'm hoping to land this branch asap.
>> https://review.openstack.org/#change,1192
>>
>> It replaces all the "kind of alike" schedulers with a single
>> DistributedScheduler.
>>
>> -S
>>
>>
>> From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net
>> [openstack-bounces+sandy.walsh=rackspace....@lists.launchpad.net] on behalf
>> of Christian Wittwer [wittwe...@gmail.com]
>> Sent: Tuesday, November 01, 2011 5:38 AM
>> To: Lorin Hochstein
>> Cc: openstack@lists.launchpad.net
>> Subject: Re: [Openstack] Which nova scheduler for different hardware sizes?
>>
>> Lorin,
>> Thanks for your reply. Well the least cost scheduler with these cost
>> functions looks interesting.
>> Unfortunately there is not much documenation about it. Can somebody
>> give me an example how to switch to that scheduler using the memory
>> cost function which already exist?
>>
>> Cheers,
>> Christian
>>
>> 2011/10/24 Lorin Hochstein :
>>> Christian:
>>> You could use the least cost scheduler, but I think you'd have to write your
>>> own cost function to take into account the different number of cores.
>>> Looking at the source, the only cost function it comes with only takes into
>>> account the amount of memory that's free, not loading in terms of total
Re: [Openstack] Which nova scheduler for different hardware sizes?
Christian: Sandy's branch just landed in the repository. You should be able to use the distributed scheduler with the least cost functionality by specifying the following flag in nova.conf for the nova-scheduler service: --compute_scheduler_driver=nova.scheduler.distributed_scheduler.DistributedScheduler By default, this uses the nova.scheduler.least_cost.compute_fill_first_cost_fn weighting function. Note, however, that this function will favor scheduling instances to nodes that have the smallest amount of RAM available that can still fit the instance. If you're looking for the opposite effect (deploy to the node that has the most amount of RAM free), then you'll have to write your own cost function. One way would be to add the following method to least_cost.py: def compute_least_loaded_cost_fn(host_info): return -compute_fill_first_cost_fn(host_info) Then add the following flag to your nova.conf --least_cost_functions=nova.scheduler.least_cost.compute_least_loaded_cost_fn Lorin -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin On Nov 1, 2011, at 11:37 AM, Sandy Walsh wrote: > I'm hoping to land this branch asap. > https://review.openstack.org/#change,1192 > > It replaces all the "kind of alike" schedulers with a single > DistributedScheduler. > > -S > > > From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net > [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf > of Christian Wittwer [wittwe...@gmail.com] > Sent: Tuesday, November 01, 2011 5:38 AM > To: Lorin Hochstein > Cc: openstack@lists.launchpad.net > Subject: Re: [Openstack] Which nova scheduler for different hardware sizes? > > Lorin, > Thanks for your reply. Well the least cost scheduler with these cost > functions looks interesting. > Unfortunately there is not much documenation about it. Can somebody > give me an example how to switch to that scheduler using the memory > cost function which already exist? > > Cheers, > Christian > > 2011/10/24 Lorin Hochstein : >> Christian: >> You could use the least cost scheduler, but I think you'd have to write your >> own cost function to take into account the different number of cores. >> Looking at the source, the only cost function it comes with only takes into >> account the amount of memory that's free, not loading in terms of total >> physical cores and allocated virtual cores. (We use a custom scheduler at >> our site, so I don't have any firsthand experience with the least-cost >> scheduler). >> Lorin >> -- >> Lorin Hochstein, Computer Scientist >> USC Information Sciences Institute >> 703.812.3710 >> http://www.east.isi.edu/~lorin >> >> >> >> On Oct 22, 2011, at 3:17 AM, Christian Wittwer wrote: >> >> I'm planning to build a openstack nova installation with older >> hardware. These servers obviously doesn't have the same hardware >> configuration like memory and cores. >> It ranges from 2 core and 4GB memory to 16 core and 64GB memory. I >> know that there are different scheduler, but I'm not sure which one to >> choose. >> The simple scheduler tries to find the least used host, but the amount >> of used cores per host (max_cores) is a constant, which doesn't work >> for me. >> Maybe the least cost scheduler would be the right one? But I'm not >> sure, because I did not find any documenation about how to use it. >> >> Cheers, >> Christian >> >> ___ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp >> >> > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Which nova scheduler for different hardware sizes?
I'm hoping to land this branch asap. https://review.openstack.org/#change,1192 It replaces all the "kind of alike" schedulers with a single DistributedScheduler. -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Christian Wittwer [wittwe...@gmail.com] Sent: Tuesday, November 01, 2011 5:38 AM To: Lorin Hochstein Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Which nova scheduler for different hardware sizes? Lorin, Thanks for your reply. Well the least cost scheduler with these cost functions looks interesting. Unfortunately there is not much documenation about it. Can somebody give me an example how to switch to that scheduler using the memory cost function which already exist? Cheers, Christian 2011/10/24 Lorin Hochstein : > Christian: > You could use the least cost scheduler, but I think you'd have to write your > own cost function to take into account the different number of cores. > Looking at the source, the only cost function it comes with only takes into > account the amount of memory that's free, not loading in terms of total > physical cores and allocated virtual cores. (We use a custom scheduler at > our site, so I don't have any firsthand experience with the least-cost > scheduler). > Lorin > -- > Lorin Hochstein, Computer Scientist > USC Information Sciences Institute > 703.812.3710 > http://www.east.isi.edu/~lorin > > > > On Oct 22, 2011, at 3:17 AM, Christian Wittwer wrote: > > I'm planning to build a openstack nova installation with older > hardware. These servers obviously doesn't have the same hardware > configuration like memory and cores. > It ranges from 2 core and 4GB memory to 16 core and 64GB memory. I > know that there are different scheduler, but I'm not sure which one to > choose. > The simple scheduler tries to find the least used host, but the amount > of used cores per host (max_cores) is a constant, which doesn't work > for me. > Maybe the least cost scheduler would be the right one? But I'm not > sure, because I did not find any documenation about how to use it. > > Cheers, > Christian > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Which nova scheduler for different hardware sizes?
Lorin, Thanks for your reply. Well the least cost scheduler with these cost functions looks interesting. Unfortunately there is not much documenation about it. Can somebody give me an example how to switch to that scheduler using the memory cost function which already exist? Cheers, Christian 2011/10/24 Lorin Hochstein : > Christian: > You could use the least cost scheduler, but I think you'd have to write your > own cost function to take into account the different number of cores. > Looking at the source, the only cost function it comes with only takes into > account the amount of memory that's free, not loading in terms of total > physical cores and allocated virtual cores. (We use a custom scheduler at > our site, so I don't have any firsthand experience with the least-cost > scheduler). > Lorin > -- > Lorin Hochstein, Computer Scientist > USC Information Sciences Institute > 703.812.3710 > http://www.east.isi.edu/~lorin > > > > On Oct 22, 2011, at 3:17 AM, Christian Wittwer wrote: > > I'm planning to build a openstack nova installation with older > hardware. These servers obviously doesn't have the same hardware > configuration like memory and cores. > It ranges from 2 core and 4GB memory to 16 core and 64GB memory. I > know that there are different scheduler, but I'm not sure which one to > choose. > The simple scheduler tries to find the least used host, but the amount > of used cores per host (max_cores) is a constant, which doesn't work > for me. > Maybe the least cost scheduler would be the right one? But I'm not > sure, because I did not find any documenation about how to use it. > > Cheers, > Christian > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Which nova scheduler for different hardware sizes?
Christian: You could use the least cost scheduler, but I think you'd have to write your own cost function to take into account the different number of cores. Looking at the source, the only cost function it comes with only takes into account the amount of memory that's free, not loading in terms of total physical cores and allocated virtual cores. (We use a custom scheduler at our site, so I don't have any firsthand experience with the least-cost scheduler). Lorin -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin On Oct 22, 2011, at 3:17 AM, Christian Wittwer wrote: > I'm planning to build a openstack nova installation with older > hardware. These servers obviously doesn't have the same hardware > configuration like memory and cores. > It ranges from 2 core and 4GB memory to 16 core and 64GB memory. I > know that there are different scheduler, but I'm not sure which one to > choose. > The simple scheduler tries to find the least used host, but the amount > of used cores per host (max_cores) is a constant, which doesn't work > for me. > Maybe the least cost scheduler would be the right one? But I'm not > sure, because I did not find any documenation about how to use it. > > Cheers, > Christian > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
