Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Jay Pipes]

On 05/04/2017 04:59 AM, Giuseppe Di Lena wrote:

Hi Chris,


I'm pretty sure a regular user can create a server group and specify the 
anti-affinity filter.


yes, but we want that the user specifies just the Robustness; the way in which 
we assign the instances to the compute nodes should be a black box for the 
regular user(and also for the admin).


Server groups *are* a black box though. You create a server group and 
set the policy of the group to "anti-affinity" and that's it. There's no 
need for the user or admin to know anything else...



Why do you need to track which compute nodes the instances are on?


Because putting the instances in the correct compute nodes is just the first 
step of the algorithm that we are implementing, for the next steps we need to 
know where is each instance.


In a cloud, it shouldn't matter which specific compute node an instance 
is on -- in fact, in clouds, an instance (workload) may not even know 
it's on a hypervisor vs. a baremetal machine vs. a privileged container.


What is important for the user in a cloud to specify is the amount of 
resources the workload will consume (this is the flavor in Nova) and a 
set of characteristics (traits) that the eventual host system should have.


I think it would help if you describe in a little more detail what is 
the eventual outcome you are trying to achieve and what use case that 
outcome serves. Then we can assist you in showing you how to get to that 
outcome.


Best,
-jay


Thank you for the question.

Best regards Giuseppe


Il giorno 03 mag 2017, alle ore 21:01, Chris Friesen 
 ha scritto:

On 05/03/2017 03:08 AM, Giuseppe Di Lena wrote:

Thank you a lot for the help!

I think that the problem can be solved using the anti-affinity filter, but we want 
a regular user can choose an instance and set the property(image, flavour, 
network, etc.) and a parameter Robustness >= 1(that is the number of copies of 
this particular instance).


I'm pretty sure a regular user can create a server group and specify the 
anti-affinity filter.  And a regular user can certainly specify --min-count and 
--max-count to specify the number of copies.


After that, we put every copy of this instance in a different compute, but we 
need to track where we put every copy of the instance (we need to know it for 
the algorithm that we would implement);


Normally only admin-level users are allowed to know which compute nodes a given 
instance is placed on.  Why do you need to track which compute nodes the 
instances are on?

Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Giuseppe Di Lena]

Hi Chris,

> I'm pretty sure a regular user can create a server group and specify the 
> anti-affinity filter. 

yes, but we want that the user specifies just the Robustness; the way in which 
we assign the instances to the compute nodes should be a black box for the 
regular user(and also for the admin).

> Why do you need to track which compute nodes the instances are on?

Because putting the instances in the correct compute nodes is just the first 
step of the algorithm that we are implementing, for the next steps we need to 
know where is each instance.

Thank you for the question.

Best regards Giuseppe 

> Il giorno 03 mag 2017, alle ore 21:01, Chris Friesen 
>  ha scritto:
> 
> On 05/03/2017 03:08 AM, Giuseppe Di Lena wrote:
>> Thank you a lot for the help!
>> 
>> I think that the problem can be solved using the anti-affinity filter, but 
>> we want a regular user can choose an instance and set the property(image, 
>> flavour, network, etc.) and a parameter Robustness >= 1(that is the number 
>> of copies of this particular instance).
> 
> I'm pretty sure a regular user can create a server group and specify the 
> anti-affinity filter.  And a regular user can certainly specify --min-count 
> and --max-count to specify the number of copies.
> 
>> After that, we put every copy of this instance in a different compute, but 
>> we need to track where we put every copy of the instance (we need to know it 
>> for the algorithm that we would implement);
> 
> Normally only admin-level users are allowed to know which compute nodes a 
> given instance is placed on.  Why do you need to track which compute nodes 
> the instances are on?
> 
> Chris
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Chris Friesen]

On 05/03/2017 03:08 AM, Giuseppe Di Lena wrote:

Thank you a lot for the help!

I think that the problem can be solved using the anti-affinity filter, but we want 
a regular user can choose an instance and set the property(image, flavour, 
network, etc.) and a parameter Robustness >= 1(that is the number of copies of 
this particular instance).


I'm pretty sure a regular user can create a server group and specify the 
anti-affinity filter.  And a regular user can certainly specify --min-count and 
--max-count to specify the number of copies.



After that, we put every copy of this instance in a different compute, but we 
need to track where we put every copy of the instance (we need to know it for 
the algorithm that we would implement);


Normally only admin-level users are allowed to know which compute nodes a given 
instance is placed on.  Why do you need to track which compute nodes the 
instances are on?


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Giuseppe Di Lena]

Thank you a lot for the help!

I think that the problem can be solved using the anti-affinity filter, but we 
want a regular user can choose an instance and set the property(image, flavour, 
network, etc.) and a parameter Robustness >= 1(that is the number of copies of 
this particular instance).

After that, we put every copy of this instance in a different compute, but we 
need to track where we put every copy of the instance (we need to know it for 
the algorithm that we would implement);

EXAMPLE. With 4 compute nodes

User create an instance(Server_1) with Robustness = 3.

Server_1_copy1 ==> compute2
Server_1_copy2 ==> compute1
Server_1_copy3 ==> compute4

User create another instance (Host_1) with Robustness = 2

Host_1_copy1 ==> compute2
Host_1_copy2 ==> compute3

and we want to track in a table where are the copies of a particular instance.
For Server_1

copy1 ==> compute2 
copy2 ==> compute1
copy3 ==> compute4

same thing for Host_1
copy1 ==> compute2
copy2 ==> compute3

I hope that now is more clear.
In your opinion what is the best way to do it?

Best Regards Giuseppe

> Il giorno 02 mag 2017, alle ore 22:08, Matt Riedemann  
> ha scritto:
> 
> On 5/2/2017 2:15 PM, Chris Friesen wrote:
>> It sounds to me that the problem could be solved by specifying
>> --min-count and --max-count to specify the number of copies, and using
>> server groups with the anti-affinity filter to ensure that they end up
>> on different compute nodes.
> 
> This is exactly what I was thinking. What else is involved or needed that 
> existing parts of the compute API don't already provide for that use case?
> 
> -- 
> 
> Thanks,
> 
> Matt
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Matt Riedemann]

On 5/2/2017 2:15 PM, Chris Friesen wrote:

It sounds to me that the problem could be solved by specifying
--min-count and --max-count to specify the number of copies, and using
server groups with the anti-affinity filter to ensure that they end up
on different compute nodes.


This is exactly what I was thinking. What else is involved or needed 
that existing parts of the compute API don't already provide for that 
use case?


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Chris Friesen]

On 05/02/2017 10:59 AM, Jay Pipes wrote:

On 05/02/2017 12:33 PM, Giuseppe Di Lena wrote:

Thank you a lot! :-).
Actually, we are also working in parallel to implement the algorithm with
tacker, but for this project we will only use the basic modules in OpenStack
and Heat.

If the scheduler hints are no longer supported, what is the correct way to
give the scheduler personalized input with the instance details?
Best regards Giuseppe


Again, it depends on what problem you are trying to solve with this personalized
input... what would the scheduler do with the length of the service chain as an
input? What are you attempting to solve?


It sounds to me that he's not specifying the length of the service chain, but 
the amount of instance redundancy (i.e. the number of copies of the "same" 
instance).


From the earlier message:

"When we create an instance, we give as input the robustness(integer value >=1) 
 that is the number of copy of an instance. For example Robustness = 3, it will 
create 3 instances and put them in 3 different compute nodes."


It sounds to me that the problem could be solved by specifying --min-count and 
--max-count to specify the number of copies, and using server groups with the 
anti-affinity filter to ensure that they end up on different compute nodes.


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Mooney, Sean K]

> -Original Message-
> From: Jay Pipes [mailto:jaypi...@gmail.com]
> Sent: Tuesday, May 2, 2017 5:59 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [nova-scheduler] Get scheduler hint
> 
> On 05/02/2017 12:33 PM, Giuseppe Di Lena wrote:
> > Thank you a lot! :-).
> > Actually, we are also working in parallel to implement the algorithm
> with tacker, but for this project we will only use the basic modules in
> OpenStack and Heat.
[Mooney, Sean K] if you can use heat you can use the server anti affinity filter
To create a server group per port-pair group and ensure that two instance of 
the same
port-pair-group do not reside on the same server. You could also use 
heat's/senlin's
Scaling groups to define the insantce count of each SF in the chain. 
There was a presentation on this in Barcelona 
https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/15037/on-building-an-auto-healing-resource-cluster-using-senlin
https://www.youtube.com/watch?v=bmdU_m6vRZc
But as jay says below it depends on what you are trying to solve.
If you are trying to model ha constraints for a service chain the above may
Help however it is likely outside the scope of nova/placement api to support 
this directly.

> >
> > If the scheduler hints are no longer supported, what is the correct
> way to give the scheduler personalized input with the instance details?
> > Best regards Giuseppe
> 
> Again, it depends on what problem you are trying to solve with this
> personalized input... what would the scheduler do with the length of
> the service chain as an input? What are you attempting to solve?
> 
> Best,
> -jay
> 
> ___
> ___
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Jay Pipes]

On 05/02/2017 12:33 PM, Giuseppe Di Lena wrote:

Thank you a lot! :-).
Actually, we are also working in parallel to implement the algorithm with 
tacker, but for this project we will only use the basic modules in OpenStack 
and Heat.

If the scheduler hints are no longer supported, what is the correct way to give 
the scheduler personalized input with the instance details?
Best regards Giuseppe


Again, it depends on what problem you are trying to solve with this 
personalized input... what would the scheduler do with the length of the 
service chain as an input? What are you attempting to solve?


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Giuseppe Di Lena]

Thank you a lot! :-).
Actually, we are also working in parallel to implement the algorithm with 
tacker, but for this project we will only use the basic modules in OpenStack 
and Heat.

If the scheduler hints are no longer supported, what is the correct way to give 
the scheduler personalized input with the instance details? 
Best regards Giuseppe

> Il giorno 02 mag 2017, alle ore 16:47, Jay Pipes  ha 
> scritto:
> 
> On 05/02/2017 09:58 AM, Giuseppe Di Lena wrote:
>> Thank you for your answer :-)
>> 
>> I'm pretty new in OpenStack and open source in general(I started two
>> months ago, and I don't have any experience in developing big project).
>> What we are trying to do is to implement our algorithm for Robust NFV
>> chain; to do it we need to create multiple copies of the same instance
>> and put them in different compute nodes.
>> When we create an instance, we give as input the robustness(integer
>> value >=1)  that is the number of copy of an instance.
>> For example Robustness = 3, it will create 3 instances and put them in 3
>> different compute nodes.
> 
> OK, got it. What you want to do is actually not in Nova but rather in the 
> Neutron SFC and Tacker projects:
> 
> https://github.com/openstack/networking-sfc
> https://github.com/openstack/tacker
> 
> Go ahead and read up on those two projects and get a feel for how they are 
> structured. You can head over to Freenode IRC on the #openstack-neutron 
> channel to chat about Neutron service function chaining. The #tacker channel 
> has folks interested in NFV orchestration and should be able to help you out.
> 
> All the best,
> -jay
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Jay Pipes]

On 05/02/2017 09:58 AM, Giuseppe Di Lena wrote:

Thank you for your answer :-)

I'm pretty new in OpenStack and open source in general(I started two
months ago, and I don't have any experience in developing big project).
What we are trying to do is to implement our algorithm for Robust NFV
chain; to do it we need to create multiple copies of the same instance
and put them in different compute nodes.
When we create an instance, we give as input the robustness(integer
value >=1)  that is the number of copy of an instance.
For example Robustness = 3, it will create 3 instances and put them in 3
different compute nodes.


OK, got it. What you want to do is actually not in Nova but rather in 
the Neutron SFC and Tacker projects:


https://github.com/openstack/networking-sfc
https://github.com/openstack/tacker

Go ahead and read up on those two projects and get a feel for how they 
are structured. You can head over to Freenode IRC on the 
#openstack-neutron channel to chat about Neutron service function 
chaining. The #tacker channel has folks interested in NFV orchestration 
and should be able to help you out.


All the best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Giuseppe Di Lena]

Thank you for your answer :-)

I'm pretty new in OpenStack and open source in general(I started two months 
ago, and I don't have any experience in developing big project).
What we are trying to do is to implement our algorithm for Robust NFV chain; to 
do it we need to create multiple copies of the same instance and put them in 
different compute nodes.
When we create an instance, we give as input the robustness(integer value >=1)  
that is the number of copy of an instance.
For example Robustness = 3, it will create 3 instances and put them in 3 
different compute nodes.

Best regards


> Il giorno 02 mag 2017, alle ore 14:25, Jay Pipes  ha 
> scritto:
> 
> Scheduler hints are neither portable nor going to be supported in Nova 
> long-term, so describe to us what problem you're trying to solve and maybe we 
> can help guide you in what you need to change in the scheduler.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova-scheduler] Get scheduler hint

[Jay Pipes]

On 05/02/2017 05:14 AM, Giuseppe Di Lena wrote:

Hello all,
I’m modifying  nova-scheduler, implementing my own scheduler;
there is a way to get all the list of the scheduler hints(for example when I 
lunch a new Instance, add a custom Hint MY_HINT, with value 100)?

I tried with

def select_destination(self, context, spec_obj):
…..
spec_obj.get_scheduler_hint(‘MY_HINT’) # but it return None
…..

thank you


Scheduler hints are neither portable nor going to be supported in Nova 
long-term, so describe to us what problem you're trying to solve and 
maybe we can help guide you in what you need to change in the scheduler.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova-scheduler] Get scheduler hint

[Giuseppe Di Lena]

Hello all,
I’m modifying  nova-scheduler, implementing my own scheduler;
there is a way to get all the list of the scheduler hints(for example when I 
lunch a new Instance, add a custom Hint MY_HINT, with value 100)?

I tried with 
 
def select_destination(self, context, spec_obj):
…..
spec_obj.get_scheduler_hint(‘MY_HINT’) # but it return None
…..

thank you 

Giuseppe Di Lena
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev