Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Ihar Hrachyshka]

Paul Carver  wrote:


On 11/12/2015 3:50 PM, Ihar Hrachyshka wrote:

All I am saying is that IF we merge some classifier API into neutron
core and start using it for core, non-experimental, features, we cannot
later move to some newer version of this API [that you will iterate on]
without leaving a huge pile of compatibility code that would not exist
in the first place if only we thought about proper API in advance. If
that’s what you envision, fine; but I believe it will make adoption of
the ‘evolving’ API a lot slower than it could otherwise be.


I don't think I disagree at all. But we don't have a classifier API in  
neutron core (unless we consider security groups to be it) and I don't  
think anyone is saying that the classifier in networking-sfc should be  
merged straight into core as-is. In fact I think we're saying exactly the  
opposite, that *a* classifier will sit in networking-sfc, outside of core  
neutron, until *some* classifier is merged into core neutron.




That’s why I raised service groups spec in this thread: it seems it’s  
planned to be added into core, with all compatibility guarantees; and was  
planned for adoption for the new fwaas API (as per summit sessions). At  
least I haven’t found anything in their spec that would suggest it’s  
experimental.


It may mean that at the moment when you arrive to some classifier API that  
you can claim the best we can get, there can be a rival classifier API in  
the core tree already.


The point of networking-sfc isn't the classifier. A classifier is simply  
a prerequisite. So by all means let's work on defining and merging into  
core neutron a classifier that we can consider non-experimental and  
stable for all features to share and depend on, but we don't want SFC to  
be non-functional while we wait for that to happen. We can call the  
networking-sfc classifier experimental a slap a warning on that it'll be  
replaced with the core neutron classifier once such a thing has been  
implemented.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Ihar Hrachyshka]

German  wrote:


Ihar,

The Service Group API will be added to FWaaS only and remain experimental  
in M. If the community finds it useful for other areas it can be added to  
Neutron core once it is matures. I think incubating it inside FWaaS will  
give us the velocity to iterate quickly and once it matures a strong API  
for adoption in the wider Neutron.


On the other hand if classifiers mature more quickly/provide the same  
functionality more elegantly we can quickly pivot to that. Since we are  
trying to address an immediate need I would like to experiment with  
Service Groups inside FWaaS first and then use that to shape the  
classifier API discussion. Also, as Sean suggested, we might use the  
classifier data models under the hood — so despite being a different API  
at first we might arrive at the same shared implementation…


Thanks for clarification! If we are aware of compatibility issues and use  
API in experimental features only, I think that’s indeed a better approach.  
From the first sight at the service group spec, I haven’t seen any marks  
of its experimental nature, hence was asking.


Thanks again,
Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Eichberger, German]

Ihar,

The Service Group API will be added to FWaaS only and remain experimental in M. 
If the community finds it useful for other areas it can be added to Neutron 
core once it is matures. I think incubating it inside FWaaS will give us the 
velocity to iterate quickly and once it matures a strong API for adoption in 
the wider Neutron.

On the other hand if classifiers mature more quickly/provide the same 
functionality more elegantly we can quickly pivot to that. Since we are trying 
to address an immediate need I would like to experiment with Service Groups 
inside FWaaS first and then use that to shape the classifier API discussion. 
Also, as Sean suggested, we might use the classifier data models under the hood 
— so despite being a different API at first we might arrive at the same shared 
implementation…

Thanks,
German



On 11/13/15, 1:12 AM, "Ihar Hrachyshka"  wrote:

>Paul Carver  wrote:
>
>> On 11/12/2015 3:50 PM, Ihar Hrachyshka wrote:
>>> All I am saying is that IF we merge some classifier API into neutron
>>> core and start using it for core, non-experimental, features, we cannot
>>> later move to some newer version of this API [that you will iterate on]
>>> without leaving a huge pile of compatibility code that would not exist
>>> in the first place if only we thought about proper API in advance. If
>>> that’s what you envision, fine; but I believe it will make adoption of
>>> the ‘evolving’ API a lot slower than it could otherwise be.
>>
>> I don't think I disagree at all. But we don't have a classifier API in  
>> neutron core (unless we consider security groups to be it) and I don't  
>> think anyone is saying that the classifier in networking-sfc should be  
>> merged straight into core as-is. In fact I think we're saying exactly the  
>> opposite, that *a* classifier will sit in networking-sfc, outside of core  
>> neutron, until *some* classifier is merged into core neutron.
>>
>
>That’s why I raised service groups spec in this thread: it seems it’s  
>planned to be added into core, with all compatibility guarantees; and was  
>planned for adoption for the new fwaas API (as per summit sessions). At  
>least I haven’t found anything in their spec that would suggest it’s  
>experimental.
>
>It may mean that at the moment when you arrive to some classifier API that  
>you can claim the best we can get, there can be a rival classifier API in  
>the core tree already.
>
>> The point of networking-sfc isn't the classifier. A classifier is simply  
>> a prerequisite. So by all means let's work on defining and merging into  
>> core neutron a classifier that we can consider non-experimental and  
>> stable for all features to share and depend on, but we don't want SFC to  
>> be non-functional while we wait for that to happen. We can call the  
>> networking-sfc classifier experimental a slap a warning on that it'll be  
>> replaced with the core neutron classifier once such a thing has been  
>> implemented.
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Sean M. Collins]

On Fri, Nov 13, 2015 at 01:53:49AM EST, Paul Carver wrote:
> On 11/3/2015 1:03 PM, Sean M. Collins wrote:
> >Anyway, the code is currently up on GitHub - I just threw it on there
> >because I wanted to scratch my hacking itch quickly.
> >
> >https://github.com/sc68cal/neutron-classifier
> >
> 
> Sean,
> 
> How much is needed to turn your models into something runnable to the extent
> of populating a database? I'm not really all that proficient with SQL
> Alchemy or SQL in general so I can't really visualize what the polymorphism
> statements in your model actually create.

Yep - the doc links are still not well organized, but when I was writing
the code I had the same issue, so I stuck a lot of debugging stuff in so
I could see the CREATE statements.

https://github.com/sc68cal/neutron-classifier/blob/master/doc/source/usage.rst

It's a little basic right now, it's just one classifier added to a
chain - I'm going to take that and build it out more to show the full
classifier chain that would be created in the DB for something like a
security group or a firewall rule, so you see all the classifiers being
created. Bear with me :)

> I'd like to create a few classifier rules and see what gets populated into
> the database and also to understand complicated of an SQL query is SQL
> Alchemy generating in order to reassemble each rule from its polymorphic
> representation in the database.


Yeah. Right now my unit tests are super hacky, since I'm creating a DB
engine and session and passing them into the API. It's dumb and ugly but
I'm still reading through oslo_db's dev docs and some of how Neutron
creates the database session and packs it into a context so it can be
passed around, so my code doesn't make you want to claw your eyes out.

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Ihar Hrachyshka]

Cathy Zhang  wrote:

Agree with Paul and Louis. The networking-sfc repo should be preserved to  
support the service function chain functionality. Flow classifier is just  
needed to specify what flows will go through the service port chain.


The flow classifier API is designed as a separate plugin which is  
independent of the port chain plugin. We will support the effort of  
evolving it to a common service classifier API and moving it out of the  
networking-sfc repo when the time comes.


The problem with that kind of thinking is that TC is needed for other  
features beyond sfc, some of them with strict compatibility requirements  
(like the new fwaas API; or security groups). Once we adopt some API for  
those features, we can’t just drop their support, saying we still iterate  
on it. I am all for being more flexible and not get in the backwards  
compatible van if we are really explicit that those APIs based on new TC  
are not supported whatsoever (I know fwaas had that EXPERIMENTAL tag  
throughout its whole history and lived with that; but I am not sure it  
helped the project adoption; that said, for new projects with no binding  
guarantees to users like SFC it may be a burden to consider this API  
stable).


All I am saying is that IF we merge some classifier API into neutron core  
and start using it for core, non-experimental, features, we cannot later  
move to some newer version of this API [that you will iterate on] without  
leaving a huge pile of compatibility code that would not exist in the first  
place if only we thought about proper API in advance. If that’s what you  
envision, fine; but I believe it will make adoption of the ‘evolving’ API a  
lot slower than it could otherwise be.


Are experimental features the only features we see adopt the TC API in the  
near future?


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Paul Carver]

On 11/12/2015 3:50 PM, Ihar Hrachyshka wrote:


All I am saying is that IF we merge some classifier API into neutron
core and start using it for core, non-experimental, features, we cannot
later move to some newer version of this API [that you will iterate on]
without leaving a huge pile of compatibility code that would not exist
in the first place if only we thought about proper API in advance. If
that’s what you envision, fine; but I believe it will make adoption of
the ‘evolving’ API a lot slower than it could otherwise be.


I don't think I disagree at all. But we don't have a classifier API in 
neutron core (unless we consider security groups to be it) and I don't 
think anyone is saying that the classifier in networking-sfc should be 
merged straight into core as-is. In fact I think we're saying exactly 
the opposite, that *a* classifier will sit in networking-sfc, outside of 
core neutron, until *some* classifier is merged into core neutron.


The point of networking-sfc isn't the classifier. A classifier is simply 
a prerequisite. So by all means let's work on defining and merging into 
core neutron a classifier that we can consider non-experimental and 
stable for all features to share and depend on, but we don't want SFC to 
be non-functional while we wait for that to happen. We can call the 
networking-sfc classifier experimental a slap a warning on that it'll be 
replaced with the core neutron classifier once such a thing has been 
implemented.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Paul Carver]

On 11/3/2015 1:03 PM, Sean M. Collins wrote:

Anyway, the code is currently up on GitHub - I just threw it on there
because I wanted to scratch my hacking itch quickly.

https://github.com/sc68cal/neutron-classifier



Sean,

How much is needed to turn your models into something runnable to the 
extent of populating a database? I'm not really all that proficient with 
SQL Alchemy or SQL in general so I can't really visualize what the 
polymorphism statements in your model actually create.


I'd like to create a few classifier rules and see what gets populated 
into the database and also to understand complicated of an SQL query is 
SQL Alchemy generating in order to reassemble each rule from its 
polymorphic representation in the database.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Cathy Zhang]

Agree with Paul and Louis. The networking-sfc repo should be preserved to 
support the service function chain functionality. Flow classifier is just 
needed to specify what flows will go through the service port chain. 

The flow classifier API is designed as a separate plugin which is independent 
of the port chain plugin. We will support the effort of evolving it to a common 
service classifier API and moving it out of the networking-sfc repo when the 
time comes. 

Thanks,
Cathy

-Original Message-
From: Henry Fourie [mailto:louis.fou...@huawei.com] 
Sent: Wednesday, November 11, 2015 2:33 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic 
classifiers

Paul,
   Agree completely that the networking-sfc repo should be preserved as it 
includes functionality beyond that of just a classifier - it defines the 
service chain structure. 

Work on a common service classifier API could be done by the networking-sfc 
team to help in evaluating that API.

 - Louis   

-Original Message-
From: Paul Carver [mailto:pcar...@paulcarver.us]
Sent: Wednesday, November 11, 2015 1:07 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic 
classifiers

On 11/10/2015 8:30 AM, Sean M. Collins wrote:
> On Mon, Nov 09, 2015 at 07:58:34AM EST, Jay Pipes wrote:
>
>> 2) Keep the security-group API as-is to keep outward compatibility with AWS.
>> Create a single, new service-groups and service-group-rules API for
>> L2 to L7 traffic classification using mostly the modeling that Sean has put 
>> together.
>> Remove the networking-sfc repo and obselete the classifier spec. Not 
>> sure what should/would happen to the FWaaS API, frankly.
>
> As to the REST-ful API for creating classifiers, I don't know if it 
> should reside in the networking-sfc project. It's a big enough piece 
> that it will most likely need to be its own endpoint and repo, and 
> have stakeholders from other projects, not just networking-sfc. That 
> will take time and quite a bit of wrangling, so I'd like to defer that 
> for a bit and just work on all the services having the same data 
> model, where we can make changes quickly, since they are not visible 
> to API consumers.
>

I agree that the service classifier API should NOT reside in the networking-sfc 
project, but I don't understand why Jay suggests removing the networking-sfc 
repo. The classifier specified by networking-sfc is needed only because there 
isn't a pre-existing classifier API. As soon as we can converge on a common 
classifier API I am completely in favor of using it in place of the one in the 
networking-sfc repo, but SFC is more than just classifying traffic. We need a 
classifier in order to determine which traffic to redirect, but we also need 
the API to specify how to redirect the traffic that has been identified by 
classifiers.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Paul Carver]

On 11/10/2015 8:30 AM, Sean M. Collins wrote:

On Mon, Nov 09, 2015 at 07:58:34AM EST, Jay Pipes wrote:


2) Keep the security-group API as-is to keep outward compatibility with AWS.
Create a single, new service-groups and service-group-rules API for L2 to L7
traffic classification using mostly the modeling that Sean has put together.
Remove the networking-sfc repo and obselete the classifier spec. Not sure
what should/would happen to the FWaaS API, frankly.


As to the REST-ful API for creating classifiers, I don't know if it
should reside in the networking-sfc project. It's a big enough piece
that it will most likely need to be its own endpoint and repo, and have
stakeholders from other projects, not just networking-sfc. That will
take time and quite a bit of wrangling, so I'd like to defer that for a
bit and just work on all the services having the same data model, where
we can make changes quickly, since they are not visible to API
consumers.



I agree that the service classifier API should NOT reside in the 
networking-sfc project, but I don't understand why Jay suggests removing 
the networking-sfc repo. The classifier specified by networking-sfc is 
needed only because there isn't a pre-existing classifier API. As soon 
as we can converge on a common classifier API I am completely in favor 
of using it in place of the one in the networking-sfc repo, but SFC is 
more than just classifying traffic. We need a classifier in order to 
determine which traffic to redirect, but we also need the API to specify 
how to redirect the traffic that has been identified by classifiers.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Henry Fourie]

Paul,
   Agree completely that the networking-sfc repo should be preserved
as it includes functionality beyond that of just a classifier -
it defines the service chain structure. 

Work on a common service classifier API could be done by
the networking-sfc team to help in evaluating that API.

 - Louis   

-Original Message-
From: Paul Carver [mailto:pcar...@paulcarver.us] 
Sent: Wednesday, November 11, 2015 1:07 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic 
classifiers

On 11/10/2015 8:30 AM, Sean M. Collins wrote:
> On Mon, Nov 09, 2015 at 07:58:34AM EST, Jay Pipes wrote:
>
>> 2) Keep the security-group API as-is to keep outward compatibility with AWS.
>> Create a single, new service-groups and service-group-rules API for 
>> L2 to L7 traffic classification using mostly the modeling that Sean has put 
>> together.
>> Remove the networking-sfc repo and obselete the classifier spec. Not 
>> sure what should/would happen to the FWaaS API, frankly.
>
> As to the REST-ful API for creating classifiers, I don't know if it 
> should reside in the networking-sfc project. It's a big enough piece 
> that it will most likely need to be its own endpoint and repo, and 
> have stakeholders from other projects, not just networking-sfc. That 
> will take time and quite a bit of wrangling, so I'd like to defer that 
> for a bit and just work on all the services having the same data 
> model, where we can make changes quickly, since they are not visible 
> to API consumers.
>

I agree that the service classifier API should NOT reside in the networking-sfc 
project, but I don't understand why Jay suggests removing the networking-sfc 
repo. The classifier specified by networking-sfc is needed only because there 
isn't a pre-existing classifier API. As soon as we can converge on a common 
classifier API I am completely in favor of using it in place of the one in the 
networking-sfc repo, but SFC is more than just classifying traffic. We need a 
classifier in order to determine which traffic to redirect, but we also need 
the API to specify how to redirect the traffic that has been identified by 
classifiers.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Eichberger, German]

Sean and Mickey +1

As much as I like us using the same API to create classifiers (users only need 
to learn one way) I think for the short term we should work with our own 
constructs and rely on a common data model. That will allow us to iterate 
faster on the REST API level and not have premature constraints (as Mickey 
mentions).

Midterm we should create some common API which is a superset of the 
functionality of all projects using it.

German



On 11/10/15, 5:30 AM, "Sean M. Collins"  wrote:

>On Mon, Nov 09, 2015 at 07:58:34AM EST, Jay Pipes wrote:
>> In short, my preference, in order, would be:
>> 
>> 1) Enhance/evolve the existing security-groups and security-group-rules API
>> in Neutron to support more generic classification of traffic from L2 to L7,
>> using mostly the modeling that Sean has put together in his PoC library.
>
>
>
>> 2) Keep the security-group API as-is to keep outward compatibility with AWS.
>> Create a single, new service-groups and service-group-rules API for L2 to L7
>> traffic classification using mostly the modeling that Sean has put together.
>> Remove the networking-sfc repo and obselete the classifier spec. Not sure
>> what should/would happen to the FWaaS API, frankly.
>> 
>
>I'd prefer that since we're already redesigning the Firewall API that we
>go ahead and make the Firewall API more expressive, so users can
>classify L2 to L7 traffic and then make filtering decisions. Let's not
>complicate the Security Group API with more advanced features that we
>just bolt on. So my vote is for #2 - with slight adjustments. I still
>think the networking-sfc repo should stay around, and that collaboration
>on the common classifier framework should happen, so that we can start
>both projects (sfc and fwaas) with a common datamodel for the classifier
>piece.
>
>As to the REST-ful API for creating classifiers, I don't know if it
>should reside in the networking-sfc project. It's a big enough piece
>that it will most likely need to be its own endpoint and repo, and have
>stakeholders from other projects, not just networking-sfc. That will
>take time and quite a bit of wrangling, so I'd like to defer that for a
>bit and just work on all the services having the same data model, where
>we can make changes quickly, since they are not visible to API
>consumers.
>
>-- 
>Sean M. Collins
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Sean M. Collins]

On Mon, Nov 09, 2015 at 07:58:34AM EST, Jay Pipes wrote:
> In short, my preference, in order, would be:
> 
> 1) Enhance/evolve the existing security-groups and security-group-rules API
> in Neutron to support more generic classification of traffic from L2 to L7,
> using mostly the modeling that Sean has put together in his PoC library.



> 2) Keep the security-group API as-is to keep outward compatibility with AWS.
> Create a single, new service-groups and service-group-rules API for L2 to L7
> traffic classification using mostly the modeling that Sean has put together.
> Remove the networking-sfc repo and obselete the classifier spec. Not sure
> what should/would happen to the FWaaS API, frankly.
> 

I'd prefer that since we're already redesigning the Firewall API that we
go ahead and make the Firewall API more expressive, so users can
classify L2 to L7 traffic and then make filtering decisions. Let's not
complicate the Security Group API with more advanced features that we
just bolt on. So my vote is for #2 - with slight adjustments. I still
think the networking-sfc repo should stay around, and that collaboration
on the common classifier framework should happen, so that we can start
both projects (sfc and fwaas) with a common datamodel for the classifier
piece.

As to the REST-ful API for creating classifiers, I don't know if it
should reside in the networking-sfc project. It's a big enough piece
that it will most likely need to be its own endpoint and repo, and have
stakeholders from other projects, not just networking-sfc. That will
take time and quite a bit of wrangling, so I'd like to defer that for a
bit and just work on all the services having the same data model, where
we can make changes quickly, since they are not visible to API
consumers.

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Jay Pipes]

On 11/03/2015 12:19 PM, Ihar Hrachyshka wrote:

Now, I don't think that we need two APIs for the same thing. I would
be glad if we instead converge on a single API, making sure all cases
are covered. In the end, the feature is just a building block for
other features, like fwaas, security groups, or QoS.

We could build traffic classifier use cases on top of service groups,
though the name of the latter is a bit specific, and could use some
more generalization to cover other cases where we need to classify
traffic that may belong to different services; or vice versa, may
split into several categories, even while having a single service source.

I encourage those who work on traffic classifier, and those who will
implement and review service group feature, to start discussion on how
we converge and avoid multiple APIs for similar things.


+1000

Please do not create 3 APIs that essentially do the exact same thing. I 
think Sean's on the right track with the modeling he's been doing in his 
PoC library.


I think the classifier spec from Yuji Azama is better than the 
networking-sfc service function chaining API proposal because, well, it 
actually describes an API instead of a series of CLI commands.


The problem with the classifier proposal, though, is that it flattens 
the API into just the rules part, discarding the grouping part that 
allows one to define groups of rules. The service-group[-rules] spec 
gets this part correct.


At this point, I'd recommend just enhancing the existing security-group 
and security-group-rules APIs, though, since having both "service-group" 
and "security-group" in the API referring to similar concepts will be 
quite confusing IMHO. But then, Sean has told me he doesn't want to 
change the original AWS security group concept in the Neutron API... I'm 
on the fence about whether that would really be problematic if the 
security-group[-rules] API is enhanced/evolved appropriately.


In short, my preference, in order, would be:

1) Enhance/evolve the existing security-groups and security-group-rules 
API in Neutron to support more generic classification of traffic from L2 
to L7, using mostly the modeling that Sean has put together in his PoC 
library.


2) Keep the security-group API as-is to keep outward compatibility with 
AWS. Create a single, new service-groups and service-group-rules API for 
L2 to L7 traffic classification using mostly the modeling that Sean has 
put together. Remove the networking-sfc repo and obselete the classifier 
spec. Not sure what should/would happen to the FWaaS API, frankly.


Best,
-jay



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Mickey Spiegel]

I agree that the timing is good for FWaaS to look at the possibility of using a 
common traffic classifier in the new FWaaS 2.0 API. If we can agree on a 
definition and get moving on an implementation in the Mitaka time frame, we 
could avoid a painful API migration later on. Sean has started moving us down 
this path with his proposal for a common classifier database. Assuming that we 
move in this direction, the service groups feature would be aimed at the common 
traffic classifier rather than the FWaaS API itself.

Although various features such as QoS, FWaaS, security groups, SFC want to do 
traffic classification based on similar parameters such as source and 
destination IP address, protocol, source and destination L4 port, etc, there 
are different ways to structure this information. Through grouping and 
indirection, we can structure the model so that only experts deal with IP 
addresses and L4 ports, while users reference reusable groups. Thinking about 
sharing and reuse, it makes sense to break the classification parameters into 
two parts: identity (including source and destination IP address, source and 
destination MAC address), and service (protocol, source and destination L4 
ports, L7 parameters). Perhaps QoS related parameters and encapsulation related 
parameters would add more parts.

Service groups are not aimed at the entirety of  traffic classifier, they are 
aimed at only the service part of the traffic classifier. A tenant may be 
running the same application in several places. In these different places, the 
identity parameters will vary, but the service parameters (protocol, source and 
destination L4 ports, L7 parameters) will be the same. One expert could define 
a service group for a particular application, then multiple users could 
reference those service groups as they roll out that application in different 
places in the cloud. The expert could be someone within the tenant's 
organization, or it could be a service provider admin. To address this later 
case, we should introduce the notion of shared service groups, typically 
defined by the admin but used by different tenants.

For some applications, there may be multiple sets of L4 ports that must be 
allowed. This is addressed through the grouping aspect of service groups.

In the future, as deep packet inspection comes into the picture, the service 
can be identified through application ID rather than or in addition to L4 port. 
This would involve enhancements to the L7 parameters within service groups. The 
indirection through service groups would isolate this change from the base 
traffic classifier itself.

Note that security groups and service groups are different things, aimed at 
different aspects of traffic classification. Security groups are aimed at the 
identity part of the traffic classifier, allowing users to refer to groups of 
Neutron ports rather than having to replicate rules over and over again for 
different IP addresses. Service groups are aimed at the service part of the 
traffic classifier, allowing users to refer to application-related service 
groups rather than having to specify L4 ports.

For the upcoming FWaaS 2.0 API, we will want traffic classification using both 
the notions of security groups (generalized to get away from existing 
connotations of the term "security group") and service groups. I hope that we 
can agree on a common traffic classifier built on these concepts.

Mickey

-Jay Pipes <jaypi...@gmail.com> wrote: -
To: openstack-dev@lists.openstack.org
From: Jay Pipes <jaypi...@gmail.com>
Date: 11/09/2015 05:04AM
Subject: Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic 
classifiers

On 11/03/2015 12:19 PM, Ihar Hrachyshka wrote:
> Now, I don't think that we need two APIs for the same thing. I would
> be glad if we instead converge on a single API, making sure all cases
> are covered. In the end, the feature is just a building block for
> other features, like fwaas, security groups, or QoS.
>
> We could build traffic classifier use cases on top of service groups,
> though the name of the latter is a bit specific, and could use some
> more generalization to cover other cases where we need to classify
> traffic that may belong to different services; or vice versa, may
> split into several categories, even while having a single service source.
>
> I encourage those who work on traffic classifier, and those who will
> implement and review service group feature, to start discussion on how
> we converge and avoid multiple APIs for similar things.

+1000

Please do not create 3 APIs that essentially do the exact same thing. I 
think Sean's on the right track with the modeling he's been doing in his 
PoC library.

I think the classifier spec from Yuji Azama is better than the 
networking-sfc service function chaining API proposal because, well, it 
actually describes an API instead of a series of

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Sean M. Collins]

On Tue, Nov 03, 2015 at 02:08:26PM EST, Vikram Choudhary wrote:
> Thanks for all your efforts Sean.
> 
> I was actually thinking a separate IRC for this effort would be great and
> will help all the interested people to come together and develop.
> 
> Any thoughts on this?

Unless it becomes super popular I think it's fine to just discuss on
#openstack-neutron. 

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Ihar Hrachyshka]

Sean M. Collins  wrote:


Hi Ihar,

This sounds good. I actually had a draft e-mail that I've been saving
until I got back, that may be relevant. Some contributors met on Friday
to discuss the packet classification framework, mostly centered around
just building a reusable library that can be shared among multiple
services.

It was my view that just getting the different APIs to share a common
data model would be a big first step, since we can refactor a lot of
common internal data structures without any user facing API changes.



Yes, code reuse could help a lot. Though I believe that convergence should  
also be achieved on API level. There is still no API merged for either  
traffic classifier or service groups, so I would be glad if we step back  
and think how we cover both cases with single API.


APIs are not that easy to refactor or deprecate since they are user  
visible. I better take a bit more time to polish single API than throw  
multiple application specific API as needed.


I quickly went back to my hotel room on Friday (after stealing some red  
bulls from the

dev lounge) to start hacking on a shared library for packet
classification, that can be re-used by other projects.

At this point, the code is mostly SQLAlchemy models, but the objective is  
to
try and see if the models are actually useful, and can be re-used by  
multiple services.


On the FwaaS side I plan on proving out the models by attempting to
replace some of the FwaaS database models with models from the
common-classifier. I also plan on putting together some simple tests to
see if it can also handle classifiers for security groups in the future,
since there has already been some ideas about creating a common backend
for both FwaaS and the Security Group implementation.

Anyway, the code is currently up on GitHub - I just threw it on there
because I wanted to scratch my hacking itch quickly.

https://github.com/sc68cal/neutron-classifier

Hopefully this can help spur more discussion.



That’s a good start, and I like the approach with inheritance. Let’s  
iterate on the neutron-specs review for traffic classifier to get to some  
result.


In the meantime, I highly recommend we don’t merge anything for service  
groups.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Vikram Choudhary]

I am fine with this!

-Original Message-
From: Sean M. Collins [mailto:s...@coreitpro.com] 
Sent: 04 November 2015 14:28
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic 
classifiers

On Tue, Nov 03, 2015 at 02:08:26PM EST, Vikram Choudhary wrote:
> Thanks for all your efforts Sean.
> 
> I was actually thinking a separate IRC for this effort would be great 
> and will help all the interested people to come together and develop.
> 
> Any thoughts on this?

Unless it becomes super popular I think it's fine to just discuss on 
#openstack-neutron. 

--
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi all,

on fwaas design session in Tokyo it was pointed out that for new fwaas
API we may want to use service groups [1] that were approved but never
merged. As far as I can tell, service groups are designed to catch
several criteria to describe a type of networking application
'service'. Possible criteria are e.g.: port, protocol, icmp code or
type, ...

I need to admit this is was the first time when I heard about this new
feature proposed. And it immediately hit me that it somehow resembles
traffic classifier feature [2] that we look into QoS context for
Mitaka. The classifier thingy is designed to describe traffic types,
and is expected to support multiple criteria, including: port, mac,
ether type, protocol, ... You can see that the lists of possible
criteria are quite similar, and it's of no surprise since for what I
wonder both features are designed to do the same thing: to allow to
match and classify traffic based on criteria, and then use those sets
of criteria to apply different policies (whether it's firewall, QoS
marks, or any other action you can think of for specific traffic type).

Now, I don't think that we need two APIs for the same thing. I would
be glad if we instead converge on a single API, making sure all cases
are covered. In the end, the feature is just a building block for
other features, like fwaas, security groups, or QoS.

We could build traffic classifier use cases on top of service groups,
though the name of the latter is a bit specific, and could use some
more generalization to cover other cases where we need to classify
traffic that may belong to different services; or vice versa, may
split into several categories, even while having a single service source
.

I encourage those who work on traffic classifier, and those who will
implement and review service group feature, to start discussion on how
we converge and avoid multiple APIs for similar things.

Am I making any sense?

[1]:
http://specs.openstack.org/openstack/neutron-specs/specs/juno/service-gr
oup.html
[2]: https://review.openstack.org/#/c/190463/

Ihar
-BEGIN PGP SIGNATURE-

iQEcBAEBAgAGBQJWOOy5AAoJEC5aWaUY1u57TF0IAMgfcJosHFPTIOObNxOVdSBv
SnCRrTWOH0KKP1omyFh3oEWyiZJcAWarAUYPCdpKDo9PNF73jCUJcE0ieiWnIjzN
2N7Km1c7nxDNla5oGhHIlckVkeVKwrt8y1JiuJkqCB59FlgJ1wCYKiKipx3hQKTN
TqmU7kjpt5VL7L1uRCJIQ5GN1vwpEA4xzcBG39xdZe6PzP41ztGDO0Cdkeo63xYj
AvvFfW/KxZ332+PyZS4ZtYYLFd33s0PCe70g4CcnfuM/3Ma350gUdJAPdz4knrDx
5f9oPdkJDfBJTmxmz5GJJFKjc/FwFydy5J69jEWSeWKx+dM1tUbCS5hwaloSqk4=
=kJ70
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Sean M. Collins]

Hi Ihar,

This sounds good. I actually had a draft e-mail that I've been saving
until I got back, that may be relevant. Some contributors met on Friday
to discuss the packet classification framework, mostly centered around
just building a reusable library that can be shared among multiple
services.

It was my view that just getting the different APIs to share a common
data model would be a big first step, since we can refactor a lot of
common internal data structures without any user facing API changes. 

I quickly went back to my hotel room on Friday (after stealing some red bulls 
from the
dev lounge) to start hacking on a shared library for packet
classification, that can be re-used by other projects.

At this point, the code is mostly SQLAlchemy models, but the objective is to
try and see if the models are actually useful, and can be re-used by multiple 
services.

On the FwaaS side I plan on proving out the models by attempting to
replace some of the FwaaS database models with models from the
common-classifier. I also plan on putting together some simple tests to
see if it can also handle classifiers for security groups in the future,
since there has already been some ideas about creating a common backend
for both FwaaS and the Security Group implementation.

Anyway, the code is currently up on GitHub - I just threw it on there
because I wanted to scratch my hacking itch quickly.

https://github.com/sc68cal/neutron-classifier

Hopefully this can help spur more discussion.

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Vikram Choudhary]

Thanks for all your efforts Sean.

I was actually thinking a separate IRC for this effort would be great and
will help all the interested people to come together and develop.

Any thoughts on this?

Thanks
Vikram
On Nov 3, 2015 11:54 PM, "Sean M. Collins"  wrote:

> I made a very quick attempt to jot down my thoughts about how it could
> be used. It's based off what I proposed in
> https://review.openstack.org/238812,
> and is my attempt to take that review and use SQLAlchemy to make it
> actually work.
>
>
> https://github.com/sc68cal/neutron-classifier/blob/master/doc/source/usage.rst
>
> It's very basic, it was just me hacking away at a proof of concept to
> demonstrate that what I was proposing was possible, that we didn't need
> a single database table with 10+ columns.
> --
> Sean M. Collins
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

[Sean M. Collins]

I made a very quick attempt to jot down my thoughts about how it could
be used. It's based off what I proposed in https://review.openstack.org/238812, 
and is my attempt to take that review and use SQLAlchemy to make it
actually work.

https://github.com/sc68cal/neutron-classifier/blob/master/doc/source/usage.rst

It's very basic, it was just me hacking away at a proof of concept to
demonstrate that what I was proposing was possible, that we didn't need
a single database table with 10+ columns.
-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev