On 11/14/2013 11:39 AM, Tim Hinrichs wrote:
I completely agree that making Congress successful will rely crucially on
addressing performance and scalability issues. Some thoughts...
1. We're definitely intending to cache data locally to avoid repeated API
calls. In fact, a prototype cache is already in place. We haven't yet hooked
up API calls (other than to AD). We envision some data sources pushing us data
(updates) and some data sources requiring us to periodically pull.
So, I think that you should start with what we already have working.
Policy distribution is a part of Keystone now. Policy processing uses
Oslo policy.py. I am not sure why it would make sense to have a
separate program. I would be more than happy to house this work inside
the existing Keystone architecture.
2. My main concern for scalability/performance is for proactive enforcement,
where at least conceptually Congress is on the critical path for API calls.
One thought is that we could splice out, say, the network portion of the
Congress policy and push it down into neutron, assuming neutron could enforce
that policy. This would at least eliminate cross-component communication. It
would require a policy engine on each of the OS components, but (a) there
already is one on many components and (b) if there isn't, we can rely on
reactive enforcement.
The downside with policy-caching on other OS components are the usual problems
with staleness and data replication, e.g. maybe we'd end up copying all of
nova's VM data into neutron so that neutron could enforce its policy. But
because we have reactive enforcement to rely on, we could always approximate
the policy that we push down (conservatively) to catch the common mistakes and
leave the remainder to reactive enforcement. For example, we might be able to
auto-generate the current policy.json files for each component from Congress's
policy.
Keeping Congress out of the critical path for every API call is one of the
reasons it was designed to do reactive enforcement as well as proactive
enforcement.
I think all these questions have come up multiple times with Keystone.
There was a signifcant discussion about Configuration management, which
would encompass policy. We were discussing the Key Distribution Service
as well. It bascially comes down to "inside the undercloud" versus
outside (end user facing).
Termie's suggestion was that we needed to consider using something like
Zookeeper. I think he is right.
3. Another option is to build high-performance optimizations for certain
fragments of the policy language. Then the cloud architect can decide whether
she wants to utilize a more expressive language whose performance is worse or a
less expressive language whose performance is better.
So we should not be writing yet another policy language. We have a
simple one in Open Stack already, and there are many, many ones out
there already. XACML is the standard for authorization decisions, as an
example. Puppet falls into this role as well.
I don't think this effort makes sense as a stand alone project. Either
it should be part of Keystone if it just for the existing acces policy,
or it should be part of a larger configuration management effort. I
realize that there is a commonality of effort in policy enforcement that
sounds like it should be a stand alone project, but I think that there
is much more to the conversation than just that.
Tim
----- Original Message -----
| From: "Flavio Percoco" <fla...@redhat.com>
| To: "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev@lists.openstack.org>
| Sent: Thursday, November 14, 2013 6:05:46 AM
| Subject: Re: [openstack-dev] Congress: an open policy framework
|
| On 14/11/13 04:40 -0800, Morgan Fainberg wrote:
| >On Wed, Nov 13, 2013 at 10:40 AM, Tim Hinrichs
| ><thinri...@vmware.com> wrote:
| >> We're just getting started with Congress and understanding how it
| >> will integrate with the OS ecosystem, but here's our current
| >> thinking about how Congress relates to Oslo's policy engine and
| >> to Keystone. Comments and suggestions are welcome.
| >>
| >>
| >> Congress and Oslo
| >> --------------------
| >> Three dimensions for comparison: policy language, data sources,
| >> and policy engine.
| >>
| >> We've always planned to make Congress compatible with existing
| >> policy languages like the one in oslo. The plan is to build a
| >> front-end for a number of policy languages/formats, e.g.
| >> oslo-policy language, XACML, JSON, YAML, SQL, etc. The idea
| >> being that the syntax/language you use is irrelevant as long as
| >> it can be mapped into Congress's native policy language. As of
| >> now, Congress is using Datalog, which is a variant of SQL and is
| >> at least as expressive as all of the policy languages we've run
| >> across in the cloud domain, including the oslo-policy language.
| >>
| >> In terms of the data sources you can reference in the policy,
| >> Congress is designed to enable policies that reference arbitrary
| >> data sources in the cloud. For example, we could write a Nova
| >> authorization policy that permits a new VM to be created if that
| >> VM is connected to a network owned by a tenant (info stored in
| >> Neutron) where the VM owner (info in the request) is in the same
| >> group as the network owner (info stored in Keystone/LDAP).
| >> Oslo's handles some of these data sources with its terminal
| >> rules, but it's not involved in data integration to the same
| >> extent Congress is.
| >>
| >> In terms of policy engines, Congress is intended to enforce
| >> policies in 2 different ways: proactively (stopping policy
| >> violations before they occur) and reactively (acting to eliminate
| >> a violation after it occurs). Ideally we wouldn't need reactive
| >> enforcement, but there will always be cases where proactive
| >> enforcement is not possible (e.g. a DOS attack brings app
| >> latencies out of compliance). The oslo-engine does proactive
| >> enforcement only--stopping API calls before they violate the
| >> policy.
| >>
| >
| >Does this mean all policy decisions need to ask this new service?
| >There are many policy checks that occur across even a given action
| >(in
| >some cases). Could this have a significant performance implication
| >on
| >larger scale cloud deployments? I like the idea of having reactive
| >(DOS prevention) policy enforcement as well as external (arbitrary)
| >data to help make policy decisions, I don't want to see Congress be
| >limited in deployment because large scale clouds getting
| >bottle-necked
| >trying to communicate with it.
|
| This is exactly what worries me about Congress. I mentioned in my
| last
| email that some kind of 'local' cache managed by the Confress library
| is a must to avoid the performance penalty.
|
| >There might be some value to seeing some work being done to provide
| >more information to Keystone, but I think this will become more
| >apparent as Congress develops.
|
| +1
|
| Cheers,
| FF
|
| --
| @flaper87
| Flavio Percoco
|
| _______________________________________________
| OpenStack-dev mailing list
| OpenStack-dev@lists.openstack.org
| http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
|
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
