[openstack-dev] [Ceilometer] Ceilometer API RBAC enhancement proposal (work in progress)

[Pendergrass, Eric]

To anyone who's interested, we're working on a proposal and implementation
of a fine-grained v3 compatible RBAC scheme for the API.

The WIP spec change is here:  
https://review.openstack.org/#/c/112137/

I realize this will need to move from Juno to Kilo because it won't make
Juno.

And the code which is mostly implemented is here:  
https://review.openstack.org/#/c/115717/

I'd be interested in any comments from the community on this.  Those
following the recent discussions on hooks and decorators will know why we
chose the @secure decorators instead of Hooks.

Please keep in mind this is not final code or blueprints, but an early work
in progress.  Still, any suggestions appreciated.

Thanks!
Eric


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Way to get wrapped method's name/class using Pecan secure decorators?

[Pendergrass, Eric]

Hi Ryan,

We tried globally applying the hook but could not get execution to enter the
hook class. 

Perhaps we made a mistake, but we concluded the Controller still had to
inherit from HookController using the project-wide method.  Otherwise we
would have been satisfied applying it project-wide.

Thanks,
Eric

> Eric,

> 
> Doug's correct - this looks like a bug in pecan that occurs when you
subclass both rest.RestController and hooks.HookController.  I'm working on
a bug fix as we speak.  In the meantime, have you tried applying hooks at a
global application level?  This approach should still work.
> 
> On 08/14/14 04:38 PM, Pendergrass, Eric wrote:
> > Sure, Doug.  We want the ability to selectively apply policies to 
> > certain Ceilometer API methods based on user/tenant roles.
> > 
> > For example, we want to restrict the ability to execute Alarm deletes 
> > to admins and user/tenants who have a special role, say "domainadmin".
> > 
> > The policy file might look like this:
> > {
> > "context_is_admin":  [["role:admin"]],
> > "admin_and_matching_project_domain_id":  [["role:domainadmin"]],
> > "admin_or_cloud_admin": [["rule:context_is_admin"], 
> > ["rule:admin_and_matching_project_domain_id"]],
> > "telemetry:delete_alarms":  [["rule:admin_or_cloud_admin"]] }
> > 
> > The current acl.py and _query_to_kwargs access control setup either 
> > sets project_id scope to None (do everything) or to the project_id in 
> > the request header 'X-Project-Id'.  This allows for admin or project 
> > scope, but nothing in between.
> > 
> > We tried hooks.  Unfortunately we can't seem to turn the API 
> > controllers into HookControllers just by adding HookController to the 
> > Controller class definition.  It causes infinite recursion on API 
> > startup.  For example, this doesn't work because ceilometer-api will 
> > not start with it:
> > class MetersController(rest.RestController, HookController):
> > 
> > If there was a way to use hooks with the v2. API controllers that 
> > might work really well.
> > 
> > So we are left using the @secure decorator and deriving the method 
> > name from the request environ PATH_INFO and REQUEST_METHOD values.  
> > This is how we determine the wrapped method within the class 
> > (REQUEST_METHOD + PATH_INFO = "telemetry:delete_alarms" with some 
> > munging).  We need the method name in order to selectively apply acces 
> > control to certain methods.
> > 
> > Deriving the method this way isn't ideal but it's the only thing we've 
> > gotten working between hooks, @secure, and regular decorators.
> > 
> > I submitted a WIP BP here: https://review.openstack.org/#/c/112137/3.  
> > It is slightly out of date but should give you a beter idea of our
goals.
> > 
> > Thanks
> > 
> > > Eric,
> > >
> > > If you can give us some more information about your end goal, 
> > > independent
> > of the implementation, maybe we can propose an alternate technique to 
> > achieve the same thing.
> > >
> > > Doug
> > >
> > > On Aug 12, 2014, at 6:21 PM, Ryan Petrello 
> > > 
> > wrote:
> > >
> > > > Yep, you're right, this doesn't seem to work.  The issue is that 
> > > > security is enforced at routing time (while the controller is 
> > > > still actually being discovered).  In order to do this sort of 
> > > > thing with the `check_permissions`, we'd probably need to add a
feature to pecan.
> > > >
> > > > On 08/12/14 06:38 PM, Pendergrass, Eric wrote:
> > > > > Sure, here's the decorated method from v2.py:
> > > > >
> > > > >class MetersController(rest.RestController):
> > > > >"""Works on meters."""
> > > > >
> > > > >@pecan.expose()
> > > > >def _lookup(self, meter_name, *remainder):
> > > > >return MeterController(meter_name), remainder
> > > > >
> > > > >@wsme_pecan.wsexpose([Meter], [Query])
> > > > >@secure(RBACController.check_permissions)
> > > > >def get_all(self, q=None):
> > > > >
> > > > > and here's the decorator called by the secure tag:
> > > > >
> > > > >class RBACController(object):
> >

Re: [openstack-dev] [Ceilometer] Way to get wrapped method's name/class using Pecan secure decorators?

[Pendergrass, Eric]

Sure, Doug.  We want the ability to selectively apply policies to certain
Ceilometer
API methods based on user/tenant roles.

For example, we want to restrict the ability to execute Alarm deletes to
admins and user/tenants who have a special role, say "domainadmin".

The policy file might look like this:
{
"context_is_admin":  [["role:admin"]],
"admin_and_matching_project_domain_id":  [["role:domainadmin"]],
"admin_or_cloud_admin": [["rule:context_is_admin"],
["rule:admin_and_matching_project_domain_id"]],
"telemetry:delete_alarms":  [["rule:admin_or_cloud_admin"]]
}

The current acl.py and _query_to_kwargs access control setup either sets
project_id scope to None (do everything) or to the project_id in the request
header 'X-Project-Id'.  This allows for admin or project scope, but nothing
in
between.

We tried hooks.  Unfortunately we can't seem to turn the API controllers
into
HookControllers just by adding HookController to the Controller class
definition.  It causes infinite recursion on API startup.  For example, this
doesn't work because ceilometer-api will not start with it:
class MetersController(rest.RestController, HookController):

If there was a way to use hooks with the v2. API controllers that might work
really well.

So we are left using the @secure decorator and deriving the method name from
the request environ PATH_INFO and REQUEST_METHOD values.  This is how we
determine the wrapped method within the class (REQUEST_METHOD + PATH_INFO =
"telemetry:delete_alarms" with some munging).  We need the method name in
order to
selectively apply acces control to certain methods.

Deriving the method this way isn't ideal but it's the only thing we've
gotten working 
between hooks, @secure, and regular decorators.

I submitted a WIP BP here: https://review.openstack.org/#/c/112137/3.  It is
slightly out of date but should give you a beter idea of our goals.

Thanks

> Eric,
>
> If you can give us some more information about your end goal, independent
of the implementation, maybe we can propose an alternate technique to
achieve the same thing.
>
> Doug
>
> On Aug 12, 2014, at 6:21 PM, Ryan Petrello 
wrote:
>
> > Yep, you're right, this doesn't seem to work.  The issue is that
> > security is enforced at routing time (while the controller is still
> > actually being discovered).  In order to do this sort of thing with
> > the `check_permissions`, we'd probably need to add a feature to pecan.
> >
> > On 08/12/14 06:38 PM, Pendergrass, Eric wrote:
> >> Sure, here's the decorated method from v2.py:
> >>
> >>class MetersController(rest.RestController):
> >>"""Works on meters."""
> >>
> >>@pecan.expose()
> >>def _lookup(self, meter_name, *remainder):
> >>return MeterController(meter_name), remainder
> >>
> >>@wsme_pecan.wsexpose([Meter], [Query])
> >>@secure(RBACController.check_permissions)
> >>def get_all(self, q=None):
> >>
> >> and here's the decorator called by the secure tag:
> >>
> >>class RBACController(object):
> >>global _ENFORCER
> >>if not _ENFORCER:
> >>_ENFORCER = policy.Enforcer()
> >>
> >>
> >>@classmethod
> >>    def check_permissions(cls):
> >># do some stuff
> >>
> >> In check_permissions I'd like to know the class and method with the
@secure tag that caused check_permissions to be invoked.  In this case, that
would be MetersController.get_all.
> >>
> >> Thanks
> >>
> >>
> >>> Can you share some code?  What do you mean by, "is there a way for the
decorator code to know it was called by MetersController.get_all"
> >>>
> >>> On 08/12/14 04:46 PM, Pendergrass, Eric wrote:
> >>>> Thanks Ryan, but for some reason the controller attribute is None:
> >>>>
> >>>> (Pdb) from pecan.core import state
> >>>> (Pdb) state.__dict__
> >>>> {'hooks': [,
> >>>> ,
> >>>> ,
> >>>> ], 'app':
> >>>> , 'request':  >>>> 0x3ed7390 GET http://localhost:8777/v2/meters>, 'controller': None,
> >>>> 'response': }
> >>>>
> >>>>> -Original Message-
> >>>>> From: Ryan Petrello [mailto:ryan.petre...@dreamhost.com]
> >>>>> Sent:

Re: [openstack-dev] [Ceilometer] Way to get wrapped method's name/class using Pecan secure decorators?

[Pendergrass, Eric]

Sure, here's the decorated method from v2.py:

class MetersController(rest.RestController):
"""Works on meters."""

@pecan.expose()
def _lookup(self, meter_name, *remainder):
return MeterController(meter_name), remainder

@wsme_pecan.wsexpose([Meter], [Query])
@secure(RBACController.check_permissions)
def get_all(self, q=None):

and here's the decorator called by the secure tag:

class RBACController(object):
global _ENFORCER
if not _ENFORCER:
_ENFORCER = policy.Enforcer()


@classmethod
def check_permissions(cls):
# do some stuff

In check_permissions I'd like to know the class and method with the @secure tag 
that caused check_permissions to be invoked.  In this case, that would be 
MetersController.get_all.

Thanks


> Can you share some code?  What do you mean by, "is there a way for the 
> decorator code to know it was called by MetersController.get_all"
>
> On 08/12/14 04:46 PM, Pendergrass, Eric wrote:
> > Thanks Ryan, but for some reason the controller attribute is None:
> >
> > (Pdb) from pecan.core import state
> > (Pdb) state.__dict__
> > {'hooks': [,
> > ,
> > ,
> > ], 'app':
> > , 'request':  > 0x3ed7390 GET http://localhost:8777/v2/meters>, 'controller': None,
> > 'response': }
> >
> > > -Original Message-
> > > From: Ryan Petrello [mailto:ryan.petre...@dreamhost.com]
> > > Sent: Tuesday, August 12, 2014 10:34 AM
> > > To: OpenStack Development Mailing List (not for usage questions)
> > > Subject: Re: [openstack-dev] [Ceilometer] Way to get wrapped method's 
> > > name/class using Pecan secure decorators?
> > >
> > > This should give you what you need:
> > >
> > > from pecan.core import state
> > > state.controller
> > >
> > > On 08/12/14 04:08 PM, Pendergrass, Eric wrote:
> > > > Hi, I'm trying to use the built in secure decorator in Pecan for access 
> > > > control, and I'ld like to get the name of the method that is wrapped 
> > > > from within the decorator.
> > > >
> > > > For instance, if I'm wrapping MetersController.get_all with an @secure 
> > > > decorator, is there a way for the decorator code to know it was called 
> > > > by MetersController.get_all?
> > > >
> > > > I don't see any global objects that provide this information.  I can 
> > > > get the endpoint, v2/meters, with pecan.request.path, but that's not as 
> > > > elegant.
> > > >
> > > > Is there a way to derive the caller or otherwise pass this information 
> > > > to the decorator?
> > > >
> > > > Thanks
> > > > Eric Pendergrass
> > >
> > > > ___
> > > > OpenStack-dev mailing list
> > > > OpenStack-dev@lists.openstack.org
> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> > >
> > > --
> > > Ryan Petrello
> > > Senior Developer, DreamHost
> > > ryan.petre...@dreamhost.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Way to get wrapped method's name/class using Pecan secure decorators?

[Pendergrass, Eric]

Thanks Ryan, but for some reason the controller attribute is None:

(Pdb) from pecan.core import state
(Pdb) state.__dict__
{'hooks': [, 
, 
, 
], 'app': 
, 'request': http://localhost:8777/v2/meters>, 'controller': None, 'response': }

> -Original Message-
> From: Ryan Petrello [mailto:ryan.petre...@dreamhost.com]
> Sent: Tuesday, August 12, 2014 10:34 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [Ceilometer] Way to get wrapped method's 
> name/class using Pecan secure decorators?
>
> This should give you what you need:
>
> from pecan.core import state
> state.controller
>
> On 08/12/14 04:08 PM, Pendergrass, Eric wrote:
> > Hi, I'm trying to use the built in secure decorator in Pecan for access 
> > control, and I'ld like to get the name of the method that is wrapped from 
> > within the decorator.
> >
> > For instance, if I'm wrapping MetersController.get_all with an @secure 
> > decorator, is there a way for the decorator code to know it was called by 
> > MetersController.get_all?
> >
> > I don't see any global objects that provide this information.  I can get 
> > the endpoint, v2/meters, with pecan.request.path, but that's not as elegant.
> >
> > Is there a way to derive the caller or otherwise pass this information to 
> > the decorator?
> >
> > Thanks
> > Eric Pendergrass
>
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> --
> Ryan Petrello
> Senior Developer, DreamHost
> ryan.petre...@dreamhost.com
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Way to get wrapped method's name/class using Pecan secure decorators?

[Pendergrass, Eric]

Hi, I'm trying to use the built in secure decorator in Pecan for access 
control, and I'ld like to get the name of the method that is wrapped from 
within the decorator.

For instance, if I'm wrapping MetersController.get_all with an @secure 
decorator, is there a way for the decorator code to know it was called by 
MetersController.get_all?

I don't see any global objects that provide this information.  I can get the 
endpoint, v2/meters, with pecan.request.path, but that's not as elegant.

Is there a way to derive the caller or otherwise pass this information to the 
decorator?

Thanks
Eric Pendergrass
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Question on decorators in Ceilometer pecan framework

[Pendergrass, Eric]

> From: David Stanek [mailto:dsta...@dstanek.com]
> Sent: Friday, August 08, 2014 7:25 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [Ceilometer] Question on decorators in
> Ceilometer pecan framework

> It looks like maybe WSME or Pecan is inspecting the method signature. Have
you
> tried to change the order of the decorators?

Good suggestion and we did try it.  Unfortunately the wsme decorator must
come
first else the endpoint isn't found and the client gets a 404.

Eric


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Question on decorators in Ceilometer pecan framework

[Pendergrass, Eric]

Wrong link again, this is embarrassing :( 
https://review.openstack.org/#/c/112137/3

From: Pendergrass, Eric
Sent: Friday, August 08, 2014 7:15 AM
To: openstack-dev@lists.openstack.org
Subject: RE: [Ceilometer] Question on decorators in Ceilometer pecan framework

Sorry, wrong BP review link below.  Here is the correct one:  
https://review.openstack.org/#/c/112127/3.  Please disregard the wiki link.

From: Pendergrass, Eric
Sent: Friday, August 08, 2014 6:50 AM
To: openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>
Cc: Giannetti, Fabio
Subject: [Ceilometer] Question on decorators in Ceilometer pecan framework

Hi,

We have been struggling to get a decorator working for proposed new RBAC 
functionality in ceilometer-api.  We're hitting a problem where GET request 
query parameters are mucked up by our decorator.  Here's an example call:

curl -H "X-Auth-Token:$TOKEN" 
'http://localhost:8777/v2/meters?q.field=project_id&q.value=8c678720fb5b4e3bb18dee222d7d7933'

And here's the decorator method (we've tried changing the kwargs, args, etc. 
with no luck):

_ENFORCER = None

def protected(controller_class):

global _ENFORCER
if not _ENFORCER:
_ENFORCER = policy.Enforcer()

def wrapper(f):
@functools.wraps(f)
def inner(self, **kwargs):
pdb.set_trace()
self._rbac_context = {}
if not _ENFORCER.enforce('context_is_admin',
 {},
 {'roles': 
pecan.request.headers.get('X-Roles', "").split(",")}):
self._rbac_context['project_id'] = 
pecan.request.headers.get('X-Project-Id')
self._rbac_context['user_id'] = 
pecan.request.headers.get('X-User-Id')
return f(self, **kwargs)
return inner
return wrapper

tried this too:

_ENFORCER = None

def protected(*args):

controller_class = 'meter'
global _ENFORCER
if not _ENFORCER:
_ENFORCER = policy.Enforcer()

def wrapper(f, *args):
def inner(self, *args):
pdb.set_trace()
#self._rbac_context = {}
#if not _ENFORCER.enforce('context_is_admin',
# {},
# {'roles': 
pecan.request.headers.get('X-Roles', "").split(",")}):
#self._rbac_context['project_id'] = 
pecan.request.headers.get('X-Project-Id')
#self._rbac_context['user_id'] = 
pecan.request.headers.get('X-User-Id')
#return f(*args)
f(self, *args)
return inner
return wrapper

and here's how it's used:

class MetersController(rest.RestController):
"""Works on meters."""

_rbac_context = {}
@pecan.expose()
def _lookup(self, meter_name, *remainder):
return MeterController(meter_name), remainder

@wsme_pecan.wsexpose([Meter], [Query])
@rbac_validate.protected('meters')
def get_all(self, q=None):
"""Return all known meters, based on the data recorded so far.

:param q: Filter rules for the meters to be returned.
"""
q = q or [] ...


but we get errors similar to below where the arg parser cannot find the query 
parameter because the decorator doesn't take a q argument as 
MetersController.get_all does.

Is there any way to get a decorator to work within the v2 API code and wsme 
framework or should we consider another approach?  Decorators would really 
simplify the RBAC idea we're working on, which is mostly code-implemented save 
for this fairly major problem.

I have a WIP registered BP on this at 
https://blueprints.launchpad.net/ceilometer/+spec/ready-ceilometer-rbac-keystone-v3.

If I can provide more details I'll be happy to.

Thanks
Eric

  /usr/local/bin/ceilometer-api(10)()
-> sys.exit(api())
  /opt/stack/ceilometer/ceilometer/cli.py(96)api()
-> srv.serve_forever()
  /usr/lib/python2.7/SocketServer.py(227)serve_forever()
-> self._handle_request_noblock()
  /usr/lib/python2.7/SocketServer.py(284)_handle_request_noblock()
-> self.process_request(request, client_address)
  /usr/lib/python2.7/SocketServer.py(310)process_request()
-> self.finish_request(request, client_address)
  /usr/lib/python2.7/SocketServer.py(323)finish_request()
-> self.RequestHandlerClass(request, client_address, self)
  /usr/lib/python2.7/SocketServer.py(638)__init__()
-> self.handle()
  /usr/lib/python2.7/wsgiref/simple_server.py(124)handle()
-> handler.run(self.server.get_app())
  /usr/lib/python2.7/wsgiref/handlers.py(85)run()
-> self.result = application(self.environ, self.start_response)
  
/opt/stack/pyt

Re: [openstack-dev] [Ceilometer] Question on decorators in Ceilometer pecan framework

[Pendergrass, Eric]

Sorry, wrong BP review link below.  Here is the correct one:  
https://review.openstack.org/#/c/112127/3.  Please disregard the wiki link.

From: Pendergrass, Eric
Sent: Friday, August 08, 2014 6:50 AM
To: openstack-dev@lists.openstack.org
Cc: Giannetti, Fabio
Subject: [Ceilometer] Question on decorators in Ceilometer pecan framework

Hi,

We have been struggling to get a decorator working for proposed new RBAC 
functionality in ceilometer-api.  We're hitting a problem where GET request 
query parameters are mucked up by our decorator.  Here's an example call:

curl -H "X-Auth-Token:$TOKEN" 
'http://localhost:8777/v2/meters?q.field=project_id&q.value=8c678720fb5b4e3bb18dee222d7d7933'

And here's the decorator method (we've tried changing the kwargs, args, etc. 
with no luck):

_ENFORCER = None

def protected(controller_class):

global _ENFORCER
if not _ENFORCER:
_ENFORCER = policy.Enforcer()

def wrapper(f):
@functools.wraps(f)
def inner(self, **kwargs):
pdb.set_trace()
self._rbac_context = {}
if not _ENFORCER.enforce('context_is_admin',
 {},
 {'roles': 
pecan.request.headers.get('X-Roles', "").split(",")}):
self._rbac_context['project_id'] = 
pecan.request.headers.get('X-Project-Id')
self._rbac_context['user_id'] = 
pecan.request.headers.get('X-User-Id')
return f(self, **kwargs)
return inner
return wrapper

tried this too:

_ENFORCER = None

def protected(*args):

controller_class = 'meter'
global _ENFORCER
if not _ENFORCER:
_ENFORCER = policy.Enforcer()

def wrapper(f, *args):
def inner(self, *args):
pdb.set_trace()
#self._rbac_context = {}
#if not _ENFORCER.enforce('context_is_admin',
# {},
# {'roles': 
pecan.request.headers.get('X-Roles', "").split(",")}):
#self._rbac_context['project_id'] = 
pecan.request.headers.get('X-Project-Id')
#self._rbac_context['user_id'] = 
pecan.request.headers.get('X-User-Id')
#return f(*args)
f(self, *args)
return inner
return wrapper

and here's how it's used:

class MetersController(rest.RestController):
"""Works on meters."""

_rbac_context = {}
@pecan.expose()
def _lookup(self, meter_name, *remainder):
return MeterController(meter_name), remainder

@wsme_pecan.wsexpose([Meter], [Query])
@rbac_validate.protected('meters')
def get_all(self, q=None):
"""Return all known meters, based on the data recorded so far.

:param q: Filter rules for the meters to be returned.
"""
q = q or [] ...


but we get errors similar to below where the arg parser cannot find the query 
parameter because the decorator doesn't take a q argument as 
MetersController.get_all does.

Is there any way to get a decorator to work within the v2 API code and wsme 
framework or should we consider another approach?  Decorators would really 
simplify the RBAC idea we're working on, which is mostly code-implemented save 
for this fairly major problem.

I have a WIP registered BP on this at 
https://blueprints.launchpad.net/ceilometer/+spec/ready-ceilometer-rbac-keystone-v3.

If I can provide more details I'll be happy to.

Thanks
Eric

  /usr/local/bin/ceilometer-api(10)()
-> sys.exit(api())
  /opt/stack/ceilometer/ceilometer/cli.py(96)api()
-> srv.serve_forever()
  /usr/lib/python2.7/SocketServer.py(227)serve_forever()
-> self._handle_request_noblock()
  /usr/lib/python2.7/SocketServer.py(284)_handle_request_noblock()
-> self.process_request(request, client_address)
  /usr/lib/python2.7/SocketServer.py(310)process_request()
-> self.finish_request(request, client_address)
  /usr/lib/python2.7/SocketServer.py(323)finish_request()
-> self.RequestHandlerClass(request, client_address, self)
  /usr/lib/python2.7/SocketServer.py(638)__init__()
-> self.handle()
  /usr/lib/python2.7/wsgiref/simple_server.py(124)handle()
-> handler.run(self.server.get_app())
  /usr/lib/python2.7/wsgiref/handlers.py(85)run()
-> self.result = application(self.environ, self.start_response)
  
/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py(663)__call__()
-> return self.app(env, start_response)
  /opt/stack/ceilometer/ceilometer/api/app.py(97)__call__()
-> return self.v2(environ, start_response)
  
/usr/local/lib/python2.7/dist-packages/pecan/middleware/static.py(151)__call__()
-> retur

[openstack-dev] [Ceilometer] Question on decorators in Ceilometer pecan framework

[Pendergrass, Eric]

Hi,

We have been struggling to get a decorator working for proposed new RBAC 
functionality in ceilometer-api.  We're hitting a problem where GET request 
query parameters are mucked up by our decorator.  Here's an example call:

curl -H "X-Auth-Token:$TOKEN" 
'http://localhost:8777/v2/meters?q.field=project_id&q.value=8c678720fb5b4e3bb18dee222d7d7933'

And here's the decorator method (we've tried changing the kwargs, args, etc. 
with no luck):

_ENFORCER = None

def protected(controller_class):

global _ENFORCER
if not _ENFORCER:
_ENFORCER = policy.Enforcer()

def wrapper(f):
@functools.wraps(f)
def inner(self, **kwargs):
pdb.set_trace()
self._rbac_context = {}
if not _ENFORCER.enforce('context_is_admin',
 {},
 {'roles': 
pecan.request.headers.get('X-Roles', "").split(",")}):
self._rbac_context['project_id'] = 
pecan.request.headers.get('X-Project-Id')
self._rbac_context['user_id'] = 
pecan.request.headers.get('X-User-Id')
return f(self, **kwargs)
return inner
return wrapper

tried this too:

_ENFORCER = None

def protected(*args):

controller_class = 'meter'
global _ENFORCER
if not _ENFORCER:
_ENFORCER = policy.Enforcer()

def wrapper(f, *args):
def inner(self, *args):
pdb.set_trace()
#self._rbac_context = {}
#if not _ENFORCER.enforce('context_is_admin',
# {},
# {'roles': 
pecan.request.headers.get('X-Roles', "").split(",")}):
#self._rbac_context['project_id'] = 
pecan.request.headers.get('X-Project-Id')
#self._rbac_context['user_id'] = 
pecan.request.headers.get('X-User-Id')
#return f(*args)
f(self, *args)
return inner
return wrapper

and here's how it's used:

class MetersController(rest.RestController):
"""Works on meters."""

_rbac_context = {}
@pecan.expose()
def _lookup(self, meter_name, *remainder):
return MeterController(meter_name), remainder

@wsme_pecan.wsexpose([Meter], [Query])
@rbac_validate.protected('meters')
def get_all(self, q=None):
"""Return all known meters, based on the data recorded so far.

:param q: Filter rules for the meters to be returned.
"""
q = q or [] ...


but we get errors similar to below where the arg parser cannot find the query 
parameter because the decorator doesn't take a q argument as 
MetersController.get_all does.

Is there any way to get a decorator to work within the v2 API code and wsme 
framework or should we consider another approach?  Decorators would really 
simplify the RBAC idea we're working on, which is mostly code-implemented save 
for this fairly major problem.

I have a WIP registered BP on this at 
https://blueprints.launchpad.net/ceilometer/+spec/ready-ceilometer-rbac-keystone-v3.

If I can provide more details I'll be happy to.

Thanks
Eric

  /usr/local/bin/ceilometer-api(10)()
-> sys.exit(api())
  /opt/stack/ceilometer/ceilometer/cli.py(96)api()
-> srv.serve_forever()
  /usr/lib/python2.7/SocketServer.py(227)serve_forever()
-> self._handle_request_noblock()
  /usr/lib/python2.7/SocketServer.py(284)_handle_request_noblock()
-> self.process_request(request, client_address)
  /usr/lib/python2.7/SocketServer.py(310)process_request()
-> self.finish_request(request, client_address)
  /usr/lib/python2.7/SocketServer.py(323)finish_request()
-> self.RequestHandlerClass(request, client_address, self)
  /usr/lib/python2.7/SocketServer.py(638)__init__()
-> self.handle()
  /usr/lib/python2.7/wsgiref/simple_server.py(124)handle()
-> handler.run(self.server.get_app())
  /usr/lib/python2.7/wsgiref/handlers.py(85)run()
-> self.result = application(self.environ, self.start_response)
  
/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py(663)__call__()
-> return self.app(env, start_response)
  /opt/stack/ceilometer/ceilometer/api/app.py(97)__call__()
-> return self.v2(environ, start_response)
  
/usr/local/lib/python2.7/dist-packages/pecan/middleware/static.py(151)__call__()
-> return self.app(environ, start_response)
  
/usr/local/lib/python2.7/dist-packages/pecan/middleware/debug.py(289)__call__()
-> return self.app(environ, start_response)
  
/usr/local/lib/python2.7/dist-packages/pecan/middleware/recursive.py(56)__call__()
-> return self.application(environ, start_response)
  /opt/stack/ceilometer/ceilometer/api/middleware.py(83)__call__()
-> app_iter = self.app(environ, replacement_start_response)
  /usr/local/lib/python2.7/dist-packages/pecan/core.py(750)__call__()
-> return super(Pecan, self).__call__(environ, start_response)
  /usr/local/lib/python2.7/dist-packages/pecan/core.py(616)__call__()
-> self.invoke_controller(controller, args, kwargs, state)
  /usr/local/

[openstack-dev] [Keystone] Removed admin role from admin user/tenant, can't add back

[Pendergrass, Eric]

In an effort to test ceilometer roles I removed the admin role from the
admin tenant and user.  Now I can't add it back since I don't have a
user/tenant combo with the admin role:

 

keystone user-role-add --role e4252b63c308470b8cb7f77c37d27632 --user
8c678720fb5b4e3bb18dee222d7d7933 --tenant 9229d9ffed3d4fe2aa00d7575acd7ada

You are not authorized to perform the requested action: admin_required
(Disable debug mode to suppress these details.) (HTTP 403)

 

Is there a way to do this in the mysql database if I know the
user/tenant/role IDs?  Or, is there another way with keystone client?

 

Thanks,

Eric



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa] Debugging tox tests with pdb?

[Pendergrass, Eric]

Hi, I've read much of the documentation around Openstack tests, tox, and
testr.  All I've found indicates debugging can be done, but only by running
the entire test suite.

 

I'd like the ability to run a single test module with pdb.set_trace()
breakpoints inserted, then step through the test.  I've tried this but it
causes test failures on a test that would otherewise succeed.  The command I
use to run the test is similar to this:  tox -e py27 

 

Is there some way to debug single tests that I haven't found?  If not, how
is everyone doing test development without the ability to debug?

 

Thanks,

Eric Pendergrass



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Error running ceilometer tox tests

[Pendergrass, Eric]

Hi, I pulled devstack yesterday and have been trying to get any test to run
successfully.

 

My process is this:

Install tox >=1.6,<1.7

Install libmysqlclient-dev

Install mongodb-server 

Source .tox/py27/bin/activate

Install test-requuirements packages in venv

Install pytidylib 0.2.1 from tarball

Execute tox -e py27 - api.v2

 

Here is the result (many lines of output omitted):

error: testr failed (3)

+ clean_exit

+ local error_code=1

+ rm -rf /tmp/CEILO-MONGODB-ezBtk

++ jobs -p

+ kill 13439 13463

+ return 1

ERROR: InvocationError: '/bin/bash -x
/opt/stack/ceilometer/setup-test-env.sh python setup.py testr --slowest
--testr-args=api.v2'


 summary

_

ERROR:   py27: commands failed

 

I've seen this work on a copy of devstack I cloned several weeks ago.

 

Does anyone know what's causing the testr failures?

 

Thank you,

Eric Pendergrass



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Add a filter between auth_token and v2

[Pendergrass, Eric]

I need to add an additional layer of authorization between auth_token and
the reporting API.  

 

I know it's as simple as creating a WSGI element and adding it to the
pipeline.  Examining the code I haven't figured out where to begin doing
this.

 

I'm not using Apache and mod_wsgi, just the reporting API and Pecan.

 

Any pointers on where to start and what files control the pipeline would be
a big help.

 

Thanks

Eric



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Compute meter names prefaced by "instance:"

[Pendergrass, Eric]

Hi, I've been out for nearly 3 weeks and noticed Compute meter names are now
prefaced by "instance:"  

 

http://docs.openstack.org/developer/ceilometer/measurements.html

 

Not sure when this happened but I was wondering if the change applies across
all OpenStack.  Will Nova use the change for its events?

 

Also, is the purpose of the change to identify that instance types are
undefined and may vary by installation?

 

Many thanks,

Eric Pendergrass



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Common] context-is-admin causes project_id to not be passed to API layer

[Pendergrass, Eric]

While debugging a token auth problem I noticed that the enforcer searches the 
role list in a token for a role called 'admin' (any case).  If it's present, 
the enforcer returns true and the acl does not set the X-Project-Id header on 
the request.

I was wondering what the reason for not setting project id is in this case.  I 
assume it is a mechanism for privilege scoping for a highly-privileged user.

Also, the name 'admin' seems like a sensible choice to denote an admin user.  
Is there any other meaning behind the role name than this?

Many thanks,
Eric
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Token's project (tenant) not passed to API layer (v2.py) from auth_token.py

[Pendergrass, Eric]

Hi, I'm struggling with a problem related to tokens.  I have one token for
which the project ID gets passed to v2.MeterController.get_all() in the
kwargs:

 

(Pdb) kwargs

{'project': u'10032339952700', 'meter': u'network.outgoing.bytes'}

 

I have another token for a different tenant and the project does not get
passed in:

 

(Pdb) kwargs

{'meter': u'network.outgoing.bytes'}

 

As far as I can tell the token data is roughly the same, except for the
details about the tenant.  I'm stepping through the auth_token code trying
to find the cause.

 

Could someone point me in a direction with some possible causes for why one
token would pass the tenant and another wouldn't?

 

Thank you,

Eric Pendergrass



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Allow multiple projects (tenants) in the received kwargs for v2 queries

[Pendergrass, Eric]

I'm implementing a role based access control system where I'll use a list of
projects to determine which projects a user can query.  This project list
will come from an upstream filter based on roles associated with the user's
auth token.

 

In the current v2/meters/ (Sample) code, the kwargs received are
{'project': u'10032339952700', 'meter': u'network.incoming.bytes'}.  This
indicates one project is received.

 

I'd like to increase the flexibility of some of the v2 API controllers by
allowing a list of projects to be received and subsequently sent to the
storage layer for use in filtering.

 

My questions are:

What do you think about this?

Can the 'project' key change to 'projects' to support this?

 

Any thoughts greatly appreciated.  If this sounds reasonable I'll put out a
blueprint for the change.

 

Thanks

Eric Pendergrass



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer] Disable keystone authorization in ceilometer-api?

[Pendergrass, Eric]

Does anyone know how to disable keystone authorization in ceilometer-api?
Is there a ceilometer.conf option for this?

Thank you


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Ceilometer]How to change Keystone properties

[Pendergrass, Eric]

I've been editing /etc/ceilometer/ceilometer.conf trying to change the
keystone provider:

 

[keystone]

auth_host=

auth_port=35357

auth_protocol=https

auth_uri=https://:35357

admin_user=

admin_password=

 

Tracing throught auth_token.py, I see none of the configuration info is
picked up:

 

257 def __init__(self, app, conf):

258 pdb.set_trace()

259 self.LOG = logging.getLogger(conf.get('log_name',
__name__))

260  ->   self.LOG.info('Starting keystone auth_token middleware')

261 self.conf = conf

262 self.app = app

263 

264 # delay_auth_decision means we still allow
unauthenticated requests

265 # through and we let the downstream service make the
final decision

(Pdb) p conf

{'memcache_secret_key': None, 'auth_protocol': 'https', 'admin_user': None,
'admin_token': None, 'http_handler': None, 'signing_dir':
'/root/keystone-signing', 'auth_uri': None, 'auth_admin_prefix': '',
'memcache_security_strategy': None, 'admin_tenant_name': 'admin',
'auth_port': 35357, 'memcache_servers': None, 'delay_auth_decision': False,
'cache': None, 'auth_version': None, 'token_cache_time': 300,
'admin_password': None, 'auth_host': '127.0.0.1', 'certfile': None,
'http_connect_timeout': None, 'keyfile': None}

 

I assumed the configuration would be within a [keystone] section in
ceilometer.conf.  Have not found much documentation about this.  Am I
missing something?

 

Thanks

Eric



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Ceilometer: Proposal to add resource_metadata to reporting Meter object

[Pendergrass, Eric]

Our group would like to have resource_metadata available on the Meter object
(currently it's not there).  The reason is so we may supply additional
customized attributes associated with a Meter.  Examples include other
timestamp values, a project owner, and processing status.

 

Currently, to get these values we would need to request a  Sample for the
Meter.  This is problematic because we often want to get Meter lists
associated with certain customized attributes, such as the project owner.

 

Changing the Meter spec
(http://docs.openstack.org/developer/ceilometer/webapi/v2.html#Meter) to
include resource_metadata would not require others to use the new field.
Clients could ignore it as they do for Samples.

 

It wouldn't necessarily require any code changes if we documented the field
as not implemented.  The only change it would require is to the Meter table
schema, so that if someone were to supply the field it could be stored.

 

Here is the wishlist item along with discussion.  Any comments appreciated.

 

https://bugs.launchpad.net/ceilometer/+bug/1194593

 

Thank you,

Eric Pendergrass



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev