Re: [Openstack-operators] logging for Keystone on user/project delete/create operations

2015-04-16 Thread gordon chung 
> Standing up Ceilometer (and patching things) just to be able to log 
> this stuff to a file seems rather... heavy handed? We understand that 
> these things are emitted via notifications, but as of right now trying 
> to do anything with those notifications such as simply logging them 
> requires too much additional infrastructure. 

agreed. it's really dependent on what your use case is... just pointing out an 
option to the statement: "hoping that there's a way to consume the notification 
events and then write them to a log". if you're just looking to track 
operations of a single keystone service, then you probably don't want 
notifications. if you're looking to collate them against various other things 
then you'll need some service/tool to either listen to notifications or 
grab/process logs from multiple places.

cheers,
gord


> ___ 
>> From: morgan.fainb...@gmail.com<mailto:morgan.fainb...@gmail.com> 
>> Date: Thu, 16 Apr 2015 07:50:43 -0700 
>> To: dsta...@dstanek.com<mailto:dsta...@dstanek.com> 
>> CC: 
> openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>
>  
>> Subject: Re: [Openstack-operators] logging for Keystone on user/project 
>> delete/create operations 
>> 
>> 
>> 
>> On Apr 16, 2015, at 04:56, David Stanek 
>> 
> mailto:dsta...@dstanek.com><mailto:dsta...@dstanek.com<mailto:dsta...@dstanek.com>>>
>  
> wrote: 
>> 
>> 
>> 
>> On Thu, Apr 16, 2015 at 1:10 AM, Miguel Angel Ajo Pelayo 
>> 
> mailto:mangel...@redhat.com><mailto:mangel...@redhat.com<mailto:mangel...@redhat.com>>>
>  
> wrote: 
>> I’m not involved in the keystone project, but I’d recommend you to 
>> start by filling a blueprint 
>> asking for it, and explaining what you just said here: 
>> 
>> https://blueprints.launchpad.net/keystone 
>> 
>> Adding a blueprint for discussion would be a good idea if you think you 
>> want a change to the project. 
>> 
>> 
>> 
>> I’d also try to contact Keystone PTL (I’m not sure who is the PTL). 
>> 
>> Morgan Fainberg is out PTL. 
>> 
>> 
>> 
>> Best regards, 
>> Miguel Ángel 
>> 
>> On 16/4/2015, at 3:23, Matt Fischer 
>> 
> mailto:m...@mattfischer.com><mailto:m...@mattfischer.com<mailto:m...@mattfischer.com>>>
>  
> wrote: 
>> 
>> I'd like to have some better logging when certain CRUD operations 
>> happen in Keystone, for example, when a project is deleted. I 
>> specifically mean "any" when I say better since right now I'm not 
>> seeing anything even when Verbose is enabled. 
>> 
>> This is pretty frustrating for me because these are rather important 
>> events, certainly more important than my load balancers hitting 
>> Keystone which it's happily logging twice a second. 
>> 
>> I know that Keystone supports some audit event notifications [1]. Can I 
>> simply have these reflect back into the main logs somehow? 
>> 
>> It would be possible (and trivial) to add logging messages at the INFO 
>> level, but I'm not sure that is what you really want. I don't know much 
>> about the operational side at this point, but I'm hoping that there's a 
>> way to consume the notification events and then write them to a log if 
>> that's what you wish to do. 
> 
> Ceilometer listens to these notifications currently and it's possible 
> to write them to a file rather than a database. a lot of this 
> functionality was worked on in Kilo but there may be a way to support 
> this in Juno and Icehouse (disclaimer: may require some patching and 
> even more patching, respectively) 
> 
> cheers, 
> gord 
> ___ 
> OpenStack-operators mailing list 
> OpenStack-operators@lists.openstack.org<mailto:OpenStack-operators@lists.openstack.org>
>  
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators 
> 
> 
> ___ OpenStack-operators 
> mailing list OpenStack-operators@lists.openstack.org 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators 
  
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] logging for Keystone on user/project delete/create operations

2015-04-16 Thread Matt Fischer 
So I've not had time to fully test this, but Steve Martinelli let me know
that you can simply set the notification_driver to "log" and keystone will
log them. The format is a bit more obtuse than I'd like to see and it uses
ids rather than names which means I'd need to do some correlation later.
Here's what I get out of Juno on a user-create:

2015-04-16 15:07:10.137 23939 INFO
oslo.messaging.notification.identity.user.created [-] {"priority": "INFO",
"event_type": "identity.user.created", "timestamp": "2015-04-16
15:07:10.137145", "publisher_id": "identity.dev01-keystone-001", "payload":
{"resource_info": "c79e7f9f64d04ae78b3e069e5cafb9fd"}, "message_id":
"3b1bce1f-8049-4b4f-986f-2ada3d918ee7"}

In the message above the c79e is the ID of the user I just created.

Another downside so far seems to be that its logging too much, including
every time someone authenticates and that it's at INFO level which I hate
because it logs my LB connection. I think the former may be configurable,
the latter is not unless I hack Keystone.

Also in K you can set the format for these either to basic or cadf. I've
pushed a puppet change to allow this to be set via puppet.

This gets me way ahead of where I wanted to be without using either
rabbitmq or ceilometer (which we've gutted from our environment anyway).



On Thu, Apr 16, 2015 at 10:50 AM, Jesse Keating  wrote:

> Standing up Ceilometer (and patching things) just to be able to log this
> stuff to a file seems rather... heavy handed? We understand that these
> things are emitted via notifications, but as of right now trying to do
> anything with those notifications such as simply logging them requires too
> much additional infrastructure.
>
>
> - jlk
>
> On Thu, Apr 16, 2015 at 7:58 AM, gordon chung  wrote:
>
>>
>>
>> ___________________
>> > From: morgan.fainb...@gmail.com
>> > Date: Thu, 16 Apr 2015 07:50:43 -0700
>> > To: dsta...@dstanek.com
>> > CC: openstack-operators@lists.openstack.org
>> > Subject: Re: [Openstack-operators] logging for Keystone on user/project
>> > delete/create operations
>> >
>> >
>> >
>> > On Apr 16, 2015, at 04:56, David Stanek
>> > mailto:dsta...@dstanek.com>> wrote:
>> >
>> >
>> >
>> > On Thu, Apr 16, 2015 at 1:10 AM, Miguel Angel Ajo Pelayo
>> > mailto:mangel...@redhat.com>> wrote:
>> > I’m not involved in the keystone project, but I’d recommend you to
>> > start by filling a blueprint
>> > asking for it, and explaining what you just said here:
>> >
>> > https://blueprints.launchpad.net/keystone
>> >
>> > Adding a blueprint for discussion would be a good idea if you think you
>> > want a change to the project.
>> >
>> >
>> >
>> > I’d also try to contact Keystone PTL (I’m not sure who is the PTL).
>> >
>> > Morgan Fainberg is out PTL.
>> >
>> >
>> >
>> > Best regards,
>> > Miguel Ángel
>> >
>> > On 16/4/2015, at 3:23, Matt Fischer
>> > mailto:m...@mattfischer.com>> wrote:
>> >
>> > I'd like to have some better logging when certain CRUD operations
>> > happen in Keystone, for example, when a project is deleted. I
>> > specifically mean "any" when I say better since right now I'm not
>> > seeing anything even when Verbose is enabled.
>> >
>> > This is pretty frustrating for me because these are rather important
>> > events, certainly more important than my load balancers hitting
>> > Keystone which it's happily logging twice a second.
>> >
>> > I know that Keystone supports some audit event notifications [1]. Can I
>> > simply have these reflect back into the main logs somehow?
>> >
>> > It would be possible (and trivial) to add logging messages at the INFO
>> > level, but I'm not sure that is what you really want. I don't know much
>> > about the operational side at this point, but I'm hoping that there's a
>> > way to consume the notification events and then write them to a log if
>> > that's what you wish to do.
>>
>> Ceilometer listens to these notifications currently and it's possible to
>> write them to a file rather than a database. a lot of this functionality
>> was worked on in Kilo but there may be a way to support this in Juno and
>> Icehouse (disclaimer: may require some patching and even more patching,
>> respectively)
>>
>> cheers,
>> gord
>> ___
>> OpenStack-operators mailing list
>> OpenStack-operators@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] logging for Keystone on user/project delete/create operations

2015-04-16 Thread Jesse Keating 
Standing up Ceilometer (and patching things) just to be able to log this
stuff to a file seems rather... heavy handed? We understand that these
things are emitted via notifications, but as of right now trying to do
anything with those notifications such as simply logging them requires too
much additional infrastructure.


- jlk

On Thu, Apr 16, 2015 at 7:58 AM, gordon chung  wrote:

>
>
> ___
> > From: morgan.fainb...@gmail.com
> > Date: Thu, 16 Apr 2015 07:50:43 -0700
> > To: dsta...@dstanek.com
> > CC: openstack-operators@lists.openstack.org
> > Subject: Re: [Openstack-operators] logging for Keystone on user/project
> > delete/create operations
> >
> >
> >
> > On Apr 16, 2015, at 04:56, David Stanek
> > mailto:dsta...@dstanek.com>> wrote:
> >
> >
> >
> > On Thu, Apr 16, 2015 at 1:10 AM, Miguel Angel Ajo Pelayo
> > mailto:mangel...@redhat.com>> wrote:
> > I’m not involved in the keystone project, but I’d recommend you to
> > start by filling a blueprint
> > asking for it, and explaining what you just said here:
> >
> > https://blueprints.launchpad.net/keystone
> >
> > Adding a blueprint for discussion would be a good idea if you think you
> > want a change to the project.
> >
> >
> >
> > I’d also try to contact Keystone PTL (I’m not sure who is the PTL).
> >
> > Morgan Fainberg is out PTL.
> >
> >
> >
> > Best regards,
> > Miguel Ángel
> >
> > On 16/4/2015, at 3:23, Matt Fischer
> > mailto:m...@mattfischer.com>> wrote:
> >
> > I'd like to have some better logging when certain CRUD operations
> > happen in Keystone, for example, when a project is deleted. I
> > specifically mean "any" when I say better since right now I'm not
> > seeing anything even when Verbose is enabled.
> >
> > This is pretty frustrating for me because these are rather important
> > events, certainly more important than my load balancers hitting
> > Keystone which it's happily logging twice a second.
> >
> > I know that Keystone supports some audit event notifications [1]. Can I
> > simply have these reflect back into the main logs somehow?
> >
> > It would be possible (and trivial) to add logging messages at the INFO
> > level, but I'm not sure that is what you really want. I don't know much
> > about the operational side at this point, but I'm hoping that there's a
> > way to consume the notification events and then write them to a log if
> > that's what you wish to do.
>
> Ceilometer listens to these notifications currently and it's possible to
> write them to a file rather than a database. a lot of this functionality
> was worked on in Kilo but there may be a way to support this in Juno and
> Icehouse (disclaimer: may require some patching and even more patching,
> respectively)
>
> cheers,
> gord
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] logging for Keystone on user/project delete/create operations

2015-04-16 Thread gordon chung 


___
> From: morgan.fainb...@gmail.com 
> Date: Thu, 16 Apr 2015 07:50:43 -0700 
> To: dsta...@dstanek.com 
> CC: openstack-operators@lists.openstack.org 
> Subject: Re: [Openstack-operators] logging for Keystone on user/project 
> delete/create operations 
> 
> 
> 
> On Apr 16, 2015, at 04:56, David Stanek 
> mailto:dsta...@dstanek.com>> wrote: 
> 
> 
> 
> On Thu, Apr 16, 2015 at 1:10 AM, Miguel Angel Ajo Pelayo 
> mailto:mangel...@redhat.com>> wrote: 
> I’m not involved in the keystone project, but I’d recommend you to 
> start by filling a blueprint 
> asking for it, and explaining what you just said here: 
> 
> https://blueprints.launchpad.net/keystone 
> 
> Adding a blueprint for discussion would be a good idea if you think you 
> want a change to the project. 
> 
> 
> 
> I’d also try to contact Keystone PTL (I’m not sure who is the PTL). 
> 
> Morgan Fainberg is out PTL. 
> 
> 
> 
> Best regards, 
> Miguel Ángel 
> 
> On 16/4/2015, at 3:23, Matt Fischer 
> mailto:m...@mattfischer.com>> wrote: 
> 
> I'd like to have some better logging when certain CRUD operations 
> happen in Keystone, for example, when a project is deleted. I 
> specifically mean "any" when I say better since right now I'm not 
> seeing anything even when Verbose is enabled. 
> 
> This is pretty frustrating for me because these are rather important 
> events, certainly more important than my load balancers hitting 
> Keystone which it's happily logging twice a second. 
> 
> I know that Keystone supports some audit event notifications [1]. Can I 
> simply have these reflect back into the main logs somehow? 
> 
> It would be possible (and trivial) to add logging messages at the INFO 
> level, but I'm not sure that is what you really want. I don't know much 
> about the operational side at this point, but I'm hoping that there's a 
> way to consume the notification events and then write them to a log if 
> that's what you wish to do. 

Ceilometer listens to these notifications currently and it's possible to write 
them to a file rather than a database. a lot of this functionality was worked 
on in Kilo but there may be a way to support this in Juno and Icehouse 
(disclaimer: may require some patching and even more patching, respectively)

cheers,
gord  
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] logging for Keystone on user/project delete/create operations

2015-04-16 Thread Morgan Fainberg 


> On Apr 16, 2015, at 04:56, David Stanek  wrote:
> 
> 
> 
>> On Thu, Apr 16, 2015 at 1:10 AM, Miguel Angel Ajo Pelayo 
>>  wrote:
>> I’m not involved in the keystone project, but I’d recommend you to start by 
>> filling a blueprint
>> asking for it, and explaining what you just said here:
>> 
>> https://blueprints.launchpad.net/keystone
> 
> Adding a blueprint for discussion would be a good idea if you think you want 
> a change to the project.  
> 
> 
>> 
>> I’d also try to contact Keystone PTL (I’m not sure who is the PTL).
> 
> Morgan Fainberg is out PTL.
> 
>  
>> 
>> Best regards,
>> Miguel Ángel
>> 
>>> On 16/4/2015, at 3:23, Matt Fischer  wrote:
>>> 
>>> I'd like to have some better logging when certain CRUD operations happen in 
>>> Keystone, for example, when a project is deleted. I specifically mean "any" 
>>> when I say better since right now I'm not seeing anything even when Verbose 
>>> is enabled. 
>>> 
>>> This is pretty frustrating for me because these are rather important 
>>> events, certainly more important than my load balancers hitting Keystone 
>>> which it's happily logging twice a second.
>>> 
>>> I know that Keystone supports some audit event notifications [1]. Can I 
>>> simply have these reflect back into the main logs somehow?
> 
> It would be possible (and trivial) to add logging messages at the INFO level, 
> but I'm not sure that is what you really want. I don't know much about the 
> operational side at this point, but I'm hoping that there's a way to consume 
> the notification events and then write them to a log if that's what you wish 
> to do. 
> 

It wouldn't hurt us to see logging expanded upon. As long as the logging 
conforms to the cross-project logging spec[1].


[1] 
https://github.com/openstack/openstack-specs/blob/master/specs/log-guidelines.rst

--Morgan

Sent via mobile___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] logging for Keystone on user/project delete/create operations

2015-04-16 Thread David Stanek 
On Thu, Apr 16, 2015 at 1:10 AM, Miguel Angel Ajo Pelayo <
mangel...@redhat.com> wrote:

> I’m not involved in the keystone project, but I’d recommend you to start
> by filling a blueprint
> asking for it, and explaining what you just said here:
>
> https://blueprints.launchpad.net/keystone
>

Adding a blueprint for discussion would be a good idea if you think you
want a change to the project.



> I’d also try to contact Keystone PTL (I’m not sure who is the PTL).
>

Morgan Fainberg is out PTL.



>
> Best regards,
> Miguel Ángel
>
> On 16/4/2015, at 3:23, Matt Fischer  wrote:
>
> I'd like to have some better logging when certain CRUD operations happen
> in Keystone, for example, when a project is deleted. I specifically mean
> "any" when I say better since right now I'm not seeing anything even when
> Verbose is enabled.
>
> This is pretty frustrating for me because these are rather important
> events, certainly more important than my load balancers hitting Keystone
> which it's happily logging twice a second.
>
> I know that Keystone supports some audit event notifications [1]. Can I
> simply have these reflect back into the main logs somehow?
>
>
It would be possible (and trivial) to add logging messages at the INFO
level, but I'm not sure that is what you really want. I don't know much
about the operational side at this point, but I'm hoping that there's a way
to consume the notification events and then write them to a log if that's
what you wish to do.


>
> [1] -
> http://docs.openstack.org/developer/keystone/event_notifications.html
>
>
-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] logging for Keystone on user/project delete/create operations

2015-04-15 Thread Miguel Angel Ajo Pelayo 
I’m not involved in the keystone project, but I’d recommend you to start by 
filling a blueprint
asking for it, and explaining what you just said here:

https://blueprints.launchpad.net/keystone 


I’d also try to contact Keystone PTL (I’m not sure who is the PTL).

Best regards,
Miguel Ángel

> On 16/4/2015, at 3:23, Matt Fischer  wrote:
> 
> I'd like to have some better logging when certain CRUD operations happen in 
> Keystone, for example, when a project is deleted. I specifically mean "any" 
> when I say better since right now I'm not seeing anything even when Verbose 
> is enabled. 
> 
> This is pretty frustrating for me because these are rather important events, 
> certainly more important than my load balancers hitting Keystone which it's 
> happily logging twice a second.
> 
> I know that Keystone supports some audit event notifications [1]. Can I 
> simply have these reflect back into the main logs somehow?
> 
> 
> [1] - http://docs.openstack.org/developer/keystone/event_notifications.html 
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Miguel Angel Ajo



___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators