----- Original Message -----
> From: "Trevor Vaughan" <[email protected]>
> To: "puppet-dev" <[email protected]>
> Sent: Wednesday, 2 March, 2016 11:50:27
> Subject: Re: [Puppet-dev] metaparam question
> Hi all,
>
> This is a very interesting conversation and I'm monitoring to see where it
> ends up.
>
> You're going to want to have a central location for organizing this
> information between nodes since there are no guarantees that any two nodes
> can get bi-directional communications between each other.
>
> Take, for instance, a highly regulated HIPAA environment. You may be able
> to send data one way between two managed environments but bi-directional
> communication would be unable to be processed. Therefore, you would need
> some level of controlled and shared trigger between your host catalogs.
> (OK, fine, I still want language-level semaphores).
quite, that's what the puppet orchastrator and node catalogs will do eventually.
In the mean time as a building block what I am proposing:
service{"your_app":
ensure => "running",
validate => "check_my_app -H localhost -p 80"
}
Might connect to a local web app that was configured with its DB details and it
might hit something like /status which might return:
{"database": "ok", "health": "ok", "metrics": {"requests": 1234,
"avg_response_time": 0.2}}
and so you know the service up and happy and working much more than just a:
service you_app start
would know. At a deeper level additional per resource reporting could include
things like this additional status and using the nagios metric protocol you
could
communicate the metrics requests and avg_response_time back to Puppet for
surfacing
in the various consoles.
I'd as a side effect feed that into let say sensu as well so your other
monitoring
will automagically become aware of this resoure and its status.
>
> Trevor
>
> On Wed, Mar 2, 2016 at 4:43 AM, R.I.Pienaar <[email protected]> wrote:
>
>>
>>
>> ----- Original Message -----
>> > From: "Kylo Ginsberg" <[email protected]>
>> > To: "puppet-dev" <[email protected]>
>> > Sent: Wednesday, 2 March, 2016 06:59:08
>> > Subject: Re: [Puppet-dev] metaparam question
>>
>> > On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar <[email protected]> wrote:
>> >
>> >> hello,
>> >>
>> >> Thanks for the replies, think I'll reply to both in one go
>> >>
>> >> ----- Original Message -----
>> >> > From: "Luke Kanies" <[email protected]>
>> >> > To: "puppet-dev" <[email protected]>
>> >> > Sent: Friday, February 5, 2016 6:15:28 AM
>> >> > Subject: Re: [Puppet-dev] metaparam question
>> >>
>> >> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg <[email protected]>
>> wrote:
>> >> >>
>> >> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar <[email protected]> wrote:
>> >> >>> hello,
>> >> >>>
>> >> >>> I would like to add a metaparameter - which I think is easy now via
>> >> >>> Type.newmetaparam.
>> >> >>
>> >> >> We haven't been thinking of metaparameters as a general purpose
>> >> extension point.
>> >> >> This came up once before that I know of, about a year ago, and
>> there's
>> >> a little
>> >> >> discussion of this in https://tickets.puppetlabs.com/browse/PUP-4281
>> .
>> >> The
>> >> >> conclusion we reached at the time was, more or less, to explore
>> whether
>> >> the
>> >> >> desired change could be accomplished with a puppet function and/or a
>> >> change to
>> >> >> core puppet.
>> >> >
>> >> > I don't remember my original logic for metaparams not being an
>> extension
>> >> point.
>> >> > Given the system's support for easily loading this kind of plugin, it
>> >> should be
>> >> > easy to add.
>> >> >
>> >> > The discussion on the ticket is pretty limited. We've got multiple use
>> >> cases
>> >> > here, and I have seen use cases in the past brought up, but they were
>> >> mostly on
>> >> > lists or in person, so I don't think they made it into the ticketing
>> >> system.
>> >> >
>> >> > RI - you up for a pull request to add the autoloading? Have you
>> thought
>> >> much
>> >> > about how this might go wrong?
>> >>
>> >> Unfortunately my time is pretty limited and I'll prefer to see what I
>> can
>> >> do
>> >> with this information in the catalog rather than making a merge worthy
>> >> patch
>> >> at present.
>> >>
>> >> I'm happy to just patch my actual puppet code with no modules to get to
>> >> that
>> >> point, I'll have a look at the validate_cmd in file to see if there is a
>> >> way
>> >> in the provider base class to do it so all providers get it.
>> >>
>> >
>> > I kept meaning to get back to this thread and it slipped my mind. Mea
>> culpa
>> > on that.
>> >
>> > I'm curious where you got with this approach? E.g. did you end up
>> patching
>> > an additional metaparameter and were there any lessons worth sharing in
>> > that?
>>
>> Unfortunately work is a bit busy right now so I've not had time to really
>> give it
>> any screen time :(
>>
>> >> For me this being in every provider is a nice to have I guess - it
>> would be
>> >> great to be able to improve puppets ability to check if something worked
>> >> in the broad sense, and it's going to apply to all resource types
>> equally.
>> >>
>> >
>> >> I could take the route that https://github.com/nanliu/puppet-transport
>> >> does
>> >> and instead of making the meta param do something use it just to store
>> this
>> >> information and then some other module do something with this
>> information
>> >> and
>> >> that would let me play with what I want, but it seems a small step to
>> >> improve
>> >> puppet in general.
>> >>
>> >
>> > I'm intrigued by the idea of a generic 'validate' or 'health check'. It
>> > sounds like you're thinking of it as separate from the rest of the
>> > type/provider, i.e. an external command that could be run, but I also
>> > wonder (this is half-baked) if, rather than a separate metaparameter, it
>> > could use a standard provider method to do the validation. I.e. when
>> puppet
>> > is applying the catalog it has to essentially ask the same question that
>> a
>> > 'validate' method would to figure out if it needs to remediate, so can
>> the
>> > same need be met with some refinement of the generic type/provider api?
>>
>> As a POC yes, I'd like it external - there's a ton of Nagios plugins out
>> there
>> and they would magically all be useful without any extra work, with the
>> time
>> I have minimising the yaks have value :)
>>
>> But there are common patterns to these things, so for sure you could mark
>> up a module/resource with some additional properties or allow provider
>> authors
>> to do better than we do today - but then they more or less can already
>> code their
>> modules to do in depth health tests if they wanted. Probably the
>> type/provider
>> API needs some love in general and if that was something you'd make front
>> and
>> centre it would be great. I imagine puppet might get some built in
>> 'checks'
>> like this, say a generic built in TCP checker that we can reference here
>> instead
>> of calling out to a nagios plugin (which you then have to manage), and
>> that would
>> be pretty neat and a good place to add pretty good deep introspection value
>> for module authors. Putting some nasty CLI call with a bunch of arguments
>> hard coded into a resource is not elegant at all - like nagios based check
>> validation would cause, that would not be my final destination but a
>> reasonable
>> stop along the way.
>>
>> The meta parameter though has value - I see this to a large extend as user
>> serviceable. I get a module from the forge that has some provider, for me
>> a 'responds well' check might be < 10ms for others it might be < 100ms and
>> for others it might not even be a time based expression.. And the
>> parameters
>> that makes up what is good - here I used response time - are actually very
>> varied
>> so no doubt some space for getting this in a magical way for now a meta
>> param
>> seems a easy thing to reason about for now while I look at how this works
>> and
>> how feasible it is.
>>
>> Another approach might be to introduce a more formal observer pattern into
>> the
>> system, I've toyed with this a few times and there's a interesting example
>> over
>> at https://forge.puppetlabs.com/ryanuber/tell though it's not what I want
>> but
>> as a thing to look at and think about it's quite interesting. My thoughts
>> are
>> not really well formed around it, but I did built a network wide observer
>> for
>> puppet events and could do pretty crazy things with it, not sure they were
>> good ideas though.
>>
>>
>> >
>> > (And btw, thanks for the pointer to puppet-transport. I looked at that
>> long
>> > ago and had forgotten (or more likely didn't internalize) the use there
>> of
>> > a do-nothing metaparameter to store connection information.)
>> >
>> >
>> >> I've told many people this story so no doubt you remember the context
>> Luke
>> >> but what I'd like to do is allow people to attach these checks to
>> resources
>> >> and of course use them in catalog apply stage to get a deeper sense of
>> >> 'done'
>> >> in the same way that the File validate_cmd does but later I want to
>> consume
>> >> the catalog in other tools and extract the same - or rather hopefully
>> speak
>> >> to some API and get this.
>> >>
>> >> Imagine I had a bridge to Sensu where it loads the catalog and just
>> checks
>> >> everything that has a validate attached for free with no configuration
>> >> required,
>> >> this is actually really easy with sensu as it doesnt require you to
>> declare
>> >> the checks upfront on the server and all you have to do is write JSON
>> to a
>> >> socket for every check, it'll promiscuously take them and show them and
>> do
>> >> alerts etc.
>> >>
>> >> Once you have this information and can surface it additionally into the
>> >> Puppet
>> >> eco system you can gather the state regularly like a monitoring system
>> does
>> >> and it becomes quite interesting later down the line when perhaps you
>> can
>> >> use this to influence the node assignment logic in the orchastrator so
>> that
>> >> a Web app will be configured to talk to a known good DB server - or that
>> >> you can detect that the web app should be reconfigured once it's chosen
>> >> DB isn't good anymore. That's quite far down the line though, I think
>> just
>> >> being able to do the monitoring part into sensu would already be a huge
>> win
>> >> for many.
>> >>
>> >
>> > I haven't heard this story before but it's super interesting. Since you
>> > mention the orchestrator and you're thinking about monitoring/validation
>> to
>> > gate or to assign connections between nodes, have you seen
>> >
>> http://docs.puppetlabs.com/pe/latest/app_orchestration_availability_tests.html#using-availability-test
>> ?
>> > Availability tests as described there are just a start but they seem like
>> > they're in the neighborhood of what you're describing above.
>>
>> Yes, I saw those after Eric mentioned it to me at the contributor summit,
>> they
>> are interesting for sure and along the lines of what I mean above about
>> the generic
>> patterns in checks and module authors bringing their own built in ones.
>> Though
>> you'll find few/no successful monitoring systems that told people to
>> rewrite every
>> check from fresh when thousands of nagios ones exist. The nagios layer
>> could well
>> be one of these checks though.
>>
>> > The added idea here of integrating such a thing, be it availability test
>> or
>> > validate metaparameter, with a monitoring system could be really
>> powerful.
>>
>> yes in as much as monitoring isn't what monitoring is about. It's about
>> the data
>> thats the side effect of monitoring and what you can do with it. I think
>> Puppet
>> is fairly uniquely situated to do things with that data very few others
>> can do.
>>
>> > Wrt catalog format (or perhaps report format?) it would be interesting to
>> > explore whether we could adapt the format to facilitate such
>> integrations.
>> > Currently both catalog format and report format are more serializations
>> of
>> > internal data structures than they are designed as external integration
>> > points. I'd have to noodle on this some more, but it's definitely a
>> course
>> > worth exploring.
>>
>> Yeah, I've done nasty little things to extract information from catalogs
>> and
>> they all broke recently with 4 :P Public APIs around interacting with it
>> would be good. A library to do so would be fantastic
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/puppet-dev/454494855.237167.1456911800479.JavaMail.zimbra%40devco.net
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Trevor Vaughan
> Vice President, Onyx Point, Inc
> (410) 541-6699
>
> -- This account not approved for unencrypted proprietary information --
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email
> to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoXjF%3DTh1o5vJBU9hW%3DHhmxNDcDf5CHAkc-TUM9HF4hxDA%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/435304898.284658.1456916431327.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/d/optout.
