On Aug 28, 2010, at 3:44 AM, Daniel Pittman wrote:
> Luke Kanies <[email protected]> writes:
>
> G'day. Having finally gotten free of some crash-priority engineering I have a
> chance to look this over.
>
>> Rein, Paul, and I had a call today discussing whether we should produce a
>> 1.6 (I said no, unless there are high priority tickets that really need to
>> be worked on), and then what the design goals of 2.0 should be. I took
>> notes on our discussion and atempted to produce a doc capturing it all:
>>
>> http://projects.puppetlabs.com/projects/facter/wiki/ArchitectureForTwoDotOh
>>
>> Comments appreciated.
>
> It looks pretty good to me, and the subsequent discussion has clarified some
> of the bits I was uncertain about when it came to our internal use of the
> facts.
>
> From my PoV one of the big gains would be making adding a new fact more like
> writing a munin plugin[1] than it currently is, although it is fairly simple
> and direct right now.
Yeah, I want to make this easier, but especially easier for sysadmins to do in
a way they're familiar with, which usually means not writing ruby and not
knowing special Facterisms.
> The main thing missing from this documentation, and which has bitten us in
> practice, is a lack of "community standards" for how facts should be
> presented.
>
> For example, we have a "mem_in_mb" fact to work around the human-focused
> values being returned from the default memory fact, or the difficulty in
> returning a boolean fact to puppet. (0 and Ruby false are both "true",
> apparently. :)
Hmm. Yeah, this looks to be missing. What's the best way to fix that?
> I would also be very happy to see more explicitness than is mentioned here
> about what sort of data types Facter handles: It sounds like y'all are
> thinking of something akin to JSON-level "rich" data structures, which I would
> be very happy with, rather than YAML-with-Ruby-classes "rich" data structures.
> (...or even plain "any Ruby object is fine" results. :)
Yeah - this is definitely raw data, not ruby objects. We already support
providing Facter output as a YAML hash, but we're going to support hashes of
hashes of hashes of arrays of... You get the idea. And it'll all be in
multiple formats.
> WRT the point about grouping of facts, and resolution: to my mind, this would
> be a nice place to use a qualified name, and a search path:
>
> com.puppetlabs.memory
> net.rimspace.memory
>
> facter search path: rimspace.net, puppetlabs.com
>
> That would allow for qualified and unqualified facts to be used, and
> appropriate searching through them; mostly, I would imagine this being
> centrally configured in whatever tool was going to reference them.
>
> That same hierarchy lends itself, SNMP-like, to grouping facts as leaf nodes
> in a tree of naming.
>
> com.puppetlabs.ipaddress.eth0 => 192.168.1.1
> com.puppetlabs.ipaddress.eth2 => [172.16.23.1, 172.16.24.1]
Hmm. I've never thought of this. I've always kind of hated the reverse-domain
style naming, but it's certainly common, and my opinions have often been wrong
on this. Anyone else desirous of this?
It wouldn't translate all that well to Puppet variables -
$com::puppetlabs::ipaddress::eth0?
I think it would also make overrides and things a bit complicated - is a fact
in com.puppetlabs a fact that we shipped the code for, or a fact about a Puppet
Labs project? Should a Solaris IP address have a different path than a Red Hat
IP? What if you override the default IP fact with a custom one?
However, I can totally see naming the resolution mechanisms that way - we need
a unique way of naming and finding resolutions, both for logging and testing,
and this is probably a good option. That way we can track where the code is
from (i.e., com.puppetlabs fact resolutions are part of the core).
> Finally, the new DSL as proposed in the ticket looks good to me. It removes
> much of the boilerplate code from the system. The one additional thing that
> would be convenient to know would be what assurances of namespace and
> execution I had as a developer:
>
> If I use a global variable, am I going to trample anyone else?
Probably. :)
> When, and how often, will my code be run?
Yeah, this should be clearer.
> I am thinking of facts like this one:
>
> Facter.add("grub2") do
> installed = %x{complex and costly command}.match(/installed/)
> setcode { installed ? 'true' : '' }
> end
>
> Knowing when, and how often, that costly command runs would be good, and
> having that promise as part of the specification for building facts would make
> it easier to be comfortable.
This is why we're adding support for TTL and back-end data collectors. As
discussed, it's unlikely for Facter to cache this data, but the caller (usually
Puppet) certainly could.
> The namespace part is more important when using the DSL that doesn't make the
> blocks (and, so, variable scope) so obvious.
>
> [...]
>
> Luke Kanies <[email protected]> writes:
>> On Aug 26, 2010, at 11:27 AM, Nigel Kersten wrote:
>>> On Wed, Aug 25, 2010 at 10:09 PM, Luke Kanies <[email protected]> wrote:
>>>> On Aug 25, 2010, at 1:53 PM, Nigel Kersten wrote:
>>>>> On Wed, Aug 25, 2010 at 11:17 AM, Luke Kanies <[email protected]> wrote:
>
> [...]
>
>>>>> I have a few thoughts churning around about Facter having native
>>>>> support for storing fact evaluation history on the client, which ties
>>>>> into the open feature request for caching fact values, and I noticed
>>>>> you reference a ttl option in #4565.
>>>>
>>>> It'd be great to hear these.
>>>
>>> So I feel that it would be immensely useful for Facter to optionally store
>>> a certain amount of historical data about the fact evaluation.
>>>
>>> It would be great to be able to simply interrogate info like "when did the
>>> amount of RAM in this machine change?" "what is my kernel version history?"
>>> etc etc.
>>>
>>> To get there however, we need a persistent store for facts, which seems to
>>> tie in quite nicely to the idea of having certain facts be cached, and
>>> easily marked as "refresh once per boot" etc.
>
> FWIW, I would be worried about any fact that was "refresh once per boot": an
> awful lot of things can change dynamically, including hostname, memory
> capacity, disk capacity, number of CPUs, and a bunch of other frequently
> static things about a host.
>
> (In fact, much of the engineering we did was to give us more capacity to
> dynamically change many of those aspects, and part of it done by renaming
> hosts as we rebuild them on the fly. Gotta love emergencies.)
I have the same worry, but we need some ability to store fact data over time.
>>> Facter becomes much more useful as a standalone product with these
>>> capabilities, and ideally we could hook Puppet/Puppet Dashboard into this
>>> to store historical fact data. We could do this at the Puppet/Dashboard
>>> layer, but if we decided to accept the feature request for caching fact
>>> evaluation, then it appears to make more sense to have Facter support this
>>> directly.
>>
>> Interesting. So in this scenario, Facter would develop a decent bit of its
>> own functionality - maybe not a daemon, but at least long-term storage.
>
> I think that like R. I. Pienaar I would lean towards making this a third tool,
> which used facter but was not part of it. This sort of trending and
> historical analysis is a very different use of the system than configuration
> management.
Yep.
>> Would it be acceptable if, say, the puppet agent provided a simple interface
>> to the server-side fact storage, which will already have this? We're
>> working on designing something like this right now, although it's more
>> mental goo than real ideas right now.
>
> To me, some sort of API between fact storage and third party applications
> would be desirable, but whatever form it took would be pretty much fine.
>
> We don't so much care about talking to facter, as obtaining the information
> that is stored in a specific fact, possibly from a specific node.
>
>
> The only other capability that would be interesting would be to be able to
> dynamically query facts from other nodes: at the moment we use mcollective to
> query facter facts dynamically on a manual basis, but anything we do manually
> is usually a pointer to something that we want to automate eventually...
We'll be having a central inventory (fact) store very soon, which will give you
this, but can you provide more about what you do with it? How concerned are
you about security? I've been really hesitant to build a system that allows
anyone to read and write anyone else's facts.
--
The surest sign that intelligent life exists elsewhere in the universe
is that it has never tried to contact us.
--Calvin and Hobbes (Bill Watterson)
---------------------------------------------------------------------
Luke Kanies -|- http://puppetlabs.com -|- +1(615)594-8199
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en.
