On May 28, 2009, at 7:51 AM, Bart Vanbrabant wrote:
>
>>> Conflict handling
>>> -----------------
>>>
>
>> This is really interesting. Still trying to absorb the implications,
>> so I can't really make a real assessment at this point.
>>
>> Other than the sorting handler for doing file concatenation, what
>> useful conflict handlers do you see existing?
>>
>> I've long been wanting a builtin mechanism for providing what this
>> sorting handler provides -- the ability to specify file snippets in
>> the language and have them turned into files automatically on the
>> client -- so this is of special interest to me. I'm trying to see
>> what other useful handlers might exist that would justify this
>> being a
>> general service rather than a specific solution for file snippets.
>>
>> This makes me think of the work I'd like to do around making it
>> easier
>> for native types to generate other native resources -- e.g.,
>> something
>> like what you have but that retained the semantically rich logging,
>> and having the host -> file conversion take place on the client
>> rather
>> than on the server.
>
> At the moment the main use would be replacing the file concatenation
> hack. Because I wanted to be able to write puppet modules at higher
> abstraction levels I used this quite a lot instead of writing native
> resources. After this work I started a PhD and I am still working on
> these kind of things. One of the things I am pretty convinced of is
> that file editing is evil. This are not only the line edit definitions
> but also all editing native resources that exist in puppet. The only
> sane way to use them is by setting purge to true.
I've long said that file editing was the wrong approach.
>
> Why do I think they are evil? When you edit files you are only sure
> that what is defined in your manifests is in the file. You can not be
> sure about anything else in the file (unless purge is set to true).
> When you do edit files you lose a huge advantage of the declarative
> nature of Puppet: you can not be sure you can go from an undefined
> state to the correct state in described in your manifests. In the
> Puppet training this week Teyo said as a best practice, he tries to
> make sure you can correctly configure starting from a greenfield state
> or the previous state. This is something that should not be required
> if your system is truly declarative which is not possible when you
> manage resources partially. I know that bad initscripts or bad
> packages can do this as well but in that case it is a bug that needs
> to be fixed.
I see here that you go further than I.
I would generally say that having an abstraction (e.g., a host type)
over a system that edits a file that's composed of records is not
evil, but attempting to force an abstraction over a file that's *not*
composed of records is evil.
In other words, I don't have much problem doing file editing if 1) the
edited file is composed if essentially independent records and 2) the
editing is via an abstraction such as a resource type, rather than in
a way that encodes both the semantics and the file format at the same
level.
>
> At the moment I see some other use cases for a conflicthandler as
> well. When Teyo explained virtual resources he gave a good use for
> them when you need to define a cronjob for a backup that is used in
> more than one class or when you need a user in more than one class.
> You make it virtual and you realize it in the places where you need
> the user. If there are none, the user is not used, if there is more
> than one you get no duplicate resource exception. You could create a
> conflicthandler that does not raise an exception if conflicting
> resources with the same parameters are defined. Because in a true
> declarative specification, if you specify things more than once but
> they do not conflict, it does not need to be a problem.
I can see how you would think the conflicthandlers are easier than
using virtual resources, and they might be.
Certainly in some ways he virtual resources are "simpler" because they
force you to have that resource defined in only one place, and you can
always find that one place.
However, they do force each manifest developer to be more cognizant of
which resources they're managing are used by other parts of the
configuration, which isn't as simple.
As to how this would behave in a truly declarative system, it's hard
to say - a language being declarative doesn't automatically determine
all of its semantics, and Puppet's concept of a list of unique
resources isn't exactly common among declarative systems.
>
>>> example usage
>>> +++++++++++++
>>>
>>> # A host entry in /etc/hosts. It replaces the native resource and is
>>> functional
>>> # equivalent to the native host type if Host { purge => true } is
>>> set.
>>
>> There is one crucial difference between this and native Host support:
>> Logging is semantically rich with the native type (e.g., 'changed ip
>> address from X to Y') but not with the file ('changed contents from
>> <hash> to <hash>'). You might make 50 changes in a hosts file in one
>> transaction and it would show up as a single change.
>>
>> Not saying this kills the feature, just that it does make a
>> difference, and that difference is one of the reasons I frequently
>> espouse native types.
>
> I do not really agree with this. This information is also available in
> your specification repository. Unauthorized manual changes to a
> resources are not logged this way, but I can not think of a reason why
> you would want that, except for detection but you can do this with the
> md5sums as well.
The resulting configuration (the hosts and their parameters) is in
your specification repository, but not the means by which you got there.
My point is that logging for this kind of file editing is almost
entirely lacking in semantics, and even worse, the few semantics you
do have (the file name) are at the wrong abstraction level.
I don't even want to know where hosts are stored (ldap, netinfo, /etc/
hosts, xml), much less how they're stored. But if I use this
conflicthandler to make my changes and then there's a problem, I have
to track down logs for the specific file the changes happened in,
rather than being able to search for, say, logs that say that Host foo
had its ip address changed.
And if I come in in the morning to find an outage, and I determine
that it's a problem with hosts but that there were multiple changes
to /etc/hosts during the night, it's now much harder for me to track
down the source of the outage.
>
>>> Todo
>>> ++++
>>>
>>> - add a better api for writing conflicthandlers, especially to work
>>> with the
>>> resources
>>> - only allow conflicthandlers on native resources or make them
>>> work on
>>> non-native resources. But this can only work IF the conflicting
>>> define does
>>> NOT generate any new conflicting defines for the same resource.
>
>> I would think that the handlers would work on defined resources as
>> well as native resources, as long as they were triggered at the right
>> time. The complexity would be making sure that that happened -- you
>> could quite easily have conflicting native resources at drastically
>> different depths in the configuration, so it'd be very difficult to
>> have both of them in their final state at the same time without one
>> of
>> them having already been evaluated.
>
> This is somewhat related to the comments you made on the patch itself.
> I 'catch' the conflicts when they are added to the catalog. From what
> I remember at that time I can be sure that the resource is conflicting
> with an already added resource. A conflicthandler is added to the
> resource that is already in the catalog. All conflicting resources are
> then added to that conflicthandler. After all evaluation is finished,
> all conflicthandlers are invoked to do there thing. This also means
> that a conflicthandler is limited to what it can do, because all
> evaluation has already been done.
I see, I missed this -- all conflicthandlers are invoked at the very
end of the run, after all resource evaluation happens. Clever.
--
Every generation laughs at the old fashions, but follows religiously
the new. -- Henry David Thoreau
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
