Re: [Puppet Users] Re: Thoughts about extlookup: http://blog.wl0.org/2011/05/thoughts-about-extlookup-in-puppet/
john.bollin...@stjude.org (jcbollinger) writes:
On May 14, 8:52 am, Simon J Mudd sjm...@pobox.com wrote:
However, from the discussion a few things strike me:
1. the use of parameterised classes is recommended heavily. I've just
found out about this new feature inspite of using puppet for a long
time. Hence many recipes that I'm using are not paremeterised and I
have a large number of similar classes. This is certainly worth fixing
but is quite a painful transition to make given the number of systems
in use and the chance of creating heavy breakage during that change.
So if you haven't used parameterised classes much that's the first
thing that needs looking at.
If you have a complex set of manifests that do not use parameterized
classes, then I think it much safer to coalesce similar classes with
the help of extlookup() than by attempting to switch to parameterized
classes.
Perhaps. To some extend my non-parameterised classes are _very_ similar
in many ways except for various parameters (creation of logical volumes
and filesystems, version of mysql to use, creation of certain cron jobs
for importing informtation into a configuration database, and other crons
for doing standard jobs, etc) Most of the completely common stuff
has been taken out. However, using extlookup() doesn't really work for
me right now for a simple reason I haven't yet mentioned. I'd further
like to have a lookup_by_key() function [let's say similar to the extlookup()
one provided now] but also a lookup_by_regexp() function where I can match
groups of boxes, in my case mainly based on regexps of their hostnames
rather than on specific $hostname, or $domain, etc Without that
I need to add more entries to match by lots of hostnames, which doesn't
work.
So I'd like something like
$mount_point = lookup_by_regexp( 'mount_point', $hostname, 'regexp_file', '' )
# parameters: lookup_item, lookup_value, regexp file to lookup, default
$lvsize = lookup_by_regexp( 'lvsize', $hostname, 'regexp_file', '' )
class db_lvconfig { db_lvconfig: mount_point = $mount_point, lvsize = $lvsize
} # note: other default parameters not shown like: owner, group, mode, vgname,
lvname
and have in the .csv file something like
mount_point,/dbservera/,/mount_point/for/servera
mount_point,/dbserverb/,/mount_point/for/serverb
mount_point,/dbserverc/,/mount_point/for/serverc
lvsize,/dbservera/,100G
lvsize,/dbserverb/,200G
lvsize,/dbserverc/,300G
Then if $hostname = 'dbservera-001'
it would get $lvsize = '100G', and $mount_point = '/mount_point/for/servera'
If I have 10 dbservera-xxx boxes then this keeps the config simple and the
.csv files much smaller.
There are a couple of important ways (other than
parameterization itself) by which parameterized classes differ from
non-parameterized ones, and the more complex your current manifests,
the more likely these are to bite you if you parameterize existing
classes.
Perhaps. for most of my cases I don't think that's the case. I'm
slowly adjusting the configuration of several classes to use
parameterised classes but even these still work nicely with
extlookup() alternatives such as the currently unimplemented
lookup_by_key() or lookup_by_regexp() alternatives I'm thinking of
using.
Note, however, that it is the use of extlookup() *as an alternative to
class parameterization* that the style guide recommends against. Do
not take it as a blanket recommendation against using the feature.
I'm not. You need to have a very considered mindset to build
_everything_ top down using an ENC and classes or globalvariables. If
using global variables as heavily as puppet seem to imply then ensuing
that you have a clear naming policy for this is quite important to
avoid unexpected name clashes (or mistakes) which might go unnoticed.
So I'd certainly like to see real world examples of complete
configurations to be fully convinced this works well. Perhaps after
moving a configuration to use extlookup() type mechanisms it then
becomes easier to make a further step up to get to the ENCs style
Puppetlabs reccommends. However, with system configuration being a
continual ongoing process I tend to think that a 100% ENC-only style
build using only parameterised classes is going to be a hard thing to
achieve in practice. And the global variables just don't convince me
yet. Perhaps I still have to see the light.
Note also that another part of the Style Guide's approach is that
classes should avoid including other classes. Although it may not be
immediately obvious, that goes hand-in-hand with extensive use of
parameterized classes. Do take that into consideration as you
evaluate the Style Guide's recommendations.
Hm.. well building bigger blocks from smaller ones tends to be a good
practice to follow, so this recommendation seems counterintuitive.
I'll have to lookup the section and see if they explain why, but guess
it's because the intention is you build your server
Re: [Puppet Users] Re: Thoughts about extlookup: http://blog.wl0.org/2011/05/thoughts-about-extlookup-in-puppet/
Hi John, john.bollin...@stjude.org (jcbollinger) writes: ... Let's not cast things in terms of correctness, except insomuch as whether they reliably produce the desired effect on clients. Indeed. As always there is more than one way to solve a problem. Depending on what you're trying to do with Puppet, however, it is certainly true that some approaches to structuring your manifests will make them easier to write and maintain than will others. Personally, when writing specializations into my manifests, I find it useful to keep in mind the question of *why* a particular host or group of hosts is different from others. I have yet to run into an answer that didn't fall into one of these categories, or a combination of them: 1) It has a functional role that some other systems do not In my case 90% of the differences are functional differences, often expressed as part of the hostname. 2) Its network environment requires differences from some other systems. There may be some minor differences here but they are the exceptions rather than the rule. 3) It's just special I have a few of these too. It has worked well for me so far to model roles via (unparameterized) classes, assigned to nodes via their node declarations. That leaves only one level between common and node-specific where I might need to customize data. Yes, I have db_server::type_a and db_server::type_b, ... (That intermediate level could in principal be parameterized by network domain, but in my case it is by subnet.) Most of my nodes do not require per-node customization, so I don't end up with many data files for extlookup. In my case one db_server is pretty much similar to another one. Things vary such as mysql version to use (normally pretty constant), partitions to mount and their location(s), cron jobs to setup. All of these are lower level classes which are almost the same except for a few parameters. I also started with non-parameterised classes so created a large number of nearly duplicate classes and now I'm beginning to parameterise some of these the total number of classes should drop much more easily. Of course if I can lookup the parameters from a data source (extlookup or similar) then I can pretty much make many classes identical: all they do is pickup the required parameters and apply them. Moreover, I disagree with several of the opinions and conclusions in your post: 1) You write: The extlookup() functionality only allows [... specifying implicitly ...] where to look for this value. That is false. Extlookup does provide for configuration of a standard set of CSV files to search (which can be parameterized by nodes' facts), but the function also has an optional parameter to specify a specific file to be searched first on any given invocation. Perhaps coming from a database background, I'd like to mirror what seems more _natural_ and having values spread around over potentially a large number of files seems non-intuitive. If extlookup use would indeed require you to maintain a large number of separate files then that might be a good reason to find something better, but in all likelihood you can avoid that, or else structure it in a sane way. Consider also: When you work with a database, do you normally focus on how it maps data to the host filesystem? Focus no. Again as mentioned I try to simplify the structure by normalising it. Here I see I want to lookup a parameter ( say partition size ) from somewhere and I need to find it for a server called $hostname. If I can't find a parameter for $hostname, I may be happy if I find the same parameter for $domain, or if not I may be happy with some default value. _My_ preference is to look that up in one place. Also as mentioned in a different message, doing this lookup via a regexp might nicely enable me to keep the list of entries short. Given the diverse data parameterization you described, if you created a database for your configuration data, would you really organize everything into a single table? (And what would be its key?) yes: CREATE TABLE lookup_table ( config_item VARCHAR(200) NOT NULL, lookup_value VARCHAR(200) NOT NULL, return_value VARCHAR(200) NOT NULL PRIMARY KEY ( config_item, lookup_value ) ) SELECT return_value FROM lookup_table WHERE config_item = 'lvsize' AND lookup_value = 'myhostname001' provides a fast lookup of the value. If that fails you can do SELECT return_value FROM lookup_table WHERE config_item = 'lvsize' AND lookup_value = 'example.com' for a more generic response. If that fails you can do: SELECT return_value FROM lookup_table WHERE config_item = 'lvsize' AND lookup_value = 'DEFAULT' Yes, I'm fully aware this could be normalised better but given the limited number of entries it's trivial to understand, trivial to maintain fast to access. The 3 SELECTs could be optimised into a _single_ more complex SELECT adding
Re: [Puppet Users] Re: Thoughts about extlookup: http://blog.wl0.org/2011/05/thoughts-about-extlookup-in-puppet/
r...@devco.net (R.I.Pienaar) writes: - Original Message - john.bollin...@stjude.org (jcbollinger) writes: Perhaps. To some extend my non-parameterised classes are _very_ similar in many ways except for various parameters (creation of logical volumes and filesystems, version of mysql to use, creation of certain cron jobs for importing informtation into a configuration database, and other crons for doing standard jobs, etc) Most of the completely common stuff has been taken out. However, using extlookup() doesn't really work for me right now for a simple reason I haven't yet mentioned. I'd further like to have a lookup_by_key() function [let's say similar to the extlookup() one provided now] but also a lookup_by_regexp() function where I can match groups of boxes, in my case mainly based on regexps of their hostnames rather than on specific $hostname, or $domain, etc Without that I need to add more entries to match by lots of hostnames, which doesn't work. I think you're really trying to just avoid thinking in the puppet way instead trying to force some previous ideas on how things should work on it rather than figure out how it works. For example, you could just define a fact on your machines that indicate what role they have - perhaps based on your regex hostnames - and then use that in extlookup to select csv file to use. That would achieve your goal and it would be the puppet way of solving it. Currently I build nodes.pp from an external source and that provides the node with it's basic class (and hence role). So that's not needed. One of the irritating things I see when we discuss puppet is the inevitable I can't describe my complete problem as it's [confidential] my site specific, so you can't have a full understanding of what I'm doing or why I'm doing it wrong, or provide me a precise, clear explanation of how better to solve my original problem. We talk in generics and that means we never fully understand each other. I guess the reasons are obvious but it's probably frustrating to us all. Nevertheless this thread I started has helped me clarify several things so I appreciate the time you've spent discussing them with me. Simon -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Thoughts about extlookup: http://blog.wl0.org/2011/05/thoughts-about-extlookup-in-puppet/
Hi John,
While you obviously disagree with me, thanks for taking the time to
reply. I may be looking at the problem the wrong way which is why
I'm trying to figure out if that's the case and why.
john.bollin...@stjude.org (jcbollinger) writes:
On May 13, 8:07 am, jcbollinger john.bollin...@stjude.org wrote:
I agree that it is silly to suppose that extlookup() is a universal
solution, but personally, I don't take that impression from web
comments in the first place. In fact, Puppet Labs's own recently
updated style guide recommends against using extlookup(), though that
position is controversial.
Interesting. I wasn't aware of that. A colleague is starting to
use extlookup() for other functionality and if this may not be
ideal we may need to reconsider.
I also agree that for some sites, a richer / more powerful lookup
feature might be needed.
On the other hand, extlookup() is easy to set up and use, and it is
powerful enough for most needs. If you need a more powerful data
lookup function then you would be lucky to find a canned one that
meets all your needs.
Perhaps but I don't see it as being that bad, and it seems to be
the closest thing to a solution I could find at the moment. My current
recipes (perhaps incorrectly) are heavily customised in a way
which often is not just host specific or domain specific. Hence
the need to override the default settings. Other settings though
may need this thus requiring this to be more dynamic.
Moreover, I disagree with several of the opinions and conclusions in
your post:
1) You write: The extlookup() functionality only allows [...
specifying implicitly ...] where to look for this value. That is
false. Extlookup does provide for configuration of a standard set of
CSV files to search (which can be parameterized by nodes' facts), but
the function also has an optional parameter to specify a specific file
to be searched first on any given invocation.
Perhaps coming from a database background, I'd like to mirror what
seems more _natural_ and having values spread around over potentially
a large number of files seems non-intuitive.
2) You would prefer looking up data via a compound key (config_item,
lookup_value) rather than via a simple key (config_item). You rightly
observe that this would sometimes allow fewer files to be used than
are needed in the current implementation, because different values for
the same item could be distinguished by the lookup_value instead of
by the file in which they reside. The trade-off, however, is that
every lookup is more expensive, because on average the function would
need to read more lines, and to perform a more complex analysis on
each one. In your blog you suggest a DB backend, but that's not
really relevant because by that point you have a completely different
function than extlookup(). And even a DB performs multi-key queries
slower than single-key searches.
Not if they are part of the primary key. That's part of the point.
Any modern database can sustain thousands of queries a second doing this.
However, you are right small sites may not want to go this way to
start with unless configuration using a database is trivial.
3) You argue that your suggested formulation of extlookup would be
clearer as the configuration is more explicit then the current
extlookup() definition. I think you're missing the point. It would
indeed be clearer from which file the data would come, but the
objective of extlookup is to separate the *definition* of the data as
much as possible from the *use* of the data. And I like that. I
prefer that my manifests _not_ specify a bunch of details to every
extlookup() call, because that would obscure the details that are
important at the manifest level.
ok. This perhaps is a different point of view we have. extlookup()
looks like a function. As such it's nice if the function given the same
input provides the same output. Otherwise confusion is easy.
As written it's clear if you do extlookup('dns_server') that you are
getting a dns server result. In many of the examples provided
the expected use case is: check first for the host, then the domain,
then ., but that's not specific, _and_ may be important.
Where the dns_server info is coming from is IMO important, as is how
you are trying to look it up (the order of the parameters in the search).
_How_ it gets retrieved (.csv file, db backend, ...) is _not_
something which I care about as probably that mechanism is likely to
be static and stable amongst all recipes.
4) You seem to want to use an arbitrary-length list of values as your
lookup key, but I don't see how that can work while retaining the
apparent objective of reducing the number of data files that must be
associated with your configuration and retaining the use of CSV.
The .csv files are currently lookup_key,lookup_value. AFAIK there is
no limit on either parameter's size. I'm only suggesting a third
grouping/column item_type,
Re: [Puppet Users] Re: Thoughts about extlookup: http://blog.wl0.org/2011/05/thoughts-about-extlookup-in-puppet/
r...@devco.net (R.I.Pienaar) writes:
Let me not be taken as an extlookup() zealot. I do like the design
as it is, but that doesn't mean it is ideal -- or even good -- for every
purpose. One of the advantages of Puppet, however, is that it is
fairly easy to modify and extend. If you would like an alternative
lookup facility then by all means, write one. If you are community-
minded then you could consider sharing it; the Puppet Forge (http://
forge.puppetlabs.com/) makes it easy to do that.
You may also be interested in some of the extlookup() author's
comments about it (http://www.devco.net/archives/2009/08/31/
complex_data_and_puppet.php). He hangs out here, so you may also
hear from him directly.
So extlookup wasnt written as a be-all and end-all of data sources.
I recognized we want data and code separation and of course I could write
an ENC but I also recognized that I could do something generic that would
solve a large % of cases for people to push out the point at which they
need an ENC a bit later or in many cases entirely. I hoped I could solve
it in a way that adds some value like make it easier to write configurable
modules and to do so in a way that people without programming backgrounds
can grasp and use and make their manifests a bit better.
With this in mind I wrote extlookup aiming to solve the most common problems.
I am not against ENCs I think big/complex shops absolutely must have them and
I
think a lot of the extlookup bashers missed this point - if you are big
enough or complex enough where extlookup doesnt work for you then you've
probably reached a legitimate point where investing time in writing an
ENC is time well spent. And you're also not any more the the target audience
extlookup was created for.
That's an interesting thought and also something I'm not sure of the answer to.
Personally I'm responsible for managing quite a few database servers, but the
company of course has several more severs of different types what we manage
besides that.
When I looked at the ENCs it seems to approach (if I understand how
they should be written) the problem from a different point of
view. That is provide: a list of global variables, and a list of
classes which should be defined for each node. That requires thinking
about the problem in quite a different way to the extlookup()
functionality which basically says: give me this value, and now based
on that I can decide what to do... That's my understanding of the view
views of how to configure a system. If that's right one thing I don't like
with the ENCs is the fact that it appears to use and rely on a large number
of global variables which are then used by the classes. As systems get more
complex that strikes me as not being good as the class behaviour may
change depending on the scope of where something comes from and how it's
included
and that's not clear. I've already been tripped up by not setting a global
variable
correctly (or using the wrong name) and not noticing and of course behaviour
changes. So I still think it's clearer if you can do:
$ip_address = extlookup( parameters to get an ip address )
$interface = extlookup( parameters to get an interface name )
interface::check{ $interface: ip_address = $ip_address }
than
inteface::check{ $global_interface1: ip_address = $global_ip_address1 }
as in the latter case it's not clear where these 2 values come
from. Their names are also less intuitive and if you have a number of
these to set then it gets worse.
You should though just look at extlookups code, its extremely simple and easy
to understand (especially since Jesse rewrote my ancient crappy code) and
this
should let you hack up solutions that solves your problems.
Ive tried to keep away from Ruby. Perhaps that's a real issue but yes, given
the idea is certainly sound, extending in a way that's more appropriate to my
perceived needs seems a sane approach.
Puppet is a framework, its very hackable, its easy to write functions and
types,
easy to copy them out to all your servers and make it work in new and
interesting
ways. So I totally encourage you to use extlookup as a example and solve the
problem in your own way or even to evaluate writing your own ENC.
One way or another I am probably going to go down that path. For the reasons
stated earlier building a complete ENC means that everything depends on it.
using a extlookup() type functionaliy initially at least makes the change seem
less imposing. However, if used heavily in the end it's doing just the same
thing,
but a different way.
Thanks for your comments.
Simon
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
Re: [Puppet Users] Re: Thoughts about extlookup: http://blog.wl0.org/2011/05/thoughts-about-extlookup-in-puppet/
kc7...@gmail.com (Patrick) writes: On May 14, 2011, at 12:55 AM, Simon J Mudd wrote: ... Not if they are part of the primary key. That's part of the point. Any modern database can sustain thousands of queries a second doing this. However, you are right small sites may not want to go this way to start with unless configuration using a database is trivial. As a side note, putting the information in a csv file should make it much easier to version along with the rest of your files. I assume, though I could easily be wrong, that most DBs aren't easy to version control using GIT and Subversion. Using GIT or subversion, no. But versioning the configuration in the database is hardly difficult to do. (If you got that far you have enough infrastructure and expertise to solve this). That means (using csv files) you can keep your data in lockstep with the version of the config files. Indeed and I think that's nice. So the only time this becomes an issue is when the load on the puppet master to provide the lookup based on the extlookup() calls is sufficiently high to be a problem. When it's a problem you override the csv lookup to work the same based on some sort of db lookup alternative. Simon -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Thoughts about extlookup: http://blog.wl0.org/2011/05/thoughts-about-extlookup-in-puppet/
john.bollin...@stjude.org (jcbollinger) writes: In fact, Puppet Labs's own recently updated style guide recommends against using extlookup(), though that position is controversial. I found the URL: http://docs.puppetlabs.com/guides/style_guide.html The extlookup() Function Modules should avoid the use of extlookup() in favor of ENCs or other alternatives. The statement is clear, but it would be nice to know the reasoning. That is if there's a problem with extlookup() then it would be good to know what it is. This may have been discussed elsewhere but it shouldn't be necessary to have to go and search for this. Simon -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Thoughts about extlookup: http://blog.wl0.org/2011/05/thoughts-about-extlookup-in-puppet/
sjm...@pobox.com (Simon J Mudd) writes: john.bollin...@stjude.org (jcbollinger) writes: In fact, Puppet Labs's own recently updated style guide recommends against using extlookup(), though that position is controversial. I found the URL: http://docs.puppetlabs.com/guides/style_guide.html The extlookup() Function Modules should avoid the use of extlookup() in favor of ENCs or other alternatives. The statement is clear, but it would be nice to know the reasoning. That is if there's a problem with extlookup() then it would be good to know what it is. This may have been discussed elsewhere but it shouldn't be necessary to have to go and search for this. In the end I did. http://groups.google.com/group/puppet-users/browse_thread/thread/34e4dfb4828a2835/d9f6d8d6c7bba841?show_docid=d9f6d8d6c7bba841 It is a good read. However, from the discussion a few things strike me: 1. the use of parameterised classes is recommended heavily. I've just found out about this new feature inspite of using puppet for a long time. Hence many recipes that I'm using are not paremeterised and I have a large number of similar classes. This is certainly worth fixing but is quite a painful transition to make given the number of systems in use and the chance of creating heavy breakage during that change. So if you haven't used parameterised classes much that's the first thing that needs looking at. 2. Does puppet provide an ENC? The dashboard? Again if you're not using one switching over is a heavy tasks as you have to define _everything_ first before you can use it. (that's how it seems to me). I recently attended a puppet master course and it's content was interesting, but despite there was little emphasis on this sort of thing getting the high level design right and more on the nitty gritty of the usage of the lower-level resource types. To push the use of an ENC it seems that this higher level planning would be most helpful. Also a real-life overview of how this would be implemented. Saying how to configure an ssh server doesn't count. I'd be interested in the deployment of a series of database servers, their configuration, setup, and how to take into account differences between one set of boxes and another. If I can relate to a use case that fits my own (using an ENC) and there's a way to slowly bring in more systems into using this ENC then it makes it easier to follow that path. The alternative of using an extlookup() behaviour does indeed have the inconvenience of hiding some behaviour depending on external aspects which are picked up by the extlookups, but if you already have working recipes perhaps and aren't using an ENC this gives you a way to clearly remove the conditional logic the recipes currently have. It's then a smaller step up to using an ENC if you want to go that way. For those that don't it still becomes reasonably clear where the configuration is parameterised and how/why that's done, so it's still a more flexible approach. Judging from people I've listened to at the last puppet camp there are many ways to use puppet. Sites vary from small to very large and there's no clear one solution fits all. So I'm inclined to agree that a extlookup() [ or get_config_by_key lookup] mechanism may work fine for a lot of people. As long as you know and understand the limitations of the approach you are taking that's fine. Again the ENC setup described to me at the course I went on did not mention parameterised classes, but simple variable settings and classes. I guess that will change as perhaps if you use parameterised classes the enc should do nothing other than provide a definition of the parameterised classes to use and their initial setup valus. Removing the global variables would ensure that each class's behaviour is encapsulated only by it's input parameters and that does sound much more reasonable. Simon -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: MySQL Restart
doug.garst...@gmail.com (Douglas Garstang) writes:
I've got a mostly working puppet manifest for mysql, except for one problem.
I'm using the RPM from the MySQL web site, and the packagers of this
RPM in their infinite wisdom decided that the mysql service should be
automatically started when the RPM was installed.
See http://bugs.mysql.com/bug.php?id=27072
If you don't like this behaviour tell Sun.
I use something like this, simplified (somewhat) which causes no problems.
# Installation of MySQL server binaries
class mysql::server {
package {
[ MySQL-server-communitiy ]:
ensure = installed,
notify = Exec[ chkconfig_mysql ]
}
# Configure the service.
# - Do NOT ensure enabled as this is carried out by chkconfig_mysql.
service { mysql:
hasrestart = true,
hasstatus = true,
require= Package[MySQL-server-community],
}
exec { chkconfig_mysql:
path= [ /sbin, /bin ],
command = chkconfig --levels 345 mysql on,
unless = chkconfig --list mysql | grep -q
'3:$chkconfig_state.*4:$chkconfig_state.*5:on',
require = Package[MySQL-server-community],
}
}
Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
[Puppet Users] Re: copying files back to puppet master?
Hi Luke, l...@madstop.com (Luke Kanies) writes: ... I think you should have a look to the filebucket[1]. It might not be exactly what you are asking for, but it might still help you. No, not really but thanks for the pointer. As I said my idea would be to use this to manage local configuration files, but to manage them centrally. In a previous job using cvs and cfengine I used this to allow us to maintain local configuration files normally pushed to the central server, but under configuration control could also be used to push out the same files on to a different box for example to replace a failed server. To be fair I don't expect puppet to do all of this, but it would be nice if the current file: resource/protocol could potentially work in both directions. This would open up a lot of possibilities. This will at least be possible internally with the code in 0.25 (which looks like it'll go rc1 this week, I think just one more ticket), but I don't know how it would actually be useful for the client. On the client it's NOT useful, but that's not the point. Perhaps my vision of a system like puppet is slightly different, but to manage a whole group of servers, and not just think about a single specific server. Hence my comment before about using a technique like this to allow files perhaps not pushed out by puppet to be managed centrally and if needed these older configurations could be later pushed back to the same or perhaps a different server. I haven't thought out all the details but I can see this as an alternative way of cloning configs from a specific server. I've been thinking about what you're looking for, though, and I think it would make more sense to directly integrate filebuckets into a version control system - the client would back modified files up to the server, the server would automagically check those files into a version control repository (into a branch named after the host, I assume), and then you could do whatever comparisons you wanted to your heart's content. Yes, that's basically the idea. It allows sysadmins to not be fearful about changes they make as they would be tracked and thus easier to revert should the need arise. I can see this being useful for managing changes in: - system and user cron jobs - user ~/.[bash]profile or ~/.bashrc - user ~/.ssh/config type files - ... While puppet may not be pushing out these changes all of a sudden you have these files under control. Potentially you could also add some sort of extra hooks (on the server) when these changes are collected to do certain actions such as send out emails or whatever. I've actually got a basic proof of concept of at least replacing the filebucket store with git done[1]. It's just the client-side pieces, you'd need to add the server-side pieces that created the branch and such, but I don't think that would be a ton of work, and it'd provide everything you want while integrating nicely with how Puppet already backs files up to the server. 1 - http://gist.github.com/77811 I'll have a look. Thanks for taking the time to considering what I was asking. Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] copying files back to puppet master?
Hi all, Puppet is used mainly for managing resources on the servers it manages, and as such files and software may be distributed from the puppet master to the clients as part of the management or configuration process. That's fine. The puppet installation where I work has evolved starting from a site with no puppet to a site where most of the servers are puppetised with the intention being on doing a complete deploy exclusively using puppet. However, as not all aspects of all resources on the server are managed by puppet there is room for the administators to make mistakes. As such we often want to version control configuration information on the server which protects against mistakes but also allows us to see when changes were made, either automatically (by puppet) or manually. If I'm not mistaken it's not possible to use the file resource for copying files back to the puppet master. Is that correct? This functionality would be most useful, especially if we combine this with some revision control system. If we did something like this the files to be returned back to the puppet master (assuming they change) would need to be copied back to a location like /puppet_master_path/site_and_hostname/host_path/filename. The advantage of doing this in puppet is that you don't need to ensure that extra firewall holes are opened, and that in principal at least the same file resource could be used. It might be necesssary to additionally qualify/authorise back to master transfers for security reasons. A feature like this is much easier to manage than doing this outside of puppet. It is also much easier to compare the configuration of a number of different similar servers if the files are located on the same server, especially as the number of managed servers increases. So is this possible at the moment in puppet, and if not is it a feature that might be interesting to others? How are others tackling problems like this? Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] Re: copying files back to puppet master?
brice-pup...@daysofwonder.com (Brice Figureau) writes: ... If I'm not mistaken it's not possible to use the file resource for copying files back to the puppet master. Is that correct? Yes it is correct. ok. ... I think you should have a look to the filebucket[1]. It might not be exactly what you are asking for, but it might still help you. No, not really but thanks for the pointer. As I said my idea would be to use this to manage local configuration files, but to manage them centrally. In a previous job using cvs and cfengine I used this to allow us to maintain local configuration files normally pushed to the central server, but under configuration control could also be used to push out the same files on to a different box for example to replace a failed server. To be fair I don't expect puppet to do all of this, but it would be nice if the current file: resource/protocol could potentially work in both directions. This would open up a lot of possibilities. Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] Re: How to use definitions with common resources?
philipp.hanselm...@gmail.com (philipp Hanselmann) writes:
Simon J Mudd schrieb:
...
#NOTE: generate is executed on the puppetmaster.
$download_dir = generate(/usr/bin/env,bash,'-c',/bin/
mktemp)
With this every time your definition get used it will choose an random
folder in your /tmp.
Thanks. The idea is good, but it doesn't quite work.
1. The script unfortunately contains account information
user/passwords for the application's initial installation. Hence
/root/tmp is better as only root users can access this directory.
1.) With
mktemp -p /root
Sure. If I'd wanted a random directory I would have done something like that.
The point is that puppet is SLOW at copying files from the master to the client.
Anything that's not a small file takes a while and syncing a directory like
this (with say 50-100 scripts) gets very tedious. So I'm reluctant to:
copy from master, run, deleteevery 30 minutes on several hundred boxes.
Hence the optimisation of leaving the install script in one place and using
it to decide if any work needs doing. This is just a wrapper around the
vendor provided install binary which is NOT copied with puppet as puppet is
too slow (and the binary is about 20MB).
3.) Clean-up of the random folder?
may something like this helps?
exec { $install_script/$software_env:
command = $download_dir/$install_script ... appropriate
parameters ... rm Rf $download_dir,
onlyif = ... test if software not installed ...,
require = File[$download_dir/$install_script]
}
Not quite. You'd need to have a separate Exec run dependent on the
File[$download_dir/$install_script] as the command shown is not
always run (see onlyif) and thus the rm would not be triggered.
In many ways a post_exec or cleanup = would be handy which _is_
always run even if command isn't.
And remember part of the problem of this original post was that the
setup of the temporary directory fails if you have 2 similar
installations requiring the same base directory and installation
script. That's what bothers me (and I'd like to solve) more than the
details of the single install problem which works fine for me.
Your suggestions are valid but my implementation is different for the
reasons stated.
Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
[Puppet Users] Re: onlyif functionality for the File type
[EMAIL PROTECTED] (Paul Lathrop) writes: Why are you doing this? This kind of situation usually indicates a need to adjust one's mindset in order to work within the Puppet model rather than fighting against it. That said, I think you could do this a couple ways: 1) An Exec resource with 'onlyif' and make the File resource depend on it. 2) A custom fact that checks for the existence of the directory, combined with 'if' statements around the File resource. Still, I think you'll get more mileage out of telling us what problem you are trying to solve so we can help you figure out the best way to solve that problem within the Puppet model. I posted almost exactly the same problem a couple of days ago: - you want to install via a custom installer an application, so you need to do the following: 1. copy the file and installer over 2. run installer and cleanup Step 1 above needs to be done ONLY IF the installation has NOT been completed. (check for some final installation file/directory) Step 2 depends on step 1 so only takes place if the files are copied. So as I suggested in my posting a few days ago an onlyif option would be very nice. I phrased it in my posting as unless some condition, which is just the negative condition of the onlyif condition. My guess the poster wants to do the same thing. Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] Re: file type and unless method
[EMAIL PROTECTED] (Joshua Timberman) writes: On Sat, Nov 8, 2008 at 3:42 AM, Simon J Mudd [EMAIL PROTECTED] wrote: One thought I had was to have an unless method/attribute in the file type which would allow us to disable the configured file behaviour: This seems like it would provide a nice easy way to run custom install scripts and only copy over the files once. The install script would clean up afterwards. The file resource already has something like this in the replace parameter. replace Whether or not to replace a file that is sourced but exists. This is useful for using file sources purely for initialization. Valid values are true (also called yes), false (also called no). Alternately, just add the 'refreshonly' parameter on the exec resource so it only gets run if the installation script is replaced. thanks for the suggestion. I'll see if I can use that instead. Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~--~~~~--~~--~--~---
[Puppet Users] puppet, facter and using non-scalar values
Hello all,
I've been using puppet for a while now and have an issue I've not
seen clearly answered. One of the issues I want to address in puppet
is to manage LVM, the volume groups and logical volumes on a series
of servers.
Starting from a small base configuration I would like to configure the
logical volumes on the server.
However this doesn't seem to be a thing that puppet knows about directly.
My first thought is to create some custom facts which store the volume
group names and within each volume group the logical volumes. This is
not ideally stored as a couple of scalar values.
So how should I store this? Facter would seem to be the best thing to
adjust adding some custom facts but from what I can see all facter facts
are single scalar values.
So am I approaching the problem incorrectly? My initial thought is
that I'd like to populate some sort of variable volume_group_info as a
hash of hashes, the first hash being indexed on the volume group names
and the second on the logical volume group names in each volume group.
Still being new to ruby I'm trying to work out how to do this straight
in ruby but I'm sure this can be done, and if so my question would by
how to translate this into puppet code.
I'm initally looking at doing something like:
-- snip --
vol_groups = Hash.new
puts vol_groups.to_s
if FileTest.exists?(/usr/sbin/lvs)
%x{/usr/sbin/lvs --noheadings 2/dev/null}.split(\n).each do |line|
if line =~ /\s+(\S+)\s+(\S+)/
puts match vg: + $2 + , lv: + $1
lvname = $1
vgname = $2
# now fill lvname into vol_groups by vgname...
end
end
end
-- snip --
Can anyone comment on this idea?
Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~--~~~~--~~--~--~---
