No problem! Feeling lost is a little normal -- Puppet is a deceptively
complex tool. You will probably go through MANY iterations of
architecture, module layout, etc, as you ramp up your Puppet skills.
When I first started with Puppet, I wrote a very clever ENC that did
some internal-business-logic stuff. It worked great, and I was very
proud of myself. Fast forward a few months, and I realized that I had
basically reinvented Hiera, in a way that wasn't as good as Hiera, and
all I had to do to use pure Hiera was write a custom fact and use it in
my Hiera hierarchy.
The stuff I described is a little hard to understand if you haven't
dealt with modules before, because it's very abstract. However, it seems
like you're on the right track. Group everything to make an "ELK
server" into a profile module -- including packages, configuration
files, and network ports, then just include that into a role module that
gets assigned to a node. If there is special "data" (say, database
credentials), that should come from Hiera. You may find that you may
need to (or it might just be easier to) split out your ELK module into
its 3 components, then have the ELK module tie them all together.
However, I would caution you against completely rewriting everything
from scratch -- there are a lot of really great modules on the Forge --
Elasticache, for example, has a really good module that is provided by
the vendor. Don't reinvent the wheel. :) Just write enough 'glue' code
to tie everything together to meet your business needs.
I would start by watching the video linked to by Rich in his reply to
this thread. I haven't watched the whole thing, but what I've watched
is a pretty awesome intro to a lot of best practices. There are a
couple other good videos of the roles & profiles model floating around
that may be worth your time to watch as well.
On 8/13/2016 9:53 AM, Robert Poulson wrote:
Hi Peter,
thank you for the valuable input!
I read your answer yesterday many times and asked myself: "yes, it's
clear that I'm doing it wrong, but what is the Right Way™then?".
But many hours later, I might have one of those "aha moments" :)
I always wanted to write modules - or use them in a manner - which
define the state of the node after the module gets assigned to it. The
modules either contain the information themselves (like the mounts,
apache modules etc.) or get these as a parameter; and they always got
assigned directly to the node.
But it seems to me that in this "roles & profiles" model, one uses the
modules more like if they were just various libs; they're not assigned
to the nodes per se, but are just "called" (included) in the node
classifiers with the parameters.
This way, I don't have to open the port 5601 for Kibana in my ELK
module, but include the firewall class with port 5601 as a parameter
within the ELK profile.
Or something similar. Or something different. But I think I got your
point :)
The good news is, I have very much work ahead of me so I don't mind to
start creating new modules, and re-doing the existing ones in this
philosophy.
Thanks; and feel free to correct me if I'm still wrong :)
Rp
On Thu, Aug 11, 2016 at 11:30 PM, Peter Kristolaitis
<alte...@alter3d.ca <mailto:alte...@alter3d.ca>> wrote:
This is probably a case of trying to use Hiera for stuff you
shouldn't be using it for. It's important to keep in mind that
Hiera is not really a replacement for manifest files -- or even
better, modules that follow a role & profile architecture. You
can hack Hiera to do that, but you're going to rip your hair out
debugging it, and there are better architectural ways of dealing
with it anyways. create_resources() seems really cool when you
first learn about it. That fades quickly as you scale. ;)
To use your firewall example, "ports opened" depend on the profile
of the system (e.g. what software is installed). All nodes that
have Apache need port 80 and 443 opened, for example. All SSH
servers need port 22.
In a "roles & profiles" model, every node probably gets a
"baseline firewall" profile assigned to it. This is where you
would set up iptables policies, default-drop rules, etc. If
you're using the puppetlabs-firewall module, you'll probably tell
it to purge all non-Puppet-managed firewall rules.
Then in every profile class that requires an open network port,
you would have it add the appropriate firewall rule definition.
The SSH profile class adds a firewall rule for port 22. The
Apache class adds firewall rules for ports 80 and 443.
This way, when Puppet is configuring your system, the logic is "I
have Apache installed, so I need ports 80 and 443 opened", instead
of "I'm node ABC and I have port 80 and 443 opened, but I might or
might not have Apache installed", which is a situation you can
easily get into if you're shoving as much as you can into Hiera.
This approach seems like much more work at first, because breaking
out all your logic into profile modules can be quite verbose and
time-consuming. Sometimes you will get a profile class that does
nothing but install a single package (e.g. if you have multiple
profiles that require a single Apache module). That's OK. It
will be MUCH easier and less error-prone to manage over the long
run, and you'll appreciate the architecture if you start using
exported resources and other features of Puppet.
On 2016-08-11 04:46 PM, Robert Poulson wrote:
Hey List,
during the last weeks I finally managed to spend some quality
time :) with Puppet and I deepened my knowledge a bit: learned
some ruby, wrote my first custom ruby facts, used hiera in my
latest module... it's been fun.
There is one concept with Hiera though which is still unclear for me.
Let's say all of our servers have only port 22 open. But I assign
my module to a node which installs and configures Apache, so it
should add port 80 and 443 to the open ports list.
Or I assign my other module to a node which uses SSO and has to
enable the mod_auth_kerb Apache module. On my ELK nodes I'd like
to use some more Apache modules as well.
Or a Docker node needs to add some rules to the /etc/sudoers,
another application some more rules...
I could name a few more examples, but the point is: I can't
always define something static in a hiera value, like open_ports:
/ apache_modules_enabled: / sudo_commands: etc. but need to
dynamically manage these arrays when a module is assigned to a node.
How should be this achieved?
A _very_ primitive analogy is like
$PATH="$PATH:/usr/local/my_app"
where I don't have to take care of the actual content of $PATH,
but can simply extend it. Something similar would be great in
Hiera (or elsewhere) instead of using defines for every resource
I have.
Thanks
Rp
--
You received this message because you are subscribed to the
Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to puppet-users+unsubscr...@googlegroups.com
<mailto:puppet-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CANwwCtz0nz6%2BzZMX4ksfrfy%3D8sMjmsSi6vrSpZqS49fVUVO-dA%40mail.gmail.com
<https://groups.google.com/d/msgid/puppet-users/CANwwCtz0nz6%2BzZMX4ksfrfy%3D8sMjmsSi6vrSpZqS49fVUVO-dA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to puppet-users+unsubscr...@googlegroups.com
<mailto:puppet-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/efe67a83-ce29-07ef-3e69-1bcdb32f2d05%40alter3d.ca
<https://groups.google.com/d/msgid/puppet-users/efe67a83-ce29-07ef-3e69-1bcdb32f2d05%40alter3d.ca?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to puppet-users+unsubscr...@googlegroups.com
<mailto:puppet-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CANwwCtyX0jER-zaQwXfuYhxs8hz5ngMKqbSu%2Bv8RnDAS72s8yw%40mail.gmail.com
<https://groups.google.com/d/msgid/puppet-users/CANwwCtyX0jER-zaQwXfuYhxs8hz5ngMKqbSu%2Bv8RnDAS72s8yw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Puppet
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/bdcda760-379b-1da0-d2e8-9f6c47571fd3%40alter3d.ca.
For more options, visit https://groups.google.com/d/optout.
