Re: [Puppet Users] Strategies for "boring" packages

[Dirk Heinrichs]

Am 19.04.2016 um 02:47 schrieb J.T. Conklin:

> What strategies do you use for "boring" modules so you're not
> overwhelmed by hundreds of small boilerplate modules?

I use a simple module with one class for mass-installing packages and
another one for mass-uninstalling, like:

class linux_base::install ($packages = undef) {
  $packages.each |String $pkg| {
package { $pkg: ensure => latest, }
  }
}

where $packages is an array of package names (strings), which can easily
be provided for each (group of) host(s) via Hiera.

The ::uninstall class has "purge" instead of "latest". Of course, input
array for both classes must be mutually exclusive, if both are used at
the same time.

HTH...

Dirk
-- 

*Dirk Heinrichs*, Senior Systems Engineer, Engineering Solutions
*Recommind GmbH*, Von-Liebig-Straße 1, 53359 Rheinbach
*Tel*: +49 2226 159 (Ansage) 1149
*Email*: d...@recommind.com 
*Skype*: dirk.heinrichs.recommind
www.recommind.com 

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/57178BA2.6020801%40recommind.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Strategies for "boring" packages

[Ramin K]

On 4/18/16 5:47 PM, J.T. Conklin wrote:

At work, we've written about 120 modules in our puppet code repository.
About two dozen are "interesting", in that they have lots of parameters
and configuration that is specific to our environment.  The balance are
"boring", rather they are mostly boilerplate with minimal configuration.
For example, our modules abstract the differences in package and service
names between RedHat and Debian based systems.

However, there is some disagreement amongst our puppeteers about how to
handle these "boring" modules. One side objects to the amount of boiler-
plate and duplication, and would prefer that we simply define packages
in our role/profile modules. The other side claims that abstracting
package and service names is value enough to justify the overhead, and
that "boring" packages often become "interesting" over time as new
requirements for flexibility and customization develop over time. Each
group is firmly convinced that their opinion is the right one.

So I throw the question to the puppet community... What strategies do
you use for "boring" modules so you're not overwhelmed by hundreds of
small boilerplate modules?

Thanks for sharing,

 --jtc



	At the previous job I preferred to promote packages to full modules 
from a holding module.


class vpackages {
  include vpackages::params

  @package { 'curl':tag => 'utils',}
  @package { 'htop':tag => 'utils',}
  @package { 'iftop':   tag => 'utils',}
  @package { 'whois':   tag => 'utils', name => 
$vpackages::params::whois, }


  # Mysql packages
  @package { 'percona-toolkit': tag => 'mysql', }
  @package { 'mysql-client':tag => 'mysql', name => 
$vpackages::params::mysqlclient, }

}

mysql-client is good example.

class mysql::install {
  package { 'mysql-server':
ensure => present,
name   => $mysql::params::packagename,
  }

  # realize packages like the client
  Package<| tag == 'mysql' |>
}

Later our Mysql installations were more complicated in several different 
roles with more versions. At that point it made sense to move 
mysqlclient into it's own parameterized module.


What I liked about the vpackages modules is that it's a visual 
representation of the shared namespace.


Ramin

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5716775B.7020007%40badapple.net.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Strategies for "boring" packages

[J.T. Conklin]

Luke Bigum  writes:
> In my mind the "purest" way would be to go individual modules for each
> package/service combination. If the only requirement is that you are
> handling the differences between Red Hat and Debian flavours, then a
> module per package/service. These modules would be wholly self
> contained and rely on some of the standard set of Facter facts. And
> then you could publish them :-) It would also avoid future duplicate
> resource declarations where someone's embedded "packageX" into one
> profile, and it clashes with "packageX" in another profile.
>
> I can see the argument for putting package installs and service starts
> into a profile but only if it's global for every operating system. So
> if there was profile::webserver that needed Package[openssl] and that
> was correct for all operating systems, then fine. However if you have
> to start doing conditional logic to find the right name of
> Package[openssl] for Red Hat and Debian, then profile::webserver is
> not the place. profile::webserver is a container of business logic
> that relates wholly and only to your business and your team. The exact
> implementation of Package[openssl] has nothing to do with
> profile::webserver, as long as openssl gets there somehow, that should
> be all you care about at the Profile level. Implementing
> Package[openssl] really depends on the operating system Facts alone,
> and this should be in it's own module... and... all of a sudden your
> profile::webserver is operating system agnostic, which is cool.

I agree 100%.

> Question - why is it taking your team getting annoyed at generating
> boilerplate code? Surely you have some sort of "puppet module create"
> wrapper script or you use https://github.com/voxpupuli/modulesync? If
> you've got so much overhead adding boiler plate code for your boring
> modules then I think you're tackling the wrong problem... If you can
> bring the boiler plate code problem down to 1-2 minutes, it's got to
> only take another 5-10 minutes tops to refactor one package{} and one
> service{} resource out of the profile and into it's own module, and
> then your team argument kind of goes away.

Lately we've been creating a new module every 3-4 weeks. So it's been
faster to copy an existing module, run a perl script that renames the
module, packages, and services, than it would be to write/adapt a script
to generate new modules from a template + parameters. It only takes me a
minute or two to create a new module. The counter-argument is that it
only takes a few seconds to add a "package { 'foo': }" to a profile, and
that a module per package/service leads to a unmanagable set of hundreds
of modules.

While I'm in the camp that separate modules for each package/service are
a good thing, I started this thread in order to double check my opinion.

> Question - why are you writing 120 modules yourself? Are there really
> no other implementations of these things on the Forge or GitHub?

In some cases we've found existing modules, we even use a few.  But in
the general case, we've found it useful to write our own modules so they
have the same "look and feel" i.e. use the same sets of parameters and
facts with the same semantics. Our basic package/service boilerplate is
based off the example42 modules (at least as they were ~2-3 years ago).

   --jtc

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/87bn55d2bj.fsf%40wopr.acorntoolworks.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Strategies for "boring" packages

[Luke Bigum]

In my mind the "purest" way would be to go individual modules for each 
package/service combination. If the only requirement is that you are handling 
the differences between Red Hat and Debian flavours, then a module per 
package/service. These modules would be wholly self contained and rely on some 
of the standard set of Facter facts. And then you could publish them :-) It 
would also avoid future duplicate resource declarations where someone's 
embedded "packageX" into one profile, and it clashes with "packageX" in another 
profile.

I can see the argument for putting package installs and service starts into a 
profile but only if it's global for every operating system. So if there was 
profile::webserver that needed Package[openssl] and that was correct for all 
operating systems, then fine. However if you have to start doing conditional 
logic to find the right name of Package[openssl] for Red Hat and Debian, then 
profile::webserver is not the place. profile::webserver is a container of 
business logic that relates wholly and only to your business and your team. The 
exact implementation of Package[openssl] has nothing to do with 
profile::webserver, as long as openssl gets there somehow, that should be all 
you care about at the Profile level. Implementing Package[openssl] really 
depends on the operating system Facts alone, and this should be in it's own 
module... and... all of a sudden your profile::webserver is operating system 
agnostic, which is cool.

Question - why is it taking your team getting annoyed at generating boilerplate 
code? Surely you have some sort of "puppet module create" wrapper script or you 
use https://github.com/voxpupuli/modulesync? If you've got so much overhead 
adding boiler plate code for your boring modules then I think you're tackling 
the wrong problem... If you can bring the boiler plate code problem down to 1-2 
minutes, it's got to only take another 5-10 minutes tops to refactor one 
package{} and one service{} resource out of the profile and into it's own 
module, and then your team argument kind of goes away.

Question - why are you writing 120 modules yourself? Are there really no other 
implementations of these things on the Forge or GitHub?

--
Luke Bigum


- Original Message -
From: "J.T. Conklin" <j...@acorntoolworks.com>
To: "puppet-users" <puppet-users@googlegroups.com>
Sent: Tuesday, 19 April, 2016 01:47:37
Subject: [Puppet Users] Strategies for "boring" packages

At work, we've written about 120 modules in our puppet code repository.
About two dozen are "interesting", in that they have lots of parameters
and configuration that is specific to our environment.  The balance are
"boring", rather they are mostly boilerplate with minimal configuration.
For example, our modules abstract the differences in package and service
names between RedHat and Debian based systems.

However, there is some disagreement amongst our puppeteers about how to
handle these "boring" modules. One side objects to the amount of boiler-
plate and duplication, and would prefer that we simply define packages
in our role/profile modules. The other side claims that abstracting
package and service names is value enough to justify the overhead, and
that "boring" packages often become "interesting" over time as new
requirements for flexibility and customization develop over time. Each
group is firmly convinced that their opinion is the right one.

So I throw the question to the puppet community... What strategies do
you use for "boring" modules so you're not overwhelmed by hundreds of
small boilerplate modules?

Thanks for sharing,

--jtc

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/87shyio252.fsf%40wopr.acorntoolworks.com.
For more options, visit https://groups.google.com/d/optout.
---

LMAX Exchange, Yellow Building, 1A Nicholas Road, London W11 4AN
http://www.LMAX.com/

Recognised by the most prestigious business and technology awards
 
2016 Best Trading & Execution, HFM US Technology Awards
2016, 2015, 2014, 2013 Best FX Trading Venue - ECN/MTF, WSL Institutional 
Trading Awards

2015 Winner, Deloitte UK Technology Fast 50
2015, 2014, 2013, One of the UK's fastest growing technology firms, The Sunday 
Times Tech Track 100
2015 Winner, Deloitte EMEA Technology Fast 500
2015, 2014, 2013 Best Margin Sector Platform, Profit & Loss Readers' Choice 
Awards

---

FX and CFDs are leveraged products that can result in losses exceeding your 
deposit. They are not suitable for everyone so please ensure you fully 
understand the risks involved.

This mes

[Puppet Users] Strategies for "boring" packages

[J.T. Conklin]

At work, we've written about 120 modules in our puppet code repository.
About two dozen are "interesting", in that they have lots of parameters
and configuration that is specific to our environment.  The balance are
"boring", rather they are mostly boilerplate with minimal configuration.
For example, our modules abstract the differences in package and service
names between RedHat and Debian based systems.

However, there is some disagreement amongst our puppeteers about how to
handle these "boring" modules. One side objects to the amount of boiler-
plate and duplication, and would prefer that we simply define packages
in our role/profile modules. The other side claims that abstracting
package and service names is value enough to justify the overhead, and
that "boring" packages often become "interesting" over time as new
requirements for flexibility and customization develop over time. Each
group is firmly convinced that their opinion is the right one.

So I throw the question to the puppet community... What strategies do
you use for "boring" modules so you're not overwhelmed by hundreds of
small boilerplate modules?

Thanks for sharing,

--jtc

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/87shyio252.fsf%40wopr.acorntoolworks.com.
For more options, visit https://groups.google.com/d/optout.