[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
On Tuesday, May 20, 2014 12:54:15 PM UTC-5, Jesse Cotton wrote: Understood. However without creating a custom fact, I am not aware of a way to say 'apache::keepalive = 1' only applies when a node has the profile 'profile::apache_phpfpm'. There is no good way to say that to Puppet. It reflects a non-idiomatic thought process. Moreover, it's unlikely to be what you really want to say -- you might in the future want other nodes to have keepalive enabled, too. You should instead consider how to assign apache::keepalive: 1 to exactly those nodes that get profile::apache_phpfpm. There are at least two ways you could approach that problem: 1. If you are using hiera to assign roles and/or profiles to nodes then it should be entirely straightforward to override apache::keepalive at the same hierarchy level where you assign profile::apache_phpfpm. 2. You can perform a resource parameter override in profile::apache_phpfpm. If it is viable, the former is preferable to the latter. A resource parameter override is a bit messy because it relies on implementation details of module 'apache'. (It does not work to override class parameters; you must instead override one or more parameters of a bona fide resource, which can be of either defined or native type.) And the foregoing is based on using only the built-in YAML back end. Hiera supports pluggable back ends, usable together or separately. A custom back end can employ whatever lookup or computation you want to serve whichever data you choose. We're really trying to avoid this. I don't blame you. A custom back-end should certainly be a last resort. As you keep that alternative in reserve, however, don't forget that such a custom component does not necessarily need to replace the YAML back-end; hiera can use two (or more) at the same time. class profile::drupal ( $apache_listen = ['80'], $apache_name_virtual_hosts = ['*:80'], $apache_modules = ['fastcgi'], $apache_fastcgi_servers = { 'www' = { host = '127.0.0.1:9000', faux_path = '/var/www/php.fcgi', alias = '/php.fcgi', file_type = 'application/x-httpd-php' } }, $phpfpm_pools = { 'www' = { listen = '127.0.0.1:9000', user = 'apache', pm_max_requests = 500, catch_workers_output = 'no', php_admin_value = {}, php_value = {} } }, $php_modules = [], $firewall_rules = {}, $backup_jobs = {}, $cron_jobs = {} ) { include ::apache ::apache::listen { $apache_listen: } ::apache::namevirtualhost { $apache_name_virtual_hosts: } ::apache::mod { $apache_modules: } create_resources(::apache::fastcgi::server, $apache_fastcgi_servers) include ::php::fpm::daemon create_resources(::php::fpm::conf, $phpfpm_pools) ::php::module { $php_modules: } ~ Service['php-fpm'] # So the apache user is created before # php-fpm starts Class['::apache'] - Class['::php::fpm::daemon'] create_resources(firewall, $firewall_rules) create_resources(::duplicity::job, $backup_jobs) create_resources(::cron::job, $cron_jobs) } Either you're missing something or I am. I see nothing in that class that would inherently preclude it being composed. In particular, the two class declarations it contains both use 'include', not the resource-like class declaration syntax. If there is a barrier to composition it would be related to composition with another class that declares some of the same resources. That problem has a solution, however: factor out the multiply-declared resources into their own class or classes, which the then-composable classes declare instead of declaring the resources directly. Duplicate declaration: Apache::Listen[80] is already declared in file ... As I said, a duplicate resource. As I said, factor it (and the others) out. I think it's excellent practice to avoid resource declarations directly in profile classes, unless possibly of resources that are assuredly specific to the profile in which they appear. Yes, the class could be broken into separate classes but this only exacerbates the issue of assigning variables based on the profile. But that's just it: you don't *want* to assign data based (strictly) on profile. Or if you do, then understand that it is inherently inconsistent with composable profiles. Since the node characteristic that determines the combination of profiles in use is its role, it is on that basis that you want to assign data. Indeed, that's what you already said with respect to apache::keepalive. Note that with hiera, that doesn't mean you must avoid assigning data based on profile; it just means you must (also) be able to assign data based on role, with the role-based data having higher priority. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
But that's just it: you don't *want* to assign data based (strictly) on profile. Or if you do, then understand that it is inherently inconsistent with composable profiles. Since the node characteristic that determines the combination of profiles in use is its role, it is on that basis that you want to assign data. Indeed, that's what you already said with respect to apache::keepalive. Note that with hiera, that doesn't mean you must avoid assigning data based on profile; it just means you must (also) be able to assign data based on role, with the role-based data having higher priority. I understand this better now however the problem we were really struggling with was how to assign variables based on role or profile. Ie, how do you incorporate the the role or profile into the hierarchy so a variable only applies to nodes that have a particular role or profile. Your first suggestion is good although thats not how we're currently assigning classes. On Wednesday, May 21, 2014 10:03:33 AM UTC-4, jcbollinger wrote: On Tuesday, May 20, 2014 12:54:15 PM UTC-5, Jesse Cotton wrote: Understood. However without creating a custom fact, I am not aware of a way to say 'apache::keepalive = 1' only applies when a node has the profile 'profile::apache_phpfpm'. There is no good way to say that to Puppet. It reflects a non-idiomatic thought process. Moreover, it's unlikely to be what you really want to say -- you might in the future want other nodes to have keepalive enabled, too. You should instead consider how to assign apache::keepalive: 1 to exactly those nodes that get profile::apache_phpfpm. There are at least two ways you could approach that problem: 1. If you are using hiera to assign roles and/or profiles to nodes then it should be entirely straightforward to override apache::keepalive at the same hierarchy level where you assign profile::apache_phpfpm. 2. You can perform a resource parameter override in profile::apache_phpfpm. If it is viable, the former is preferable to the latter. A resource parameter override is a bit messy because it relies on implementation details of module 'apache'. (It does not work to override class parameters; you must instead override one or more parameters of a bona fide resource, which can be of either defined or native type.) And the foregoing is based on using only the built-in YAML back end. Hiera supports pluggable back ends, usable together or separately. A custom back end can employ whatever lookup or computation you want to serve whichever data you choose. We're really trying to avoid this. I don't blame you. A custom back-end should certainly be a last resort. As you keep that alternative in reserve, however, don't forget that such a custom component does not necessarily need to replace the YAML back-end; hiera can use two (or more) at the same time. class profile::drupal ( $apache_listen = ['80'], $apache_name_virtual_hosts = ['*:80'], $apache_modules = ['fastcgi'], $apache_fastcgi_servers = { 'www' = { host = '127.0.0.1:9000', faux_path = '/var/www/php.fcgi', alias = '/php.fcgi', file_type = 'application/x-httpd-php' } }, $phpfpm_pools = { 'www' = { listen = '127.0.0.1:9000', user = 'apache', pm_max_requests = 500, catch_workers_output = 'no', php_admin_value = {}, php_value = {} } }, $php_modules = [], $firewall_rules = {}, $backup_jobs = {}, $cron_jobs = {} ) { include ::apache ::apache::listen { $apache_listen: } ::apache::namevirtualhost { $apache_name_virtual_hosts: } ::apache::mod { $apache_modules: } create_resources(::apache::fastcgi::server, $apache_fastcgi_servers) include ::php::fpm::daemon create_resources(::php::fpm::conf, $phpfpm_pools) ::php::module { $php_modules: } ~ Service['php-fpm'] # So the apache user is created before # php-fpm starts Class['::apache'] - Class['::php::fpm::daemon'] create_resources(firewall, $firewall_rules) create_resources(::duplicity::job, $backup_jobs) create_resources(::cron::job, $cron_jobs) } Either you're missing something or I am. I see nothing in that class that would inherently preclude it being composed. In particular, the two class declarations it contains both use 'include', not the resource-like class declaration syntax. If there is a barrier to composition it would be related to composition with another class that declares some of the same resources. That problem has a solution, however: factor out the multiply-declared resources into their own class or classes, which the then-composable classes declare instead of declaring the resources directly. Duplicate declaration: Apache::Listen[80] is already declared in file ... As I said, a duplicate resource. As I said, factor it (and the
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
On Monday, May 19, 2014 1:42:08 PM UTC-5, Jesse Cotton wrote: On Monday, May 19, 2014 10:18:11 AM UTC-4, jcbollinger wrote: On Friday, May 16, 2014 10:55:35 AM UTC-5, Jesse Cotton wrote: I work with Matt and am filling in for him since I posed this question to him originally. Our confusion really lies around how you layout profiles for a multi-function box. For example, suppose you have a role, CMS App Server that will host various CMS like Wordpress, Drupal, and others. They are all built on top of the same technologies, Apache, PHP, and MySQL. I don't believe you can build a separate profile for each CMS b/c they will conflict (at least within Puppet). Each will require a certain set of php modules and settings, apache modules and settings, etc. So do you build a single profile like profile::wordpress_drupal_cms3_cm4 or do you build a profile::apachefpm profile? The later seems more logical to me however you lose the ability to define profile specific hiera variables b/c profile::apachefpm is generic. You can declare the same class multiple times, provided that you use the 'include' function or its cousins to do so. That is a tremendously useful technique for managing configurations where multiple facilities share requirements on some of the same core resources. Using 'include' instead of resource-like class declarations means you must feed data to your classes via hiera instead of via your manifests, but separating your data from your manifests is good practice anyway. That's what Christopher Wood was pointing you toward, and it is good advice. We're aware of most of this and agree with most of this. However when you always call include, you lose the ability to say a particular hiera variable is attached to the profile. For example If you define: class profile::apache_phpfpm { include ::apache } with the following in hiera: apache::keepalive = 1 keepalive = 1 applies anywhere apache is included vs class profile::apache_phpfpm ( $keepalive = 1 ) { class { ::apache: keepalive = $keepalive } } profile::apache_phpfpm::keepalive = 1 So with the later you can build a somewhat self-contained profile. With the former you have to set variables globally or on a node. That is what your data hierarchy is for. Hiera does not limit you to only global and per-node data. You can define as many hierarchy levels as you need, each grouping your nodes in its own way (though it usually makes the most sense for the groups at each level to be subsets of the groups at the next lower level). And the foregoing is based on using only the built-in YAML back end. Hiera supports pluggable back ends, usable together or separately. A custom back end can employ whatever lookup or computation you want to serve whichever data you choose. It is thus possible to find and/or write composable classes and modules for managing the components you want. As David Schmitt observes, composability is not automatic, but I don't see why your particular case of an apache-based PHP app server with firewall rules and a specific collection of PHP and apache modules should present any special problem. Thus, the answer to your question is neither. You build a drupal profile, and a wordpress profile, etc., and you use them all together. So below is an example for Drupal. It could literally be cloned for Wordpress and Joomla. Unfortunately, b/c of the class declarations, it is not composable. class profile::drupal ( $apache_listen = ['80'], $apache_name_virtual_hosts = ['*:80'], $apache_modules = ['fastcgi'], $apache_fastcgi_servers = { 'www' = { host = '127.0.0.1:9000', faux_path = '/var/www/php.fcgi', alias = '/php.fcgi', file_type = 'application/x-httpd-php' } }, $phpfpm_pools = { 'www' = { listen = '127.0.0.1:9000', user = 'apache', pm_max_requests = 500, catch_workers_output = 'no', php_admin_value = {}, php_value = {} } }, $php_modules = [], $firewall_rules = {}, $backup_jobs = {}, $cron_jobs = {} ) { include ::apache ::apache::listen { $apache_listen: } ::apache::namevirtualhost { $apache_name_virtual_hosts: } ::apache::mod { $apache_modules: } create_resources(::apache::fastcgi::server, $apache_fastcgi_servers) include ::php::fpm::daemon create_resources(::php::fpm::conf, $phpfpm_pools) ::php::module { $php_modules: } ~ Service['php-fpm'] # So the apache user is created before # php-fpm starts Class['::apache'] - Class['::php::fpm::daemon'] create_resources(firewall, $firewall_rules) create_resources(::duplicity::job, $backup_jobs) create_resources(::cron::job, $cron_jobs) } Either you're missing something or I am. I see nothing in that class that would inherently preclude it being composed. In particular, the two class
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
We're aware of most of this and agree with most of this. However when you always call include, you lose the ability to say a particular hiera variable is attached to the profile. For example If you define: class profile::apache_phpfpm { include ::apache } with the following in hiera: apache::keepalive = 1 keepalive = 1 applies anywhere apache is included vs class profile::apache_phpfpm ( $keepalive = 1 ) { class { ::apache: keepalive = $keepalive } } profile::apache_phpfpm::keepalive = 1 So with the later you can build a somewhat self-contained profile. With the former you have to set variables globally or on a node. That is what your data hierarchy is for. Hiera does not limit you to only global and per-node data. You can define as many hierarchy levels as you need, each grouping your nodes in its own way (though it usually makes the most sense for the groups at each level to be subsets of the groups at the next lower level). Understood. However without creating a custom fact, I am not aware of a way to say 'apache::keepalive = 1' only applies when a node has the profile 'profile::apache_phpfpm'. And the foregoing is based on using only the built-in YAML back end. Hiera supports pluggable back ends, usable together or separately. A custom back end can employ whatever lookup or computation you want to serve whichever data you choose. We're really trying to avoid this. It is thus possible to find and/or write composable classes and modules for managing the components you want. As David Schmitt observes, composability is not automatic, but I don't see why your particular case of an apache-based PHP app server with firewall rules and a specific collection of PHP and apache modules should present any special problem. Thus, the answer to your question is neither. You build a drupal profile, and a wordpress profile, etc., and you use them all together. So below is an example for Drupal. It could literally be cloned for Wordpress and Joomla. Unfortunately, b/c of the class declarations, it is not composable. class profile::drupal ( $apache_listen = ['80'], $apache_name_virtual_hosts = ['*:80'], $apache_modules = ['fastcgi'], $apache_fastcgi_servers = { 'www' = { host = '127.0.0.1:9000', faux_path = '/var/www/php.fcgi', alias = '/php.fcgi', file_type = 'application/x-httpd-php' } }, $phpfpm_pools = { 'www' = { listen = '127.0.0.1:9000', user = 'apache', pm_max_requests = 500, catch_workers_output = 'no', php_admin_value = {}, php_value = {} } }, $php_modules = [], $firewall_rules = {}, $backup_jobs = {}, $cron_jobs = {} ) { include ::apache ::apache::listen { $apache_listen: } ::apache::namevirtualhost { $apache_name_virtual_hosts: } ::apache::mod { $apache_modules: } create_resources(::apache::fastcgi::server, $apache_fastcgi_servers) include ::php::fpm::daemon create_resources(::php::fpm::conf, $phpfpm_pools) ::php::module { $php_modules: } ~ Service['php-fpm'] # So the apache user is created before # php-fpm starts Class['::apache'] - Class['::php::fpm::daemon'] create_resources(firewall, $firewall_rules) create_resources(::duplicity::job, $backup_jobs) create_resources(::cron::job, $cron_jobs) } Either you're missing something or I am. I see nothing in that class that would inherently preclude it being composed. In particular, the two class declarations it contains both use 'include', not the resource-like class declaration syntax. If there is a barrier to composition it would be related to composition with another class that declares some of the same resources. That problem has a solution, however: factor out the multiply-declared resources into their own class or classes, which the then-composable classes declare instead of declaring the resources directly. Duplicate declaration: Apache::Listen[80] is already declared in file ... Yes, the class could be broken into separate classes but this only exacerbates the issue of assigning variables based on the profile. How are others constructing their hierarchies when using the Role Profile pattern? Are you assigning variables based on a nodes role or profiles? If so, how? Custom facts, parameterized classes as we've done in the example above? John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/b6df42fd-0a33-4b63-ba8c-b56b54986d3e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
On Tue, May 20, 2014 at 10:54:15AM -0700, Jesse Cotton wrote: We're aware of most of this and agree with most of this. However when you always call include, you lose the ability to say a particular hiera variable is attached to the profile. For example If you define: class profile::apache_phpfpm { include ::apache } with the following in hiera: apache::keepalive = 1 keepalive = 1 applies anywhere apache is included vs class profile::apache_phpfpm ( $keepalive = 1 ) { class { ::apache: keepalive = $keepalive } } profile::apache_phpfpm::keepalive = 1 So with the later you can build a somewhat self-contained profile. With the former you have to set variables globally or on a node. That is what your data hierarchy is for. Hiera does not limit you to only global and per-node data. You can define as many hierarchy levels as you need, each grouping your nodes in its own way (though it usually makes the most sense for the groups at each level to be subsets of the groups at the next lower level). Understood. However without creating a custom fact, I am not aware of a way to say 'apache::keepalive = 1' only applies when a node has the profile 'profile::apache_phpfpm'. It sounds like there's some generalizable rule where some hosts get certain settings (ex: keepalive) setting and some don't. Perhaps this could be broken out into another hiera layer? Cheapest possible example: :hierarchy: - 'nodes/%{fqdn}' - 'roles/%{role}' - 'common' There's this workflow: puppetmaster uses ENC to get the node's role (named mytype here) the role is used to name the class for hiera_include (role::mytype) mytype.yaml is also a yaml file as part of that hiera level Then in the yaml for phpfpm and needkeepalive you have apache::keepalive: true and in common.yaml you have apache::keepalive: false. And the foregoing is based on using only the built-in YAML back end. Hiera supports pluggable back ends, usable together or separately. A custom back end can employ whatever lookup or computation you want to serve whichever data you choose. We're really trying to avoid this. It is thus possible to find and/or write composable classes and modules for managing the components you want. As David Schmitt observes, composability is not automatic, but I don't see why your particular case of an apache-based PHP app server with firewall rules and a specific collection of PHP and apache modules should present any special problem. Thus, the answer to your question is neither. You build a drupal profile, and a wordpress profile, etc., and you use them all together. So below is an example for Drupal. It could literally be cloned for Wordpress and Joomla. Unfortunately, b/c of the class declarations, it is not composable. class profile::drupal ( $apache_listen = ['80'], $apache_name_virtual_hosts = ['*:80'], $apache_modules = ['fastcgi'], $apache_fastcgi_servers = { 'www' = { host = '[1]127.0.0.1:9000', faux_path = '/var/www/php.fcgi', alias = '/php.fcgi', file_type = 'application/x-httpd-php' } }, $phpfpm_pools = { 'www' = { listen = '[2]127.0.0.1:9000', user = 'apache', pm_max_requests = 500, catch_workers_output = 'no', php_admin_value = {}, php_value = {} } }, $php_modules = [], $firewall_rules = {}, $backup_jobs = {}, $cron_jobs = {} ) { include ::apache ::apache::listen { $apache_listen: } ::apache::namevirtualhost { $apache_name_virtual_hosts: } ::apache::mod { $apache_modules: } create_resources(::apache::fastcgi::server, $apache_fastcgi_servers) include ::php::fpm::daemon create_resources(::php::fpm::conf, $phpfpm_pools) ::php::module { $php_modules: } ~ Service['php-fpm'] # So the apache user is created before # php-fpm starts Class['::apache'] - Class['::php::fpm::daemon'] create_resources(firewall, $firewall_rules) create_resources(::duplicity::job, $backup_jobs) create_resources(::cron::job, $cron_jobs) } Either you're missing something or I am. I see nothing in that class that would inherently preclude it being composed. In particular, the two class declarations it contains both use 'include', not the resource-like class declaration
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
Out of curiosity, what exactly are all the components you are trying to install, e.g. CMSes? Drupal? WordPress? Joomla? Is that database going to be on the same system? Note, segue here before I divulge a solution, perhaps sacrilegious, is that the role/profile might not be the best design pattern for, as it seems to be a top-down with functional decomposition, rather than a bottom-up approach (e.g. starting with apache, app-engine child class, then cms child class, each one inheriting properties and actions of the parent). There are other OOP design patterns (yet to be documented) that could work for Puppet. Still, here's a way to maybe get this to work with roles/profiles: You could write maybe a fat_php module (not profile), but has parameters to turn on/off the features you want through parametrization. Thus you can have this fat_php module support all the requirements to install everything under the sun, then a profile will toggle them on and off through parameters. Thus, a package's ensure attribute can be set to present depending on what CMS the profile will build. The role would call whatever profile CMSes that are needed for that role, so it can call two or more CMS profiles. Each one will toggle various options, such as a package's ensure to present, appropriate to the that CMS. When the catalog is compiled, and 2+ CMSes were enabled, packages will only be installed once. If any custom code should have some way to avoid collisions, if it has already been invoked previously, so you need to be careful to ensure it is idempotent, especially for exec { command: }. When you advanced the code base to use Hiera, you can just use class includes, to have pure idempotence, and avoid specifying parameters through class { param=value }, which might cause some unexpected, ala honey badger (don't give a shit), results when classes might try to get installed twice or out of order between profiles included in the roles. Similarly, you can have a fat MySQL module which has the needed schema and account configurations needed to make it work with whatever CMS. These sets of configurations and actions are toggled on/off, and which will be set by the profile (or hiera config related to the profile). On Saturday, May 17, 2014 5:39:05 AM UTC-7, Jesse Cotton wrote: Yes it is feasible :) So really the server is just a PHP app server. What makes it useable for all of the CMSs is that the correct set of apache and php modules are installed, and it has the correct firewall rules, cron jobs and backup jobs configured. With this in mind, my inclination is to build a cms_app_server profile but that seems more like a role then a profile. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a4b895df-02b6-4014-bbce-176d93c3ee07%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
On Friday, May 16, 2014 10:55:35 AM UTC-5, Jesse Cotton wrote: I work with Matt and am filling in for him since I posed this question to him originally. Our confusion really lies around how you layout profiles for a multi-function box. For example, suppose you have a role, CMS App Server that will host various CMS like Wordpress, Drupal, and others. They are all built on top of the same technologies, Apache, PHP, and MySQL. I don't believe you can build a separate profile for each CMS b/c they will conflict (at least within Puppet). Each will require a certain set of php modules and settings, apache modules and settings, etc. So do you build a single profile like profile::wordpress_drupal_cms3_cm4 or do you build a profile::apachefpm profile? The later seems more logical to me however you lose the ability to define profile specific hiera variables b/c profile::apachefpm is generic. You can declare the same class multiple times, provided that you use the 'include' function or its cousins to do so. That is a tremendously useful technique for managing configurations where multiple facilities share requirements on some of the same core resources. Using 'include' instead of resource-like class declarations means you must feed data to your classes via hiera instead of via your manifests, but separating your data from your manifests is good practice anyway. That's what Christopher Wood was pointing you toward, and it is good advice. It is thus possible to find and/or write composable classes and modules for managing the components you want. As David Schmitt observes, composability is not automatic, but I don't see why your particular case of an apache-based PHP app server with firewall rules and a specific collection of PHP and apache modules should present any special problem. Thus, the answer to your question is neither. You build a drupal profile, and a wordpress profile, etc., and you use them all together. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/56d766c8-64b4-4d94-8134-4fb8d0d516f1%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
On Monday, May 19, 2014 10:18:11 AM UTC-4, jcbollinger wrote: On Friday, May 16, 2014 10:55:35 AM UTC-5, Jesse Cotton wrote: I work with Matt and am filling in for him since I posed this question to him originally. Our confusion really lies around how you layout profiles for a multi-function box. For example, suppose you have a role, CMS App Server that will host various CMS like Wordpress, Drupal, and others. They are all built on top of the same technologies, Apache, PHP, and MySQL. I don't believe you can build a separate profile for each CMS b/c they will conflict (at least within Puppet). Each will require a certain set of php modules and settings, apache modules and settings, etc. So do you build a single profile like profile::wordpress_drupal_cms3_cm4 or do you build a profile::apachefpm profile? The later seems more logical to me however you lose the ability to define profile specific hiera variables b/c profile::apachefpm is generic. You can declare the same class multiple times, provided that you use the 'include' function or its cousins to do so. That is a tremendously useful technique for managing configurations where multiple facilities share requirements on some of the same core resources. Using 'include' instead of resource-like class declarations means you must feed data to your classes via hiera instead of via your manifests, but separating your data from your manifests is good practice anyway. That's what Christopher Wood was pointing you toward, and it is good advice. We're aware of most of this and agree with most of this. However when you always call include, you lose the ability to say a particular hiera variable is attached to the profile. For example If you define: class profile::apache_phpfpm { include ::apache } with the following in hiera: apache::keepalive = 1 keepalive = 1 applies anywhere apache is included vs class profile::apache_phpfpm ( $keepalive = 1 ) { class { ::apache: keepalive = $keepalive } } profile::apache_phpfpm::keepalive = 1 So with the later you can build a somewhat self-contained profile. With the former you have to set variables globally or on a node. It is thus possible to find and/or write composable classes and modules for managing the components you want. As David Schmitt observes, composability is not automatic, but I don't see why your particular case of an apache-based PHP app server with firewall rules and a specific collection of PHP and apache modules should present any special problem. Thus, the answer to your question is neither. You build a drupal profile, and a wordpress profile, etc., and you use them all together. So below is an example for Drupal. It could literally be cloned for Wordpress and Joomla. Unfortunately, b/c of the class declarations, it is not composable. class profile::drupal ( $apache_listen = ['80'], $apache_name_virtual_hosts = ['*:80'], $apache_modules = ['fastcgi'], $apache_fastcgi_servers = { 'www' = { host = '127.0.0.1:9000', faux_path = '/var/www/php.fcgi', alias = '/php.fcgi', file_type = 'application/x-httpd-php' } }, $phpfpm_pools = { 'www' = { listen = '127.0.0.1:9000', user = 'apache', pm_max_requests = 500, catch_workers_output = 'no', php_admin_value = {}, php_value = {} } }, $php_modules = [], $firewall_rules = {}, $backup_jobs = {}, $cron_jobs = {} ) { include ::apache ::apache::listen { $apache_listen: } ::apache::namevirtualhost { $apache_name_virtual_hosts: } ::apache::mod { $apache_modules: } create_resources(::apache::fastcgi::server, $apache_fastcgi_servers) include ::php::fpm::daemon create_resources(::php::fpm::conf, $phpfpm_pools) ::php::module { $php_modules: } ~ Service['php-fpm'] # So the apache user is created before # php-fpm starts Class['::apache'] - Class['::php::fpm::daemon'] create_resources(firewall, $firewall_rules) create_resources(::duplicity::job, $backup_jobs) create_resources(::cron::job, $cron_jobs) } John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/6a4218c6-fb00-4434-aa56-56de3fb33eff%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
You could do global variables, but you couldn't one set the hierarchy to the profile level? Thus you isolate variables in preference to the profile, rather than a global mismash. Stick with parametrization for now if this works, refactor later after getting it working, as you'll have a functional base state. We're aware of most of this and agree with most of this. However when you always call include, you lose the ability to say a particular hiera variable is attached to the profile. For example If you define: class profile::apache_phpfpm { include ::apache } with the following in hiera: apache::keepalive = 1 keepalive = 1 applies anywhere apache is included vs class profile::apache_phpfpm ( $keepalive = 1 ) { class { ::apache: keepalive = $keepalive } } profile::apache_phpfpm::keepalive = 1 I am not sure about this... Personally, I get scared of having too many sub-classes beyond install.pp, config.pp, service.pp, etc. It might make those files verbose, but then it is easier to contain and debug. Afterwards, parts into separate pieces, if this helps. Is cron a separate class? Personally, I put my bottom level modules into their product role, which may require cron functionality. I avoid crafting classes based on their functionality, as this would be the same as having a class called package that accepts an array packages to be installed. It gets hard to identify what is involved in installing a product or scheduling cleanup tasks for the same product; the code to implement a particular product gets scattered across many files, rather than self-contained in one module. So below is an example for Drupal. It could literally be cloned for Wordpress and Joomla. Unfortunately, b/c of the class declarations, it is not composable. class profile::drupal ( $apache_listen = ['80'], $apache_name_virtual_hosts = ['*:80'], $apache_modules = ['fastcgi'], $apache_fastcgi_servers = { 'www' = { host = '127.0.0.1:9000', faux_path = '/var/www/php.fcgi', alias = '/php.fcgi', file_type = 'application/x-httpd-php' } }, $phpfpm_pools = { 'www' = { listen = '127.0.0.1:9000', user = 'apache', pm_max_requests = 500, catch_workers_output = 'no', php_admin_value = {}, php_value = {} } }, $php_modules = [], $firewall_rules = {}, $backup_jobs = {}, $cron_jobs = {} ) { include ::apache ::apache::listen { $apache_listen: } ::apache::namevirtualhost { $apache_name_virtual_hosts: } ::apache::mod { $apache_modules: } create_resources(::apache::fastcgi::server, $apache_fastcgi_servers) include ::php::fpm::daemon create_resources(::php::fpm::conf, $phpfpm_pools) ::php::module { $php_modules: } ~ Service['php-fpm'] # So the apache user is created before # php-fpm starts Class['::apache'] - Class['::php::fpm::daemon'] create_resources(firewall, $firewall_rules) create_resources(::duplicity::job, $backup_jobs) create_resources(::cron::job, $cron_jobs) } John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/91d7bbf0-e290-444a-8741-07a1c763d325%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
Perhaps I am overlooking something. How would you construct the hierarchy so you could set apache::keepalive for this profile? class profile::apache_phpfpm { include ::apache } You could do global variables, but you couldn't one set the hierarchy to the profile level? Thus you isolate variables in preference to the profile, rather than a global mismash. Stick with parametrization for now if this works, refactor later after getting it working, as you'll have a functional base state. We're aware of most of this and agree with most of this. However when you always call include, you lose the ability to say a particular hiera variable is attached to the profile. For example If you define: class profile::apache_phpfpm { include ::apache } with the following in hiera: apache::keepalive = 1 keepalive = 1 applies anywhere apache is included vs class profile::apache_phpfpm ( $keepalive = 1 ) { class { ::apache: keepalive = $keepalive } } profile::apache_phpfpm::keepalive = 1 I am not sure about this... Personally, I get scared of having too many sub-classes beyond install.pp, config.pp, service.pp, etc. It might make those files verbose, but then it is easier to contain and debug. Afterwards, parts into separate pieces, if this helps. Is cron a separate class? Personally, I put my bottom level modules into their product role, which may require cron functionality. I avoid crafting classes based on their functionality, as this would be the same as having a class called package that accepts an array packages to be installed. It gets hard to identify what is involved in installing a product or scheduling cleanup tasks for the same product; the code to implement a particular product gets scattered across many files, rather than self-contained in one module. I am not sure I understand what you're suggesting here. The point of the create_resources calls at the very bottom is so we can attach backup jobs, cron jobs, firewall rules to the profile. You seem to be suggesting we should assign these at the role level. Is that correct? I see valid reasons for doing it either way but experience may dictate one is better then the other. Yes, apache, php, cron, duplicity, and firewall are all separate classes. So below is an example for Drupal. It could literally be cloned for Wordpress and Joomla. Unfortunately, b/c of the class declarations, it is not composable. class profile::drupal ( $apache_listen = ['80'], $apache_name_virtual_hosts = ['*:80'], $apache_modules = ['fastcgi'], $apache_fastcgi_servers = { 'www' = { host = '127.0.0.1:9000', faux_path = '/var/www/php.fcgi', alias = '/php.fcgi', file_type = 'application/x-httpd-php' } }, $phpfpm_pools = { 'www' = { listen = '127.0.0.1:9000', user = 'apache', pm_max_requests = 500, catch_workers_output = 'no', php_admin_value = {}, php_value = {} } }, $php_modules = [], $firewall_rules = {}, $backup_jobs = {}, $cron_jobs = {} ) { include ::apache ::apache::listen { $apache_listen: } ::apache::namevirtualhost { $apache_name_virtual_hosts: } ::apache::mod { $apache_modules: } create_resources(::apache::fastcgi::server, $apache_fastcgi_servers) include ::php::fpm::daemon create_resources(::php::fpm::conf, $phpfpm_pools) ::php::module { $php_modules: } ~ Service['php-fpm'] # So the apache user is created before # php-fpm starts Class['::apache'] - Class['::php::fpm::daemon'] create_resources(firewall, $firewall_rules) create_resources(::duplicity::job, $backup_jobs) create_resources(::cron::job, $cron_jobs) } John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/d499acf3-d7d7-401f-8692-4d9c44b02e6c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
Hi, To answer the original qustion with a somewhat cheap truism: It is possible, but not required that profiles are composable. E.g. profile::apache_server and profile::mysql_server should be trivially composable, because they manage disjunct sets of resources. On the other hand, profile::apache_server and profile::nginx_server might not be composable, because both need to use port 80, which cannot be shared. So to the question of building composable profiles, if your profiles often need a specific set of php modules, it might ease the situation by creating a profile::fat_php_installation, which contains all required php modules and becomes the focal point for intra-cms-php-development. A different approach would be finegrained classes for each php module, which can be included multiple times across many profiles. A totally different approach would be to create local php installations for each CMS which do not talk to each other at all, leaving the CMS consumer to choose the exact patch level and composition required. Surely, there are several other possibilities. Which one is the right one really depends on the structure of your development team and systems. Regards, David On 2014-05-17 14:39, Jesse Cotton wrote: Yes it is feasible :) So really the server is just a PHP app server. What makes it useable for all of the CMSs is that the correct set of apache and php modules are installed, and it has the correct firewall rules, cron jobs and backup jobs configured. With this in mind, my inclination is to build a cms_app_server profile but that seems more like a role then a profile. On Friday, May 16, 2014 3:08:00 PM UTC-4, Joaquin Menchaca wrote: Hello: This might be a dumb question, but can you build a system manually with the various CMSes? Are there a cross-set of technologies (Apache, PHP, MySQL) that will work on all CMSes in your organization? I would think in terms in how you would build these systems manually w/o puppet in some organized way, then see how this can fit into puppet. If you can build some reference platform that can host some or all of these, then you have your basic profile for that wonder-web-box system. Complexity might come in when you start separating, such as putting php in it's own module and apache in another, then sequencing them in profile. You can use augues to scissor in php specific configurations (or alternatively creative erb ruby code to other configurations). This is what I do for a nginx + php-fm configuration. class profile::webserver { class { nginx: } - class { php: } } On Friday, May 16, 2014 8:55:35 AM UTC-7, Jesse Cotton wrote: I work with Matt and am filling in for him since I posed this question to him originally. Our confusion really lies around how you layout profiles for a multi-function box. For example, suppose you have a role, CMS App Server that will host various CMS like Wordpress, Drupal, and others. They are all built on top of the same technologies, Apache, PHP, and MySQL. I don't believe you can build a separate profile for each CMS b/c they will conflict (at least within Puppet). Each will require a certain set of php modules and settings, apache modules and settings, etc. So do you build a single profile like profile::wordpress_drupal_cms3_cm4 or do you build a profile::apachefpm profile? The later seems more logical to me however you lose the ability to define profile specific hiera variables b/c profile::apachefpm is generic. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com mailto:puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/be9a1775-2d62-49f2-8b25-ae072729c059%40googlegroups.com https://groups.google.com/d/msgid/puppet-users/be9a1775-2d62-49f2-8b25-ae072729c059%40googlegroups.com?utm_medium=emailutm_source=footer. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5378A048.7010204%40dasz.at. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
Yes it is feasible :) So really the server is just a PHP app server. What makes it useable for all of the CMSs is that the correct set of apache and php modules are installed, and it has the correct firewall rules, cron jobs and backup jobs configured. With this in mind, my inclination is to build a cms_app_server profile but that seems more like a role then a profile. On Friday, May 16, 2014 3:08:00 PM UTC-4, Joaquin Menchaca wrote: Hello: This might be a dumb question, but can you build a system manually with the various CMSes? Are there a cross-set of technologies (Apache, PHP, MySQL) that will work on all CMSes in your organization? I would think in terms in how you would build these systems manually w/o puppet in some organized way, then see how this can fit into puppet. If you can build some reference platform that can host some or all of these, then you have your basic profile for that wonder-web-box system. Complexity might come in when you start separating, such as putting php in it's own module and apache in another, then sequencing them in profile. You can use augues to scissor in php specific configurations (or alternatively creative erb ruby code to other configurations). This is what I do for a nginx + php-fm configuration. class profile::webserver { class { nginx: } - class { php: } } On Friday, May 16, 2014 8:55:35 AM UTC-7, Jesse Cotton wrote: I work with Matt and am filling in for him since I posed this question to him originally. Our confusion really lies around how you layout profiles for a multi-function box. For example, suppose you have a role, CMS App Server that will host various CMS like Wordpress, Drupal, and others. They are all built on top of the same technologies, Apache, PHP, and MySQL. I don't believe you can build a separate profile for each CMS b/c they will conflict (at least within Puppet). Each will require a certain set of php modules and settings, apache modules and settings, etc. So do you build a single profile like profile::wordpress_drupal_cms3_cm4 or do you build a profile::apachefpm profile? The later seems more logical to me however you lose the ability to define profile specific hiera variables b/c profile::apachefpm is generic. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/be9a1775-2d62-49f2-8b25-ae072729c059%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
I work with Matt and am filling in for him since I posed this question to him originally. Our confusion really lies around how you layout profiles for a multi-function box. For example, suppose you have a role, CMS App Server that will host various CMS like Wordpress, Drupal, and others. They are all built on top of the same technologies, Apache, PHP, and MySQL. I don't believe you can build a separate profile for each CMS b/c they will conflict (at least within Puppet). Each will require a certain set of php modules and settings, apache modules and settings, etc. So do you build a single profile like profile::wordpress_drupal_cms3_cm4 or do you build a profile::apachefpm profile? The later seems more logical to me however you lose the ability to define profile specific hiera variables b/c profile::apachefpm is generic. On Thursday, May 15, 2014 1:22:02 AM UTC-4, mjus...@gmail.com wrote: Hi all, We use the roles/profiles/components model originally suggested by Craig Dunn fairly heavily. In our case: - The role is a business name, like Application X App Server - The profile is the technical name, like Base Components or Webserver - The components are either wrapper classes around modules or modules themselves, like PHP or Apache. For the most part, this works well. We can have, for example: - MyFace Application Server - Base Components - SSSD - Sudo - NTP - PHP Webserver - PHP - Apache - PHP-FPM - Memcache However, we're running into trouble how to handle the situation where you're running a box with multiple functions... for example, WordPress and Drupal. In that case, how do you handle configuration conflicts? On the surface, it seems like we would create a more generic profile like PHP Webserver (like I did in the above example). If I do this, however, I lose the ability to define profile specific variables such as firewall rules, cron jobs, etc. Any thoughts on this? Best, Matt -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/457a1f16-7d8f-4bad-9d93-1f666127fb89%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
Hello Jesse Matt. This might be a dumb question, but can you build a system manually with the various CMSes? Are there a cross-set of technologies (Apache, PHP, MySQL) that will work on all CMSes in your organization? I would think in terms in how you would build these systems manually w/o puppet in some organized way, then see how this can fit into puppet. If you can build some reference platform that can host some or all of these, then you have your basic profile for that wonder-web-box system. Complexity might come in when you start separating, such as putting php in it's own module and apache in another, then sequencing them in profile. You can use augues to scissor php specific configurations. This is what I do for a nginx + php-fm configuration. class profile::webserver { class { nginx: } - class { php: } } On Friday, May 16, 2014 8:55:35 AM UTC-7, Jesse Cotton wrote: I work with Matt and am filling in for him since I posed this question to him originally. Our confusion really lies around how you layout profiles for a multi-function box. For example, suppose you have a role, CMS App Server that will host various CMS like Wordpress, Drupal, and others. They are all built on top of the same technologies, Apache, PHP, and MySQL. I don't believe you can build a separate profile for each CMS b/c they will conflict (at least within Puppet). Each will require a certain set of php modules and settings, apache modules and settings, etc. So do you build a single profile like profile::wordpress_drupal_cms3_cm4 or do you build a profile::apachefpm profile? The later seems more logical to me however you lose the ability to define profile specific hiera variables b/c profile::apachefpm is generic. On Thursday, May 15, 2014 1:22:02 AM UTC-4, mjus...@gmail.com wrote: Hi all, We use the roles/profiles/components model originally suggested by Craig Dunn fairly heavily. In our case: - The role is a business name, like Application X App Server - The profile is the technical name, like Base Components or Webserver - The components are either wrapper classes around modules or modules themselves, like PHP or Apache. For the most part, this works well. We can have, for example: - MyFace Application Server - Base Components - SSSD - Sudo - NTP - PHP Webserver - PHP - Apache - PHP-FPM - Memcache However, we're running into trouble how to handle the situation where you're running a box with multiple functions... for example, WordPress and Drupal. In that case, how do you handle configuration conflicts? On the surface, it seems like we would create a more generic profile like PHP Webserver (like I did in the above example). If I do this, however, I lose the ability to define profile specific variables such as firewall rules, cron jobs, etc. Any thoughts on this? Best, Matt -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/b5e35060-be8e-4dcb-9755-159c20e1fc9d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
Hello: This might be a dumb question, but can you build a system manually with the various CMSes? Are there a cross-set of technologies (Apache, PHP, MySQL) that will work on all CMSes in your organization? I would think in terms in how you would build these systems manually w/o puppet in some organized way, then see how this can fit into puppet. If you can build some reference platform that can host some or all of these, then you have your basic profile for that wonder-web-box system. Complexity might come in when you start separating, such as putting php in it's own module and apache in another, then sequencing them in profile. You can use augues to scissor in php specific configurations (or alternatively creative erb ruby code to other configurations). This is what I do for a nginx + php-fm configuration. class profile::webserver { class { nginx: } - class { php: } } On Friday, May 16, 2014 8:55:35 AM UTC-7, Jesse Cotton wrote: I work with Matt and am filling in for him since I posed this question to him originally. Our confusion really lies around how you layout profiles for a multi-function box. For example, suppose you have a role, CMS App Server that will host various CMS like Wordpress, Drupal, and others. They are all built on top of the same technologies, Apache, PHP, and MySQL. I don't believe you can build a separate profile for each CMS b/c they will conflict (at least within Puppet). Each will require a certain set of php modules and settings, apache modules and settings, etc. So do you build a single profile like profile::wordpress_drupal_cms3_cm4 or do you build a profile::apachefpm profile? The later seems more logical to me however you lose the ability to define profile specific hiera variables b/c profile::apachefpm is generic. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1b526acb-d961-484c-8525-4ded8f0c9472%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Re: Craig Dunn's Roles/Profiles/Components Conflicts
On Thursday, May 15, 2014 12:22:02 AM UTC-5, mjus...@gmail.com wrote: Hi all, We use the roles/profiles/components model originally suggested by Craig Dunn fairly heavily. In our case: - The role is a business name, like Application X App Server - The profile is the technical name, like Base Components or Webserver - The components are either wrapper classes around modules or modules themselves, like PHP or Apache. For the most part, this works well. We can have, for example: - MyFace Application Server - Base Components - SSSD - Sudo - NTP - PHP Webserver - PHP - Apache - PHP-FPM - Memcache However, we're running into trouble how to handle the situation where you're running a box with multiple functions... for example, WordPress and Drupal. In that case, how do you handle configuration conflicts? On the surface, it seems like we would create a more generic profile like PHP Webserver (like I did in the above example). If I do this, however, I lose the ability to define profile specific variables such as firewall rules, cron jobs, etc. Any thoughts on this? If you follow Dunn to the letter then each node has exactly one role. Therefore, if you want a box with multiple functions then you need to define a role that encompasses all of them. That's a useful exercise in itself, as it makes you think about *why* those functions are all served by the same machine. Suppose, then, that you have a role for what you're talking about, something such as Satellite Office Omnibus Server or whatever. You then proceed as normal for the pattern, by defining one or more profiles for machines having that role. Those profiles define all the needed components, just as your other profiles do. Presumably the profiles for such roles will be larger than most. I think the key hurdle here is defining appropriate roles. Multi-function computer isn't a role, it's just a vague description. Given genuine, coherent roles for these computers, I think everything else will fall out fairly naturally. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/442f6d7c-6696-420e-9933-1edd95c98f56%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.