[Puppet Users] Re: [Puppet-dev] [Puppet-Users] Puppet Platform 6 Update

[Martin Alfke]

> On 18. Jul 2018, at 00:43, Eric Sorenson  wrote:
> 
> On Jul 17, 2018, at 2:26 AM, Martin Alfke  wrote:
>> 
>>> On 17. Jul 2018, at 01:40, Eric Sorenson  wrote:
>>> 
>>> So my question is - 
>>> - do you current use/rely on 'gem install puppet' for your workflows? If 
>>> so, what do you do with it? (does anybody use a 'gem install puppet' as 
>>> their production "puppet agent" daemon?)
>> 
>> We install puppet as a gem in CI/CD unit testing.
>> 
> 
> Hi Martin! Does this use depend on types and providers in puppet's lib/ 
> directory? Or is it just having the core puppet code available?

AFAIK it is needed to compile the catalog using rspec-puppet. (I hope I am 
right on this).
On acceptance testing (with beaker) we install the OS puppet-agent package.

> 
> 
>>> - given the above, what would be the easiest/most intuitive way to get 
>>> those extracted types into your puppet installation? some ideas we've 
>>> kicked around are 
>>>  * a puppet type 'meta module' that, akin to a rpm/deb metapackage, doesn't 
>>> have content, just dependencies on the actual modules at particular pinned 
>>> versions that match the agent package versions
>>>  * a Puppetfile that you could point r10k at to get the modules installed
>>>  * individual gems for each of the extracted modules with Gemfile 
>>> dependencies (note: this is a Bad Idea™)
>> 
>> We need at least a note how we have to add the module with the separated 
>> types/providers.
> 
> Yes absolutely
> 
>> 
>>> 
>>> WDYT?
>>> --eric0
>>> 
>>> 
 On Jul 16, 2018, at 10:20 AM, Josh Cooper  wrote:
 
 I wanted to share some significant developments as we progress towards a 
 Puppet Platform 6 release. I encourage you to try out nightly builds 
 available in the puppet6 repos:
 
 http://nightlies.puppet.com/yum/puppet6-nightly/
 http://nightlies.puppet.com/apt/puppet6-nightly/
 http://nightlies.puppet.com/downloads/{mac,windows}/puppet6-nightly/
 
 1. Unvendoring Semantic Puppet
 
 Previously, the puppet repo, puppet-agent and puppetserver 
 vendored/packaged different versions of the semantic_puppet gem. We've 
 untangled that mess so that in Platform 6:
 
 * puppet has a runtime gem dependency on the semantic_puppet gem
 * puppet-agent bundles the semantic_puppet 1.0.2 gem
 * puppetserver no longer knows about puppet's transitive gem dependencies
 * we can bump the semantic_puppet version in puppet-agent in the future 
 without breaking puppetserver running on the same host. The same is true 
 for other puppet runtime gem dependencies like fast_gettext and multi_json.
 
 See https://tickets.puppetlabs.com/browse/PA-1880 for more details.
 
 2. Puppet Platform 6 requires Ruby 2.3
 
 Puppet Platform 6 requires Ruby 2.3 or up, so we can now use modern syntax 
 such as keyword arguments, dig, squiggly heredocs, etc. Puppet will error 
 when running on unsupported ruby versions such as 2.2, which went EOL on 
 March 31, 2018.
 
 Since puppetserver runs puppet code in a JRuby interpreter and JRuby 1.7 
 conforms to the 1.9.3 Ruby language, we first had to move puppetserver 
 from JRuby 1.7 to 9K. In Platform 5, we made it possible to opt into using 
 JRuby 9K. In Platform 6, we will drop JRuby 1.7 and only support JRuby 
 9.1.x.x, which conforms to Ruby 2.3.
 
 To ensure puppet code does not break puppetserver/JRuby, we've started 
 running puppet PRs against JRuby 9K in TravisCI.
 
 See https://tickets.puppetlabs.com/browse/PUP-6893 and 
 https://tickets.puppetlabs.com/browse/SERVER-2155 for more details.
 
 3. Intermediate CA improvements
 
 Currently, customers can set up Puppet to use an intermediate CA by 
 manually generating and distributing certificates and keys, installing 
 them in the proper locations on disk, for both the master and agent. This 
 is time intensive, error prone, and even once these certs have been put in 
 place, full validation using CRL chains was not possible.
 
 For Puppet 6, we we are making both tooling and functionality improvements 
 to this process. In this increment, we have implemented full validation 
 with chained certificates and CRLs, and we have changed the agent-side SSL 
 bootstrapping to automatically download these full chains from the master 
 and store and use them appropriately. It is now no longer necessary for 
 intermediate CA users to manually distribute SSL files to their agents. On 
 the server side, we are working to create a puppetserver CLI for setting 
 up and interacting with the CA. See 
 https://tickets.puppetlabs.com/browse/SERVER-2171.
 
 4. Server-stack containerization
 
 We’ve been working primarily on the automation and tooling to improve 
 building and shipping updated containers for the Puppet Platform server 
 components (puppet

[Puppet Users] Re: [Puppet-dev] [Puppet-Users] Puppet Platform 6 Update

[Eric Sorenson]

On Jul 17, 2018, at 2:26 AM, Martin Alfke  wrote:
> 
>> On 17. Jul 2018, at 01:40, Eric Sorenson > > wrote:
>> 
>> So my question is - 
>> - do you current use/rely on 'gem install puppet' for your workflows? If so, 
>> what do you do with it? (does anybody use a 'gem install puppet' as their 
>> production "puppet agent" daemon?)
> 
> We install puppet as a gem in CI/CD unit testing.
> 

Hi Martin! Does this use depend on types and providers in puppet's lib/ 
directory? Or is it just having the core puppet code available?


>> - given the above, what would be the easiest/most intuitive way to get those 
>> extracted types into your puppet installation? some ideas we've kicked 
>> around are 
>>  * a puppet type 'meta module' that, akin to a rpm/deb metapackage, doesn't 
>> have content, just dependencies on the actual modules at particular pinned 
>> versions that match the agent package versions
>>  * a Puppetfile that you could point r10k at to get the modules installed
>>  * individual gems for each of the extracted modules with Gemfile 
>> dependencies (note: this is a Bad Idea™)
> 
> We need at least a note how we have to add the module with the separated 
> types/providers.

Yes absolutely

> 
>> 
>> WDYT?
>> --eric0
>> 
>> 
>>> On Jul 16, 2018, at 10:20 AM, Josh Cooper  wrote:
>>> 
>>> I wanted to share some significant developments as we progress towards a 
>>> Puppet Platform 6 release. I encourage you to try out nightly builds 
>>> available in the puppet6 repos:
>>> 
>>> http://nightlies.puppet.com/yum/puppet6-nightly/
>>> http://nightlies.puppet.com/apt/puppet6-nightly/
>>> http://nightlies.puppet.com/downloads/{mac,windows}/puppet6-nightly/
>>> 
>>> 1. Unvendoring Semantic Puppet
>>> 
>>> Previously, the puppet repo, puppet-agent and puppetserver 
>>> vendored/packaged different versions of the semantic_puppet gem. We've 
>>> untangled that mess so that in Platform 6:
>>> 
>>> * puppet has a runtime gem dependency on the semantic_puppet gem
>>> * puppet-agent bundles the semantic_puppet 1.0.2 gem
>>> * puppetserver no longer knows about puppet's transitive gem dependencies
>>> * we can bump the semantic_puppet version in puppet-agent in the future 
>>> without breaking puppetserver running on the same host. The same is true 
>>> for other puppet runtime gem dependencies like fast_gettext and multi_json.
>>> 
>>> See https://tickets.puppetlabs.com/browse/PA-1880 for more details.
>>> 
>>> 2. Puppet Platform 6 requires Ruby 2.3
>>> 
>>> Puppet Platform 6 requires Ruby 2.3 or up, so we can now use modern syntax 
>>> such as keyword arguments, dig, squiggly heredocs, etc. Puppet will error 
>>> when running on unsupported ruby versions such as 2.2, which went EOL on 
>>> March 31, 2018.
>>> 
>>> Since puppetserver runs puppet code in a JRuby interpreter and JRuby 1.7 
>>> conforms to the 1.9.3 Ruby language, we first had to move puppetserver from 
>>> JRuby 1.7 to 9K. In Platform 5, we made it possible to opt into using JRuby 
>>> 9K. In Platform 6, we will drop JRuby 1.7 and only support JRuby 9.1.x.x, 
>>> which conforms to Ruby 2.3.
>>> 
>>> To ensure puppet code does not break puppetserver/JRuby, we've started 
>>> running puppet PRs against JRuby 9K in TravisCI.
>>> 
>>> See https://tickets.puppetlabs.com/browse/PUP-6893 and 
>>> https://tickets.puppetlabs.com/browse/SERVER-2155 for more details.
>>> 
>>> 3. Intermediate CA improvements
>>> 
>>> Currently, customers can set up Puppet to use an intermediate CA by 
>>> manually generating and distributing certificates and keys, installing them 
>>> in the proper locations on disk, for both the master and agent. This is 
>>> time intensive, error prone, and even once these certs have been put in 
>>> place, full validation using CRL chains was not possible.
>>> 
>>> For Puppet 6, we we are making both tooling and functionality improvements 
>>> to this process. In this increment, we have implemented full validation 
>>> with chained certificates and CRLs, and we have changed the agent-side SSL 
>>> bootstrapping to automatically download these full chains from the master 
>>> and store and use them appropriately. It is now no longer necessary for 
>>> intermediate CA users to manually distribute SSL files to their agents. On 
>>> the server side, we are working to create a puppetserver CLI for setting up 
>>> and interacting with the CA. See 
>>> https://tickets.puppetlabs.com/browse/SERVER-2171.
>>> 
>>> 4. Server-stack containerization
>>> 
>>> We’ve been working primarily on the automation and tooling to improve 
>>> building and shipping updated containers for the Puppet Platform server 
>>> components (puppetserver, puppetdb, and r10k). The build tooling for these 
>>> containers has moved into the individual project repos, and we’re getting 
>>> very close to having containers that will auto-publish to dockerhub.
>>> 
>>> We also have a number of workflow improvements planned for running the 
>>> server stack in a 

[Puppet Users] Re: [Puppet-dev] [Puppet-Users] Puppet Platform 6 Update

[Martin Alfke]

> On 17. Jul 2018, at 01:40, Eric Sorenson  wrote:
> 
> Another effort that's underway but not yet complete is the extraction of 
> non-core types/providers into modules. This addresses some long-standing 
> requests to, for example, be able to change the nagios types and OS-specific 
> resources without needing to get a full agent release out. The extracted 
> types will be available in a modulepath structure in the puppet agent 
> package, so (with a few targeted exceptions) there won't be any user-visible 
> changes to what's available when you get the package, but an implication that 
> hasn't really come up is around using Puppet in rubygem format. The extracted 
> types are available on github and on the forge as separate modules, so if you 
> currently use some of these extracted types, you'd need a way to get them 
> installed locally.
> 
> So my question is - 
> - do you current use/rely on 'gem install puppet' for your workflows? If so, 
> what do you do with it? (does anybody use a 'gem install puppet' as their 
> production "puppet agent" daemon?)

We install puppet as a gem in CI/CD unit testing.

> - given the above, what would be the easiest/most intuitive way to get those 
> extracted types into your puppet installation? some ideas we've kicked around 
> are 
>   * a puppet type 'meta module' that, akin to a rpm/deb metapackage, doesn't 
> have content, just dependencies on the actual modules at particular pinned 
> versions that match the agent package versions
>   * a Puppetfile that you could point r10k at to get the modules installed
>   * individual gems for each of the extracted modules with Gemfile 
> dependencies (note: this is a Bad Idea™)

We need at least a note how we have to add the module with the separated 
types/providers.

> 
> WDYT?
> --eric0
> 
> 
>> On Jul 16, 2018, at 10:20 AM, Josh Cooper  wrote:
>> 
>> I wanted to share some significant developments as we progress towards a 
>> Puppet Platform 6 release. I encourage you to try out nightly builds 
>> available in the puppet6 repos:
>> 
>> http://nightlies.puppet.com/yum/puppet6-nightly/
>> http://nightlies.puppet.com/apt/puppet6-nightly/
>> http://nightlies.puppet.com/downloads/{mac,windows}/puppet6-nightly/
>> 
>> 1. Unvendoring Semantic Puppet
>> 
>> Previously, the puppet repo, puppet-agent and puppetserver vendored/packaged 
>> different versions of the semantic_puppet gem. We've untangled that mess so 
>> that in Platform 6:
>> 
>> * puppet has a runtime gem dependency on the semantic_puppet gem
>> * puppet-agent bundles the semantic_puppet 1.0.2 gem
>> * puppetserver no longer knows about puppet's transitive gem dependencies
>> * we can bump the semantic_puppet version in puppet-agent in the future 
>> without breaking puppetserver running on the same host. The same is true for 
>> other puppet runtime gem dependencies like fast_gettext and multi_json.
>> 
>> See https://tickets.puppetlabs.com/browse/PA-1880 for more details.
>> 
>> 2. Puppet Platform 6 requires Ruby 2.3
>> 
>> Puppet Platform 6 requires Ruby 2.3 or up, so we can now use modern syntax 
>> such as keyword arguments, dig, squiggly heredocs, etc. Puppet will error 
>> when running on unsupported ruby versions such as 2.2, which went EOL on 
>> March 31, 2018.
>> 
>> Since puppetserver runs puppet code in a JRuby interpreter and JRuby 1.7 
>> conforms to the 1.9.3 Ruby language, we first had to move puppetserver from 
>> JRuby 1.7 to 9K. In Platform 5, we made it possible to opt into using JRuby 
>> 9K. In Platform 6, we will drop JRuby 1.7 and only support JRuby 9.1.x.x, 
>> which conforms to Ruby 2.3.
>> 
>> To ensure puppet code does not break puppetserver/JRuby, we've started 
>> running puppet PRs against JRuby 9K in TravisCI.
>> 
>> See https://tickets.puppetlabs.com/browse/PUP-6893 and 
>> https://tickets.puppetlabs.com/browse/SERVER-2155 for more details.
>> 
>> 3. Intermediate CA improvements
>> 
>> Currently, customers can set up Puppet to use an intermediate CA by manually 
>> generating and distributing certificates and keys, installing them in the 
>> proper locations on disk, for both the master and agent. This is time 
>> intensive, error prone, and even once these certs have been put in place, 
>> full validation using CRL chains was not possible.
>> 
>> For Puppet 6, we we are making both tooling and functionality improvements 
>> to this process. In this increment, we have implemented full validation with 
>> chained certificates and CRLs, and we have changed the agent-side SSL 
>> bootstrapping to automatically download these full chains from the master 
>> and store and use them appropriately. It is now no longer necessary for 
>> intermediate CA users to manually distribute SSL files to their agents. On 
>> the server side, we are working to create a puppetserver CLI for setting up 
>> and interacting with the CA. See 
>> https://tickets.puppetlabs.com/browse/SERVER-2171.
>> 
>> 4. Server-stack containerization
>> 
>> We’ve be

[Puppet Users] Re: [Puppet-dev] [Puppet-Users] Puppet Platform 6 Update

[Eric Sorenson]

Another effort that's underway but not yet complete is the extraction of 
non-core types/providers into modules. This addresses some long-standing 
requests to, for example, be able to change the nagios types and OS-specific 
resources without needing to get a full agent release out. The extracted types 
will be available in a modulepath structure in the puppet agent package, so 
(with a few targeted exceptions) there won't be any user-visible changes to 
what's available when you get the package, but an implication that hasn't 
really come up is around using Puppet in rubygem format. The extracted types 
are available on github and on the forge as separate modules, so if you 
currently use some of these extracted types, you'd need a way to get them 
installed locally.

So my question is - 
- do you current use/rely on 'gem install puppet' for your workflows? If so, 
what do you do with it? (does anybody use a 'gem install puppet' as their 
production "puppet agent" daemon?)
- given the above, what would be the easiest/most intuitive way to get those 
extracted types into your puppet installation? some ideas we've kicked around 
are 
  * a puppet type 'meta module' that, akin to a rpm/deb metapackage, doesn't 
have content, just dependencies on the actual modules at particular pinned 
versions that match the agent package versions
  * a Puppetfile that you could point r10k at to get the modules installed
  * individual gems for each of the extracted modules with Gemfile dependencies 
(note: this is a Bad Idea™)

WDYT?
--eric0


> On Jul 16, 2018, at 10:20 AM, Josh Cooper  wrote:
> 
> I wanted to share some significant developments as we progress towards a 
> Puppet Platform 6 release. I encourage you to try out nightly builds 
> available in the puppet6 repos:
> 
> http://nightlies.puppet.com/yum/puppet6-nightly/ 
> 
> http://nightlies.puppet.com/apt/puppet6-nightly/ 
> 
> http://nightlies.puppet.com/downloads/{mac,windows}/puppet6-nightly/ 
> 
> 
> 1. Unvendoring Semantic Puppet
> 
> Previously, the puppet repo, puppet-agent and puppetserver vendored/packaged 
> different versions of the semantic_puppet gem. We've untangled that mess so 
> that in Platform 6:
> 
> * puppet has a runtime gem dependency on the semantic_puppet gem
> * puppet-agent bundles the semantic_puppet 1.0.2 gem
> * puppetserver no longer knows about puppet's transitive gem dependencies
> * we can bump the semantic_puppet version in puppet-agent in the future 
> without breaking puppetserver running on the same host. The same is true for 
> other puppet runtime gem dependencies like fast_gettext and multi_json.
> 
> See https://tickets.puppetlabs.com/browse/PA-1880 
>  for more details.
> 
> 2. Puppet Platform 6 requires Ruby 2.3
> 
> Puppet Platform 6 requires Ruby 2.3 or up, so we can now use modern syntax 
> such as keyword arguments, dig, squiggly heredocs, etc. Puppet will error 
> when running on unsupported ruby versions such as 2.2, which went EOL on 
> March 31, 2018.
> 
> Since puppetserver runs puppet code in a JRuby interpreter and JRuby 1.7 
> conforms to the 1.9.3 Ruby language, we first had to move puppetserver from 
> JRuby 1.7 to 9K. In Platform 5, we made it possible to opt into using JRuby 
> 9K. In Platform 6, we will drop JRuby 1.7 and only support JRuby 9.1.x.x, 
> which conforms to Ruby 2.3.
> 
> To ensure puppet code does not break puppetserver/JRuby, we've started 
> running puppet PRs against JRuby 9K in TravisCI.
> 
> See https://tickets.puppetlabs.com/browse/PUP-6893 
>  and 
> https://tickets.puppetlabs.com/browse/SERVER-2155 
>  for more details.
> 
> 3. Intermediate CA improvements
> 
> Currently, customers can set up Puppet to use an intermediate CA by manually 
> generating and distributing certificates and keys, installing them in the 
> proper locations on disk, for both the master and agent. This is time 
> intensive, error prone, and even once these certs have been put in place, 
> full validation using CRL chains was not possible.
> 
> For Puppet 6, we we are making both tooling and functionality improvements to 
> this process. In this increment, we have implemented full validation with 
> chained certificates and CRLs, and we have changed the agent-side SSL 
> bootstrapping to automatically download these full chains from the master and 
> store and use them appropriately. It is now no longer necessary for 
> intermediate CA users to manually distribute SSL files to their agents. On 
> the server side, we are working to create a puppetserver CLI for setting up 
> and interacting with the CA. See 
> https://tickets.puppetlabs.com/browse/SERVER-2171 
>