Re: [Puppet Users] Use nodes with defined resource in nginx configuration

[Justin Rowles]

Hi Gareth,

I'd be interested on your feedback with a gem that we at BGCH have just 
released: bgch-hiera-consul.  It's on rubygems.

It's derived from lynxman's hiera-consul, but the behaviour is different. 
 It permits storing yaml and/or json as a value and parses it for hiera 
variables.

Hope you like it!

Justin.

On Tuesday, 19 January 2016 12:39:11 UTC, garethr wrote:
>
> On 19 January 2016 at 11:34, Lowe Schmidt  > wrote: 
> > Exported resources would accomplish what you are trying to do, as long 
> as 
> > you don't often and rapidly remove and add upstream app-hosts. 
> > 
>
> If you find you do want to change things more rapidly, integrating 
> with a service discovery mechanism is a nice approach. 
>
> This repo, and accompanying presentation I gave last year demonstrate 
> using Consul, along with the Consul module and Hiera backend to 
> accomplish dynamic changes to configuration of an nginx upstream. The 
> demo uses Puppet Enterprise but the approach is general purpose, and 
> the modules are all open source and from the wider Puppet community. 
>
> https://github.com/puppetlabs/puppet-enterprise-consul-docker-demo 
>
> https://speakerdeck.com/garethr/service-discovery-and-configuration-management
>  
>
> The key trick is to use some sort of watcher (in my case the build-in 
> consul watcher setup using the ::consul::watch Puppet type) to detect 
> changes across the cluster as they happen and to run Puppet to update 
> the proxy straight away. 
>
> Gareth 
>
>
> > -- 
> > Lowe Schmidt | +46 723 867 157 
> > 
> > On 18 January 2016 at 14:29, Frederik Bjerggaard Nielsen <
> f...@firstcom.dk > 
> > wrote: 
> >> 
> >> Hi, 
> >> 
> >> I am working on making our puppet infrastructure more dynamic where I 
> can 
> >> add any number of nodes and these will automatically be applied in 
> another 
> >> nodes configuration. 
> >> 
> >> An example: 
> >> I have X number of hosts running a Java application, which are defined 
> by 
> >> our javaapp module with the following define in it (simplified): 
> >> 
> >> define javaapp::site ( 
> >>   $site_name, 
> >>   $site_port ) { 
> >>   # Doing some stuff regarding deployment etc here 
> >> } 
> >> 
> >> These are defined in the site.pp file like this: 
> >> 
> >> javaapp::site { 'app01': 
> >>   play_site_name=> 'app01', 
> >>   play_site_port=> 9100, 
> >> } 
> >> 
> >> We have a nginx reverse proxy in front of these application servers 
> where 
> >> I need to forward requests to a range of servers. Currently my nginx 
> >> template looks like this: 
> >> 
> >> upstream app-hosts { 
> >>   server <%= @host_app_01 %>:9100; 
> >>   server <%= @host_app_02 %>:9100; 
> >> } 
> >> 
> >> Where @host_app_01 and @host_app_02 have been defined globally in the 
> >> site.pp file with the IP of the nodes. 
> >> 
> >> What I would like to do is to collect an array of the nodes with a 
> >> specific javaapp site name and use it in the nginx template. 
> >> 
> >> I have read a little about exported resources, but I can't figure out 
> >> whether it is the right thing for this situation? 
> >> 
> >> Is it even possible to do with puppet? 
> >> 
> >> We are running puppet version 3.7.5. 
> >> 
> >> Thanks in advance. 
> >> 
> >> Regards 
> >> Frederik Nielsen 
> >> 
> >> -- 
> >> You received this message because you are subscribed to the Google 
> Groups 
> >> "Puppet Users" group. 
> >> To unsubscribe from this group and stop receiving emails from it, send 
> an 
> >> email to puppet-users...@googlegroups.com . 
> >> To view this discussion on the web visit 
> >> 
> https://groups.google.com/d/msgid/puppet-users/a3bcb7c2-f8bb-4b4b-a901-a328662e2266%40googlegroups.com.
>  
>
> >> For more options, visit https://groups.google.com/d/optout. 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "Puppet Users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to puppet-users...@googlegroups.com . 
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/puppet-users/CAC-wWcQTTqcZe1x3rfswQmTyV1pzMKNcMSFe5RHNZtCB43Q8pw%40mail.gmail.com.
>  
>
> > 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
>
> -- 
> Gareth Rushgrove 
> @garethr 
>
> devopsweekly.com 
> morethanseven.net 
> garethrushgrove.com 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/01c0d4a6-8db4-4ece-929c-c5e679147279%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Use nodes with defined resource in nginx configuration

[Lowe Schmidt]

Exported resources would accomplish what you are trying to do, as long as
you don't often and rapidly remove and add upstream app-hosts.

--
Lowe Schmidt | +46 723 867 157

On 18 January 2016 at 14:29, Frederik Bjerggaard Nielsen 
wrote:

> Hi,
>
> I am working on making our puppet infrastructure more dynamic where I can
> add any number of nodes and these will automatically be applied in another
> nodes configuration.
>
> *An example:*
> I have X number of hosts running a Java application, which are defined by
> our *javaapp* module with the following define in it (simplified):
>
> define javaapp::site (
>   $site_name,
>   $site_port ) {
>   # Doing some stuff regarding deployment etc here
> }
>
> These are defined in the site.pp file like this:
>
> javaapp::site { 'app01':
>   play_site_name=> 'app01',
>   play_site_port=> 9100,
> }
>
> We have a nginx reverse proxy in front of these application servers where
> I need to forward requests to a range of servers. Currently my nginx
> template looks like this:
>
> upstream app-hosts {
>   server <%= @host_app_01 %>:9100;
>   server <%= @host_app_02 %>:9100;
> }
>
> Where *@host_app_01* and *@host_app_02* have been defined globally in the
> site.pp file with the IP of the nodes.
>
> What I would like to do is to collect an array of the nodes with a
> specific javaapp site name and use it in the nginx template.
>
> I have read a little about exported resources, but I can't figure out
> whether it is the right thing for this situation?
>
> Is it even possible to do with puppet?
>
> We are running puppet version 3.7.5.
>
> Thanks in advance.
>
> Regards
> Frederik Nielsen
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/a3bcb7c2-f8bb-4b4b-a901-a328662e2266%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAC-wWcQTTqcZe1x3rfswQmTyV1pzMKNcMSFe5RHNZtCB43Q8pw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Use nodes with defined resource in nginx configuration

[Gareth Rushgrove]

On 19 January 2016 at 11:34, Lowe Schmidt  wrote:
> Exported resources would accomplish what you are trying to do, as long as
> you don't often and rapidly remove and add upstream app-hosts.
>

If you find you do want to change things more rapidly, integrating
with a service discovery mechanism is a nice approach.

This repo, and accompanying presentation I gave last year demonstrate
using Consul, along with the Consul module and Hiera backend to
accomplish dynamic changes to configuration of an nginx upstream. The
demo uses Puppet Enterprise but the approach is general purpose, and
the modules are all open source and from the wider Puppet community.

https://github.com/puppetlabs/puppet-enterprise-consul-docker-demo
https://speakerdeck.com/garethr/service-discovery-and-configuration-management

The key trick is to use some sort of watcher (in my case the build-in
consul watcher setup using the ::consul::watch Puppet type) to detect
changes across the cluster as they happen and to run Puppet to update
the proxy straight away.

Gareth


> --
> Lowe Schmidt | +46 723 867 157
>
> On 18 January 2016 at 14:29, Frederik Bjerggaard Nielsen 
> wrote:
>>
>> Hi,
>>
>> I am working on making our puppet infrastructure more dynamic where I can
>> add any number of nodes and these will automatically be applied in another
>> nodes configuration.
>>
>> An example:
>> I have X number of hosts running a Java application, which are defined by
>> our javaapp module with the following define in it (simplified):
>>
>> define javaapp::site (
>>   $site_name,
>>   $site_port ) {
>>   # Doing some stuff regarding deployment etc here
>> }
>>
>> These are defined in the site.pp file like this:
>>
>> javaapp::site { 'app01':
>>   play_site_name=> 'app01',
>>   play_site_port=> 9100,
>> }
>>
>> We have a nginx reverse proxy in front of these application servers where
>> I need to forward requests to a range of servers. Currently my nginx
>> template looks like this:
>>
>> upstream app-hosts {
>>   server <%= @host_app_01 %>:9100;
>>   server <%= @host_app_02 %>:9100;
>> }
>>
>> Where @host_app_01 and @host_app_02 have been defined globally in the
>> site.pp file with the IP of the nodes.
>>
>> What I would like to do is to collect an array of the nodes with a
>> specific javaapp site name and use it in the nginx template.
>>
>> I have read a little about exported resources, but I can't figure out
>> whether it is the right thing for this situation?
>>
>> Is it even possible to do with puppet?
>>
>> We are running puppet version 3.7.5.
>>
>> Thanks in advance.
>>
>> Regards
>> Frederik Nielsen
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to puppet-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/puppet-users/a3bcb7c2-f8bb-4b4b-a901-a328662e2266%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/CAC-wWcQTTqcZe1x3rfswQmTyV1pzMKNcMSFe5RHNZtCB43Q8pw%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.



-- 
Gareth Rushgrove
@garethr

devopsweekly.com
morethanseven.net
garethrushgrove.com

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAFi_6y%2BuGBmHjuAeQ2YTYVM%2B9K3yixpRe3FvffzHLFuu2jby2w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Use nodes with defined resource in nginx configuration

[Frederik Bjerggaard Nielsen]

Hi,

I am working on making our puppet infrastructure more dynamic where I can 
add any number of nodes and these will automatically be applied in another 
nodes configuration.

*An example:*
I have X number of hosts running a Java application, which are defined by 
our *javaapp* module with the following define in it (simplified):

define javaapp::site (
  $site_name,
  $site_port ) {
  # Doing some stuff regarding deployment etc here
}

These are defined in the site.pp file like this:

javaapp::site { 'app01':
  play_site_name=> 'app01',
  play_site_port=> 9100,
}

We have a nginx reverse proxy in front of these application servers where I 
need to forward requests to a range of servers. Currently my nginx template 
looks like this:

upstream app-hosts {
  server <%= @host_app_01 %>:9100;
  server <%= @host_app_02 %>:9100;
}

Where *@host_app_01* and *@host_app_02* have been defined globally in the 
site.pp file with the IP of the nodes.

What I would like to do is to collect an array of the nodes with a specific 
javaapp site name and use it in the nginx template.

I have read a little about exported resources, but I can't figure out 
whether it is the right thing for this situation?

Is it even possible to do with puppet?

We are running puppet version 3.7.5.

Thanks in advance.

Regards
Frederik Nielsen

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/a3bcb7c2-f8bb-4b4b-a901-a328662e2266%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.