Re: [Puppet Users] Re: EC2 autoscaling reusing hostnames

[Felipe Salum]

I use a different approach to clean up certificates and the node on the
puppet dashboard, but it is a ugly hack. I'm writing something in python to
read the autoscaling termination message posted to SNS->SQS and I should
have something up tonight. I will share here and get feedback, I'm planning
to replace my ugly hack by this python script.


On Wed, May 28, 2014 at 3:05 AM, Hugh Cole-Baker  wrote:

> Here is the code that we use to pull Auto Scaling messages off an SQS
> queue and add/remove the respective nodes to autosign.conf and sign or
> clean up their certificates.
>
> https://gist.github.com/sigmaris/bda78c1857034031eb76
>
> It's copied and pasted out of a larger application that handles various
> events from Auto Scaling and applies changes to other systems than Puppet,
> so some parts may be missing, but it should be understandable... It assumes
> that all the instance hostnames will be - instance ID> and the domain appended will be based on the AWS region like
> 'east.internal'. You'd have to change the code in message.py to alter that
> behaviour.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/m_fffsKR9aM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/45c97a85-6783-4239-b04d-bf7b378bfa2c%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CALNTNeMGNQaXaa9ofVDH8j27cjSHGh6J_AW%3D%2B7ZQsmJMy%3D4W9A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: EC2 autoscaling reusing hostnames

[Felipe Salum]

I have prod, qa, staging all using autoscaling and my master uses node
regex based on the - hostnames to apply the right
roles modules. I have been using it for a long time and no issue at all to
use the full power of the cloud and autoscale :)



On Tue, May 27, 2014 at 8:34 AM, Jeremy T. Bouse  wrote:

> On 27.05.2014 11:06, Felipe Salum wrote:
>
>> I work around this by using a cloudinit script during the autoscale
>> instance launch that gets the instance-id of the instance, rename the
>> hostname and update /etc/hosts before running puppet.
>>
>>
> Cloud-init helps but there are limitations. I use cloud-init to deploy a
> script that generates the extra attributes file prior to puppet being
> deployed to include the instance id in the CSR and this works if your
> master can then otherwise determine how to configure the host. If the
> master is deterministic on the client cert name (read: hostname) then you
> can override that with cloud-init but it fails to be useful when using the
> full power of the cloud architecture and use auto scaling groups.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/puppet-users/m_fffsKR9aM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/puppet-users/85960579c1c0eab21f9068aa33299130%40undergrid.net.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CALNTNeMWR9Tuibd-UYdg5wQ4r_MO4V9%2Bu7pw2kEzhxFP5yj%3D-A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: EC2 autoscaling reusing hostnames

[Jeremy T. Bouse]

On 27.05.2014 11:06, Felipe Salum wrote:

I work around this by using a cloudinit script during the autoscale
instance launch that gets the instance-id of the instance, rename the
hostname and update /etc/hosts before running puppet.



Cloud-init helps but there are limitations. I use cloud-init to deploy 
a script that generates the extra attributes file prior to puppet being 
deployed to include the instance id in the CSR and this works if your 
master can then otherwise determine how to configure the host. If the 
master is deterministic on the client cert name (read: hostname) then 
you can override that with cloud-init but it fails to be useful when 
using the full power of the cloud architecture and use auto scaling 
groups.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/85960579c1c0eab21f9068aa33299130%40undergrid.net.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: EC2 autoscaling reusing hostnames

[Jeremy T. Bouse]

On 27.05.2014 08:00, Hugh Cole-Baker wrote:

On Tuesday, 27 May 2014 12:56:05 UTC+1, Jeremy wrote:


This actually sounds like a useful tool. Is this something you're
(or
would) consider releasing as OSS for others to make use of?

I've put my autosign script up on a GitHub gist and at least one
other
has considered it useful so I've included a header releasing it
under
Apache 2.0 license.


It's a bit complicated with code to do various other things that our
infrastructure needs, and it makes some assumptions (for example we
always use - for our hostnames, so instances
in "mygroup" are always named things like "mygroup-abcd1234", which
avoids us having to call the EC2 API to find the hostname), but I 
will

see if I can separate out the useful parts and publish them.



Yeah I can understand that. My autosign script made use of the instance 
id being embedded as an extra attribute within the CSR. It left out 
anything specific in processing beyond showing how to pull the instance 
id from the CSR and validate it was a valid running instance using Fog.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5e99cd0c29d55a61c0bf5bd1a405eb2e%40undergrid.net.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: EC2 autoscaling reusing hostnames

[Hugh Cole-Baker]

On Tuesday, 27 May 2014 12:56:05 UTC+1, Jeremy wrote:

>
> This actually sounds like a useful tool. Is this something you're (or 
> would) consider releasing as OSS for others to make use of? 
>
> I've put my autosign script up on a GitHub gist and at least one other 
> has considered it useful so I've included a header releasing it under 
> Apache 2.0 license. 
>

It's a bit complicated with code to do various other things that our 
infrastructure needs, and it makes some assumptions (for example we always 
use - for our hostnames, so instances in "mygroup" 
are always named things like "mygroup-abcd1234", which avoids us having to 
call the EC2 API to find the hostname), but I will see if I can separate 
out the useful parts and publish them.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/53042581-78c0-4319-a5d4-fcdc194c5466%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: EC2 autoscaling reusing hostnames

[Jeremy T. Bouse]

On 27.05.2014 07:23, Hugh Cole-Baker wrote:

Am I missing a configuration option in the manual to somehow disable
SSL certificate validation? Does everybody add a cron job to their
puppet master to stop the puppetmaster daemon and blow away its SSL
directory then restart it at exactly 12:00AM every day, and the same
on the instances at exactly 12:02AM every day? Or are we the only
people on the planet who actually use Amazon's auto-scaling feature
*plus* use Puppet at the same time? Curious penguins are... curious!


We have enabled the Amazon SNS notifications from Autoscaling, and
subscribed a SQS queue to the SNS topic. We have a written a small
daemon, which runs on the puppet master and consumes from that queue,
and calls "puppet cert clean" when it receives messages about
instances being terminated by autoscaling.

We also have it listen for instance launch messages and add their
certnames into /etc/puppet/autosign.conf and call "puppet cert sign"
on them, which is also useful for security (you don't have to turn on
auto signing for everything that way).



This actually sounds like a useful tool. Is this something you're (or 
would) consider releasing as OSS for others to make use of?


I've put my autosign script up on a GitHub gist and at least one other 
has considered it useful so I've included a header releasing it under 
Apache 2.0 license.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/61925ab12dbf194f6d6ea546180a34db%40undergrid.net.
For more options, visit https://groups.google.com/d/optout.