Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-15 Thread Felix Frank


On 12/15/2010 12:04 PM, Patrick wrote:
> I'm looking for a way to run more than one puppetmaster on the same server 
> under passenger.  Most of the puppet CPU load is waiting for the catalogs to 
> compile.  This also seems to be mostly what takes large amounts of RAM.  I 
> have storedconfigs on.
> 
> I want to be able to move the fileserver to a different pool of puppetmaster 
> processes.  Is there an easy way to tell the client, either in the catalog or 
> in the config file, to get the files from a different port than the catalog?
> 
> Is there a better way to do this?  What I really want is for the cheap file 
> requests to stop being blocked by the expensive catalog requests and keep the 
> RAM usage low on the file serving processes.
> 

Hi,

what springs to mind is a webserver with mod_proxy up front (or in fact,
any intelligent reverse proxy), that chooses your actual webserver with
respect to request URIs.

Fileserver requests do go to a different root directory, yes?

HTH,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-15 Thread Patrick

On Dec 15, 2010, at 3:09 AM, Felix Frank wrote:

> 
> 
> On 12/15/2010 12:04 PM, Patrick wrote:
>> I'm looking for a way to run more than one puppetmaster on the same server 
>> under passenger.  Most of the puppet CPU load is waiting for the catalogs to 
>> compile.  This also seems to be mostly what takes large amounts of RAM.  I 
>> have storedconfigs on.
>> 
>> I want to be able to move the fileserver to a different pool of puppetmaster 
>> processes.  Is there an easy way to tell the client, either in the catalog 
>> or in the config file, to get the files from a different port than the 
>> catalog?
>> 
>> Is there a better way to do this?  What I really want is for the cheap file 
>> requests to stop being blocked by the expensive catalog requests and keep 
>> the RAM usage low on the file serving processes.
>> 
> 
> Hi,
> 
> what springs to mind is a webserver with mod_proxy up front (or in fact,
> any intelligent reverse proxy), that chooses your actual webserver with
> respect to request URIs.
> 
> Fileserver requests do go to a different root directory, yes?

Technically, I decided to just pass the catalog requests through instead.  
Catalog requests are sent to "/production/catalog/" so it should be easy to do.

Do you have any advice for me before I try separating the proxy into a 
different (third) VirtualServer?

I setup a second server on 8141 and I can send puppet requests to that server 
just fine too.

Then I added these lines in my VirtualHost block:
ProxyPass /production/catalog/ 
https://localhost:8141/production/catalog/
ProxyPassReverse /production/catalog/ 
https://localhost:8141/production/catalog/

They don't seem to have any effect though.

I can see the log entries like this one (I chopped off 3 pages of facts) which 
shows the URL:
Simba.Outer:8140 192.168.2.252 - - [15/Dec/2010:10:21:07 -0800] "GET 
/production/catalog/simba.outer?facts=eNp1 HTTP/1.1" 200 95433 "-" "-"

My config file for the primary virtual server is here:
http://pastie.org/1380225

In summery, both servers work, but no redirection is taking place.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-15 Thread Brice Figureau
On 15/12/10 12:04, Patrick wrote:
> I'm looking for a way to run more than one puppetmaster on the same
> server under passenger.  Most of the puppet CPU load is waiting for
> the catalogs to compile.  This also seems to be mostly what takes
> large amounts of RAM.  I have storedconfigs on.

If you don't need the full storedconfigs, you can use thin_storedconfigs
for wy better performance.

> I want to be able to move the fileserver to a different pool of
> puppetmaster processes.  Is there an easy way to tell the client,
> either in the catalog or in the config file, to get the files from a
> different port than the catalog?

In every puppet:// url you can specify a different server. You can
dedicate some masters to serve files only and others to server catalogs.

> Is there a better way to do this?  What I really want is for the
> cheap file requests to stop being blocked by the expensive catalog
> requests and keep the RAM usage low on the file serving processes.

You can use what I called file serving offloading:
http://www.masterzen.fr/2010/03/21/more-puppet-offloading/

-- 
Brice Figureau
My Blog: http://www.masterzen.fr/

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-15 Thread Patrick

On Dec 15, 2010, at 1:48 PM, Brice Figureau wrote:

> On 15/12/10 12:04, Patrick wrote:
>> I'm looking for a way to run more than one puppetmaster on the same
>> server under passenger.  Most of the puppet CPU load is waiting for
>> the catalogs to compile.  This also seems to be mostly what takes
>> large amounts of RAM.  I have storedconfigs on.
> 
> If you don't need the full storedconfigs, you can use thin_storedconfigs
> for wy better performance.

Thanks.  I'm actually doing that, and misspoke in the first post.

>> Is there a better way to do this?  What I really want is for the
>> cheap file requests to stop being blocked by the expensive catalog
>> requests and keep the RAM usage low on the file serving processes.
> 
> You can use what I called file serving offloading:
> http://www.masterzen.fr/2010/03/21/more-puppet-offloading/

The file offloading is interesting.  So if I'm reading that right, that only 
makes a difference if some of the files are not in sync?




My original error was that I didn't set:
SSLProxyEngine on

Now I'm just getting errors that say all requests are forbidden.  I assume this 
is because the puppetmaster isn't seeing the headers from apache that have the 
SSL information.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Felix Frank
On 12/15/2010 07:40 PM, Patrick wrote:
> 
> On Dec 15, 2010, at 3:09 AM, Felix Frank wrote:
> 
>>
>>
>> On 12/15/2010 12:04 PM, Patrick wrote:
>>> I'm looking for a way to run more than one puppetmaster on the same
>>> server under passenger.  Most of the puppet CPU load is waiting for
>>> the catalogs to compile.  This also seems to be mostly what takes
>>> large amounts of RAM.  I have storedconfigs on.
>>>
>>> I want to be able to move the fileserver to a different pool of
>>> puppetmaster processes.  Is there an easy way to tell the client,
>>> either in the catalog or in the config file, to get the files from a
>>> different port than the catalog?
>>>
>>> Is there a better way to do this?  What I really want is for the
>>> cheap file requests to stop being blocked by the expensive catalog
>>> requests and keep the RAM usage low on the file serving processes.
>>>
>>
>> Hi,
>>
>> what springs to mind is a webserver with mod_proxy up front (or in fact,
>> any intelligent reverse proxy), that chooses your actual webserver with
>> respect to request URIs.
>>
>> Fileserver requests do go to a different root directory, yes?
> 
> Technically, I decided to just pass the catalog requests through
> instead.  Catalog requests are sent to "/production/catalog/" so it
> /should/ be easy to do.
> 
> Do you have any advice for me before I try separating the proxy into a
> different (third) VirtualServer?
> 
> I setup a second server on 8141 and I can send puppet requests to that
> server just fine too.
> 
> Then I added these lines in my VirtualHost block:
> ProxyPass /production/catalog/
> https://localhost:8141/production/catalog/
> ProxyPassReverse /production/catalog/
> https://localhost:8141/production/catalog/
> 
> They don't seem to have any effect though.
> 
> I can see the log entries like this one (I chopped off 3 pages of facts)
> which shows the URL:
> Simba.Outer:8140 192.168.2.252 - - [15/Dec/2010:10:21:07 -0800] "GET
> /production/catalog/simba.outer?facts=eNp1 HTTP/1.1" 200 95433 "-" "-"
> 
> My config file for the primary virtual server is here:
> http://pastie.org/1380225
> 
> In summery, both servers work, but no redirection is taking place.

Hum, I'm not in the habit of using ProxyPass directives. I rather add
RewriteRules that include the [P] flag.

One stupid idea is to try without trailing slash in the ProxyPass pattern.

It may even be necessary to enable mod_rpaf, but it is weird that no
proxied request gets logged.

Sorry to be of no real help.

Felix

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Brice Figureau
On Wed, 2010-12-15 at 20:15 -0800, Patrick wrote:
> On Dec 15, 2010, at 1:48 PM, Brice Figureau wrote:
> 
> > On 15/12/10 12:04, Patrick wrote:
> >> I'm looking for a way to run more than one puppetmaster on the same
> >> server under passenger.  Most of the puppet CPU load is waiting for
> >> the catalogs to compile.  This also seems to be mostly what takes
> >> large amounts of RAM.  I have storedconfigs on.
> > 
> > If you don't need the full storedconfigs, you can use
> thin_storedconfigs
> > for wy better performance.
> 
> Thanks.  I'm actually doing that, and misspoke in the first post.
> 
> >> Is there a better way to do this?  What I really want is for the
> >> cheap file requests to stop being blocked by the expensive catalog
> >> requests and keep the RAM usage low on the file serving processes.
> > 
> > You can use what I called file serving offloading:
> > http://www.masterzen.fr/2010/03/21/more-puppet-offloading/
> 
> The file offloading is interesting.  So if I'm reading that right,
> that only makes a difference if some of the files are not in sync?

Actually yes, because the file content is sent only if the checksum
differs (and if you provision many new nodes at the same time, then it
can help). One solution would be to offload metadata computation to a
native nginx module (it's something easy to do once you know how to code
nginx module).

> My original error was that I didn't set:
> SSLProxyEngine on
> 
> Now I'm just getting errors that say all requests are forbidden.  I
> assume this is because the puppetmaster isn't seeing the headers from
> apache that have the SSL information.

You must setup your file serving master exactly like your catalog (or
general) master.

-- 
Brice Figureau
Follow the latest Puppet Community evolutions on www.planetpuppet.org!

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Patrick

On Dec 16, 2010, at 12:45 AM, Felix Frank wrote:

> On 12/15/2010 07:40 PM, Patrick wrote:
>> 
>> On Dec 15, 2010, at 3:09 AM, Felix Frank wrote:
>> 
>>> 
>>> 
>>> On 12/15/2010 12:04 PM, Patrick wrote:
 I'm looking for a way to run more than one puppetmaster on the same
 server under passenger.  Most of the puppet CPU load is waiting for
 the catalogs to compile.  This also seems to be mostly what takes
 large amounts of RAM.  I have storedconfigs on.
 
 I want to be able to move the fileserver to a different pool of
 puppetmaster processes.  Is there an easy way to tell the client,
 either in the catalog or in the config file, to get the files from a
 different port than the catalog?
 
 Is there a better way to do this?  What I really want is for the
 cheap file requests to stop being blocked by the expensive catalog
 requests and keep the RAM usage low on the file serving processes.
 
>>> 
>>> Hi,
>>> 
>>> what springs to mind is a webserver with mod_proxy up front (or in fact,
>>> any intelligent reverse proxy), that chooses your actual webserver with
>>> respect to request URIs.
>>> 
>>> Fileserver requests do go to a different root directory, yes?
>> 
>> Technically, I decided to just pass the catalog requests through
>> instead.  Catalog requests are sent to "/production/catalog/" so it
>> /should/ be easy to do.
>> 
>> Do you have any advice for me before I try separating the proxy into a
>> different (third) VirtualServer?
>> 
>> I setup a second server on 8141 and I can send puppet requests to that
>> server just fine too.
>> 
>> Then I added these lines in my VirtualHost block:
>>ProxyPass /production/catalog/
>> https://localhost:8141/production/catalog/
>>ProxyPassReverse /production/catalog/
>> https://localhost:8141/production/catalog/
>> 
>> They don't seem to have any effect though.
>> 
>> I can see the log entries like this one (I chopped off 3 pages of facts)
>> which shows the URL:
>> Simba.Outer:8140 192.168.2.252 - - [15/Dec/2010:10:21:07 -0800] "GET
>> /production/catalog/simba.outer?facts=eNp1 HTTP/1.1" 200 95433 "-" "-"
>> 
>> My config file for the primary virtual server is here:
>> http://pastie.org/1380225
>> 
>> In summery, both servers work, but no redirection is taking place.
> 
> Hum, I'm not in the habit of using ProxyPass directives. I rather add
> RewriteRules that include the [P] flag.
> 
> One stupid idea is to try without trailing slash in the ProxyPass pattern.
> 
> It may even be necessary to enable mod_rpaf, but it is weird that no
> proxied request gets logged.
> 
> Sorry to be of no real help.
> 
> Felix
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to 
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/puppet-users?hl=en.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Patrick

On Dec 16, 2010, at 12:45 AM, Felix Frank wrote:

> On 12/15/2010 07:40 PM, Patrick wrote:
>> 
>> In summery, both servers work, but no redirection is taking place.
> 
> Hum, I'm not in the habit of using ProxyPass directives. I rather add
> RewriteRules that include the [P] flag.
> 
> One stupid idea is to try without trailing slash in the ProxyPass pattern.
> 
> It may even be necessary to enable mod_rpaf, but it is weird that no
> proxied request gets logged.

Actually, I got that working.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Patrick


On Dec 16, 2010, at 1:04 AM, Brice Figureau wrote:
>> My original error was that I didn't set:
>> SSLProxyEngine on
>> 
>> Now I'm just getting errors that say all requests are forbidden.  I
>> assume this is because the puppetmaster isn't seeing the headers from
>> apache that have the SSL information.
> 
> You must setup your file serving master exactly like your catalog (or
> general) master.

I did.  The problem is that I don't know enough about apache so I'm doing 
something wrong.

I think this is the problem:
The first layer is stripping out the client's certificate.  Then the second 
layer is stripping out the success headers leaving the puppetmaster with not 
authentication information.

The real problem is that I don't know how to tell Apache to "send on the 
request and don't touch anything".

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Felix Frank
On 12/16/2010 10:28 AM, Patrick wrote:
> 
> 
> On Dec 16, 2010, at 1:04 AM, Brice Figureau wrote:
>>> My original error was that I didn't set:
>>> SSLProxyEngine on
>>>
>>> Now I'm just getting errors that say all requests are forbidden.  I
>>> assume this is because the puppetmaster isn't seeing the headers from
>>> apache that have the SSL information.
>>
>> You must setup your file serving master exactly like your catalog (or
>> general) master.
> 
> I did.  The problem is that I don't know enough about apache so I'm doing 
> something wrong.
> 
> I think this is the problem:
> The first layer is stripping out the client's certificate.  Then the second 
> layer is stripping out the success headers leaving the puppetmaster with not 
> authentication information.
> 
> The real problem is that I don't know how to tell Apache to "send on the 
> request and don't touch anything".

Tough call. There is no such thing as a "transparent SSL proxy" afaik,
because without decrypting requests, the proxy cannot make any header
based decisions.

This may well be a dead end then.

Is it possible to have the fileserving subset of puppetmasters running
without any SSL support? That's throwing security out of the windows of
course, so the proxy should be able to determine (say, by IP rule?) what
clients are allowed and which aren't.

If such an approach is at all possible, the complete implementation
would include giving the proxy the means to recognize valid client
certificates.

Even if this should work - is it work all that hassle?

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Richard Crowley
> Is it possible to have the fileserving subset of puppetmasters running
> without any SSL support? That's throwing security out of the windows of
> course, so the proxy should be able to determine (say, by IP rule?) what
> clients are allowed and which aren't.

This seems like a job for a new file provider.  If memory serves, this
is more involved than just adding a provider because of something
about files not working like other resource types.  Bueller?

I think it'd be very valuable to be able to pick file providers that
grabbed file content from arbitrary HTTP servers, from tarballs, from
stdout of an arbitrary command, etc.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Felix Frank
On 12/16/2010 05:06 PM, Richard Crowley wrote:
>> Is it possible to have the fileserving subset of puppetmasters running
>> without any SSL support? That's throwing security out of the windows of
>> course, so the proxy should be able to determine (say, by IP rule?) what
>> clients are allowed and which aren't.
> 
> This seems like a job for a new file provider.  If memory serves, this
> is more involved than just adding a provider because of something
> about files not working like other resource types.  Bueller?
> 
> I think it'd be very valuable to be able to pick file providers that
> grabbed file content from arbitrary HTTP servers, from tarballs, from
> stdout of an arbitrary command, etc.
> 

Ah, misunderstanding.

The client does use SSL. It is terminated at the HTTP proxy (that runs
on the master host). The trick is to make the puppetmaster swallow up
unencrypted (and thus unauthenticated) traffic from the proxy.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-16 Thread Patrick

On Dec 16, 2010, at 7:55 AM, Felix Frank wrote:

> On 12/16/2010 10:28 AM, Patrick wrote:
>> 
>> 
>> On Dec 16, 2010, at 1:04 AM, Brice Figureau wrote:
 My original error was that I didn't set:
 SSLProxyEngine on
 
 Now I'm just getting errors that say all requests are forbidden.  I
 assume this is because the puppetmaster isn't seeing the headers from
 apache that have the SSL information.
>>> 
>>> You must setup your file serving master exactly like your catalog (or
>>> general) master.
>> 
>> I did.  The problem is that I don't know enough about apache so I'm doing 
>> something wrong.
>> 
>> I think this is the problem:
>> The first layer is stripping out the client's certificate.  Then the second 
>> layer is stripping out the success headers leaving the puppetmaster with not 
>> authentication information.
>> 
>> The real problem is that I don't know how to tell Apache to "send on the 
>> request and don't touch anything".
> 
> Tough call. There is no such thing as a "transparent SSL proxy" afaik,
> because without decrypting requests, the proxy cannot make any header
> based decisions.
> 
> This may well be a dead end then.

Ah.  See below for a different idea then.

> If such an approach is at all possible, the complete implementation
> would include giving the proxy the means to recognize valid client
> certificates.

The proxy can and is recognizing valid certificates.  The problem is passing 
that information on to the puppetmaster because I really don't know how to do 
that.  I also don't know exactly which headers the puppetmaster uses.

I'm thinking that if I do this, I need to remove the SSL from the file server 
VirtualHost and just pass the information directly through.

> Even if this should work - is it work all that hassle?

This is a much better question.  I'm going to work on it a little more though.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-20 Thread Felix Frank
Sorry for the late answer, my provider had a downtime this weekend.

>> Tough call. There is no such thing as a "transparent SSL proxy" afaik,
>> because without decrypting requests, the proxy cannot make any header
>> based decisions.
>>
>> This may well be a dead end then.
> 
> Ah.  See below for a different idea then.
> 
>> If such an approach is at all possible, the complete implementation
>> would include giving the proxy the means to recognize valid client
>> certificates.
> 
> The proxy can and is recognizing valid certificates.  The problem is passing 
> that information on to the puppetmaster because I really don't know how to do 
> that.  I also don't know exactly which headers the puppetmaster uses.
> 
> I'm thinking that if I do this, I need to remove the SSL from the file server 
> VirtualHost and just pass the information directly through.

Hm, no good. This is just the "transparent SSL proxying I'm afraid is
not at all possible (at least while still making use of the request URI
to decide on the backend HTTP server).

>> Even if this should work - is it work all that hassle?
> 
> This is a much better question.  I'm going to work on it a little more though.

The weird thing is: This might as well work, because you proxy pass to
https://localhost...
That means that your proxy actually reencrypts the requests (or should
do that, at least).

Are you seeing requests at the backend Apache now?
Does puppet show any reaction to that?

Are you performing basic tests using wget on distinct URLs that should
be served by this setup?

Regards,
Felix

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Separating puppetmaster file serving and catalogs

2010-12-20 Thread Patrick

On Dec 20, 2010, at 2:43 AM, Felix Frank wrote:

> Sorry for the late answer, my provider had a downtime this weekend.
> 
>>> Tough call. There is no such thing as a "transparent SSL proxy" afaik,
>>> because without decrypting requests, the proxy cannot make any header
>>> based decisions.
>>> 
>>> This may well be a dead end then.
>> 
>> Ah.  See below for a different idea then.
>> 
>>> If such an approach is at all possible, the complete implementation
>>> would include giving the proxy the means to recognize valid client
>>> certificates.
>> 
>> The proxy can and is recognizing valid certificates.  The problem is passing 
>> that information on to the puppetmaster because I really don't know how to 
>> do that.  I also don't know exactly which headers the puppetmaster uses.
>> 
>> I'm thinking that if I do this, I need to remove the SSL from the file 
>> server VirtualHost and just pass the information directly through.
> 
> Hm, no good. This is just the "transparent SSL proxying I'm afraid is
> not at all possible (at least while still making use of the request URI
> to decide on the backend HTTP server).
> 
>>> Even if this should work - is it work all that hassle?
>> 
>> This is a much better question.  I'm going to work on it a little more 
>> though.
> 
> The weird thing is: This might as well work, because you proxy pass to
> https://localhost...
> That means that your proxy actually reencrypts the requests (or should
> do that, at least).
> 
> Are you seeing requests at the backend Apache now?
> Does puppet show any reaction to that?
> 
> Are you performing basic tests using wget on distinct URLs that should
> be served by this setup?

Actually I've gotten a lot farther.  At first, I spent a lot of time fumbling 
around until I finally gave up because I didn't have enough information to 
debug the problems.  Here's what I did as a hack to give me more information:


Added this to config.ru:
# if you want debugging:
 ARGV << "--debug"

# Log to a file in addition to syslog
ARGV << "--logdest" << "/var/log/puppet/puppetpassenger.log"


Added this to near the end of 
/usr/lib/ruby/1.8/puppet/network/http/rack/rest.rb.  Make sure you know where 
to add this.  Do not blindly add this at the end of the file:
if dn.nil?
  dn="nil"
end
status = request.env[Puppet[:ssl_client_verify_header]]
if status.nil?
  status = "nil"
end
Puppet.debug "Custom: client_name=\""+dn+"\" status=\""+status+"\""

#Print out all headers
request.env.each { |elem|
  Puppet.debug "#{elem[1]}, #{elem[0]}"
}



After this I found out that apache was adding a prefix of "HTTP_" to the 
beginning of all custom headers.
So, I was able to get the puppet client to connect if I added:
RequestHeader set SSL_CLIENT_S_DN "/CN=Simba.Outer"
RequestHeader set SSL_CLIENT_VERIFY "SUCCESS"
to the apache config and add:
ssl_client_header = HTTP_SSL_CLIENT_S_DN
ssl_client_verify_header = HTTP_SSL_CLIENT_VERIFY
to the "[master]" section of puppet.conf.

Obviously, this is not suitable in a production environment.

Now I'm trying to get the second apache instance to resend the header it 
receives from the first instance.  I haven't actually started on this yet.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.