Commit 2118888d3b99ed1e23b05ad203e05f2c62096e3b:
LDAP fallover and configuration:
- Read multiple LDAP uris from /etc/ldap
- Select from those hosts randomly
- Verify that the host is alive
- Use Puppet configuration as a primary source for configuration information
- Have configure intelligently update as well as set LDAP configuration
values
Branch: refs/heads/master
Author: Sam Ruby <[email protected]>
Committer: Sam Ruby <[email protected]>
Pusher: rubys <[email protected]>
------------------------------------------------------------
README.md | +++++++ ------
lib/whimsy/asf/ldap.rb | ++++++++ ---
------------------------------------------------------------
204 changes: 141 additions, 63 deletions.
------------------------------------------------------------
diff --git a/README.md b/README.md
index 068e1cf..2b48bb1 100644
--- a/README.md
+++ b/README.md
@@ -24,7 +24,7 @@ Overview
This directory has two main subdirectories...
-1. [lib/whimsy/asf](lib/whimsy/asf) contains the "model", i.e., a set of
classes
+1. [lib/whimsy/asf](lib/whimsy/asf) contains the "model", i.e., a set of
classes
which encapsulate access
to a number of data sources such as LDAP, ICLAs, auth lists, etc. This
code originally was developed as a part of separate tools and was later
@@ -34,13 +34,13 @@ This directory has two main subdirectories...
2. [www](www) contains the "view", largely a set of CGI scripts that produce
HTML.
Generally a CGI script is self contained, including all of the CSS,
scripts, AJAX logic (client and server), SVG images, etc. A single script
- may also produce a set (subtree) of web pages.
+ may also produce a set (subtree) of web pages.
Some of the directories (like the roster tool) contain
[rack](http://rack.github.io/)
applications. These can be run independently, or under the Apache web
server through
- the use of [Phusion Passenger](https://www.phusionpassenger.com/).
-
- The board agenda tool is currently hosted
[separately](https://github.com/rubys/whimsy-agenda)
+ the use of [Phusion Passenger](https://www.phusionpassenger.com/).
+
+ The board agenda tool is currently hosted
[separately](https://github.com/rubys/whimsy-agenda)
on github, but this will be consolidated into this repository as a part of
the effort
to move whimsy.apache.org to a new VM.
@@ -60,11 +60,11 @@ Skip this section if you are running a Docker container or
a Vagrant VM.
'cd' in bash).
For more information:
-
+
1. [Understanding
Shims](https://github.com/sstephenson/rbenv#understanding-shims)
2. [Understanding
Binstubs](https://github.com/sstephenson/rbenv/wiki/Understanding-binstubs)
3. [Ruby Version Manager](https://rvm.io/)
-
+
2. Make sure that the `whimsy-asf` gem installed. If it is not, run
@@ -87,8 +87,8 @@ Skip this section if you are running a Docker container or a
Vagrant VM.
4. Access to LDAP requires configuration, and a cert.
1. The model code determines what host and port to connect to by parsing
- either `/etc/ldap/ldap.conf` or `/etc/ldap/ldap.conf` for a line that
looks
- like the following:
+ either `/etc/ldap/ldap.conf` or `/etc/ldap/ldap.conf` for a line that
+ looks like the following:
`uri ldaps://ldap1-us-east.apache.org:636`
2. A `TLS_CACERT` can be obtained via either of the following commands:
@@ -96,22 +96,26 @@ Skip this section if you are running a Docker container or
a Vagrant VM.
`ruby -r whimsy/asf -e "puts ASF::LDAP.cert"`<br/>
`openssl s_client -connect ldap1-us-east.apache.org:636 </dev/null`
- Copy from `BEGIN` to `END` inclusive into the file
+ Copy from `BEGIN` to `END` inclusive into the file
`/etc/ldap/asf-ldap-client.pem`. Point to the file in
`/etc/ldap/ldap.conf` with a line like the following:
``` TLS_CACERT /etc/ldap/asf-ldap-client.pem```
- N.B. OpenLDAP uses `/etc/openldap/` instead of `/etc/ldap/`
- Adjust the paths above as necessary
+ N.B. OpenLDAP on Mac OS/X uses `/etc/openldap/` instead of `/etc/ldap/`
+ Adjust the paths above as necessary. Additionally ensure that
+ that `TLS_REQCERT` is set to `allow`.
- Note: the certificate is needed because the LDAP hosts use a self-signed
- certificate
-
- These above updates can be done for you with the following command:
+ Note: the certificate is needed because the ASF LDAP hosts use a
+ self-signed certificate.
+
+ All these updates can be done for you with the following command:
sudo ruby -r whimsy/asf -e "ASF::LDAP.configure"
+ These above command can also be used to update your configuration as
+ the ASF changes LDAP servers.
+
5. Verify that the configuration is correct by running:
`ruby examples/board.rb`
@@ -121,12 +125,12 @@ Skip this section if you are running a Docker container
or a Vagrant VM.
Further Reading
===============
-An [example](https://github.com/rubys/whimsy-agenda#readme) of a complete tool
-that makes full use of the library factoring, has a suite of test cases, and
-client componentization (using ReactJS), and provides instructions for setting
+An [example](https://github.com/rubys/whimsy-agenda#readme) of a complete tool
+that makes full use of the library factoring, has a suite of test cases, and
+client componentization (using ReactJS), and provides instructions for setting
up both a Docker component and a Vagrant VM:
If you would like to understand how the view code works, you can get started
-by looking at a
-few of the [Wunderbar
demos](https://github.com/rubys/wunderbar/tree/master/demo)
+by looking at a few of the
+[Wunderbar demos](https://github.com/rubys/wunderbar/tree/master/demo)
and [README](https://github.com/rubys/wunderbar/blob/master/README.md).
diff --git a/lib/whimsy/asf/ldap.rb b/lib/whimsy/asf/ldap.rb
index 6b6bae8..33948c7 100644
--- a/lib/whimsy/asf/ldap.rb
+++ b/lib/whimsy/asf/ldap.rb
@@ -1,6 +1,7 @@
require 'wunderbar'
require 'ldap'
require 'weakref'
+require 'net/http'
module ASF
module LDAP
@@ -14,6 +15,68 @@ module LDAP
ldaps://snappy5.apache.org:636
ldaps://ldap2-lw-us.apache.org:636
)
+
+ # fetch configuration from apache/infrastructure-puppet
+ def self.puppet_config
+ return @puppet if @puppet
+ file = '/apache/infrastructure-puppet/deployment/data/common.yaml'
+ http = Net::HTTP.new('raw.githubusercontent.com', 443)
+ http.use_ssl = true
+ @puppet = YAML.load(http.request(Net::HTTP::Get.new(file)).body)
+ end
+
+ # extract the ldapcert from the puppet configuration
+ def self.puppet_cert
+ puppet_config['ldapclient::ldapcert']
+ end
+
+ # extract the ldap servers from the puppet configuration
+ def self.puppet_ldapservers
+ puppet_config['ldapserver::slapd_peers'].values.
+ map {|host| "ldaps://#{host}:636"}
+ rescue
+ nil
+ end
+
+ # connect to LDAP
+ def self.connect
+ hosts.shuffle.each do |host|
+ Wunderbar.info "Connecting to LDAP server: #{host}"
+
+ begin
+ # request connection
+ uri = URI.parse(host)
+ if uri.scheme == 'ldaps'
+ ldap = ::LDAP::SSLConn.new(uri.host, uri.port)
+ else
+ ldap = ::LDAP::Conn.new(uri.host, uri.port)
+ end
+
+ # test the connection
+ ldap.bind
+ ldap.unbind rescue nil
+
+ # save the host
+ @host = host
+
+ return ldap
+ rescue ::LDAP::ResultError => re
+ Wunderbar.error "Error connecting to LDAP server #{host}: " +
+ re.message
+ end
+
+ return nil
+ end
+ end
+ end
+
+ # backwards compatibility for tools that called this interface, and
+ # a part of the refresh strategy (something that should be revisited
+ # with WeakReferences instead).
+ def self.init_ldap
+ return @ldap if @ldap
+ @mtime = Time.now
+ @ldap = ASF::LDAP.init
end
# determine where ldap.conf resides
@@ -23,27 +86,6 @@ module LDAP
ETCLDAP = '/etc/ldap'
end
- # determine whether or not the LDAP API can be used
- def self.init_ldap
- @ldap = nil
- @mtime = Time.now
-
- host = ASF::LDAP.host
-
- Wunderbar.info "Connecting to LDAP server: #{host}"
-
- begin
- uri = URI.parse(host)
- if uri.scheme == 'ldaps'
- @ldap = ::LDAP::SSLConn.new(uri.host, uri.port)
- else
- @ldap = ::LDAP::Conn.new(uri.host, uri.port)
- end
- rescue ::LDAP::ResultError=>re
- Wunderbar.error "Error binding to LDAP server: message: ["+ re.message +
"]"
- end
- end
-
def self.ldap
@ldap || self.init_ldap
end
@@ -355,59 +397,91 @@ def add(people)
module LDAP
def self.bind(user, password, &block)
dn = ASF::Person.new(user).dn
+ ASF.ldap.unbind rescue nil
if block
ASF.ldap.bind(dn, password, &block)
+ ASF.init_ldap
else
ASF.ldap.bind(dn, password)
end
- ASF.init_ldap
end
- # select LDAP host
- def self.host
+ # determine what LDAP hosts are available
+ def self.hosts
# try whimsy config
- host = ASF::Config.get(:ldap)
+ hosts = Array(ASF::Config.get(:ldap))
# check system configuration
- unless host
+ if hosts.empty?
conf = "#{ETCLDAP}/ldap.conf"
if File.exist? conf
- host = File.read(conf)[/^uri\s+(ldaps?:\/\/\S+?:\d+)/i, 1]
+ uris = File.read(conf)[/^uri\s+(.*)/i, 1].to_s
+ hosts = uris.scan(/ldaps?:\/\/\S+?:\d+/)
end
end
- # if all else fails, pick one at random
- host = ASF::LDAP::HOSTS.sample unless host
+ # if all else fails, use default list
+ hosts = ASF::LDAP::HOSTS if hosts.empty?
+
+ hosts
+ end
- host
+ # select LDAP host
+ def self.host
+ @host ||= hosts.sample
end
# query and extract cert from openssl output
- def self.cert
+ def self.extract_cert
host = LDAP.host[%r{//(.*?)(/|$)}, 1]
- query = "openssl s_client -connect #{host} -showcerts"
- output = `#{query} < /dev/null 2> /dev/null`
- output[/^-+BEGIN.*?\n-+END[^\n]+\n/m]
+ puts ['openssl', 's_client', '-connect', host, '-showcerts'].join(' ')
+ out, err, rc = Open3.capture3 'openssl', 's_client',
+ '-connect', host, '-showcerts'
+ out[/^-+BEGIN.*?\n-+END[^\n]+\n/m]
end
# update /etc/ldap.conf. Usage:
+ #
# sudo ruby -r whimsy/asf -e "ASF::LDAP.configure"
+ #
def self.configure
- if not File.exist? "#{ETCLDAP}/asf-ldap-client.pem"
- File.write "#{ETCLDAP}/asf-ldap-client.pem", self.cert
+ cert = Dir["#{ETCLDAP}/asf*-ldap-client.pem"].first
+
+ # verify/obtain/write the cert
+ if not cert
+ cert = "#{ETCLDAP}/asf-ldap-client.pem"
+ File.write cert, ASF::LDAP.puppet_cert || self.extract_cert
end
+ # read the current configuration file
ldap_conf = "#{ETCLDAP}/ldap.conf"
content = File.read(ldap_conf)
- unless content.include? 'asf-ldap-client.pem'
- content.gsub!(/^TLS_CACERT/, '# TLS_CACERT')
- content.gsub!(/^TLS_REQCERT/, '# TLS_REQCERT')
+
+ # ensure that the right cert is used
+ unless content =~ /asf.*-ldap-client\.pem/
+ content.gsub!(/^TLS_CACERT/i, '# TLS_CACERT')
content += "TLS_CACERT #{ETCLDAP}/asf-ldap-client.pem\n"
- content += "uri #{LDAP.host}\n"
+ end
+
+ # provide the URIs of the ldap hosts
+ content.gsub!(/^URI/, '# URI')
+ content += "uri \n" unless content =~ /^uri /
+ content[/uri (.*)\n/, 1] = hosts.join(' ')
+
+ # verify/set the base
+ unless content.include? 'base dc=apache'
+ content.gsub!(/^BASE/i, '# BASE')
content += "base dc=apache,dc=org\n"
- content += "TLS_REQCERT allow\n" if ETCLDAP.include? 'openldap'
- File.write(ldap_conf, content)
end
+
+ # ensure TLS_REQCERT is allow (Mac OS/X only)
+ if ETCLDAP.include? 'openldap' and not content.include? 'REQCERT allow'
+ content.gsub!(/^TLS_REQCERT/i, '# TLS_REQCERT')
+ content += "TLS_REQCERT allow\n"
+ end
+
+ # write the configuration if there were any changes
+ File.write(ldap_conf, content) unless content == File.read(ldap_conf)
end
end
end
