Faidon has submitted this change and it was merged.
Change subject: Create self-standing IPython Notebook Puppet module
......................................................................
Create self-standing IPython Notebook Puppet module
- For the sake of portability, replaced dependency on 'systemuser' resource
(from generic-definitions.pp) with equivalent user & group resources.
- Dito generic-definitions.pp's 'upstart_job' resource (which could stand to be
improved..) Replaced with a proper service resource, specifying upstart as
the provider, but also creating an init.d symlink to /lib/init/upstart-job.
- Added README, expanded parameter description, and documented default values.
- Change default IPYTHONDIR from '/var/lib/ipython' to '/srv/ipython', which
better conforms to the Filesystem Hierarchy Standard.
- Changed ill-named 'graphs_dir' to 'mplconfigdir', in keeping with the name of
the environment variable it shadows.
- Followed Puppet's style guide for specifying default values for parametrized
classes that are based on other variables.
- In erb templates, use '@varname' rather than 'varname'. (The latter notation
is liable to conflict with Ruby built-ins and is thus discouraged.)
I tested it using Vagrant and it works well.
Change-Id: I2db8206d91e929c943fddab8e09db60f458739b0
---
A modules/eventlogging/manifests/analysis.pp
M modules/eventlogging/manifests/init.pp
D modules/eventlogging/manifests/notebook.pp
D modules/eventlogging/templates/ipython-notebook.conf.erb
A modules/ipython/README
A modules/ipython/manifests/init.pp
A modules/ipython/manifests/notebook.pp
A modules/ipython/manifests/profile.pp
A modules/ipython/templates/ipython-notebook.conf.erb
R modules/ipython/templates/ipython_notebook_config.py.erb
10 files changed, 267 insertions(+), 140 deletions(-)
Approvals:
Faidon: Verified; Looks good to me, approved
diff --git a/modules/eventlogging/manifests/analysis.pp
b/modules/eventlogging/manifests/analysis.pp
new file mode 100644
index 0000000..1ee5633
--- /dev/null
+++ b/modules/eventlogging/manifests/analysis.pp
@@ -0,0 +1,10 @@
+# Data analysis environment for EventLogging, based on IPython.
+class eventlogging::analysis {
+ package { [ 'python-pandas', 'python-sympy' ]:
+ ensure => latest,
+ }
+
+ class { 'ipython::notebook':
+ exec_files => [ '/var/lib/ipython/helpers/helpers.py' ],
+ }
+}
diff --git a/modules/eventlogging/manifests/init.pp
b/modules/eventlogging/manifests/init.pp
index 16b44e0..f999bb6 100644
--- a/modules/eventlogging/manifests/init.pp
+++ b/modules/eventlogging/manifests/init.pp
@@ -3,9 +3,7 @@
class eventlogging( $archive_destinations = [] ) {
class { 'eventlogging::supervisor': }
- class { 'eventlogging::notebook':
- exec_files => [ '/var/lib/ipython/helpers/helpers.py' ],
- }
+ class { 'eventlogging::analysis': }
class { 'eventlogging::ganglia': }
class { 'eventlogging::archive':
destinations => $archive_destinations,
diff --git a/modules/eventlogging/manifests/notebook.pp
b/modules/eventlogging/manifests/notebook.pp
deleted file mode 100644
index 504c470..0000000
--- a/modules/eventlogging/manifests/notebook.pp
+++ /dev/null
@@ -1,110 +0,0 @@
-# == Class: eventlogging::notebook
-#
-# Configures an integrated, web-based numeric computation environment
-# (like Mathematica), based on PyLab and IPython Notebook.
-# See <http://ipython.org/ipython-doc/dev/interactive/htmlnotebook.html>
-# and <http://www.scipy.org/PyLab>.
-#
-# === Parameters
-#
-# [*ipython_dir*]
-# IPython working directory.
-#
-# [*ipython_profile*]
-# Name of IPython profile to create.
-#
-# [*ipython_user*]
-# Create this user account and run IPython as this user.
-#
-# [*notebook_dir*]
-# Specifies where IPython should archive notebooks.
-#
-# [*notebook_ip*]
-# Interface on which the notebook server will listen (or '*' for all
-# interfaces).
-#
-# [*notebook_port*]
-# The port the notebook server will listen on.
-#
-# [*exec_files*]
-# Array of Python files to execute at start of Notebook session.
-#
-class eventlogging::notebook(
- $ipython_dir = '/var/lib/ipython',
- $ipython_profile = 'nbserver',
- $ipython_user = 'ipython',
- $graph_dir = '/var/lib/ipython/graphs',
- $notebook_dir = '/var/lib/ipython/notebooks',
- $notebook_ip = '*',
- $notebook_port = 8888,
- $exec_files = []
-) {
-
- package { [
- 'ipython-notebook',
- 'python-matplotlib',
- 'python-numpy',
- 'python-pandas',
- 'python-pexpect',
- 'python-scipy',
- ]:
- ensure => latest,
- }
-
- systemuser { $ipython_user:
- name => $ipython_user,
- home => $ipython_dir,
- }
-
- file { $notebook_dir:
- ensure => directory,
- owner => $ipython_user,
- group => $ipython_user,
- mode => '0775',
- require => Systemuser[$ipython_user],
- }
-
- file { $graph_dir:
- ensure => directory,
- owner => $ipython_user,
- group => $ipython_user,
- mode => '0775',
- require => Systemuser[$ipython_user],
- }
-
- exec { "Create IPython profile ${ipython_profile}":
- command => "/usr/bin/ipython profile create
${ipython_profile} --ipython-dir=\"${ipython_dir}\"",
- creates => "${ipython_dir}/profile_${ipython_profile}",
- environment => "IPYTHONDIR=${ipython_dir}",
- user => $ipython_user,
- require => [ Package['ipython-notebook'],
Systemuser[$ipython_user] ],
- }
-
- file { "Configure IPython profile ${ipython_profile}":
- path =>
"${ipython_dir}/profile_${ipython_profile}/ipython_notebook_config.py",
- content =>
template('eventlogging/ipython_notebook_config.py.erb'),
- require => Exec["Create IPython profile ${ipython_profile}"],
- owner => $ipython_user,
- group => $ipython_user,
- mode => '0444',
- }
-
- file { '/etc/init.d/ipython-notebook':
- ensure => link,
- target => '/lib/init/upstart-job',
- }
-
- file { '/etc/init/ipython-notebook.conf':
- content => template('eventlogging/ipython-notebook.conf.erb'),
- require => File['/etc/init.d/ipython-notebook'],
- notify => Exec['start ipython-notebook'],
- }
-
- exec { 'Start IPython Notebook':
- command => '/sbin/start ipython-notebook',
- refreshonly => true,
- require => [ Package['ipython-notebook'],
File[$notebook_dir], File[$graph_dir] ],
- returns => [ 0, 1 ], # OK if "Job is already running"
- }
-
-}
diff --git a/modules/eventlogging/templates/ipython-notebook.conf.erb
b/modules/eventlogging/templates/ipython-notebook.conf.erb
deleted file mode 100644
index 2471fd4..0000000
--- a/modules/eventlogging/templates/ipython-notebook.conf.erb
+++ /dev/null
@@ -1,24 +0,0 @@
-# vim: set ft=upstart:
-
-# Upstart job configuration for IPython Notebook
-# This file is managed by Puppet
-
-description "IPython Notebook"
-author "Ori Livneh <o...@wikimedia.org>"
-
-start on (local-filesystems and net-device-up IFACE!=lo)
-stop on runlevel [!2345]
-
-setuid <%= ipython_user %>
-setgid <%= ipython_dir %>
-
-chdir "<%= ipython_dir %>"
-
-env HOME="<%= ipython_dir %>"
-env IPYTHONDIR="<%= ipython_dir %>"
-env MPLCONFIGDIR="<%= graph_dir %>"
-
-exec ipython notebook \
- --ipython-dir="<%= ipython_dir %>" \
- --profile="<%= ipython_profile %>"
-respawn
diff --git a/modules/ipython/README b/modules/ipython/README
new file mode 100644
index 0000000..5a02dc7
--- /dev/null
+++ b/modules/ipython/README
@@ -0,0 +1,34 @@
+== Puppet module for IPython
+
+=== What is it?
+
+This folder contains a Puppet module for configuring an integrated, web-based
+numeric computation environment (like Mathematica), based on PyLab and IPython
+Notebook.
+
+See RDoc in manifests/init.pp for a listing of various configuration options
+and their default values.
+
+
+=== Usage
+
+To use the default options, you can simply add this to your manifest:
+
+ class { 'ipython::notebook': }
+
+
+=== Limitations
+
+ * Ubuntu-specific, as it depends on Upstart.
+
+
+=== Links
+
+ * http://ipython.org/ipython-doc/dev/interactive/htmlnotebook.html
+ * http://www.scipy.org/PyLab
+
+
+=== Copyright
+
+ Copyright (C) 2013 by Ori Livneh <o...@wikimedia.org>
+ Licensed under the GNU Public License, version 2 or later.
diff --git a/modules/ipython/manifests/init.pp
b/modules/ipython/manifests/init.pp
new file mode 100644
index 0000000..b5d24ba
--- /dev/null
+++ b/modules/ipython/manifests/init.pp
@@ -0,0 +1,60 @@
+# == Class: ipython
+#
+# Base class for IPython setups. IPython is an interactive interpreter for
+# Python which provides facilities for scientific computing, such as
+# distributed computation and graphical development environment.
+#
+# Note that in most cases, rather than including this class directly, you will
+# want to simply accept the defaults and skip directly to including a specific
+# IPython service, such as 'ipython::notebook'.
+#
+# === Parameters
+#
+# [*ipythondir*]
+# IPython working directory. Equivalent to the IPYTHONDIR environment
+# variable. Defaults to '/srv/ipython'.
+#
+# [*user*]
+# Any IPython services will run under this user's uid. The user account will
+# be created if it does not already exist. Defaults to 'ipython'.
+#
+# [*group*]
+# Run services under this gid. Will be created if it does not already exist.
+# Defaults to 'ipython'.
+#
+# === Authors
+#
+# Ori Livneh <o...@wikimedia.org>
+#
+# === Copyright
+#
+# Copyright (C) 2013 Ori Livneh
+# Licensed under the GNU Public License, version 2
+#
+class ipython(
+ $ipythondir = '/srv/ipython',
+ $user = 'ipython',
+ $group = 'ipython'
+) {
+
+ package { 'ipython':
+ ensure => latest,
+ }
+
+ if ! defined(Group[$group]) {
+ group { $user:
+ ensure => present,
+ }
+ }
+
+ if ! defined(User[$user]) {
+ user { $user:
+ ensure => present,
+ gid => $group,
+ shell => '/bin/false',
+ home => $ipythondir,
+ managehome => true,
+ system => true,
+ }
+ }
+}
diff --git a/modules/ipython/manifests/notebook.pp
b/modules/ipython/manifests/notebook.pp
new file mode 100644
index 0000000..85206c4
--- /dev/null
+++ b/modules/ipython/manifests/notebook.pp
@@ -0,0 +1,99 @@
+# == Class: ipython::notebook
+#
+# Configures an integrated, web-based numeric computation environment
+# (like Mathematica), based on PyLab and IPython Notebook.
+# See <http://ipython.org/ipython-doc/dev/interactive/htmlnotebook.html>
+# and <http://www.scipy.org/PyLab>.
+#
+# === Parameters
+#
+# [*profile*]
+# Name of IPython profile to create (default: 'nbserver').
+#
+# [*user*]
+# Run the IPython Notebook service as this user. Defaults to 'ipython'.
+#
+# [*group*]
+# Run the IPython Notebook service under this group's gid.
+# Defaults to 'ipython'.
+#
+# [*ipythondir*]
+# Same as the IPYTHONDIR environment variable. The base folder for IPython
+# profiles and data.
+#
+# [*mplconfigdir*]
+# This is the directory used to store user customizations to matplotlib, as
+# well as some caches to improve performance. Equivalent to the MPLCONFIGDIR
+# environment variable. Defaults to "$ipythondir/matplotlib".
+#
+# [*notebookdir*]
+# Directory where IPython should store notebooks.
+# Default: "$ipythondir/notebooks".
+#
+# [*ip*]
+# Interface on which the Notebook server will listen, or '*' for all
+# interfaces (default: '*').
+#
+# [*port*]
+# The port the Notebook server will listen on (default: 8888).
+#
+# [*exec_files*]
+# Array of fully qualified paths to Python files to execute at start of
+# Notebook session (default: none).
+#
+class ipython::notebook(
+ $profile = 'nbserver',
+ $user = $ipython::user,
+ $group = $ipython::group,
+ $ipythondir = $ipython::ipythondir,
+ $mplconfigdir = "${ipython::ipythondir}/matplotlib",
+ $notebookdir = "${ipython::ipythondir}/notebooks",
+ $port = 8888,
+ $ip = '*',
+ $exec_files = []
+) inherits ipython {
+
+ ipython::profile { $profile: }
+
+ package { [ 'ipython-notebook', 'python-matplotlib', 'python-scipy' ]:
+ ensure => latest,
+ }
+
+ file { $notebookdir:
+ ensure => directory,
+ owner => $user,
+ group => $group,
+ mode => '0775',
+ }
+
+ file { $mplconfigdir:
+ ensure => directory,
+ owner => $user,
+ group => $group,
+ mode => '0775',
+ }
+
+ file { "${profile} notebook config":
+ path =>
"${ipythondir}/profile_${profile}/ipython_notebook_config.py",
+ require => Ipython::Profile[$profile],
+ content => template('ipython/ipython_notebook_config.py.erb'),
+ owner => $user,
+ group => $group,
+ mode => '0444',
+ }
+
+ file { '/etc/init/ipython-notebook.conf':
+ content => template('ipython/ipython-notebook.conf.erb'),
+ require => File["${profile} notebook config"],
+ }
+
+ service { 'ipython-notebook':
+ ensure => running,
+ provider => 'upstart',
+ subscribe => File['/etc/init/ipython-notebook.conf'],
+ require => [
+ Package['ipython-notebook'],
+ File['/etc/init/ipython-notebook.conf'],
+ ]
+ }
+}
diff --git a/modules/ipython/manifests/profile.pp
b/modules/ipython/manifests/profile.pp
new file mode 100644
index 0000000..86636ac
--- /dev/null
+++ b/modules/ipython/manifests/profile.pp
@@ -0,0 +1,36 @@
+# == Define: ipython::profile
+#
+# Creates an IPython configuration profile.
+#
+# === Parameters
+#
+# [*user*]
+# Create a configuration profile for this user. Defaults to 'ipython'.
+#
+# [*ipythondir*]
+# Use this folder as the base IPYTHONDIR. Defaults to '/srv/ipython'.
+#
+# [*parallel*]
+# If true, include the config files for parallel computing apps.
+# False by default.
+#
+define ipython::profile(
+ $user = $ipython::user,
+ $ipythondir = $ipython::ipythondir,
+ $parallel = false
+) {
+ include ipython
+
+ $options = $parallel ? {
+ true => '--parallel',
+ default => '',
+ }
+
+ exec { "ipython profile ${title}":
+ command => "ipython profile create ${title} ${options}",
+ creates => "${ipythondir}/profile_${title}",
+ environment => "IPYTHONDIR=${ipythondir}",
+ require => Package['ipython'],
+ user => $user,
+ }
+}
diff --git a/modules/ipython/templates/ipython-notebook.conf.erb
b/modules/ipython/templates/ipython-notebook.conf.erb
new file mode 100644
index 0000000..b0b70a8
--- /dev/null
+++ b/modules/ipython/templates/ipython-notebook.conf.erb
@@ -0,0 +1,24 @@
+# vim: set ft=upstart:
+
+# Upstart job configuration for IPython Notebook
+# This file is managed by Puppet
+
+description "IPython Notebook"
+author "Ori Livneh <o...@wikimedia.org>"
+
+start on (local-filesystems and net-device-up IFACE!=lo)
+stop on runlevel [!2345]
+
+setuid <%= @user %>
+setgid <%= @group %>
+
+chdir "<%= @ipythondir %>"
+
+env HOME="<%= @ipythondir %>"
+env IPYTHONDIR="<%= @ipythondir %>"
+env MPLCONFIGDIR="<%= @mplconfigdir %>"
+
+exec ipython notebook \
+ --ipython-dir="<%= @ipythondir %>" \
+ --profile="<%= @profile %>"
+respawn
diff --git a/modules/eventlogging/templates/ipython_notebook_config.py.erb
b/modules/ipython/templates/ipython_notebook_config.py.erb
similarity index 68%
rename from modules/eventlogging/templates/ipython_notebook_config.py.erb
rename to modules/ipython/templates/ipython_notebook_config.py.erb
index b881cbf..da81fd6 100644
--- a/modules/eventlogging/templates/ipython_notebook_config.py.erb
+++ b/modules/ipython/templates/ipython_notebook_config.py.erb
@@ -5,9 +5,9 @@
c.IPKernelApp.pylab = 'inline'
c.NotebookApp.enable_mathjax = True
c.NotebookApp.open_browser = False
-c.NotebookApp.ip = '<%= notebook_ip %>'
-c.NotebookApp.port = <%= notebook_port %>
-c.NotebookManager.notebook_dir = '<%= notebook_dir %>'
+c.NotebookApp.ip = '<%= @ip %>'
+c.NotebookApp.port = <%= @port %>
+c.NotebookManager.notebook_dir = '<%= @notebookdir %>'
c.InteractiveShellApp.exec_files = [
<% exec_files.each do |file| -%>
--
To view, visit https://gerrit.wikimedia.org/r/60187
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I2db8206d91e929c943fddab8e09db60f458739b0
Gerrit-PatchSet: 6
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ori.livneh <o...@wikimedia.org>
Gerrit-Reviewer: Faidon <fai...@wikimedia.org>
Gerrit-Reviewer: Ori.livneh <o...@wikimedia.org>
Gerrit-Reviewer: Ottomata <o...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
