[MediaWiki-commits] [Gerrit] Trim leading blanks from servernames - change (mediawiki...LdapAuthentication)
Ryan Lane has submitted this change and it was merged. Change subject: Trim leading blanks from servernames .. Trim leading blanks from servernames Bug: T56968 Change-Id: I15437b21a9e73660defd201ef804762fbb2b8ba3 --- M LdapAuthentication.php 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved diff --git a/LdapAuthentication.php b/LdapAuthentication.php index d9014a3..4b75450 100644 --- a/LdapAuthentication.php +++ b/LdapAuthentication.php @@ -598,7 +598,7 @@ $servers = $servers . " " . $serverpre . $tok . ":" . $this->getConf( 'Port', $domain ); $tok = strtok( " " ); } - $servers = rtrim( $servers ); + $servers = trim( $servers ); $this->printDebug( "Using servers: $servers", SENSITIVE ); -- To view, visit https://gerrit.wikimedia.org/r/206645 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I15437b21a9e73660defd201ef804762fbb2b8ba3 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/LdapAuthentication Gerrit-Branch: master Gerrit-Owner: 01tonythomas <01tonytho...@gmail.com> Gerrit-Reviewer: Reedy Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Fix repos with checked-in .gitmodules - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/121574 Change subject: Fix repos with checked-in .gitmodules .. Fix repos with checked-in .gitmodules Some repos have checked-in .gitmodules files from repositories that are checked directly into repos, rather than being submodules. For repos that leave the git configurations around, trebuchet breaks during the fetch phase. This change checks to see if the .gitmodules location is a valid top level repo by checking the status of 'git submodule status --quiet'. Change-Id: I8b1fe930452a811d50fe3b0c8319f46d2faa318b --- M modules/deployment/files/modules/deploy.py 1 file changed, 9 insertions(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/74/121574/1 diff --git a/modules/deployment/files/modules/deploy.py b/modules/deployment/files/modules/deploy.py index 213f528..9561d32 100644 --- a/modules/deployment/files/modules/deploy.py +++ b/modules/deployment/files/modules/deploy.py @@ -263,7 +263,15 @@ gitmodules_list = __salt__['file.find'](location, name='.gitmodules') for gitmodules in gitmodules_list: gitmodules_dir = os.path.dirname(gitmodules) -# First ensure we're working with an unmodified .gitmodules file +# Check to see if this is even a repo with submodules. Some repos +# have git repositories checked into the repository and kept the +# git configuration files when doing so. This will cause our submodule +# calls to fail. +cmd = '/usr/bin/git submodule status --quiet' +status = __salt__['cmd.retcode'](cmd, gitmodules_dir) +if status != 0: +continue +# Ensure we're working with an unmodified .gitmodules file cmd = '/usr/bin/git checkout .gitmodules' status = __salt__['cmd.retcode'](cmd, gitmodules_dir) if status != 0: -- To view, visit https://gerrit.wikimedia.org/r/121574 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8b1fe930452a811d50fe3b0c8319f46d2faa318b Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add missing config from role::deployment::salt_masters::labs - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add missing config from role::deployment::salt_masters::labs .. Add missing config from role::deployment::salt_masters::labs Change-Id: Ia460f7a8583694dcc6248e0fa6acb3342f7567ec --- M manifests/role/deployment.pp 1 file changed, 5 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index f93eb11..d31ad55 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -223,6 +223,11 @@ 'db' => '0', }, } + class { '::role::deployment::config': } + class { 'deployment::salt_master': +repo_config => $role::deployment::config::repo_config, +deployment_config => $deployment_config, + } } class role::deployment::deployment_servers::labs { -- To view, visit https://gerrit.wikimedia.org/r/118432 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ia460f7a8583694dcc6248e0fa6acb3342f7567ec Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add missing config from role::deployment::salt_masters::labs - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/118432 Change subject: Add missing config from role::deployment::salt_masters::labs .. Add missing config from role::deployment::salt_masters::labs Change-Id: Ia460f7a8583694dcc6248e0fa6acb3342f7567ec --- M manifests/role/deployment.pp 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/32/118432/1 diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index f93eb11..d31ad55 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -223,6 +223,11 @@ 'db' => '0', }, } + class { '::role::deployment::config': } + class { 'deployment::salt_master': +repo_config => $role::deployment::config::repo_config, +deployment_config => $deployment_config, + } } class role::deployment::deployment_servers::labs { -- To view, visit https://gerrit.wikimedia.org/r/118432 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia460f7a8583694dcc6248e0fa6acb3342f7567ec Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Simplify trebuchet developer environment creation - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Simplify trebuchet developer environment creation .. Simplify trebuchet developer environment creation This change simplifies the creation of developer environments in labs, allowing users to specify generic labs roles and override masters/deployment servers as necessary. Change-Id: I9297b297e26489f149ea1701756d7313acfaf042 --- M manifests/role/deployment.pp M manifests/role/salt.pp 2 files changed, 38 insertions(+), 94 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index 5741c65..f93eb11 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -1,14 +1,7 @@ # vim: sw=2 ts=2 et -# repo not showing up on tin even after puppet has run on -# sockpuppet, palladium and tin? one possible explanation: -# Ryan_Lane: https://gerrit.wikimedia.org/r/operations/ocg-config.git -# Ryan_Lane: ^^ that's wrong -# Ryan_Lane: just use https://gerrit.wikimedia.org/r/operations/ocg-config -# Ryan_Lane: I ran this on tin: salt-call deploy.deployment_server_init -# Ryan_Lane: to see that -# Ryan_Lane: it showed a git exit code of 128 - +# Configuration info: https://wikitech.wikimedia.org/wiki/Trebuchet#Adding_a_new_repo +# Troubleshooting: https://wikitech.wikimedia.org/wiki/Trebuchet#Troubleshooting class role::deployment::config { $repo_config = { 'integration/kss' => { @@ -155,46 +148,6 @@ } } -class role::deployment::salt_masters::labs { - $deployment_config = { -'parent_dir' => '/srv/deployment', -'servers'=> { -'pmtpa' => 'i-0390.pmtpa.wmflabs', -'eqiad' => 'i-0390.pmtpa.wmflabs', -}, -'redis' => { - 'host' => 'i-0390.pmtpa.wmflabs', - 'port' => '6379', - 'db' => '0', -}, - } - class { '::role::deployment::config': } - class { 'deployment::salt_master': -repo_config => $role::deployment::config::repo_config, -deployment_config => $deployment_config, - } -} - -class role::deployment::salt_masters::sartoris { - $deployment_config = { -'parent_dir' => '/srv/deployment', -'servers'=> { -'pmtpa' => 'i-0822.pmtpa.wmflabs', -'eqiad' => 'i-0822.pmtpa.wmflabs', -}, -'redis' => { - 'host' => 'i-0822.pmtpa.wmflabs', - 'port' => '6379', - 'db' => '0', -}, - } - class { '::role::deployment::config': } - class { 'deployment::salt_master': -repo_config => $role::deployment::config::repo_config, -deployment_config => $deployment_config, - } -} - class role::deployment::deployment_servers::common { # Can't include this while scap is present on tin: # include misc::deployment::scripts @@ -251,46 +204,42 @@ } } -class role::deployment::deployment_servers::labs { - include role::deployment::deployment_servers::common - - apache::vhost { "i-0390.pmtpa.wmflabs": -priority => 10, -vhost_name => "10.4.0.58", -port => 80, -docroot=> "/srv/deployment", -docroot_owner => "sartoris", -docroot_group => "project-deployment-prep", -docroot_dir_allows => ["10.4.0.0/16"], -serveradmin=> "n...@wikimedia.org", -configure_firewall => false, +class role::deployment::salt_masters::labs { + # Enable multiple test environments within a single project + if ( $::deployment_server_override != undef ) { +$deployment_server = $::deployment_server_override + } else { +$deployment_server = "${::instanceproject}-deploy.eqiad.wmflabs" } - class { "redis": -dir => "/srv/redis", -maxmemory => "500Mb", -monitor => "false", - } - sudo_group { "project_deployment_prep_deployment_server": -privileges => [ - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json pillar.data", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.fetch *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.checkout *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json publish.runner deploy.restart *", -], -group => "project-deployment-prep", + $deployment_config = { +'parent_dir' => '/srv/deployment', +'servers'=> { +'pmtpa' => $deployment_server, +'eqiad' => $deployment_server, +}, +'redis' => { + 'host' => $deployment_server, + 'port' => '6379', + 'db' => '0', +}, } } -class role::deployment::deployment_servers::sartoris { +class role::deployment::deployment_servers::labs { include role::deployment::deployment_servers::common - apache::vhost { "i-0822.pmtpa.wmflabs": + # Enable multiple test environments within a single project + if ( $::deploy
[MediaWiki-commits] [Gerrit] Fix eqiad labs range - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/118431 Change subject: Fix eqiad labs range .. Fix eqiad labs range Change-Id: I4e175227a79437cea8a63f0a24fd3a63306881b4 --- M manifests/role/openstack.pp M modules/puppet/manifests/self/master.pp 2 files changed, 3 insertions(+), 3 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/31/118431/1 diff --git a/manifests/role/openstack.pp b/manifests/role/openstack.pp index 032e9cd..fe94685 100644 --- a/manifests/role/openstack.pp +++ b/manifests/role/openstack.pp @@ -9,7 +9,7 @@ private_interface => 'eth1', internal_address=> '10.64.20.4', floating_range => '208.80.153.177/32', -fixed_range => '10.68.0.0/21', +fixed_range => '10.68.16.0/21', multi_host => true, network_manager => 'nova.network.manager.FlatDHCPManager', admin_email => 'root@localhost', @@ -41,7 +41,7 @@ private_interface => 'eth1', internal_address => $::ipaddress_eth0, libvirt_type => 'kvm', -fixed_range=> '10.68.0.0/21', +fixed_range=> '10.68.16.0/21', network_manager=> 'nova.network.manager.FlatDHCPManager', multi_host => true, rabbit_host=> '10.64.20.4', diff --git a/modules/puppet/manifests/self/master.pp b/modules/puppet/manifests/self/master.pp index 9f446b8..9fcc753 100644 --- a/modules/puppet/manifests/self/master.pp +++ b/modules/puppet/manifests/self/master.pp @@ -32,7 +32,7 @@ 'localhost' => '127.0.0.1', default => $::site ? { 'pmtpa' => '10.4.0.0/21', -'eqiad' => '10.68.0.0/21', +'eqiad' => '10.68.16.0/21', } } -- To view, visit https://gerrit.wikimedia.org/r/118431 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4e175227a79437cea8a63f0a24fd3a63306881b4 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] fluoride is no longer a target, switch to eventlogging - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: fluoride is no longer a target, switch to eventlogging .. fluoride is no longer a target, switch to eventlogging Change-Id: I1545087c1b815a365685c4f140cabf8f65ddc3d1 --- M manifests/role/deployment.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index 0c6a1cf..5741c65 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -101,7 +101,7 @@ 'checkout_submodules' => true, }, 'fluoride/fluoride' => { -'grain'=> 'fluoride', +'grain'=> 'eventlogging', 'upstream' => 'https://gerrit.wikimedia.org/r/mediawiki/tools/fluoride', }, 'mwprof/mwprof' => { -- To view, visit https://gerrit.wikimedia.org/r/116927 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1545087c1b815a365685c4f140cabf8f65ddc3d1 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Change required umask for deployment in trigger to 002 - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Change required umask for deployment in trigger to 002 .. Change required umask for deployment in trigger to 002 Change-Id: I48a04d6667b115b50cd83f80220fdb62e0645d05 --- M modules/deployment/templates/gitconfig.erb 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/deployment/templates/gitconfig.erb b/modules/deployment/templates/gitconfig.erb index 95e86e8..9917470 100644 --- a/modules/deployment/templates/gitconfig.erb +++ b/modules/deployment/templates/gitconfig.erb @@ -1,2 +1,2 @@ [deploy] -required-umask = '022' +required-umask = '002' -- To view, visit https://gerrit.wikimedia.org/r/116926 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I48a04d6667b115b50cd83f80220fdb62e0645d05 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] fluoride is no longer a target, switch to eventlogging - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/116927 Change subject: fluoride is no longer a target, switch to eventlogging .. fluoride is no longer a target, switch to eventlogging Change-Id: I1545087c1b815a365685c4f140cabf8f65ddc3d1 --- M manifests/role/deployment.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/27/116927/1 diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index 0c6a1cf..5741c65 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -101,7 +101,7 @@ 'checkout_submodules' => true, }, 'fluoride/fluoride' => { -'grain'=> 'fluoride', +'grain'=> 'eventlogging', 'upstream' => 'https://gerrit.wikimedia.org/r/mediawiki/tools/fluoride', }, 'mwprof/mwprof' => { -- To view, visit https://gerrit.wikimedia.org/r/116927 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1545087c1b815a365685c4f140cabf8f65ddc3d1 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Change required umask for deployment in trigger to 002 - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/116926 Change subject: Change required umask for deployment in trigger to 002 .. Change required umask for deployment in trigger to 002 Change-Id: I48a04d6667b115b50cd83f80220fdb62e0645d05 --- M modules/deployment/templates/gitconfig.erb 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/26/116926/1 diff --git a/modules/deployment/templates/gitconfig.erb b/modules/deployment/templates/gitconfig.erb index 95e86e8..9917470 100644 --- a/modules/deployment/templates/gitconfig.erb +++ b/modules/deployment/templates/gitconfig.erb @@ -1,2 +1,2 @@ [deploy] -required-umask = '022' +required-umask = '002' -- To view, visit https://gerrit.wikimedia.org/r/116926 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I48a04d6667b115b50cd83f80220fdb62e0645d05 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Adding python-gitdb dependency - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Adding python-gitdb dependency .. Adding python-gitdb dependency Change-Id: I49a133b9d75bb1f4da7de66cc95757117753beee --- M modules/deployment/manifests/deployment_server.pp 1 file changed, 3 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/deployment/manifests/deployment_server.pp b/modules/deployment/manifests/deployment_server.pp index 1cfd641..8182932 100644 --- a/modules/deployment/manifests/deployment_server.pp +++ b/modules/deployment/manifests/deployment_server.pp @@ -28,6 +28,9 @@ ensure => present; } } +package { 'python-gitdb': +ensure => present; +} package { 'trebuchet-trigger': ensure => present; } -- To view, visit https://gerrit.wikimedia.org/r/116924 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I49a133b9d75bb1f4da7de66cc95757117753beee Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Change sudo format for pillar fetching and service restarts ... - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Change sudo format for pillar fetching and service restarts for trigger .. Change sudo format for pillar fetching and service restarts for trigger Change-Id: I11188410a5680076a5e9680c1f09a31b02650a5c --- M manifests/role/deployment.pp 1 file changed, 6 insertions(+), 6 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index ab8554e..0c6a1cf 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -242,10 +242,10 @@ } sudo_group { "wikidev_deployment_server": privileges => [ - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json pillar.data", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json pillar.data", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.fetch *", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.checkout *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json publish.runner deploy.restart *", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json publish.runner deploy.restart *", ], group => "wikidev", } @@ -272,10 +272,10 @@ } sudo_group { "project_deployment_prep_deployment_server": privileges => [ - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json pillar.data", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json pillar.data", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.fetch *", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.checkout *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json publish.runner deploy.restart *", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json publish.runner deploy.restart *", ], group => "project-deployment-prep", } @@ -302,10 +302,10 @@ } sudo_group { "project_deployment_prep_deployment_server": privileges => [ - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json pillar.data", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json pillar.data", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.fetch *", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.checkout *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json publish.runner deploy.restart *", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json publish.runner deploy.restart *", ], group => "project-sartoris", } -- To view, visit https://gerrit.wikimedia.org/r/116925 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I11188410a5680076a5e9680c1f09a31b02650a5c Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Change sudo format for pillar fetching and service restarts ... - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/116925 Change subject: Change sudo format for pillar fetching and service restarts for trigger .. Change sudo format for pillar fetching and service restarts for trigger Change-Id: I11188410a5680076a5e9680c1f09a31b02650a5c --- M manifests/role/deployment.pp 1 file changed, 6 insertions(+), 6 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/25/116925/1 diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index ab8554e..0c6a1cf 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -242,10 +242,10 @@ } sudo_group { "wikidev_deployment_server": privileges => [ - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json pillar.data", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json pillar.data", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.fetch *", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.checkout *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json publish.runner deploy.restart *", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json publish.runner deploy.restart *", ], group => "wikidev", } @@ -272,10 +272,10 @@ } sudo_group { "project_deployment_prep_deployment_server": privileges => [ - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json pillar.data", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json pillar.data", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.fetch *", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.checkout *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json publish.runner deploy.restart *", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json publish.runner deploy.restart *", ], group => "project-deployment-prep", } @@ -302,10 +302,10 @@ } sudo_group { "project_deployment_prep_deployment_server": privileges => [ - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json pillar.data", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json pillar.data", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.fetch *", "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.checkout *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json publish.runner deploy.restart *", + "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out=json publish.runner deploy.restart *", ], group => "project-sartoris", } -- To view, visit https://gerrit.wikimedia.org/r/116925 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I11188410a5680076a5e9680c1f09a31b02650a5c Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Adding python-gitdb dependency - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/116924 Change subject: Adding python-gitdb dependency .. Adding python-gitdb dependency Change-Id: I49a133b9d75bb1f4da7de66cc95757117753beee --- M modules/deployment/manifests/deployment_server.pp 1 file changed, 3 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/24/116924/1 diff --git a/modules/deployment/manifests/deployment_server.pp b/modules/deployment/manifests/deployment_server.pp index 1cfd641..8182932 100644 --- a/modules/deployment/manifests/deployment_server.pp +++ b/modules/deployment/manifests/deployment_server.pp @@ -28,6 +28,9 @@ ensure => present; } } +package { 'python-gitdb': +ensure => present; +} package { 'trebuchet-trigger': ensure => present; } -- To view, visit https://gerrit.wikimedia.org/r/116924 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I49a133b9d75bb1f4da7de66cc95757117753beee Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Deployment module changes for trebuchet-trigger - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Deployment module changes for trebuchet-trigger .. Deployment module changes for trebuchet-trigger Change-Id: I4f736f833e85498acddda60f4ea3a8797f44672b --- M manifests/role/deployment.pp D modules/deployment/files/git-deploy/dependencies/l10nupdate-quick D modules/deployment/files/git-deploy/hooks/depends.py D modules/deployment/files/git-deploy/hooks/deploylib.py D modules/deployment/files/git-deploy/hooks/shared.py M modules/deployment/files/modules/deploy.py M modules/deployment/manifests/deployment_server.pp D modules/deployment/templates/git-deploy/git-deploy.conf.erb D modules/deployment/templates/git-deploy/gitconfig.erb D modules/deployment/templates/git-deploy/gitignore.erb A modules/deployment/templates/gitconfig.erb 11 files changed, 28 insertions(+), 534 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index dd6f2b8..ab8554e 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -226,7 +226,7 @@ vhost_name => "10.64.0.196", port => 80, docroot=> "/srv/deployment", -docroot_owner => "sartoris", +docroot_owner => "trebuchet", docroot_group => "wikidev", docroot_dir_allows => ["10.0.0.0/16","10.64.0.0/16","208.80.152.0/22"], serveradmin=> "n...@wikimedia.org", diff --git a/modules/deployment/files/git-deploy/dependencies/l10nupdate-quick b/modules/deployment/files/git-deploy/dependencies/l10nupdate-quick deleted file mode 100755 index 45e17ae..000 --- a/modules/deployment/files/git-deploy/dependencies/l10nupdate-quick +++ /dev/null @@ -1,123 +0,0 @@ -#!/bin/bash - -set -e - -BINDIR=/usr/local/bin - -. /usr/local/lib/mw-deployment-vars.sh - -umask 0002 -echo "Starting l10nupdate-quick at `date`." - -mwVerDbSets=$($BINDIR/mwversionsinuse --withdb) -if [ -z "$mwVerDbSets" ]; then - echo "Obtaining MediaWiki version list FAILED" - exit 1 -fi - -# Update l10n cache -for i in ${mwVerDbSets[@]} -do - mwVerNum=${i%=*} - mwDbName=${i#*=} -slot=`basename "$(readlink -e $MW_COMMON/l10n-$mwVerNum)"` - - if [ ! -z "$1" -a "$1" != "$slot" ] - then - continue - fi - - if [ ! -d "$MW_COMMON/l10n-$mwVerNum" ] - then - echo "Update for $mwVerNum failed: $MW_COMMON/l10n-$mwVerNum does not exist" - continue - fi - - cd $MW_COMMON/l10n-$mwVerNum - - git deploy start - set +e - FAILMSG="" - - trap "{ - echo Cleaning up after signal - git clean -d -f - git reset --hard - git deploy abort - exit 255 - }" SIGINT SIGTERM - - if [ ! -d "$MW_COMMON/l10n-$mwVerNum/cache" ] - then - mkdir $MW_COMMON/l10n-$mwVerNum/cache - fi - - if [ ! -e "$MW_COMMON/l10n-$mwVerNum/ExtensionMessages.php" ] - then - touch $MW_COMMON/l10n-$mwVerNum/ExtensionMessages.php - fi - - if [ ! -e "$MW_COMMON/l10n-$mwVerNum/cache/l10n_cache-en.cdb" ] - then - echo "Building initial localisation cache for $mwVerNum (on $mwDbName)" - if $BINDIR/mwscript rebuildLocalisationCache.php --wiki="$mwDbName" \ - --outdir=$MW_COMMON/l10n-$mwVerNum/cache \ - --threads=12 - then - true - else - FAILMSG="Localisation cache build failed" - fi - fi - - if [ -z "$FAILMSG" ] - then - echo "Updating ExtensionMessages.php for $mwVerNum (on $mwDbName)" - if $BINDIR/mwscript mergeMessageFileList.php --wiki="$mwDbName" \ - --list-file=$MW_COMMON/wmf-config/extension-list \ - --output=$MW_COMMON/l10n-$mwVerNum/ExtensionMessages.php - then - true - else - FAILMSG="ExtensionMessages update failed" - fi - fi - - if [ -z "$FAILMSG" ] - then - echo "Rebuilding localisation cache for $mwVerNum (on $mwDbName)" - if $BINDIR/mwscript rebuildLocalisationCache.php --wiki="$mwDbName" \ - --outdir=$MW_COMMON/l10n-$mwVerNum/cache \ - --threads=12 - then - true - else - FAILMSG="Localisation cache rebuild failed" - fi - fi - - if [ -z "$FAILMSG" ] - then - git add ExtensionMessages.php cache - - if git status --porcelain | grep -q '^[MAD
[MediaWiki-commits] [Gerrit] Use Token and not TokenNoList redis driver for folsom keystone - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Use Token and not TokenNoList redis driver for folsom keystone .. Use Token and not TokenNoList redis driver for folsom keystone Change-Id: I30bd927d53f6a8ceb738441ea713e21645add42e --- M templates/openstack/folsom/keystone/keystone.conf.erb 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/templates/openstack/folsom/keystone/keystone.conf.erb b/templates/openstack/folsom/keystone/keystone.conf.erb index 32a2673..c63cbf4 100644 --- a/templates/openstack/folsom/keystone/keystone.conf.erb +++ b/templates/openstack/folsom/keystone/keystone.conf.erb @@ -74,7 +74,7 @@ [token] <% if keystoneconfig["token_driver"] == 'redis' %> -driver = keystoneredis.token.TokenNoList +driver = keystoneredis.token.Token <% else %> driver = keystone.token.backends.sql.Token <% end %> -- To view, visit https://gerrit.wikimedia.org/r/116066 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I30bd927d53f6a8ceb738441ea713e21645add42e Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Use Token and not TokenNoList redis driver for folsom keystone - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/116066 Change subject: Use Token and not TokenNoList redis driver for folsom keystone .. Use Token and not TokenNoList redis driver for folsom keystone Change-Id: I30bd927d53f6a8ceb738441ea713e21645add42e --- M templates/openstack/folsom/keystone/keystone.conf.erb 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/66/116066/1 diff --git a/templates/openstack/folsom/keystone/keystone.conf.erb b/templates/openstack/folsom/keystone/keystone.conf.erb index 32a2673..c63cbf4 100644 --- a/templates/openstack/folsom/keystone/keystone.conf.erb +++ b/templates/openstack/folsom/keystone/keystone.conf.erb @@ -74,7 +74,7 @@ [token] <% if keystoneconfig["token_driver"] == 'redis' %> -driver = keystoneredis.token.TokenNoList +driver = keystoneredis.token.Token <% else %> driver = keystone.token.backends.sql.Token <% end %> -- To view, visit https://gerrit.wikimedia.org/r/116066 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I30bd927d53f6a8ceb738441ea713e21645add42e Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Enable keystone redis driver and switch replication around - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Enable keystone redis driver and switch replication around .. Enable keystone redis driver and switch replication around Change-Id: Ia7e76379b275837e675c366d702c1eba75cca2a7 --- M manifests/role/keystone.pp 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 380ecdf..4dc971c 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -35,7 +35,7 @@ "labs" => "127.0.0.1", }, token_driver => $realm ? { - 'production' => 'sql', + 'production' => 'redis', 'labs' => 'redis', }, } @@ -83,7 +83,7 @@ if ($::realm == 'production') { $replication = { -'virt0' => 'virt1000.wikimedia.org' +'virt1000' => 'virt0.wikimedia.org' } } else { $replication = { -- To view, visit https://gerrit.wikimedia.org/r/116065 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ia7e76379b275837e675c366d702c1eba75cca2a7 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Enable keystone redis driver and switch replication around - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/116065 Change subject: Enable keystone redis driver and switch replication around .. Enable keystone redis driver and switch replication around Change-Id: Ia7e76379b275837e675c366d702c1eba75cca2a7 --- M manifests/role/keystone.pp 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/65/116065/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 380ecdf..4dc971c 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -35,7 +35,7 @@ "labs" => "127.0.0.1", }, token_driver => $realm ? { - 'production' => 'sql', + 'production' => 'redis', 'labs' => 'redis', }, } @@ -83,7 +83,7 @@ if ($::realm == 'production') { $replication = { -'virt0' => 'virt1000.wikimedia.org' +'virt1000' => 'virt0.wikimedia.org' } } else { $replication = { -- To view, visit https://gerrit.wikimedia.org/r/116065 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia7e76379b275837e675c366d702c1eba75cca2a7 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add an wgOpenStackManagerRestrictedRegions option - change (mediawiki...OpenStackManager)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/116063 Change subject: Add an wgOpenStackManagerRestrictedRegions option .. Add an wgOpenStackManagerRestrictedRegions option This change adds an wgOpenStackManagerRestrictedRegions option to restrict a list of users in a group that is granted the accessrestrictedregions right. Change-Id: Ia097f9627ce334d4d9559bf9fac9393544d601ac --- M OpenStackManager.php M nova/OpenStackNovaController.php 2 files changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/OpenStackManager refs/changes/63/116063/1 diff --git a/OpenStackManager.php b/OpenStackManager.php index 1d27855..99bb8e5 100644 --- a/OpenStackManager.php +++ b/OpenStackManager.php @@ -37,6 +37,7 @@ $wgAvailableRights[] = 'managednsdomain'; $wgAvailableRights[] = 'manageglobalpuppet'; $wgAvailableRights[] = 'loginviashell'; +$wgAvailableRights[] = 'accessrestrictedregions'; $wgHooks['UserRights'][] = 'OpenStackNovaUser::manageShellAccess'; @@ -139,6 +140,9 @@ // will be deemed stale $wgPuppetInterval = 1440; +// A list of regions restricted to a group by right +$wgOpenStackManagerRestrictedRegions = array(); + $dir = dirname( __FILE__ ) . '/'; $wgExtensionMessagesFiles['OpenStackManager'] = $dir . 'OpenStackManager.i18n.php'; diff --git a/nova/OpenStackNovaController.php b/nova/OpenStackNovaController.php index e89fdea..729ca82 100644 --- a/nova/OpenStackNovaController.php +++ b/nova/OpenStackNovaController.php @@ -83,6 +83,8 @@ function getRegions( $service ) { global $wgMemc; + global $wgUser; + global $wgOpenStackManagerRestrictedRegions; // We need to ensure the project token has been // fetched before we can get the regions. @@ -94,6 +96,9 @@ foreach ( $serviceCatalog as $entry ) { if ( $entry->type === "identity" ) { foreach ( $entry->endpoints as $endpoint ) { + if ( !$wgUser->isAllowed( 'accessrestrictedregions' ) && in_array( $wgOpenStackManagerRestrictedRegions, $endpoint->region ) ) { + continue; + } $regions[] = $endpoint->region; } } -- To view, visit https://gerrit.wikimedia.org/r/116063 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia097f9627ce334d4d9559bf9fac9393544d601ac Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/OpenStackManager Gerrit-Branch: master Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Revert "Revert "Reenable redis for keystone in eqiad"" - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Revert "Revert "Reenable redis for keystone in eqiad"" .. Revert "Revert "Reenable redis for keystone in eqiad"" This reverts commit 6a91a745330c0cfd0d2c007864a32a31276a2256. Change-Id: Ifbe1f69748a78530048a2adf2a812d4db262b29b --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 6a0ab07..380ecdf 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -57,7 +57,7 @@ "labs" => "127.0.0.1", }, token_driver => $realm ? { - 'production' => 'sql', + 'production' => 'redis', 'labs' => 'redis', }, } -- To view, visit https://gerrit.wikimedia.org/r/114757 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ifbe1f69748a78530048a2adf2a812d4db262b29b Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Use the Token keystone redis driver rather than the TokenNoL... - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Use the Token keystone redis driver rather than the TokenNoList driver .. Use the Token keystone redis driver rather than the TokenNoList driver Change-Id: Id49ff4a5eb838ce22eecafd96b59695c15f3731c --- M templates/openstack/havana/keystone/keystone.conf.erb 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/templates/openstack/havana/keystone/keystone.conf.erb b/templates/openstack/havana/keystone/keystone.conf.erb index b1cc367..3bf5e93 100644 --- a/templates/openstack/havana/keystone/keystone.conf.erb +++ b/templates/openstack/havana/keystone/keystone.conf.erb @@ -74,7 +74,7 @@ [token] <% if keystoneconfig["token_driver"] == 'redis' %> -driver = keystoneredis.token.TokenNoList +driver = keystoneredis.token.Token <% else %> driver = keystone.token.backends.sql.Token <% end %> -- To view, visit https://gerrit.wikimedia.org/r/114756 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Id49ff4a5eb838ce22eecafd96b59695c15f3731c Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Revert "Revert "Reenable redis for keystone in eqiad"" - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114757 Change subject: Revert "Revert "Reenable redis for keystone in eqiad"" .. Revert "Revert "Reenable redis for keystone in eqiad"" This reverts commit 6a91a745330c0cfd0d2c007864a32a31276a2256. Change-Id: Ifbe1f69748a78530048a2adf2a812d4db262b29b --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/57/114757/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 6a0ab07..380ecdf 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -57,7 +57,7 @@ "labs" => "127.0.0.1", }, token_driver => $realm ? { - 'production' => 'sql', + 'production' => 'redis', 'labs' => 'redis', }, } -- To view, visit https://gerrit.wikimedia.org/r/114757 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifbe1f69748a78530048a2adf2a812d4db262b29b Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Use the Token keystone redis driver rather than the TokenNoL... - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114756 Change subject: Use the Token keystone redis driver rather than the TokenNoList driver .. Use the Token keystone redis driver rather than the TokenNoList driver Change-Id: Id49ff4a5eb838ce22eecafd96b59695c15f3731c --- M templates/openstack/havana/keystone/keystone.conf.erb 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/56/114756/1 diff --git a/templates/openstack/havana/keystone/keystone.conf.erb b/templates/openstack/havana/keystone/keystone.conf.erb index b1cc367..3bf5e93 100644 --- a/templates/openstack/havana/keystone/keystone.conf.erb +++ b/templates/openstack/havana/keystone/keystone.conf.erb @@ -74,7 +74,7 @@ [token] <% if keystoneconfig["token_driver"] == 'redis' %> -driver = keystoneredis.token.TokenNoList +driver = keystoneredis.token.Token <% else %> driver = keystone.token.backends.sql.Token <% end %> -- To view, visit https://gerrit.wikimedia.org/r/114756 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id49ff4a5eb838ce22eecafd96b59695c15f3731c Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Reenable redis for keystone in eqiad - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Reenable redis for keystone in eqiad .. Reenable redis for keystone in eqiad Change-Id: Ic4db20f7f03d96779f635b192b2b24fa05af329b --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 6a0ab07..380ecdf 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -57,7 +57,7 @@ "labs" => "127.0.0.1", }, token_driver => $realm ? { - 'production' => 'sql', + 'production' => 'redis', 'labs' => 'redis', }, } -- To view, visit https://gerrit.wikimedia.org/r/114506 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic4db20f7f03d96779f635b192b2b24fa05af329b Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Fix virt1000 hostname in redis config - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Fix virt1000 hostname in redis config .. Fix virt1000 hostname in redis config Change-Id: I828dcb848c66c7c3f7795af8756446d8d0baab3b --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 10d1529..6a0ab07 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -83,7 +83,7 @@ if ($::realm == 'production') { $replication = { -'virt0' => 'virt1000.eqiad.wmnet' +'virt0' => 'virt1000.wikimedia.org' } } else { $replication = { -- To view, visit https://gerrit.wikimedia.org/r/114505 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I828dcb848c66c7c3f7795af8756446d8d0baab3b Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Reenable redis for keystone in eqiad - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114506 Change subject: Reenable redis for keystone in eqiad .. Reenable redis for keystone in eqiad Change-Id: Ic4db20f7f03d96779f635b192b2b24fa05af329b --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/06/114506/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 6a0ab07..380ecdf 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -57,7 +57,7 @@ "labs" => "127.0.0.1", }, token_driver => $realm ? { - 'production' => 'sql', + 'production' => 'redis', 'labs' => 'redis', }, } -- To view, visit https://gerrit.wikimedia.org/r/114506 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic4db20f7f03d96779f635b192b2b24fa05af329b Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Fix virt1000 hostname in redis config - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114505 Change subject: Fix virt1000 hostname in redis config .. Fix virt1000 hostname in redis config Change-Id: I828dcb848c66c7c3f7795af8756446d8d0baab3b --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/05/114505/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 10d1529..6a0ab07 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -83,7 +83,7 @@ if ($::realm == 'production') { $replication = { -'virt0' => 'virt1000.eqiad.wmnet' +'virt0' => 'virt1000.wikimedia.org' } } else { $replication = { -- To view, visit https://gerrit.wikimedia.org/r/114505 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I828dcb848c66c7c3f7795af8756446d8d0baab3b Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Switch back to sql driver for keystone in eqiad - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Switch back to sql driver for keystone in eqiad .. Switch back to sql driver for keystone in eqiad Change-Id: I7105bd0868a4fb3b625fc14d82655c850aeb6df7 --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index e6cbf3d..10d1529 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -57,7 +57,7 @@ "labs" => "127.0.0.1", }, token_driver => $realm ? { - 'production' => 'redis', + 'production' => 'sql', 'labs' => 'redis', }, } -- To view, visit https://gerrit.wikimedia.org/r/114437 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I7105bd0868a4fb3b625fc14d82655c850aeb6df7 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Switch back to sql driver for keystone in eqiad - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114437 Change subject: Switch back to sql driver for keystone in eqiad .. Switch back to sql driver for keystone in eqiad Change-Id: I7105bd0868a4fb3b625fc14d82655c850aeb6df7 --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/37/114437/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index e6cbf3d..10d1529 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -57,7 +57,7 @@ "labs" => "127.0.0.1", }, token_driver => $realm ? { - 'production' => 'redis', + 'production' => 'sql', 'labs' => 'redis', }, } -- To view, visit https://gerrit.wikimedia.org/r/114437 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7105bd0868a4fb3b625fc14d82655c850aeb6df7 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add DNS ferm rules for labs DNS - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add DNS ferm rules for labs DNS .. Add DNS ferm rules for labs DNS Change-Id: I8a566583c783b298751d974e072053f8fae621f0 --- M manifests/openstack.pp 1 file changed, 5 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Verified; Looks good to me, approved diff --git a/manifests/openstack.pp b/manifests/openstack.pp index c0751be..f2c63f3 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -17,6 +17,11 @@ rule => 'saddr (0.0.0.0/0) proto tcp dport (http https) ACCEPT;', } +# Labs DNS +ferm::rule { 'dns_public': +rule => 'saddr (0.0.0.0/0) proto (udp tcp) dport 53 ACCEPT;', +} + # LDAP ferm::rule { 'ldap_private_labs': rule => 'saddr (10.0.0.0/8 208.80.152.0/22) proto tcp dport (ldap ldaps) ACCEPT;', -- To view, visit https://gerrit.wikimedia.org/r/114436 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8a566583c783b298751d974e072053f8fae621f0 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add DNS ferm rules for labs DNS - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114436 Change subject: Add DNS ferm rules for labs DNS .. Add DNS ferm rules for labs DNS Change-Id: I8a566583c783b298751d974e072053f8fae621f0 --- M manifests/openstack.pp 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/36/114436/1 diff --git a/manifests/openstack.pp b/manifests/openstack.pp index c0751be..f2c63f3 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -17,6 +17,11 @@ rule => 'saddr (0.0.0.0/0) proto tcp dport (http https) ACCEPT;', } +# Labs DNS +ferm::rule { 'dns_public': +rule => 'saddr (0.0.0.0/0) proto (udp tcp) dport 53 ACCEPT;', +} + # LDAP ferm::rule { 'ldap_private_labs': rule => 'saddr (10.0.0.0/8 208.80.152.0/22) proto tcp dport (ldap ldaps) ACCEPT;', -- To view, visit https://gerrit.wikimedia.org/r/114436 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8a566583c783b298751d974e072053f8fae621f0 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add redis config to havana's keystone config file - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add redis config to havana's keystone config file .. Add redis config to havana's keystone config file Change-Id: Id7599673710a5307d0ad73f84bef560ae702e534 --- M templates/openstack/havana/keystone/keystone.conf.erb 1 file changed, 9 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/templates/openstack/havana/keystone/keystone.conf.erb b/templates/openstack/havana/keystone/keystone.conf.erb index 56703c4..b1cc367 100644 --- a/templates/openstack/havana/keystone/keystone.conf.erb +++ b/templates/openstack/havana/keystone/keystone.conf.erb @@ -73,12 +73,21 @@ # template_file = default_catalog.templates [token] +<% if keystoneconfig["token_driver"] == 'redis' %> +driver = keystoneredis.token.TokenNoList +<% else %> driver = keystone.token.backends.sql.Token +<% end %> # Amount of time a token should remain valid (in seconds) # Using 7.1 days, as we'll set MediaWiki to 7 days expiration = 613440 +<% if keystoneconfig["token_driver"] == 'redis' %> +[redis] +password = <%= keystoneconfig["token_driver_password"] %> +<% end -%> + [policy] driver = keystone.policy.backends.rules.Policy -- To view, visit https://gerrit.wikimedia.org/r/114435 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Id7599673710a5307d0ad73f84bef560ae702e534 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add redis config to havana's keystone config file - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114435 Change subject: Add redis config to havana's keystone config file .. Add redis config to havana's keystone config file Change-Id: Id7599673710a5307d0ad73f84bef560ae702e534 --- M templates/openstack/havana/keystone/keystone.conf.erb 1 file changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/35/114435/1 diff --git a/templates/openstack/havana/keystone/keystone.conf.erb b/templates/openstack/havana/keystone/keystone.conf.erb index 56703c4..b1cc367 100644 --- a/templates/openstack/havana/keystone/keystone.conf.erb +++ b/templates/openstack/havana/keystone/keystone.conf.erb @@ -73,12 +73,21 @@ # template_file = default_catalog.templates [token] +<% if keystoneconfig["token_driver"] == 'redis' %> +driver = keystoneredis.token.TokenNoList +<% else %> driver = keystone.token.backends.sql.Token +<% end %> # Amount of time a token should remain valid (in seconds) # Using 7.1 days, as we'll set MediaWiki to 7 days expiration = 613440 +<% if keystoneconfig["token_driver"] == 'redis' %> +[redis] +password = <%= keystoneconfig["token_driver_password"] %> +<% end -%> + [policy] driver = keystone.policy.backends.rules.Policy -- To view, visit https://gerrit.wikimedia.org/r/114435 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id7599673710a5307d0ad73f84bef560ae702e534 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Fix labs_nodes subnet in firewall config - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Fix labs_nodes subnet in firewall config .. Fix labs_nodes subnet in firewall config Change-Id: I623508f449d0cba463935a073684e364ddbb9976 --- M manifests/openstack.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Verified; Looks good to me, approved diff --git a/manifests/openstack.pp b/manifests/openstack.pp index 141f595..c0751be 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -7,7 +7,7 @@ # virt1000 $other_master = '208.80.154.18' } elsif ($::site == 'eqiad') { -$labs_nodes = '10.68.20.0/24' +$labs_nodes = '10.64.20.0/24' # virt0 $other_master = '208.80.152.32' } -- To view, visit https://gerrit.wikimedia.org/r/114434 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I623508f449d0cba463935a073684e364ddbb9976 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Fix labs_nodes subnet in firewall config - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114434 Change subject: Fix labs_nodes subnet in firewall config .. Fix labs_nodes subnet in firewall config Change-Id: I623508f449d0cba463935a073684e364ddbb9976 --- M manifests/openstack.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/34/114434/1 diff --git a/manifests/openstack.pp b/manifests/openstack.pp index 141f595..c0751be 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -7,7 +7,7 @@ # virt1000 $other_master = '208.80.154.18' } elsif ($::site == 'eqiad') { -$labs_nodes = '10.68.20.0/24' +$labs_nodes = '10.64.20.0/24' # virt0 $other_master = '208.80.152.32' } -- To view, visit https://gerrit.wikimedia.org/r/114434 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I623508f449d0cba463935a073684e364ddbb9976 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add http/https to ferm rules on wikitech - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add http/https to ferm rules on wikitech .. Add http/https to ferm rules on wikitech Change-Id: If8ea4d161aa3ee6ff3cf7cedfd7fff48999c3bc3 --- M manifests/openstack.pp 1 file changed, 5 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Verified; Looks good to me, approved Andrew Bogott: Looks good to me, but someone else must approve diff --git a/manifests/openstack.pp b/manifests/openstack.pp index 069fd78..141f595 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -12,6 +12,11 @@ $other_master = '208.80.152.32' } +# Wikitech HTTP/HTTPS +ferm::rule { 'http_public': +rule => 'saddr (0.0.0.0/0) proto tcp dport (http https) ACCEPT;', +} + # LDAP ferm::rule { 'ldap_private_labs': rule => 'saddr (10.0.0.0/8 208.80.152.0/22) proto tcp dport (ldap ldaps) ACCEPT;', -- To view, visit https://gerrit.wikimedia.org/r/114431 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: If8ea4d161aa3ee6ff3cf7cedfd7fff48999c3bc3 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Andrew Bogott Gerrit-Reviewer: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add http/https to ferm rules on wikitech - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114431 Change subject: Add http/https to ferm rules on wikitech .. Add http/https to ferm rules on wikitech Change-Id: If8ea4d161aa3ee6ff3cf7cedfd7fff48999c3bc3 --- M manifests/openstack.pp 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/31/114431/1 diff --git a/manifests/openstack.pp b/manifests/openstack.pp index 069fd78..141f595 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -12,6 +12,11 @@ $other_master = '208.80.152.32' } +# Wikitech HTTP/HTTPS +ferm::rule { 'http_public': +rule => 'saddr (0.0.0.0/0) proto tcp dport (http https) ACCEPT;', +} + # LDAP ferm::rule { 'ldap_private_labs': rule => 'saddr (10.0.0.0/8 208.80.152.0/22) proto tcp dport (ldap ldaps) ACCEPT;', -- To view, visit https://gerrit.wikimedia.org/r/114431 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If8ea4d161aa3ee6ff3cf7cedfd7fff48999c3bc3 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add base::firewall to openstack's firewall class - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add base::firewall to openstack's firewall class .. Add base::firewall to openstack's firewall class Change-Id: Id14b905fbd83c99c0306580a6bd0e8c92bb743b8 --- M manifests/openstack.pp 1 file changed, 2 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Verified; Looks good to me, approved diff --git a/manifests/openstack.pp b/manifests/openstack.pp index c417d84..069fd78 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -1,4 +1,6 @@ class openstack::firewall { +include base::firewall + $labs_private_net = '10.0.0.0/0' if ($::site == 'pmtpa') { $labs_nodes = '10.4.16.0/24' -- To view, visit https://gerrit.wikimedia.org/r/114430 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Id14b905fbd83c99c0306580a6bd0e8c92bb743b8 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add base::firewall to openstack's firewall class - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114430 Change subject: Add base::firewall to openstack's firewall class .. Add base::firewall to openstack's firewall class Change-Id: Id14b905fbd83c99c0306580a6bd0e8c92bb743b8 --- M manifests/openstack.pp 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/30/114430/1 diff --git a/manifests/openstack.pp b/manifests/openstack.pp index c417d84..069fd78 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -1,4 +1,6 @@ class openstack::firewall { +include base::firewall + $labs_private_net = '10.0.0.0/0' if ($::site == 'pmtpa') { $labs_nodes = '10.4.16.0/24' -- To view, visit https://gerrit.wikimedia.org/r/114430 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id14b905fbd83c99c0306580a6bd0e8c92bb743b8 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Change redis config hash key to hostname rather than fqdn - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Change redis config hash key to hostname rather than fqdn .. Change redis config hash key to hostname rather than fqdn Seems the hash key is based on hostname, not fqdn. Change-Id: I7e549635d9f801ea725597a80122231e431984ed --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 419e411..e6cbf3d 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -83,7 +83,7 @@ if ($::realm == 'production') { $replication = { -'virt0.pmtpa.wmnet' => 'virt1000.eqiad.wmnet' +'virt0' => 'virt1000.eqiad.wmnet' } } else { $replication = { -- To view, visit https://gerrit.wikimedia.org/r/114429 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I7e549635d9f801ea725597a80122231e431984ed Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Change redis config hash key to hostname rather than fqdn - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114429 Change subject: Change redis config hash key to hostname rather than fqdn .. Change redis config hash key to hostname rather than fqdn Seems the hash key is based on hostname, not fqdn. Change-Id: I7e549635d9f801ea725597a80122231e431984ed --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/29/114429/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 419e411..e6cbf3d 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -83,7 +83,7 @@ if ($::realm == 'production') { $replication = { -'virt0.pmtpa.wmnet' => 'virt1000.eqiad.wmnet' +'virt0' => 'virt1000.eqiad.wmnet' } } else { $replication = { -- To view, visit https://gerrit.wikimedia.org/r/114429 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7e549635d9f801ea725597a80122231e431984ed Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Move the keystone token driver password back into common - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Move the keystone token driver password back into common .. Move the keystone token driver password back into common There's no need to split this config by realm or site. Change-Id: I183615141cb08c7c4d2c12c114f3a7d1a6f239ab --- M manifests/role/keystone.pp 1 file changed, 1 insertion(+), 6 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index c82dc07..419e411 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -17,6 +17,7 @@ auth_protocol => "http", auth_port => "35357", admin_token => $passwords::openstack::keystone::keystone_admin_token, + token_driver_password => $passwords::openstack::keystone::keystone_db_pass, } } class role::keystone::config::pmtpa inherits role::keystone::config { @@ -36,9 +37,6 @@ token_driver => $realm ? { 'production' => 'sql', 'labs' => 'redis', - }, - token_driver_password => $realm ? { - 'labs' => $passwords::openstack::keystone::keystone_db_pass, }, } $keystoneconfig = merge($pmtpakeystoneconfig, $commonkeystoneconfig) @@ -61,9 +59,6 @@ token_driver => $realm ? { 'production' => 'redis', 'labs' => 'redis', - }, - token_driver_password => $realm ? { - 'labs' => $passwords::openstack::keystone::keystone_db_pass, }, } $keystoneconfig = merge($eqiadkeystoneconfig, $commonkeystoneconfig) -- To view, visit https://gerrit.wikimedia.org/r/114425 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I183615141cb08c7c4d2c12c114f3a7d1a6f239ab Gerrit-PatchSet: 2 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Don't split keystone token password config by realm - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114425 Change subject: Don't split keystone token password config by realm .. Don't split keystone token password config by realm Labs and production are already split for passwords via the private repo. There's no need to split these in the config by realm. Change-Id: I183615141cb08c7c4d2c12c114f3a7d1a6f239ab --- M manifests/role/keystone.pp 1 file changed, 2 insertions(+), 6 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/25/114425/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index c82dc07..6dece03 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -37,9 +37,7 @@ 'production' => 'sql', 'labs' => 'redis', }, - token_driver_password => $realm ? { - 'labs' => $passwords::openstack::keystone::keystone_db_pass, - }, + token_driver_password => $passwords::openstack::keystone::keystone_db_pass, } $keystoneconfig = merge($pmtpakeystoneconfig, $commonkeystoneconfig) } @@ -62,9 +60,7 @@ 'production' => 'redis', 'labs' => 'redis', }, - token_driver_password => $realm ? { - 'labs' => $passwords::openstack::keystone::keystone_db_pass, - }, + token_driver_password => $passwords::openstack::keystone::keystone_db_pass, } $keystoneconfig = merge($eqiadkeystoneconfig, $commonkeystoneconfig) } -- To view, visit https://gerrit.wikimedia.org/r/114425 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I183615141cb08c7c4d2c12c114f3a7d1a6f239ab Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Combine labs and production classes for keystone redis and i... - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Combine labs and production classes for keystone redis and include with keystone server .. Combine labs and production classes for keystone redis and include with keystone server Change-Id: I36d1824f1278b441540882e232a0363ce0e12f96 --- M manifests/role/keystone.pp 1 file changed, 18 insertions(+), 19 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 9073cf1..c82dc07 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -70,37 +70,36 @@ } class role::keystone::server { - include role::keystone::config::pmtpa, - role::keystone::config::eqiad +include role::keystone::config::pmtpa, +role::keystone::config::eqiad - $keystoneconfig = $site ? { - "pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, - "eqiad" => $role::keystone::config::eqiad::keystoneconfig, - } +$keystoneconfig = $site ? { +"pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, +"eqiad" => $role::keystone::config::eqiad::keystoneconfig, +} - class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } +class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } + +include role::keystone::redis } class role::keystone::redis { include passwords::openstack::keystone -class { "::redis": -maxmemory => "250mb", -persist => "aof", -redis_replication => { 'virt0.pmtpa.wmnet' => 'virt1000.eqiad.wmnet' }, -password => $passwords::openstack::keystone::keystone_db_pass, -dir => "/var/lib/redis/", -auto_aof_rewrite_min_size => "64mb", +if ($::realm == 'production') { +$replication = { +'virt0.pmtpa.wmnet' => 'virt1000.eqiad.wmnet' +} +} else { +$replication = { +'nova-precise3' => 'nova-precise2' +} } -} - -class role::keystone::redis::labs { -include passwords::openstack::keystone class { "::redis": maxmemory => "250mb", persist => "aof", -redis_replication => { 'nova-precise3' => 'nova-precise2' }, +redis_replication => $replication, password => $passwords::openstack::keystone::keystone_db_pass, dir => "/var/lib/redis/", auto_aof_rewrite_min_size => "64mb", -- To view, visit https://gerrit.wikimedia.org/r/114424 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I36d1824f1278b441540882e232a0363ce0e12f96 Gerrit-PatchSet: 3 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Combine labs and production classes for keystone redis and i... - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114424 Change subject: Combine labs and production classes for keystone redis and include with keystone server .. Combine labs and production classes for keystone redis and include with keystone server Change-Id: I36d1824f1278b441540882e232a0363ce0e12f96 --- M manifests/role/keystone.pp 1 file changed, 13 insertions(+), 21 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/24/114424/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 9073cf1..924d340 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -70,15 +70,17 @@ } class role::keystone::server { - include role::keystone::config::pmtpa, - role::keystone::config::eqiad +include role::keystone::config::pmtpa, +role::keystone::config::eqiad - $keystoneconfig = $site ? { - "pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, - "eqiad" => $role::keystone::config::eqiad::keystoneconfig, - } +$keystoneconfig = $site ? { +"pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, +"eqiad" => $role::keystone::config::eqiad::keystoneconfig, +} - class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } +class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } + +include role::keystone::redis } class role::keystone::redis { @@ -87,20 +89,10 @@ class { "::redis": maxmemory => "250mb", persist => "aof", -redis_replication => { 'virt0.pmtpa.wmnet' => 'virt1000.eqiad.wmnet' }, -password => $passwords::openstack::keystone::keystone_db_pass, -dir => "/var/lib/redis/", -auto_aof_rewrite_min_size => "64mb", -} -} - -class role::keystone::redis::labs { -include passwords::openstack::keystone - -class { "::redis": -maxmemory => "250mb", -persist => "aof", -redis_replication => { 'nova-precise3' => 'nova-precise2' }, +redis_replication => $realm ? { +'production' => { 'virt0.pmtpa.wmnet' => 'virt1000.eqiad.wmnet' }, +'labs' => { 'nova-precise3' => 'nova-precise2' }, +}, password => $passwords::openstack::keystone::keystone_db_pass, dir => "/var/lib/redis/", auto_aof_rewrite_min_size => "64mb", -- To view, visit https://gerrit.wikimedia.org/r/114424 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I36d1824f1278b441540882e232a0363ce0e12f96 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Enable keystone redis driver for eqiad. - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Enable keystone redis driver for eqiad. .. Enable keystone redis driver for eqiad. Change-Id: Ide28fd3edcce6e0dc312533b25126107d26ab318 --- M manifests/openstack.pp M manifests/role/keystone.pp 2 files changed, 33 insertions(+), 8 deletions(-) Approvals: Ryan Lane: Looks good to me, approved Andrew Bogott: Looks good to me, but someone else must approve jenkins-bot: Verified diff --git a/manifests/openstack.pp b/manifests/openstack.pp index e2e8440..c417d84 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -17,10 +17,15 @@ ferm::rule { 'ldap_backend_private_labs': rule => 'saddr (10.0.0.0/8 208.80.152.0/22) proto tcp dport (1389 1636) ACCEPT;', } -ferm::rule {' ldap_admin_replication': +ferm::rule { 'ldap_admin_replication': rule => "saddr (10.0.0.244 $other_master) proto tcp dport ( 8989) ACCEPT;", } +# Redis replication for keystone +ferm::rule { 'redis_replication': +rule => "saddr ($other_master) proto tcp dport (6379) ACCEPT;", +} + # internal services to Labs virt servers ferm::rule { 'keystone': rule => "saddr ($other_master $labs_nodes) proto tcp dport (5000 35357) ACCEPT;", diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 4964463..9073cf1 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -5,13 +5,6 @@ db_name => "keystone", db_user => "keystone", db_pass => $passwords::openstack::keystone::keystone_db_pass, - token_driver => $realm ? { - 'production' => 'sql', - 'labs' => 'redis', - }, - token_driver_password => $realm ? { - 'labs' => $passwords::openstack::keystone::keystone_db_pass, - }, ldap_base_dn => "dc=wikimedia,dc=org", ldap_user_dn => "uid=novaadmin,ou=people,dc=wikimedia,dc=org", ldap_user_id_attribute => "uid", @@ -40,6 +33,13 @@ "production" => "208.80.152.32", "labs" => "127.0.0.1", }, + token_driver => $realm ? { + 'production' => 'sql', + 'labs' => 'redis', + }, + token_driver_password => $realm ? { + 'labs' => $passwords::openstack::keystone::keystone_db_pass, + }, } $keystoneconfig = merge($pmtpakeystoneconfig, $commonkeystoneconfig) } @@ -58,6 +58,13 @@ "production" => "208.80.154.18", "labs" => "127.0.0.1", }, + token_driver => $realm ? { + 'production' => 'redis', + 'labs' => 'redis', + }, + token_driver_password => $realm ? { + 'labs' => $passwords::openstack::keystone::keystone_db_pass, + }, } $keystoneconfig = merge($eqiadkeystoneconfig, $commonkeystoneconfig) } @@ -74,6 +81,19 @@ class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } } +class role::keystone::redis { +include passwords::openstack::keystone + +class { "::redis": +maxmemory => "250mb", +persist => "aof", +redis_replication => { 'virt0.pmtpa.wmnet' => 'virt1000.eqiad.wmnet' }, +password => $passwords::openstack::keystone::keystone_db_pass, +dir => "/var/lib/redis/", +auto_aof_rewrite_min_size => "64mb", +} +} + class role::keystone::redis::labs { include passwords::openstack::keystone -- To view, visit https://gerrit.wikimedia.org/r/114423 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ide28fd3edcce6e0dc312533b25126107d26ab318 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Andrew Bogott Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Enable keystone redis driver for eqiad. - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/114423 Change subject: Enable keystone redis driver for eqiad. .. Enable keystone redis driver for eqiad. Change-Id: Ide28fd3edcce6e0dc312533b25126107d26ab318 --- M manifests/openstack.pp M manifests/role/keystone.pp 2 files changed, 33 insertions(+), 8 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/23/114423/1 diff --git a/manifests/openstack.pp b/manifests/openstack.pp index e2e8440..c417d84 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -17,10 +17,15 @@ ferm::rule { 'ldap_backend_private_labs': rule => 'saddr (10.0.0.0/8 208.80.152.0/22) proto tcp dport (1389 1636) ACCEPT;', } -ferm::rule {' ldap_admin_replication': +ferm::rule { 'ldap_admin_replication': rule => "saddr (10.0.0.244 $other_master) proto tcp dport ( 8989) ACCEPT;", } +# Redis replication for keystone +ferm::rule { 'redis_replication': +rule => "saddr ($other_master) proto tcp dport (6379) ACCEPT;", +} + # internal services to Labs virt servers ferm::rule { 'keystone': rule => "saddr ($other_master $labs_nodes) proto tcp dport (5000 35357) ACCEPT;", diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index 4964463..9073cf1 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -5,13 +5,6 @@ db_name => "keystone", db_user => "keystone", db_pass => $passwords::openstack::keystone::keystone_db_pass, - token_driver => $realm ? { - 'production' => 'sql', - 'labs' => 'redis', - }, - token_driver_password => $realm ? { - 'labs' => $passwords::openstack::keystone::keystone_db_pass, - }, ldap_base_dn => "dc=wikimedia,dc=org", ldap_user_dn => "uid=novaadmin,ou=people,dc=wikimedia,dc=org", ldap_user_id_attribute => "uid", @@ -40,6 +33,13 @@ "production" => "208.80.152.32", "labs" => "127.0.0.1", }, + token_driver => $realm ? { + 'production' => 'sql', + 'labs' => 'redis', + }, + token_driver_password => $realm ? { + 'labs' => $passwords::openstack::keystone::keystone_db_pass, + }, } $keystoneconfig = merge($pmtpakeystoneconfig, $commonkeystoneconfig) } @@ -58,6 +58,13 @@ "production" => "208.80.154.18", "labs" => "127.0.0.1", }, + token_driver => $realm ? { + 'production' => 'redis', + 'labs' => 'redis', + }, + token_driver_password => $realm ? { + 'labs' => $passwords::openstack::keystone::keystone_db_pass, + }, } $keystoneconfig = merge($eqiadkeystoneconfig, $commonkeystoneconfig) } @@ -74,6 +81,19 @@ class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } } +class role::keystone::redis { +include passwords::openstack::keystone + +class { "::redis": +maxmemory => "250mb", +persist => "aof", +redis_replication => { 'virt0.pmtpa.wmnet' => 'virt1000.eqiad.wmnet' }, +password => $passwords::openstack::keystone::keystone_db_pass, +dir => "/var/lib/redis/", +auto_aof_rewrite_min_size => "64mb", +} +} + class role::keystone::redis::labs { include passwords::openstack::keystone -- To view, visit https://gerrit.wikimedia.org/r/114423 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ide28fd3edcce6e0dc312533b25126107d26ab318 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add redis support to keystone in labs - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add redis support to keystone in labs .. Add redis support to keystone in labs Adding redis support to support token replication between multiple regions in labs. Change-Id: I89cf4cde92a1ccd0f7fd1c3034752e48f4c2a750 --- M manifests/openstack.pp M manifests/role/keystone.pp M templates/openstack/folsom/keystone/keystone.conf.erb 3 files changed, 23 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/openstack.pp b/manifests/openstack.pp index e54deca..e2e8440 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -935,6 +935,13 @@ require => Class["openstack::repo"]; } +if $keystoneconfig['token_driver'] == 'redis' { +package { [ "python-keystone-redis" ]: +ensure => present; +} +} + + service { "keystone": ensure => running, subscribe => File['/etc/keystone/keystone.conf'], diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index f3954f2..4964463 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -5,6 +5,13 @@ db_name => "keystone", db_user => "keystone", db_pass => $passwords::openstack::keystone::keystone_db_pass, + token_driver => $realm ? { + 'production' => 'sql', + 'labs' => 'redis', + }, + token_driver_password => $realm ? { + 'labs' => $passwords::openstack::keystone::keystone_db_pass, + }, ldap_base_dn => "dc=wikimedia,dc=org", ldap_user_dn => "uid=novaadmin,ou=people,dc=wikimedia,dc=org", ldap_user_id_attribute => "uid", diff --git a/templates/openstack/folsom/keystone/keystone.conf.erb b/templates/openstack/folsom/keystone/keystone.conf.erb index 7b0aac5..32a2673 100644 --- a/templates/openstack/folsom/keystone/keystone.conf.erb +++ b/templates/openstack/folsom/keystone/keystone.conf.erb @@ -73,12 +73,21 @@ # template_file = default_catalog.templates [token] +<% if keystoneconfig["token_driver"] == 'redis' %> +driver = keystoneredis.token.TokenNoList +<% else %> driver = keystone.token.backends.sql.Token +<% end %> # Amount of time a token should remain valid (in seconds) # Using 7.1 days, as we'll set MediaWiki to 7 days expiration = 613440 +<% if keystoneconfig["token_driver"] == 'redis' %> +[redis] +password = <%= keystoneconfig["token_driver_password"] %> +<% end -%> + [policy] driver = keystone.policy.backends.rules.Policy -- To view, visit https://gerrit.wikimedia.org/r/105139 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I89cf4cde92a1ccd0f7fd1c3034752e48f4c2a750 Gerrit-PatchSet: 4 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Andrew Bogott Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add redis config for keystone in labs - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add redis config for keystone in labs .. Add redis config for keystone in labs For testing the redis token driver with replication for tokens this change adds a redis server for keystone. Change-Id: Ia2a3bbc7dc6e02aa223b9bde780843f8a3f10322 --- M manifests/role/keystone.pp 1 file changed, 13 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Looks good to me, approved Andrew Bogott: Looks good to me, but someone else must approve jenkins-bot: Verified diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index e1d44d3..f3954f2 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -66,3 +66,16 @@ class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } } + +class role::keystone::redis::labs { +include passwords::openstack::keystone + +class { "::redis": +maxmemory => "250mb", +persist => "aof", +redis_replication => { 'nova-precise3' => 'nova-precise2' }, +password => $passwords::openstack::keystone::keystone_db_pass, +dir => "/var/lib/redis/", +auto_aof_rewrite_min_size => "64mb", +} +} -- To view, visit https://gerrit.wikimedia.org/r/104322 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ia2a3bbc7dc6e02aa223b9bde780843f8a3f10322 Gerrit-PatchSet: 3 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Andrew Bogott Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Ensure resolv.conf is generated properly in labs images - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/113092 Change subject: Ensure resolv.conf is generated properly in labs images .. Ensure resolv.conf is generated properly in labs images When vmbuilder creates new images resolv.conf info is baked into the image, making it datacenter specific. This change empties the /etc/resolvconf/resolv.conf.d/original file in the image so that dhclient's reconfiguration of resolv.conf will work properly. Change-Id: I5d32813d31188fabf77bf0c55344353c9a66e69b --- M modules/labs_vmbuilder/files/postinst.sh 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/92/113092/1 diff --git a/modules/labs_vmbuilder/files/postinst.sh b/modules/labs_vmbuilder/files/postinst.sh index 51b3256..0e1d24b 100644 --- a/modules/labs_vmbuilder/files/postinst.sh +++ b/modules/labs_vmbuilder/files/postinst.sh @@ -1,5 +1,6 @@ #!/bin/bash +chroot $1 echo '' > /etc/resolvconf/resolv.conf.d/original chroot $1 passwd -ld root chroot $1 passwd -ld ubuntu chroot $1 printf "%s\t%s\t%s\t%s\n" cloud-init cloud-init/datasources multiselect "ConfigDrive, Ec2" | debconf-set-selections -- To view, visit https://gerrit.wikimedia.org/r/113092 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5d32813d31188fabf77bf0c55344353c9a66e69b Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Code documentation for trebuchet's deployment module - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/112855 Change subject: Code documentation for trebuchet's deployment module .. Code documentation for trebuchet's deployment module Change-Id: Ide861f9f3edfc90124b8c1cf1f5cbff125bc5bb0 --- M modules/deployment/files/modules/deploy.py 1 file changed, 74 insertions(+), 11 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/55/112855/1 diff --git a/modules/deployment/files/modules/deploy.py b/modules/deployment/files/modules/deploy.py index d3ab312..9fc5ac4 100644 --- a/modules/deployment/files/modules/deploy.py +++ b/modules/deployment/files/modules/deploy.py @@ -81,6 +81,9 @@ config = __pillar__.get('repo_config') config = config[repo] config.setdefault('type', 'git-http') +# location is the location on the filesystem of the repository +# shadow_location is the location on the filesystem of the shadow +# reference repository. if 'location' in config: location = config['location'] shadow_location = '{0}/.{1}'.format(os.path.dirname(location), @@ -107,14 +110,37 @@ scheme = 'git' else: scheme = 'http' +# The url of the repository on the deployment server config['url'] = '{0}://{1}/{2}'.format(scheme, server, repo) +# checkout_submodules determines whether or not this repo should +# recursively fetch and checkout submodules. config.setdefault('checkout_submodules', False) +# dependencies are a set of repositories that should be fetched +# and checked out before this repo. This is a deprecated feature. config.setdefault('dependencies', {}) -config.setdefault('checkout_module_calls', {}) +# fetch_module_calls is a hash of salt modules with a list of arguments +# that will be called at the end of the fetch stage. +# TODO (ryan-lane): add a pre-fetch option config.setdefault('fetch_module_calls', {}) +# checkout_module_calls is a hash of salt modules with a list of arguments +# that will be called at the end of the checkout stage. +# TODO (ryan-lane): add a pre-checkout option +config.setdefault('checkout_module_calls', {}) +# sync_script specifies the script that should be linked to on the +# deployment server for the perl git-deploy. This option is deprecated. config.setdefault('sync_script', 'shared.py') +# upstream specifies the upstream url of the repository and is used +# to clone repositories on the deployment server. config.setdefault('upstream', None) +# shadow_reference determines whether or not to make a reference clone +# of a repository on the minions during the fetch stage. This feature +# enables fetch_module_calls modules to run commands against the current +# checkout of code before it's made live. config.setdefault('shadow_reference', False) +# service_name is the service associated with this repository and +# allows the deployment module to run service restart/stop/start/etc +# for services without allowing end-users the ability to restart all +# services on the targets. config.setdefault('service_name', None) return config @@ -138,6 +164,7 @@ repo_config = __pillar__.get('repo_config') for repo in repo_config: config = get_config(repo) +# Begin deprecated perl git-deploy support repo_sync_dir = '{0}/sync/{1}'.format(hook_dir, os.path.dirname(repo)) sync_link = '{0}/{1}.sync'.format(repo_sync_dir, os.path.basename(repo)) @@ -149,6 +176,7 @@ sync_script = '{0}/sync/{1}'.format(hook_dir, config['sync_script']) __salt__['file.symlink'](sync_script, sync_link) +# End deprecated perl git-deploy support # Clone repo from upstream or init repo with no upstream if not __salt__['file.directory_exists'](config['location'] + '/.git'): if config['upstream']: @@ -211,6 +239,7 @@ stats = {} for repo, config in repo_config.items(): +# Ensure the minion is a deployment target for this repo if config['grain'] not in deployment_target: continue if repo not in stats: @@ -242,10 +271,12 @@ gitmodules_list = __salt__['file.find'](location, name='.gitmodules') for gitmodules in gitmodules_list: gitmodules_dir = os.path.dirname(gitmodules) +# First ensure we're working with an unmodified .gitmodules file cmd = '/usr/bin/git checkout .gitmodules' status = __salt__['cmd.retcode'](cmd, gitmodules_dir) if status != 0: return status +# Get a list of the submodules submodules = [] f = open(gitmodules, 'r') for line in f.readlines(): @@ -254,7 +285,8 @@
[MediaWiki-commits] [Gerrit] Add Sphinx function documentation for deploy module - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/112649 Change subject: Add Sphinx function documentation for deploy module .. Add Sphinx function documentation for deploy module This change adds basic Sphinx documentation to the deploy module. Change-Id: Ia8f04c8bc95ef521192e1ea9fa1a176b4bd58b8a --- M modules/deployment/files/modules/deploy.py 1 file changed, 152 insertions(+), 9 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/49/112649/1 diff --git a/modules/deployment/files/modules/deploy.py b/modules/deployment/files/modules/deploy.py index c024c07..13848d1 100644 --- a/modules/deployment/files/modules/deploy.py +++ b/modules/deployment/files/modules/deploy.py @@ -12,6 +12,8 @@ def _get_redis_serv(): ''' Return a redis server object + +:rtype: A Redis object ''' deployment_config = __pillar__.get('deployment_config') deploy_redis = deployment_config['redis'] @@ -22,6 +24,16 @@ def _check_in(function, repo): +""" +Private function used for reporting that a function has started. +Writes to redis with basic status information. + +:param function: The function being reported on. +:type function: str +:param repo: The repository being acted on. +:type repo: str +:rtype: None +""" serv = _get_redis_serv() minion = __grains__.get('id') timestamp = time.time() @@ -39,6 +51,16 @@ def _map_args(repo, args): +""" +Maps a set of arguments to a predefined set of values. Currently only +__REPO__ is support and will be replaced with the repository name. + +:param repo: The repo name used for mapping. +:type repo: str +:param args: An array of arguments to map. +:type args: list +:rtype: list +""" arg_map = {'__REPO__': repo} mapped_args = [] for arg in args: @@ -47,6 +69,14 @@ def get_config(repo): +""" +Fetches the configuration for this repo from the pillars and returns +a hash with the munged configuration (with defaults and helper config). + +:param repo: The specific repo for which to return config data. +:type repo: str +:rtype: hash +""" deployment_config = __pillar__.get('deployment_config') config = __pillar__.get('repo_config') config = config[repo] @@ -90,6 +120,14 @@ def deployment_server_init(): +""" +Initializes a set of repositories on the deployment server. This +function will only run on the deployment server and will initialize +any repository defined in the pillar configuration. This function is +safe to call at any point. + +:rtype: int +""" serv = _get_redis_serv() is_deployment_server = __grains__.get('deployment_server') hook_dir = __grains__.get('deployment_global_hook_dir') @@ -134,11 +172,20 @@ def sync_all(): ''' -Sync all repositories. If a repo doesn't exist on target, clone as well. +Sync all repositories for this minion. If a repo doesn't exist on target, +clone it as well. This function will ensure all repositories for the +minion are at the current tag as defined by the master and is +be safe to call at any point. -CLI Example:: +CLI Example (from the master): -salt -G 'cluster:appservers' deploy.sync_all +salt -G 'deployment_target:test' deploy.sync_all + +CLI Example (from a minion): + +salt-call deploy.sync_all + +:rtype: hash ''' repo_config = __pillar__.get('repo_config') deployment_target = __grains__.get('deployment_target') @@ -157,6 +204,23 @@ def _update_gitmodules(config, location, shadow=False): +""" +Finds all .gitmodules in a repository, changes all submodules within them +to point to the correct submodule on the deployment server, then runs +a submodule sync. This function is in support of recursive submodules. + +In the case we need to update a shadow reference repo, the .gitmodules +files will have their submodules point to the reference clone. + +:param config: The config hash for the repo (as pulled from get_config). +:type config: hash +:param location: The location on the filesystem to find the .gitmodules + files. +:type location: str +:param shadow: Defines whether or not this is a shadow reference repo. +:type shadow: bool +:rtype: int +""" gitmodules_list = __salt__['file.find'](location, name='.gitmodules') for gitmodules in gitmodules_list: gitmodules_dir = os.path.dirname(gitmodules) @@ -203,6 +267,21 @@ def _clone(config, location, tag, shadow=False): +""" +Perform a clone of a repo at a specified location, and +do a fetch and checkout of the repo to ensure it's at the +current deployment tag. + +:param config: Config hash as fetched from get_config +:type config:
[MediaWiki-commits] [Gerrit] Fix submodule fetching in trebuchet - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/112605 Change subject: Fix submodule fetching in trebuchet .. Fix submodule fetching in trebuchet At some point fetch && fetch --tags was replaces with fetch --all, which doesn't do the same thing. This change reverts that code so that recursive submodule checkout will again work correctly. Change-Id: I8a878f4a79e0e6688bc7b98ab48e2284980a6517 --- M modules/deployment/files/modules/deploy.py 1 file changed, 11 insertions(+), 5 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/05/112605/1 diff --git a/modules/deployment/files/modules/deploy.py b/modules/deployment/files/modules/deploy.py index 6ec43c9..c024c07 100644 --- a/modules/deployment/files/modules/deploy.py +++ b/modules/deployment/files/modules/deploy.py @@ -286,20 +286,26 @@ def _fetch_location(config, location, shadow=False): -cmd = '/usr/bin/git fetch --all' +cmd = '/usr/bin/git fetch' +status = __salt__['cmd.retcode'](cmd, location) +if status != 0: +return status +cmd = '/usr/bin/git fetch --tags' status = __salt__['cmd.retcode'](cmd, location) if status != 0: return status -# TODO: update .gitmodules recursively, then run submodule commands -# recursively. if config['checkout_submodules']: ret = _update_gitmodules(config, location, shadow) if ret != 0: return ret -# fetch all submodules and tag for submodules -cmd = '/usr/bin/git submodule foreach --recursive git fetch --all' +# fetch all submodules and tags for submodules +cmd = '/usr/bin/git submodule foreach --recursive git fetch' +status = __salt__['cmd.retcode'](cmd, location) +if status != 0: +return status +cmd = '/usr/bin/git submodule foreach --recursive git fetch --tags' status = __salt__['cmd.retcode'](cmd, location) if status != 0: return status -- To view, visit https://gerrit.wikimedia.org/r/112605 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8a878f4a79e0e6688bc7b98ab48e2284980a6517 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Ensure submodules are checked out on the deployment server - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/112319 Change subject: Ensure submodules are checked out on the deployment server .. Ensure submodules are checked out on the deployment server If checkout_submodules is enabled on minions at any point and the deployment server doesn't have the submodules checked out then the minions will enter an unrecoverable state after the initial deploy. This change always does a submodule recursive init for repos to ensure this condition can't happen. Change-Id: I125e9275d813dc34baed10878c19be94e4a0251e --- M modules/deployment/files/modules/deploy.py 1 file changed, 17 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/19/112319/1 diff --git a/modules/deployment/files/modules/deploy.py b/modules/deployment/files/modules/deploy.py index 6ec43c9..614f60e 100644 --- a/modules/deployment/files/modules/deploy.py +++ b/modules/deployment/files/modules/deploy.py @@ -115,10 +115,25 @@ if config['upstream']: cmd = '/usr/bin/git clone %s/.git %s' % (config['upstream'], config['location']) +status = __salt__['cmd.retcode'](cmd, runas=deploy_user, + umask=002) +if status != 0: +ret_status = 1 +continue +# We don't check the checkout_submodules config flag here +# on purpose. The deployment server should always have a +# fully recursive clone and minions should decide whether +# or not they'll use the submodules. This avoids consistency +# issues in the case where submodules are later enabled, but +# someone forgets to check them out. +cmd = '/usr/bin/git submodule update --init --recursive' +status = __salt__['cmd.retcode'](cmd, runas=deploy_user, + umask=002, + config['location']) else: cmd = '/usr/bin/git init %s' % (config['location']) -status = __salt__['cmd.retcode'](cmd, runas=deploy_user, - umask=002) +status = __salt__['cmd.retcode'](cmd, runas=deploy_user, + umask=002) if status != 0: return status # git clone does ignores umask and does explicit mkdir with 755 -- To view, visit https://gerrit.wikimedia.org/r/112319 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I125e9275d813dc34baed10878c19be94e4a0251e Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Continue on single repo failures for deployment server init - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/112317 Change subject: Continue on single repo failures for deployment server init .. Continue on single repo failures for deployment server init Rather than failing immediately if a single repo is broken (for instance if the upstream is listed incorrectly), continue on to other non-broken repos. Change-Id: I27add880f7f4b864f7357a298d42cd5c8bd7e8cd --- M modules/deployment/files/modules/deploy.py 1 file changed, 7 insertions(+), 4 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/17/112317/1 diff --git a/modules/deployment/files/modules/deploy.py b/modules/deployment/files/modules/deploy.py index 6ec43c9..ce592ad 100644 --- a/modules/deployment/files/modules/deploy.py +++ b/modules/deployment/files/modules/deploy.py @@ -90,11 +90,12 @@ def deployment_server_init(): +ret_status = 0 serv = _get_redis_serv() is_deployment_server = __grains__.get('deployment_server') hook_dir = __grains__.get('deployment_global_hook_dir') if not is_deployment_server: -return 0 +return ret_status deploy_user = __grains__.get('deployment_repo_user') repo_config = __pillar__.get('repo_config') for repo in repo_config: @@ -120,7 +121,8 @@ status = __salt__['cmd.retcode'](cmd, runas=deploy_user, umask=002) if status != 0: -return status +ret_status = 1 +continue # git clone does ignores umask and does explicit mkdir with 755 __salt__['file.set_mode'](config['location'], 2775) # Set the repo name in the repo's config @@ -128,8 +130,9 @@ status = __salt__['cmd.retcode'](cmd, cwd=config['location'], runas=deploy_user, umask=002) if status != 0: -return status -return 0 +ret_status = 1 +continue +return ret_status def sync_all(): -- To view, visit https://gerrit.wikimedia.org/r/112317 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I27add880f7f4b864f7357a298d42cd5c8bd7e8cd Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Remove old parsoid deploy repos and remove parsoid salt module - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/112316 Change subject: Remove old parsoid deploy repos and remove parsoid salt module .. Remove old parsoid deploy repos and remove parsoid salt module It seems at some point the parsoid module was removed from the repo, but wasn't removed from the salt config. The missing module breaks the module/grain/pillar refresh across the cluster. This change cleans that up and removes any reference to it (which removes the old parsoid deploy repos). Change-Id: I74d63e9dfd516f7afc27f111d5919f6f81db813a --- M manifests/role/deployment.pp M modules/deployment/manifests/salt_master.pp 2 files changed, 0 insertions(+), 20 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/16/112316/1 diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index 38ef7ec..cb20a3c 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -80,17 +80,6 @@ 'grain'=> 'gdash', 'upstream' => 'https://gerrit.wikimedia.org/r/operations/software/gdash', }, -'parsoid/Parsoid'=> { -'grain' => 'parsoid', -'upstream' => 'https://gerrit.wikimedia.org/r/mediawiki/extensions/Parsoid', -'checkout_module_calls' => { -'parsoid.config_symlink' => ['__REPO__'], -}, -'service_name' => 'parsoid', -}, -'parsoid/config' => { -'grain' => 'parsoid', -}, 'parsoid/deploy'=> { 'grain' => 'parsoid', 'upstream' => 'https://gerrit.wikimedia.org/r/p/mediawiki/services/parsoid/deploy', diff --git a/modules/deployment/manifests/salt_master.pp b/modules/deployment/manifests/salt_master.pp index 288bccc..dbac3a4 100644 --- a/modules/deployment/manifests/salt_master.pp +++ b/modules/deployment/manifests/salt_master.pp @@ -95,14 +95,6 @@ require => [File[$returner_dir]], } -file { "${module_dir}/parsoid.py": -source => 'puppet:///modules/deployment/modules/parsoid.py', -mode=> '0555', -owner => 'root', -group => 'root', -require => [File[$module_dir]], -} - file { "${module_dir}/mwprof.py": source => 'puppet:///modules/deployment/modules/mwprof.py', mode=> '0555', @@ -139,7 +131,6 @@ exec { 'refresh_deployment_modules': command => "/usr/bin/salt -G 'deployment_target:*' saltutil.sync_modules", subscribe => [File["${module_dir}/deploy.py"], -File["${module_dir}/parsoid.py"], File["${module_dir}/mwprof.py"], File["${module_dir}/mediawiki.py"]], refreshonly => true, -- To view, visit https://gerrit.wikimedia.org/r/112316 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I74d63e9dfd516f7afc27f111d5919f6f81db813a Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Simplify trebuchet developer environment creation - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/112315 Change subject: Simplify trebuchet developer environment creation .. Simplify trebuchet developer environment creation This change simplifies the creation of developer environments in labs, allowing users to specify generic labs roles and override masters/deployment servers as necessary. Change-Id: I9297b297e26489f149ea1701756d7313acfaf042 --- M manifests/role/deployment.pp M manifests/role/salt.pp 2 files changed, 42 insertions(+), 83 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/15/112315/1 diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index 38ef7ec..5f59c90 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -1,14 +1,7 @@ # vim: sw=2 ts=2 et -# repo not showing up on tin even after puppet has run on -# sockpuppet, palladium and tin? one possible explanation: -# Ryan_Lane: https://gerrit.wikimedia.org/r/operations/ocg-config.git -# Ryan_Lane: ^^ that's wrong -# Ryan_Lane: just use https://gerrit.wikimedia.org/r/operations/ocg-config -# Ryan_Lane: I ran this on tin: salt-call deploy.deployment_server_init -# Ryan_Lane: to see that -# Ryan_Lane: it showed a git exit code of 128 - +# Configuration info: https://wikitech.wikimedia.org/wiki/Trebuchet#Adding_a_new_repo +# Troubleshooting: https://wikitech.wikimedia.org/wiki/Trebuchet#Troubleshooting class role::deployment::config { $repo_config = { 'integration/kss' => { @@ -162,46 +155,6 @@ } } -class role::deployment::salt_masters::labs { - $deployment_config = { -'parent_dir' => '/srv/deployment', -'servers'=> { -'pmtpa' => 'i-0390.pmtpa.wmflabs', -'eqiad' => 'i-0390.pmtpa.wmflabs', -}, -'redis' => { - 'host' => 'i-0390.pmtpa.wmflabs', - 'port' => '6379', - 'db' => '0', -}, - } - class { '::role::deployment::config': } - class { 'deployment::salt_master': -repo_config => $role::deployment::config::repo_config, -deployment_config => $deployment_config, - } -} - -class role::deployment::salt_masters::sartoris { - $deployment_config = { -'parent_dir' => '/srv/deployment', -'servers'=> { -'pmtpa' => 'i-0822.pmtpa.wmflabs', -'eqiad' => 'i-0822.pmtpa.wmflabs', -}, -'redis' => { - 'host' => 'i-0822.pmtpa.wmflabs', - 'port' => '6379', - 'db' => '0', -}, - } - class { '::role::deployment::config': } - class { 'deployment::salt_master': -repo_config => $role::deployment::config::repo_config, -deployment_config => $deployment_config, - } -} - class role::deployment::deployment_servers::common { # Can't include this while scap is present on tin: # include misc::deployment::scripts @@ -258,46 +211,47 @@ } } -class role::deployment::deployment_servers::labs { - include role::deployment::deployment_servers::common - - apache::vhost { "i-0390.pmtpa.wmflabs": -priority => 10, -vhost_name => "10.4.0.58", -port => 80, -docroot=> "/srv/deployment", -docroot_owner => "sartoris", -docroot_group => "project-deployment-prep", -docroot_dir_allows => ["10.4.0.0/16"], -serveradmin=> "n...@wikimedia.org", -configure_firewall => false, +class role::deployment::salt_masters::labs { + # Enable multiple test environments within a single project + if ( $::deployment_server_override != undef ) { +$deployment_server = $::deployment_server_override + } else { +$deployment_server = "${::instanceproject}-deploy.pmtpa.wmflabs" } - class { "redis": -dir => "/srv/redis", -maxmemory => "500Mb", -monitor => "false", + $deployment_config = { +'parent_dir' => '/srv/deployment', +'servers'=> { +'pmtpa' => $deployment_server, +'eqiad' => $deployment_server, +}, +'redis' => { + 'host' => $deployment_server, + 'port' => '6379', + 'db' => '0', +}, } - sudo_group { "project_deployment_prep_deployment_server": -privileges => [ - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json pillar.data", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.fetch *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet publish.runner deploy.checkout *", - "ALL = (root) NOPASSWD: /usr/bin/salt-call -l quiet --out json publish.runner deploy.restart *", -], -group => "project-deployment-prep", + class { '::role::deployment::config': } + class { 'deployment::salt_master': +repo_config => $role::deployment::config::repo_config, +deployment_config => $deployment_config, } } -class role::deployment::deployment_servers::sartoris {
[MediaWiki-commits] [Gerrit] Revoking my access - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/111960 Change subject: Revoking my access .. Revoking my access Change-Id: I16ba0b9fb35c706298234749bdfffad26a37e0c6 --- M manifests/admins.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/60/111960/1 diff --git a/manifests/admins.pp b/manifests/admins.pp index fa259c2..d446e6f 100644 --- a/manifests/admins.pp +++ b/manifests/admins.pp @@ -1584,7 +1584,7 @@ key => "B3NzaC1yc2EBIwAAAQEA5i6EW2Qwvv8bEEVOM9UQnSU9i+83pz0tmJ9zU37jimdMNmuxUb/2hi1mzmJlDRYDiZ08dIIO02MhkkQROQ629kWU+Dyx2RkxAtHF+vDmShpsp/PNSsPs6+3qDJs89Af7SRvAQJ3jVmQqJ1TzqniiLu1Ab87TDJoFNE2WjqlPlUWDLZa88023CO65dL8e907QR7OHYPLxbpiJMLYFvdJ1nByquo9t+iV3Iu8/WQS1JOPsGriN282qyc3EErir03et75kS7h+1Zhr+Z6BB0MO2cd6SJDl1cChcIrlHzs4zpufUzWXq9ELBmIaxYBH5iUYYM4ezSyA+qEbDnEpweJiW5w==" } ssh_authorized_key { "laner@Free-Public-Wifi.local": - ensure => present, + ensure => absent, user=> $username, type=> "ssh-rsa", key => "B3NzaC1yc2EDAQABAAABAQDRsK78adkRJfbYrsZznpbwldoSpQyyQXrXG6WzrJEBAVIAKz5gPSM8zmJ/kj89QygYRaKRPWAcuF5GZhSho15dwDXm5M0ZTva4/m/Hu4H3j7oxx3PKjZKBiygP7mSu/32TJs7FynPGAFVl/B766Snn9Ll/xwrx4lg3v9ZNEpNMJZ0DQTFZ1xXD2Ns08JvxW1csAEoNrpqH6tTdXdHmhurXdKQq1G/JmKR3/KVWbB1MNvUwCY0mQbN1icuy+JsOXbvXEftumigXRV16reLvX3q4sNmYSFfOGOMMW7K9d+nDc4TRNrUjm8R0AEZ6BxTJsvpahDi1gCOfZnGmpGKUEWgZ" -- To view, visit https://gerrit.wikimedia.org/r/111960 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I16ba0b9fb35c706298234749bdfffad26a37e0c6 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add an eventual consistency call for deploy.deployment_serve... - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/111749 Change subject: Add an eventual consistency call for deploy.deployment_server_init .. Add an eventual consistency call for deploy.deployment_server_init In case the salt master fails to call deploy.deployment_server_init make puppet call the function on every puppet run. This call will simply bring repositories into a consistent configuration on the deployment server and won't modify their repository state, so it's safe to run this on every puppet run. Change-Id: Iab6367f9d6e4fd0c7a2043d6c2dbf681b017744e --- M modules/deployment/manifests/deployment_server.pp 1 file changed, 6 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/49/111749/1 diff --git a/modules/deployment/manifests/deployment_server.pp b/modules/deployment/manifests/deployment_server.pp index 69ecb12..c716e9c 100644 --- a/modules/deployment/manifests/deployment_server.pp +++ b/modules/deployment/manifests/deployment_server.pp @@ -31,6 +31,12 @@ } } +exec { 'eventual_consistency_deployment_server_init': +path=> ['/usr/bin'], +command => 'salt-call deploy.deployment_server_init', +require => [Package['salt-minion']]; +} + $deployment_global_hook_dir = "${deployment_git_deploy_dir}/hooks" $deployment_dependencies_dir = "${deployment_git_deploy_dir}/dependencies" file { $deployment_global_hook_dir: -- To view, visit https://gerrit.wikimedia.org/r/111749 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iab6367f9d6e4fd0c7a2043d6c2dbf681b017744e Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Temporarily disable multi-master salt - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/111746 Change subject: Temporarily disable multi-master salt .. Temporarily disable multi-master salt Change-Id: Ie3c8426ac2a19175580c1b4d26faa5e0eab4ded1 --- M manifests/role/salt.pp M modules/salt/templates/minion.erb 2 files changed, 15 insertions(+), 6 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/46/111746/1 diff --git a/manifests/role/salt.pp b/manifests/role/salt.pp index 64aa816..3b30486 100644 --- a/manifests/role/salt.pp +++ b/manifests/role/salt.pp @@ -120,7 +120,16 @@ "cluster" => $cluster, } } else { - $salt_master = [ "sockpuppet.pmtpa.wmnet", "palladium.eqiad.wmnet" ] + ## Disabling multi-master salt for now, until synchronization + ## issues are handled for puppet managing salt. + ## When minions fetch modules/returners/pillars/etc. it's necessary + ## for both salt masters to have the same sets of data or inconsistencies + ## can occur. + #$salt_master = $site ? { + # "pmtpa" => [ "sockpuppet.pmtpa.wmnet", "palladium.eqiad.wmnet" ], + # "eqiad" => [ "palladium.eqiad.wmnet", "sockpuppet.pmtpa.wmnet" ], + #} + $salt_master = "palladium.eqiad.wmnet" $salt_client_id = "${fqdn}" $salt_grains = { "realm" => $realm, diff --git a/modules/salt/templates/minion.erb b/modules/salt/templates/minion.erb index 100cfa6..b324632 100644 --- a/modules/salt/templates/minion.erb +++ b/modules/salt/templates/minion.erb @@ -8,14 +8,14 @@ # Set the location of the salt master server, if the master server cannot be # resolved, then the minion will fail to start. -<% if salt_master.is_a? Array %> +<% if salt_master.is_a? Array -%> master: -<% salt_master.each do |name| %> +<% salt_master.each do |name| -%> - <%= name %> -<% end %> -<% else %> +<% end -%> +<% else -%> master: <%= salt_master %> -<% end %> +<% end -%> # Set the number of seconds to wait before attempting to resolve # the master hostname if name resolution fails. Defaults to 30 seconds. -- To view, visit https://gerrit.wikimedia.org/r/111746 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie3c8426ac2a19175580c1b4d26faa5e0eab4ded1 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Deployment module changes for trebuchet-trigger - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/110239 Change subject: Deployment module changes for trebuchet-trigger .. Deployment module changes for trebuchet-trigger Change-Id: I4f736f833e85498acddda60f4ea3a8797f44672b --- M manifests/role/deployment.pp D modules/deployment/files/git-deploy/dependencies/l10nupdate-quick D modules/deployment/files/git-deploy/hooks/depends.py D modules/deployment/files/git-deploy/hooks/deploylib.py D modules/deployment/files/git-deploy/hooks/shared.py M modules/deployment/files/modules/deploy.py M modules/deployment/manifests/deployment_server.pp D modules/deployment/templates/git-deploy/git-deploy.conf.erb D modules/deployment/templates/git-deploy/gitconfig.erb D modules/deployment/templates/git-deploy/gitignore.erb 10 files changed, 17 insertions(+), 508 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/39/110239/1 diff --git a/manifests/role/deployment.pp b/manifests/role/deployment.pp index 30e0967..7f483a6 100644 --- a/manifests/role/deployment.pp +++ b/manifests/role/deployment.pp @@ -227,7 +227,7 @@ vhost_name => "10.64.0.196", port => 80, docroot=> "/srv/deployment", -docroot_owner => "sartoris", +docroot_owner => "trebuchet", docroot_group => "wikidev", docroot_dir_allows => ["10.0.0.0/16","10.64.0.0/16","208.80.152.0/22"], serveradmin=> "n...@wikimedia.org", @@ -260,7 +260,7 @@ vhost_name => "10.4.0.58", port => 80, docroot=> "/srv/deployment", -docroot_owner => "sartoris", +docroot_owner => "trebuchet", docroot_group => "project-deployment-prep", docroot_dir_allows => ["10.4.0.0/16"], serveradmin=> "n...@wikimedia.org", diff --git a/modules/deployment/files/git-deploy/dependencies/l10nupdate-quick b/modules/deployment/files/git-deploy/dependencies/l10nupdate-quick deleted file mode 100755 index 45e17ae..000 --- a/modules/deployment/files/git-deploy/dependencies/l10nupdate-quick +++ /dev/null @@ -1,123 +0,0 @@ -#!/bin/bash - -set -e - -BINDIR=/usr/local/bin - -. /usr/local/lib/mw-deployment-vars.sh - -umask 0002 -echo "Starting l10nupdate-quick at `date`." - -mwVerDbSets=$($BINDIR/mwversionsinuse --withdb) -if [ -z "$mwVerDbSets" ]; then - echo "Obtaining MediaWiki version list FAILED" - exit 1 -fi - -# Update l10n cache -for i in ${mwVerDbSets[@]} -do - mwVerNum=${i%=*} - mwDbName=${i#*=} -slot=`basename "$(readlink -e $MW_COMMON/l10n-$mwVerNum)"` - - if [ ! -z "$1" -a "$1" != "$slot" ] - then - continue - fi - - if [ ! -d "$MW_COMMON/l10n-$mwVerNum" ] - then - echo "Update for $mwVerNum failed: $MW_COMMON/l10n-$mwVerNum does not exist" - continue - fi - - cd $MW_COMMON/l10n-$mwVerNum - - git deploy start - set +e - FAILMSG="" - - trap "{ - echo Cleaning up after signal - git clean -d -f - git reset --hard - git deploy abort - exit 255 - }" SIGINT SIGTERM - - if [ ! -d "$MW_COMMON/l10n-$mwVerNum/cache" ] - then - mkdir $MW_COMMON/l10n-$mwVerNum/cache - fi - - if [ ! -e "$MW_COMMON/l10n-$mwVerNum/ExtensionMessages.php" ] - then - touch $MW_COMMON/l10n-$mwVerNum/ExtensionMessages.php - fi - - if [ ! -e "$MW_COMMON/l10n-$mwVerNum/cache/l10n_cache-en.cdb" ] - then - echo "Building initial localisation cache for $mwVerNum (on $mwDbName)" - if $BINDIR/mwscript rebuildLocalisationCache.php --wiki="$mwDbName" \ - --outdir=$MW_COMMON/l10n-$mwVerNum/cache \ - --threads=12 - then - true - else - FAILMSG="Localisation cache build failed" - fi - fi - - if [ -z "$FAILMSG" ] - then - echo "Updating ExtensionMessages.php for $mwVerNum (on $mwDbName)" - if $BINDIR/mwscript mergeMessageFileList.php --wiki="$mwDbName" \ - --list-file=$MW_COMMON/wmf-config/extension-list \ - --output=$MW_COMMON/l10n-$mwVerNum/ExtensionMessages.php - then - true - else - FAILMSG="ExtensionMessages update failed" - fi - fi - - if [ -z "$FAILMSG" ] - then - echo "Rebuilding localisation cache for $mwVerNum (on $mwDbName)" - if $BINDIR/mwscript rebuildLocalisationCache.php --wiki="$mwDbName" \ - --outdir=$MW_COMMON/l10n-$mwVerNu
[MediaWiki-commits] [Gerrit] Add redis support to keystone - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/105139 Change subject: Add redis support to keystone .. Add redis support to keystone Adding redis support to support token replication between multiple regions in labs. Change-Id: I89cf4cde92a1ccd0f7fd1c3034752e48f4c2a750 --- M manifests/openstack.pp M manifests/role/keystone.pp M templates/openstack/folsom/keystone/keystone.conf.erb 3 files changed, 22 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/39/105139/1 diff --git a/manifests/openstack.pp b/manifests/openstack.pp index 307117d..50625cf 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -715,6 +715,12 @@ require => Class["openstack::repo"]; } + if $keystoneconfig['token_driver'] == 'redis' { + package { [ "python-keystone-redis" ]: + ensure => present; + } + } + service { "keystone": ensure => running, subscribe => File['/etc/keystone/keystone.conf'], diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index e1d44d3..a62034e 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -5,6 +5,13 @@ db_name => "keystone", db_user => "keystone", db_pass => $passwords::openstack::keystone::keystone_db_pass, + token_driver => $realm ? { + 'production' => 'sql', + 'labs' => 'redis', + }, + token_driver_password => $realm ? { + 'labs' => $passwords::openstack::keystone::keystone_db_pass, + }, ldap_base_dn => "dc=wikimedia,dc=org", ldap_user_dn => "uid=novaadmin,ou=people,dc=wikimedia,dc=org", ldap_user_id_attribute => "uid", diff --git a/templates/openstack/folsom/keystone/keystone.conf.erb b/templates/openstack/folsom/keystone/keystone.conf.erb index 7b0aac5..32a2673 100644 --- a/templates/openstack/folsom/keystone/keystone.conf.erb +++ b/templates/openstack/folsom/keystone/keystone.conf.erb @@ -73,12 +73,21 @@ # template_file = default_catalog.templates [token] +<% if keystoneconfig["token_driver"] == 'redis' %> +driver = keystoneredis.token.TokenNoList +<% else %> driver = keystone.token.backends.sql.Token +<% end %> # Amount of time a token should remain valid (in seconds) # Using 7.1 days, as we'll set MediaWiki to 7 days expiration = 613440 +<% if keystoneconfig["token_driver"] == 'redis' %> +[redis] +password = <%= keystoneconfig["token_driver_password"] %> +<% end -%> + [policy] driver = keystone.policy.backends.rules.Policy -- To view, visit https://gerrit.wikimedia.org/r/105139 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I89cf4cde92a1ccd0f7fd1c3034752e48f4c2a750 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Fix misplaced closing brace for manage-exports - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Fix misplaced closing brace for manage-exports .. Fix misplaced closing brace for manage-exports Seems the cron exists because there was a misplaced brace. Change-Id: I12f17c580fc50154fa487ec05455ea6e8b8a7e4a --- M modules/ldap/manifests/client.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/ldap/manifests/client.pp b/modules/ldap/manifests/client.pp index b513ba0..1f3bd9a 100644 --- a/modules/ldap/manifests/client.pp +++ b/modules/ldap/manifests/client.pp @@ -305,7 +305,6 @@ $ircecho_nick = "labs-home-wm" $ircecho_server = "chat.freenode.net" include role::echoirc -} cron { "manage-exports": command => "/usr/sbin/nscd -i passwd; /usr/sbin/nscd -i group; /usr/bin/python /usr/local/sbin/manage-exports --logfile=/var/log/manage-exports.log >/dev/null 2>&1", @@ -318,4 +317,5 @@ ensure => absent; } } +} } -- To view, visit https://gerrit.wikimedia.org/r/105137 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I12f17c580fc50154fa487ec05455ea6e8b8a7e4a Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Fix misplaced closing brace for manage-exports - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/105137 Change subject: Fix misplaced closing brace for manage-exports .. Fix misplaced closing brace for manage-exports Seems the cron exists because there was a misplaced brace. Change-Id: I12f17c580fc50154fa487ec05455ea6e8b8a7e4a --- M modules/ldap/manifests/client.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/37/105137/1 diff --git a/modules/ldap/manifests/client.pp b/modules/ldap/manifests/client.pp index b513ba0..1f3bd9a 100644 --- a/modules/ldap/manifests/client.pp +++ b/modules/ldap/manifests/client.pp @@ -305,7 +305,6 @@ $ircecho_nick = "labs-home-wm" $ircecho_server = "chat.freenode.net" include role::echoirc -} cron { "manage-exports": command => "/usr/sbin/nscd -i passwd; /usr/sbin/nscd -i group; /usr/bin/python /usr/local/sbin/manage-exports --logfile=/var/log/manage-exports.log >/dev/null 2>&1", @@ -318,4 +317,5 @@ ensure => absent; } } +} } -- To view, visit https://gerrit.wikimedia.org/r/105137 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I12f17c580fc50154fa487ec05455ea6e8b8a7e4a Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Specify the command to remove for manage-exports - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Specify the command to remove for manage-exports .. Specify the command to remove for manage-exports The manage-exports cron was still running on a large number of nodes. This change specifies the command, so that it can be found and removed properly. Change-Id: I68ec7836c7f85a64a7ecb19e40e29c7aafba6731 --- M modules/ldap/manifests/client.pp 1 file changed, 1 insertion(+), 0 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/ldap/manifests/client.pp b/modules/ldap/manifests/client.pp index ee23a24..b513ba0 100644 --- a/modules/ldap/manifests/client.pp +++ b/modules/ldap/manifests/client.pp @@ -314,6 +314,7 @@ } else { # This was added to all nodes accidentally cron { "manage-exports": +command => "/usr/sbin/nscd -i passwd; /usr/sbin/nscd -i group; /usr/bin/python /usr/local/sbin/manage-exports --logfile=/var/log/manage-exports.log >/dev/null 2>&1", ensure => absent; } } -- To view, visit https://gerrit.wikimedia.org/r/105136 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I68ec7836c7f85a64a7ecb19e40e29c7aafba6731 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Specify the command to remove for manage-exports - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/105136 Change subject: Specify the command to remove for manage-exports .. Specify the command to remove for manage-exports The manage-exports cron was still running on a large number of nodes. This change specifies the command, so that it can be found and removed properly. Change-Id: I68ec7836c7f85a64a7ecb19e40e29c7aafba6731 --- M modules/ldap/manifests/client.pp 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/36/105136/1 diff --git a/modules/ldap/manifests/client.pp b/modules/ldap/manifests/client.pp index ee23a24..b513ba0 100644 --- a/modules/ldap/manifests/client.pp +++ b/modules/ldap/manifests/client.pp @@ -314,6 +314,7 @@ } else { # This was added to all nodes accidentally cron { "manage-exports": +command => "/usr/sbin/nscd -i passwd; /usr/sbin/nscd -i group; /usr/bin/python /usr/local/sbin/manage-exports --logfile=/var/log/manage-exports.log >/dev/null 2>&1", ensure => absent; } } -- To view, visit https://gerrit.wikimedia.org/r/105136 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I68ec7836c7f85a64a7ecb19e40e29c7aafba6731 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add redis config for keystone in labs - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/104322 Change subject: Add redis config for keystone in labs .. Add redis config for keystone in labs For testing the redis token driver with replication for tokens this change adds a redis server for keystone. Change-Id: Ia2a3bbc7dc6e02aa223b9bde780843f8a3f10322 --- M manifests/role/keystone.pp 1 file changed, 13 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/22/104322/1 diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index e1d44d3..1fefb70 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -66,3 +66,16 @@ class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } } + +class role::keystone::redis::labs { +include passwords::openstack::keystone + +class { "::redis": +maxmemory => "250mb", +persist => "aof", +redis_replication => { 'nova-precise2' => 'nova-precise3' }, +password => $passwords::openstack::keystone::keystone_db_pass, +dir => "/var/lib/redis/", +auto_aof_rewrite_min_size => "64mb", +} +} -- To view, visit https://gerrit.wikimedia.org/r/104322 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia2a3bbc7dc6e02aa223b9bde780843f8a3f10322 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Refactor pub/priv DNS for multi-region support - change (mediawiki...OpenStackManager)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/104320 Change subject: Refactor pub/priv DNS for multi-region support .. Refactor pub/priv DNS for multi-region support Public and private DNS was mostly segregated in the last refactor, but this refactoring fully segregates them into two subclasses of OpenStackNovaHost. This refactor was necessary because the code was previously getting private DNS by specifying the instanceid, but no region, which would return the incorrect record if the instanceid was identical. By splitting the classes apart it made it easier to modify the constructor for private DNS without changing the public DNS code. This change also modifies how private DNS domains are fetched. Rather than fetching a domain based on the instance ID. It fetches the domain based on the region, which simplifies the logic and number of LDAP queries. Change-Id: Idf2cc62d8916eb9e6cac218c16be76e0087d1333 --- M OpenStackManager.php M nova/OpenStackNovaController.php M nova/OpenStackNovaDomain.php M nova/OpenStackNovaHost.php M nova/OpenStackNovaInstance.php A nova/OpenStackNovaPrivateHost.php A nova/OpenStackNovaPublicHost.php M special/SpecialNovaInstance.php 8 files changed, 320 insertions(+), 336 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/OpenStackManager refs/changes/20/104320/1 diff --git a/OpenStackManager.php b/OpenStackManager.php index 8b006b9..3077643 100644 --- a/OpenStackManager.php +++ b/OpenStackManager.php @@ -149,6 +149,8 @@ $wgAutoloadClasses['OpenStackNovaUser'] = $dir . 'nova/OpenStackNovaUser.php'; $wgAutoloadClasses['OpenStackNovaDomain'] = $dir . 'nova/OpenStackNovaDomain.php'; $wgAutoloadClasses['OpenStackNovaHost'] = $dir . 'nova/OpenStackNovaHost.php'; +$wgAutoloadClasses['OpenStackNovaPublicHost'] = $dir . 'nova/OpenStackNovaPublicHost.php'; +$wgAutoloadClasses['OpenStackNovaPrivateHost'] = $dir . 'nova/OpenStackNovaPrivateHost.php'; $wgAutoloadClasses['OpenStackNovaAddress'] = $dir . 'nova/OpenStackNovaAddress.php'; $wgAutoloadClasses['OpenStackNovaSecurityGroup'] = $dir . 'nova/OpenStackNovaSecurityGroup.php'; $wgAutoloadClasses['OpenStackNovaSecurityGroupRule'] = $dir . 'nova/OpenStackNovaSecurityGroupRule.php'; diff --git a/nova/OpenStackNovaController.php b/nova/OpenStackNovaController.php index b5cfaa6..10603b4 100644 --- a/nova/OpenStackNovaController.php +++ b/nova/OpenStackNovaController.php @@ -149,7 +149,7 @@ if ( $ret['code'] === 200 ) { $server = self::_get_property( $ret['body'], 'server' ); if ( $server ) { - return new OpenStackNovaInstance( $server, true ); + return new OpenStackNovaInstance( $server, $this->getRegion(), true ); } } return null; @@ -243,7 +243,7 @@ return $instancesarr; } foreach ( $instances as $instance ) { - $instance = new OpenStackNovaInstance( $instance, true ); + $instance = new OpenStackNovaInstance( $instance, $this->getRegion(), true ); $id = $instance->getInstanceOSId(); $instancesarr[$id] = $instance; } @@ -457,7 +457,7 @@ if ( $ret['code'] !== 202 ) { return null; } - $instance = new OpenStackNovaInstance( $ret['body']->server ); + $instance = new OpenStackNovaInstance( $ret['body']->server, $this->getRegion() ); return $instance; } diff --git a/nova/OpenStackNovaDomain.php b/nova/OpenStackNovaDomain.php index 1871456..f895f4d 100644 --- a/nova/OpenStackNovaDomain.php +++ b/nova/OpenStackNovaDomain.php @@ -190,34 +190,21 @@ } /** -* Get a domain by an instance's ID. Return null if the instance ID entry +* Get a domain by a region. Return null if the region * does not exist. * * @static * @param $instanceid * @return null|OpenStackNovaDomain */ - static function getDomainByInstanceId( $instanceid ) { - global $wgAuth; - global $wgOpenStackManagerLDAPInstanceBaseDN; - - OpenStackNovaLdapConnection::connect(); - - $result = LdapAuthenticationPlugin::ldap_search( $wgAuth->ldapconn, $wgOpenStackManagerLDAPInstanceBaseDN, - '(associateddomain=' . $instanceid . '.*)' ); - $hostInfo = LdapAuthenticationPlugin::ldap_get_entries( $wgAuth->ldapconn, $result ); - if ( $hostInfo['count'] == "0" ) { - return null; + static function getDomainByRegion( $region ) { +
[MediaWiki-commits] [Gerrit] Fully qualify instance resource pages - change (mediawiki...OpenStackManager)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/104144 Change subject: Fully qualify instance resource pages .. Fully qualify instance resource pages With multi-region support it's necessary to use FQDNs for instance resource pages rather than basic instance IDs. This change switches all instance links to FQDNs as well as article creation and deletion. This change also adds a maintenance script that will rename existing articles from IDs to FQDNs. Change-Id: I828c216b45ef8b56e579b0e0b378fbb0388ab240 --- A maintenance/qualifyInstancePages.php M nova/OpenStackNovaInstance.php M special/SpecialNovaAddress.php M special/SpecialNovaInstance.php M special/SpecialNovaResources.php 5 files changed, 106 insertions(+), 24 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/OpenStackManager refs/changes/44/104144/1 diff --git a/maintenance/qualifyInstancePages.php b/maintenance/qualifyInstancePages.php new file mode 100644 index 000..923cf1e --- /dev/null +++ b/maintenance/qualifyInstancePages.php @@ -0,0 +1,61 @@ +mDescription = "Move instance pages from id to fqdn."; + } + + public function execute() { + global $wgAuth; + global $wgOpenStackManagerLDAPUsername; + global $wgOpenStackManagerLDAPUserPassword; + + $user = new OpenStackNovaUser( $wgOpenStackManagerLDAPUsername ); + $userNova = OpenStackNovaController::newFromUser( $user ); + $projects = OpenStackNovaProject::getAllProjects(); + # HACK (please fix): Keystone doesn't deliver services and endpoints unless + # a project token is returned, so we need to feed it a project. Ideally this + # should be configurable, and not hardcoded like this. + $userNova->setProject( 'bastion' ); + $userNova->authenticate( $wgOpenStackManagerLDAPUsername, $wgOpenStackManagerLDAPUserPassword ); + $regions = $userNova->getRegions( 'compute' ); + foreach ( $regions as $region ) { + $this->output( "Running region: " . $region . "\n" ); + foreach ( $projects as $project ) { + $projectName = $project->getProjectName(); + $this->output( "Running project: " . $projectName . "\n" ); + $userNova->setProject( $projectName ); + $userNova->setRegion( $region ); + $instances = $userNova->getInstances(); + if ( ! $instances ) { + $wgAuth->printDebug( "No instance, continuing", NONSENSITIVE ); + continue; + } + foreach ( $instances as $instance ) { + $host = $instance->getHost(); + if ( !$host ) { + $this->output( "Skipping instance due to missing host entry: " . $instance->getInstanceId() . "\n" ); + continue; + } + $this->output( "Renaming instance: " . $instance->getInstanceId() . "\n" ); + $ot = Title::newFromText( $instance->getInstanceId(), NS_NOVA_RESOURCE ); + $nt = Title::newFromText( $host->getFullyQualifiedHostName(), NS_NOVA_RESOURCE ); + $ot->moveTo( $nt, false, 'Maintenance script move from id to fqdn.' ); + } + } + } + + $this->output( "Done.\n" ); + } + +} + +$maintClass = "OpenStackNovaQualifyInstancePages"; +require_once( RUN_MAINTENANCE_IF_MAIN ); diff --git a/nova/OpenStackNovaInstance.php b/nova/OpenStackNovaInstance.php index 44d5325..bfd2197 100644 --- a/nova/OpenStackNovaInstance.php +++ b/nova/OpenStackNovaInstance.php @@ -279,22 +279,24 @@ return; } -// There might already be an autogenerated instance status on this page, -// so set it aside in $instanceStatus. We'll re-insert it at -// the start of the new page. -$instanceStatus = ''; -$oldtext = OpenStackNovaArticle::getText( $this->getInstanceId() ); -if ( $oldtext ) { -$startFlag = ''; -$endFlag = ''; -$statusStart = strpos( $oldtext, $startFlag ); -if ($statusStart !== false) { -$statusEnd = strpos( $oldtext, $endFlag, $statusStart ); -if ( $statusEnd !== false ) { -$instanceStatus = substr( $o
[MediaWiki-commits] [Gerrit] Initial fixes for cross-regional support - change (mediawiki...OpenStackManager)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/104129 Change subject: Initial fixes for cross-regional support .. Initial fixes for cross-regional support This change has fixes for fetching of scoped and unscoped tokens and for enumerating service endpoints. When checking to see if a user has nova credentials, the code was checking if the user had a scoped token for a project, but was also checking without a project listed (which would check an unscoped token). However, the code handled this poorly, which would sometimes inject empty service catalogs into memcache. This change breaks this into two calls: getProjectToken and getUnscopedToken. Additionally, the way the code was previously getting endpoints was leading to only a single region being discovered in the endpoints. Rather than returning the first entry in the endpoints list, the code now enumerates all of them. Also, getEndpoints now returns an array of actual endpoints rather than an array of endpoint arrays, making it easier to enumerate the endpoints. Change-Id: If0f9d9416dea7bd20cd89d54e2096f943b1ab34d --- M nova/OpenStackNovaController.php M nova/OpenStackNovaUser.php 2 files changed, 31 insertions(+), 15 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/OpenStackManager refs/changes/29/104129/1 diff --git a/nova/OpenStackNovaController.php b/nova/OpenStackNovaController.php index b5cfaa6..5d98b24 100644 --- a/nova/OpenStackNovaController.php +++ b/nova/OpenStackNovaController.php @@ -92,7 +92,7 @@ $regions = array(); if ( $serviceCatalog ) { foreach ( $serviceCatalog as $entry ) { - if ( $entry->type === "compute" ) { + if ( $entry->type === "identity" ) { foreach ( $entry->endpoints as $endpoint ) { $regions[] = $endpoint->region; } @@ -716,16 +716,10 @@ return $this->token; } - function getProjectToken( $project ) { + function getUnscopedToken() { global $wgMemc; - // Try to fetch the project token - $projectkey = wfMemcKey( 'openstackmanager', "fulltoken-$project", $this->username ); - $projecttoken = $wgMemc->get( $projectkey ); - if ( is_string( $projecttoken ) ) { - return $projecttoken; - } - // Try to fetch the non-project token + $token = ''; $key = wfMemcKey( 'openstackmanager', "fulltoken", $this->username ); $fulltoken = $wgMemc->get( $key ); if ( is_string( $fulltoken ) ) { @@ -735,14 +729,30 @@ $wikiuser = User::newFromName( $this->user->getUsername() ); $token = OpenStackNovaUser::loadToken( $wikiuser ); if ( !$token ) { - // If there's no non-project token, there's nothing to do, the - // user will need to re-authenticate. return ''; } $wgMemc->set( $key, $token ); } else { $token = $this->token; } + } + return $token; + } + + function getProjectToken( $project ) { + global $wgMemc; + + // Try to fetch the project token + $projectkey = wfMemcKey( 'openstackmanager', "fulltoken-$project", $this->username ); + $projecttoken = $wgMemc->get( $projectkey ); + if ( is_string( $projecttoken ) ) { + return $projecttoken; + } + $token = $this->getUnscopedToken(); + if ( !$token ) { + // If there's no non-project token, there's nothing to do, the + // user will need to re-authenticate. + return ''; } $headers = array( 'Accept: application/json', @@ -773,7 +783,9 @@ if ( $serviceCatalog ) { foreach ( $serviceCatalog as $entry ) { if ( $entry->type === $service ) { - $endpoints[] = $entry->endpoints; + foreach ( $entry->endpoints as $endpoint ) { + $endpoints[] = $endpoint; + } } } } @@ -809,
[MediaWiki-commits] [Gerrit] Fix duplicate definition for openstack in labs - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Fix duplicate definition for openstack in labs .. Fix duplicate definition for openstack in labs Change-Id: I199ddf5ee98a48cbc3f5476f8356497654c6e0f8 --- M manifests/role/nova.pp 1 file changed, 3 insertions(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/nova.pp b/manifests/role/nova.pp index 2652a27..6f4d2b5 100644 --- a/manifests/role/nova.pp +++ b/manifests/role/nova.pp @@ -308,7 +308,9 @@ class role::nova::wikiupdates { -package { 'python-mwclient': ensure => latest; } +if $::realm == "production" { +package { 'python-mwclient': ensure => latest; } +} if ($openstack_version == "essex") { if ($::lsbdistcodename == "lucid") { -- To view, visit https://gerrit.wikimedia.org/r/102349 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I199ddf5ee98a48cbc3f5476f8356497654c6e0f8 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Fix duplicate definition for openstack in labs - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102349 Change subject: Fix duplicate definition for openstack in labs .. Fix duplicate definition for openstack in labs Change-Id: I199ddf5ee98a48cbc3f5476f8356497654c6e0f8 --- M manifests/role/nova.pp 1 file changed, 3 insertions(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/49/102349/1 diff --git a/manifests/role/nova.pp b/manifests/role/nova.pp index 2652a27..6f4d2b5 100644 --- a/manifests/role/nova.pp +++ b/manifests/role/nova.pp @@ -308,7 +308,9 @@ class role::nova::wikiupdates { -package { 'python-mwclient': ensure => latest; } +if $::realm == "production" { +package { 'python-mwclient': ensure => latest; } +} if ($openstack_version == "essex") { if ($::lsbdistcodename == "lucid") { -- To view, visit https://gerrit.wikimedia.org/r/102349 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I199ddf5ee98a48cbc3f5476f8356497654c6e0f8 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Use eth0 IP rather than localhost for multi-region - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102345 Change subject: Use eth0 IP rather than localhost for multi-region .. Use eth0 IP rather than localhost for multi-region For testing multi-region support in labs it's necessary to use the eth0 IP address rather than localhost so that the services will be reachable from the OpenStackManager instance. Change-Id: Ie92c14c0db940e6ccf5e8cc18add1e35c73d975a --- M manifests/role/glance.pp M manifests/role/keystone.pp M manifests/role/nova.pp 3 files changed, 21 insertions(+), 21 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/45/102345/1 diff --git a/manifests/role/glance.pp b/manifests/role/glance.pp index a034e86..d89ba25 100644 --- a/manifests/role/glance.pp +++ b/manifests/role/glance.pp @@ -16,11 +16,11 @@ $pmtpaglanceconfig = { db_host => $realm ? { "production" => "virt0.wikimedia.org", - "labs" => "localhost", + "labs" => $::ipaddress_eth0, }, bind_ip => $realm ? { "production" => "208.80.152.32", - "labs" => "127.0.0.1", + "labs" => $::ipaddress_eth0, }, keystone_admin_token => $keystoneconfig["admin_token"], keystone_auth_host => $keystoneconfig["bind_ip"], @@ -38,11 +38,11 @@ $eqiadglanceconfig = { db_host => $realm ? { "production" => "virt1000.wikimedia.org", - "labs" => "localhost", + "labs" => $::ipaddress_eth0, }, bind_ip => $realm ? { "production" => "208.80.154.18", - "labs" => "127.0.0.1", + "labs" => $::ipaddress_eth0, }, keystone_admin_token => $keystoneconfig["admin_token"], keystone_auth_host => $keystoneconfig["bind_ip"], diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index be5d576..5839d47 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -23,15 +23,15 @@ $pmtpakeystoneconfig = { db_host => $realm ? { "production" => "virt0.wikimedia.org", - "labs" => "localhost", + "labs" => $::ipaddress_eth0, }, ldap_host => $realm ? { "production" => "virt0.wikimedia.org", - "labs" => "localhost", + "labs" => $::ipaddress_eth0, }, bind_ip => $realm ? { "production" => "208.80.152.32", - "labs" => "127.0.0.1", + "labs" => $::ipaddress_eth0, }, } $keystoneconfig = merge($pmtpakeystoneconfig, $commonkeystoneconfig) @@ -41,15 +41,15 @@ $eqiadkeystoneconfig = { db_host => $realm ? { "production" => "virt1000.wikimedia.org", - "labs" => "localhost", + "labs" => $::ipaddress_eth0, }, ldap_host => $realm ? { "production" => "virt1000.wikimedia.org", - "labs" => "localhost", + "labs" => $::ipaddress_eth0, }, bind_ip => $realm ? { "production" => "208.80.154.18", - "labs" => "127.0.0.1", + "labs" => $::ipaddress_eth0, }, } $keystoneconfig = merge($eqiadkeystoneconfig, $commonkeystoneconfig) diff --git a/manifests/role/nova.pp b/manifests/role/nova.pp index 20bd2ee..dcdb65b 100644 --- a/manifests/role/nova.pp +++ b/manifests/role/nova.pp @@ -22,7 +22,7 @@ db_name => "nova", db_user => "nova", db_pass => $passwords::openstack::nova::nova_db_pass, - my_ip => $ipaddress_eth0, + my_ip => $::ipaddress_eth0, ldap_base_dn => "dc=wikimedia,dc=org", ldap_user_dn => "uid=novaadmin,ou=people,dc=wikimedia,dc=org", ldap_user_pass => $passwords::openstack::nova::nova_ldap_user_pass, @@ -49,7 +49,7 @@ $keystoneconfig = $role::keystone::config::pmtpa::keystoneconfig $controller_hostname = $realm ? { "production" => "virt0.wikimedia.org", - "labs" => "localhost", + "labs" => $::ipaddress_eth0, } @@ -72,15 +72,15 @@ network_public_interface => "eth0", network_host => $realm ? { "production" => "10.4.0.1", -
[MediaWiki-commits] [Gerrit] Only install mysql on openstack database node - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Only install mysql on openstack database node .. Only install mysql on openstack database node Change-Id: I663d89cb09b175544008988e719eb882c31ed644 --- M manifests/openstack.pp 1 file changed, 3 insertions(+), 1 deletion(-) Approvals: Ryan Lane: Verified; Looks good to me, approved diff --git a/manifests/openstack.pp b/manifests/openstack.pp index ed77a54..550a923 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -152,7 +152,7 @@ require => Class["openstack::repo"]; } - require mysql, mysql::server::package + require mysql # For IPv6 support package { [ "python-netaddr", "radvd" ]: @@ -294,6 +294,8 @@ $keystone_db_user = $keystoneconfig["db_user"] $keystone_db_pass = $keystoneconfig["db_pass"] + require mysql::server::package + if !defined(Service['mysql']) { service { "mysql": enable => true, -- To view, visit https://gerrit.wikimedia.org/r/102309 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I663d89cb09b175544008988e719eb882c31ed644 Gerrit-PatchSet: 2 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Only install mysql on openstack database node - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102309 Change subject: Only install mysql on openstack database node .. Only install mysql on openstack database node Change-Id: I663d89cb09b175544008988e719eb882c31ed644 --- M manifests/openstack.pp 1 file changed, 3 insertions(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/09/102309/1 diff --git a/manifests/openstack.pp b/manifests/openstack.pp index 0703dc6..0152a65 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -155,7 +155,7 @@ require => Class["openstack::repo"]; } - require mysql, mysql::server::package + require mysql # For IPv6 support package { [ "python-netaddr", "radvd" ]: @@ -297,6 +297,8 @@ $keystone_db_user = $keystoneconfig["db_user"] $keystone_db_pass = $keystoneconfig["db_pass"] + require mysql::server::package + if !defined(Service['mysql']) { service { "mysql": enable => true, -- To view, visit https://gerrit.wikimedia.org/r/102309 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I663d89cb09b175544008988e719eb882c31ed644 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Use image metadata for hidden and default images. - change (mediawiki...OpenStackManager)
Ryan Lane has submitted this change and it was merged. Change subject: Use image metadata for hidden and default images. .. Use image metadata for hidden and default images. Rather than keeping a list of images in the configuration, set metadata items in glance for the images and filter/default based on that. Change-Id: I67162e5fb2de12512e350c29c8b72c77c0ec7df6 --- M OpenStackManager.php M nova/OpenStackNovaImage.php M special/SpecialNovaInstance.php 3 files changed, 12 insertions(+), 7 deletions(-) Approvals: Ryan Lane: Verified; Looks good to me, approved Andrew Bogott: Looks good to me, but someone else must approve diff --git a/OpenStackManager.php b/OpenStackManager.php index a7e174d..56479b3 100644 --- a/OpenStackManager.php +++ b/OpenStackManager.php @@ -108,10 +108,6 @@ ); // Default security rules to add to a project when created $wgOpenStackManagerDefaultSecurityGroupRules = array(); -// Image ID to default to in the instance creation interface -$wgOpenStackManagerInstanceDefaultImage = ""; -// List of image IDs to not display on instance creation interface -$wgOpenStackManagerInstanceBannedImages = array(); // List of instance type names to not display on instance creation interface $wgOpenStackManagerInstanceBannedInstanceTypes = array(); // Whether resource pages should be managed on instance/project creation/deletion diff --git a/nova/OpenStackNovaImage.php b/nova/OpenStackNovaImage.php index 75ed4ed..0caf480 100644 --- a/nova/OpenStackNovaImage.php +++ b/nova/OpenStackNovaImage.php @@ -45,4 +45,12 @@ return $this->image->status; } + /** +* Return the value of the metadata key requested +* +* @return string +*/ + function getImageMetadata( $key ) { + return OpenStackNovaController::_get_property( $this->image->metadata, $key ); + } } diff --git a/special/SpecialNovaInstance.php b/special/SpecialNovaInstance.php index 01500d2..5e42e9f 100644 --- a/special/SpecialNovaInstance.php +++ b/special/SpecialNovaInstance.php @@ -93,7 +93,6 @@ global $wgOpenStackManagerPuppetOptions; global $wgOpenStackManagerInstanceBannedInstanceTypes; global $wgOpenStackManagerInstanceDefaultImage; - global $wgOpenStackManagerInstanceBannedImages; $this->setHeaders(); $this->getOutput()->setPagetitle( $this->msg( 'openstackmanager-createinstance' ) ); @@ -157,11 +156,13 @@ if ( $imageName === '' ) { continue; } - if ( in_array( $image->getImageId(), $wgOpenStackManagerInstanceBannedImages ) ) { + $showImage = $image->getImageMetadata( 'show' ); + if ( !$showImage ) { continue; } $imageLabel = $imageName; - if ( $image->getImageId() === $wgOpenStackManagerInstanceDefaultImage ) { + $isDefault = $image->getImageMetadata( 'default' ); + if ( $isDefault ) { $default = $imageLabel; } $image_keys[$imageLabel] = $image->getImageId(); -- To view, visit https://gerrit.wikimedia.org/r/102285 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I67162e5fb2de12512e350c29c8b72c77c0ec7df6 Gerrit-PatchSet: 2 Gerrit-Project: mediawiki/extensions/OpenStackManager Gerrit-Branch: master Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Andrew Bogott Gerrit-Reviewer: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add localhost permissions for labs testing. - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102214 Change subject: Add localhost permissions for labs testing. .. Add localhost permissions for labs testing. Change-Id: I30b9c8dd54aa9493a22e0ec6c1f8c45e43d926ff --- M templates/openstack/common/controller/glance-user.sql.erb M templates/openstack/common/controller/keystone-user.sql.erb M templates/openstack/common/controller/nova-user.sql.erb M templates/openstack/common/controller/puppet-user.sql.erb 4 files changed, 12 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/14/102214/1 diff --git a/templates/openstack/common/controller/glance-user.sql.erb b/templates/openstack/common/controller/glance-user.sql.erb index dc6450c..6c25ffa 100644 --- a/templates/openstack/common/controller/glance-user.sql.erb +++ b/templates/openstack/common/controller/glance-user.sql.erb @@ -1,2 +1,5 @@ GRANT USAGE ON *.* to '<%= glance_db_user %>'@'%' IDENTIFIED BY '<%= glance_db_pass %>'; +GRANT USAGE ON *.* to '<%= glance_db_user %>'@'127.0.0.1' IDENTIFIED BY '<%= glance_db_pass %>'; +GRANT USAGE ON *.* to '<%= glance_db_user %>'@'localhost' IDENTIFIED BY '<%= glance_db_pass %>'; GRANT ALL PRIVILEGES ON <%= glance_db_name %>.* to '<%= glance_db_user %>'@'%'; +FLUSH PRIVILEGES; diff --git a/templates/openstack/common/controller/keystone-user.sql.erb b/templates/openstack/common/controller/keystone-user.sql.erb index 1789808..39577cf 100644 --- a/templates/openstack/common/controller/keystone-user.sql.erb +++ b/templates/openstack/common/controller/keystone-user.sql.erb @@ -1,2 +1,5 @@ GRANT USAGE ON *.* to '<%= keystone_db_user %>'@'%' IDENTIFIED BY '<%= keystone_db_pass %>'; +GRANT USAGE ON *.* to '<%= keystone_db_user %>'@'127.0.0.1' IDENTIFIED BY '<%= keystone_db_pass %>'; +GRANT USAGE ON *.* to '<%= keystone_db_user %>'@'localhost' IDENTIFIED BY '<%= keystone_db_pass %>'; GRANT ALL PRIVILEGES ON <%= keystone_db_name %>.* to '<%= keystone_db_user %>'@'%'; +FLUSH PRIVILEGES; diff --git a/templates/openstack/common/controller/nova-user.sql.erb b/templates/openstack/common/controller/nova-user.sql.erb index 375e8bf..4cb3042 100644 --- a/templates/openstack/common/controller/nova-user.sql.erb +++ b/templates/openstack/common/controller/nova-user.sql.erb @@ -1,2 +1,5 @@ GRANT USAGE ON *.* to '<%= nova_db_user %>'@'%' IDENTIFIED BY '<%=nova_db_pass %>'; +GRANT USAGE ON *.* to '<%= nova_db_user %>'@'127.0.0.1' IDENTIFIED BY '<%=nova_db_pass %>'; +GRANT USAGE ON *.* to '<%= nova_db_user %>'@'localhost' IDENTIFIED BY '<%=nova_db_pass %>'; GRANT ALL PRIVILEGES ON <%= nova_db_name %>.* to '<%= nova_db_user %>'@'%'; +FLUSH PRIVILEGES; diff --git a/templates/openstack/common/controller/puppet-user.sql.erb b/templates/openstack/common/controller/puppet-user.sql.erb index 1baa3bf..01d0937 100644 --- a/templates/openstack/common/controller/puppet-user.sql.erb +++ b/templates/openstack/common/controller/puppet-user.sql.erb @@ -1,2 +1,5 @@ GRANT USAGE ON *.* to '<%= puppet_db_user %>'@'%' IDENTIFIED BY '<%= puppet_db_pass %>'; +GRANT USAGE ON *.* to '<%= puppet_db_user %>'@'127.0.0.1' IDENTIFIED BY '<%= puppet_db_pass %>'; +GRANT USAGE ON *.* to '<%= puppet_db_user %>'@'localhost' IDENTIFIED BY '<%= puppet_db_pass %>'; GRANT ALL PRIVILEGES ON <%= puppet_db_name %>.* to '<%= puppet_db_user %>'@'%'; +FLUSH PRIVILEGES; -- To view, visit https://gerrit.wikimedia.org/r/102214 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I30b9c8dd54aa9493a22e0ec6c1f8c45e43d926ff Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add localhost permissions for labs testing. - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add localhost permissions for labs testing. .. Add localhost permissions for labs testing. Change-Id: I30b9c8dd54aa9493a22e0ec6c1f8c45e43d926ff --- M templates/openstack/common/controller/glance-user.sql.erb M templates/openstack/common/controller/keystone-user.sql.erb M templates/openstack/common/controller/nova-user.sql.erb M templates/openstack/common/controller/puppet-user.sql.erb 4 files changed, 12 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/templates/openstack/common/controller/glance-user.sql.erb b/templates/openstack/common/controller/glance-user.sql.erb index dc6450c..6c25ffa 100644 --- a/templates/openstack/common/controller/glance-user.sql.erb +++ b/templates/openstack/common/controller/glance-user.sql.erb @@ -1,2 +1,5 @@ GRANT USAGE ON *.* to '<%= glance_db_user %>'@'%' IDENTIFIED BY '<%= glance_db_pass %>'; +GRANT USAGE ON *.* to '<%= glance_db_user %>'@'127.0.0.1' IDENTIFIED BY '<%= glance_db_pass %>'; +GRANT USAGE ON *.* to '<%= glance_db_user %>'@'localhost' IDENTIFIED BY '<%= glance_db_pass %>'; GRANT ALL PRIVILEGES ON <%= glance_db_name %>.* to '<%= glance_db_user %>'@'%'; +FLUSH PRIVILEGES; diff --git a/templates/openstack/common/controller/keystone-user.sql.erb b/templates/openstack/common/controller/keystone-user.sql.erb index 1789808..39577cf 100644 --- a/templates/openstack/common/controller/keystone-user.sql.erb +++ b/templates/openstack/common/controller/keystone-user.sql.erb @@ -1,2 +1,5 @@ GRANT USAGE ON *.* to '<%= keystone_db_user %>'@'%' IDENTIFIED BY '<%= keystone_db_pass %>'; +GRANT USAGE ON *.* to '<%= keystone_db_user %>'@'127.0.0.1' IDENTIFIED BY '<%= keystone_db_pass %>'; +GRANT USAGE ON *.* to '<%= keystone_db_user %>'@'localhost' IDENTIFIED BY '<%= keystone_db_pass %>'; GRANT ALL PRIVILEGES ON <%= keystone_db_name %>.* to '<%= keystone_db_user %>'@'%'; +FLUSH PRIVILEGES; diff --git a/templates/openstack/common/controller/nova-user.sql.erb b/templates/openstack/common/controller/nova-user.sql.erb index 375e8bf..4cb3042 100644 --- a/templates/openstack/common/controller/nova-user.sql.erb +++ b/templates/openstack/common/controller/nova-user.sql.erb @@ -1,2 +1,5 @@ GRANT USAGE ON *.* to '<%= nova_db_user %>'@'%' IDENTIFIED BY '<%=nova_db_pass %>'; +GRANT USAGE ON *.* to '<%= nova_db_user %>'@'127.0.0.1' IDENTIFIED BY '<%=nova_db_pass %>'; +GRANT USAGE ON *.* to '<%= nova_db_user %>'@'localhost' IDENTIFIED BY '<%=nova_db_pass %>'; GRANT ALL PRIVILEGES ON <%= nova_db_name %>.* to '<%= nova_db_user %>'@'%'; +FLUSH PRIVILEGES; diff --git a/templates/openstack/common/controller/puppet-user.sql.erb b/templates/openstack/common/controller/puppet-user.sql.erb index 1baa3bf..01d0937 100644 --- a/templates/openstack/common/controller/puppet-user.sql.erb +++ b/templates/openstack/common/controller/puppet-user.sql.erb @@ -1,2 +1,5 @@ GRANT USAGE ON *.* to '<%= puppet_db_user %>'@'%' IDENTIFIED BY '<%= puppet_db_pass %>'; +GRANT USAGE ON *.* to '<%= puppet_db_user %>'@'127.0.0.1' IDENTIFIED BY '<%= puppet_db_pass %>'; +GRANT USAGE ON *.* to '<%= puppet_db_user %>'@'localhost' IDENTIFIED BY '<%= puppet_db_pass %>'; GRANT ALL PRIVILEGES ON <%= puppet_db_name %>.* to '<%= puppet_db_user %>'@'%'; +FLUSH PRIVILEGES; -- To view, visit https://gerrit.wikimedia.org/r/102214 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I30b9c8dd54aa9493a22e0ec6c1f8c45e43d926ff Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Use image metadata for hidden and default images. - change (mediawiki...OpenStackManager)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102285 Change subject: Use image metadata for hidden and default images. .. Use image metadata for hidden and default images. Rather than keeping a list of images in the configuration, set metadata items in glance for the images and filter/default based on that. Change-Id: I67162e5fb2de12512e350c29c8b72c77c0ec7df6 --- M OpenStackManager.php M nova/OpenStackNovaImage.php M special/SpecialNovaInstance.php 3 files changed, 12 insertions(+), 7 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/OpenStackManager refs/changes/85/102285/1 diff --git a/OpenStackManager.php b/OpenStackManager.php index a7e174d..56479b3 100644 --- a/OpenStackManager.php +++ b/OpenStackManager.php @@ -108,10 +108,6 @@ ); // Default security rules to add to a project when created $wgOpenStackManagerDefaultSecurityGroupRules = array(); -// Image ID to default to in the instance creation interface -$wgOpenStackManagerInstanceDefaultImage = ""; -// List of image IDs to not display on instance creation interface -$wgOpenStackManagerInstanceBannedImages = array(); // List of instance type names to not display on instance creation interface $wgOpenStackManagerInstanceBannedInstanceTypes = array(); // Whether resource pages should be managed on instance/project creation/deletion diff --git a/nova/OpenStackNovaImage.php b/nova/OpenStackNovaImage.php index 75ed4ed..0caf480 100644 --- a/nova/OpenStackNovaImage.php +++ b/nova/OpenStackNovaImage.php @@ -45,4 +45,12 @@ return $this->image->status; } + /** +* Return the value of the metadata key requested +* +* @return string +*/ + function getImageMetadata( $key ) { + return OpenStackNovaController::_get_property( $this->image->metadata, $key ); + } } diff --git a/special/SpecialNovaInstance.php b/special/SpecialNovaInstance.php index 01500d2..fa33fe4 100644 --- a/special/SpecialNovaInstance.php +++ b/special/SpecialNovaInstance.php @@ -93,7 +93,6 @@ global $wgOpenStackManagerPuppetOptions; global $wgOpenStackManagerInstanceBannedInstanceTypes; global $wgOpenStackManagerInstanceDefaultImage; - global $wgOpenStackManagerInstanceBannedImages; $this->setHeaders(); $this->getOutput()->setPagetitle( $this->msg( 'openstackmanager-createinstance' ) ); @@ -157,11 +156,13 @@ if ( $imageName === '' ) { continue; } - if ( in_array( $image->getImageId(), $wgOpenStackManagerInstanceBannedImages ) ) { + $isHidden = $image->getImageMetadata( 'hidden' ); + if ( $isHidden ) { continue; } $imageLabel = $imageName; - if ( $image->getImageId() === $wgOpenStackManagerInstanceDefaultImage ) { + $isDefault = $image->getImageMetadata( 'default' ); + if ( $isDefault ) { $default = $imageLabel; } $image_keys[$imageLabel] = $image->getImageId(); -- To view, visit https://gerrit.wikimedia.org/r/102285 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I67162e5fb2de12512e350c29c8b72c77c0ec7df6 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/OpenStackManager Gerrit-Branch: master Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Enable a labs site override option for nova config - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102185 Change subject: Enable a labs site override option for nova config .. Enable a labs site override option for nova config It's necessary to test multiple regions of openstack within labs so this change allows the configuration to pretend it's a different site. Change-Id: I4308d0ac71051044179acfc408c32607ea751033 --- M manifests/role/glance.pp M manifests/role/keystone.pp M manifests/role/nova.pp 3 files changed, 49 insertions(+), 17 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/85/102185/1 diff --git a/manifests/role/glance.pp b/manifests/role/glance.pp index 772ff77..a034e86 100644 --- a/manifests/role/glance.pp +++ b/manifests/role/glance.pp @@ -56,9 +56,16 @@ include role::glance::config::pmtpa, role::glance::config::eqiad - $glanceconfig = $site ? { - "pmtpa" => $role::glance::config::pmtpa::glanceconfig, - "eqiad" => $role::glance::config::eqiad::glanceconfig, + if $::realm == "labs" and $::openstack_site_override != undef { + $glanceconfig = $::openstack_site_override ? { + "pmtpa" => $role::glance::config::pmtpa::glanceconfig, + "eqiad" => $role::glance::config::eqiad::glanceconfig, + } + } else { + $glanceconfig = $::site ? { + "pmtpa" => $role::glance::config::pmtpa::glanceconfig, + "eqiad" => $role::glance::config::eqiad::glanceconfig, + } } class { "openstack::glance-service": openstack_version => $openstack_version, glanceconfig => $glanceconfig } diff --git a/manifests/role/keystone.pp b/manifests/role/keystone.pp index e1d44d3..be5d576 100644 --- a/manifests/role/keystone.pp +++ b/manifests/role/keystone.pp @@ -59,9 +59,16 @@ include role::keystone::config::pmtpa, role::keystone::config::eqiad - $keystoneconfig = $site ? { - "pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, - "eqiad" => $role::keystone::config::eqiad::keystoneconfig, + if $::realm == "labs" and $::openstack_site_override != undef { + $keystoneconfig = $::openstack_site_override ? { + "pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, + "eqiad" => $role::keystone::config::eqiad::keystoneconfig, + } + } else { + $keystoneconfig = $::site ? { + "pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, + "eqiad" => $role::keystone::config::eqiad::keystoneconfig, + } } class { "openstack::keystone-service": openstack_version => $openstack_version, keystoneconfig => $keystoneconfig } diff --git a/manifests/role/nova.pp b/manifests/role/nova.pp index 2652a27..20bd2ee 100644 --- a/manifests/role/nova.pp +++ b/manifests/role/nova.pp @@ -2,9 +2,16 @@ include role::nova::config::pmtpa, role::nova::config::eqiad - $novaconfig = $site ? { - "pmtpa" => $role::nova::config::pmtpa::novaconfig, - "eqiad" => $role::nova::config::eqiad::novaconfig, + if $::realm == "labs" and $::openstack_site_override != undef { + $novaconfig = $::openstack_site_override ? { + "pmtpa" => $role::nova::config::pmtpa::novaconfig, + "eqiad" => $role::nova::config::eqiad::novaconfig, + } + } else { + $novaconfig = $::site ? { + "pmtpa" => $role::nova::config::pmtpa::novaconfig, + "eqiad" => $role::nova::config::eqiad::novaconfig, + } } } @@ -243,13 +250,24 @@ role::glance::config::pmtpa, role::glance::config::eqiad - $glanceconfig = $site ? { - "pmtpa" => $role::glance::config::pmtpa::glanceconfig, - "eqiad" => $role::glance::config::eqiad::glanceconfig, - } - $keystoneconfig = $site ? { - "pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, - "eqiad" => $role::keystone::config::eqiad::keystoneconfig, + if $::realm == "labs" and $::openstack_site_override != undef { + $glanceconfig = $::openstack_site_override ? { + "pmtpa" => $role::glance::config::pmtpa::glanceconfig, + "eqiad" => $role::glance::config::eqiad::glanceconfig, + } + $keystoneconfig = $::openstack_site_override ? { + "pmtpa" => $role::keystone::config::pmtpa::keystoneconfig, + "eqiad" => $role::keystone::config::eqiad::keystoneconfig, +
[MediaWiki-commits] [Gerrit] Restart nscd and nslcd after reconfiguration - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Restart nscd and nslcd after reconfiguration .. Restart nscd and nslcd after reconfiguration On firstboot for labs instances, it's necessary to restart nslcd and nscd after they are reconfigured. Change-Id: I75e8b017449ceacdfe48786b81a6a1c6a5e1975a --- M modules/labs_vmbuilder/files/firstboot.sh 1 file changed, 2 insertions(+), 0 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/labs_vmbuilder/files/firstboot.sh b/modules/labs_vmbuilder/files/firstboot.sh index e8e5ece..8703b5b 100644 --- a/modules/labs_vmbuilder/files/firstboot.sh +++ b/modules/labs_vmbuilder/files/firstboot.sh @@ -33,6 +33,8 @@ sed -i "s/_MASTER_/${master}/g" /etc/puppet/puppet.conf /etc/init.d/autofs restart +/etc/init.d/nslcd restart +/etc/init.d/nscd restart dpkg-reconfigure -fnoninteractive -pcritical openssh-server /etc/init.d/ssh stop /etc/init.d/ssh start -- To view, visit https://gerrit.wikimedia.org/r/102058 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I75e8b017449ceacdfe48786b81a6a1c6a5e1975a Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Restart nscd and nslcd after reconfiguration - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102058 Change subject: Restart nscd and nslcd after reconfiguration .. Restart nscd and nslcd after reconfiguration On firstboot for labs instances, it's necessary to restart nslcd and nscd after they are reconfigured. Change-Id: I75e8b017449ceacdfe48786b81a6a1c6a5e1975a --- M modules/labs_vmbuilder/files/firstboot.sh 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/58/102058/1 diff --git a/modules/labs_vmbuilder/files/firstboot.sh b/modules/labs_vmbuilder/files/firstboot.sh index e8e5ece..8703b5b 100644 --- a/modules/labs_vmbuilder/files/firstboot.sh +++ b/modules/labs_vmbuilder/files/firstboot.sh @@ -33,6 +33,8 @@ sed -i "s/_MASTER_/${master}/g" /etc/puppet/puppet.conf /etc/init.d/autofs restart +/etc/init.d/nslcd restart +/etc/init.d/nscd restart dpkg-reconfigure -fnoninteractive -pcritical openssh-server /etc/init.d/ssh stop /etc/init.d/ssh start -- To view, visit https://gerrit.wikimedia.org/r/102058 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I75e8b017449ceacdfe48786b81a6a1c6a5e1975a Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Allow salt/puppet access from pmtpa and eqiad labs - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Allow salt/puppet access from pmtpa and eqiad labs .. Allow salt/puppet access from pmtpa and eqiad labs Change-Id: Ia41a80bf171653f260d4a193531f3b68d3dd9035 --- M manifests/openstack.pp 1 file changed, 3 insertions(+), 6 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/openstack.pp b/manifests/openstack.pp index 0703dc6..ed77a54 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -55,10 +55,10 @@ iptables_add_service{ "keystone_service_nova_virt1000": source => "208.80.154.18", service => "keystone_service", jump => "ACCEPT" } iptables_add_service{ "keystone_admin_nova_virt1000": source => "208.80.154.18", service => "keystone_admin", jump => "ACCEPT" } iptables_add_service{ "amanda": source => "208.80.152.170", service => "inetd", jump => "ACCEPT" } + iptables_add_service{ "puppet_private": source => "10.0.0.0/8", service => "puppetmaster", jump => "ACCEPT" } + iptables_add_service{ "salt_publish_private": source => "10.0.0.0/8", service => "salt_publish", jump => "ACCEPT" } + iptables_add_service{ "salt_ret_private": source => "10.0.0.0/8", service => "salt_ret", jump => "ACCEPT" } if ($site == "pmtpa") { - iptables_add_service{ "puppet_private": source => "10.4.0.0/16", service => "puppetmaster", jump => "ACCEPT" } - iptables_add_service{ "salt_publish_private": source => "10.4.0.0/16", service => "salt_publish", jump => "ACCEPT" } - iptables_add_service{ "salt_ret_private": source => "10.4.0.0/16", service => "salt_ret", jump => "ACCEPT" } iptables_add_service{ "mysql_nova": source => "10.4.16.0/24", service => "mysql", jump => "ACCEPT" } iptables_add_service{ "glance_api_nova": source => "10.4.16.0/24", service => "glance_api", jump => "ACCEPT" } iptables_add_service{ "beam2_nova": source => "10.4.16.0/24", service => "beam2", jump => "ACCEPT" } @@ -67,9 +67,6 @@ iptables_add_service{ "keystone_admin_nova": source => "10.4.16.0/24", service => "keystone_admin", jump => "ACCEPT" } } if ($site == "eqiad") { - iptables_add_service{ "puppet_private": source => "10.68.0.0/16", service => "puppetmaster", jump => "ACCEPT" } - iptables_add_service{ "salt_publish_private": source => "10.68.0.0/16", service => "salt_publish", jump => "ACCEPT" } - iptables_add_service{ "salt_ret_private": source => "10.68.0.0/16", service => "salt_ret", jump => "ACCEPT" } iptables_add_service{ "mysql_nova": source => "10.64.20.0/24", service => "mysql", jump => "ACCEPT" } iptables_add_service{ "glance_api_nova": source => "10.64.20.0/24", service => "glance_api", jump => "ACCEPT" } iptables_add_service{ "beam2_nova": source => "10.64.20.0/24", service => "beam2", jump => "ACCEPT" } -- To view, visit https://gerrit.wikimedia.org/r/102052 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ia41a80bf171653f260d4a193531f3b68d3dd9035 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Allow salt/puppet access from pmtpa and eqiad labs - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102052 Change subject: Allow salt/puppet access from pmtpa and eqiad labs .. Allow salt/puppet access from pmtpa and eqiad labs Change-Id: Ia41a80bf171653f260d4a193531f3b68d3dd9035 --- M manifests/openstack.pp 1 file changed, 3 insertions(+), 6 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/52/102052/1 diff --git a/manifests/openstack.pp b/manifests/openstack.pp index 0703dc6..ed77a54 100644 --- a/manifests/openstack.pp +++ b/manifests/openstack.pp @@ -55,10 +55,10 @@ iptables_add_service{ "keystone_service_nova_virt1000": source => "208.80.154.18", service => "keystone_service", jump => "ACCEPT" } iptables_add_service{ "keystone_admin_nova_virt1000": source => "208.80.154.18", service => "keystone_admin", jump => "ACCEPT" } iptables_add_service{ "amanda": source => "208.80.152.170", service => "inetd", jump => "ACCEPT" } + iptables_add_service{ "puppet_private": source => "10.0.0.0/8", service => "puppetmaster", jump => "ACCEPT" } + iptables_add_service{ "salt_publish_private": source => "10.0.0.0/8", service => "salt_publish", jump => "ACCEPT" } + iptables_add_service{ "salt_ret_private": source => "10.0.0.0/8", service => "salt_ret", jump => "ACCEPT" } if ($site == "pmtpa") { - iptables_add_service{ "puppet_private": source => "10.4.0.0/16", service => "puppetmaster", jump => "ACCEPT" } - iptables_add_service{ "salt_publish_private": source => "10.4.0.0/16", service => "salt_publish", jump => "ACCEPT" } - iptables_add_service{ "salt_ret_private": source => "10.4.0.0/16", service => "salt_ret", jump => "ACCEPT" } iptables_add_service{ "mysql_nova": source => "10.4.16.0/24", service => "mysql", jump => "ACCEPT" } iptables_add_service{ "glance_api_nova": source => "10.4.16.0/24", service => "glance_api", jump => "ACCEPT" } iptables_add_service{ "beam2_nova": source => "10.4.16.0/24", service => "beam2", jump => "ACCEPT" } @@ -67,9 +67,6 @@ iptables_add_service{ "keystone_admin_nova": source => "10.4.16.0/24", service => "keystone_admin", jump => "ACCEPT" } } if ($site == "eqiad") { - iptables_add_service{ "puppet_private": source => "10.68.0.0/16", service => "puppetmaster", jump => "ACCEPT" } - iptables_add_service{ "salt_publish_private": source => "10.68.0.0/16", service => "salt_publish", jump => "ACCEPT" } - iptables_add_service{ "salt_ret_private": source => "10.68.0.0/16", service => "salt_ret", jump => "ACCEPT" } iptables_add_service{ "mysql_nova": source => "10.64.20.0/24", service => "mysql", jump => "ACCEPT" } iptables_add_service{ "glance_api_nova": source => "10.64.20.0/24", service => "glance_api", jump => "ACCEPT" } iptables_add_service{ "beam2_nova": source => "10.64.20.0/24", service => "beam2", jump => "ACCEPT" } -- To view, visit https://gerrit.wikimedia.org/r/102052 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia41a80bf171653f260d4a193531f3b68d3dd9035 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Remove run once logic from firstboot.sh - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Remove run once logic from firstboot.sh .. Remove run once logic from firstboot.sh Run once logic is already added by vm-builder itself, so there's no need for the logic in the script. Change-Id: Icaeafcb477858f3be6de295314789357844fb2a7 --- M modules/labs_vmbuilder/files/firstboot.sh 1 file changed, 0 insertions(+), 8 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/labs_vmbuilder/files/firstboot.sh b/modules/labs_vmbuilder/files/firstboot.sh index 978b5a3..e8e5ece 100644 --- a/modules/labs_vmbuilder/files/firstboot.sh +++ b/modules/labs_vmbuilder/files/firstboot.sh @@ -1,11 +1,5 @@ #!/bin/bash -if [ -f '/root/.firstboot' ] -then - # Only run firstboot once - exit -fi - echo 'Enabling console logging for puppet while it does the initial run' echo 'daemon.* |/dev/console' > /etc/rsyslog.d/60-puppet.conf restart rsyslog @@ -51,5 +45,3 @@ # Force initial puppet run puppet agent --onetime --verbose --no-daemonize --no-splay --show_diff --waitforcert=10 --certname=${idfqdn} --server=${master} - -touch /root/.firstboot -- To view, visit https://gerrit.wikimedia.org/r/102034 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Icaeafcb477858f3be6de295314789357844fb2a7 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add -y condition to salt-key for puppetsigner script - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add -y condition to salt-key for puppetsigner script .. Add -y condition to salt-key for puppetsigner script Change-Id: I97df4333f44af09536fffb1522750bbf8e8a30e9 --- M modules/ldap/files/scripts/puppetsigner.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/ldap/files/scripts/puppetsigner.py b/modules/ldap/files/scripts/puppetsigner.py index 323df24..074d7be 100644 --- a/modules/ldap/files/scripts/puppetsigner.py +++ b/modules/ldap/files/scripts/puppetsigner.py @@ -58,7 +58,7 @@ if not PosixData: subprocess.Popen(['/usr/bin/salt-key -y -d ' + host], shell=True, stdout=subprocess.PIPE) else: -subprocess.Popen(['/usr/bin/salt-key -a ' + host], shell=True, stderr=subprocess.PIPE) +subprocess.Popen(['/usr/bin/salt-key -y -a ' + host], shell=True, stderr=subprocess.PIPE) except ldap.PROTOCOL_ERROR: sys.stderr.write("There was an LDAP protocol error; see traceback.\n") traceback.print_exc(file=sys.stderr) -- To view, visit https://gerrit.wikimedia.org/r/102038 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I97df4333f44af09536fffb1522750bbf8e8a30e9 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add -y condition to salt-key for puppetsigner script - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102038 Change subject: Add -y condition to salt-key for puppetsigner script .. Add -y condition to salt-key for puppetsigner script Change-Id: I97df4333f44af09536fffb1522750bbf8e8a30e9 --- M modules/ldap/files/scripts/puppetsigner.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/38/102038/1 diff --git a/modules/ldap/files/scripts/puppetsigner.py b/modules/ldap/files/scripts/puppetsigner.py index 323df24..074d7be 100644 --- a/modules/ldap/files/scripts/puppetsigner.py +++ b/modules/ldap/files/scripts/puppetsigner.py @@ -58,7 +58,7 @@ if not PosixData: subprocess.Popen(['/usr/bin/salt-key -y -d ' + host], shell=True, stdout=subprocess.PIPE) else: -subprocess.Popen(['/usr/bin/salt-key -a ' + host], shell=True, stderr=subprocess.PIPE) +subprocess.Popen(['/usr/bin/salt-key -y -a ' + host], shell=True, stderr=subprocess.PIPE) except ldap.PROTOCOL_ERROR: sys.stderr.write("There was an LDAP protocol error; see traceback.\n") traceback.print_exc(file=sys.stderr) -- To view, visit https://gerrit.wikimedia.org/r/102038 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I97df4333f44af09536fffb1522750bbf8e8a30e9 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add secondary salt master into labs minion config - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add secondary salt master into labs minion config .. Add secondary salt master into labs minion config Change-Id: Ie813f157b070ce3f1388dd3a0765f38c905d80e7 --- M manifests/role/salt.pp 1 file changed, 4 insertions(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/salt.pp b/manifests/role/salt.pp index 8fb021e..64aa816 100644 --- a/manifests/role/salt.pp +++ b/manifests/role/salt.pp @@ -102,7 +102,10 @@ if ( $::salt_master_override != undef ) { $salt_master = $::salt_master_override } else { - $salt_master = "virt0.wikimedia.org" + $salt_master = $site ? { + "pmtpa" => ["virt0.wikimedia.org", "virt1000.wikimedia.org"], + "eqiad" => ["virt1000.wikimedia.org", "virt0.wikimedia.org"], + } } if ( $::salt_master_finger_override != undef ) { $salt_master_finger = $::salt_master_finger_override -- To view, visit https://gerrit.wikimedia.org/r/102029 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie813f157b070ce3f1388dd3a0765f38c905d80e7 Gerrit-PatchSet: 3 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Make virt1000 a secondary salt master for labs - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Make virt1000 a secondary salt master for labs .. Make virt1000 a secondary salt master for labs Change-Id: I53f1bb02b2555dd87649aa2ea26de2353ad44939 --- M manifests/site.pp 1 file changed, 3 insertions(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/site.pp b/manifests/site.pp index 64742e9..c9665ad 100644 --- a/manifests/site.pp +++ b/manifests/site.pp @@ -2874,7 +2874,9 @@ role::dns::ldap, ldap::role::server::labs, ldap::role::client::labs, -role::nova::controller +role::nova::controller, +role::salt::masters::labs, +role::deployment::salt_masters::labs } node "virt0.wikimedia.org" { -- To view, visit https://gerrit.wikimedia.org/r/102026 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I53f1bb02b2555dd87649aa2ea26de2353ad44939 Gerrit-PatchSet: 4 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Remove run once logic from firstboot.sh - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102034 Change subject: Remove run once logic from firstboot.sh .. Remove run once logic from firstboot.sh Run once logic is already added by vm-builder itself, so there's no need for the logic in the script. Change-Id: Icaeafcb477858f3be6de295314789357844fb2a7 --- M modules/labs_vmbuilder/files/firstboot.sh 1 file changed, 0 insertions(+), 8 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/34/102034/1 diff --git a/modules/labs_vmbuilder/files/firstboot.sh b/modules/labs_vmbuilder/files/firstboot.sh index 978b5a3..e8e5ece 100644 --- a/modules/labs_vmbuilder/files/firstboot.sh +++ b/modules/labs_vmbuilder/files/firstboot.sh @@ -1,11 +1,5 @@ #!/bin/bash -if [ -f '/root/.firstboot' ] -then - # Only run firstboot once - exit -fi - echo 'Enabling console logging for puppet while it does the initial run' echo 'daemon.* |/dev/console' > /etc/rsyslog.d/60-puppet.conf restart rsyslog @@ -51,5 +45,3 @@ # Force initial puppet run puppet agent --onetime --verbose --no-daemonize --no-splay --show_diff --waitforcert=10 --certname=${idfqdn} --server=${master} - -touch /root/.firstboot -- To view, visit https://gerrit.wikimedia.org/r/102034 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Icaeafcb477858f3be6de295314789357844fb2a7 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add secondary salt master into labs minion config - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102029 Change subject: Add secondary salt master into labs minion config .. Add secondary salt master into labs minion config Change-Id: Ie813f157b070ce3f1388dd3a0765f38c905d80e7 --- M manifests/role/salt.pp 1 file changed, 4 insertions(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/29/102029/1 diff --git a/manifests/role/salt.pp b/manifests/role/salt.pp index 8fb021e..cf9609e 100644 --- a/manifests/role/salt.pp +++ b/manifests/role/salt.pp @@ -102,7 +102,10 @@ if ( $::salt_master_override != undef ) { $salt_master = $::salt_master_override } else { - $salt_master = "virt0.wikimedia.org" + $salt_master = ? $site { + "pmtpa" => ["virt0.wikimedia.org", "virt1000.wikimedia.org"], + "eqiad" => ["virt1000.wikimedia.org", "virt0.wikimedia.org"], + } } if ( $::salt_master_finger_override != undef ) { $salt_master_finger = $::salt_master_finger_override -- To view, visit https://gerrit.wikimedia.org/r/102029 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie813f157b070ce3f1388dd3a0765f38c905d80e7 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Make virt1000 a secondary salt master for labs - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102026 Change subject: Make virt1000 a secondary salt master for labs .. Make virt1000 a secondary salt master for labs Change-Id: I53f1bb02b2555dd87649aa2ea26de2353ad44939 --- M manifests/site.pp 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/26/102026/1 diff --git a/manifests/site.pp b/manifests/site.pp index e335379..531aad4 100644 --- a/manifests/site.pp +++ b/manifests/site.pp @@ -2875,6 +2875,8 @@ ldap::role::server::labs, ldap::role::client::labs, role::nova::controller +role::salt::masters::labs, +role::deployment::salt_masters::labs, } node "virt0.wikimedia.org" { -- To view, visit https://gerrit.wikimedia.org/r/102026 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I53f1bb02b2555dd87649aa2ea26de2353ad44939 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Up vmbuilder version to 3 - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Up vmbuilder version to 3 .. Up vmbuilder version to 3 Change-Id: I8380d08a48ac896ab2a50b6c9419e1eaa5a474e1 --- M manifests/role/labsvmbuilder.pp 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/labsvmbuilder.pp b/manifests/role/labsvmbuilder.pp index 5f18e23..d60274b 100644 --- a/manifests/role/labsvmbuilder.pp +++ b/manifests/role/labsvmbuilder.pp @@ -1,5 +1,5 @@ class role::labs_vmbuilder { class { "::labs_vmbuilder": -vmbuilder_version => "2"; +vmbuilder_version => "3"; } } -- To view, visit https://gerrit.wikimedia.org/r/102016 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8380d08a48ac896ab2a50b6c9419e1eaa5a474e1 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Up vmbuilder version to 3 - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102016 Change subject: Up vmbuilder version to 3 .. Up vmbuilder version to 3 Change-Id: I8380d08a48ac896ab2a50b6c9419e1eaa5a474e1 --- M manifests/role/labsvmbuilder.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/16/102016/1 diff --git a/manifests/role/labsvmbuilder.pp b/manifests/role/labsvmbuilder.pp index 5f18e23..d60274b 100644 --- a/manifests/role/labsvmbuilder.pp +++ b/manifests/role/labsvmbuilder.pp @@ -1,5 +1,5 @@ class role::labs_vmbuilder { class { "::labs_vmbuilder": -vmbuilder_version => "2"; +vmbuilder_version => "3"; } } -- To view, visit https://gerrit.wikimedia.org/r/102016 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8380d08a48ac896ab2a50b6c9419e1eaa5a474e1 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add firstboot script and ubuntu-standard package - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Add firstboot script and ubuntu-standard package .. Add firstboot script and ubuntu-standard package This change adds the ubuntu-standard package for bug 54080 and also adds the firstboot script that will be called on initial instance boot. Change-Id: If28081eb19917281caf5f8eec085ccec95253b2f --- A modules/labs_vmbuilder/files/firstboot.sh M modules/labs_vmbuilder/manifests/init.pp M modules/labs_vmbuilder/templates/vmbuilder.cfg.erb 3 files changed, 64 insertions(+), 2 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/labs_vmbuilder/files/firstboot.sh b/modules/labs_vmbuilder/files/firstboot.sh new file mode 100644 index 000..978b5a3 --- /dev/null +++ b/modules/labs_vmbuilder/files/firstboot.sh @@ -0,0 +1,55 @@ +#!/bin/bash + +if [ -f '/root/.firstboot' ] +then + # Only run firstboot once + exit +fi + +echo 'Enabling console logging for puppet while it does the initial run' +echo 'daemon.* |/dev/console' > /etc/rsyslog.d/60-puppet.conf +restart rsyslog + +binddn=`grep 'binddn' /etc/ldap.conf | sed 's/.* //'` +bindpw=`grep 'bindpw' /etc/ldap.conf | sed 's/.* //'` +hostsou=`grep 'nss_base_hosts' /etc/ldap.conf | sed 's/.* //'` +id=`curl http://169.254.169.254/1.0/meta-data/instance-id 2> /dev/null` +domain=`hostname -d` +idfqdn=${id}.${domain} +#TODO: get project a saner way +project=`ldapsearch -x -D ${binddn} -w ${bindpw} -b ${hostsou} "dc=${idfqdn}" puppetvar | grep 'instanceproject' | sed 's/.*=//'` +saltfinger="c5:b1:35:45:3e:0a:19:70:aa:5f:3a:cf:bf:a0:61:dd" +if [ "${domain}" == "pmtpa.wmflabs" ] +then + master="virt0.wikimedia.org" + master_secondary="virt1000.wikimedia.org" +elif [ "${domain}" == "eqiad.wmflabs" ] +then + master="virt1000.wikimedia.org" + master_secondary="virt0.wikimedia.org" +fi + +# Finish LDAP configuration +sed -i "s/_PROJECT_/${project}/g" /etc/security/access.conf +sed -i "s/_PROJECT_/${project}/g" /etc/ldap/ldap.conf +sed -i "s/_PROJECT_/${project}/g" /etc/sudo-ldap.conf +sed -i "s/_PROJECT_/${project}/g" /etc/default/autofs +sed -i "s/_PROJECT_/${project}/g" /etc/nslcd.conf +sed -i "s/_FQDN_/${idfqdn}/g" /etc/puppet/puppet.conf +sed -i "s/_MASTER_/${master}/g" /etc/puppet/puppet.conf + +/etc/init.d/autofs restart +dpkg-reconfigure -fnoninteractive -pcritical openssh-server +/etc/init.d/ssh stop +/etc/init.d/ssh start + +# Initial salt config +echo -e "master:\n - ${master}\n - ${master_secondary}" > /etc/salt/minion +echo "id: ${idfqdn}" >> /etc/salt/minion +echo "master_finger: ${saltfinger}" >> /etc/salt/minion +/etc/init.d/salt-minion restart + +# Force initial puppet run +puppet agent --onetime --verbose --no-daemonize --no-splay --show_diff --waitforcert=10 --certname=${idfqdn} --server=${master} + +touch /root/.firstboot diff --git a/modules/labs_vmbuilder/manifests/init.pp b/modules/labs_vmbuilder/manifests/init.pp index c9b3ed7..a2ec940 100644 --- a/modules/labs_vmbuilder/manifests/init.pp +++ b/modules/labs_vmbuilder/manifests/init.pp @@ -27,6 +27,13 @@ Package['python-vm-builder'], ]; } +file { '/etc/vmbuilder/firstscripts/firstboot.sh': +mode=> 0555, +source => 'puppet:///labs_vmbuilder/firstboot.sh', +require => [ +Package['python-vm-builder'], +]; +} file { "${vmbuilder_filepath}": ensure => directory, diff --git a/modules/labs_vmbuilder/templates/vmbuilder.cfg.erb b/modules/labs_vmbuilder/templates/vmbuilder.cfg.erb index 0eb82a0..073a3e5 100644 --- a/modules/labs_vmbuilder/templates/vmbuilder.cfg.erb +++ b/modules/labs_vmbuilder/templates/vmbuilder.cfg.erb @@ -3,7 +3,7 @@ part = /etc/vmbuilder/files/vmbuilder.partition copy = /etc/vmbuilder/postinst/postinst.copy execscript = /etc/vmbuilder/postinst/postinst.sh -#firstboot = /etc/vmbuilder/firstscripts/firstboot.sh +firstboot = /etc/vmbuilder/firstscripts/firstboot.sh lock-user = true # Required for sudo-ldap. We're going to disable this # first boot. @@ -17,5 +17,5 @@ proxy = http://brewster.wikimedia.org:8080 mirror = http://ubuntu.wikimedia.org/ubuntu/ components = main,restricted,multiverse,universe -addpkg = coreutils, snmp, wipe, tzdata, zsh-beta, jfsutils, xfsprogs, screen, gdb, iperf, atop, htop, vim, sysstat, ngrep, acct, git-core, lldpd, emacs23, libpam-ldapd, autofs5, autofs5-ldap, ldap-utils, libnss-ldapd, nss-updatedb, libnss-db, nscd, libpam-ldapd, python-ldap, python-pycurl, openssl, ca-certificates, ssl-cert, rsyslog, exim4-config, exim4-daemon-light, cloud-init, cloud-utils, euca2ools, openssh-server, curl, apparmor, libapparmor1 +addpkg = coreutils, snmp, wipe, tzdata, zsh-beta, jfsutils, xfsprogs, screen, gdb, iperf, atop, htop, vim, sysstat, ngrep, acct, git-core, lldpd, emacs23, libpam-ldapd, autofs5, autofs5-ldap, ldap-utils
[MediaWiki-commits] [Gerrit] Make restart runner and info util more dependable - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: Make restart runner and info util more dependable .. Make restart runner and info util more dependable Change-Id: I59033d7ed594735623f917ad5696700d79339b01 --- M modules/deployment/files/git-deploy/utils/service-restart M modules/deployment/files/runners/deploy.py 2 files changed, 10 insertions(+), 6 deletions(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/deployment/files/git-deploy/utils/service-restart b/modules/deployment/files/git-deploy/utils/service-restart index 66c3774..856e520 100644 --- a/modules/deployment/files/git-deploy/utils/service-restart +++ b/modules/deployment/files/git-deploy/utils/service-restart @@ -66,11 +66,15 @@ raise SystemExit(1) minion_data = minion_data['local'] for i in minion_data: -for minion, data in i.items(): -try: -LOG.info('{0}: {1}'.format(minion, data['status'])) -except KeyError: -LOG.info('{0}: No status available'.format(minion)) +try: +for minion, data in i.items(): +try: +LOG.info('{0}: {1}'.format(minion, data['status'])) +except KeyError: +LOG.info('{0}: No status available'.format(minion)) +except AttributeError: +LOG.error('Got bad return from salt. Here is the raw data:') +LOG.error('{}'.format(i)) if __name__ == "__main__": main() diff --git a/modules/deployment/files/runners/deploy.py b/modules/deployment/files/runners/deploy.py index 6f2ec33..9a5db48 100755 --- a/modules/deployment/files/runners/deploy.py +++ b/modules/deployment/files/runners/deploy.py @@ -68,7 +68,7 @@ arg = (repo,) ret = [] for data in client.cmd_batch(grain, cmd, expr_form='grain', arg=arg, - timeout=30, ret='deploy_redis', batch=batch): + timeout=60, ret='deploy_redis', batch=batch): ret.append(data) print "Restart completed" return ret -- To view, visit https://gerrit.wikimedia.org/r/102003 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I59033d7ed594735623f917ad5696700d79339b01 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Make restart runner and info util more dependable - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102003 Change subject: Make restart runner and info util more dependable .. Make restart runner and info util more dependable Change-Id: I59033d7ed594735623f917ad5696700d79339b01 --- M modules/deployment/files/git-deploy/utils/service-restart M modules/deployment/files/runners/deploy.py 2 files changed, 10 insertions(+), 6 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/03/102003/1 diff --git a/modules/deployment/files/git-deploy/utils/service-restart b/modules/deployment/files/git-deploy/utils/service-restart index 66c3774..856e520 100644 --- a/modules/deployment/files/git-deploy/utils/service-restart +++ b/modules/deployment/files/git-deploy/utils/service-restart @@ -66,11 +66,15 @@ raise SystemExit(1) minion_data = minion_data['local'] for i in minion_data: -for minion, data in i.items(): -try: -LOG.info('{0}: {1}'.format(minion, data['status'])) -except KeyError: -LOG.info('{0}: No status available'.format(minion)) +try: +for minion, data in i.items(): +try: +LOG.info('{0}: {1}'.format(minion, data['status'])) +except KeyError: +LOG.info('{0}: No status available'.format(minion)) +except AttributeError: +LOG.error('Got bad return from salt. Here is the raw data:') +LOG.error('{}'.format(i)) if __name__ == "__main__": main() diff --git a/modules/deployment/files/runners/deploy.py b/modules/deployment/files/runners/deploy.py index 6f2ec33..9a5db48 100755 --- a/modules/deployment/files/runners/deploy.py +++ b/modules/deployment/files/runners/deploy.py @@ -68,7 +68,7 @@ arg = (repo,) ret = [] for data in client.cmd_batch(grain, cmd, expr_form='grain', arg=arg, - timeout=30, ret='deploy_redis', batch=batch): + timeout=60, ret='deploy_redis', batch=batch): ret.append(data) print "Restart completed" return ret -- To view, visit https://gerrit.wikimedia.org/r/102003 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I59033d7ed594735623f917ad5696700d79339b01 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Add firstboot script and ubuntu-standard package - change (operations/puppet)
Ryan Lane has uploaded a new change for review. https://gerrit.wikimedia.org/r/102000 Change subject: Add firstboot script and ubuntu-standard package .. Add firstboot script and ubuntu-standard package This change adds the ubuntu-standard package for bug 54080 and also adds the firstboot script that will be called on initial instance boot. Change-Id: If28081eb19917281caf5f8eec085ccec95253b2f --- A modules/labs_vmbuilder/files/firstboot.sh M modules/labs_vmbuilder/manifests/init.pp M modules/labs_vmbuilder/templates/vmbuilder.cfg.erb 3 files changed, 64 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/00/102000/1 diff --git a/modules/labs_vmbuilder/files/firstboot.sh b/modules/labs_vmbuilder/files/firstboot.sh new file mode 100644 index 000..978b5a3 --- /dev/null +++ b/modules/labs_vmbuilder/files/firstboot.sh @@ -0,0 +1,55 @@ +#!/bin/bash + +if [ -f '/root/.firstboot' ] +then + # Only run firstboot once + exit +fi + +echo 'Enabling console logging for puppet while it does the initial run' +echo 'daemon.* |/dev/console' > /etc/rsyslog.d/60-puppet.conf +restart rsyslog + +binddn=`grep 'binddn' /etc/ldap.conf | sed 's/.* //'` +bindpw=`grep 'bindpw' /etc/ldap.conf | sed 's/.* //'` +hostsou=`grep 'nss_base_hosts' /etc/ldap.conf | sed 's/.* //'` +id=`curl http://169.254.169.254/1.0/meta-data/instance-id 2> /dev/null` +domain=`hostname -d` +idfqdn=${id}.${domain} +#TODO: get project a saner way +project=`ldapsearch -x -D ${binddn} -w ${bindpw} -b ${hostsou} "dc=${idfqdn}" puppetvar | grep 'instanceproject' | sed 's/.*=//'` +saltfinger="c5:b1:35:45:3e:0a:19:70:aa:5f:3a:cf:bf:a0:61:dd" +if [ "${domain}" == "pmtpa.wmflabs" ] +then + master="virt0.wikimedia.org" + master_secondary="virt1000.wikimedia.org" +elif [ "${domain}" == "eqiad.wmflabs" ] +then + master="virt1000.wikimedia.org" + master_secondary="virt0.wikimedia.org" +fi + +# Finish LDAP configuration +sed -i "s/_PROJECT_/${project}/g" /etc/security/access.conf +sed -i "s/_PROJECT_/${project}/g" /etc/ldap/ldap.conf +sed -i "s/_PROJECT_/${project}/g" /etc/sudo-ldap.conf +sed -i "s/_PROJECT_/${project}/g" /etc/default/autofs +sed -i "s/_PROJECT_/${project}/g" /etc/nslcd.conf +sed -i "s/_FQDN_/${idfqdn}/g" /etc/puppet/puppet.conf +sed -i "s/_MASTER_/${master}/g" /etc/puppet/puppet.conf + +/etc/init.d/autofs restart +dpkg-reconfigure -fnoninteractive -pcritical openssh-server +/etc/init.d/ssh stop +/etc/init.d/ssh start + +# Initial salt config +echo -e "master:\n - ${master}\n - ${master_secondary}" > /etc/salt/minion +echo "id: ${idfqdn}" >> /etc/salt/minion +echo "master_finger: ${saltfinger}" >> /etc/salt/minion +/etc/init.d/salt-minion restart + +# Force initial puppet run +puppet agent --onetime --verbose --no-daemonize --no-splay --show_diff --waitforcert=10 --certname=${idfqdn} --server=${master} + +touch /root/.firstboot diff --git a/modules/labs_vmbuilder/manifests/init.pp b/modules/labs_vmbuilder/manifests/init.pp index c9b3ed7..a2ec940 100644 --- a/modules/labs_vmbuilder/manifests/init.pp +++ b/modules/labs_vmbuilder/manifests/init.pp @@ -27,6 +27,13 @@ Package['python-vm-builder'], ]; } +file { '/etc/vmbuilder/firstscripts/firstboot.sh': +mode=> 0555, +source => 'puppet:///labs_vmbuilder/firstboot.sh', +require => [ +Package['python-vm-builder'], +]; +} file { "${vmbuilder_filepath}": ensure => directory, diff --git a/modules/labs_vmbuilder/templates/vmbuilder.cfg.erb b/modules/labs_vmbuilder/templates/vmbuilder.cfg.erb index 0eb82a0..073a3e5 100644 --- a/modules/labs_vmbuilder/templates/vmbuilder.cfg.erb +++ b/modules/labs_vmbuilder/templates/vmbuilder.cfg.erb @@ -3,7 +3,7 @@ part = /etc/vmbuilder/files/vmbuilder.partition copy = /etc/vmbuilder/postinst/postinst.copy execscript = /etc/vmbuilder/postinst/postinst.sh -#firstboot = /etc/vmbuilder/firstscripts/firstboot.sh +firstboot = /etc/vmbuilder/firstscripts/firstboot.sh lock-user = true # Required for sudo-ldap. We're going to disable this # first boot. @@ -17,5 +17,5 @@ proxy = http://brewster.wikimedia.org:8080 mirror = http://ubuntu.wikimedia.org/ubuntu/ components = main,restricted,multiverse,universe -addpkg = coreutils, snmp, wipe, tzdata, zsh-beta, jfsutils, xfsprogs, screen, gdb, iperf, atop, htop, vim, sysstat, ngrep, acct, git-core, lldpd, emacs23, libpam-ldapd, autofs5, autofs5-ldap, ldap-utils, libnss-ldapd, nss-updatedb, libnss-db, nscd, libpam-ldapd, python-ldap, python-pycurl, openssl, ca-certificates, ssl-cert, rsyslog, exim4-config, exim4-daemon-light, cloud-init, cloud-utils, euca2ools, openssh-server, curl, apparmor, libapparmor1 +addpkg = coreutils, snmp, wipe, tzdata, zsh-beta, jfsutils, xfsprogs, screen, gdb, iperf, atop, htop, vim, sysstat, ngrep, acct, git-core, lldpd, emacs23, l
[MediaWiki-commits] [Gerrit] trebuchet: Handle service restarts with no status - change (operations/puppet)
Ryan Lane has submitted this change and it was merged. Change subject: trebuchet: Handle service restarts with no status .. trebuchet: Handle service restarts with no status Change-Id: I0f19c2a0421dee6c8e27fcb75a097a13465424dd --- M modules/deployment/files/git-deploy/utils/service-restart 1 file changed, 4 insertions(+), 1 deletion(-) Approvals: Ryan Lane: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/deployment/files/git-deploy/utils/service-restart b/modules/deployment/files/git-deploy/utils/service-restart index 67961cf..66c3774 100644 --- a/modules/deployment/files/git-deploy/utils/service-restart +++ b/modules/deployment/files/git-deploy/utils/service-restart @@ -67,7 +67,10 @@ minion_data = minion_data['local'] for i in minion_data: for minion, data in i.items(): -LOG.info('{0}: {1}'.format(minion, data['status'])) +try: +LOG.info('{0}: {1}'.format(minion, data['status'])) +except KeyError: +LOG.info('{0}: No status available'.format(minion)) if __name__ == "__main__": main() -- To view, visit https://gerrit.wikimedia.org/r/100914 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I0f19c2a0421dee6c8e27fcb75a097a13465424dd Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ryan Lane Gerrit-Reviewer: Ryan Lane Gerrit-Reviewer: jenkins-bot ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits