Gehel has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/377997 )
Change subject: Cassandra: Include only instance DNS' in the list of seeds
......................................................................
Cassandra: Include only instance DNS' in the list of seeds
In I49313adaf5fb6c1028df21a2c18ca4385df734f5 we introduced the
instance_count variable with which we were trying to filter out main DNS
entries for Cassandra hosts in cases where they are running multiple
instances. However, there are several problems with this approach,
namely that the approach relied on us knowing the exact format of the
hostname, which is not possible. Also, the question is - what to do when
there are instances defined, but there's only one? Or how to deal with
the case where the number of instances across nodes in the same cluster
is not the same (that's the case in RB-staging)?
Hence, this new approach filters out the seeds "at the source", while
the seed list is being computed. Two lists are produced: one that is
used for the ferm rules, containing all of the IPs/DNS'; and another
that is used only for the Cassandra seeds, from which we exclude the
main DNS if there are instances defined for that host.
Bug: T172610
Change-Id: I0a00f1d8efb143edfe18a8749dfbda3ba32753e3
---
M modules/cassandra/templates/cassandra.yaml-2.1.erb
M modules/cassandra/templates/cassandra.yaml-2.2.erb
M modules/cassandra/templates/cassandra.yaml-3.x.erb
M modules/profile/manifests/cassandra.pp
M modules/profile/templates/cassandra/seeds.erb
5 files changed, 14 insertions(+), 17 deletions(-)
Approvals:
Mobrovac: Looks good to me, but someone else must approve
jenkins-bot: Verified
Filippo Giunchedi: Looks good to me, but someone else must approve
Gehel: Looks good to me, approved
diff --git a/modules/cassandra/templates/cassandra.yaml-2.1.erb
b/modules/cassandra/templates/cassandra.yaml-2.1.erb
index 63775da..70a2d0c 100644
--- a/modules/cassandra/templates/cassandra.yaml-2.1.erb
+++ b/modules/cassandra/templates/cassandra.yaml-2.1.erb
@@ -273,10 +273,7 @@
x != @hostname \
&& ! (x.start_with?
"#{@hostname}-") \
&& x != @ipaddress \
- && x != @fqdn \
- &&
(/^.+-[a-z]\.(?:eqiad|codfw)\.wmnet$/ =~ x \
- || @instance_count ==
1 \
- ||
/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/ =~ x)
+ && x != @fqdn
}.join(',') :
@seeds.join(',') %>
- seeds: <%= clean_seeds %>
# For workloads with more data than can fit in memory, Cassandra's
diff --git a/modules/cassandra/templates/cassandra.yaml-2.2.erb
b/modules/cassandra/templates/cassandra.yaml-2.2.erb
index cdf6404..ebcf691 100644
--- a/modules/cassandra/templates/cassandra.yaml-2.2.erb
+++ b/modules/cassandra/templates/cassandra.yaml-2.2.erb
@@ -318,10 +318,7 @@
x != @hostname \
&& ! (x.start_with?
"#{@hostname}-") \
&& x != @ipaddress \
- && x != @fqdn \
- &&
(/^.+-[a-z]\.(?:eqiad|codfw)\.wmnet$/ =~ x \
- || @instance_count ==
1 \
- ||
/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/ =~ x)
+ && x != @fqdn
}.join(',') :
@seeds.join(',') %>
- seeds: <%= clean_seeds %>
# For workloads with more data than can fit in memory, Cassandra's
diff --git a/modules/cassandra/templates/cassandra.yaml-3.x.erb
b/modules/cassandra/templates/cassandra.yaml-3.x.erb
index 46edbb3..9bb151a 100644
--- a/modules/cassandra/templates/cassandra.yaml-3.x.erb
+++ b/modules/cassandra/templates/cassandra.yaml-3.x.erb
@@ -404,10 +404,7 @@
x != @hostname \
&& ! (x.start_with?
"#{@hostname}-") \
&& x != @ipaddress \
- && x != @fqdn \
- &&
(/^.+-[a-z]\.(?:eqiad|codfw)\.wmnet$/ =~ x \
- || @instance_count ==
1 \
- ||
/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/ =~ x)
+ && x != @fqdn
}.join(',') :
@seeds.join(',') %>
- seeds: <%= clean_seeds %>
# For workloads with more data than can fit in memory, Cassandra's
diff --git a/modules/profile/manifests/cassandra.pp
b/modules/profile/manifests/cassandra.pp
index 629b0f4..217724d 100644
--- a/modules/profile/manifests/cassandra.pp
+++ b/modules/profile/manifests/cassandra.pp
@@ -14,7 +14,9 @@
$instances = $all_instances[$::fqdn]
# We get the cassandra seeds from $all_instances, with a template hack
# This is preferred over a very specialized parser function.
- $seeds = split(template('profile/cassandra/seeds.erb'), '\|')
+ $all_seeds = split(template('profile/cassandra/seeds.erb'), '\|')
+ $seeds = split($all_seeds[0], ',')
+ $ferm_seeds = split($all_seeds[1], ',')
$base_settings = {
'instances' => $instances,
@@ -70,7 +72,7 @@
description => 'Cassandra server',
}
- $cassandra_hosts_ferm = join($seeds, ' ')
+ $cassandra_hosts_ferm = join($ferm_seeds, ' ')
$prometheus_nodes_ferm = join($prometheus_nodes, ' ')
# Cassandra intra-node messaging
diff --git a/modules/profile/templates/cassandra/seeds.erb
b/modules/profile/templates/cassandra/seeds.erb
index 2ab00ec..7ba8c8f 100644
--- a/modules/profile/templates/cassandra/seeds.erb
+++ b/modules/profile/templates/cassandra/seeds.erb
@@ -1,9 +1,13 @@
<%-
seeds = []
+ferm_seeds = []
@all_instances.each do |host, instances|
- seeds << host
+ ferm_seeds << host
+ seeds << host if instances.empty?
hostname = host.split(".").shift()
instances.each do |instance, _|
- seeds << host.gsub(hostname, "#{hostname}-#{instance}")
+ instance_host = host.gsub(hostname, "#{hostname}-#{instance}")
+ seeds << instance_host
+ ferm_seeds << instance_host
end
-end -%><%= seeds.join "|" -%>
+end -%><%= "#{seeds.join ','}|#{ferm_seeds.join ','}" -%>
--
To view, visit https://gerrit.wikimedia.org/r/377997
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I0a00f1d8efb143edfe18a8749dfbda3ba32753e3
Gerrit-PatchSet: 5
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Mobrovac <mobro...@wikimedia.org>
Gerrit-Reviewer: Eevans <eev...@wikimedia.org>
Gerrit-Reviewer: Elukey <ltosc...@wikimedia.org>
Gerrit-Reviewer: Filippo Giunchedi <fgiunch...@wikimedia.org>
Gerrit-Reviewer: Gehel <guillaume.leder...@wikimedia.org>
Gerrit-Reviewer: Giuseppe Lavagetto <glavage...@wikimedia.org>
Gerrit-Reviewer: Mobrovac <mobro...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
