Eevans has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/370098 )
Change subject: WIP: Reshape RESTBase Cassandra production cluster; Provision
new 3.x cluster
......................................................................
WIP: Reshape RESTBase Cassandra production cluster; Provision new 3.x cluster
Basic steps:
- 9T of storage will be freed
- restbase1016.eqiad.wmnet is decommissioned
- this changeset is merged
- restbase1016.eqiad.wmnet is re-imaged
- restbase1016.eqiad.wmnet is bootstrapped
- new changeset to comment/uncomment instances is merged
- lather
- rinse
- repeat
Todo:
- Figure out what to do w/ restbase::seeds
Bug: T169939
Change-Id: Ife09538ec52e8f48b8d0899d81b1c961a8476b94
---
M hieradata/role/common/restbase/production.yaml
A hieradata/role/common/restbase/production_new.yaml
2 files changed, 123 insertions(+), 7 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/98/370098/1
diff --git a/hieradata/role/common/restbase/production.yaml
b/hieradata/role/common/restbase/production.yaml
index 82f3b2b..4e99d3b 100644
--- a/hieradata/role/common/restbase/production.yaml
+++ b/hieradata/role/common/restbase/production.yaml
@@ -127,13 +127,13 @@
listen_address: 10.64.48.139
c:
listen_address: 10.64.48.140
- "restbase1016.eqiad.wmnet":
- a:
- listen_address: 10.64.0.32
- b:
- listen_address: 10.64.0.33
- c:
- listen_address: 10.64.0.34
+ # "restbase1016.eqiad.wmnet":
+ # a:
+ # listen_address: 10.64.0.32
+ # b:
+ # listen_address: 10.64.0.33
+ # c:
+ # listen_address: 10.64.0.34
"restbase1017.eqiad.wmnet":
a:
listen_address: 10.64.32.130
diff --git a/hieradata/role/common/restbase/production_new.yaml
b/hieradata/role/common/restbase/production_new.yaml
new file mode 100644
index 0000000..29b0487
--- /dev/null
+++ b/hieradata/role/common/restbase/production_new.yaml
@@ -0,0 +1,116 @@
+#
+# Cassandra
+#
+graphite_host: 'graphite1003.eqiad.wmnet'
+profile::cassandra::allow_analytics: false
+
+profile::cassandra::metrics_whitelist:
+ -
.*\.metrics\.ColumnFamily\.local_group_.*\.meta\.CoordinatorReadLatency\..*$
+ -
.*\.metrics\.ColumnFamily\.local_group_.*\.meta\.CoordinatorScanLatency\..*$
+ -
.*\.metrics\.ColumnFamily\.local_group_.*\.meta\.EstimatedColumnCountHistogram\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.EstimatedRowCount\..*$
+ -
.*\.metrics\.ColumnFamily\.local_group_.*\.meta\.EstimatedRowSizeHistogram\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.LiveDiskSpaceUsed\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.LiveSSTableCount\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.LiveScannedHistogram\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.MaxRowSize\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.MeanRowSize\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.MinRowSize\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.PendingCompactions\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.PendingFlushes\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.RangeLatency\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.ReadLatency\..*$
+ -
.*\.metrics\.ColumnFamily\.local_group_.*\.meta\.SSTablesPerReadHistogram\..*$
+ -
.*\.metrics\.ColumnFamily\.local_group_.*\.meta\.TombstoneScannedHistogram\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.TotalDiskSpaceUsed\..*$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\.WriteLatency\..*$
+profile::cassandra::metrics_blacklist:
+ - .*\.15MinuteRate$
+ - .*\.5MinuteRate$
+ - .*\.98percentile$
+ - .*\.999percentile$
+ - .*\.mean$
+ - .*\.meanRate$
+ - .*\.min$
+ - .*\.stddev$
+ - .*\.metrics\.ColumnFamily\.local_group_.*\.meta\..*$
+ - .*\.metrics\.Keyspace\..*$
+ - .*\.metrics\.Client\..*$
+ - .*\.metrics\.Table\..*$
+## Instances
+profile::cassandra::instances:
+ "restbase1016.eqiad.wmnet":
+ a:
+ listen_address: 10.64.0.32
+ b:
+ listen_address: 10.64.0.33
+ c:
+ listen_address: 10.64.0.34
+ # "restbase1017.eqiad.wmnet":
+ # a:
+ # listen_address: 10.64.32.130
+ # b:
+ # listen_address: 10.64.32.131
+ # c:
+ # listen_address: 10.64.32.132
+ # "restbase1018.eqiad.wmnet":
+ # a:
+ # listen_address: 10.64.48.98
+ # b:
+ # listen_address: 10.64.48.99
+ # c:
+ # listen_address: 10.64.48.100
+ # "restbase2010.codfw.wmnet":
+ # a:
+ # listen_address: 10.192.16.186
+ # b:
+ # listen_address: 10.192.16.187
+ # c:
+ # listen_address: 10.192.16.188
+ # "restbase2011.codfw.wmnet":
+ # a:
+ # listen_address: 10.192.32.152
+ # b:
+ # listen_address: 10.192.32.153
+ # c:
+ # listen_address: 10.192.32.154
+ # "restbase2012.codfw.wmnet":
+ # a:
+ # listen_address: 10.192.48.68
+ # b:
+ # listen_address: 10.192.48.69
+ # c:
+ # listen_address: 10.192.48.70
+
+
+profile::cassandra::settings:
+ start_rpc: 'false'
+ max_heap_size: 12g
+ # 1/4 heap size, no more than 100m/thread
+ heap_newsize: 2048m
+ compaction_throughput_mb_per_sec: 20
+ concurrent_compactors: 10
+ concurrent_writes: 18
+ concurrent_reads: 18
+ tls_cluster_name: "services"
+ internode_encryption: dc
+ dc: "%{::site}"
+ cluster_name: "restbase"
+ application_username: restb
+ application_password: "%{passwords::cassandra::restbase}"
+ target_version: '3.x'
+ permissions_validity_in_ms: 600000
+ trickle_fsync: false
+ client_encryption_enabled: true
+
+
+#
+# General
+#
+role::lvs::realserver::pools:
+ restbase: {}
+service::configuration::log_dir: "/var/log"
+
+admin::groups:
+ - restbase-roots
+ - restbase-admins
--
To view, visit https://gerrit.wikimedia.org/r/370098
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ife09538ec52e8f48b8d0899d81b1c961a8476b94
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Eevans <eev...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
